Tryag File Manager

//usr/share/doc/device-mapper-multipath-0.4.7/Multipath-usage.txt

RHEL4 U3 Device Mapper Multipath Usage

Maintainer
------------
Benjamin Marzinski
bmarzins@redhat.com

Overview
------------
Device Mapper Multipath (DM-MP) allows nodes to route I/O over multiple paths to
a storage controller. A path refers to the connection from an HBA port to a
storage controller port. As paths fail and new paths come up, DM-MP reroutes 
the I/O over the available paths.

When there are multiple paths to a storage controller, each path
appears as a separate device.  DM-MP creates a new device on top of
those devices. For example, a node with two HBAs attached to a storage
controller with two ports via a single unzoned FC switch sees four
devices: /dev/sda, /dev/sdb, /dev/sdc, and /dev/sdd. DM-MP creates a
single device, /dev/mpath/mpath1 that reroutes I/O to those four
underlying devices.

DM-MP consists of the
following components:

o dm-multipath kernel module --	This module reroutes I/O and fails
  over paths and path groups.

o multipath command -- This command configures, lists, and removes multipath
  devices. The command is run in rc.sysinit during startup, and by
  udev, whenever a block device	is added.

o multipathd daemon -- 	This daemon monitors paths, checking to see if faulty
  paths have been fixed. As paths come back up, multipathd may also
  initiate path group switches to ensure that the optimal path group
  is being used. Also, it is possible to interactively modify a
  multipath device.

o kpartx command -- This command creates Device Mapper devices for the
  partitions on a device. It is necessary to use this command for DOS-
  based partitions with DM-MP.

DM-MP works with a variety of storage arrays. It
auto-configures the following storage arrays:

o 3PARdata VV
o Compaq HSV110
o Compaq MSA1000
o DDN SAN DataDirector
o DEC HSG80
o EMC SYMMETRIX
o EMC CLARiiON
o FSC CentricStor
o GNBD
o HITACHI DF400
o HITACHI DF500
o HITACHI DF600
o HP HSV110
o HP HSV210
o HP A6189A
o HP Open-
o IBM 3542
o IBM ProFibre 4000R
o NETAPP
o SGI TP9100
o SGI TP9300
o SGI TP9400
o SGI TP9500
o STK OPENstroage D280
o SUN StorEdge 3510
o SUN T4

Storage arrays not included in the list may require entries in the
/etc/multipath.conf file.

NOTE: Some storage arrays require special handling of I/O errors and
      path-group switching. Those require separate hardware handler
      kernel modules.

Terms and Concepts
---------------------

Hardware Handler:
	A kernel module that performs hardware-specific actions when switching
	path groups and dealing with I/O errors.

Path:
	The connection from an HBA port to a storage controller port for a LUN.
	Each path appears as a separate device. Paths can be in
	various states (refer to "Path States").

Path States:
	ready  - Path is able to handle I/O requests.
	shaky  - Path is up, but temporarily not available for normal
		 operations.
	faulty - Path is unable to handle I/O requests.
	ghost  - Path is a passive path, on an active/passive controller.
	
        NOTE: The shaky and ghost states only exist for certain
	      storage arrays.

Path Group:
	A grouping of paths. With DM-MP, only one path group--the
	active path group--receives I/O at any time. Within a path
	group, DM-MP selects which ready path should receive I/O in a
        round robin fashion. Path groups can be in various states (refer to
        "Path Group States").

Path Group States:
	active   - Path group currently receiving I/O requests.
	enabled  - Path groups to try if the active path group has no paths
		   in the ready state.
	disabled - Path groups to try if the active path group and all
		   enabled path groups have no paths in the active state.
	
        NOTE: The disabled state only exists for certain storage arrays.

Path Priority:
	Each path can have a priority assigned to it by a callout program.
	Path priorities can be used to group paths by priority and change
	their relative weights for the round robin path selector.

Path Group Priority:
	Each path group has a priority that is equal to the sum of the
	priorities of all the non-faulty paths in the group. By default, the
	multipathd daemon tries to ensure that the path group with the
	highest priority is always in the active state.

Failover:
	When I/O to a path fails, the dm-multipath module tries to switch to 
	an enabled path group. If there are no enabled path groups with
	any paths in the ready state, dm-multipath tries to switch to a disabled 
	path group. If necessary, dm-multipath runs the hardware handler for the
	multipath device.

Failback:
	At regular intervals, multipathd checks the current priority of
	all path groups. If the current path group is not the highest priority
	path group, multipathd reacts according to the failback mode.
	By default, multipathd immediately switches to the highest priority
	path group. Other options for multipathd are to (a) wait for a
	user-defined length of time (for the path groups to stabilize)
	and then switch or (b) for multipathd to do nothing and wait for
	manual intervention.  Failback can be forced at any time by
	running the multipath command.

Multipath device:
	The multipath device is the device mapper device created by
	dm-multipath. A multipath device can be identified by either
	its WWID or its alias. A multipath device has one or more path
	groups. It also has numerous attributes defined in the
	following file:
	/usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.annotated

alias:
	The alias is the name of a multipath device. By default, the
	alias is set to the WWID. However, by setting the
	"user_friendly_names" configuration option, the alias is set to a
	unique name of the form mpath<n>. The alias name can also be
	explicitly set for each multipath device in the configuration file.

NOTE: While the alias in guaranteed to be unique on a node, it
	      is not guaranteed to be the same on all nodes using the
	      multipath device. Also, it may change.

WWID:
	The WWID (World Wide Identifier) is an identifier for the
	multipath device that is guaranteed to be globally unique and
	unchanging. It is determined by the getuid callout program.

Using DM-MP
------------------------------------

Initial setup:

1. If it is not already installed. Install the device-mapper-multipath
   package.

2. Edit /etc/multipath.conf. For new installations, all devices are blacklisted.
   The default blacklist is listed in the commented out section of
   /etc/multipath.conf.  If you comment out or delete the following lines in
   /etc/multipath.conf, the default blacklist takes effect:

devnode_blacklist {
           devnode "*"
   }

For some conditions, that may not be sufficient. If DM-MP is
   multipathing devices that you do not want it to work on, you can
   blacklist the devices by either device name or WWID.

NOTE: It is safest to blacklist individual devices by WWID, because
         their device names may change.

Several other configuration options are detailed later in this
   document. To check the  effects of configuration changes, you can
   do a dry run with the following command:

# multipath -v2 -d

3. Set the multipathd init script to run at boot time. by issuing the commands

# chkconfig --add multipathd
   # chkconfig multipathd on

4. start dm-multipath (This is only necessary the first time.  On
   reboot, this should happen automatically).

# multipath
   # /etc/init.d/multipathd start

After initial setup, all access to the multipathed storage should go through the
multipath device.

GNBD devices will not be automatically multipathed after they are imported.
The command

# multipath

must be run after every time the devices are imported. Otherwise the Multipath
devices will not be created.

Configuration File:

Many features of DM-MP are configurable using the configuration file,
/etc/multipath.conf.

For a complete list of all options with descriptions, refer to
/usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.annotated

The configuration file is divided into four sections: system defaults,
blacklisted devices (devnode_blacklist), per storage array model settings
(devices), and per multipath device settings (multipaths).  The per multipath
device settings are used for the multipath device with a matching "wwid"
value. The per storage array model settings are used for all multipath devices
with matching "vendor" and "product" values. To determine the attributes of a
multipath device, first the per multipath settings are checked, then the per
controller settings, then the system defaults.  The blacklisted device section
is described setup step 2.

NOTE: There are compiled-in defaults for the "defaults", "devnode_blacklist",
and "devices" sections of the configuration file. To see what these
are, refer to the following file:

/usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.defaults

If you are using one of the storage arrays listed in the preceding
text (in "Overview"), you probably do not need to modify the "devices"
subsection. If you are using a simple disk enclosure, the defaults
should work. If you are using a storage array that is not
listed, you may need to create a "devices" subsection for your array.

Reconfiguring a running system:

If any changes are make to the configuration file that will effect multipathd
(check /usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.annotated
to see if mulipathd is in the options scope), you must restart multipathd
for these to take effect. To do that, run

# /etc/init.d/multipathd restart

Explanation of output
-----------------------
When you create, modify, or list a multipath device, you get a printout of
the current device setup. The format is as follows.

For each multipath device:

action_if_any: alias (wwid_if_different_from_alias)
[size][features][hardware_handler]

For each path group:

\_ scheduling_policy [path_group_priority_if_known][path_group_status_if_known]

For each path:

\_ host:channel:id:lun devnode major:minor [dm_status_if_known][path_status]

The dm status (dm_status_if_known) is like the path status
(path_status), but from the kernel's point of view.  The dm status has two
states: "failed", which is analogous to "faulty", and "active" which
covers all other path states. Occasionally, the path state and the 
dm state of a device will temporarily not agree.

NOTE: When a multipath device is being created or modified, the path group
status and the dm status are not known.  Also, the features are not always
correct. When a multipath device is being listed, the path group priority is not
known.

Restrictions
---------------
DM-MP cannot be run on either the root or boot device.

Other Sources of information
----------------------------
Configuration file explanation:
/usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.annotated

Upstream documentation:
http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=Home

mailing list:
dm-devel@redhat.com
Subscribe to this from https://www.redhat.com/mailman/listinfo/dm-devel.
The list archives are at https://www.redhat.com/archives/dm-devel/

Man pages:
multipath.8, multipathd.8, kpartx.8 mpath_ctl.8