• VMware

    Learn about VMware virtualization for its products like vsphere ESX and ESXi, vCenter Server, VMware View, VMware P2V and many more

  • Linux

    Step by step configuration tutorials for many of the Linux services like DNS, DHCP, FTP, Samba4 etc including many tips and tricks in Red Hat Linux.

  • Database

    Learn installation and configuration of databases like Oracle, My SQL, Postgresql, etc including many other related tutorials in Linux.

  • Tutorial/Cheatsheet: Begineer's Guide to Understanding Device Mapper Multipath for Linux

    Before starting with going into the various commands we use for multipath let me give you a
    overview of multipath

    Without DM Multipath, each path from a server node to a storage controller is treated by the system as a separate device, even when the I/O path connects the same server node to the same storage controller. DM Multipath provides a way of organizing the I/O paths logically, by creating a single multipath device on top of the underlying devices.


    To make multipath work make sure below module is loaded
    # lsmod | grep dm_multipath
    dm_multipath           27427  4 dm_round_robin,dm_service_time

    # modinfo dm_multipath
    filename:       /lib/modules/3.10.0-693.21.1.el7.x86_64/kernel/drivers/md/dm-multipath.ko.xz
    license:        GPL
    author:         Sistina Software <dm-devel@redhat.com>
    description:    device-mapper multipath target
    retpoline:      Y
    rhelversion:    7.4
    srcversion:     063067F9F167E7B653A4773
    depends:        dm-mod
    intree:         Y
    vermagic:       3.10.0-693.21.1.el7.x86_64 SMP mod_unload modversions
    signer:         Red Hat Enterprise Linux kernel signing key
    sig_key:        B1:C3:31:09:FB:DA:94:AD:2F:E1:E8:E5:C1:E5:52:BD:22:57:60:FE
    sig_hashalgo:   sha256

    Below is the main device mapper rpm
    device-mapper-multipath
    device-mapper

    NOTE: These rpms will also bring in multiple dependencies so it is recommended to use yum for instaling these rpms

    Main configuration file
    /etc/multipath.conf

    Viewing the connected and scanned multipath devices
    # multipath -l






    Get the WWID of the connected/scanned LUN

    Each multipath device has a World Wide Identifier (WWID), which is guaranteed to be globally unique and unchanging.

    You can get the WWID using below commands
    # multipath -l | grep dm
    3600c0ff00013b88f9058e35a01000000 dm-0 HP      ,P2000 G3 FC
    3600c0ff00013b88f9c58e35a01000000 dm-6 HP      ,P2000 G3 FC

    OR using below command
    # ls -ld /dev/disk/by-id/scsi-*
    lrwxrwxrwx 1 root root 9 May 22 18:29 /dev/disk/by-id/scsi-3600c0ff00013b88f9058e35a01000000 -> ../../sde
    lrwxrwxrwx 1 root root 9 May 22 18:29 /dev/disk/by-id/scsi-3600c0ff00013b88f9c58e35a01000000 -> ../../sdd



    Get the WWID of the disks

    Each LUN is mapped to a disk for which also a unique WWID is generated and the same can be used for representing them in the configuration as the device name may change post reboot while the WWID will not change and are reboot persistent.
    # lsscsi --scsi_id
    [0:0:0:0]    disk    HP       P2000 G3 FC      T252  /dev/sda   3600c0ff00013b88f9058e35a01000000
    [0:0:0:1]    disk    HP       P2000 G3 FC      T252  /dev/sdb   3600c0ff00013b88f9c58e35a01000000
    [0:0:1:0]    disk    HP       P2000 G3 FC      T252  /dev/sdc   3600c0ff00013b88f9058e35a01000000
    [0:0:1:1]    disk    HP       P2000 G3 FC      T252  /dev/sdd   3600c0ff00013b88f9c58e35a01000000
    [1:0:0:0]    disk    HP       P2000 G3 FC      T252  /dev/sde   3600c0ff00013b88f9058e35a01000000
    [1:0:0:1]    disk    HP       P2000 G3 FC      T252  /dev/sdf   3600c0ff00013b88f9c58e35a01000000
    [1:0:1:0]    disk    HP       P2000 G3 FC      T252  /dev/sdg   3600c0ff00013b88f9058e35a01000000
    [1:0:1:1]    disk    HP       P2000 G3 FC      T252  /dev/sdh   3600c0ff00013b88f9c58e35a01000000

    To get the WWID of the disks you can also use below command
    # /usr/lib/udev/scsi_id -g -u -d /dev/sda
    3600c0ff00013b88f9058e35a01000000




    Get the list of disks mapped via HBA to the Linux box

    A node with two HBAs attached to a storage controller with two ports by means of a single unzoned FC switch sees four devices: /dev/sda, /dev/sdb, dev/sdc, and /dev/sdd. DM Multipath creates a single device with a unique WWID that reroutes I/O to those four underlying devices according to the multipath configuration.

    For our case since we have two LUNs connected to the Linux box hence we see 8 devices
    # ls -ld /sys/block/sd*/device 
    lrwxrwxrwx 1 root root 0 May 22 18:29 /sys/block/sda/device -> ../../../0:0:0:0
    lrwxrwxrwx 1 root root 0 May 22 18:29 /sys/block/sdb/device -> ../../../0:0:0:1
    lrwxrwxrwx 1 root root 0 May 22 18:29 /sys/block/sdc/device -> ../../../0:0:1:0
    lrwxrwxrwx 1 root root 0 May 22 18:29 /sys/block/sdd/device -> ../../../0:0:1:1
    lrwxrwxrwx 1 root root 0 May 22 18:29 /sys/block/sde/device -> ../../../1:0:0:0
    lrwxrwxrwx 1 root root 0 May 22 18:29 /sys/block/sdf/device -> ../../../1:0:0:1
    lrwxrwxrwx 1 root root 0 May 22 18:29 /sys/block/sdg/device -> ../../../1:0:1:0
    lrwxrwxrwx 1 root root 0 May 22 18:29 /sys/block/sdh/device -> ../../../1:0:1:1

    The same can be seen via below command where LUN to device mapping does easier on the eyes
    # ls -l  /dev/disk/by-path/ 
    total 0
    lrwxrwxrwx 1 root root 9 May 22 18:29 pci-0000:04:00.2-fc-0x207000c0ff13d3a7-lun-0 -> ../../sda
    lrwxrwxrwx 1 root root 9 May 22 18:29 pci-0000:04:00.2-fc-0x207000c0ff13d3a7-lun-1 -> ../../sdb
    lrwxrwxrwx 1 root root 9 May 22 18:29 pci-0000:04:00.2-fc-0x247000c0ff13d3a7-lun-0 -> ../../sdc
    lrwxrwxrwx 1 root root 9 May 22 18:29 pci-0000:04:00.2-fc-0x247000c0ff13d3a7-lun-1 -> ../../sdd
    lrwxrwxrwx 1 root root 9 May 22 18:29 pci-0000:04:00.3-fc-0x217000c0ff13d3a7-lun-0 -> ../../sdg
    lrwxrwxrwx 1 root root 9 May 22 18:29 pci-0000:04:00.3-fc-0x217000c0ff13d3a7-lun-1 -> ../../sdh
    lrwxrwxrwx 1 root root 9 May 22 18:29 pci-0000:04:00.3-fc-0x257000c0ff13d3a7-lun-0 -> ../../sde
    lrwxrwxrwx 1 root root 9 May 22 18:29 pci-0000:04:00.3-fc-0x257000c0ff13d3a7-lun-1 -> ../../sdf


    Understanding the Device ID

    In multipath output we see below highlighted entry being used (and also in the commands above you would see similar representation)
    # multipath -l
    3600c0ff00013b88f9058e35a01000000 dm-0 HP      ,P2000 G3 FC
    size=102G features='1 queue_if_no_path' hwhandler='0' wp=rw
    |-+- policy='service-time 0' prio=0 status=active
    | |- 0:0:0:0 sda 8:0   active undef unknown
    | `- 1:0:1:0 sdg 8:96  active undef unknown
    `-+- policy='service-time 0' prio=0 status=enabled
      |- 0:0:1:0 sdc 8:32  active undef unknown
      `- 1:0:0:0 sde 8:64  active undef unknown
    3600c0ff00013b88f9c58e35a01000000 dm-6 HP      ,P2000 G3 FC
    size=359G features='1 queue_if_no_path' hwhandler='0' wp=rw
    |-+- policy='service-time 0' prio=0 status=active
    | |- 0:0:0:1 sdb 8:16  active undef unknown
    | `- 1:0:1:1 sdh 8:112 active undef unknown
    `-+- policy='service-time 0' prio=0 status=enabled
      |- 0:0:1:1 sdd 8:48  active undef unknown
      `- 1:0:0:1 sdf 8:80  active undef unknown

    Below value explains what each field means
    1:0:0:1
    ^ ^ ^ ^
    | | | |
       
    H C T L

    Where H is the HBA number, C is the channel on the HBA, T is the SCSI target ID, and L is the LUN from the Storage.


    Flush/Remove all the multipath device

    # multipath -F




    Flush/Remove selected multipath device.

    First of all the the WWID of the multipath device which you want to flush using multipath -ll
    # multipath -ll
    3600c0ff00013b88fdc53e35a01000000 dm-0 HP      ,P2000 G3 FC
    size=186G features='1 queue_if_no_path' hwhandler='0' wp=rw
    |-+- policy='service-time 0' prio=50 status=active
    | |- 1:0:0:0 sda 8:0  active ready running
    | `- 2:0:1:0 sdd 8:48 active ready running
    `-+- policy='service-time 0' prio=10 status=enabled
      |- 1:0:1:0 sdb 8:16 active ready running
      `- 2:0:0:0 sdc 8:32 active ready running

    Next flush the selected multipath using below command
    # multipath -f 3600c0ff000144876065be35a01000000

    NOTE: This multipath must not be in used state


    Show the existing configuration in detail

    Below command will show the existing configuration loaded on your Linux box
    # multipathd show config
    defaults {
            verbosity 2
            polling_interval 5
            max_polling_interval 20
            reassign_maps "yes"
            multipath_dir "/lib64/multipath"
            path_selector "service-time 0"
            path_grouping_policy "failover"
            uid_attribute "ID_SERIAL"
            prio "const"
            prio_args ""
            features "0"
            path_checker "directio"
            alias_prefix "mpath"
            failback "immediate"
            rr_min_io 1000
            rr_min_io_rq 1
            max_fds 1048576
            rr_weight "uniform"
            queue_without_daemon "no"
            flush_on_last_del "no"
            user_friendly_names "no"
            fast_io_fail_tmo 5
            bindings_file "/etc/multipath/bindings"
            wwids_file /etc/multipath/wwids
            prkeys_file /etc/multipath/prkeys
            log_checker_err always
            find_multipaths no
            retain_attached_hw_handler no
            detect_prio no
            detect_path_checker no
            hw_str_match no
            force_sync no
            deferred_remove no
            ignore_new_boot_devs no
            skip_kpartx no
            config_dir "/etc/multipath/conf.d"
            delay_watch_checks no
            delay_wait_checks no
            retrigger_tries 3
            retrigger_delay 10
            missing_uev_wait_timeout 30
            new_bindings_in_boot no
            remove_retries 0
            disable_changed_wwids no
            unpriv_sgio no
    }
    blacklist {
            devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
            devnode "^hd[a-z][[0-9]*]"
            devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
            devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
            devnode "^(td|hd|vd)[a-z]"
            devnode "^dcssblk[0-9]*"
            device {
                    vendor "DGC"
                    product "LUNZ"
            }
            device {
                    vendor "EMC"
                    product "LUNZ"
            }
            device {
                    vendor "IBM"
                    product "Universal Xport"
            }
            device {
                    vendor "IBM"
                    product "S/390.*"
            }
            device {
                    vendor "DELL"
                    product "Universal Xport"
            }
            device {
                    vendor "SGI"
                    product "Universal Xport"
            }
            device {
                    vendor "STK"
                    product "Universal Xport"
            }
            device {
                    vendor "SUN"
                    product "Universal Xport"
            }
            device {
                    vendor "(NETAPP|LSI|ENGENIO)"
                    product "Universal Xport"
            }
    }
    blacklist_exceptions {
    }
    devices {
            device {
                    vendor "HP"
                    product "MSA2012sa|MSA23(12|24)(fc|i|sa)|MSA2000s VOLUME"
                    path_grouping_policy "group_by_prio"
                    path_checker "tur"
                    features "0"
                    hardware_handler "0"
                    prio "alua"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry 18
                    rr_min_io 100
            }
            device {
                    vendor "HP"
                    product "MSA (1|2)040 SA(N|S)"
                    path_grouping_policy "group_by_prio"
                    path_checker "tur"
                    features "0"
                    hardware_handler "0"
                    prio "alua"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry 18
                    rr_min_io 100
            }
            device {
                    vendor "HP"
                    product "HSVX700"
                    path_grouping_policy "group_by_prio"
                    path_checker "tur"
                    features "0"
                    hardware_handler "1 alua"
                    prio "alua"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry 12
                    rr_min_io 100
            }
            device {
                    vendor "HP"
                    product "LOGICAL VOLUME.*"
                    path_grouping_policy "multibus"
                    path_checker "tur"
                    features "0"
                    hardware_handler "0"
                    prio "const"
                    rr_weight "uniform"
                    no_path_retry 12
            }
            device {
                    vendor "HP"
                    product "P2000 G3 FC|P2000G3 FC/iSCSI|P2000 G3 SAS|P2000 G3 iSCSI"
                    path_grouping_policy "group_by_prio"
                    path_checker "tur"
                    features "0"
                    hardware_handler "0"
                    prio "alua"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry 18
                    rr_min_io 100
            }
            device {
                    vendor "IBM"
                    product "^1722-600"
                    product_blacklist "Universal Xport"
                    path_grouping_policy "group_by_prio"
                    path_checker "rdac"
                    features "1 queue_if_no_path"
                    hardware_handler "1 rdac"
                    prio "rdac"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry 300
            }
            device {
                    vendor "IBM"
                    product "^1724"
                    product_blacklist "Universal Xport"
                    path_grouping_policy "group_by_prio"
                    path_checker "rdac"
                    features "1 queue_if_no_path"
                    hardware_handler "1 rdac"
                    prio "rdac"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry 300
            }
            device {
                    vendor "IBM"
                    product "^1726"
                    product_blacklist "Universal Xport"
                    path_grouping_policy "group_by_prio"
                    path_checker "rdac"
                    features "1 queue_if_no_path"
                    hardware_handler "1 rdac"
                    prio "rdac"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry 300
            }
            device {
                    vendor "IBM"
                    product "^1742"
                    product_blacklist "Universal Xport"
                    path_grouping_policy "group_by_prio"
                    path_checker "rdac"
                    features "0"
                    hardware_handler "1 rdac"
                    prio "rdac"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry "queue"
            }
            device {
                    vendor "IBM"
                    product "^1745|^1746"
                    product_blacklist "Universal Xport"
                    path_grouping_policy "group_by_prio"
                    path_checker "rdac"
                    features "2 pg_init_retries 50"
                    hardware_handler "1 rdac"
                    prio "rdac"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry 15
            }
            device {
                    vendor "IBM"
                    product "^1814"
                    product_blacklist "Universal Xport"
                    path_grouping_policy "group_by_prio"
                    path_checker "rdac"
                    features "0"
                    hardware_handler "1 rdac"
                    prio "rdac"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry "queue"
            }
            device {
                    vendor "IBM"
                    product "^1815"
                    product_blacklist "Universal Xport"
                    path_grouping_policy "group_by_prio"
                    path_checker "rdac"
                    features "0"
                    hardware_handler "1 rdac"
                    prio "rdac"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry "queue"
            }
            device {
                    vendor "IBM"
                    product "^1818"
                    product_blacklist "Universal Xport"
                    path_grouping_policy "group_by_prio"
                    path_checker "rdac"
                    features "0"
                    hardware_handler "1 rdac"
                    prio "rdac"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry "queue"
            }
            device {
                    vendor "IBM"
                    product "^3526"
                    product_blacklist "Universal Xport"
                    path_grouping_policy "group_by_prio"
                    path_checker "rdac"
                    features "0"
                    hardware_handler "1 rdac"
                    prio "rdac"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry "queue"
            }
            device {
                    vendor "IBM"
                    product "^3542"
                    path_grouping_policy "group_by_serial"
                    path_checker "tur"
                    features "0"
                    hardware_handler "0"
                    prio "const"
                    rr_weight "uniform"
            }
            device {
                    vendor "IBM"
                    product "^2105800"
                    path_grouping_policy "group_by_serial"
                    path_checker "tur"
                    features "1 queue_if_no_path"
                    hardware_handler "0"
                    prio "const"
                    rr_weight "uniform"
            }
            device {
                    vendor "IBM"
                    product "2810XIV"
                    path_grouping_policy "multibus"
                    path_checker "tur"
                    features "1 queue_if_no_path"
                    hardware_handler "0"
                    prio "const"
                    failback 15
                    rr_weight "uniform"
                    rr_min_io 15
            }
            device {
                    vendor "AIX"
                    product "VDASD"
                    path_grouping_policy "multibus"
                    path_checker "directio"
                    features "0"
                    hardware_handler "0"
                    prio "const"
                    failback immediate
                    rr_weight "uniform"
                    no_path_retry 60
            }
            device {
                    vendor "PURE"
                    product "FlashArray"
                    path_grouping_policy "multibus"
                    path_selector "queue-length 0"
                    path_checker "tur"
                    features "0"
                    hardware_handler "0"
                    prio "const"
                    failback immediate
                    fast_io_fail_tmo 10
                    dev_loss_tmo 60
                    user_friendly_names no
            }
            device {
                    vendor "HUAWEI"
                    product "XSG1"
                    path_grouping_policy "multibus"
                    path_checker "tur"
                    features "0"
                    hardware_handler "0"
                    failback immediate
            }
            device {
                    vendor "Ceph"
                    product "RBD"
                    path_grouping_policy "failover"
                    path_checker "rbd"
                    no_path_retry "fail"
            }
    }
    multipaths {
    }


    Get complete HBA details


    systool uses APIs provided by libsysfs to gather information and will give detailed information about the available HBA and the LUN status

    NOTE: systool is delivered with sysfsutils rpm so make sure it is installed before using this tool
    # systool -c fc_host -v
    Class = "fc_host"

      Class Device = "host0"
      Class Device path = "/sys/devices/pci0000:00/0000:00:02.0/0000:04:00.2/host0/fc_host/host0"
        active_fc4s         = "0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 "
        dev_loss_tmo        = "30"
        fabric_name         = "0x2003547fee1188d9"
        issue_lip           = <store method only>
        max_npiv_vports     = "255"
        maxframe_size       = "2048 bytes"
        node_name           = "0x50060b0000c2a67d"
        npiv_vports_inuse   = "0"
        port_id             = "0x310305"
        port_name           = "0x50060b0000c2a67c"
        port_state          = "Online"
        port_type           = "NPort (fabric via point-to-point)"
        speed               = "unknown"
        supported_classes   = "Class 3"
        supported_fc4s      = "0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 "
        supported_speeds    = "10 Gbit"
        symbolic_name       = "Emulex 554FLB FV11.1.183.23 DV11.2.0.6 HN:ban33-be002-2b OS:Linux"
        tgtid_bind_type     = "wwpn (World Wide Port Name)"
        uevent              =
        vport_create        = <store method only>
        vport_delete        = <store method only>

        Device = "host0"
        Device path = "/sys/devices/pci0000:00/0000:00:02.0/0000:04:00.2/host0"
          uevent              = "DEVTYPE=scsi_host"


      Class Device = "host1"
      Class Device path = "/sys/devices/pci0000:00/0000:00:02.0/0000:04:00.3/host1/fc_host/host1"
        active_fc4s         = "0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 "
        dev_loss_tmo        = "30"
        fabric_name         = "0x2003547fee357a81"
        issue_lip           = <store method only>
        max_npiv_vports     = "255"
        maxframe_size       = "2048 bytes"
        node_name           = "0x50060b0000c2a67f"
        npiv_vports_inuse   = "0"
        port_id             = "0x25030f"
        port_name           = "0x50060b0000c2a67e"
        port_state          = "Online"
        port_type           = "NPort (fabric via point-to-point)"
        speed               = "unknown"
        supported_classes   = "Class 3"
        supported_fc4s      = "0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 "
        supported_speeds    = "10 Gbit"
        symbolic_name       = "Emulex 554FLB FV11.1.183.23 DV11.2.0.6 HN:ban33-be002-2b OS:Linux"
        tgtid_bind_type     = "wwpn (World Wide Port Name)"
        uevent              =
        vport_create        = <store method only>
        vport_delete        = <store method only>

        Device = "host1"
        Device path = "/sys/devices/pci0000:00/0000:00:02.0/0000:04:00.3/host1"
          uevent              = "DEVTYPE=scsi_host"

    This tool can also be used to get other values using below syntax

    To get the port name
    # systool -c fc_host -A port_name
    Class = "fc_host"

      Class Device = "host0"
        port_name           = "0x50060b0000c2a67c"

        Device = "host0"


      Class Device = "host1"
        port_name           = "0x50060b0000c2a67e"

        Device = "host1"



    To get the port id
    # systool -c fc_host -A port_id
    Class = "fc_host"

      Class Device = "host0"
        port_id             = "0x310305"

        Device = "host0"


      Class Device = "host1"
        port_id             = "0x25030f"

        Device = "host1"


    Show the existing configuration in brief

    This is an alternate command which can be used to check the loaded configuration for selected values as shown below.
    # mpathconf
    multipath is enabled
    find_multipaths is disabled
    user_friendly_names is enabled
    dm_multipath module is loaded
    multipathd is running


    Using user friendly names for multipath

    Each multipath device has a World Wide Identifier (WWID), which is guaranteed to be globally unique and unchanging. By default, the name of a multipath device is set to its WWID. Alternately, you can set the user_friendly_names option in the multipath configuration file, which sets the alias to a nodeunique name of the form mpathn

    With user_friendly name disabled we will only see the WWID of the multipath connected
    # multipath -l
    3600c0ff000144876f85ae35a01000000 dm-0 HP      ,P2000 G3 FC
    size=102G features='1 queue_if_no_path' hwhandler='0' wp=rw
    |-+- policy='service-time 0' prio=0 status=active
    | `- 2:0:0:0 sda 8:0  active undef unknown
    `-+- policy='service-time 0' prio=0 status=enabled
      `- 2:0:1:0 sdc 8:32 active undef unknown
    3600c0ff000144876065be35a01000000 dm-6 HP      ,P2000 G3 FC
    size=359G features='1 queue_if_no_path' hwhandler='0' wp=rw
    |-+- policy='service-time 0' prio=0 status=active
    | `- 2:0:0:1 sdb 8:16 active undef unknown
    `-+- policy='service-time 0' prio=0 status=enabled
      `- 2:0:1:1 sdd 8:48 active undef unknown

    Next enable user_friendly name
    # /sbin/mpathconf --user_friendly_names y

    Check the multipath
    # multipath -l
    mpathb (3600c0ff000144876065be35a01000000) dm-6 HP      ,P2000 G3 FC
    size=359G features='1 queue_if_no_path' hwhandler='0' wp=rw
    |-+- policy='service-time 0' prio=0 status=active
    | `- 2:0:0:1 sdb 8:16 active undef unknown
    `-+- policy='service-time 0' prio=0 status=enabled
      `- 2:0:1:1 sdd 8:48 active undef unknown
    mpatha (3600c0ff000144876f85ae35a01000000) dm-0 HP      ,P2000 G3 FC
    size=102G features='1 queue_if_no_path' hwhandler='0' wp=rw
    |-+- policy='service-time 0' prio=0 status=active
    | `- 2:0:0:0 sda 8:0  active undef unknown
    `-+- policy='service-time 0' prio=0 status=enabled
      `- 2:0:1:0 sdc 8:32 active undef unknown

    The mapping is done using /etc/multipath/bindings.


    Check the number of HBA on your Linux box

    # ls -l /sys/class/fc_host/
    total 0
    lrwxrwxrwx 1 root root 0 May 22 18:29 host0 -> ../../devices/pci0000:00/0000:00:02.0/0000:04:00.2/host0/fc_host/host0
    lrwxrwxrwx 1 root root 0 May 22 18:29 host1 -> ../../devices/pci0000:00/0000:00:02.0/0000:04:00.3/host1/fc_host/host1

    You can also check the same using
    # lspci -nn |grep -i "Fibre"
    04:00.2 Fibre Channel [0c04]: Emulex Corporation OneConnect 10Gb FCoE Initiator (be3) [19a2:0714] (rev 01)
    04:00.3 Fibre Channel [0c04]: Emulex Corporation OneConnect 10Gb FCoE Initiator (be3) [19a2:0714] (rev 01)

    On some Linux variant you can also grep for HBA
    # lspci -nn |grep -i "HBA"

    So here I have two HBA




    Check the WWPN of the multipath

    A worldwide port number (WWPN) is unique identifier for each FC port of any Fibre Channel device.
    • For server, we have a WWPN for each port of the HBA.
    • For a SAN switch, the WWPN is available for each port in the chassis.
    • For Storage, each host port has an individual WWPN

    Here we have two HBA so each HBA will have unique World Wide Port Name (WWPN) which will be connected to the SAN switch then to the Storage Box.

    From my Linux Box
    # cat /sys/class/fc_host/host0/port_name
    0x50060b0000c2a67c

    # cat /sys/class/fc_host/host1/port_name
    0x50060b0000c2a67e

    These are the WWPN number.


    Check the WWNN of the multipath

    A worldwide node name (WWNN) is a globally unique 64-bit identifier that is assigned to each Fibre Channel node or device.
    • For servers and hosts, WWNN is unique for each HBA (host bus adapter), and in a case of a server with two HBAs, they have two WWNNs.
    • For a SAN switch, the WWNN is a common for the chassis.
    • For storage, the WWNN is common for each controller unit of midrage storage

    Since here I have two HBA hence there are two WWNN for each HBA

    From my Linux Box
    # cat /sys/class/fc_host/host1/node_name
    0x50060b0000c2a67f

    # cat /sys/class/fc_host/host0/node_name
    0x50060b0000c2a67d


    Perform rescan of the connected HBA for new LUNs

    If you have new LUNs connected to your Linux box then you will need to rescan the HBA

    Assuming you know the WWPN for which the new LUN was added
    # echo 1 > /sys/class/fc_host/host2/issue_lip

    # systemctl reload multipathd

    # multipath -v2

    Next validate if the new LUN is visible using
    # multipath -ll


    Blacklist a device

    Create a function like below in /etc/multipath.conf where you can give the list of devices which you wish to blacklist

    Here I am blacklisting multiple devices which I do not wish to be created as multipath
    blacklist {
            devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
            devnode "^hd[a-z][[0-9]*]"
            devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
    }

    To blacklist using WWID

    You can get the WWID of a device using below command
    # /usr/lib/udev/scsi_id  -g -u -d /dev/sda
    3600c0ff000144876f85ae35a01000000

    Here we are blacklisting /dev/sda which is our internal disk and I donot want it to be a multipath
    blacklist {
    wwid 3600c0ff000144876f85ae35a01000000
    }

    Perform a multipathd reload to take the changes affect
    # systemctl reload multipathd.service


    Add blacklist exception for selected disks

    Similar to blacklisting we can also add a blacklist exception so that these devices are not blacklisted

    Add below function in /etc/multipath.conf with the list of wwid which you wish to add as an exception for blacklist
    blacklist_exceptions {
    wwid "3600d0230000000000e13955cc3757803"
    }

    Followed by reload of multipathd
    # systemctl reload multipathd.service


    Change configuration (path selector) of the multipath device

    Below is the total list of supported path_selector


    Add your choice of path_selector in /etc/multipath.conf as shown below
    defaults {
            user_friendly_names yes
            failback                immediate
            path_selector           "round-robin 0"
    }

    Either you add it under defaults function or you can add a device function if you wish to add to change path selector only for a selected device
    devices {
       device {
            vendor "HP"
            product "P2000 G3 FC|P2000G3 FC/iSCSI|P2000 G3 SAS|P2000 G3 iSCSI"
            path_grouping_policy "failover"
            path_selector "round-robin 0"
       }
    }

    NOTE: You can get the device detail from "multipathd show config"

    Add below changes
    devices {
            device {
                    vendor "HP"
                    product "P2000 G3 FC|P2000G3 FC/iSCSI|P2000 G3 SAS|P2000 G3 iSCSI"
                    user_friendly_names yes
            }
    }

    I hope the article was useful.

    Deepak Prasad

    is a techie and an author who is still trying to survive in this IT generation with very little knowledge he has on Linux/Unix, VMware, SAN Storage, Automation, networking etc

    You can follow him on Facebook or Google+


    Do you also have something to share here?
    Join GoLinuxHub Team as an Author, Click here for more information
    Tutorial/Cheatsheet: Begineer's Guide to Understanding Device Mapper Multipath for Linux Tutorial/Cheatsheet: Begineer's Guide to Understanding Device Mapper Multipath for Linux Reviewed by Deepak Prasad on Thursday, May 24, 2018 Rating: 5

    No comments:

    Powered by Blogger.