Skip to main content

iSCSI and multipath on CentOS 7

系統環境
  • iSCSI Storage: NetApp FAS8200
  • Linux OS: CentOS 7
  • Server H/W: HP DL380 Gen10
  • iSCSI 用網路卡: 10G SFP 光纖網路卡 x 2
設定網路卡

使用指令 nmtui 分別設定兩張網卡的 IP 與 Mask

注意: 這兩張網卡不要作 Bonding。

設定 iSCSI

安裝 iscsi 套件

yum install iscsi-initiator-utils

設定 iscsid.conf: /etc/iscsi/iscsid.conf

# 調整 timeout
#node.session.timeo.replacement_timeout = 120
node.session.timeo.replacement_timeout = 5

# 調整 node 自動 Login
node.startup = automatic

設定 initiator name: /etc/iscsi/initiatorname.iscsi

命名規則可以參考

  • iqn.<year-to-date>.<your-domain>.<your-hostname>:redhat-disk
    例如 InitiatorName=iqn.2019-02.mycompany.bdb7-a:redhat-disk
  • initiator name 必須與 Storage 端的設定相同
  • 不同的 Host 之間的名稱不能相同

安裝 multipath 與 netapp_linux_unified_host_utilities
yum install device-mapper-multipath
yum install libhbaapi
rpm -ivh netapp_linux_unified_host_utilities-7-1.x86_64.rpm

如果沒有安裝 libhbaapi,執行 sanlun 會出現錯誤。

連接 iSCSI storage

新增網卡裝置

iscsiadm -m iface -I iscsi_ens2f0 --op=new
iscsiadm -m iface -I iscsi_ens3f0 --op=new

Discovery for the storage

iscsiadm -m discovery -t sendtargets -p <ip-to-iscsi-storage>:3260

如果 NetApp storage 有設置多個 iSCSI IPs,只需要對其中一個 IP 作偵測,輸出結果就會顯示所有 IP。

[root@bdb7-b RPMs]# iscsiadm -m discovery -t sendtargets -p 10.18.104.32:3260
    10.18.104.32:3260,1038 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.34:3260,1040 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.33:3260,1039 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.31:3260,1037 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.32:3260,1038 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.34:3260,1040 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.33:3260,1039 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.31:3260,1037 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16

Login nodes

iscsiadm -m node -l all

輸出結果會顯示多行如下資訊

[root@bdb7-b RPMs]# iscsiadm -m node -l all
    Logging in to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.32,3260] (multiple)
    Logging in to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.32,3260] (multiple)
    Logging in to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.34,3260] (multiple)
    Logging in to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.34,3260] (multiple)
    Logging in to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.33,3260] (multiple)
    Logging in to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.33,3260] (multiple)
    Logging in to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.31,3260] (multiple)
    Logging in to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.31,3260] (multiple)
    Login to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.32,3260] successful.
    Login to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.32,3260] successful.
    Login to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.34,3260] successful.
    Login to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.34,3260] successful.
    Login to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.33,3260] successful.
    Login to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.33,3260] successful.
    Login to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.31,3260] successful.
    Login to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.31,3260] successful.

使用 sanlun 確認 LUN Disks

[root@bdb7-b RPMs]# sanlun lun show
    controller(7mode/E-Series)/                                  device          host                  lun
    vserver(cDOT/FlashRay)        lun-pathname                   filename        adapter    protocol   size    product
    ---------------------------------------------------------------------------------------------------------------
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdag       host19     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdaf       host20     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdae       host17     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdad       host18     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdac       host16     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdab       host15     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdz        host20     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdaa       host19     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdy        host14     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdx        host18     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdw        host17     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdt        host16     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdu        host15     iSCSI      500.1g  cDOT
    ...
設定 multipath