互联网服务SAN（存储区域网络）
【笔记】san学习笔记
内容写的不一定都对，如果错了，希望大家能够指出，谢谢！
hp-ux
# fcmsutil /dev/td0
Vendor ID is = 0x00103c
Device ID is = 0x001028
TL Chip Revision No is = 2.3
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x000006
Topology = PTTOPT_FABRIC
Local N_Port_id is = 0x021700
N_Port Node World Wide Name = 0x50060b00000781df
N_Port Port World Wide Name = 0x50060b00000781de
Driver state = ONLINE
Hardware Path is = 0/4/0/0
Number of Assisted IOs = 1188
Number of Active Login Sessions = 0
拓扑PTTOPT_FABRIC表示连接到switch
Local N_Port_id表示连接到switch的位置
0x021700中最左边的byte1（02）表示domain id
第3个字节的17中1表示switch board 这里写的有问题
7表示switch board的port号
如果是switch连接，则字节3总是0
今天在另一个switch上面做一实验，发现这里写的有问题
应该17表示port
17是16进制，10进制就是23，表示端口23，然后在switch上面在查看时那个blade上的那个端口
下面进行验证，因为我没有aix小机，只能验证hp-ux
# fcmsutil /dev/fcd0
Vendor ID is = 0x1077
Device ID is = 0x2422
PCI Sub-system Vendor ID is = 0x103C
PCI Sub-system ID is = 0x12D6
PCI Mode = PCI-X 266 MHz
ISP Code version = 4.4.4
ISP Chip version = 3
Topology = PTTOPT_FABRIC
Link Speed = 4Gb
Local N_Port_id is = 0x010500
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x50060b000061228f
N_Port Port World Wide Name = 0x50060b000061228e
Switch Port World Wide Name = 0x200500051e885d42
Switch Node World Wide Name = 0x100000051e885d42
N_Port Symbolic Port Name = rx6600-2_fcd0
N_Port Symbolic Node Name = rx6600-2_HP-UX_B.11.31
Driver state = ONLINE
Hardware Path is = 0/3/1/0
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) fcd B.11.31.0909 Jun 8 2009
# fcmsutil /dev/fcd1
Vendor ID is = 0x1077
Device ID is = 0x2422
PCI Sub-system Vendor ID is = 0x103C
PCI Sub-system ID is = 0x12D6
PCI Mode = PCI-X 266 MHz
ISP Code version = 4.4.4
ISP Chip version = 3
Topology = PTTOPT_FABRIC
Link Speed = 4Gb
Local N_Port_id is = 0x020a00
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x50060b000061256b
N_Port Port World Wide Name = 0x50060b000061256a
Switch Port World Wide Name = 0x200a00051e882b89
Switch Node World Wide Name = 0x100000051e882b89
N_Port Symbolic Port Name = rx6600-2_fcd1
N_Port Symbolic Node Name = rx6600-2_HP-UX_B.11.31
Driver state = ONLINE
Hardware Path is = 0/7/1/0
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) fcd B.11.31.0909 Jun 8 2009
可以看出fcd0是连接到domain id为1的switch的port5，fcd1是连接到domain id为2的switch的port10
下面登录到相关的san switch 上面验证
swd77:admin> switchsshow
rbash: switchsshow: command not found
swd77:admin> switchshow
switchName: swd77
switchType: 71.2
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 1
switchId: fffc01
switchWwn: 10:00:00:05:1e:88:5d:42
zoning: ON (xp1200)
switchBeacon: OFF
Index Port Address Media Speed State Proto
==============================================
0 0 010000 id N2 Online FC L-Port
1 1 010100 id N4 Online FC F-Port 50:01:43:80:00:c4:18:40
2 2 010200 id N2 Online FC F-Port 50:06:0e:80:04:7d:db:00
3 3 010300 id N8 No_Light FC
4 4 010400 id N4 Online FC F-Port 10:00:00:00:c9:87:ae:d8
5 5 010500 id N4 Online FC F-Port 50:06:0b:00:00:61:22:8e
6 6 010600 id N4 Online FC F-Port 10:00:00:00:c9:88:43:5a
7 7 010700 id N8 No_Light FC
8 8 010800 id N8 No_Light FC
9 9 010900 id N8 No_Light FC
10 10 010a00 id N8 No_Light FC
11 11 010b00 id N2 Online FC F-Port 50:06:0e:80:04:7d:db:10
12 12 010c00 id N4 Online FC F-Port 51:8a:90:56:e2:f4:a0:10
13 13 010d00 id N4 Online FC F-Port 50:01:43:80:01:32:12:68
14 14 010e00 id N4 Online FC F-Port 10:00:00:e0:02:43:e2:0a
15 15 010f00 id N4 Online FC F-Port 10:00:00:e0:02:23:5c:c8
16 16 011000 id N8 No_Light FC (No POD License) Disabled
17 17 011100 id N8 No_Light FC (No POD License) Disabled
18 18 011200 id N8 No_Light FC (No POD License) Disabled
19 19 011300 id N8 No_Light FC (No POD License) Disabled
20 20 011400 id N8 No_Light FC (No POD License) Disabled
21 21 011500 id N8 No_Light FC (No POD License) Disabled
22 22 011600 id N8 No_Light FC (No POD License) Disabled
23 23 011700 id N8 No_Light FC (No POD License) Disabled
然后再登录到domain为2的san switch上面验证
sw11:admin> switchshsow
rbash: switchshsow: command not found
sw11:admin> switchshow
switchName: sw11
switchType: 71.2
switchState: Online
switchMode: Native
switchRole: Subordinate
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:00:05:1e:88:2b:89
zoning: ON (testcfg)
switchBeacon: OFF
Index Port Address Media Speed State Proto
==============================================
0 0 020000 id N4 Online FC F-Port 10:00:00:e0:02:43:5c:c8
1 1 020100 id N2 Online FC L-Port
2 2 020200 id N2 Online FC F-Port 50:06:0e:80:04:7d:db:02
3 3 020300 id N4 Online FC F-Port 51:8a:90:56:e2:f4:a0:00
4 4 020400 id N4 Online FC F-Port 10:00:00:00:c9:87:a9:bc
5 5 020500 id N4 Online FC F-Port 50:01:43:80:00:c4:43:38
6 6 020600 id N8 No_Light FC
7 7 020700 id N8 No_Light FC
8 8 020800 id N4 Online FC F-Port 50:01:43:80:01:32:13:2c
9 9 020900 id N4 Online FC F-Port 10:00:00:e0:02:23:e2:0a
10 10 020a00 id N4 Online FC F-Port 50:06:0b:00:00:61:25:6a
11 11 020b00 id N8 No_Light FC
12 12 020c00 id N4 Online FC F-Port 50:06:0b:00:00:69:fa:1e
13 13 020d00 id N4 Online FC E-Port 10:00:00:05:1e:04:b3:25 ")
14 14 020e00 id N4 Online FC F-Port 50:06:0b:00:00:69:f9:0e
15 15 020f00 id N8 No_Light FC Disabled
16 16 021000 id N8 No_Light FC (No POD License) Disabled
17 17 021100 id N8 No_Light FC (No POD License) Disabled
18 18 021200 id N8 No_Light FC (No POD License) Disabled
19 19 021300 id N8 No_Light FC (No POD License) Disabled
20 20 021400 id N8 No_Light FC (No POD License) Disabled
21 21 021500 id N8 No_Light FC (No POD License) Disabled
22 22 021600 id N8 No_Light FC (No POD License) Disabled
23 23 021700 id N8 No_Light FC (No POD License) Disabled
先再举个例子
fcmsutil /dev/fclp2
Vendor ID is = 0x10df
Device ID is = 0xfe00
PCI Sub-system Vendor ID is = 0x103c
PCI Sub-system ID is = 0x7046
Chip version = 2
Firmware Version = 2.70X5 SLI-3 (Z3D2.70X5)
EFI Version = ZE3.21A3
EFI Boot = ENABLED
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Topology = PTTOPT_FABRIC
Link Speed = 4Gb
Local N_Port_id is = 0x789400
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x5001438002a2b3df
N_Port Port World Wide Name = 0x5001438002a2b3de
Switch Port World Wide Name = 0x209400051e3627aa
Switch Node World Wide Name = 0x100000051e3627aa
Driver state = ONLINE
Hardware Path is = 1/0/12/0/0/0/0
Maximum Frame Size = 2048
Driver Version = @(#) FCLP: PCIe Fibre Channel driver (FibrChanl-02), B.11.31.0903, Dec 12 2008, FCLP_IFC (3,2)
Local N_Port_id is = 0x789400这里的0x表示是16进制，78表示这个hba卡连接的san switch的domain id，换算为16进制是120，表示连接的domain id是120的交换机，94表示端口148
#[/]ioscan -m hwpath
Lun H/W Path Lunpath H/W Path Legacy H/W Path
====================================================================
64000/0xfa00/0x18
0/0/12/0/0/0/0.0x50060e80047f3168.0x4001000000000000 0/0/12/0/0/0/0.120.220.0.0.0.1
0/0/14/0/0/0/0.0x50060e80047f3178.0x4001000000000000 0/0/14/0/0/0/0.110.220.0.0.0.1
1/0/12/0/0/0/0.0x50060e80047f3168.0x4001000000000000 1/0/12/0/0/0/0.120.220.0.0.0.1
1/0/14/0/0/0/0.0x50060e80047f3178.0x4001000000000000 1/0/14/0/0/0/0.110.220.0.0.0.1
我们可以看到这个disk，他有4条路径，分别通过4个hba，hba卡和存储端口是2对1，即2个hba卡对一个存储端口，
0/0/12/0/0/0/0表示hba卡的硬件地址，
0x50060e80047f3178表示存储端口的wwn号，hba卡1和3都对应存储同一个的端口，
0x4001000000000000中1表示这个lun的lunid是1，
0/0/12/0/0/0/0.120.220.0.0.0.1中120表示存储连接的san switch的domain id，这里可以看出主机和存储连接的同一个交换机，
如果是这个domain id和刚才fcmsutil所有的hba卡看到的domain id都不一样，表示可能存储和主机连接的是不同的交换机，
然后交换机之间级联来做san的通信。从这里可以看出，这个lun是从存储的2个端口出来，分别连接110和120 san switch的220端口
最后的1也是表示lunid，逢8进1
hp-ux
# fcmsutil /dev/td0
Vendor ID is = 0x00103c
Device ID is = 0x001028
TL Chip Revision No is = 2.3
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x000006
Topology = PTTOPT_FABRIC
Local N_Port_id is = 0x021700
N_Port Node World Wide Name = 0x50060b00000781df
N_Port Port World Wide Name = 0x50060b00000781de
Driver state = ONLINE
Hardware Path is = 0/4/0/0
Number of Assisted IOs = 1188
Number of Active Login Sessions = 0
拓扑PTTOPT_FABRIC表示连接到switch
Local N_Port_id表示连接到switch的位置
0x021700中最左边的byte1（02）表示domain id
第3个字节的17中1表示switch board 这里写的有问题
7表示switch board的port号
如果是switch连接，则字节3总是0
今天在另一个switch上面做一实验，发现这里写的有问题
应该17表示port
17是16进制，10进制就是23，表示端口23，然后在switch上面在查看时那个blade上的那个端口
下面进行验证，因为我没有aix小机，只能验证hp-ux
# fcmsutil /dev/fcd0
Vendor ID is = 0x1077
Device ID is = 0x2422
PCI Sub-system Vendor ID is = 0x103C
PCI Sub-system ID is = 0x12D6
PCI Mode = PCI-X 266 MHz
ISP Code version = 4.4.4
ISP Chip version = 3
Topology = PTTOPT_FABRIC
Link Speed = 4Gb
Local N_Port_id is = 0x010500
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x50060b000061228f
N_Port Port World Wide Name = 0x50060b000061228e
Switch Port World Wide Name = 0x200500051e885d42
Switch Node World Wide Name = 0x100000051e885d42
N_Port Symbolic Port Name = rx6600-2_fcd0
N_Port Symbolic Node Name = rx6600-2_HP-UX_B.11.31
Driver state = ONLINE
Hardware Path is = 0/3/1/0
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) fcd B.11.31.0909 Jun 8 2009
# fcmsutil /dev/fcd1
Vendor ID is = 0x1077
Device ID is = 0x2422
PCI Sub-system Vendor ID is = 0x103C
PCI Sub-system ID is = 0x12D6
PCI Mode = PCI-X 266 MHz
ISP Code version = 4.4.4
ISP Chip version = 3
Topology = PTTOPT_FABRIC
Link Speed = 4Gb
Local N_Port_id is = 0x020a00
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x50060b000061256b
N_Port Port World Wide Name = 0x50060b000061256a
Switch Port World Wide Name = 0x200a00051e882b89
Switch Node World Wide Name = 0x100000051e882b89
N_Port Symbolic Port Name = rx6600-2_fcd1
N_Port Symbolic Node Name = rx6600-2_HP-UX_B.11.31
Driver state = ONLINE
Hardware Path is = 0/7/1/0
Maximum Frame Size = 2048
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Driver Version = @(#) fcd B.11.31.0909 Jun 8 2009
可以看出fcd0是连接到domain id为1的switch的port5，fcd1是连接到domain id为2的switch的port10
下面登录到相关的san switch 上面验证
swd77:admin> switchsshow
rbash: switchsshow: command not found
swd77:admin> switchshow
switchName: swd77
switchType: 71.2
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 1
switchId: fffc01
switchWwn: 10:00:00:05:1e:88:5d:42
zoning: ON (xp1200)
switchBeacon: OFF
Index Port Address Media Speed State Proto
==============================================
0 0 010000 id N2 Online FC L-Port
1 1 010100 id N4 Online FC F-Port 50:01:43:80:00:c4:18:40
2 2 010200 id N2 Online FC F-Port 50:06:0e:80:04:7d:db:00
3 3 010300 id N8 No_Light FC
4 4 010400 id N4 Online FC F-Port 10:00:00:00:c9:87:ae:d8
5 5 010500 id N4 Online FC F-Port 50:06:0b:00:00:61:22:8e
6 6 010600 id N4 Online FC F-Port 10:00:00:00:c9:88:43:5a
7 7 010700 id N8 No_Light FC
8 8 010800 id N8 No_Light FC
9 9 010900 id N8 No_Light FC
10 10 010a00 id N8 No_Light FC
11 11 010b00 id N2 Online FC F-Port 50:06:0e:80:04:7d:db:10
12 12 010c00 id N4 Online FC F-Port 51:8a:90:56:e2:f4:a0:10
13 13 010d00 id N4 Online FC F-Port 50:01:43:80:01:32:12:68
14 14 010e00 id N4 Online FC F-Port 10:00:00:e0:02:43:e2:0a
15 15 010f00 id N4 Online FC F-Port 10:00:00:e0:02:23:5c:c8
16 16 011000 id N8 No_Light FC (No POD License) Disabled
17 17 011100 id N8 No_Light FC (No POD License) Disabled
18 18 011200 id N8 No_Light FC (No POD License) Disabled
19 19 011300 id N8 No_Light FC (No POD License) Disabled
20 20 011400 id N8 No_Light FC (No POD License) Disabled
21 21 011500 id N8 No_Light FC (No POD License) Disabled
22 22 011600 id N8 No_Light FC (No POD License) Disabled
23 23 011700 id N8 No_Light FC (No POD License) Disabled
然后再登录到domain为2的san switch上面验证
sw11:admin> switchshsow
rbash: switchshsow: command not found
sw11:admin> switchshow
switchName: sw11
switchType: 71.2
switchState: Online
switchMode: Native
switchRole: Subordinate
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:00:05:1e:88:2b:89
zoning: ON (testcfg)
switchBeacon: OFF
Index Port Address Media Speed State Proto
==============================================
0 0 020000 id N4 Online FC F-Port 10:00:00:e0:02:43:5c:c8
1 1 020100 id N2 Online FC L-Port
2 2 020200 id N2 Online FC F-Port 50:06:0e:80:04:7d:db:02
3 3 020300 id N4 Online FC F-Port 51:8a:90:56:e2:f4:a0:00
4 4 020400 id N4 Online FC F-Port 10:00:00:00:c9:87:a9:bc
5 5 020500 id N4 Online FC F-Port 50:01:43:80:00:c4:43:38
6 6 020600 id N8 No_Light FC
7 7 020700 id N8 No_Light FC
8 8 020800 id N4 Online FC F-Port 50:01:43:80:01:32:13:2c
9 9 020900 id N4 Online FC F-Port 10:00:00:e0:02:23:e2:0a
10 10 020a00 id N4 Online FC F-Port 50:06:0b:00:00:61:25:6a
11 11 020b00 id N8 No_Light FC
12 12 020c00 id N4 Online FC F-Port 50:06:0b:00:00:69:fa:1e
13 13 020d00 id N4 Online FC E-Port 10:00:00:05:1e:04:b3:25 ")
14 14 020e00 id N4 Online FC F-Port 50:06:0b:00:00:69:f9:0e
15 15 020f00 id N8 No_Light FC Disabled
16 16 021000 id N8 No_Light FC (No POD License) Disabled
17 17 021100 id N8 No_Light FC (No POD License) Disabled
18 18 021200 id N8 No_Light FC (No POD License) Disabled
19 19 021300 id N8 No_Light FC (No POD License) Disabled
20 20 021400 id N8 No_Light FC (No POD License) Disabled
21 21 021500 id N8 No_Light FC (No POD License) Disabled
22 22 021600 id N8 No_Light FC (No POD License) Disabled
23 23 021700 id N8 No_Light FC (No POD License) Disabled
先再举个例子
fcmsutil /dev/fclp2
Vendor ID is = 0x10df
Device ID is = 0xfe00
PCI Sub-system Vendor ID is = 0x103c
PCI Sub-system ID is = 0x7046
Chip version = 2
Firmware Version = 2.70X5 SLI-3 (Z3D2.70X5)
EFI Version = ZE3.21A3
EFI Boot = ENABLED
Driver-Firmware Dump Available = NO
Driver-Firmware Dump Timestamp = N/A
Topology = PTTOPT_FABRIC
Link Speed = 4Gb
Local N_Port_id is = 0x789400
Previous N_Port_id is = None
N_Port Node World Wide Name = 0x5001438002a2b3df
N_Port Port World Wide Name = 0x5001438002a2b3de
Switch Port World Wide Name = 0x209400051e3627aa
Switch Node World Wide Name = 0x100000051e3627aa
Driver state = ONLINE
Hardware Path is = 1/0/12/0/0/0/0
Maximum Frame Size = 2048
Driver Version = @(#) FCLP: PCIe Fibre Channel driver (FibrChanl-02), B.11.31.0903, Dec 12 2008, FCLP_IFC (3,2)
Local N_Port_id is = 0x789400这里的0x表示是16进制，78表示这个hba卡连接的san switch的domain id，换算为16进制是120，表示连接的domain id是120的交换机，94表示端口148
#[/]ioscan -m hwpath
Lun H/W Path Lunpath H/W Path Legacy H/W Path
====================================================================
64000/0xfa00/0x18
0/0/12/0/0/0/0.0x50060e80047f3168.0x4001000000000000 0/0/12/0/0/0/0.120.220.0.0.0.1
0/0/14/0/0/0/0.0x50060e80047f3178.0x4001000000000000 0/0/14/0/0/0/0.110.220.0.0.0.1
1/0/12/0/0/0/0.0x50060e80047f3168.0x4001000000000000 1/0/12/0/0/0/0.120.220.0.0.0.1
1/0/14/0/0/0/0.0x50060e80047f3178.0x4001000000000000 1/0/14/0/0/0/0.110.220.0.0.0.1
我们可以看到这个disk，他有4条路径，分别通过4个hba，hba卡和存储端口是2对1，即2个hba卡对一个存储端口，
0/0/12/0/0/0/0表示hba卡的硬件地址，
0x50060e80047f3178表示存储端口的wwn号，hba卡1和3都对应存储同一个的端口，
0x4001000000000000中1表示这个lun的lunid是1，
0/0/12/0/0/0/0.120.220.0.0.0.1中120表示存储连接的san switch的domain id，这里可以看出主机和存储连接的同一个交换机，
如果是这个domain id和刚才fcmsutil所有的hba卡看到的domain id都不一样，表示可能存储和主机连接的是不同的交换机，
然后交换机之间级联来做san的通信。从这里可以看出，这个lun是从存储的2个端口出来，分别连接110和120 san switch的220端口
最后的1也是表示lunid，逢8进1