ERRPT报错信息:
shell> errpt
IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION
D666A8C7 0712172319 T H fcs2 ADAPTER ERROR
D666A8C7 0712172319 T H fcs2 ADAPTER ERROR
DC73C03A 0712172319 T S fscsi2 SOFTWARE PROGRAM ERROR
DC73C03A 0712172319 T S fscsi2 SOFTWARE PROGRAM ERROR
shell> errpt -aj D666A8C7 |more
---------------------------------------------------------------------------
LABEL: FCA_ERR2
IDENTIFIER: D666A8C7
Date/Time: Fri Jul 12 18:44:10 GMT+08:00 2019
Sequence Number: 2986137
Machine Id: 00F76B704C00
Node Id: sapprd2
Class: H
Type: TEMP
WPAR: Global
Resource Name: fcs2
Resource Class: adapter
Resource Type: df1000fe
Location: U78A0.001.DNWKLL8-P1-C2-T1
VPD:
Part Number.................10N7255
Serial Number...............1A20203641
Manufacturer................001A
EC Level....................D77040
Customer Card ID Number.....5774
FRU Number..................10N7255
Device Specific.(ZM)........3
Network Address.............10000000C9E89270
ROS Level and ID............02E8277F
Device Specific.(Z0)........2057706D
Device Specific.(Z1)........00000000
Device Specific.(Z2)........00000000
Device Specific.(Z3)........03000909
Device Specific.(Z4)........FFE01212
Device Specific.(Z5)........02E8277F
Device Specific.(Z6)........06E12715
Device Specific.(Z7)........07E1277F
Device Specific.(Z8)........20000000C9E89270
Device Specific.(Z9)........ZS2.71X15
Device Specific.(ZA)........Z1F2.70A5
Device Specific.(ZB)........Z2F2.71X15
Device Specific.(ZC)........00000000
Description
ADAPTER ERROR
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
SENSE DATA
0000 0011 0000 0027 0500 0000 0000 0000 0000 0000 0ABA 7C2C 0004 A2EC 0000 012C
...
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
Diagnostic Analysis
Diagnostic Log sequence number: 597748
Resource tested: fcs2
Menu Number: 2603902
Description:
Error Log Analysis has detected multiple communication
errors. These errors can be caused by attached devices,
a switch, a hub, or a SCSI-to-FC convertor.
If connected to a switch, refer to the Storage Area
Network (SAN) problem determination procedures for
additional problem resolution.
If not connected to a switch, run diagnostics on the
attached devices. If a hub or SCSI-to-FC convertor is
attached, refer to the product documentation for problem
resolution.
---------------------------------------------------------------------------
shell> errpt -aj DC73C03A |more
---------------------------------------------------------------------------
LABEL: FCP_ERR6
IDENTIFIER: DC73C03A
Date/Time: Fri Jul 12 18:45:00 GMT+08:00 2019
Sequence Number: 2986146
Machine Id: 00F76B704C00
Node Id: sapprd2
Class: S
Type: TEMP
WPAR: Global
Resource Name: fscsi2
Description
SOFTWARE PROGRAM ERROR
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
SENSE DATA
0000 0010 0000 00A1 0000 0005 0203 0000 0000 0000 0000 0000 0000 0000 0000 0000
...
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
这是报错涉及到的排查信息:
shell> lsattr -El fscsi2
attach switch How this adapter is CONNECTED False
dyntrk no Dynamic Tracking of FC Devices True
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True
scsi_id 0x10300 Adapter SCSI ID False
sw_fc_class 3 FC Class for Fabric True
shell> lsdev -Cc adapter
ent0 Available 02-08 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent1 Available 02-09 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent2 Available 03-08 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent3 Available 03-09 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent4 Available Logical Host Ethernet Port (lp-hea)
ent5 Available Logical Host Ethernet Port (lp-hea)
ent6 Available Logical Host Ethernet Port (lp-hea)
ent7 Available Logical Host Ethernet Port (lp-hea)
ent8 Available EtherChannel / IEEE 802.3ad Link Aggregation
ent9 Available EtherChannel / IEEE 802.3ad Link Aggregation
fcs0 Available 04-00 4Gb FC PCI Express Adapter (df1000fe)
fcs1 Available 04-01 4Gb FC PCI Express Adapter (df1000fe)
fcs2 Available 05-00 4Gb FC PCI Express Adapter (df1000fe)
fcs3 Available 05-01 4Gb FC PCI Express Adapter (df1000fe)
lhea0 Available Logical Host Ethernet Adapter (l-hea)
sa0 Available 08-00 4 Port Async EIA-232 PCIe Adapter
sissas0 Available 00-08 PCI-X266 Planar 3Gb SAS Adapter
usbhc0 Available 01-08 USB Host Controller (33103500)
usbhc1 Available 01-09 USB Host Controller (33103500)
usbhc2 Available 01-0a USB Enhanced Host Controller (3310e000)
vsa0 Available LPAR Virtual Serial Adapter
shell> lspath
Enabled hdisk0 sas0
Enabled hdisk1 sas0
Enabled hdisk2 sas0
Enabled hdisk3 sas0
Available ses0 sas0
Available ses1 sas0
Enabled hdisk4 fscsi0
Enabled hdisk5 fscsi0
Enabled hdisk6 fscsi0
Enabled hdisk7 fscsi0
Enabled hdisk8 fscsi0
Enabled hdisk9 fscsi0
Enabled hdisk10 fscsi0
Enabled hdisk11 fscsi0
Enabled hdisk12 fscsi0
Enabled hdisk13 fscsi0
Enabled hdisk14 fscsi0
Enabled hdisk15 fscsi0
Enabled hdisk16 fscsi0
Enabled hdisk17 fscsi0
Enabled hdisk18 fscsi0
Enabled hdisk19 fscsi0
Enabled hdisk20 fscsi0
Enabled hdisk21 fscsi0
Enabled hdisk22 fscsi0
Enabled hdisk23 fscsi0
Enabled hdisk24 fscsi0
Enabled hdisk25 fscsi0
Enabled hdisk26 fscsi0
Enabled hdisk27 fscsi0
Enabled hdisk28 fscsi0
Enabled hdisk29 fscsi0
Enabled hdisk30 fscsi0
Enabled hdisk31 fscsi0
Enabled dac0 fscsi0
Enabled hdisk4 fscsi0
Enabled hdisk5 fscsi0
Enabled hdisk6 fscsi0
Enabled hdisk7 fscsi0
Enabled hdisk8 fscsi0
Enabled hdisk9 fscsi0
Enabled hdisk10 fscsi0
Enabled hdisk11 fscsi0
Enabled hdisk12 fscsi0
Enabled hdisk13 fscsi0
Enabled hdisk14 fscsi0
Enabled hdisk15 fscsi0
Enabled hdisk16 fscsi0
Enabled hdisk17 fscsi0
Enabled hdisk18 fscsi0
Enabled hdisk19 fscsi0
Enabled hdisk20 fscsi0
Enabled hdisk21 fscsi0
Enabled hdisk22 fscsi0
Enabled hdisk23 fscsi0
Enabled hdisk24 fscsi0
Enabled hdisk25 fscsi0
Enabled hdisk26 fscsi0
Enabled hdisk27 fscsi0
Enabled hdisk28 fscsi0
Enabled hdisk29 fscsi0
Enabled hdisk30 fscsi0
Enabled hdisk31 fscsi0
Enabled dac1 fscsi0
Enabled hdisk4 fscsi2
Enabled hdisk5 fscsi2
Enabled hdisk6 fscsi2
Enabled hdisk7 fscsi2
Failed hdisk8 fscsi2
Failed hdisk9 fscsi2
Failed hdisk10 fscsi2
Failed hdisk11 fscsi2
Failed hdisk12 fscsi2
Failed hdisk13 fscsi2
Failed hdisk14 fscsi2
Failed hdisk15 fscsi2
Failed hdisk16 fscsi2
Failed hdisk17 fscsi2
Failed hdisk18 fscsi2
Failed hdisk19 fscsi2
Failed hdisk20 fscsi2
Failed hdisk21 fscsi2
Failed hdisk22 fscsi2
Failed hdisk23 fscsi2
Failed hdisk24 fscsi2
Failed hdisk25 fscsi2
Failed hdisk26 fscsi2
Failed hdisk27 fscsi2
Failed hdisk28 fscsi2
Failed hdisk29 fscsi2
Failed hdisk30 fscsi2
Failed hdisk31 fscsi2
Enabled dac0 fscsi2
Enabled hdisk4 fscsi2
Enabled hdisk5 fscsi2
Enabled hdisk6 fscsi2
Enabled hdisk7 fscsi2
Failed hdisk8 fscsi2
Failed hdisk9 fscsi2
Failed hdisk10 fscsi2
Failed hdisk11 fscsi2
Failed hdisk12 fscsi2
Failed hdisk13 fscsi2
Failed hdisk14 fscsi2
Failed hdisk15 fscsi2
Failed hdisk16 fscsi2
Failed hdisk17 fscsi2
Failed hdisk18 fscsi2
Failed hdisk19 fscsi2
Failed hdisk20 fscsi2
Failed hdisk21 fscsi2
Failed hdisk22 fscsi2
Failed hdisk23 fscsi2
Failed hdisk24 fscsi2
Failed hdisk25 fscsi2
Failed hdisk26 fscsi2
Failed hdisk27 fscsi2
Failed hdisk28 fscsi2
Failed hdisk29 fscsi2
Failed hdisk30 fscsi2
Failed hdisk31 fscsi2
Enabled dac1 fscsi2
MPIO中,fscsi2的链路几乎是Failed,光纤交换机也检查过端口,都是online状态,存储是DS4800,在映射的主机界面看不到对端主机的状态。
问下大家,根据以上提供的信息,是否能判断是硬件故障,谢谢
收起类似问题突然发生在稳定运行的系统中,建议排查光纤链路以及HBA,SFP卡等设备。如果是发生在新建环境或者调整过系统环境后的场景,建议排查软件版本的兼容性。例如AIX版本,HACMP版本,多路径软件版本等等,将软件版本更新到兼容版本即可解决问题。