ONTAP Discussions

REBOOT (panic)) WARNING - lost aggregates

jeffshelly
15,081 Views

Our DR target filer Panicked and rebooted and an aggregate is completely gone (not offline, gone)

The problem is it looks like 12 disks failed at once. Since this is an off-site filer, I don't have easy access to the box and I need to troubleshoot remotely.

 

The bottom line is I'm not sure if it's a FC interconnect, a shelf, or something else.

 

What I believe to be the relevant log entries are in red below.

Any help would be greatly appreciated

 

(Edit: Due to the large number of characters, this post has been edited to display the error. The entire log has been attached)

 

Thu Jul 24 09:00:03 EDT [callhome.performance.data:info]: Call home for PERFORMANCE DATA

Thu Jul 24 09:20:25 EDT [fci.device.login.reject:info]: The PLOGI transmitted by port 1c was rejected by port_id 0xcd with reason code 'Reserved', explanation code 'Service Parm Error - Rec Data Field Size'.

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.16: excessive victim abort: delta time 98920: retry CDB 0x2f:4959a800:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.18: excessive victim abort: delta time 98920: retry CDB 0x2f:4959a380:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.19: excessive victim abort: delta time 98920: retry CDB 0x2f:4959a380:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.20: excessive victim abort: delta time 98922: retry CDB 0x2f:4959a800:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.21: excessive victim abort: delta time 98923: retry CDB 0x2f:4959a800:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.22: excessive victim abort: delta time 98924: retry CDB 0x2f:4959a380:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.23: excessive victim abort: delta time 98925: retry CDB 0x2f:49599f00:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.24: excessive victim abort: delta time 98926: retry CDB 0x2f:4959a800:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.26: excessive victim abort: delta time 98926: retry CDB 0x2f:49599f00:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.27: excessive victim abort: delta time 98927: retry CDB 0x2f:4959a800:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.28: excessive victim abort: delta time 98927: retry CDB 0x2f:4959a380:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:03 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.29: excessive victim abort: delta time 98928: retry CDB 0x2f:4959a800:0480: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:05 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.15: excessive victim abort: delta time 98826: retry CDB 0x1c: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:05 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.17: excessive victim abort: delta time 98476: retry CDB 0x28:4b9e95ef:0009: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:05 EDT [scsi.cmd.excessiveVictim:warning]: device 1c.25: excessive victim abort: delta time 98478: retry CDB 0x28:4b9e95ef:0009: victim retry count 100: retry count 0: timeout retry count 0: path retry count 0: adapter status 0x15: target status 0x0: sense data SCSI:no sense -  (0x0 - 0x0 0x0 ).

Thu Jul 24 09:22:05 EDT [disk.reserveFailed:error]: Disk reservation failed on 1c.16 CDB 0x5e:01 - SCSI:no sense (0 0 0)

Thu Jul 24 09:22:05 EDT [disk.reserveFailed:error]: Disk reservation failed on 1c.18 CDB 0x5e:01 - SCSI:no sense (0 0 0)

Thu Jul 24 09:22:05 EDT [diskown.errorDuringIO:error]: error 25 (no valid path to disk) on disk 1c.16 (S/N WD-WMATV3672756) while setting disk reservation

Thu Jul 24 09:22:05 EDT [diskown.errorDuringIO:error]: error 25 (no valid path to disk) on disk 1c.18 (S/N WD-WMATV3734801) while setting disk reservation

Thu Jul 24 09:22:05 EDT [disk.reserveFailed:error]: Disk reservation failed on 1c.19 CDB 0x5e:01 - SCSI:no sense (0 0 0)

Thu Jul 24 09:22:05 EDT [disk.reserveFailed:error]: Disk reservation failed on 1c.20 CDB 0x5e:01 - SCSI:no sense (0 0 0)

Thu Jul 24 09:22:05 EDT [disk.readReservationFailed:error]: Disk read reservation failed on 1c.16 CDB 0x5e:01 - SCSI:no sense (0 0 0)

Thu Jul 24 09:22:05 EDT [disk.readReservationFailed:error]: Disk read reservation failed on 1c.18 CDB 0x5e:01 - SCSI:no sense (0 0 0)

Thu Jul 24 09:22:05 EDT [diskown.errorDuringIO:error]: error 25 (no valid path to disk) on disk 1c.19 (S/N WD-WMATV3660301) while setting disk reservation

 

 

1 ACCEPTED SOLUTION

aborzenkov
15,080 Views

Oh, that makes a difference. If port 1d was never connected in the first place, then it could be anything - adapter in slot 1, SFPs on port 1c (both of them) or AT-FCX module. In theory even defective disk could block loop. I’d start with power cycling shelf itself; obvious next step is to connect to another port.

Sorry, but the only way here is to troubleshoot on site by replacing components.

View solution in original post

9 REPLIES 9

vims
15,081 Views

Looks like a hardware failure. 12 disk -  sorry to hear that.

At least this is DR not PR filer.

1. sysconfig previous and now

2. what kind of shelf ?

jeffshelly
15,081 Views

These are DS14MK2 AT-FCX shelves

Current sysconfig - I do not have one prior to panic reboot...

sysconfig -a

NetApp Release 8.0.5 7-Mode: Sun Jan 20 21:53:54 PST 2013

System ID: 0151731760 (PCTF090002)

System Serial Number: 200000054435 (PCTF090002)

System Rev: B0          

System Storage Configuration: Single Path

System ACP Connectivity: NA

slot 0: System Board 2.4 GHz (System Board XV A5)

                Model Name:         FAS3140

                Part Number:        110-00120

                Revision:           A5

                Serial Number:      736816

                BIOS version:       4.4.0

                Loader version:     1.8

                Agent FW version:   3

                Processors:         2

                Processor ID:       0x40f13

                Microcode Version:  0x0

                Processor type:     Opteron

                Memory Size:        4096 MB

                Memory Attributes:  Bank Interleaving

                                    Hoisting

                                    Chipkill ECC

                CMOS RAM Status:    OK

                Controller:         A

Remote LAN Module           Status: Online

Part Number:        110-00057

Revision:           G0

Serial Number:      918584

Firmware Version:   4.1

Mgmt MAC Address:   00:A0:98:13:A9:CA

Ethernet Link:      up

Using DHCP:         no

IPv4 configuration:

IP Address:         10.251.90.77

Netmask:            255.255.0.0

Gateway:            10.251.0.1

slot 0: Internal BGE 10/100 Ethernet Controller

e0M MAC Address:    00:a0:98:12:dd:e6 (auto-100tx-fd-up)

                Device Type:        BCM5703C_B0

                        memory mapped I/O base 0xfdd00000, size 0x10000

slot 0: BGE 10/100/1000 Ethernet Controller

e0a MAC Address:    00:a0:98:12:dd:e4 (auto-1000t-fd-up)

e0b MAC Address:    00:a0:98:12:dd:e5 (auto-1000t-fd-up)

                Device Type:        BCM5715C

                        memory mapped I/O base 0xfe310000, size 0x10000

                        memory mapped I/O base 0xfe300000, size 0x10000

                        memory mapped I/O base 0xfe330000, size 0x10000

                        memory mapped I/O base 0xfe320000, size 0x10000

        slot 0: ATA/IDE Adapter 0e (0x000001f0)

                0e.0   STEC NACF1GM1U-B11   01/17/07  977MB  512B/sect (STI1MMV209225084409)

                        I/O base 0x000001f0, size 0x8

                        I/O base 0x000003f4, size 0x4

                        I/O base 0000000000, size 0x8

                        I/O base 0000000000, size 0x4

                        I/O base 0x00001c00, size 0x10

slot 0: NVRAM (NetApp NVRAM VII)

                Revision:           A5

                Serial Number:      736816

                Memory Size:        512 MB

                DIMM Size:          512 MB

                Battery1 Status:    Battery sufficiently charged (3816 mV)

                Charger1 Status:    OFF

                Running Firmware:   4 (4.8.940)

                Cluster Interconnect Port 1: 4x copper

                Cluster Interconnect Port 2: 4x copper

slot 0: FC Host Adapter 0a (QLogic 2432 rev. 2, L-port, <OFFLINE (hard)>)

Firmware rev:      5.4.100

Host Loop Id:      0

FC Node Name:      5:00a:098200:037584

FC Port Name:      5:00a:098200:037584

SFF Vendor:       

SFF Part Number:  

SFF Serial Number:

SFF Capabilities: 

Link Data Rate:    N/A

                        I/O base 0x0000000000007000, size 0x100

                        memory mapped I/O base 0xfe700000, size 0x4000

        slot 0: Fibre Channel Target Host Adapter 0b

                (Dual-channel, QLogic 2432 (2462) rev. 2, 64-bit, <OFFLINED BY USER/SYSTEM>)

                Firmware rev:     5.5.2

                Host Port Addr:   000000

                Cacheline size:   16 

                SRAM parity:      Yes

                FC Nodename:      50:0a:09:80:83:b1:5b:17 (500a098083b15b17)

                FC Portname:      50:0a:09:84:83:b1:5b:17 (500a098483b15b17)

                Connection:       No link

                        I/O base 0x0000000000007400, size 0x100

                        memory mapped I/O base 0xfe704000, size 0x4000

slot 0: FC Host Adapter 0c (QLogic 2432 rev. 2, L-port, <OFFLINE (hard)>)

Firmware rev:      5.4.100

Host Loop Id:      0

FC Node Name:      5:00a:098000:037584

FC Port Name:      5:00a:098000:037584

SFF Vendor:       

SFF Part Number:  

SFF Serial Number:

SFF Capabilities: 

Link Data Rate:    N/A

                        I/O base 0x0000000000006000, size 0x100

                        memory mapped I/O base 0xfe600000, size 0x4000

        slot 0: Fibre Channel Target Host Adapter 0d

                (Dual-channel, QLogic 2432 (2462) rev. 2, 64-bit, <OFFLINED BY USER/SYSTEM>)

                Firmware rev:     5.5.2

                Host Port Addr:   000000

                Cacheline size:   16 

                SRAM parity:      Yes

                FC Nodename:      50:0a:09:80:83:b1:5b:17 (500a098083b15b17)

                FC Portname:      50:0a:09:83:83:b1:5b:17 (500a098383b15b17)

                Connection:       No link

                        I/O base 0x0000000000006400, size 0x100

                        memory mapped I/O base 0xfe604000, size 0x4000

slot 1: FC Host Adapter 1a (QLogic 2432 rev. 3, L-port, <UP>)

Board name:        QLE2464

Serial Number:     LFC0929J70082

Firmware rev:      5.4.100

Host Loop Id:      7

FC Node Name:      2:000:001b32:8e62cd

FC Port Name:      2:100:001b32:8e62cd

SFF Vendor:        FINISAR CORP.

SFF Part Number:   FTLF8524E2KNL

SFF Serial Number: PFA5QH1

SFF Capabilities:  1, 2 or 4 Gbit

Link Data Rate:    2 Gbit

                 16  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MVPDE)

                 17  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NVS7E)

                 18  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82M9CFE)

                 19  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NVSDE)

                 20  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NMTHE)

                 21  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NML7E)

                 22  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MDRCE)

                 23  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82ME3ZE)

                 24  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82M9F6E)

                 25  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82LWYWE)

                 26  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MF0NE)

                 27  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NVFHE)

                 28  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82LN1BE)

                 29  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82N83NE)

                 32  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82T74QE)

                 33  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82T7D7E)

                 34  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82T6RVE)

                 35  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SD4BE)

                 36  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SDQME)

                 37  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCJ4E)

                 38  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SLKZE)

                 39  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82T5G0E)

                 40  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SM09E)

                 41  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SM4FE)

                 42  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SHMVE)

                 43  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SACCE)

                 44  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RMY3E)

                 45  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SVNQE)

                 48  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0GVF)

                 49  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJP74F)

                 50  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0HKF)

                 51  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0J5F)

                 52  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJK3H3F)

                 53  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJB3PF)

                 54  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ747F)

                 55  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0EUF)

                 56  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0ETF)

                 57  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0G7F)

                 58  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJP53F)

                 59  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0HNF)

                 60  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0G0F)

                 61  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJP4TF)

                 64  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFC1)

                 65  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFKK)

                 66  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1DKZT)

                 67  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFTF)

                 68  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFMS)

                 69  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFFA)

                 70  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EE5Z)

                 71  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EEZK)

                 72  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N02ZZWKL)

                 73  : NETAPP   X267_SGLXY500SSX AQMZ 423.1GB 512B/sect (9QG8L2KV)

                 74  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFZA)

                 75  : NETAPP   X267_SGLXY500SSX AQNZ 423.1GB 512B/sect (9QG9SS9R)

                 76  : NETAPP   X269_HSATE01TSSX A3CA 847.5GB 512B/sect (MQ0EEL5F)

                 77  : NETAPP   X267_HSATET50SSX A3CA 423.1GB 512B/sect (ME1T7R6F)

                 80  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3MXJN)

                 81  : NETAPP   X268_WMARST75SSX NA00 635.5GB 512B/sect (WD-WMATW0167796)

                 82  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3JPVQ)

                 83  : NETAPP   X269_SMUSK01TSSX NA00 847.5GB 512B/sect (9WK5GWD4)

                 84  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N00S7MPL)

                 85  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3MX5M)

                 86  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3L2X2)

                 87  : NETAPP   X269_HGEMI01TSSX A90A 847.5GB 512B/sect (PBH2R15E)

                 88  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (J81S844L)

                 89  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (J81T36ML)

                 90  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N01JVTTL)

                 91  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N036T66L)

                 92  : NETAPP   X268_WMARST75SSX NA00 635.5GB 512B/sect (WD-WMAU00169763)

                 93  : NETAPP   X269_WMARS01TSSX NA01 847.5GB 512B/sect (WD-WMATV8033696)

Shelf 1: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: --

Shelf 2: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: --

Shelf 3: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

Shelf 4: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

Shelf 5: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

                        I/O base 0x0000000000002000, size 0x100

                        memory mapped I/O base 0xfdf00000, size 0x4000

slot 1: FC Host Adapter 1b (QLogic 2432 rev. 3, L-port, <UP>)

Board name:        QLE2464

Serial Number:     LFC0929J70082

Firmware rev:      5.4.100

Host Loop Id:      7

FC Node Name:      2:001:001b32:ae62cd

FC Port Name:      2:101:001b32:ae62cd

SFF Vendor:        FINISAR CORP.

SFF Part Number:   FTLF8524E2KNL

SFF Serial Number: PFB0AP2

SFF Capabilities:  1, 2 or 4 Gbit

Link Data Rate:    2 Gbit

                 16  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MWS2E)

                 17  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MAEFE)

                 18  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82N9L1E)

                 19  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NS27E)

                 20  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82LZ9JE)

                 21  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NR3WE)

                 22  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NRJ9E)

                 23  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NS29E)

                 24  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82N86RE)

                 25  : NETAPP   X269_WMARS01TSSX NA01 847.5GB 512B/sect (WD-WMATV8389970)

                 26  : XYRATEX  RS-1402-SA-XNSSX        0.0GB 0B/sect (Failed)

                 27  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82ME7AE)

                 28  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NTBGE)

                 29  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MXBGE)

                 32  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82PNLRE)

                 33  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SVY3E)

                 34  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SG66E)

                 35  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SPWHE)

                 36  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SXYEE)

                 37  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCHJE)

                 38  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RBJ6E)

                 39  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCFEE)

                 40  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCH3E)

                 41  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82STGWE)

                 42  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCC5E)

                 43  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SFHKE)

                 44  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCDSE)

                 45  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RC5CE)

                 48  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9NSF)

                 49  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9MNF)

                 50  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJLS3PF)

                 51  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJLSR4F)

                 52  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9NHF)

                 53  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9PVF)

                 54  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9P7F)

                 55  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9NEF)

                 56  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJLSRZF)

                 57  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9PMF)

                 58  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJLSBXF)

                 59  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJKMGAF)

                 60  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9S7F)

                 61  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9NWF)

                 64  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1E6WV)

                 65  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (5QG1KJKV)

                 66  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFS2)

                 67  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1A26Y)

                 68  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFFJ)

                 69  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1E9KD)

                 70  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1DKS1)

                 71  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N02A82HL)

                 72  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG19ZT2)

                 73  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG19ZZW)

                 74  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZBG7N0SH)

                 75  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1E5W0)

                 76  : NETAPP   X267_HSATET50SSX A3CA 423.1GB 512B/sect (ME1THP6F)

                 77  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFNC)

                 80  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3MF5T)

                 81  : NETAPP   X269_SMUSK01TSSX NA00 847.5GB 512B/sect (9WK513HP)

                 82  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3L1EH)

                 83  : NETAPP   X269_SMUSK01TSSX NA00 847.5GB 512B/sect (9WK5GTVC)

                 84  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD2YBC0)

                 85  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3HLKX)

                 86  : NETAPP   X268_WMARST75SSX NA00 635.5GB 512B/sect (WD-WMAU00191004)

                 87  : NETAPP   X268_WMARST75SSX NA00 635.5GB 512B/sect (WD-WMATW0168340)

                 88  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N03D01UL)

                 89  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N02ZV6SL)

                 90  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3LJ6V)

                 91  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3LBB8)

                 92  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3EMMS)

                 93  : NETAPP   X269_SMUSK01TSSX NA00 847.5GB 512B/sect (9WK51S3K)

Shelf 1: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: --

Shelf 2: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: --

Shelf 3: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

Shelf 4: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

Shelf 5: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

                        I/O base 0x0000000000002400, size 0x100

                        memory mapped I/O base 0xfdf04000, size 0x4000

slot 1: FC Host Adapter 1c (QLogic 2432 rev. 3, L-port, <UP>)

Board name:        QLE2464

Serial Number:     LFC0929J70082

Firmware rev:      5.4.100

Host Loop Id:      7

FC Node Name:      2:002:001b32:ce62cd

FC Port Name:      2:102:001b32:ce62cd

SFF Vendor:        FINISAR CORP.

SFF Part Number:   FTLF8524E2KNL

SFF Serial Number: PFB09PB

SFF Capabilities:  1, 2 or 4 Gbit

Link Data Rate:    2 Gbit

                        I/O base 0x0000000000003000, size 0x100

                        memory mapped I/O base 0xfe000000, size 0x4000

slot 1: FC Host Adapter 1d (QLogic 2432 rev. 3, L-port, <OFFLINE (hard)>)

Board name:        QLE2464

Serial Number:     LFC0929J70082

Firmware rev:      5.4.100

Host Loop Id:      0

FC Node Name:      2:003:001b32:ee62cd

FC Port Name:      2:103:001b32:ee62cd

SFF Vendor:        FINISAR CORP.

SFF Part Number:   FTLF8524E2KNL

SFF Serial Number: PFA6E3X

SFF Capabilities:  1, 2 or 4 Gbit

Link Data Rate:    N/A

                        I/O base 0x0000000000003400, size 0x100

                        memory mapped I/O base 0xfe004000, size 0x4000

slot 4: Quad 10/100/1000 Ethernet Controller G20

e4a MAC Address:    00:15:17:c6:5e:54 (auto-unknown-down)

e4b MAC Address:    00:15:17:c6:5e:55 (auto-unknown-down)

e4c MAC Address:    00:15:17:c6:5e:56 (auto-unknown-down)

e4d MAC Address:    00:15:17:c6:5e:57 (auto-unknown-down)

Device Type:        Rev 6, PBA D45774-008

                        memory mapped I/O base 0x00000000fe420000, size 0x20000

                        memory mapped I/O base 0x00000000fe400000, size 0x20000

                        I/O base 0x0000000000004000, size 0x20

                        memory mapped I/O base 0x00000000fe460000, size 0x20000

                        memory mapped I/O base 0x00000000fe440000, size 0x20000

                        I/O base 0x0000000000004020, size 0x20

                        memory mapped I/O base 0x00000000fe520000, size 0x20000

                        memory mapped I/O base 0x00000000fe500000, size 0x20000

                        I/O base 0x0000000000005000, size 0x20

                        memory mapped I/O base 0x00000000fe560000, size 0x20000

                        memory mapped I/O base 0x00000000fe540000, size 0x20000

                        I/O base 0x0000000000005020, size 0x20

aborzenkov
15,081 Views

You can pull previous configuration from ASUP. Being able to compare would help.

Based on what you have shown so far, you have one genuinely failed disk (1b.26) as well as HBA in slot 1 - looks like second ASIC that connects to ports c and d failed. But I do not quite understand how it helps you - you would need to open case with NetApp to get replacement anyway, and it is their job to troubleshoot and determine failed part.

You have enough free ports to connect shelf using multipath. This would help to mitigate this issue (not panic, but shelves would have been found after reboot).

jeffshelly
15,080 Views

I have an older Autosupport but I was unable to find sysconfig in the Autosupport bundle that was sent (I only found sysconfig_m and sysconfig_t). Fortunately I had a previous sysconfig-A that I had pulled about a week ago.

This is a legacy system that needs to run until the end of the year until we transition to our new 8040s - It's under third party hardware support, but they essentially replace failed hardware and not much else, which is why I need to figure out what went bad.

Thanks again,

PCTF090002> sysconfig -a

NetApp Release 8.0.5 7-Mode: Sun Jan 20 21:53:54 PST 2013

System ID: 0151731760 (PCTF090002)

System Serial Number: 200000054435 (PCTF090002)

System Rev: B0          

System Storage Configuration: Single Path

System ACP Connectivity: NA

slot 0: System Board 2.4 GHz (System Board XV A5)

                Model Name:         FAS3140

                Part Number:        110-00120

                Revision:           A5

                Serial Number:      736816

                BIOS version:       4.4.0

                Loader version:     1.8

                Agent FW version:   3

                Processors:         2

                Processor ID:       0x40f13

                Microcode Version:  0x0

                Processor type:     Opteron

                Memory Size:        4096 MB

                Memory Attributes:  Bank Interleaving

                                    Hoisting

                                    Chipkill ECC

                CMOS RAM Status:    OK

                Controller:         A

Remote LAN Module           Status: Online

Part Number:        110-00057

Revision:           G0

Serial Number:      918584

Firmware Version:   4.1

Mgmt MAC Address:   00:A0:98:13:A9:CA

Ethernet Link:      up

Using DHCP:         no

IPv4 configuration:

IP Address:         10.251.90.77

Netmask:            255.255.0.0

Gateway:            10.251.0.1

slot 0: Internal BGE 10/100 Ethernet Controller

e0M MAC Address:    00:a0:98:12:dd:e6 (auto-100tx-fd-up)

                Device Type:        BCM5703C_B0

                        memory mapped I/O base 0xfdd00000, size 0x10000

slot 0: BGE 10/100/1000 Ethernet Controller

e0a MAC Address:    00:a0:98:12:dd:e4 (auto-1000t-fd-up)

e0b MAC Address:    00:a0:98:12:dd:e5 (auto-1000t-fd-up)

                Device Type:        BCM5715C

                        memory mapped I/O base 0xfe310000, size 0x10000

                        memory mapped I/O base 0xfe300000, size 0x10000

                        memory mapped I/O base 0xfe330000, size 0x10000

                        memory mapped I/O base 0xfe320000, size 0x10000

        slot 0: ATA/IDE Adapter 0e (0x000001f0)

                0e.0   STEC NACF1GM1U-B11   01/17/07  977MB  512B/sect (STI1MMV209225084409)

                        I/O base 0x000001f0, size 0x8

                        I/O base 0x000003f4, size 0x4

                        I/O base 0000000000, size 0x8

                        I/O base 0000000000, size 0x4

                        I/O base 0x00001c00, size 0x10

slot 0: NVRAM (NetApp NVRAM VII)

                Revision:           A5

                Serial Number:      736816

                Memory Size:        512 MB

                DIMM Size:          512 MB

                Battery1 Status:    Battery sufficiently charged (3870 mV)

                Charger1 Status:    OFF

                Running Firmware:   4 (4.8.940)

                Cluster Interconnect Port 1: 4x copper

                Cluster Interconnect Port 2: 4x copper

slot 0: FC Host Adapter 0a (QLogic 2432 rev. 2, L-port, <OFFLINE (hard)>)

Firmware rev:      5.4.100

Host Loop Id:      0

FC Node Name:      5:00a:098200:037584

FC Port Name:      5:00a:098200:037584

SFF Vendor:       

SFF Part Number:  

SFF Serial Number:

SFF Capabilities: 

Link Data Rate:    N/A

                        I/O base 0x0000000000007000, size 0x100

                        memory mapped I/O base 0xfe700000, size 0x4000

        slot 0: Fibre Channel Target Host Adapter 0b

                (Dual-channel, QLogic 2432 (2462) rev. 2, 64-bit, <OFFLINED BY USER/SYSTEM>)

                Firmware rev:     5.5.2

                Host Port Addr:   000000

                Cacheline size:   16 

                SRAM parity:      Yes

                FC Nodename:      50:0a:09:80:83:b1:5b:17 (500a098083b15b17)

                FC Portname:      50:0a:09:84:83:b1:5b:17 (500a098483b15b17)

                Connection:       No link

                        I/O base 0x0000000000007400, size 0x100

                        memory mapped I/O base 0xfe704000, size 0x4000

slot 0: FC Host Adapter 0c (QLogic 2432 rev. 2, L-port, <OFFLINE (hard)>)

Firmware rev:      5.4.100

Host Loop Id:      0

FC Node Name:      5:00a:098000:037584

FC Port Name:      5:00a:098000:037584

SFF Vendor:       

SFF Part Number:  

SFF Serial Number:

SFF Capabilities: 

Link Data Rate:    N/A

                        I/O base 0x0000000000006000, size 0x100

                        memory mapped I/O base 0xfe600000, size 0x4000

        slot 0: Fibre Channel Target Host Adapter 0d

                (Dual-channel, QLogic 2432 (2462) rev. 2, 64-bit, <OFFLINED BY USER/SYSTEM>)

                Firmware rev:     5.5.2

                Host Port Addr:   000000

                Cacheline size:   16 

                SRAM parity:      Yes

                FC Nodename:      50:0a:09:80:83:b1:5b:17 (500a098083b15b17)

                FC Portname:      50:0a:09:83:83:b1:5b:17 (500a098383b15b17)

                Connection:       No link

                        I/O base 0x0000000000006400, size 0x100

                        memory mapped I/O base 0xfe604000, size 0x4000

slot 1: FC Host Adapter 1a (QLogic 2432 rev. 3, L-port, <UP>)

Board name:        QLE2464

Serial Number:     LFC0929J70082

Firmware rev:      5.4.100

Host Loop Id:      7

FC Node Name:      2:000:001b32:8e62cd

FC Port Name:      2:100:001b32:8e62cd

SFF Vendor:        FINISAR CORP.

SFF Part Number:   FTLF8524E2KNL

SFF Serial Number: PFA5QH1

SFF Capabilities:  1, 2 or 4 Gbit

Link Data Rate:    2 Gbit

                 16  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MVPDE)

                 17  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NVS7E)

                 18  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82M9CFE)

                 19  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NVSDE)

                 20  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NMTHE)

                 21  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NML7E)

                 22  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MDRCE)

                 23  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82ME3ZE)

                 24  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82M9F6E)

                 25  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82LWYWE)

                 26  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MF0NE)

                 27  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NVFHE)

                 28  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82LN1BE)

                 29  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82N83NE)

                 32  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82T74QE)

                 33  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82T7D7E)

                 34  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82T6RVE)

                 35  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SD4BE)

                 36  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SDQME)

                 37  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCJ4E)

                 38  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SLKZE)

                 39  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82T5G0E)

                 40  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SM09E)

                 41  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SM4FE)

                 42  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SHMVE)

                 43  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SACCE)

                 44  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RMY3E)

                 45  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SVNQE)

                 48  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0GVF)

                 49  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJP74F)

                 50  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0HKF)

                 51  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0J5F)

                 52  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJK3H3F)

                 53  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJB3PF)

                 54  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ747F)

                 55  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0EUF)

                 56  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0ETF)

                 57  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0G7F)

                 58  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJP53F)

                 59  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0HNF)

                 60  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJ0G0F)

                 61  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJJP4TF)

                 64  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFC1)

                 65  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFKK)

                 66  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1DKZT)

                 67  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFTF)

                 68  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFMS)

                 69  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFFA)

                 70  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EE5Z)

                 71  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EEZK)

                 72  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N02ZZWKL)

                 73  : NETAPP   X267_SGLXY500SSX AQMZ 423.1GB 512B/sect (9QG8L2KV)

                 74  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFZA)

                 75  : NETAPP   X267_SGLXY500SSX AQNZ 423.1GB 512B/sect (9QG9SS9R)

                 76  : NETAPP   X269_HSATE01TSSX A3CA 847.5GB 512B/sect (MQ0EEL5F)

                 77  : NETAPP   X267_HSATET50SSX A3CA 423.1GB 512B/sect (ME1T7R6F)

                 80  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3MXJN)

                 81  : NETAPP   X268_WMARST75SSX NA00 635.5GB 512B/sect (WD-WMATW0167796)

                 82  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3JPVQ)

                 83  : NETAPP   X269_SMUSK01TSSX NA00 847.5GB 512B/sect (9WK5GWD4)

                 84  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N00S7MPL)

                 85  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3MX5M)

                 86  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3L2X2)

                 87  : NETAPP   X269_HGEMI01TSSX A90A 847.5GB 512B/sect (PBH2R15E)

                 88  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (J81S844L)

                 89  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (J81T36ML)

                 90  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N01JVTTL)

                 91  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N036T66L)

                 92  : NETAPP   X268_WMARST75SSX NA00 635.5GB 512B/sect (WD-WMAU00169763)

                 93  : NETAPP   X269_WMARS01TSSX NA01 847.5GB 512B/sect (WD-WMATV8033696)

Shelf 1: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: --

Shelf 2: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: --

Shelf 3: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

Shelf 4: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

Shelf 5: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

                        I/O base 0x0000000000002000, size 0x100

                        memory mapped I/O base 0xfdf00000, size 0x4000

slot 1: FC Host Adapter 1b (QLogic 2432 rev. 3, L-port, <UP>)

Board name:        QLE2464

Serial Number:     LFC0929J70082

Firmware rev:      5.4.100

Host Loop Id:      7

FC Node Name:      2:001:001b32:ae62cd

FC Port Name:      2:101:001b32:ae62cd

SFF Vendor:        FINISAR CORP.

SFF Part Number:   FTLF8524E2KNL

SFF Serial Number: PFB0AP2

SFF Capabilities:  1, 2 or 4 Gbit

Link Data Rate:    2 Gbit

                 16  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MWS2E)

                 17  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MAEFE)

                 18  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82N9L1E)

                 19  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NS27E)

                 20  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82LZ9JE)

                 21  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NR3WE)

                 22  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NRJ9E)

                 23  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NS29E)

                 24  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82N86RE)

                 25  : NETAPP   X269_WMARS01TSSX NA01 847.5GB 512B/sect (WD-WMATV8389970)

                 26  : XYRATEX  RS-1402-SA-XNSSX      847.5GB 512B/sect (Failed)

                 27  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82ME7AE)

                 28  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82NTBGE)

                 29  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82MXBGE)

                 32  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82PNLRE)

                 33  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SVY3E)

                 34  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SG66E)

                 35  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SPWHE)

                 36  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SXYEE)

                 37  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCHJE)

                 38  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RBJ6E)

                 39  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCFEE)

                 40  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCH3E)

                 41  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82STGWE)

                 42  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCC5E)

                 43  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82SFHKE)

                 44  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RCDSE)

                 45  : NETAPP   X266_MTOMC320PSX R5VV 274.4GB 512B/sect (A82RC5CE)

                 48  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9NSF)

                 49  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9MNF)

                 50  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJLS3PF)

                 51  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJLSR4F)

                 52  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9NHF)

                 53  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9PVF)

                 54  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9P7F)

                 55  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9NEF)

                 56  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJLSRZF)

                 57  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9PMF)

                 58  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJLSBXF)

                 59  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJKMGAF)

                 60  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9S7F)

                 61  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZAJL9NWF)

                 64  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1E6WV)

                 65  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (5QG1KJKV)

                 66  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFS2)

                 67  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1A26Y)

                 68  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFFJ)

                 69  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1E9KD)

                 70  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1DKS1)

                 71  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N02A82HL)

                 72  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG19ZT2)

                 73  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG19ZZW)

                 74  : NETAPP   X267_HKURO500SSX AB0A 423.1GB 512B/sect (ZBG7N0SH)

                 75  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1E5W0)

                 76  : NETAPP   X267_HSATET50SSX A3CA 423.1GB 512B/sect (ME1THP6F)

                 77  : NETAPP   X267_SGLXY500SSX AQJZ 423.1GB 512B/sect (9QG1EFNC)

                 80  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3MF5T)

                 81  : NETAPP   X269_SMUSK01TSSX NA00 847.5GB 512B/sect (9WK513HP)

                 82  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3L1EH)

                 83  : NETAPP   X269_SMUSK01TSSX NA00 847.5GB 512B/sect (9WK5GTVC)

                 84  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD2YBC0)

                 85  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3HLKX)

                 86  : NETAPP   X268_WMARST75SSX NA00 635.5GB 512B/sect (WD-WMAU00191004)

                 87  : NETAPP   X268_WMARST75SSX NA00 635.5GB 512B/sect (WD-WMATW0168340)

                 88  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N03D01UL)

                 89  : NETAPP   X269_HJUPI01TSSX NA01 847.5GB 512B/sect (N02ZV6SL)

                 90  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3LJ6V)

                 91  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3LBB8)

                 92  : NETAPP   X268_SGLXY750SSX AQRZ 635.5GB 512B/sect (5QD3EMMS)

                 93  : NETAPP   X269_SMUSK01TSSX NA00 847.5GB 512B/sect (9WK51S3K)

Shelf 1: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: --

Shelf 2: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: --

Shelf 3: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

Shelf 4: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

Shelf 5: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

                        I/O base 0x0000000000002400, size 0x100

                        memory mapped I/O base 0xfdf04000, size 0x4000

slot 1: FC Host Adapter 1c (QLogic 2432 rev. 3, L-port, <UP>)

Board name:        QLE2464

Serial Number:     LFC0929J70082

Firmware rev:      5.4.100

Host Loop Id:      7

FC Node Name:      2:002:001b32:ce62cd

FC Port Name:      2:102:001b32:ce62cd

SFF Vendor:        FINISAR CORP.

SFF Part Number:   FTLF8524E2KNL

SFF Serial Number: PFB09PB

SFF Capabilities:  1, 2 or 4 Gbit

Link Data Rate:    2 Gbit

                 16  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3672756)

                 17  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3673202)

                 18  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3734801)

                 19  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3660301)

                 20  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3793760)

                 21  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3673732)

                 22  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3690944)

                 23  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3673385)

                 24  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3671605)

                 25  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3792610)

                 26  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3662035)

                 27  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3803338)

                 28  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3663073)

                 29  : NETAPP   X269_WMARS01TSSX NA00 847.5GB 512B/sect (WD-WMATV3793495)

Shelf 1: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38

                        I/O base 0x0000000000003000, size 0x100

                        memory mapped I/O base 0xfe000000, size 0x4000

slot 1: FC Host Adapter 1d (QLogic 2432 rev. 3, L-port, <OFFLINE (hard)>)

Board name:        QLE2464

Serial Number:     LFC0929J70082

Firmware rev:      5.4.100

Host Loop Id:      0

FC Node Name:      2:003:001b32:ee62cd

FC Port Name:      2:103:001b32:ee62cd

SFF Vendor:        FINISAR CORP.

SFF Part Number:   FTLF8524E2KNL

SFF Serial Number: PFA6E3X

SFF Capabilities:  1, 2 or 4 Gbit

Link Data Rate:    N/A

                        I/O base 0x0000000000003400, size 0x100

                        memory mapped I/O base 0xfe004000, size 0x4000

slot 4: Quad 10/100/1000 Ethernet Controller G20

e4a MAC Address:    00:15:17:c6:5e:54 (auto-unknown-down)

e4b MAC Address:    00:15:17:c6:5e:55 (auto-unknown-down)

e4c MAC Address:    00:15:17:c6:5e:56 (auto-unknown-down)

e4d MAC Address:    00:15:17:c6:5e:57 (auto-unknown-down)

Device Type:        Rev 6, PBA D45774-008

                        memory mapped I/O base 0x00000000fe420000, size 0x20000

                        memory mapped I/O base 0x00000000fe400000, size 0x20000

                        I/O base 0x0000000000004000, size 0x20

                        memory mapped I/O base 0x00000000fe460000, size 0x20000

                        memory mapped I/O base 0x00000000fe440000, size 0x20000

                        I/O base 0x0000000000004020, size 0x20

                        memory mapped I/O base 0x00000000fe520000, size 0x20000

                        memory mapped I/O base 0x00000000fe500000, size 0x20000

                        I/O base 0x0000000000005000, size 0x20

                        memory mapped I/O base 0x00000000fe560000, size 0x20000

                        memory mapped I/O base 0x00000000fe540000, size 0x20000

                        I/O base 0x0000000000005020, size 0x20

PCTF090002> sysconfig -a   r

Aggregate aggr2 (online, raid_dp) (block checksums)

  Plex /aggr2/plex0 (online, normal, active)

    RAID group /aggr2/plex0/rg0 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      dparity 1a.481a    3   0   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      parity  1b.481b    3   0   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.491a    3   1   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.491b    3   1   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.501a    3   2   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.501b    3   2   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.511a    3   3   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.511b    3   3   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.521a    3   4   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.521b    3   4   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.531a    3   5   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.531b    3   5   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.541a    3   6   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.541b    3   6   FC:A   -  ATA   7200 423111/866531584  423889/868126304

    RAID group /aggr2/plex0/rg1 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      dparity 1b.551b    3   7   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      parity  1a.551a    3   7   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.561b    3   8   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.561a    3   8   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.571b    3   9   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.571a    3   9   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.581b    3   10  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.581a    3   10  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.591b    3   11  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.591a    3   11  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.601b    3   12  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.601a    3   12  FC:A   -  ATA   7200 423111/866531584  423889/868126304

Aggregate aggr4 (online, raid_dp) (block checksums)

  Plex /aggr4/plex0 (online, normal, active)

    RAID group /aggr4/plex0/rg0 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      dparity 1a.801a    5   0   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      parity  1b.801b    5   0   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1a.891a    5   9   FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1b.871b    5   7   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1a.821a    5   2   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1b.821b    5   2   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1b.811b    5   1   FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1b.931b    5   13  FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1a.881a    5   8   FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1a.901a    5   10  FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1a.851a    5   5   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1b.851b    5   5   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1a.861a    5   6   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1b.841b    5   4   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

    RAID group /aggr4/plex0/rg1 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      dparity 1b.251b    1   9   FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      parity  1a.931a    5   13  FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1a.921a    5   12  FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1a.831a    5   3   FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1a.761a    4   12  FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1b.831b    5   3   FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1b.901b    5   10  FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1b.861b    5   6   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1b.911b    5   11  FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1a.841a    5   4   FC:A   -  ATA   7200 635555/1301618176 847827/1736350304

      data    1b.921b    5   12  FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

      data    1a.811a    5   1   FC:A   -  ATA   7200 635555/1301618176 635858/1302238304

Aggregate aggr5 (online, raid_dp) (block checksums)

  Plex /aggr5/plex0 (online, normal, active)

    RAID group /aggr5/plex0/rg0 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      dparity 1c.171c    1   1   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      parity  1c.251c    1   9   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.161c    1   0   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.241c    1   8   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.231c    1   7   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.221c    1   6   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.211c    1   5   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.201c    1   4   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.291c    1   13  FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.281c    1   12  FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.271c    1   11  FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.191c    1   3   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.261c    1   10  FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1c.181c    1   2   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

Aggregate aggr3 (online, raid_dp) (block checksums)

  Plex /aggr3/plex0 (online, normal, active)

    RAID group /aggr3/plex0/rg0 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      dparity 1a.691a    4   5   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      parity  1b.711b    4   7   FC:A   -  ATA   7200 423111/866531584  847827/1736350304

      data    1b.721b    4   8   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.771b    4   13  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.611b    3   13  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.701b    4   6   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.651b    4   1   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.691b    4   5   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.681a    4   4   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.671a    4   3   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.701a    4   6   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.731a    4   9   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.611a    3   13  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.771a    4   13  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.741b    4   10  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.881b    5   8   FC:A   -  ATA   7200 423111/866531584  847827/1736350304

    RAID group /aggr3/plex0/rg1 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      dparity 1b.891b    5   9   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      parity  1a.721a    4   8   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

      data    1b.671b    4   3   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.751b    4   11  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.661b    4   2   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.641b    4   0   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.731b    4   9   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.741a    4   10  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.661a    4   2   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.871a    5   7   FC:A   -  ATA   7200 423111/866531584  847827/1736350304

      data    1a.651a    4   1   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.711a    4   7   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.641a    4   0   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.761b    4   12  FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1b.681b    4   4   FC:A   -  ATA   7200 423111/866531584  423889/868126304

      data    1a.911a    5   11  FC:A   -  ATA   7200 847555/1735794176 847827/1736350304

Aggregate aggr1 (online, raid4) (block checksums)

  Plex /aggr1/plex0 (online, normal, active)

    RAID group /aggr1/plex0/rg0 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      parity  1a.261a    1   10  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.191b    1   3   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.191a    1   3   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.231b    1   7   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.451a    2   13  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.171a    1   1   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.231a    1   7   FC:A   -  ATA   5400 274400/561971200  274540/562258784

    RAID group /aggr1/plex0/rg1 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      parity  1b.171b    1   1   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.271a    1   11  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.271b    1   11  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.201a    1   4   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.201b    1   4   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.241a    1   8   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.241b    1   8   FC:A   -  ATA   5400 274400/561971200  274540/562258784

    RAID group /aggr1/plex0/rg2 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      parity  1a.181a    1   2   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.281b    1   12  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.281a    1   12  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.211b    1   5   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.211a    1   5   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.161b    1   0   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.181b    1   2   FC:A   -  ATA   5400 274400/561971200  274540/562258784

    RAID group /aggr1/plex0/rg3 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      parity  1a.251a    1   9   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.291a    1   13  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.221b    1   6   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.291b    1   13  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.421a    2   10  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.421b    2   10  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.221a    1   6   FC:A   -  ATA   5400 274400/561971200  274540/562258784

    RAID group /aggr1/plex0/rg4 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      parity  1b.391b    2   7   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.391a    2   7   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.351b    2   3   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.351a    2   3   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.321b    2   0   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.321a    2   0   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.431b    2   11  FC:A   -  ATA   5400 274400/561971200  274540/562258784

    RAID group /aggr1/plex0/rg5 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      parity  1b.361b    2   4   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.431a    2   11  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.401b    2   8   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.361a    2   4   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.331b    2   1   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.401a    2   8   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.441b    2   12  FC:A   -  ATA   5400 274400/561971200  274540/562258784

    RAID group /aggr1/plex0/rg6 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      parity  1a.331a    2   1   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.371b    2   5   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.441a    2   12  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.411b    2   9   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.371a    2   5   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.341b    2   2   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.411a    2   9   FC:A   -  ATA   5400 274400/561971200  274540/562258784

    RAID group /aggr1/plex0/rg7 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      parity  1b.451b    2   13  FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.341a    2   2   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1b.381b    2   6   FC:A   -  ATA   5400 274400/561971200  274540/562258784

Volume root (online, raid4) (block checksums)

  Plex /root/plex0 (online, normal, active)

    RAID group /root/plex0/rg0 (normal)

      RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------------------------- ---- ---- ---- ----- --------------    --------------

      parity  1a.381a    2   6   FC:A   -  ATA   5400 274400/561971200  274540/562258784

      data    1a.161a    1   0   FC:A   -  ATA   5400 274400/561971200  274540/562258784

Spare disks

RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

---------------------------- ---- ---- ---- ----- --------------    --------------

Spare disks for block or zoned checksum traditional volumes or aggregates

spare   1a.751a    4   11  FC:A   -  ATA   7200 423111/866531584  423889/868126304

Broken disks

RAID DiskDeviceHA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

---------------------------- ---- ---- ---- ----- --------------    --------------

init failed1b.261b    1   10  FC:A   -    ?    ?   ?/?               ?/?              

aborzenkov
15,081 Views

Oh, that makes a difference. If port 1d was never connected in the first place, then it could be anything - adapter in slot 1, SFPs on port 1c (both of them) or AT-FCX module. In theory even defective disk could block loop. I’d start with power cycling shelf itself; obvious next step is to connect to another port.

Sorry, but the only way here is to troubleshoot on site by replacing components.

jeffshelly
15,080 Views

Yeah, I was afraid of that...

Thank you for taking a look and enjoy some points for a correct answer.

Wish me luck...

vims
15,080 Views

taking in consideration that 12 disk fails at once - the exact half if shelf, could be a back plane. Do not remember the architecture of DS14 shelf, know for sure that slots 0 and 1 has to be populated for enclosure service. 

aborzenkov
15,080 Views

How can 12 be exact half of 14?

vims
15,080 Views

oops shame on me messed myself up sorry...

ps dealing lately with 2243/6 shelves may be that's why

Public