ONTAP Hardware
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- Home
- :
- ONTAP, AFF, and FAS
- :
- ONTAP Hardware
- :
- Re: Understanding disk paths
ONTAP Hardware
Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, guys!
We have 4-node fas8200 metrocluster, ontap 9.6p5. I'm trying understand how to interpret the output of the "storage disk show" command. I did the following:
cluster_a::> storage disk show -primary-port B -secondary-port B
Usable Disk Container Container
Disk Size Shelf Bay Type Type Name Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
3.10.1 1.63TB 10 1 SAS aggregate aggr1_a cluster_a-01
3.10.2 1.63TB 10 2 SAS aggregate aggr1_a cluster_a-01
3.10.4 1.63TB 10 4 SAS aggregate aggr0_cluster_a_01
cluster_a-01
3.10.5 1.63TB 10 5 SAS aggregate aggr1_a cluster_a-01
3.10.8 1.63TB 10 8 SAS aggregate aggr1_a cluster_a-01
3.10.10 1.63TB 10 10 SAS aggregate aggr1_a cluster_a-01
3.10.11 1.63TB 10 11 SAS aggregate aggr1_a cluster_a-01
3.10.15 1.63TB 10 15 SAS aggregate aggr1_a cluster_a-01
3.10.16 1.63TB 10 16 SAS aggregate aggr1_a cluster_a-01
3.10.18 1.63TB 10 18 SAS aggregate aggr1_a cluster_a-01
3.10.20 1.63TB 10 20 SAS aggregate aggr1_a cluster_a-01
3.10.22 1.63TB 10 22 SAS aggregate aggr1_a cluster_a-01
3.10.23 1.63TB 10 23 SAS spare Pool1 cluster_a-01
3.11.0 1.63TB 11 0 SAS aggregate aggr0_cluster_a_02
cluster_a-02
3.11.2 1.63TB 11 2 SAS aggregate aggr0_cluster_a_02
cluster_a-02
3.11.3 1.63TB 11 3 SAS aggregate aggr2_a cluster_a-02
3.11.7 1.63TB 11 7 SAS aggregate aggr2_a cluster_a-02
3.11.8 1.63TB 11 8 SAS aggregate aggr2_a cluster_a-02
3.11.9 1.63TB 11 9 SAS aggregate aggr2_a cluster_a-02
3.11.11 1.63TB 11 11 SAS aggregate aggr2_a cluster_a-02
3.11.12 1.63TB 11 12 SAS aggregate aggr2_a cluster_a-02
3.11.18 1.63TB 11 18 SAS aggregate aggr2_a cluster_a-02
3.11.20 1.63TB 11 20 SAS aggregate aggr2_a cluster_a-02
3.11.21 1.63TB 11 21 SAS aggregate aggr2_a cluster_a-02
3.12.0 - 12 0 SAS remote - cluster_b-01
3.12.1 - 12 1 SAS remote - cluster_b-01
3.12.2 - 12 2 SAS remote - cluster_b-01
3.12.8 - 12 8 SAS remote - cluster_b-01
3.12.12 - 12 12 SAS remote - cluster_b-01
3.12.14 - 12 14 SAS remote - cluster_b-01
3.12.18 - 12 18 SAS remote - cluster_b-01
3.12.20 - 12 20 SAS remote - cluster_b-01
3.12.22 - 12 22 SAS remote - cluster_b-01
3.13.3 - 13 3 SAS remote - cluster_b-02
3.13.6 - 13 6 SAS remote - cluster_b-02
3.13.7 - 13 7 SAS remote - cluster_b-02
3.13.8 - 13 8 SAS remote - cluster_b-02
3.13.10 - 13 10 SAS remote - cluster_b-02
3.13.11 - 13 11 SAS remote - cluster_b-02
3.13.12 - 13 12 SAS remote - cluster_b-02
3.13.14 - 13 14 SAS remote - cluster_b-02
3.13.15 - 13 15 SAS remote - cluster_b-02
3.13.16 - 13 16 SAS remote - cluster_b-02
3.13.17 - 13 17 SAS remote - cluster_b-02
3.13.18 - 13 18 SAS remote - cluster_b-02
3.13.19 - 13 19 SAS remote - cluster_b-02
3.13.20 - 13 20 SAS remote - cluster_b-02
3.13.21 - 13 21 SAS remote - cluster_b-02
3.13.23 - 13 23 SAS remote - cluster_b-02
Usable Disk Container Container
Disk Size Shelf Bay Type Type Name Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
4.14.0 - 14 0 SSD remote - cluster_b-02
4.14.1 - 14 1 SSD remote - cluster_b-02
4.14.2 - 14 2 SSD remote - cluster_b-02
4.14.4 - 14 4 SSD remote - cluster_b-02
4.14.6 - 14 6 SSD remote - cluster_b-02
4.14.9 - 14 9 SSD remote - cluster_b-02
4.14.16 - 14 16 SSD remote - cluster_b-02
4.14.19 - 14 19 SSD remote - cluster_b-02
4.14.20 - 14 20 SSD remote - cluster_b-02
4.14.22 - 14 22 SSD remote - cluster_b-02
4.14.23 - 14 23 SSD remote - cluster_b-02
60 entries were displayed.
cluster_a::> storage disk show -primary-port A -secondary-port A
Usable Disk Container Container
Disk Size Shelf Bay Type Type Name Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
1.4.0 - 4 0 SSD unassigned - -
1.4.2 - 4 2 SSD unassigned - -
1.4.3 - 4 3 SSD unassigned - -
1.4.4 - 4 4 SSD unassigned - -
1.4.5 - 4 5 SSD unassigned - -
1.4.6 - 4 6 SSD unassigned - -
1.4.7 - 4 7 SSD unassigned - -
1.4.9 - 4 9 SSD unassigned - -
1.4.10 - 4 10 SSD unassigned - -
1.4.11 - 4 11 SSD unassigned - -
1.4.12 - 4 12 SSD unassigned - -
1.4.14 - 4 14 SSD unassigned - -
1.4.15 - 4 15 SSD unassigned - -
1.4.18 - 4 18 SSD unassigned - -
1.4.19 - 4 19 SSD unassigned - -
1.4.23 - 4 23 SSD unassigned - -
2.0.1 1.63TB 0 1 SAS aggregate aggr0_cluster_a_01
cluster_a-01
2.0.2 1.63TB 0 2 SAS aggregate aggr0_cluster_a_01
cluster_a-01
2.0.3 1.63TB 0 3 SAS aggregate aggr1_a cluster_a-01
2.0.7 1.63TB 0 7 SAS aggregate aggr1_a cluster_a-01
2.0.8 1.63TB 0 8 SAS aggregate aggr1_a cluster_a-01
2.0.13 1.63TB 0 13 SAS aggregate aggr1_a cluster_a-01
2.0.14 1.63TB 0 14 SAS aggregate aggr1_a cluster_a-01
2.0.18 1.63TB 0 18 SAS aggregate aggr1_a cluster_a-01
2.0.19 1.63TB 0 19 SAS aggregate aggr1_a cluster_a-01
2.0.20 1.63TB 0 20 SAS aggregate aggr1_a cluster_a-01
2.0.21 1.63TB 0 21 SAS aggregate aggr1_a cluster_a-01
2.0.23 1.63TB 0 23 SAS spare Pool0 cluster_a-01
2.1.0 1.63TB 1 0 SAS aggregate aggr0_cluster_a_02
cluster_a-02
2.1.1 1.63TB 1 1 SAS aggregate aggr0_cluster_a_02
cluster_a-02
2.1.6 1.63TB 1 6 SAS aggregate aggr2_a cluster_a-02
2.1.7 1.63TB 1 7 SAS aggregate aggr2_a cluster_a-02
2.1.9 1.63TB 1 9 SAS aggregate aggr2_a cluster_a-02
2.1.11 1.63TB 1 11 SAS aggregate aggr2_a cluster_a-02
2.1.12 1.63TB 1 12 SAS aggregate aggr2_a cluster_a-02
2.1.14 1.63TB 1 14 SAS aggregate aggr2_a cluster_a-02
2.1.20 1.63TB 1 20 SAS aggregate aggr2_a cluster_a-02
2.1.21 1.63TB 1 21 SAS aggregate aggr2_a cluster_a-02
2.1.22 1.63TB 1 22 SAS aggregate aggr2_a cluster_a-02
2.1.23 1.63TB 1 23 SAS spare Pool0 cluster_a-02
2.2.0 - 2 0 SAS remote - cluster_b-01
2.2.2 - 2 2 SAS remote - cluster_b-01
2.2.3 - 2 3 SAS remote - cluster_b-01
2.2.4 - 2 4 SAS remote - cluster_b-01
2.2.6 - 2 6 SAS remote - cluster_b-01
2.2.8 - 2 8 SAS remote - cluster_b-01
2.2.10 - 2 10 SAS remote - cluster_b-01
2.2.12 - 2 12 SAS remote - cluster_b-01
Usable Disk Container Container
Disk Size Shelf Bay Type Type Name Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
2.2.13 - 2 13 SAS remote - cluster_b-01
2.2.15 - 2 15 SAS remote - cluster_b-01
2.2.16 - 2 16 SAS remote - cluster_b-01
2.2.18 - 2 18 SAS remote - cluster_b-01
2.2.19 - 2 19 SAS remote - cluster_b-01
2.2.21 - 2 21 SAS remote - cluster_b-01
2.2.22 - 2 22 SAS remote - cluster_b-01
2.2.23 - 2 23 SAS remote - cluster_b-01
2.3.21 - 3 21 SAS remote - cluster_b-02
57 entries were displayed.
So, as you can see, according this output, i have 60 disks with both B paths as primary and secondary and 57 disks with both A paths as primary and secondary. Does that mean that the system is not redundant? I thought that all the disks should have A and B paths as primary and secondary, but if the "metrocluster check show" shows that everything is ok, my assumption is wrong. So, could you give me any comments about the output of the "storage disk show"?
16 REPLIES 16
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
can you provide the output of this:
::> storage disk show -fields secondary-port,primary-port,initiator,nodelist
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Here:
cluster_a::> storage disk show -fields secondary-port,primary-port,initiator,nodelist disk initiator nodelist primary-port secondary-port ------- ----------------------- ------------------------------------------------------------------------------------------------------- ------------ -------------- 1.4.0 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.1 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 1.4.2 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.3 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.4 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.5 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.6 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.7 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.8 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.9 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.10 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.11 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.12 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.13 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 1.4.14 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.15 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.16 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.17 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.18 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.19 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.20 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 1.4.21 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.22 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 1.4.23 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.0.0 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A A 2.0.1 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.2 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A A 2.0.3 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A A 2.0.4 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A A 2.0.5 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.6 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A A 2.0.7 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.8 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.9 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.10 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A A 2.0.11 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.12 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.13 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.14 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.15 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.16 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.17 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.18 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.19 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A A 2.0.20 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.21 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A A 2.0.22 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B A 2.0.23 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A A 2.1.0 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.1 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.2 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.3 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.4 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.5 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.6 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.7 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.8 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.9 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.10 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.11 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.12 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.13 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.14 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.15 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.16 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.17 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.18 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.1.19 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.20 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.21 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.22 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.1.23 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.0 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.1 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.2 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.3 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.4 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.5 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.2.6 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.7 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.2.8 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.9 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.2.10 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.11 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.2.12 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.13 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.14 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.2.15 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.16 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.17 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.2.18 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.19 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.20 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.2.21 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.22 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.2.23 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.3.0 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.1 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.2 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.3 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.4 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.5 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.6 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.7 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.8 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.9 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.10 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.11 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.12 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.13 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.14 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.15 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.16 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.17 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.18 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.19 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.20 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.21 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A A 2.3.22 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 2.3.23 1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B A 3.10.0 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.1 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.2 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.3 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.4 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.5 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.6 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.7 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.8 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.9 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.11 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A B 3.10.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.10.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B B 3.11.0 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.11.1 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.2 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.11.3 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.11.4 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.11.5 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.11.6 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.7 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.8 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.9 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.11.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.11 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.11.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.11.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.0 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.12.1 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.12.2 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.12.3 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.4 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.5 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.6 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.7 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.8 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.12.9 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.11 1a,1b,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.12.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.12.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.12.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.12.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.12.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.12.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.13.0 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.13.1 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.13.2 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.13.3 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.4 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.13.5 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.13.6 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.7 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.8 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.9 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.13.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.11 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.13.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 3.13.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 3.13.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.0 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.1 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.2 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.3 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.4 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.5 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.6 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.7 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.8 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.9 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.11 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A B 4.14.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 4.14.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B B 240 entries were displayed. cluster_a::>
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Something don't look alright in the output you have shared (Just from regular HA perspective). Probably, metrocluster check show command doesn't even meant to check the disk paths and therefore says ok.
Using one example:
For Shelf 4, only BAYS:1,13,20 shows dual-paths [Pri:A, Sec:B], rest all the Bays are showing Prim & Sec as 'A'. Could you ask someone to take a physical inspection of shelf, and ensure the cabling is done correctly ?
Could you simply run this command and share the output.
::> storage disk show -port
Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi!
I agree with you that something is wrong here, but i just don't understand, what. Let me explain: there're two stacks, on the each side, one with four shelfs with hdd drives, second one with single ssd shelf. I checked the cabling and it's done correctly, just exactly as in the metrocluster installation documentation (there's an example with two stacks - just as here).
I've already tried to find some information from the "storage disk show -port", but it looks abcolutely illogical for me. In the output, as you can see, the ssds, which are located in one shelf (well, I hope, that they are, according netapp shelf partnumbers 🙂 ), some have A-A paths, some B-A *(disks 1.4.0-1.4.23) and the same is for the second shelf with ssd drives: some have A-B and some have B-B (4.14.0-4.14.23). Just in theory, how this could happen?
Here's the output you asked:
cluster_a::> storage disk show -port Primary Port Secondary Port Type Shelf Bay --------------- ---- --------------- ---- ------ ----- --- 1.4.0 A cluster_a-01:netapp-sansw-1b:1-9.126L501 A SSD 4 0 1.4.1 A cluster_a-01:netapp-sansw-1a:1-9.126L502 A SSD 4 1 1.4.2 B cluster_a-01:netapp-sansw-1b:1-9.126L503 A SSD 4 2 1.4.3 B cluster_a-01:netapp-sansw-1b:1-9.126L504 A SSD 4 3 1.4.4 B cluster_a-01:netapp-sansw-1b:1-9.126L505 A SSD 4 4 1.4.5 B cluster_a-01:netapp-sansw-1b:1-9.126L506 A SSD 4 5 1.4.6 B cluster_a-01:netapp-sansw-1b:1-9.126L507 A SSD 4 6 1.4.7 B cluster_a-01:netapp-sansw-1b:1-9.126L508 A SSD 4 7 1.4.8 A cluster_a-01:netapp-sansw-1b:1-9.126L509 A SSD 4 8 1.4.9 A cluster_a-01:netapp-sansw-1a:1-9.126L510 A SSD 4 9 1.4.10 A cluster_a-01:netapp-sansw-1a:1-9.126L511 A SSD 4 10 1.4.11 B cluster_a-01:netapp-sansw-1b:1-9.126L512 A SSD 4 11 1.4.12 B cluster_a-01:netapp-sansw-1b:1-9.126L513 A SSD 4 12 1.4.13 A cluster_a-01:netapp-sansw-1a:1-9.126L514 A SSD 4 13 1.4.14 A cluster_a-01:netapp-sansw-1a:1-9.126L515 A SSD 4 14 1.4.15 A cluster_a-01:netapp-sansw-1a:1-9.126L516 A SSD 4 15 1.4.16 B cluster_a-01:netapp-sansw-1b:1-9.126L517 A SSD 4 16 1.4.17 A cluster_a-01:netapp-sansw-1a:1-9.126L518 A SSD 4 17 1.4.18 B cluster_a-01:netapp-sansw-1b:1-9.126L519 A SSD 4 18 1.4.19 A cluster_a-01:netapp-sansw-1a:1-9.126L520 A SSD 4 19 1.4.20 A cluster_a-01:netapp-sansw-1a:1-9.126L521 A SSD 4 20 1.4.21 A cluster_a-01:netapp-sansw-1a:1-9.126L522 A SSD 4 21 1.4.22 A cluster_a-01:netapp-sansw-1a:1-9.126L523 A SSD 4 22 1.4.23 B cluster_a-01:netapp-sansw-1b:1-9.126L524 A SSD 4 23 2.0.0 A cluster_a-01:netapp-sansw-1b:1-9.126L1 A SAS 0 0 2.0.1 B cluster_a-01:netapp-sansw-1b:1-9.126L2 A SAS 0 1 2.0.2 A cluster_a-01:netapp-sansw-1a:1-9.126L3 A SAS 0 2 2.0.3 B cluster_a-01:netapp-sansw-1b:1-9.126L4 A SAS 0 3 2.0.4 A cluster_a-01:netapp-sansw-1a:1-9.126L5 A SAS 0 4 2.0.5 B cluster_a-01:netapp-sansw-1b:1-9.126L6 A SAS 0 5 2.0.6 B cluster_a-01:netapp-sansw-1b:1-9.126L7 A SAS 0 6 2.0.7 B cluster_a-01:netapp-sansw-1b:1-9.126L8 A SAS 0 7 2.0.8 A cluster_a-01:netapp-sansw-1a:1-9.126L9 A SAS 0 8 2.0.9 A cluster_a-01:netapp-sansw-1b:1-9.126L10 A SAS 0 9 2.0.10 B cluster_a-01:netapp-sansw-1b:1-9.126L11 A SAS 0 10 2.0.11 A cluster_a-01:netapp-sansw-1b:1-9.126L12 A SAS 0 11 2.0.12 A cluster_a-01:netapp-sansw-1b:1-9.126L13 A SAS 0 12 2.0.13 B cluster_a-01:netapp-sansw-1b:1-9.126L14 A SAS 0 13 2.0.14 A cluster_a-01:netapp-sansw-1b:1-9.126L15 A SAS 0 14 2.0.15 A cluster_a-01:netapp-sansw-1a:1-9.126L16 A SAS 0 15 2.0.16 A cluster_a-01:netapp-sansw-1b:1-9.126L17 A SAS 0 16 2.0.17 B cluster_a-01:netapp-sansw-1b:1-9.126L18 A SAS 0 17 2.0.18 B cluster_a-01:netapp-sansw-1b:1-9.126L19 A SAS 0 18 2.0.19 B cluster_a-01:netapp-sansw-1b:1-9.126L20 A SAS 0 19 2.0.20 B cluster_a-01:netapp-sansw-1b:1-9.126L21 A SAS 0 20 2.0.21 A cluster_a-01:netapp-sansw-1a:1-9.126L22 A SAS 0 21 2.0.22 A cluster_a-01:netapp-sansw-1a:1-9.126L23 A SAS 0 22 2.0.23 B cluster_a-01:netapp-sansw-1b:1-9.126L24 A SAS 0 23 2.1.0 B cluster_a-02:netapp-sansw-1b:1-9.126L27 A SAS 1 0 2.1.1 B cluster_a-02:netapp-sansw-1b:1-9.126L28 A SAS 1 1 2.1.2 B cluster_a-02:netapp-sansw-1b:1-9.126L29 A SAS 1 2 2.1.3 A cluster_a-02:netapp-sansw-1b:1-9.126L30 A SAS 1 3 2.1.4 B cluster_a-02:netapp-sansw-1b:1-9.126L31 A SAS 1 4 2.1.5 A cluster_a-02:netapp-sansw-1a:1-9.126L32 A SAS 1 5 2.1.6 A cluster_a-02:netapp-sansw-1b:1-9.126L33 A SAS 1 6 2.1.7 B cluster_a-02:netapp-sansw-1b:1-9.126L34 A SAS 1 7 2.1.8 B cluster_a-02:netapp-sansw-1b:1-9.126L35 A SAS 1 8 2.1.9 A cluster_a-02:netapp-sansw-1b:1-9.126L36 A SAS 1 9 2.1.10 B cluster_a-02:netapp-sansw-1b:1-9.126L37 A SAS 1 10 2.1.11 A cluster_a-02:netapp-sansw-1a:1-9.126L38 A SAS 1 11 2.1.12 B cluster_a-02:netapp-sansw-1b:1-9.126L39 A SAS 1 12 2.1.13 A cluster_a-02:netapp-sansw-1b:1-9.126L40 A SAS 1 13 2.1.14 A cluster_a-02:netapp-sansw-1a:1-9.126L41 A SAS 1 14 2.1.15 A cluster_a-02:netapp-sansw-1b:1-9.126L42 A SAS 1 15 2.1.16 B cluster_a-02:netapp-sansw-1b:1-9.126L43 A SAS 1 16 2.1.17 B cluster_a-02:netapp-sansw-1b:1-9.126L44 A SAS 1 17 2.1.18 A cluster_a-02:netapp-sansw-1b:1-9.126L45 A SAS 1 18 2.1.19 A cluster_a-02:netapp-sansw-1a:1-9.126L46 A SAS 1 19 2.1.20 A cluster_a-02:netapp-sansw-1a:1-9.126L47 A SAS 1 20 2.1.21 B cluster_a-02:netapp-sansw-1b:1-9.126L48 A SAS 1 21 2.1.22 B cluster_a-02:netapp-sansw-1b:1-9.126L49 A SAS 1 22 2.1.23 A cluster_a-02:netapp-sansw-1b:1-9.126L50 A SAS 1 23 2.2.0 A cluster_a-01:netapp-sansw-1a:1-9.126L53 A SAS 2 0 2.2.1 B cluster_a-01:netapp-sansw-1b:1-9.126L54 A SAS 2 1 2.2.2 A cluster_a-01:netapp-sansw-1a:1-9.126L55 A SAS 2 2 2.2.3 A cluster_a-01:netapp-sansw-1a:1-9.126L56 A SAS 2 3 2.2.4 A cluster_a-01:netapp-sansw-1a:1-9.126L57 A SAS 2 4 2.2.5 B cluster_a-01:netapp-sansw-1b:1-9.126L58 A SAS 2 5 2.2.6 B cluster_a-01:netapp-sansw-1b:1-9.126L59 A SAS 2 6 2.2.7 B cluster_a-01:netapp-sansw-1b:1-9.126L60 A SAS 2 7 2.2.8 B cluster_a-01:netapp-sansw-1b:1-9.126L61 A SAS 2 8 2.2.9 B cluster_a-01:netapp-sansw-1b:1-9.126L62 A SAS 2 9 2.2.10 A cluster_a-01:netapp-sansw-1a:1-9.126L63 A SAS 2 10 2.2.11 B cluster_a-01:netapp-sansw-1b:1-9.126L64 A SAS 2 11 2.2.12 A cluster_a-01:netapp-sansw-1a:1-9.126L65 A SAS 2 12 2.2.13 A cluster_a-01:netapp-sansw-1a:1-9.126L66 A SAS 2 13 2.2.14 B cluster_a-01:netapp-sansw-1b:1-9.126L67 A SAS 2 14 2.2.15 A cluster_a-01:netapp-sansw-1b:1-9.126L68 A SAS 2 15 2.2.16 A cluster_a-01:netapp-sansw-1a:1-9.126L69 A SAS 2 16 2.2.17 B cluster_a-01:netapp-sansw-1b:1-9.126L70 A SAS 2 17 2.2.18 A cluster_a-01:netapp-sansw-1b:1-9.126L71 A SAS 2 18 2.2.19 A cluster_a-01:netapp-sansw-1b:1-9.126L72 A SAS 2 19 2.2.20 B cluster_a-01:netapp-sansw-1b:1-9.126L73 A SAS 2 20 2.2.21 A cluster_a-01:netapp-sansw-1b:1-9.126L74 A SAS 2 21 2.2.22 A cluster_a-01:netapp-sansw-1b:1-9.126L75 A SAS 2 22 2.2.23 A cluster_a-01:netapp-sansw-1a:1-9.126L76 A SAS 2 23 2.3.0 B cluster_a-01:netapp-sansw-1b:1-9.126L79 A SAS 3 0 2.3.1 B cluster_a-01:netapp-sansw-1b:1-9.126L80 A SAS 3 1 2.3.2 B cluster_a-01:netapp-sansw-1b:1-9.126L81 A SAS 3 2 2.3.3 B cluster_a-01:netapp-sansw-1b:1-9.126L82 A SAS 3 3 2.3.4 B cluster_a-01:netapp-sansw-1b:1-9.126L83 A SAS 3 4 2.3.5 B cluster_a-01:netapp-sansw-1b:1-9.126L84 A SAS 3 5 2.3.6 A cluster_a-01:netapp-sansw-1b:1-9.126L85 A SAS 3 6 2.3.7 B cluster_a-01:netapp-sansw-1b:1-9.126L86 A SAS 3 7 2.3.8 B cluster_a-01:netapp-sansw-1b:1-9.126L87 A SAS 3 8 2.3.9 B cluster_a-01:netapp-sansw-1b:1-9.126L88 A SAS 3 9 2.3.10 B cluster_a-01:netapp-sansw-1b:1-9.126L89 A SAS 3 10 2.3.11 B cluster_a-01:netapp-sansw-1b:1-9.126L90 A SAS 3 11 2.3.12 A cluster_a-01:netapp-sansw-1b:1-9.126L91 A SAS 3 12 2.3.13 A cluster_a-01:netapp-sansw-1a:1-9.126L92 A SAS 3 13 2.3.14 B cluster_a-01:netapp-sansw-1b:1-9.126L93 A SAS 3 14 2.3.15 A cluster_a-01:netapp-sansw-1a:1-9.126L94 A SAS 3 15 2.3.16 B cluster_a-01:netapp-sansw-1b:1-9.126L95 A SAS 3 16 2.3.17 B cluster_a-01:netapp-sansw-1b:1-9.126L96 A SAS 3 17 2.3.18 B cluster_a-01:netapp-sansw-1b:1-9.126L97 A SAS 3 18 2.3.19 B cluster_a-01:netapp-sansw-1b:1-9.126L98 A SAS 3 19 2.3.20 B cluster_a-01:netapp-sansw-1b:1-9.126L99 A SAS 3 20 2.3.21 A cluster_a-01:netapp-sansw-1b:1-9.126L100 A SAS 3 21 2.3.22 B cluster_a-01:netapp-sansw-1b:1-9.126L101 A SAS 3 22 2.3.23 B cluster_a-01:netapp-sansw-1b:1-9.126L102 A SAS 3 23 3.10.0 B cluster_a-01:netapp-sansw-2b:1-10.126L79 B SAS 10 0 3.10.1 B cluster_a-01:netapp-sansw-2a:1-10.126L80 B SAS 10 1 3.10.2 A cluster_a-01:netapp-sansw-2a:1-10.126L81 B SAS 10 2 3.10.3 A cluster_a-01:netapp-sansw-2a:1-10.126L82 B SAS 10 3 3.10.4 B cluster_a-01:netapp-sansw-2a:1-10.126L83 B SAS 10 4 3.10.5 B cluster_a-01:netapp-sansw-2b:1-10.126L84 B SAS 10 5 3.10.6 A cluster_a-01:netapp-sansw-2a:1-10.126L85 B SAS 10 6 3.10.7 B cluster_a-01:netapp-sansw-2a:1-10.126L86 B SAS 10 7 3.10.8 A cluster_a-01:netapp-sansw-2a:1-10.126L87 B SAS 10 8 3.10.9 A cluster_a-01:netapp-sansw-2a:1-10.126L88 B SAS 10 9 3.10.10 B cluster_a-01:netapp-sansw-2b:1-10.126L89 B SAS 10 10 3.10.11 A cluster_a-01:netapp-sansw-2a:1-10.126L90 B SAS 10 11 3.10.12 A cluster_a-01:netapp-sansw-2a:1-10.126L91 B SAS 10 12 3.10.13 A cluster_a-01:netapp-sansw-2a:1-10.126L92 B SAS 10 13 3.10.14 A cluster_a-01:netapp-sansw-2a:1-10.126L93 B SAS 10 14 3.10.15 B cluster_a-01:netapp-sansw-2b:1-10.126L94 B SAS 10 15 3.10.16 B cluster_a-01:netapp-sansw-2a:1-10.126L95 B SAS 10 16 3.10.17 A cluster_a-01:netapp-sansw-2a:1-10.126L96 B SAS 10 17 3.10.18 A cluster_a-01:netapp-sansw-2a:1-10.126L97 B SAS 10 18 3.10.19 A cluster_a-01:netapp-sansw-2a:1-10.126L98 B SAS 10 19 3.10.20 A cluster_a-01:netapp-sansw-2a:1-10.126L99 B SAS 10 20 3.10.21 A cluster_a-01:netapp-sansw-2a:1-10.126L100 B SAS 10 21 3.10.22 A cluster_a-01:netapp-sansw-2a:1-10.126L101 B SAS 10 22 3.10.23 B cluster_a-01:netapp-sansw-2b:1-10.126L102 B SAS 10 23 3.11.0 A cluster_a-02:netapp-sansw-2a:1-10.126L53 B SAS 11 0 3.11.1 A cluster_a-02:netapp-sansw-2a:1-10.126L54 B SAS 11 1 3.11.2 B cluster_a-02:netapp-sansw-2b:1-10.126L55 B SAS 11 2 3.11.3 A cluster_a-02:netapp-sansw-2a:1-10.126L56 B SAS 11 3 3.11.4 A cluster_a-02:netapp-sansw-2a:1-10.126L57 B SAS 11 4 3.11.5 B cluster_a-02:netapp-sansw-2b:1-10.126L58 B SAS 11 5 3.11.6 B cluster_a-02:netapp-sansw-2b:1-10.126L59 B SAS 11 6 3.11.7 A cluster_a-02:netapp-sansw-2a:1-10.126L60 B SAS 11 7 3.11.8 B cluster_a-02:netapp-sansw-2a:1-10.126L61 B SAS 11 8 3.11.9 A cluster_a-02:netapp-sansw-2a:1-10.126L62 B SAS 11 9 3.11.10 B cluster_a-02:netapp-sansw-2a:1-10.126L63 B SAS 11 10 3.11.11 B cluster_a-02:netapp-sansw-2a:1-10.126L64 B SAS 11 11 3.11.12 B cluster_a-02:netapp-sansw-2a:1-10.126L65 B SAS 11 12 3.11.13 B cluster_a-02:netapp-sansw-2b:1-10.126L66 B SAS 11 13 3.11.14 B cluster_a-02:netapp-sansw-2a:1-10.126L67 B SAS 11 14 3.11.15 A cluster_a-02:netapp-sansw-2a:1-10.126L68 B SAS 11 15 3.11.16 A cluster_a-02:netapp-sansw-2a:1-10.126L69 B SAS 11 16 3.11.17 A cluster_a-02:netapp-sansw-2a:1-10.126L70 B SAS 11 17 3.11.18 B cluster_a-02:netapp-sansw-2b:1-10.126L71 B SAS 11 18 3.11.19 B cluster_a-02:netapp-sansw-2b:1-10.126L72 B SAS 11 19 3.11.20 A cluster_a-02:netapp-sansw-2a:1-10.126L73 B SAS 11 20 3.11.21 B cluster_a-02:netapp-sansw-2a:1-10.126L74 B SAS 11 21 3.11.22 B cluster_a-02:netapp-sansw-2a:1-10.126L75 B SAS 11 22 3.11.23 B cluster_a-02:netapp-sansw-2a:1-10.126L76 B SAS 11 23 3.12.0 B cluster_a-01:netapp-sansw-2b:1-10.126L27 B SAS 12 0 3.12.1 B cluster_a-01:netapp-sansw-2a:1-10.126L28 B SAS 12 1 3.12.2 B cluster_a-01:netapp-sansw-2a:1-10.126L29 B SAS 12 2 3.12.3 A cluster_a-01:netapp-sansw-2a:1-10.126L30 B SAS 12 3 3.12.4 A cluster_a-01:netapp-sansw-2a:1-10.126L31 B SAS 12 4 3.12.5 A cluster_a-01:netapp-sansw-2a:1-10.126L32 B SAS 12 5 3.12.6 A cluster_a-01:netapp-sansw-2a:1-10.126L33 B SAS 12 6 3.12.7 A cluster_a-01:netapp-sansw-2a:1-10.126L34 B SAS 12 7 3.12.8 B cluster_a-01:netapp-sansw-2b:1-10.126L35 B SAS 12 8 3.12.9 A cluster_a-01:netapp-sansw-2a:1-10.126L36 B SAS 12 9 3.12.10 A cluster_a-01:netapp-sansw-2a:1-10.126L37 B SAS 12 10 3.12.11 B cluster_a-01:netapp-sansw-2b:1-10.126L38 B SAS 12 11 3.12.12 B cluster_a-01:netapp-sansw-2a:1-10.126L39 B SAS 12 12 3.12.13 B cluster_a-01:netapp-sansw-2b:1-10.126L40 B SAS 12 13 3.12.14 B cluster_a-01:netapp-sansw-2b:1-10.126L41 B SAS 12 14 3.12.15 A cluster_a-01:netapp-sansw-2a:1-10.126L42 B SAS 12 15 3.12.16 A cluster_a-01:netapp-sansw-2a:1-10.126L43 B SAS 12 16 3.12.17 B cluster_a-01:netapp-sansw-2a:1-10.126L44 B SAS 12 17 3.12.18 B cluster_a-01:netapp-sansw-2b:1-10.126L45 B SAS 12 18 3.12.19 A cluster_a-01:netapp-sansw-2a:1-10.126L46 B SAS 12 19 3.12.20 B cluster_a-01:netapp-sansw-2a:1-10.126L47 B SAS 12 20 3.12.21 A cluster_a-01:netapp-sansw-2a:1-10.126L48 B SAS 12 21 3.12.22 B cluster_a-01:netapp-sansw-2a:1-10.126L49 B SAS 12 22 3.12.23 A cluster_a-01:netapp-sansw-2a:1-10.126L50 B SAS 12 23 3.13.0 A cluster_a-01:netapp-sansw-2a:1-10.126L1 B SAS 13 0 3.13.1 A cluster_a-01:netapp-sansw-2a:1-10.126L2 B SAS 13 1 3.13.2 A cluster_a-01:netapp-sansw-2a:1-10.126L3 B SAS 13 2 3.13.3 B cluster_a-01:netapp-sansw-2b:1-10.126L4 B SAS 13 3 3.13.4 A cluster_a-01:netapp-sansw-2a:1-10.126L5 B SAS 13 4 3.13.5 A cluster_a-01:netapp-sansw-2a:1-10.126L6 B SAS 13 5 3.13.6 B cluster_a-01:netapp-sansw-2a:1-10.126L7 B SAS 13 6 3.13.7 B cluster_a-01:netapp-sansw-2a:1-10.126L8 B SAS 13 7 3.13.8 B cluster_a-01:netapp-sansw-2a:1-10.126L9 B SAS 13 8 3.13.9 A cluster_a-01:netapp-sansw-2a:1-10.126L10 B SAS 13 9 3.13.10 B cluster_a-01:netapp-sansw-2b:1-10.126L11 B SAS 13 10 3.13.11 B cluster_a-01:netapp-sansw-2b:1-10.126L12 B SAS 13 11 3.13.12 B cluster_a-01:netapp-sansw-2a:1-10.126L13 B SAS 13 12 3.13.13 B cluster_a-01:netapp-sansw-2a:1-10.126L14 B SAS 13 13 3.13.14 B cluster_a-01:netapp-sansw-2a:1-10.126L15 B SAS 13 14 3.13.15 B cluster_a-01:netapp-sansw-2b:1-10.126L16 B SAS 13 15 3.13.16 B cluster_a-01:netapp-sansw-2b:1-10.126L17 B SAS 13 16 3.13.17 B cluster_a-01:netapp-sansw-2b:1-10.126L18 B SAS 13 17 3.13.18 B cluster_a-01:netapp-sansw-2b:1-10.126L19 B SAS 13 18 3.13.19 B cluster_a-01:netapp-sansw-2a:1-10.126L20 B SAS 13 19 3.13.20 B cluster_a-01:netapp-sansw-2a:1-10.126L21 B SAS 13 20 3.13.21 B cluster_a-01:netapp-sansw-2b:1-10.126L22 B SAS 13 21 3.13.22 A cluster_a-01:netapp-sansw-2a:1-10.126L23 B SAS 13 22 3.13.23 B cluster_a-01:netapp-sansw-2b:1-10.126L24 B SAS 13 23 4.14.0 A cluster_a-01:netapp-sansw-2a:1-10.126L501 B SSD 14 0 4.14.1 A cluster_a-01:netapp-sansw-2a:1-10.126L502 B SSD 14 1 4.14.2 A cluster_a-01:netapp-sansw-2a:1-10.126L503 B SSD 14 2 4.14.3 B cluster_a-01:netapp-sansw-2b:1-10.126L504 B SSD 14 3 4.14.4 A cluster_a-01:netapp-sansw-2a:1-10.126L505 B SSD 14 4 4.14.5 A cluster_a-01:netapp-sansw-2a:1-10.126L506 B SSD 14 5 4.14.6 A cluster_a-01:netapp-sansw-2a:1-10.126L507 B SSD 14 6 4.14.7 A cluster_a-01:netapp-sansw-2a:1-10.126L508 B SSD 14 7 4.14.8 B cluster_a-01:netapp-sansw-2a:1-10.126L509 B SSD 14 8 4.14.9 B cluster_a-01:netapp-sansw-2b:1-10.126L510 B SSD 14 9 4.14.10 A cluster_a-01:netapp-sansw-2a:1-10.126L511 B SSD 14 10 4.14.11 A cluster_a-01:netapp-sansw-2a:1-10.126L512 B SSD 14 11 4.14.12 A cluster_a-01:netapp-sansw-2a:1-10.126L513 B SSD 14 12 4.14.13 A cluster_a-01:netapp-sansw-2a:1-10.126L514 B SSD 14 13 4.14.14 A cluster_a-01:netapp-sansw-2a:1-10.126L515 B SSD 14 14 4.14.15 A cluster_a-01:netapp-sansw-2a:1-10.126L516 B SSD 14 15 4.14.16 A cluster_a-01:netapp-sansw-2a:1-10.126L517 B SSD 14 16 4.14.17 A cluster_a-01:netapp-sansw-2a:1-10.126L518 B SSD 14 17 4.14.18 B cluster_a-01:netapp-sansw-2a:1-10.126L519 B SSD 14 18 4.14.19 B cluster_a-01:netapp-sansw-2a:1-10.126L520 B SSD 14 19 4.14.20 A cluster_a-01:netapp-sansw-2a:1-10.126L521 B SSD 14 20 4.14.21 B cluster_a-01:netapp-sansw-2a:1-10.126L522 B SSD 14 21 4.14.22 A cluster_a-01:netapp-sansw-2a:1-10.126L523 B SSD 14 22 4.14.23 B cluster_a-01:netapp-sansw-2a:1-10.126L524 B SSD 14 23 240 entries were displayed.
And the shelf output, which is also strange (the shelf naming):
cluster_a::> storage shelf show
Module Operational
Shelf Name Shelf ID Serial Number Model Type Status
----------------- -------- --------------- ----------- ------ -----------
1.4 4 SHFHU1852XXXXXX DS224-12 IOM12 Normal
2.0 0 SHFHU1849XXXXXX DS224-12 IOM12 Normal
2.1 1 SHFHU1850XXXXXX DS224-12 IOM12 Normal
2.2 2 SHFHU1850XXXXXX DS224-12 IOM12 Normal
2.3 3 SHFHU1848XXXXXX DS224-12 IOM12 Normal
3.10 10 SHFHU1848XXXXXX DS224-12 IOM12 Normal
3.11 11 SHFHU1848XXXXXX DS224-12 IOM12 Normal
3.12 12 SHFHU1848XXXXXX DS224-12 IOM12 Normal
3.13 13 SHFHU1848XXXXXX DS224-12 IOM12 Normal
4.14 14 SHFHU1852XXXXXX DS224-12 IOM12 Normal
10 entries were displayed.
cluster_a::>
and the shelf ports
cluster_a::> storage shelf port show
Shelf ID Module State Internal?
----- -- ------ ------------ ---------
1.4
0 A connected false
1 A disconnected false
2 A disconnected false
3 A disconnected false
4 B disconnected false
5 B disconnected false
6 B connected false
7 B disconnected false
2.0
0 A connected false
1 A disconnected false
2 A connected false
3 A disconnected false
4 B disconnected false
5 B disconnected false
6 B connected false
7 B disconnected false
2.1
0 A connected false
1 A disconnected false
2 A connected false
3 A disconnected false
4 B connected false
5 B disconnected false
6 B connected false
7 B disconnected false
2.2
0 A connected false
1 A disconnected false
2 A connected false
3 A disconnected false
4 B connected false
5 B disconnected false
6 B connected false
7 B disconnected false
2.3
0 A connected false
1 A disconnected false
2 A disconnected false
3 A disconnected false
4 B connected false
5 B disconnected false
6 B connected false
7 B disconnected false
3.10
0 A connected false
1 A disconnected false
2 A connected false
3 A disconnected false
4 B disconnected false
5 B disconnected false
6 B connected false
7 B disconnected false
3.11
0 A connected false
1 A disconnected false
2 A connected false
3 A disconnected false
4 B connected false
5 B disconnected false
6 B connected false
7 B disconnected false
3.12
0 A connected false
1 A disconnected false
2 A connected false
3 A disconnected false
4 B connected false
5 B disconnected false
6 B connected false
7 B disconnected false
3.13
0 A connected false
1 A disconnected false
2 A disconnected false
3 A disconnected false
4 B connected false
5 B disconnected false
6 B connected false
7 B disconnected false
4.14
0 A connected false
1 A disconnected false
2 A disconnected false
3 A disconnected false
4 B disconnected false
5 B disconnected false
6 B connected false
7 B disconnected false
80 entries were displayed.
cluster_a::>
The question is, how exactly should this output look like, if it's cabled as it is shown in the guide? I suspect only one that could happen: after I've checked the physical cabling, there was a test during which one SAS cable was plugged out from FC-SAS bridge and then plugged in again. After that I haven't checked anything, so i guess one cable could be plugged in a wrong port. I've checked this via CLI, but all the bridges show just the same - that everything is ok:
cluster_a::> storage bridge run-cli -name ATTO_2000001086a9d800 -command SASPortList [Job 581] Job is queued: Executing ATTO CLI command "SASPortList" on bridge "ATTO_200000XXXXXXXXX0".. ;Connector PHY Link Speed SAS Address ;============================================================= Device A 1 Up 12Gb 5001086000XXXXX0 Device A 2 Up 12Gb 5001086000XXXXX0 Device A 3 Up 12Gb 5001086000XXXXX0 Device A 4 Up 12Gb 5001086000XXXXX0 Device B 1 Up 12Gb 5001086000XXXXX4 Device B 2 Up 12Gb 5001086000XXXXX4 Device B 3 Up 12Gb 5001086000XXXXX4 Device B 4 Up 12Gb 5001086000XXXXX4 Device C 1 Disabled 12Gb 5001086000XXXXX8 Device C 2 Disabled 12Gb 5001086000XXXXX8 Device C 3 Disabled 12Gb 5001086000XXXXX8 Device C 4 Disabled 12Gb 5001086000XXXXX8 Device D 1 Disabled 12Gb 5001086000XXXXXc Device D 2 Disabled 12Gb 5001086000XXXXXc Device D 3 Disabled 12Gb 5001086000XXXXXc Device D 4 Disabled 12Gb 5001086000XXXXXc
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
"some have A-B and some have B-B (4.14.0-4.14.23). Just in theory, how this could happen?"
It is not bad cabling. It could be either IOM or ATTO bridge that skip some of the disks. The fact that it is showing this anomaly in multiple shelves tells me that it is not an IOM issue
Can you provide the output of 'sysconfig -a' and 'sysconfig -r' from both nodes? (node shell)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Could you please check your PM?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Wops, couldn't send the PM, because the length of output (500-600kb of text)... There's a lot of serial numbers there, i'm not sure that it's a good idea to post it here 😞
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Looks like the situation is not good, so i opened a support case. I hope that helps... Thanks for your answers!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Will be keen to know what is the cause here, Please do let us know later.
As far as the storage shelf output is concerend, it looks fine. You mentioned you have 2 stacks on each side, so based on the shelf naming I am assuming it is in this order.
Node1:
Stack 1: shelf 4 [1.4]
Stack 2: Shelf 0,1,2,3 [2.0,2.1,2.2,2.3]
Node2:
Stack 3: Shelf 10,11,12,13 [3.10,3.11,3.12,3.13]
Stack 4: Shelf 14 [4.14]
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As soon as i'll solve this case, i'll write here the detailed report. According the shelves: physically, they're placed identically on both sites: four hdd-only shelves from the top, then ssd shelf.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Found a defect (Could be your case or may be not but let us know)
Limited number of SAS drives programming defect resulted in the reporting of port "A" on both primary and secondary ports.
https://mysupport.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=472649
To verify:
::> node run -node NODE-01
NODE-01> storage show disk -p
PRIMARY PORT SECONDARY PORT SHELF BAY
-------- ---- --------- ---- ---------
1d.00.4 A 1b.00.4 A 0 4
Do you see PORT A for PRI & SEC for some disks.
Workaround suggested in the BUG : After confirming there are no multipath cabling problems, replace the drive to resolve this issue.
Hopefully you will have more information from NetApp support.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I've updated the firmware on the bridges, but i've found a very strange thing - either different commands show me different outputs, or I just don't understand what they show me :). Here's two outputs, according the first one, everything is ok, and all the disks have two paths - A and B, but according the another one - some disks have two identical paths - A-A or A-B:
cluster_a::> node run -node cluster_a-01
Type 'exit' or 'Ctrl-D' to return to the CLI
cluster_a-01> storage show disk -p
PRIMARY PORT SECONDARY PORT SHELF BAY
---------------------------- ---- ---------------------------- ---- ---------
netapp-sansw-1a:1-9.126L1 A netapp-sansw-1a:1-10.126L79 B 0 0
netapp-sansw-1a:1-9.126L2 A netapp-sansw-1a:1-10.126L80 B 0 1
netapp-sansw-1b:1-9.126L3 A netapp-sansw-1a:1-10.126L81 B 0 2
netapp-sansw-1b:1-10.126L82 B netapp-sansw-1b:1-9.126L4 A 0 3
netapp-sansw-1a:1-10.126L83 B netapp-sansw-1b:1-9.126L5 A 0 4
netapp-sansw-1a:1-10.126L84 B netapp-sansw-1b:1-9.126L6 A 0 5
netapp-sansw-1b:1-10.126L85 B netapp-sansw-1b:1-9.126L7 A 0 6
netapp-sansw-1a:1-10.126L86 B netapp-sansw-1b:1-9.126L8 A 0 7
netapp-sansw-1b:1-9.126L9 A netapp-sansw-1a:1-10.126L87 B 0 8
netapp-sansw-1a:1-9.126L10 A netapp-sansw-1a:1-10.126L88 B 0 9
netapp-sansw-1b:1-9.126L11 A netapp-sansw-1a:1-10.126L89 B 0 10
netapp-sansw-1a:1-10.126L90 B netapp-sansw-1b:1-9.126L12 A 0 11
netapp-sansw-1a:1-10.126L91 B netapp-sansw-1b:1-9.126L13 A 0 12
netapp-sansw-1b:1-9.126L14 A netapp-sansw-1a:1-10.126L92 B 0 13
netapp-sansw-1b:1-9.126L15 A netapp-sansw-1a:1-10.126L93 B 0 14
netapp-sansw-1a:1-10.126L94 B netapp-sansw-1b:1-9.126L16 A 0 15
netapp-sansw-1a:1-9.126L17 A netapp-sansw-1a:1-10.126L95 B 0 16
netapp-sansw-1a:1-9.126L18 A netapp-sansw-1a:1-10.126L96 B 0 17
netapp-sansw-1b:1-10.126L97 B netapp-sansw-1b:1-9.126L19 A 0 18
netapp-sansw-1b:1-10.126L98 B netapp-sansw-1b:1-9.126L20 A 0 19
netapp-sansw-1b:1-10.126L99 B netapp-sansw-1b:1-9.126L21 A 0 20
netapp-sansw-1b:1-9.126L22 A netapp-sansw-1a:1-10.126L100 B 0 21
netapp-sansw-1a:1-10.126L101 B netapp-sansw-1b:1-9.126L23 A 0 22
netapp-sansw-1a:1-9.126L24 A netapp-sansw-1a:1-10.126L102 B 0 23
netapp-sansw-1a:1-10.126L53 B netapp-sansw-1b:1-9.126L27 A 1 0
netapp-sansw-1a:1-10.126L54 B netapp-sansw-1b:1-9.126L28 A 1 1
netapp-sansw-1a:1-10.126L55 B netapp-sansw-1b:1-9.126L29 A 1 2
netapp-sansw-1b:1-9.126L30 A netapp-sansw-1a:1-10.126L56 B 1 3
netapp-sansw-1a:1-9.126L31 A netapp-sansw-1a:1-10.126L57 B 1 4
netapp-sansw-1b:1-9.126L32 A netapp-sansw-1a:1-10.126L58 B 1 5
netapp-sansw-1a:1-10.126L59 B netapp-sansw-1b:1-9.126L33 A 1 6
netapp-sansw-1a:1-10.126L60 B netapp-sansw-1b:1-9.126L34 A 1 7
netapp-sansw-1a:1-10.126L61 B netapp-sansw-1b:1-9.126L35 A 1 8
netapp-sansw-1a:1-9.126L36 A netapp-sansw-1a:1-10.126L62 B 1 9
netapp-sansw-1b:1-9.126L37 A netapp-sansw-1a:1-10.126L63 B 1 10
netapp-sansw-1a:1-10.126L64 B netapp-sansw-1b:1-9.126L38 A 1 11
netapp-sansw-1a:1-9.126L39 A netapp-sansw-1a:1-10.126L65 B 1 12
netapp-sansw-1a:1-10.126L66 B netapp-sansw-1b:1-9.126L40 A 1 13
netapp-sansw-1b:1-10.126L67 B netapp-sansw-1b:1-9.126L41 A 1 14
netapp-sansw-1a:1-10.126L68 B netapp-sansw-1b:1-9.126L42 A 1 15
netapp-sansw-1b:1-10.126L69 B netapp-sansw-1b:1-9.126L43 A 1 16
netapp-sansw-1a:1-10.126L70 B netapp-sansw-1b:1-9.126L44 A 1 17
netapp-sansw-1b:1-10.126L71 B netapp-sansw-1b:1-9.126L45 A 1 18
netapp-sansw-1b:1-10.126L72 B netapp-sansw-1b:1-9.126L46 A 1 19
netapp-sansw-1a:1-10.126L73 B netapp-sansw-1b:1-9.126L47 A 1 20
netapp-sansw-1b:1-10.126L74 B netapp-sansw-1b:1-9.126L48 A 1 21
netapp-sansw-1a:1-10.126L75 B netapp-sansw-1b:1-9.126L49 A 1 22
netapp-sansw-1b:1-10.126L76 B netapp-sansw-1b:1-9.126L50 A 1 23
netapp-sansw-1b:1-9.126L53 A netapp-sansw-1a:1-10.126L27 B 2 0
netapp-sansw-1a:1-9.126L54 A netapp-sansw-1a:1-10.126L28 B 2 1
netapp-sansw-1b:1-9.126L55 A netapp-sansw-1a:1-10.126L29 B 2 2
netapp-sansw-1b:1-9.126L56 A netapp-sansw-1a:1-10.126L30 B 2 3
netapp-sansw-1a:1-9.126L57 A netapp-sansw-1a:1-10.126L31 B 2 4
netapp-sansw-1a:1-10.126L32 B netapp-sansw-1b:1-9.126L58 A 2 5
netapp-sansw-1b:1-9.126L59 A netapp-sansw-1a:1-10.126L33 B 2 6
netapp-sansw-1a:1-9.126L60 A netapp-sansw-1a:1-10.126L34 B 2 7
netapp-sansw-1a:1-10.126L35 B netapp-sansw-1b:1-9.126L61 A 2 8
netapp-sansw-1b:1-9.126L62 A netapp-sansw-1a:1-10.126L36 B 2 9
netapp-sansw-1a:1-9.126L63 A netapp-sansw-1a:1-10.126L37 B 2 10
netapp-sansw-1a:1-10.126L38 B netapp-sansw-1b:1-9.126L64 A 2 11
netapp-sansw-1b:1-9.126L65 A netapp-sansw-1a:1-10.126L39 B 2 12
netapp-sansw-1a:1-9.126L66 A netapp-sansw-1a:1-10.126L40 B 2 13
netapp-sansw-1a:1-9.126L67 A netapp-sansw-1a:1-10.126L41 B 2 14
netapp-sansw-1a:1-10.126L42 B netapp-sansw-1b:1-9.126L68 A 2 15
netapp-sansw-1b:1-9.126L69 A netapp-sansw-1a:1-10.126L43 B 2 16
netapp-sansw-1a:1-9.126L70 A netapp-sansw-1a:1-10.126L44 B 2 17
netapp-sansw-1b:1-10.126L45 B netapp-sansw-1b:1-9.126L71 A 2 18
netapp-sansw-1b:1-10.126L46 B netapp-sansw-1b:1-9.126L72 A 2 19
netapp-sansw-1b:1-10.126L47 B netapp-sansw-1b:1-9.126L73 A 2 20
netapp-sansw-1b:1-10.126L48 B netapp-sansw-1b:1-9.126L74 A 2 21
netapp-sansw-1b:1-10.126L49 B netapp-sansw-1b:1-9.126L75 A 2 22
netapp-sansw-1b:1-10.126L50 B netapp-sansw-1b:1-9.126L76 A 2 23
netapp-sansw-1b:1-9.126L79 A netapp-sansw-1a:1-10.126L1 B 3 0
netapp-sansw-1a:1-9.126L80 A netapp-sansw-1a:1-10.126L2 B 3 1
netapp-sansw-1a:1-10.126L3 B netapp-sansw-1b:1-9.126L81 A 3 2
netapp-sansw-1a:1-9.126L82 A netapp-sansw-1a:1-10.126L4 B 3 3
netapp-sansw-1b:1-9.126L83 A netapp-sansw-1a:1-10.126L5 B 3 4
netapp-sansw-1a:1-10.126L6 B netapp-sansw-1b:1-9.126L84 A 3 5
netapp-sansw-1b:1-9.126L85 A netapp-sansw-1a:1-10.126L7 B 3 6
netapp-sansw-1a:1-9.126L86 A netapp-sansw-1a:1-10.126L8 B 3 7
netapp-sansw-1b:1-9.126L87 A netapp-sansw-1a:1-10.126L9 B 3 8
netapp-sansw-1b:1-9.126L88 A netapp-sansw-1a:1-10.126L10 B 3 9
netapp-sansw-1a:1-9.126L89 A netapp-sansw-1a:1-10.126L11 B 3 10
netapp-sansw-1a:1-9.126L90 A netapp-sansw-1a:1-10.126L12 B 3 11
netapp-sansw-1a:1-9.126L91 A netapp-sansw-1a:1-10.126L13 B 3 12
netapp-sansw-1a:1-9.126L92 A netapp-sansw-1a:1-10.126L14 B 3 13
netapp-sansw-1b:1-9.126L93 A netapp-sansw-1a:1-10.126L15 B 3 14
netapp-sansw-1b:1-9.126L94 A netapp-sansw-1a:1-10.126L16 B 3 15
netapp-sansw-1b:1-9.126L95 A netapp-sansw-1a:1-10.126L17 B 3 16
netapp-sansw-1a:1-9.126L96 A netapp-sansw-1a:1-10.126L18 B 3 17
netapp-sansw-1b:1-9.126L97 A netapp-sansw-1a:1-10.126L19 B 3 18
netapp-sansw-1a:1-9.126L98 A netapp-sansw-1a:1-10.126L20 B 3 19
netapp-sansw-1a:1-9.126L99 A netapp-sansw-1a:1-10.126L21 B 3 20
netapp-sansw-1b:1-9.126L100 A netapp-sansw-1a:1-10.126L22 B 3 21
netapp-sansw-1b:1-9.126L101 A netapp-sansw-1a:1-10.126L23 B 3 22
netapp-sansw-1b:1-9.126L102 A netapp-sansw-1a:1-10.126L24 B 3 23
netapp-sansw-1b:1-10.126L501 B netapp-sansw-1b:1-9.126L501 A 4 0
netapp-sansw-1a:1-9.126L502 A netapp-sansw-1a:1-10.126L502 B 4 1
netapp-sansw-1b:1-10.126L503 B netapp-sansw-1b:1-9.126L503 A 4 2
netapp-sansw-1b:1-10.126L504 B netapp-sansw-1b:1-9.126L504 A 4 3
netapp-sansw-1b:1-9.126L505 A netapp-sansw-1a:1-10.126L505 B 4 4
netapp-sansw-1a:1-9.126L506 A netapp-sansw-1a:1-10.126L506 B 4 5
netapp-sansw-1b:1-10.126L507 B netapp-sansw-1b:1-9.126L507 A 4 6
netapp-sansw-1b:1-10.126L508 B netapp-sansw-1b:1-9.126L508 A 4 7
netapp-sansw-1b:1-10.126L509 B netapp-sansw-1b:1-9.126L509 A 4 8
netapp-sansw-1b:1-9.126L510 A netapp-sansw-1a:1-10.126L510 B 4 9
netapp-sansw-1b:1-10.126L511 B netapp-sansw-1b:1-9.126L511 A 4 10
netapp-sansw-1b:1-10.126L512 B netapp-sansw-1b:1-9.126L512 A 4 11
netapp-sansw-1a:1-9.126L513 A netapp-sansw-1a:1-10.126L513 B 4 12
netapp-sansw-1b:1-10.126L514 B netapp-sansw-1b:1-9.126L514 A 4 13
netapp-sansw-1b:1-10.126L515 B netapp-sansw-1b:1-9.126L515 A 4 14
netapp-sansw-1a:1-9.126L516 A netapp-sansw-1a:1-10.126L516 B 4 15
netapp-sansw-1b:1-10.126L517 B netapp-sansw-1b:1-9.126L517 A 4 16
netapp-sansw-1b:1-10.126L518 B netapp-sansw-1b:1-9.126L518 A 4 17
netapp-sansw-1a:1-10.126L519 B netapp-sansw-1b:1-9.126L519 A 4 18
netapp-sansw-1b:1-10.126L520 B netapp-sansw-1b:1-9.126L520 A 4 19
netapp-sansw-1b:1-10.126L521 B netapp-sansw-1b:1-9.126L521 A 4 20
netapp-sansw-1a:1-10.126L522 B netapp-sansw-1b:1-9.126L522 A 4 21
netapp-sansw-1b:1-10.126L523 B netapp-sansw-1b:1-9.126L523 A 4 22
netapp-sansw-1b:1-10.126L524 B netapp-sansw-1b:1-9.126L524 A 4 23
netapp-sansw-2a:1-10.126L79 B netapp-sansw-2b:1-9.126L1 A 10 0
netapp-sansw-2a:1-10.126L80 B netapp-sansw-2b:1-9.126L2 A 10 1
netapp-sansw-2b:1-9.126L3 A netapp-sansw-2b:1-10.126L81 B 10 2
netapp-sansw-2a:1-9.126L4 A netapp-sansw-2b:1-10.126L82 B 10 3
netapp-sansw-2b:1-10.126L83 B netapp-sansw-2b:1-9.126L5 A 10 4
netapp-sansw-2b:1-10.126L84 B netapp-sansw-2b:1-9.126L6 A 10 5
netapp-sansw-2a:1-9.126L7 A netapp-sansw-2b:1-10.126L85 B 10 6
netapp-sansw-2b:1-10.126L86 B netapp-sansw-2b:1-9.126L8 A 10 7
netapp-sansw-2a:1-9.126L9 A netapp-sansw-2b:1-10.126L87 B 10 8
netapp-sansw-2b:1-9.126L10 A netapp-sansw-2b:1-10.126L88 B 10 9
netapp-sansw-2a:1-9.126L11 A netapp-sansw-2b:1-10.126L89 B 10 10
netapp-sansw-2b:1-9.126L12 A netapp-sansw-2b:1-10.126L90 B 10 11
netapp-sansw-2b:1-9.126L13 A netapp-sansw-2b:1-10.126L91 B 10 12
netapp-sansw-2a:1-9.126L14 A netapp-sansw-2b:1-10.126L92 B 10 13
netapp-sansw-2b:1-9.126L15 A netapp-sansw-2b:1-10.126L93 B 10 14
netapp-sansw-2b:1-9.126L16 A netapp-sansw-2b:1-10.126L94 B 10 15
netapp-sansw-2b:1-10.126L95 B netapp-sansw-2b:1-9.126L17 A 10 16
netapp-sansw-2b:1-10.126L96 B netapp-sansw-2b:1-9.126L18 A 10 17
netapp-sansw-2a:1-9.126L19 A netapp-sansw-2b:1-10.126L97 B 10 18
netapp-sansw-2a:1-10.126L98 B netapp-sansw-2b:1-9.126L20 A 10 19
netapp-sansw-2b:1-9.126L21 A netapp-sansw-2b:1-10.126L99 B 10 20
netapp-sansw-2a:1-10.126L100 B netapp-sansw-2b:1-9.126L22 A 10 21
netapp-sansw-2b:1-9.126L23 A netapp-sansw-2b:1-10.126L101 B 10 22
netapp-sansw-2a:1-10.126L102 B netapp-sansw-2b:1-9.126L24 A 10 23
netapp-sansw-2a:1-10.126L53 B netapp-sansw-2b:1-9.126L27 A 11 0
netapp-sansw-2a:1-10.126L54 B netapp-sansw-2b:1-9.126L28 A 11 1
netapp-sansw-2a:1-10.126L55 B netapp-sansw-2b:1-9.126L29 A 11 2
netapp-sansw-2b:1-9.126L30 A netapp-sansw-2b:1-10.126L56 B 11 3
netapp-sansw-2b:1-10.126L57 B netapp-sansw-2b:1-9.126L31 A 11 4
netapp-sansw-2a:1-10.126L58 B netapp-sansw-2b:1-9.126L32 A 11 5
netapp-sansw-2b:1-9.126L33 A netapp-sansw-2b:1-10.126L59 B 11 6
netapp-sansw-2b:1-10.126L60 B netapp-sansw-2b:1-9.126L34 A 11 7
netapp-sansw-2b:1-9.126L35 A netapp-sansw-2b:1-10.126L61 B 11 8
netapp-sansw-2b:1-9.126L36 A netapp-sansw-2b:1-10.126L62 B 11 9
netapp-sansw-2b:1-9.126L37 A netapp-sansw-2b:1-10.126L63 B 11 10
netapp-sansw-2b:1-9.126L38 A netapp-sansw-2b:1-10.126L64 B 11 11
netapp-sansw-2b:1-10.126L65 B netapp-sansw-2b:1-9.126L39 A 11 12
netapp-sansw-2b:1-9.126L40 A netapp-sansw-2b:1-10.126L66 B 11 13
netapp-sansw-2b:1-9.126L41 A netapp-sansw-2b:1-10.126L67 B 11 14
netapp-sansw-2b:1-9.126L42 A netapp-sansw-2b:1-10.126L68 B 11 15
netapp-sansw-2b:1-9.126L43 A netapp-sansw-2b:1-10.126L69 B 11 16
netapp-sansw-2b:1-10.126L70 B netapp-sansw-2b:1-9.126L44 A 11 17
netapp-sansw-2b:1-9.126L45 A netapp-sansw-2b:1-10.126L71 B 11 18
netapp-sansw-2a:1-9.126L46 A netapp-sansw-2b:1-10.126L72 B 11 19
netapp-sansw-2b:1-9.126L47 A netapp-sansw-2b:1-10.126L73 B 11 20
netapp-sansw-2b:1-9.126L48 A netapp-sansw-2b:1-10.126L74 B 11 21
netapp-sansw-2b:1-9.126L49 A netapp-sansw-2b:1-10.126L75 B 11 22
netapp-sansw-2a:1-9.126L50 A netapp-sansw-2b:1-10.126L76 B 11 23
netapp-sansw-2b:1-10.126L27 B netapp-sansw-2b:1-9.126L53 A 12 0
netapp-sansw-2b:1-9.126L54 A netapp-sansw-2b:1-10.126L28 B 12 1
netapp-sansw-2b:1-9.126L55 A netapp-sansw-2b:1-10.126L29 B 12 2
netapp-sansw-2b:1-10.126L30 B netapp-sansw-2b:1-9.126L56 A 12 3
netapp-sansw-2a:1-9.126L57 A netapp-sansw-2b:1-10.126L31 B 12 4
netapp-sansw-2b:1-9.126L58 A netapp-sansw-2b:1-10.126L32 B 12 5
netapp-sansw-2a:1-10.126L33 B netapp-sansw-2b:1-9.126L59 A 12 6
netapp-sansw-2a:1-9.126L60 A netapp-sansw-2b:1-10.126L34 B 12 7
netapp-sansw-2b:1-10.126L35 B netapp-sansw-2b:1-9.126L61 A 12 8
netapp-sansw-2b:1-9.126L62 A netapp-sansw-2b:1-10.126L36 B 12 9
netapp-sansw-2b:1-9.126L63 A netapp-sansw-2b:1-10.126L37 B 12 10
netapp-sansw-2a:1-10.126L38 B netapp-sansw-2b:1-9.126L64 A 12 11
netapp-sansw-2a:1-10.126L39 B netapp-sansw-2b:1-9.126L65 A 12 12
netapp-sansw-2a:1-10.126L40 B netapp-sansw-2b:1-9.126L66 A 12 13
netapp-sansw-2b:1-10.126L41 B netapp-sansw-2b:1-9.126L67 A 12 14
netapp-sansw-2b:1-10.126L42 B netapp-sansw-2b:1-9.126L68 A 12 15
netapp-sansw-2a:1-9.126L69 A netapp-sansw-2b:1-10.126L43 B 12 16
netapp-sansw-2b:1-10.126L44 B netapp-sansw-2b:1-9.126L70 A 12 17
netapp-sansw-2a:1-10.126L45 B netapp-sansw-2b:1-9.126L71 A 12 18
netapp-sansw-2a:1-10.126L46 B netapp-sansw-2b:1-9.126L72 A 12 19
netapp-sansw-2a:1-10.126L47 B netapp-sansw-2b:1-9.126L73 A 12 20
netapp-sansw-2b:1-9.126L74 A netapp-sansw-2b:1-10.126L48 B 12 21
netapp-sansw-2a:1-10.126L49 B netapp-sansw-2b:1-9.126L75 A 12 22
netapp-sansw-2b:1-10.126L50 B netapp-sansw-2b:1-9.126L76 A 12 23
netapp-sansw-2b:1-9.126L79 A netapp-sansw-2b:1-10.126L1 B 13 0
netapp-sansw-2b:1-9.126L80 A netapp-sansw-2b:1-10.126L2 B 13 1
netapp-sansw-2b:1-9.126L81 A netapp-sansw-2b:1-10.126L3 B 13 2
netapp-sansw-2b:1-10.126L4 B netapp-sansw-2b:1-9.126L82 A 13 3
netapp-sansw-2b:1-10.126L5 B netapp-sansw-2b:1-9.126L83 A 13 4
netapp-sansw-2b:1-9.126L84 A netapp-sansw-2b:1-10.126L6 B 13 5
netapp-sansw-2b:1-9.126L85 A netapp-sansw-2b:1-10.126L7 B 13 6
netapp-sansw-2b:1-9.126L86 A netapp-sansw-2b:1-10.126L8 B 13 7
netapp-sansw-2a:1-9.126L87 A netapp-sansw-2b:1-10.126L9 B 13 8
netapp-sansw-2b:1-9.126L88 A netapp-sansw-2b:1-10.126L10 B 13 9
netapp-sansw-2a:1-9.126L89 A netapp-sansw-2b:1-10.126L11 B 13 10
netapp-sansw-2b:1-9.126L90 A netapp-sansw-2b:1-10.126L12 B 13 11
netapp-sansw-2b:1-9.126L91 A netapp-sansw-2b:1-10.126L13 B 13 12
netapp-sansw-2b:1-9.126L92 A netapp-sansw-2b:1-10.126L14 B 13 13
netapp-sansw-2a:1-9.126L93 A netapp-sansw-2b:1-10.126L15 B 13 14
netapp-sansw-2b:1-9.126L94 A netapp-sansw-2b:1-10.126L16 B 13 15
netapp-sansw-2b:1-9.126L95 A netapp-sansw-2b:1-10.126L17 B 13 16
netapp-sansw-2b:1-9.126L96 A netapp-sansw-2b:1-10.126L18 B 13 17
netapp-sansw-2a:1-9.126L97 A netapp-sansw-2b:1-10.126L19 B 13 18
netapp-sansw-2b:1-9.126L98 A netapp-sansw-2b:1-10.126L20 B 13 19
netapp-sansw-2a:1-9.126L99 A netapp-sansw-2b:1-10.126L21 B 13 20
netapp-sansw-2a:1-9.126L100 A netapp-sansw-2b:1-10.126L22 B 13 21
netapp-sansw-2a:1-10.126L23 B netapp-sansw-2b:1-9.126L101 A 13 22
netapp-sansw-2b:1-9.126L102 A netapp-sansw-2b:1-10.126L24 B 13 23
netapp-sansw-2b:1-9.126L501 A netapp-sansw-2b:1-10.126L501 B 14 0
netapp-sansw-2b:1-9.126L502 A netapp-sansw-2b:1-10.126L502 B 14 1
netapp-sansw-2a:1-10.126L503 B netapp-sansw-2b:1-9.126L503 A 14 2
netapp-sansw-2b:1-10.126L504 B netapp-sansw-2b:1-9.126L504 A 14 3
netapp-sansw-2b:1-10.126L505 B netapp-sansw-2b:1-9.126L505 A 14 4
netapp-sansw-2b:1-9.126L506 A netapp-sansw-2b:1-10.126L506 B 14 5
netapp-sansw-2b:1-9.126L507 A netapp-sansw-2b:1-10.126L507 B 14 6
netapp-sansw-2b:1-9.126L508 A netapp-sansw-2b:1-10.126L508 B 14 7
netapp-sansw-2b:1-9.126L509 A netapp-sansw-2b:1-10.126L509 B 14 8
netapp-sansw-2b:1-9.126L510 A netapp-sansw-2b:1-10.126L510 B 14 9
netapp-sansw-2a:1-10.126L511 B netapp-sansw-2b:1-9.126L511 A 14 10
netapp-sansw-2a:1-10.126L512 B netapp-sansw-2b:1-9.126L512 A 14 11
netapp-sansw-2b:1-10.126L513 B netapp-sansw-2b:1-9.126L513 A 14 12
netapp-sansw-2b:1-10.126L514 B netapp-sansw-2b:1-9.126L514 A 14 13
netapp-sansw-2b:1-9.126L515 A netapp-sansw-2b:1-10.126L515 B 14 14
netapp-sansw-2b:1-9.126L516 A netapp-sansw-2b:1-10.126L516 B 14 15
netapp-sansw-2b:1-9.126L517 A netapp-sansw-2b:1-10.126L517 B 14 16
netapp-sansw-2b:1-9.126L518 A netapp-sansw-2b:1-10.126L518 B 14 17
netapp-sansw-2b:1-9.126L519 A netapp-sansw-2b:1-10.126L519 B 14 18
netapp-sansw-2b:1-9.126L520 A netapp-sansw-2b:1-10.126L520 B 14 19
netapp-sansw-2a:1-10.126L521 B netapp-sansw-2b:1-9.126L521 A 14 20
netapp-sansw-2a:1-9.126L522 A netapp-sansw-2b:1-10.126L522 B 14 21
netapp-sansw-2b:1-9.126L523 A netapp-sansw-2b:1-10.126L523 B 14 22
netapp-sansw-2a:1-9.126L524 A netapp-sansw-2b:1-10.126L524 B 14 23
cluster_a-01> exit
logout
cluster_a::> storage disk show -fields primary-port,secondary-port
disk primary-port secondary-port
------- ------------ --------------
1.4.0 A A
1.4.1 B A
1.4.2 B A
1.4.3 B A
1.4.4 B A
1.4.5 A A
1.4.6 A A
1.4.7 B A
1.4.8 B A
1.4.9 A A
1.4.10 A A
1.4.11 B A
1.4.12 A A
1.4.13 A A
1.4.14 B A
1.4.15 B A
1.4.16 B A
1.4.17 A A
1.4.18 B A
1.4.19 A A
1.4.20 B A
1.4.21 B A
1.4.22 A A
1.4.23 A A
2.0.0 A A
2.0.1 A A
2.0.2 A A
2.0.3 B A
2.0.4 B A
2.0.5 B A
2.0.6 B A
2.0.7 B A
2.0.8 A A
2.0.9 A A
2.0.10 A A
2.0.11 B A
2.0.12 B A
2.0.13 A A
2.0.14 A A
2.0.15 B A
2.0.16 A A
2.0.17 A A
2.0.18 B A
2.0.19 B A
2.0.20 B A
2.0.21 A A
2.0.22 B A
2.0.23 A A
2.1.0 B A
2.1.1 B A
2.1.2 B A
2.1.3 B A
2.1.4 B A
2.1.5 A A
2.1.6 B A
2.1.7 B A
2.1.8 A A
2.1.9 B A
2.1.10 A A
2.1.11 A A
2.1.12 B A
2.1.13 B A
2.1.14 A A
2.1.15 A A
2.1.16 A A
2.1.17 B A
2.1.18 B A
2.1.19 B A
2.1.20 A A
2.1.21 A A
2.1.22 A A
2.1.23 A A
2.2.0 A A
2.2.1 A A
2.2.2 A A
2.2.3 B A
disk primary-port secondary-port
------- ------------ --------------
2.2.4 A A
2.2.5 A A
2.2.6 A A
2.2.7 B A
2.2.8 A A
2.2.9 A A
2.2.10 A A
2.2.11 B A
2.2.12 A A
2.2.13 A A
2.2.14 A A
2.2.15 A A
2.2.16 B A
2.2.17 A A
2.2.18 A A
2.2.19 B A
2.2.20 B A
2.2.21 A A
2.2.22 B A
2.2.23 B A
2.3.0 B A
2.3.1 A A
2.3.2 A A
2.3.3 A A
2.3.4 A A
2.3.5 B A
2.3.6 A A
2.3.7 A A
2.3.8 A A
2.3.9 A A
2.3.10 A A
2.3.11 A A
2.3.12 A A
2.3.13 A A
2.3.14 A A
2.3.15 A A
2.3.16 A A
2.3.17 A A
2.3.18 A A
2.3.19 A A
2.3.20 A A
2.3.21 A A
2.3.22 A A
2.3.23 A A
3.10.0 B B
3.10.1 B B
3.10.2 A B
3.10.3 A B
3.10.4 B B
3.10.5 B B
3.10.6 A B
3.10.7 B B
3.10.8 A B
3.10.9 A B
3.10.10 A B
3.10.11 A B
3.10.12 A B
3.10.13 A B
3.10.14 A B
3.10.15 A B
3.10.16 B B
3.10.17 B B
3.10.18 A B
3.10.19 B B
3.10.20 A B
3.10.21 B B
3.10.22 A B
3.10.23 B B
3.11.0 B B
3.11.1 B B
3.11.2 B B
3.11.3 A B
3.11.4 B B
3.11.5 B B
3.11.6 A B
disk primary-port secondary-port
------- ------------ --------------
3.11.7 A B
3.11.8 A B
3.11.9 A B
3.11.10 A B
3.11.11 A B
3.11.12 A B
3.11.13 A B
3.11.14 A B
3.11.15 A B
3.11.16 A B
3.11.17 A B
3.11.18 A B
3.11.19 A B
3.11.20 B B
3.11.21 B B
3.11.22 B B
3.11.23 A B
3.12.0 B B
3.12.1 B B
3.12.2 B B
3.12.3 B B
3.12.4 B B
3.12.5 B B
3.12.6 B B
3.12.7 B B
3.12.8 B B
3.12.9 B B
3.12.10 A B
3.12.11 A B
3.12.12 A B
3.12.13 A B
3.12.14 A B
3.12.15 B B
3.12.16 B B
3.12.17 B B
3.12.18 B B
3.12.19 A B
3.12.20 B B
3.12.21 B B
3.12.22 B B
3.12.23 A B
3.13.0 A B
3.13.1 A B
3.13.2 A B
3.13.3 B B
3.13.4 B B
3.13.5 B B
3.13.6 B B
3.13.7 B B
3.13.8 B B
3.13.9 B B
3.13.10 B B
3.13.11 B B
3.13.12 B B
3.13.13 B B
3.13.14 B B
3.13.15 B B
3.13.16 B B
3.13.17 B B
3.13.18 B B
3.13.19 B B
3.13.20 B B
3.13.21 B B
3.13.22 A B
3.13.23 B B
4.14.0 A B
4.14.1 B B
4.14.2 B B
4.14.3 A B
4.14.4 A B
4.14.5 A B
4.14.6 A B
4.14.7 A B
4.14.8 A B
4.14.9 B B
disk primary-port secondary-port
------- ------------ --------------
4.14.10 A B
4.14.11 B B
4.14.12 A B
4.14.13 B B
4.14.14 A B
4.14.15 B B
4.14.16 A B
4.14.17 A B
4.14.18 A B
4.14.19 A B
4.14.20 A B
4.14.21 B B
4.14.22 A B
4.14.23 B B
240 entries were displayed.
cluster_a::>
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Interesting, as I see.
Cluster level command is reporting incorrectly
cluster_a::> storage disk show -fields primary-port,secondary-port
Shelf 4, Bay:5 & 6 are showing both as A.
1.4.5 A A
1.4.6 A A
Node level command is reporting correctly (or is it)
cluster_a-01> storage show disk -p
Shelf 4, Bay:5 & 6 are showing both showing dual-paths
netapp-sansw-1a:1-9.126L506 A netapp-sansw-1a:1-10.126L506 B 4 5
netapp-sansw-1b:1-10.126L507 B netapp-sansw-1b:1-9.126L507 A 4 6
Something I just noticed:
In the node-level command output: Normally on my system (Non-Metro) I see disks in more simple way such as:
PRIMARY PORT SECONDARY PORT SHELF BAY
0d.04.5 B 0a.04.5 A 4 5
0d.04.6 B 0a.04.6 A 4 6
On your system it is bit different, but I pick the exact separate adapter, then it shows same path.
PRIMARY PORT SECONDARY PORT SHELF BAY
netapp-sansw-1a:1-9.126L506 A netapp-sansw-1b:1-9.126L507 A 46
netapp-sansw-1a:1-10.126L506 B netapp-sansw-1b:1-10.126L507 B 45
Yes, it can get confusing, therefore it is worth verifying to this BUG:472649
Could you verify if this is the same SAS Model: X412_S15K7560A15
😆 storage disk show -disk 1.4.5
If it's not the same Model, then let NetApp investigate.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Looks like it's not the drive, from the affected by the bug, so i'm waiting for the support's answer:
cluster_a::> storage disk show -disk 1.4.5
Disk: 1.4.5
Container Type: unassigned
Owner/Home: - / -
DR Home: -
Stack ID/Shelf/Bay: 1 / 4 / 5
LUN: 506
Array: ATTO_FibreBridge7500N_1
Vendor: NETAPP
Model: X371_S1643960ATE
Serial Number: S3SENF0KC01188
UID: 5002538B:48C05440:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000
BPS: 520
Physical Size: 894.3GB
Position: present
Checksum Compatibility: block
Aggregate: -
Plex: -
Paths:
LUN Initiator Side Target Side Link
Controller Initiator ID Switch Port Switch Port Acc Use Target Port TPGN Speed I/O KB/s IOPS
------------------ --------- ----- -------------------- -------------------- --- --- ----------------------- ------ ------- ------------ ------------
cluster_a-02 1c 506 netapp-sansw-1a:1-8 netapp-sansw-1a:1-10 AO RDY 2100001086a9c800 87 16 Gb/S 0 0
cluster_a-02 1d 506 netapp-sansw-1b:1-8 netapp-sansw-1b:1-10 AO RDY 2200001086a9c800 88 16 Gb/S 0 0
cluster_a-02 1b 506 netapp-sansw-1b:1-7 netapp-sansw-1b:1-9 AO INU 2200001086a9d700 67 16 Gb/S 0 0
cluster_a-02 1a 506 netapp-sansw-1a:1-7 netapp-sansw-1a:1-9 AO RDY 2100001086a9d700 66 16 Gb/S 0 0
cluster_a-01 1c 506 netapp-sansw-1a:1-4 netapp-sansw-1a:1-10 AO RDY 2100001086a9c800 87 16 Gb/S 0 0
cluster_a-01 1d 506 netapp-sansw-1b:1-4 netapp-sansw-1b:1-10 AO RDY 2200001086a9c800 88 16 Gb/S 0 0
cluster_a-01 1b 506 netapp-sansw-1b:1-3 netapp-sansw-1b:1-9 AO RDY 2200001086a9d700 67 16 Gb/S 0 0
cluster_a-01 1a 506 netapp-sansw-1a:1-3 netapp-sansw-1a:1-9 AO INU 2100001086a9d700 66 16 Gb/S 0 0
Errors:
-
cluster_a::>
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I've got the answer from support that the command "storage disk show" doesn’t check the multipathing in the MetroCluster. So, looks like if sysconfig shows us that we have quad-path for the metrocluster - it should be enough.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for the update. Interesting!!
Hope it helps another customer who discovers the same issue. I guess NetApp kb stating the same is desirable.
Public