FAS and V-Series Storage Systems Discussions

Understanding disk paths

Hi, guys!

 We have 4-node fas8200 metrocluster, ontap 9.6p5. I'm trying understand how to interpret the output of the "storage disk show" command. I did the following:

cluster_a::> storage disk show -primary-port B -secondary-port B
                     Usable           Disk    Container   Container   
Disk                   Size Shelf Bay Type    Type        Name      Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
3.10.1               1.63TB    10   1 SAS     aggregate   aggr1_a   cluster_a-01
3.10.2               1.63TB    10   2 SAS     aggregate   aggr1_a   cluster_a-01
3.10.4               1.63TB    10   4 SAS     aggregate   aggr0_cluster_a_01 
                                                                    cluster_a-01
3.10.5               1.63TB    10   5 SAS     aggregate   aggr1_a   cluster_a-01
3.10.8               1.63TB    10   8 SAS     aggregate   aggr1_a   cluster_a-01
3.10.10              1.63TB    10  10 SAS     aggregate   aggr1_a   cluster_a-01
3.10.11              1.63TB    10  11 SAS     aggregate   aggr1_a   cluster_a-01
3.10.15              1.63TB    10  15 SAS     aggregate   aggr1_a   cluster_a-01
3.10.16              1.63TB    10  16 SAS     aggregate   aggr1_a   cluster_a-01
3.10.18              1.63TB    10  18 SAS     aggregate   aggr1_a   cluster_a-01
3.10.20              1.63TB    10  20 SAS     aggregate   aggr1_a   cluster_a-01
3.10.22              1.63TB    10  22 SAS     aggregate   aggr1_a   cluster_a-01
3.10.23              1.63TB    10  23 SAS     spare       Pool1     cluster_a-01
3.11.0               1.63TB    11   0 SAS     aggregate   aggr0_cluster_a_02 
                                                                    cluster_a-02
3.11.2               1.63TB    11   2 SAS     aggregate   aggr0_cluster_a_02 
                                                                    cluster_a-02
3.11.3               1.63TB    11   3 SAS     aggregate   aggr2_a   cluster_a-02
3.11.7               1.63TB    11   7 SAS     aggregate   aggr2_a   cluster_a-02
3.11.8               1.63TB    11   8 SAS     aggregate   aggr2_a   cluster_a-02
3.11.9               1.63TB    11   9 SAS     aggregate   aggr2_a   cluster_a-02
3.11.11              1.63TB    11  11 SAS     aggregate   aggr2_a   cluster_a-02
3.11.12              1.63TB    11  12 SAS     aggregate   aggr2_a   cluster_a-02
3.11.18              1.63TB    11  18 SAS     aggregate   aggr2_a   cluster_a-02
3.11.20              1.63TB    11  20 SAS     aggregate   aggr2_a   cluster_a-02
3.11.21              1.63TB    11  21 SAS     aggregate   aggr2_a   cluster_a-02
3.12.0                    -    12   0 SAS     remote      -         cluster_b-01
3.12.1                    -    12   1 SAS     remote      -         cluster_b-01
3.12.2                    -    12   2 SAS     remote      -         cluster_b-01
3.12.8                    -    12   8 SAS     remote      -         cluster_b-01
3.12.12                   -    12  12 SAS     remote      -         cluster_b-01
3.12.14                   -    12  14 SAS     remote      -         cluster_b-01
3.12.18                   -    12  18 SAS     remote      -         cluster_b-01
3.12.20                   -    12  20 SAS     remote      -         cluster_b-01
3.12.22                   -    12  22 SAS     remote      -         cluster_b-01
3.13.3                    -    13   3 SAS     remote      -         cluster_b-02
3.13.6                    -    13   6 SAS     remote      -         cluster_b-02
3.13.7                    -    13   7 SAS     remote      -         cluster_b-02
3.13.8                    -    13   8 SAS     remote      -         cluster_b-02
3.13.10                   -    13  10 SAS     remote      -         cluster_b-02
3.13.11                   -    13  11 SAS     remote      -         cluster_b-02
3.13.12                   -    13  12 SAS     remote      -         cluster_b-02
3.13.14                   -    13  14 SAS     remote      -         cluster_b-02
3.13.15                   -    13  15 SAS     remote      -         cluster_b-02
3.13.16                   -    13  16 SAS     remote      -         cluster_b-02
3.13.17                   -    13  17 SAS     remote      -         cluster_b-02
3.13.18                   -    13  18 SAS     remote      -         cluster_b-02
3.13.19                   -    13  19 SAS     remote      -         cluster_b-02
3.13.20                   -    13  20 SAS     remote      -         cluster_b-02
3.13.21                   -    13  21 SAS     remote      -         cluster_b-02
3.13.23                   -    13  23 SAS     remote      -         cluster_b-02
                                                                               
                     Usable           Disk    Container   Container
Disk                   Size Shelf Bay Type    Type        Name      Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
4.14.0                    -    14   0 SSD     remote      -         cluster_b-02
4.14.1                    -    14   1 SSD     remote      -         cluster_b-02
4.14.2                    -    14   2 SSD     remote      -         cluster_b-02
4.14.4                    -    14   4 SSD     remote      -         cluster_b-02
4.14.6                    -    14   6 SSD     remote      -         cluster_b-02
4.14.9                    -    14   9 SSD     remote      -         cluster_b-02
4.14.16                   -    14  16 SSD     remote      -         cluster_b-02
4.14.19                   -    14  19 SSD     remote      -         cluster_b-02
4.14.20                   -    14  20 SSD     remote      -         cluster_b-02
4.14.22                   -    14  22 SSD     remote      -         cluster_b-02
4.14.23                   -    14  23 SSD     remote      -         cluster_b-02
60 entries were displayed.

cluster_a::> storage disk show -primary-port A -secondary-port A
                     Usable           Disk    Container   Container   
Disk                   Size Shelf Bay Type    Type        Name      Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
1.4.0                     -     4   0 SSD     unassigned  -         -
1.4.2                     -     4   2 SSD     unassigned  -         -
1.4.3                     -     4   3 SSD     unassigned  -         -
1.4.4                     -     4   4 SSD     unassigned  -         -
1.4.5                     -     4   5 SSD     unassigned  -         -
1.4.6                     -     4   6 SSD     unassigned  -         -
1.4.7                     -     4   7 SSD     unassigned  -         -
1.4.9                     -     4   9 SSD     unassigned  -         -
1.4.10                    -     4  10 SSD     unassigned  -         -
1.4.11                    -     4  11 SSD     unassigned  -         -
1.4.12                    -     4  12 SSD     unassigned  -         -
1.4.14                    -     4  14 SSD     unassigned  -         -
1.4.15                    -     4  15 SSD     unassigned  -         -
1.4.18                    -     4  18 SSD     unassigned  -         -
1.4.19                    -     4  19 SSD     unassigned  -         -
1.4.23                    -     4  23 SSD     unassigned  -         -
2.0.1                1.63TB     0   1 SAS     aggregate   aggr0_cluster_a_01 
                                                                    cluster_a-01
2.0.2                1.63TB     0   2 SAS     aggregate   aggr0_cluster_a_01 
                                                                    cluster_a-01
2.0.3                1.63TB     0   3 SAS     aggregate   aggr1_a   cluster_a-01
2.0.7                1.63TB     0   7 SAS     aggregate   aggr1_a   cluster_a-01
2.0.8                1.63TB     0   8 SAS     aggregate   aggr1_a   cluster_a-01
2.0.13               1.63TB     0  13 SAS     aggregate   aggr1_a   cluster_a-01
2.0.14               1.63TB     0  14 SAS     aggregate   aggr1_a   cluster_a-01
2.0.18               1.63TB     0  18 SAS     aggregate   aggr1_a   cluster_a-01
2.0.19               1.63TB     0  19 SAS     aggregate   aggr1_a   cluster_a-01
2.0.20               1.63TB     0  20 SAS     aggregate   aggr1_a   cluster_a-01
2.0.21               1.63TB     0  21 SAS     aggregate   aggr1_a   cluster_a-01
2.0.23               1.63TB     0  23 SAS     spare       Pool0     cluster_a-01
2.1.0                1.63TB     1   0 SAS     aggregate   aggr0_cluster_a_02 
                                                                    cluster_a-02
2.1.1                1.63TB     1   1 SAS     aggregate   aggr0_cluster_a_02 
                                                                    cluster_a-02
2.1.6                1.63TB     1   6 SAS     aggregate   aggr2_a   cluster_a-02
2.1.7                1.63TB     1   7 SAS     aggregate   aggr2_a   cluster_a-02
2.1.9                1.63TB     1   9 SAS     aggregate   aggr2_a   cluster_a-02
2.1.11               1.63TB     1  11 SAS     aggregate   aggr2_a   cluster_a-02
2.1.12               1.63TB     1  12 SAS     aggregate   aggr2_a   cluster_a-02
2.1.14               1.63TB     1  14 SAS     aggregate   aggr2_a   cluster_a-02
2.1.20               1.63TB     1  20 SAS     aggregate   aggr2_a   cluster_a-02
2.1.21               1.63TB     1  21 SAS     aggregate   aggr2_a   cluster_a-02
2.1.22               1.63TB     1  22 SAS     aggregate   aggr2_a   cluster_a-02
2.1.23               1.63TB     1  23 SAS     spare       Pool0     cluster_a-02
2.2.0                     -     2   0 SAS     remote      -         cluster_b-01
2.2.2                     -     2   2 SAS     remote      -         cluster_b-01
2.2.3                     -     2   3 SAS     remote      -         cluster_b-01
2.2.4                     -     2   4 SAS     remote      -         cluster_b-01
2.2.6                     -     2   6 SAS     remote      -         cluster_b-01
2.2.8                     -     2   8 SAS     remote      -         cluster_b-01
2.2.10                    -     2  10 SAS     remote      -         cluster_b-01
2.2.12                    -     2  12 SAS     remote      -         cluster_b-01
                                                                               
                     Usable           Disk    Container   Container
Disk                   Size Shelf Bay Type    Type        Name      Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
2.2.13                    -     2  13 SAS     remote      -         cluster_b-01
2.2.15                    -     2  15 SAS     remote      -         cluster_b-01
2.2.16                    -     2  16 SAS     remote      -         cluster_b-01
2.2.18                    -     2  18 SAS     remote      -         cluster_b-01
2.2.19                    -     2  19 SAS     remote      -         cluster_b-01
2.2.21                    -     2  21 SAS     remote      -         cluster_b-01
2.2.22                    -     2  22 SAS     remote      -         cluster_b-01
2.2.23                    -     2  23 SAS     remote      -         cluster_b-01
2.3.21                    -     3  21 SAS     remote      -         cluster_b-02
57 entries were displayed.

So, as you can see, according this output, i have 60 disks with both B paths as primary and secondary and 57 disks with both A paths as primary and secondary. Does that mean that the system is not redundant? I thought that all the disks should have A and B paths as primary and secondary, but if the "metrocluster check show" shows that everything is ok, my assumption is wrong. So, could you give me any comments about the output of the "storage disk show"?

16 REPLIES 16

Re: Understanding disk paths

can you provide the output of this:

 

::> storage disk show -fields secondary-port,primary-port,initiator,nodelist

Re: Understanding disk paths

Here:

 

cluster_a::> storage disk show -fields secondary-port,primary-port,initiator,nodelist
disk    initiator               nodelist                                                                                                primary-port secondary-port 
------- ----------------------- ------------------------------------------------------------------------------------------------------- ------------ -------------- 
1.4.0   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.1   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
1.4.2   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.3   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.4   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.5   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.6   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.7   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.8   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.9   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.10  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.11  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.12  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.13  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
1.4.14  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.15  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.16  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.17  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.18  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.19  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.20  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
1.4.21  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.22  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
1.4.23  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.0.0   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            A              
2.0.1   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.2   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            A              
2.0.3   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            A              
2.0.4   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            A              
2.0.5   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.6   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            A              
2.0.7   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.8   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.9   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.10  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            A              
2.0.11  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.12  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.13  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.14  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.15  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.16  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.17  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.18  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.19  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            A              
2.0.20  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.21  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            A              
2.0.22  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            A              
2.0.23  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            A              
2.1.0   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.1   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.2   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.3   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.4   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.5   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.6   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.7   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.8   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.9   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.10  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.11  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.12  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.13  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.14  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.15  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.16  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.17  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.18  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.1.19  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.20  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.21  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.22  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.1.23  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.0   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.1   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.2   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.3   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.4   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.5   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.2.6   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.7   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.2.8   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.9   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.2.10  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.11  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.2.12  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.13  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.14  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.2.15  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.16  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.17  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.2.18  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.19  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.20  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.2.21  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.22  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.2.23  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.3.0   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.1   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.2   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.3   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.4   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.5   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.6   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.7   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.8   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.9   1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.10  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.11  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.12  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.13  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.14  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.15  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.16  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.17  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.18  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.19  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.20  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.21  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            A              
2.3.22  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
2.3.23  1c,1d,1a,1b,1c,1d,1a,1b cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            A              
3.10.0  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.1  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.2  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.3  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.4  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.5  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.6  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.7  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.8  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.9  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.11 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 A            B              
3.10.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.10.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02 B            B              
3.11.0  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.11.1  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.2  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.11.3  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.11.4  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.11.5  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.11.6  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.7  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.8  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.9  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.11.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.11 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.11.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.11.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.0  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.12.1  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.12.2  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.12.3  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.4  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.5  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.6  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.7  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.8  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.12.9  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.11 1a,1b,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.12.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.12.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.12.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.12.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.12.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.12.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.13.0  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.13.1  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.13.2  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.13.3  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.4  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.13.5  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.13.6  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.7  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.8  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.9  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.13.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.11 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.13.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
3.13.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
3.13.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.0  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.1  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.2  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.3  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.4  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.5  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.6  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.7  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.8  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.9  1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.10 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.11 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.12 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.13 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.14 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.15 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.16 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.17 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.18 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.19 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.20 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.21 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 A            B              
4.14.22 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
4.14.23 1b,1a,1d,1c,1b,1a,1d,1c cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-02,cluster_a-01,cluster_a-01,cluster_a-01,cluster_a-01 B            B              
240 entries were displayed.

cluster_a::> 

Re: Understanding disk paths

Something don't look alright in the output you have shared (Just from regular HA perspective). Probably, metrocluster check show command doesn't even meant to check the disk paths and therefore says ok.

 

Using one example:

For Shelf 4, only BAYS:1,13,20 shows dual-paths [Pri:A, Sec:B], rest all the Bays are showing Prim & Sec as 'A'. Could you ask someone to take a physical inspection of shelf, and ensure the cabling is done correctly ?

 

Could you simply run this command and share the output.
::> storage disk show -port

 

Thanks!

Re: Understanding disk paths

Hi!

I agree with you that something is wrong here, but i just don't understand, what. Let me explain: there're two stacks, on the each side, one with four shelfs with hdd drives, second one with single ssd shelf. I checked the cabling and it's done correctly, just exactly as in the metrocluster installation documentation (there's  an example with two stacks - just as here).

I've already tried to find some information from the "storage disk show -port", but it looks abcolutely illogical for me. In the output, as you can see, the ssds, which are located in one shelf (well, I hope, that they are, according netapp shelf partnumbers Smiley Happy ), some have A-A paths, some B-A *(disks 1.4.0-1.4.23) and the same is for the second shelf with ssd drives: some have A-B and some have B-B (4.14.0-4.14.23). Just in theory, how this could happen?

 

Here's the output you asked:

 

cluster_a::> storage disk show -port
Primary         Port Secondary       Port Type   Shelf Bay            
--------------- ---- --------------- ---- ------ ----- ---
1.4.0           A    cluster_a-01:netapp-sansw-1b:1-9.126L501 A SSD 4 0
1.4.1           A    cluster_a-01:netapp-sansw-1a:1-9.126L502 A SSD 4 1
1.4.2           B    cluster_a-01:netapp-sansw-1b:1-9.126L503 A SSD 4 2
1.4.3           B    cluster_a-01:netapp-sansw-1b:1-9.126L504 A SSD 4 3
1.4.4           B    cluster_a-01:netapp-sansw-1b:1-9.126L505 A SSD 4 4
1.4.5           B    cluster_a-01:netapp-sansw-1b:1-9.126L506 A SSD 4 5
1.4.6           B    cluster_a-01:netapp-sansw-1b:1-9.126L507 A SSD 4 6
1.4.7           B    cluster_a-01:netapp-sansw-1b:1-9.126L508 A SSD 4 7
1.4.8           A    cluster_a-01:netapp-sansw-1b:1-9.126L509 A SSD 4 8
1.4.9           A    cluster_a-01:netapp-sansw-1a:1-9.126L510 A SSD 4 9
1.4.10          A    cluster_a-01:netapp-sansw-1a:1-9.126L511 A SSD 4 10
1.4.11          B    cluster_a-01:netapp-sansw-1b:1-9.126L512 A SSD 4 11
1.4.12          B    cluster_a-01:netapp-sansw-1b:1-9.126L513 A SSD 4 12
1.4.13          A    cluster_a-01:netapp-sansw-1a:1-9.126L514 A SSD 4 13
1.4.14          A    cluster_a-01:netapp-sansw-1a:1-9.126L515 A SSD 4 14
1.4.15          A    cluster_a-01:netapp-sansw-1a:1-9.126L516 A SSD 4 15
1.4.16          B    cluster_a-01:netapp-sansw-1b:1-9.126L517 A SSD 4 16
1.4.17          A    cluster_a-01:netapp-sansw-1a:1-9.126L518 A SSD 4 17
1.4.18          B    cluster_a-01:netapp-sansw-1b:1-9.126L519 A SSD 4 18
1.4.19          A    cluster_a-01:netapp-sansw-1a:1-9.126L520 A SSD 4 19
1.4.20          A    cluster_a-01:netapp-sansw-1a:1-9.126L521 A SSD 4 20
1.4.21          A    cluster_a-01:netapp-sansw-1a:1-9.126L522 A SSD 4 21
1.4.22          A    cluster_a-01:netapp-sansw-1a:1-9.126L523 A SSD 4 22
1.4.23          B    cluster_a-01:netapp-sansw-1b:1-9.126L524 A SSD 4 23
2.0.0           A    cluster_a-01:netapp-sansw-1b:1-9.126L1 A SAS 0 0
2.0.1           B    cluster_a-01:netapp-sansw-1b:1-9.126L2 A SAS 0 1
2.0.2           A    cluster_a-01:netapp-sansw-1a:1-9.126L3 A SAS 0 2
2.0.3           B    cluster_a-01:netapp-sansw-1b:1-9.126L4 A SAS 0 3
2.0.4           A    cluster_a-01:netapp-sansw-1a:1-9.126L5 A SAS 0 4
2.0.5           B    cluster_a-01:netapp-sansw-1b:1-9.126L6 A SAS 0 5
2.0.6           B    cluster_a-01:netapp-sansw-1b:1-9.126L7 A SAS 0 6
2.0.7           B    cluster_a-01:netapp-sansw-1b:1-9.126L8 A SAS 0 7
2.0.8           A    cluster_a-01:netapp-sansw-1a:1-9.126L9 A SAS 0 8
2.0.9           A    cluster_a-01:netapp-sansw-1b:1-9.126L10 A SAS 0 9
2.0.10          B    cluster_a-01:netapp-sansw-1b:1-9.126L11 A SAS 0 10
2.0.11          A    cluster_a-01:netapp-sansw-1b:1-9.126L12 A SAS 0 11
2.0.12          A    cluster_a-01:netapp-sansw-1b:1-9.126L13 A SAS 0 12
2.0.13          B    cluster_a-01:netapp-sansw-1b:1-9.126L14 A SAS 0 13
2.0.14          A    cluster_a-01:netapp-sansw-1b:1-9.126L15 A SAS 0 14
2.0.15          A    cluster_a-01:netapp-sansw-1a:1-9.126L16 A SAS 0 15
2.0.16          A    cluster_a-01:netapp-sansw-1b:1-9.126L17 A SAS 0 16
2.0.17          B    cluster_a-01:netapp-sansw-1b:1-9.126L18 A SAS 0 17
2.0.18          B    cluster_a-01:netapp-sansw-1b:1-9.126L19 A SAS 0 18
2.0.19          B    cluster_a-01:netapp-sansw-1b:1-9.126L20 A SAS 0 19
2.0.20          B    cluster_a-01:netapp-sansw-1b:1-9.126L21 A SAS 0 20
2.0.21          A    cluster_a-01:netapp-sansw-1a:1-9.126L22 A SAS 0 21
2.0.22          A    cluster_a-01:netapp-sansw-1a:1-9.126L23 A SAS 0 22
2.0.23          B    cluster_a-01:netapp-sansw-1b:1-9.126L24 A SAS 0 23
2.1.0           B    cluster_a-02:netapp-sansw-1b:1-9.126L27 A SAS 1 0
2.1.1           B    cluster_a-02:netapp-sansw-1b:1-9.126L28 A SAS 1 1
2.1.2           B    cluster_a-02:netapp-sansw-1b:1-9.126L29 A SAS 1 2
2.1.3           A    cluster_a-02:netapp-sansw-1b:1-9.126L30 A SAS 1 3
2.1.4           B    cluster_a-02:netapp-sansw-1b:1-9.126L31 A SAS 1 4
2.1.5           A    cluster_a-02:netapp-sansw-1a:1-9.126L32 A SAS 1 5
2.1.6           A    cluster_a-02:netapp-sansw-1b:1-9.126L33 A SAS 1 6
2.1.7           B    cluster_a-02:netapp-sansw-1b:1-9.126L34 A SAS 1 7
2.1.8           B    cluster_a-02:netapp-sansw-1b:1-9.126L35 A SAS 1 8
2.1.9           A    cluster_a-02:netapp-sansw-1b:1-9.126L36 A SAS 1 9
2.1.10          B    cluster_a-02:netapp-sansw-1b:1-9.126L37 A SAS 1 10
2.1.11          A    cluster_a-02:netapp-sansw-1a:1-9.126L38 A SAS 1 11
2.1.12          B    cluster_a-02:netapp-sansw-1b:1-9.126L39 A SAS 1 12
2.1.13          A    cluster_a-02:netapp-sansw-1b:1-9.126L40 A SAS 1 13
2.1.14          A    cluster_a-02:netapp-sansw-1a:1-9.126L41 A SAS 1 14
2.1.15          A    cluster_a-02:netapp-sansw-1b:1-9.126L42 A SAS 1 15
2.1.16          B    cluster_a-02:netapp-sansw-1b:1-9.126L43 A SAS 1 16
2.1.17          B    cluster_a-02:netapp-sansw-1b:1-9.126L44 A SAS 1 17
2.1.18          A    cluster_a-02:netapp-sansw-1b:1-9.126L45 A SAS 1 18
2.1.19          A    cluster_a-02:netapp-sansw-1a:1-9.126L46 A SAS 1 19
2.1.20          A    cluster_a-02:netapp-sansw-1a:1-9.126L47 A SAS 1 20
2.1.21          B    cluster_a-02:netapp-sansw-1b:1-9.126L48 A SAS 1 21
2.1.22          B    cluster_a-02:netapp-sansw-1b:1-9.126L49 A SAS 1 22
2.1.23          A    cluster_a-02:netapp-sansw-1b:1-9.126L50 A SAS 1 23
2.2.0           A    cluster_a-01:netapp-sansw-1a:1-9.126L53 A SAS 2 0
2.2.1           B    cluster_a-01:netapp-sansw-1b:1-9.126L54 A SAS 2 1
2.2.2           A    cluster_a-01:netapp-sansw-1a:1-9.126L55 A SAS 2 2
2.2.3           A    cluster_a-01:netapp-sansw-1a:1-9.126L56 A SAS 2 3
2.2.4           A    cluster_a-01:netapp-sansw-1a:1-9.126L57 A SAS 2 4
2.2.5           B    cluster_a-01:netapp-sansw-1b:1-9.126L58 A SAS 2 5
2.2.6           B    cluster_a-01:netapp-sansw-1b:1-9.126L59 A SAS 2 6
2.2.7           B    cluster_a-01:netapp-sansw-1b:1-9.126L60 A SAS 2 7
2.2.8           B    cluster_a-01:netapp-sansw-1b:1-9.126L61 A SAS 2 8
2.2.9           B    cluster_a-01:netapp-sansw-1b:1-9.126L62 A SAS 2 9
2.2.10          A    cluster_a-01:netapp-sansw-1a:1-9.126L63 A SAS 2 10
2.2.11          B    cluster_a-01:netapp-sansw-1b:1-9.126L64 A SAS 2 11
2.2.12          A    cluster_a-01:netapp-sansw-1a:1-9.126L65 A SAS 2 12
2.2.13          A    cluster_a-01:netapp-sansw-1a:1-9.126L66 A SAS 2 13
2.2.14          B    cluster_a-01:netapp-sansw-1b:1-9.126L67 A SAS 2 14
2.2.15          A    cluster_a-01:netapp-sansw-1b:1-9.126L68 A SAS 2 15
2.2.16          A    cluster_a-01:netapp-sansw-1a:1-9.126L69 A SAS 2 16
2.2.17          B    cluster_a-01:netapp-sansw-1b:1-9.126L70 A SAS 2 17
2.2.18          A    cluster_a-01:netapp-sansw-1b:1-9.126L71 A SAS 2 18
2.2.19          A    cluster_a-01:netapp-sansw-1b:1-9.126L72 A SAS 2 19
2.2.20          B    cluster_a-01:netapp-sansw-1b:1-9.126L73 A SAS 2 20
2.2.21          A    cluster_a-01:netapp-sansw-1b:1-9.126L74 A SAS 2 21
2.2.22          A    cluster_a-01:netapp-sansw-1b:1-9.126L75 A SAS 2 22
2.2.23          A    cluster_a-01:netapp-sansw-1a:1-9.126L76 A SAS 2 23
2.3.0           B    cluster_a-01:netapp-sansw-1b:1-9.126L79 A SAS 3 0
2.3.1           B    cluster_a-01:netapp-sansw-1b:1-9.126L80 A SAS 3 1
2.3.2           B    cluster_a-01:netapp-sansw-1b:1-9.126L81 A SAS 3 2
2.3.3           B    cluster_a-01:netapp-sansw-1b:1-9.126L82 A SAS 3 3
2.3.4           B    cluster_a-01:netapp-sansw-1b:1-9.126L83 A SAS 3 4
2.3.5           B    cluster_a-01:netapp-sansw-1b:1-9.126L84 A SAS 3 5
2.3.6           A    cluster_a-01:netapp-sansw-1b:1-9.126L85 A SAS 3 6
2.3.7           B    cluster_a-01:netapp-sansw-1b:1-9.126L86 A SAS 3 7
2.3.8           B    cluster_a-01:netapp-sansw-1b:1-9.126L87 A SAS 3 8
2.3.9           B    cluster_a-01:netapp-sansw-1b:1-9.126L88 A SAS 3 9
2.3.10          B    cluster_a-01:netapp-sansw-1b:1-9.126L89 A SAS 3 10
2.3.11          B    cluster_a-01:netapp-sansw-1b:1-9.126L90 A SAS 3 11
2.3.12          A    cluster_a-01:netapp-sansw-1b:1-9.126L91 A SAS 3 12
2.3.13          A    cluster_a-01:netapp-sansw-1a:1-9.126L92 A SAS 3 13
2.3.14          B    cluster_a-01:netapp-sansw-1b:1-9.126L93 A SAS 3 14
2.3.15          A    cluster_a-01:netapp-sansw-1a:1-9.126L94 A SAS 3 15
2.3.16          B    cluster_a-01:netapp-sansw-1b:1-9.126L95 A SAS 3 16
2.3.17          B    cluster_a-01:netapp-sansw-1b:1-9.126L96 A SAS 3 17
2.3.18          B    cluster_a-01:netapp-sansw-1b:1-9.126L97 A SAS 3 18
2.3.19          B    cluster_a-01:netapp-sansw-1b:1-9.126L98 A SAS 3 19
2.3.20          B    cluster_a-01:netapp-sansw-1b:1-9.126L99 A SAS 3 20
2.3.21          A    cluster_a-01:netapp-sansw-1b:1-9.126L100 A SAS 3 21
2.3.22          B    cluster_a-01:netapp-sansw-1b:1-9.126L101 A SAS 3 22
2.3.23          B    cluster_a-01:netapp-sansw-1b:1-9.126L102 A SAS 3 23
3.10.0          B    cluster_a-01:netapp-sansw-2b:1-10.126L79 B SAS 10 0
3.10.1          B    cluster_a-01:netapp-sansw-2a:1-10.126L80 B SAS 10 1
3.10.2          A    cluster_a-01:netapp-sansw-2a:1-10.126L81 B SAS 10 2
3.10.3          A    cluster_a-01:netapp-sansw-2a:1-10.126L82 B SAS 10 3
3.10.4          B    cluster_a-01:netapp-sansw-2a:1-10.126L83 B SAS 10 4
3.10.5          B    cluster_a-01:netapp-sansw-2b:1-10.126L84 B SAS 10 5
3.10.6          A    cluster_a-01:netapp-sansw-2a:1-10.126L85 B SAS 10 6
3.10.7          B    cluster_a-01:netapp-sansw-2a:1-10.126L86 B SAS 10 7
3.10.8          A    cluster_a-01:netapp-sansw-2a:1-10.126L87 B SAS 10 8
3.10.9          A    cluster_a-01:netapp-sansw-2a:1-10.126L88 B SAS 10 9
3.10.10         B    cluster_a-01:netapp-sansw-2b:1-10.126L89 B SAS 10 10
3.10.11         A    cluster_a-01:netapp-sansw-2a:1-10.126L90 B SAS 10 11
3.10.12         A    cluster_a-01:netapp-sansw-2a:1-10.126L91 B SAS 10 12
3.10.13         A    cluster_a-01:netapp-sansw-2a:1-10.126L92 B SAS 10 13
3.10.14         A    cluster_a-01:netapp-sansw-2a:1-10.126L93 B SAS 10 14
3.10.15         B    cluster_a-01:netapp-sansw-2b:1-10.126L94 B SAS 10 15
3.10.16         B    cluster_a-01:netapp-sansw-2a:1-10.126L95 B SAS 10 16
3.10.17         A    cluster_a-01:netapp-sansw-2a:1-10.126L96 B SAS 10 17
3.10.18         A    cluster_a-01:netapp-sansw-2a:1-10.126L97 B SAS 10 18
3.10.19         A    cluster_a-01:netapp-sansw-2a:1-10.126L98 B SAS 10 19
3.10.20         A    cluster_a-01:netapp-sansw-2a:1-10.126L99 B SAS 10 20
3.10.21         A    cluster_a-01:netapp-sansw-2a:1-10.126L100 B SAS 10 21
3.10.22         A    cluster_a-01:netapp-sansw-2a:1-10.126L101 B SAS 10 22
3.10.23         B    cluster_a-01:netapp-sansw-2b:1-10.126L102 B SAS 10 23
3.11.0          A    cluster_a-02:netapp-sansw-2a:1-10.126L53 B SAS 11 0
3.11.1          A    cluster_a-02:netapp-sansw-2a:1-10.126L54 B SAS 11 1
3.11.2          B    cluster_a-02:netapp-sansw-2b:1-10.126L55 B SAS 11 2
3.11.3          A    cluster_a-02:netapp-sansw-2a:1-10.126L56 B SAS 11 3
3.11.4          A    cluster_a-02:netapp-sansw-2a:1-10.126L57 B SAS 11 4
3.11.5          B    cluster_a-02:netapp-sansw-2b:1-10.126L58 B SAS 11 5
3.11.6          B    cluster_a-02:netapp-sansw-2b:1-10.126L59 B SAS 11 6
3.11.7          A    cluster_a-02:netapp-sansw-2a:1-10.126L60 B SAS 11 7
3.11.8          B    cluster_a-02:netapp-sansw-2a:1-10.126L61 B SAS 11 8
3.11.9          A    cluster_a-02:netapp-sansw-2a:1-10.126L62 B SAS 11 9
3.11.10         B    cluster_a-02:netapp-sansw-2a:1-10.126L63 B SAS 11 10
3.11.11         B    cluster_a-02:netapp-sansw-2a:1-10.126L64 B SAS 11 11
3.11.12         B    cluster_a-02:netapp-sansw-2a:1-10.126L65 B SAS 11 12
3.11.13         B    cluster_a-02:netapp-sansw-2b:1-10.126L66 B SAS 11 13
3.11.14         B    cluster_a-02:netapp-sansw-2a:1-10.126L67 B SAS 11 14
3.11.15         A    cluster_a-02:netapp-sansw-2a:1-10.126L68 B SAS 11 15
3.11.16         A    cluster_a-02:netapp-sansw-2a:1-10.126L69 B SAS 11 16
3.11.17         A    cluster_a-02:netapp-sansw-2a:1-10.126L70 B SAS 11 17
3.11.18         B    cluster_a-02:netapp-sansw-2b:1-10.126L71 B SAS 11 18
3.11.19         B    cluster_a-02:netapp-sansw-2b:1-10.126L72 B SAS 11 19
3.11.20         A    cluster_a-02:netapp-sansw-2a:1-10.126L73 B SAS 11 20
3.11.21         B    cluster_a-02:netapp-sansw-2a:1-10.126L74 B SAS 11 21
3.11.22         B    cluster_a-02:netapp-sansw-2a:1-10.126L75 B SAS 11 22
3.11.23         B    cluster_a-02:netapp-sansw-2a:1-10.126L76 B SAS 11 23
3.12.0          B    cluster_a-01:netapp-sansw-2b:1-10.126L27 B SAS 12 0
3.12.1          B    cluster_a-01:netapp-sansw-2a:1-10.126L28 B SAS 12 1
3.12.2          B    cluster_a-01:netapp-sansw-2a:1-10.126L29 B SAS 12 2
3.12.3          A    cluster_a-01:netapp-sansw-2a:1-10.126L30 B SAS 12 3
3.12.4          A    cluster_a-01:netapp-sansw-2a:1-10.126L31 B SAS 12 4
3.12.5          A    cluster_a-01:netapp-sansw-2a:1-10.126L32 B SAS 12 5
3.12.6          A    cluster_a-01:netapp-sansw-2a:1-10.126L33 B SAS 12 6
3.12.7          A    cluster_a-01:netapp-sansw-2a:1-10.126L34 B SAS 12 7
3.12.8          B    cluster_a-01:netapp-sansw-2b:1-10.126L35 B SAS 12 8
3.12.9          A    cluster_a-01:netapp-sansw-2a:1-10.126L36 B SAS 12 9
3.12.10         A    cluster_a-01:netapp-sansw-2a:1-10.126L37 B SAS 12 10
3.12.11         B    cluster_a-01:netapp-sansw-2b:1-10.126L38 B SAS 12 11
3.12.12         B    cluster_a-01:netapp-sansw-2a:1-10.126L39 B SAS 12 12
3.12.13         B    cluster_a-01:netapp-sansw-2b:1-10.126L40 B SAS 12 13
3.12.14         B    cluster_a-01:netapp-sansw-2b:1-10.126L41 B SAS 12 14
3.12.15         A    cluster_a-01:netapp-sansw-2a:1-10.126L42 B SAS 12 15
3.12.16         A    cluster_a-01:netapp-sansw-2a:1-10.126L43 B SAS 12 16
3.12.17         B    cluster_a-01:netapp-sansw-2a:1-10.126L44 B SAS 12 17
3.12.18         B    cluster_a-01:netapp-sansw-2b:1-10.126L45 B SAS 12 18
3.12.19         A    cluster_a-01:netapp-sansw-2a:1-10.126L46 B SAS 12 19
3.12.20         B    cluster_a-01:netapp-sansw-2a:1-10.126L47 B SAS 12 20
3.12.21         A    cluster_a-01:netapp-sansw-2a:1-10.126L48 B SAS 12 21
3.12.22         B    cluster_a-01:netapp-sansw-2a:1-10.126L49 B SAS 12 22
3.12.23         A    cluster_a-01:netapp-sansw-2a:1-10.126L50 B SAS 12 23
3.13.0          A    cluster_a-01:netapp-sansw-2a:1-10.126L1 B SAS 13 0
3.13.1          A    cluster_a-01:netapp-sansw-2a:1-10.126L2 B SAS 13 1
3.13.2          A    cluster_a-01:netapp-sansw-2a:1-10.126L3 B SAS 13 2
3.13.3          B    cluster_a-01:netapp-sansw-2b:1-10.126L4 B SAS 13 3
3.13.4          A    cluster_a-01:netapp-sansw-2a:1-10.126L5 B SAS 13 4
3.13.5          A    cluster_a-01:netapp-sansw-2a:1-10.126L6 B SAS 13 5
3.13.6          B    cluster_a-01:netapp-sansw-2a:1-10.126L7 B SAS 13 6
3.13.7          B    cluster_a-01:netapp-sansw-2a:1-10.126L8 B SAS 13 7
3.13.8          B    cluster_a-01:netapp-sansw-2a:1-10.126L9 B SAS 13 8
3.13.9          A    cluster_a-01:netapp-sansw-2a:1-10.126L10 B SAS 13 9
3.13.10         B    cluster_a-01:netapp-sansw-2b:1-10.126L11 B SAS 13 10
3.13.11         B    cluster_a-01:netapp-sansw-2b:1-10.126L12 B SAS 13 11
3.13.12         B    cluster_a-01:netapp-sansw-2a:1-10.126L13 B SAS 13 12
3.13.13         B    cluster_a-01:netapp-sansw-2a:1-10.126L14 B SAS 13 13
3.13.14         B    cluster_a-01:netapp-sansw-2a:1-10.126L15 B SAS 13 14
3.13.15         B    cluster_a-01:netapp-sansw-2b:1-10.126L16 B SAS 13 15
3.13.16         B    cluster_a-01:netapp-sansw-2b:1-10.126L17 B SAS 13 16
3.13.17         B    cluster_a-01:netapp-sansw-2b:1-10.126L18 B SAS 13 17
3.13.18         B    cluster_a-01:netapp-sansw-2b:1-10.126L19 B SAS 13 18
3.13.19         B    cluster_a-01:netapp-sansw-2a:1-10.126L20 B SAS 13 19
3.13.20         B    cluster_a-01:netapp-sansw-2a:1-10.126L21 B SAS 13 20
3.13.21         B    cluster_a-01:netapp-sansw-2b:1-10.126L22 B SAS 13 21
3.13.22         A    cluster_a-01:netapp-sansw-2a:1-10.126L23 B SAS 13 22
3.13.23         B    cluster_a-01:netapp-sansw-2b:1-10.126L24 B SAS 13 23
4.14.0          A    cluster_a-01:netapp-sansw-2a:1-10.126L501 B SSD 14 0
4.14.1          A    cluster_a-01:netapp-sansw-2a:1-10.126L502 B SSD 14 1
4.14.2          A    cluster_a-01:netapp-sansw-2a:1-10.126L503 B SSD 14 2
4.14.3          B    cluster_a-01:netapp-sansw-2b:1-10.126L504 B SSD 14 3
4.14.4          A    cluster_a-01:netapp-sansw-2a:1-10.126L505 B SSD 14 4
4.14.5          A    cluster_a-01:netapp-sansw-2a:1-10.126L506 B SSD 14 5
4.14.6          A    cluster_a-01:netapp-sansw-2a:1-10.126L507 B SSD 14 6
4.14.7          A    cluster_a-01:netapp-sansw-2a:1-10.126L508 B SSD 14 7
4.14.8          B    cluster_a-01:netapp-sansw-2a:1-10.126L509 B SSD 14 8
4.14.9          B    cluster_a-01:netapp-sansw-2b:1-10.126L510 B SSD 14 9
4.14.10         A    cluster_a-01:netapp-sansw-2a:1-10.126L511 B SSD 14 10
4.14.11         A    cluster_a-01:netapp-sansw-2a:1-10.126L512 B SSD 14 11
4.14.12         A    cluster_a-01:netapp-sansw-2a:1-10.126L513 B SSD 14 12
4.14.13         A    cluster_a-01:netapp-sansw-2a:1-10.126L514 B SSD 14 13
4.14.14         A    cluster_a-01:netapp-sansw-2a:1-10.126L515 B SSD 14 14
4.14.15         A    cluster_a-01:netapp-sansw-2a:1-10.126L516 B SSD 14 15
4.14.16         A    cluster_a-01:netapp-sansw-2a:1-10.126L517 B SSD 14 16
4.14.17         A    cluster_a-01:netapp-sansw-2a:1-10.126L518 B SSD 14 17
4.14.18         B    cluster_a-01:netapp-sansw-2a:1-10.126L519 B SSD 14 18
4.14.19         B    cluster_a-01:netapp-sansw-2a:1-10.126L520 B SSD 14 19
4.14.20         A    cluster_a-01:netapp-sansw-2a:1-10.126L521 B SSD 14 20
4.14.21         B    cluster_a-01:netapp-sansw-2a:1-10.126L522 B SSD 14 21
4.14.22         A    cluster_a-01:netapp-sansw-2a:1-10.126L523 B SSD 14 22
4.14.23         B    cluster_a-01:netapp-sansw-2a:1-10.126L524 B SSD 14 23
240 entries were displayed.

 

And the shelf output, which is also strange (the shelf naming):

 

cluster_a::> storage shelf show 
                                                                      
                                                       Module Operational
       Shelf Name Shelf ID Serial Number   Model       Type   Status
----------------- -------- --------------- ----------- ------ -----------
              1.4        4 SHFHU1852XXXXXX DS224-12    IOM12  Normal
              2.0        0 SHFHU1849XXXXXX DS224-12    IOM12  Normal
              2.1        1 SHFHU1850XXXXXX DS224-12    IOM12  Normal
              2.2        2 SHFHU1850XXXXXX DS224-12    IOM12  Normal
              2.3        3 SHFHU1848XXXXXX DS224-12    IOM12  Normal
             3.10       10 SHFHU1848XXXXXX DS224-12    IOM12  Normal
             3.11       11 SHFHU1848XXXXXX DS224-12    IOM12  Normal
             3.12       12 SHFHU1848XXXXXX DS224-12    IOM12  Normal
             3.13       13 SHFHU1848XXXXXX DS224-12    IOM12  Normal
             4.14       14 SHFHU1852XXXXXX DS224-12    IOM12  Normal
10 entries were displayed.

cluster_a::> 

and the shelf ports

 

cluster_a::> storage shelf port show
                                                                      
Shelf ID Module State        Internal?
----- -- ------ ------------ ---------
1.4
       0 A      connected    false
       1 A      disconnected false
       2 A      disconnected false
       3 A      disconnected false
       4 B      disconnected false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
2.0
       0 A      connected    false
       1 A      disconnected false
       2 A      connected    false
       3 A      disconnected false
       4 B      disconnected false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
2.1
       0 A      connected    false
       1 A      disconnected false
       2 A      connected    false
       3 A      disconnected false
       4 B      connected    false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
2.2
       0 A      connected    false
       1 A      disconnected false
       2 A      connected    false
       3 A      disconnected false
       4 B      connected    false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
2.3
       0 A      connected    false
       1 A      disconnected false
       2 A      disconnected false
       3 A      disconnected false
       4 B      connected    false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
3.10
       0 A      connected    false
       1 A      disconnected false
       2 A      connected    false
       3 A      disconnected false
       4 B      disconnected false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
3.11
       0 A      connected    false
       1 A      disconnected false
       2 A      connected    false
       3 A      disconnected false
       4 B      connected    false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
3.12
       0 A      connected    false
       1 A      disconnected false
       2 A      connected    false
       3 A      disconnected false
       4 B      connected    false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
3.13
       0 A      connected    false
       1 A      disconnected false
       2 A      disconnected false
       3 A      disconnected false
       4 B      connected    false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
4.14
       0 A      connected    false
       1 A      disconnected false
       2 A      disconnected false
       3 A      disconnected false
       4 B      disconnected false
       5 B      disconnected false
       6 B      connected    false
       7 B      disconnected false
80 entries were displayed.

cluster_a::> 

 

The question is, how exactly should this output look like, if it's cabled as it is shown in the guide? I suspect only one that could happen: after I've checked the physical cabling, there was a test during which one SAS cable was plugged out from FC-SAS bridge and then plugged in again. After that I haven't checked anything, so i guess one cable could be plugged in a wrong port. I've checked this via CLI, but all the bridges show just the same - that everything is ok:

 

cluster_a::> storage bridge run-cli -name ATTO_2000001086a9d800 -command SASPortList
[Job 581] Job is queued: Executing ATTO CLI command "SASPortList" on bridge "ATTO_200000XXXXXXXXX0"..                                          
;Connector      PHY     Link            Speed   SAS Address
;=============================================================
Device  A       1       Up              12Gb    5001086000XXXXX0
Device  A       2       Up              12Gb    5001086000XXXXX0
Device  A       3       Up              12Gb    5001086000XXXXX0
Device  A       4       Up              12Gb    5001086000XXXXX0
Device  B       1       Up              12Gb    5001086000XXXXX4
Device  B       2       Up              12Gb    5001086000XXXXX4
Device  B       3       Up              12Gb    5001086000XXXXX4
Device  B       4       Up              12Gb    5001086000XXXXX4
Device  C       1       Disabled        12Gb    5001086000XXXXX8
Device  C       2       Disabled        12Gb    5001086000XXXXX8
Device  C       3       Disabled        12Gb    5001086000XXXXX8
Device  C       4       Disabled        12Gb    5001086000XXXXX8
Device  D       1       Disabled        12Gb    5001086000XXXXXc
Device  D       2       Disabled        12Gb    5001086000XXXXXc
Device  D       3       Disabled        12Gb    5001086000XXXXXc
Device  D       4       Disabled        12Gb    5001086000XXXXXc

Re: Understanding disk paths

"some have A-B and some have B-B (4.14.0-4.14.23). Just in theory, how this could happen?"

 

It is not bad cabling. It could be either IOM or ATTO bridge that skip some of the disks. The fact that it is showing this anomaly in multiple shelves tells me that it is not an IOM issue

 

Can you provide the output of 'sysconfig -a' and 'sysconfig -r' from both nodes? (node shell)

Re: Understanding disk paths

Could you please check your PM?

Re: Understanding disk paths

Wops, couldn't send the PM, because the length of output (500-600kb of text)... There's a lot of serial numbers there, i'm not sure that it's a good idea to post it here Smiley Sad

Re: Understanding disk paths

Looks like the situation is not good, so i opened a support case. I hope that helps... Thanks for your answers!

Re: Understanding disk paths

Will be keen to know what is the cause here, Please do let us know later.

 

As far as the storage shelf output is concerend, it looks fine. You mentioned you have 2 stacks on each side, so based on the shelf naming I am assuming it is in this order.

 


Node1:
Stack 1: shelf 4 [1.4]
Stack 2: Shelf 0,1,2,3 [2.0,2.1,2.2,2.3]

 

Node2:
Stack 3: Shelf 10,11,12,13 [3.10,3.11,3.12,3.13]
Stack 4: Shelf 14 [4.14]

Try the new BETA Support Site!
Beta Support Site
Forums