RAID disks on different channels

[larrynce01]

Hello all,

I have encountered a system with five FC shelves, each having dual connections to a single controller and the disks in RAID groups across these shelves are mixed between each channel, below is an example.

Oh and yes I have noticed the RPM issue also

dparity          4b.33     4b    2 1   FC:A   -  FCAL 10000 68000/139264000 68552/140395088
parity              0b.71 0b    4   7   FC:B -  FCAL 15000 68000/139264000   68552/140395088
data 0b.81     0b 5   1   FC:B   -  FCAL 10000 68000/139264000   69536/142410400
data 0b.53     0b 3   5   FC:B   -  FCAL 15000 68000/139264000   137104/280790184
data 0b.66     0b 4   2   FC:B   -  FCAL 10000 68000/139264000   137104/280790184
data 0b.34     0b 2   2   FC:B   -  FCAL 10000 68000/139264000   68552/140395088
data 0b.51     0b 3   3   FC:B   -  FCAL 10000 68000/139264000   68552/140395088
data 4b.82     4b 5   2   FC:A   -  FCAL 10000 68000/139264000   69536/142410400
data 0b.19     0b 1   3   FC:B   -  FCAL 15000 68000/139264000   137422/281442144
data 4b.52     4b 3   4   FC:A   -  FCAL 10000 68000/139264000   68552/140395088

Can anyone suggest why this may have been done, or how it could happen unintentionally?

Many thanks.

[ismopuuronen]

Hello,

you see the disks trough the module B and module A.

If you issue the command >storage show disk <disk-id>

example: >storage show disk 0b.19

You should see two paths for it?

So it's "random" which pat will be used as primary, and you see it in sysconfig -r / aggr status -r command output.

You have mixed rpms in the same raidgroup.

supported, not optimal.

You just have to know, that 15k disks connot work in that speed, because there is also 10k disks in the same rg.

Rg rpm would be 10k.

If you have different size of disks in the same rg, then the usable size of the disk will be mainly the smallest one.

Example: parity disks are 72GB:

You add 2TB disk to this aggregate, you'll lose 1,9xTB, from that disk. (parity needs to be calculated to smaller disks).

You can succesfully add bigger disks to rg, if the parity disks are large enough to hold the parity.

You cannot replace 72GB disk with 2TB disk. Replace command issued, it will copy the data from 72GB disk to 2TB disk and it will work as 72GB.

Disk fails, and only available spare is 2TB for this broken 72G disk. ONTAP will reconstruct the disk from the paritys, and usable size will be 72GB.

Br.

Ismo.

[thomas_glodde]

ismo,

nowadays, if a 72GB 10k disk dies, you get a 144GB 15k disk as replacement as 10k ones are out of stock. You can see it on the physical size value, dont worry about that.

[ismopuuronen]

Hi,

I'm not worried about

It was just an example what happens if you add an 15k disk to a rg where is only 10k disks, or what happens if you add bigger disk to a rg than the exsiting ones.

Br.

Ismo.

[larrynce01]

Hi Ismopuuronen, I know they are dual pathed. It's just that I'm more used to seeing the disks listed twice, once for each channel, rather than one list from a sysconfig with disks listed on only one channel, apparently at random. Is this perhaps related to this system only having one controller. All the others are HA.

[ismopuuronen]

Hi,

Maybe one of these commands helps you?

>sysconfig -r shows where the disks belongs and the active path, it also shows if there are partner disks.
>sysconfig -a shows disks twice and more, where the shelfs are connected etc.
>storage show disk -p could be what you are looking for now?

>disk show -v is good way to determine ownership of the disks.

Br.
Ismo.

[larrynce01]

Hi,

Pretty sure I know the ownership (one controller ).  It was more the question of since some disks show on 4b and some on 0b I was wondering if the I/O across aggr was also along these channels, so priority differs between disks in an aggr, or even if it was possible to define the I/O path per disk though given the way the systems work I can't see the point on a single controller setup.

[thomas_glodde · Accepted solution]

netapp supports 2 pathes to a disk but will only activly use 1. So it will pull half of the disk i/o through way a and other half through way b. if one path dies, it might switch over to way a only, you can correct that using "stoarge load balance" and it will go 50/50 again.

[larrynce01]

Thanks Thomas,  I'm still a little uncertain why the disks are unevenly distributed across two channels on a single controller but it seems to be working. I would have expected them to report the channel on which I/O is actually running or if load balanced to see a 50/50 distribution. It's an inherited system so it may have achieved its current state over time. I'll try load balance and see if everything returns to a more coherent state.