7-Mode disk replace wrong size disk

[BobMcKim]

I have a 8.1.2 7-mode box that I added 23 SAS drives to a new raidgroup in an aggregate.  Even though I specified 23@450g for the disk add, it brought four 600GB SAS dirives into the raidgroup, which used all of my 600GB spares.  The 600GB drives are the parity, dparity and two data disks in the new raidgroup.  I need to replace these with the correct 450GB size drives, which the remaining 19 drives in the rg are.  When trying a disk replace I receive a " Disk 6c.20.21 is not of appropriate size" message.  I've tried the disk replace with and without the -m flag.  I'm reluctant to try the -f flag unless someone can verify that this will work, or not.

I also tried a disk fail and received -

WARNING! There is no spare disk available to which to copy.
Are you sure you want to continue with disk fail (y/n)?

Any suggestions?  I can't do a vol move to relocate the vols to other aggrs as they are CIFS with active shares and we can't take an outage.

[AlexSun0302]

you mean is

aggr0:

rg0:19 450G SAS

rg1:4 600G SAS

spare: 1 450G SAS ?

so you want replace the 600G to 450G ?

[YIshikawa]

Please post "sysconfig -r" output.

[BobMcKim]

RAID group /aggrSvr02/plex0/rg2 (normal, block checksums)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity 6a.05.18 6a 5 18 SA:B - SAS 15000 560000/1146880000 560879/1148681096
parity 6a.04.23 6a 4 23 SA:B - SAS 15000 560000/1146880000 560879/1148681096
data 6a.05.22 6a 5 22 SA:B - SAS 15000 560000/1146880000 560879/1148681096
data 6a.05.23 6a 5 23 SA:B - SAS 15000 560000/1146880000 560879/1148681096
data 2c.20.0 2c 20 0 SA:A - SAS 15000 418000/856064000 420584/861357448
data 2a.00.8 2a 0 8 SA:A - SAS 15000 418000/856064000 420584/861357448
data 6c.20.1 6c 20 1 SA:B - SAS 15000 418000/856064000 420584/861357448
data 2a.00.10 2a 0 10 SA:A - SAS 15000 418000/856064000 420584/861357448
data 2c.20.2 2c 20 2 SA:A - SAS 15000 418000/856064000 420584/861357448
data 6c.20.3 6c 20 3 SA:B - SAS 15000 418000/856064000 420584/861357448
data 2c.20.4 2c 20 4 SA:A - SAS 15000 418000/856064000 420584/861357448
data 6c.20.5 6c 20 5 SA:B - SAS 15000 418000/856064000 420584/861357448
data 2c.20.6 2c 20 6 SA:A - SAS 15000 418000/856064000 420584/861357448
data 6c.20.7 6c 20 7 SA:B - SAS 15000 418000/856064000 420584/861357448
data 2c.20.8 2c 20 8 SA:A - SAS 15000 418000/856064000 420584/861357448
data 6c.20.9 6c 20 9 SA:B - SAS 15000 418000/856064000 420584/861357448
data 2c.20.10 2c 20 10 SA:A - SAS 15000 418000/856064000 420584/861357448
data 6c.20.11 6c 20 11 SA:B - SAS 15000 418000/856064000 420584/861357448
data 2c.20.12 2c 20 12 SA:A - SAS 15000 418000/856064000 420584/861357448
data 6c.20.13 6c 20 13 SA:B - SAS 15000 418000/856064000 420584/861357448
data 2c.20.14 2c 20 14 SA:A - SAS 15000 418000/856064000 420584/861357448
data 6c.20.15 6c 20 15 SA:B - SAS 15000 418000/856064000 420584/861357448
data 2c.20.16 2c 20 16 SA:A - SAS 15000 418000/856064000 420584/861357448

[YIshikawa]

Spare disks are omitted so I could not confirm if you have appropriate spare disks for this RAID group.

"disk replace" requires spare disk of same characteristics for replacement. You cannot select 450GB disks to replace 600GB disk.

But, I'm not sure it is really possible, RAID reconstruction may select 450GB disks. Try it by failing 600GB disks by "disk fail -i <600GB disk>".

[BobMcKim]

I tried both a disk replace and a disk fail with the samr results.  Disk fail gave me a no sufficient spares available and disk replace gives me a disk is not of appropriate size.

Spare disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
Spare disks for block checksum
spare 5a.70 5a 4 6 FC:A - FCAL 15000 272000/557056000 274845/562884296
spare 5a.71 5a 4 7 FC:A - FCAL 15000 272000/557056000 274845/562884296
spare 5a.90 5a 5 10 FC:A - FCAL 15000 272000/557056000 274845/562884296
spare 9a.58 9a 3 10 FC:B - FCAL 15000 272000/557056000 274845/562884296
spare 9a.59 9a 3 11 FC:B - FCAL 15000 272000/557056000 274845/562884296
spare 5a.86 5a 5 6 FC:A - FCAL 15000 272000/557056000 280104/573653840
spare 6c.20.23 6c 20 23 SA:B - SAS 15000 418000/856064000 420156/860480768
spare 2c.20.18 2c 20 18 SA:A - SAS 15000 418000/856064000 420584/861357448
spare 2c.20.20 2c 20 20 SA:A - SAS 15000 418000/856064000 420584/861357448
spare 2c.20.22 2c 20 22 SA:A - SAS 15000 418000/856064000 420584/861357448
spare 6c.20.17 6c 20 17 SA:B - SAS 15000 418000/856064000 420584/861357448
spare 6c.20.19 6c 20 19 SA:B - SAS 15000 418000/856064000 420584/861357448
spare 6c.20.21 6c 20 21 SA:B - SAS 15000 418000/856064000 420584/861357448

[YIshikawa]

As per this spare disk list, you have sufficient spare disks of SAS 600GB 15krpm.

Reviewing your RAID configuration I found that 'Used Size' of 600GB disks are not downsized. Usually, 'Used Size' of larger disks are downsized to that of smaller disks when adding larger disks into RAID group of smaller disks.

As previously mentioned, adding larger disk as parity brings this odd state. This behavior is described in below.

http://mysupport.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=108147

Once 'Used Size' grows, this RAID group requires larger disks to adding and replacing disks, unfortunately.

[aborzenkov]

Unfortunately, to my best knowledge this is impossible (using officially known tools at least). Parity and dparity disks consume full disk size when allocated initially, so you cannot replace them with smaller disks. They could be some undocumented internal means to do it 🙂