Re: Error message: There are not enough spare disks

MMUELLER_HC · ‎2014-05-13

Hi all,

I was today logged-in on our NetApp:

system:system:system_model:FAS2240-4

system:system:ontap_version:NetApp Release 8.2P2 7-Mode: Sat Jul 20 20:31:47 PDT 2013

On one of the filers I saw regular this error message:

[NAME:monitor.globalStatus.nonCritical:warning]: There are not enough spare disks.

vol status -r

Aggregate aggr0 (online, raid4) (block checksums)

Plex /aggr0/plex0 (online, normal, active)

RAID group /aggr0/plex0/rg0 (normal, block checksums)

RAID Disk Device	HA SHELF BAY CHAN Pool Type RPM Used (MB/blks)	Phys (MB/blks)
--------- ------	------------- ---- ---- ---- ----- --------------	--------------
parity	0a.00.7 0a	0 7 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
data	0a.00.9 0a	0 9 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816

Pool1 spare disks (empty)

Pool0 spare disks (empty)

Partner disks

RAID Disk Device	HA SHELF BAY CHAN Pool Type RPM Used (MB/blks)	Phys (MB/blks)
--------- ------	------------- ---- ---- ---- ----- --------------	--------------
partner 0a.00.5 0a	0 5 SA:B 0 SSD N/A 95146/194859520 95396/195371568
partner 0a.00.1 0a	0 1 SA:B 0 SSD N/A 95146/194859520 95396/195371568
partner 0a.00.3 0a	0 3 SA:B 0 SSD N/A 95146/194859520 95396/195371568
partner 0b.01.8 0b	1 8 SA:A 0 SAS 10000 857000/1755136000 858483/1758174768
partner 0b.01.9 0b	1 9 SA:A 0 SAS 10000 857000/1755136000 858483/1758174768
partner 0b.01.10 0b	1 10 SA:A 0 SAS 10000 857000/1755136000 858483/1758174768
partner 0a.00.12 0a	0 12 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.15 0a	0 15 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.14 0a	0 14 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.11 0a	0 11 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.23 0a	0 23 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.20 0a	0 20 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.17 0a	0 17 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.13 0a	0 13 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.21 0a	0 21 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.16 0a	0 16 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.22 0a	0 22 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.18 0a	0 18 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.19 0a	0 19 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
partner 0a.00.10 0a	0 10 SA:B 0 BSAS 7200 0/0	847884/1736466816
partner 0a.00.8 0a	0 8 SA:B 0 BSAS 7200 0/0	847884/1736466816
partner 0a.00.6 0a	0 6 SA:B 0 BSAS 7200 0/0	847884/1736466816
partner 0b.01.11 0b	1 11 SA:A 0 SAS 10000 0/0	858483/1758174768
partner 0b.01.7 0b	1 7 SA:A 0 SAS 10000 0/0	858483/1758174768
partner 0b.01.5 0b	1 5 SA:A 0 SAS 10000 0/0	858483/1758174768
partner 0b.01.6 0b	1 6 SA:A 0 SAS 10000 0/0	858483/1758174768
partner 0b.01.3 0b	1 3 SA:A 0 SAS 10000 0/0	858483/1758174768
partner 0b.01.4 0b	1 4 SA:A 0 SAS 10000 0/0	858483/1758174768
partner 0b.01.2 0b	1 2 SA:A 0 SAS 10000 0/0	858483/1758174768
partner 0b.01.1 0b	1 1 SA:A 0 SAS 10000 0/0	858483/1758174768
partner 0b.01.0 0b	1 0 SA:A 0 SAS 10000 0/0	858483/1758174768
partner 0a.00.2 0a	0 2 SA:B 0 SSD N/A 0/0	95396/195371568
partner 0a.00.0 0a	0 0 SA:B 0 SSD N/A 0/0	95396/195371568
partner 0a.00.4 0a	0 4 SA:B 0 SSD N/A 0/0	95396/195371568

On the other I haven't saw this message. I'm pretty sure the setup of the disks aren't done that well, but now it is like it is 😞

Here's the output of the second filer:

vol status -r

Aggregate aggrsata (online, raid_dp, hybrid_enabled) (block checksums)

Plex /aggrsata/plex0 (online, normal, active)

RAID group /aggrsata/plex0/rg0 (normal, block checksums)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

dparity 0a.00.6 0a 0 6 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

parity 0a.00.11 0a 0 11 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.12 0a 0 12 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.13 0a 0 13 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.14 0a 0 14 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.15 0a 0 15 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.16 0a 0 16 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.17 0a 0 17 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.18 0a 0 18 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.19 0a 0 19 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.20 0a 0 20 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.21 0a 0 21 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.22 0a 0 22 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

Aggregate aggrsas (online, raid_dp, hybrid) (block checksums)

Plex /aggrsas/plex0 (online, normal, active)

RAID group /aggrsas/plex0/rg0 (normal, block checksums)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

dparity 0b.01.0 0b 1 0 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

parity 0b.01.1 0b 1 1 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

data 0b.01.2 0b 1 2 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

data 0b.01.3 0b 1 3 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

data 0b.01.4 0b 1 4 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

data 0b.01.5 0b 1 5 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

data 0b.01.6 0b 1 6 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

data 0b.01.7 0b 1 7 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

data 0b.01.8 0b 1 8 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

data 0b.01.9 0b 1 9 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

data 0b.01.10 0b 1 10 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

RAID group /aggrsas/plex0/rg1 (normal, block checksums)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

dparity 0a.00.0 0a 0 0 SA:A 0 SSD N/A 95146/194859520 95396/195371568

parity 0a.00.1 0a 0 1 SA:A 0 SSD N/A 95146/194859520 95396/195371568

data 0a.00.2 0a 0 2 SA:A 0 SSD N/A 95146/194859520 95396/195371568

Aggregate aggr0 (online, raid4) (block checksums)

Plex /aggr0/plex0 (online, normal, active)

RAID group /aggr0/plex0/rg0 (normal, block checksums)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

parity 0a.00.8 0a 0 8 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

data 0a.00.10 0a 0 10 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

Spare disks for block checksum

spare 0b.01.11 0b 1 11 SA:B 0 SAS 10000 857000/1755136000 858483/1758174768

spare 0a.00.23 0a 0 23 SA:A 0 BSAS 7200 847555/1735794176 847884/1736466816 (not zeroed)

spare 0a.00.3 0a 0 3 SA:A 0 SSD N/A 95146/194859520 95396/195371568

spare 0a.00.4 0a 0 4 SA:A 0 SSD N/A 95146/194859520 95396/195371568

spare 0a.00.5 0a 0 5 SA:A 0 SSD N/A 95146/194859520 95396/195371568

Partner disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

partner 0a.00.7 0a 0 7 SA:A 0 BSAS 7200 0/0 847884/1736466816

partner 0a.00.9 0a 0 9 SA:A 0 BSAS 7200 0/0 847884/1736466816

Any advice how to get solved this issue or any advice what needs to be reconfigured

Any help is highly appreciated

Cheers

Michael

JGPSHNTAP · ‎2014-05-13

I'm trying to figure how you got yourself into this pickle.. You got a little mess on your hands..

It looks like you have three types of disk in this filer, SSD, SAS and SATA.

You have aggr0 as raid-4, with such a small system, you might have wanted to spec this filer differently

You essentially have only spare of each drive and you aren't balanced properly across the cluster. You should slap your netapp sales rep for specing this system like this.

MMUELLER_HC · ‎2014-05-13

Hi,

that's right. SSD, SAS, SATA.

As far as I know is the SSD used as fast cache. And then there are are dedicated aggregates for SATA and SAS.

The Setup isn't balanced at all, it is more a ACTIVE/PASSIVE setup.

Problem is that I'm not that familiar with NetApp and the guy who did it, isn't here anymore.

Anything you could recommend to change?

JGPSHNTAP · ‎2014-05-13

You're stuck my man. First, you need to zero the spares (disk zero spares)

Now, I know why the guy who set this up left, he has no clue what he was doing.

I'm counting

18 x sata

6 ssd's

That looks like it's in your internal filer

then you added a shelf for SAS and you have 12 SAS drives

What i would do is roll up aggr0 into the sas aggr, (moving root volume , etc..)

Destroy aggr0, and zero spares, and then move one spare to the other node

Therefore, aggr0 on the other side is raid4 with one spare. (should be using raid-dp, but your stuck)

Also you are burning SSD drives with no use, maybe try to swap two of them out, since you only need one spare for your flashpool.

But, i would call netapp support to verify, don't take just my word for it