Subscribe

Error message: There are not enough spare disks

Hi all,

I was today logged-in on our NetApp:

system:system:system_model:FAS2240-4

system:system:ontap_version:NetApp Release 8.2P2 7-Mode: Sat Jul 20 20:31:47 PDT 2013

On one of the filers I saw regular this error message:

[NAME:monitor.globalStatus.nonCritical:warning]: There are not enough spare disks.

vol status -r                

Aggregate aggr0 (online, raid4) (block checksums)

  Plex /aggr0/plex0 (online, normal, active)

RAID group /aggr0/plex0/rg0 (normal, block checksums)

  RAID Disk    Device HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)Phys (MB/blks)
  ---------    ------ ------------- ---- ---- ---- ----- ----------------------------
  parity 0a.00.7     0a0   7   SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
  data   0a.00.9     0a0   9   SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816

Pool1 spare disks (empty)

Pool0 spare disks (empty)

Partner disks

RAID Disk    Device HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)Phys (MB/blks)
---------    ------ ------------- ---- ---- ---- ----- ----------------------------
partner     0a.00.5     0a0   5   SA:B   0   SSD   N/A 95146/194859520   95396/195371568
partner     0a.00.1     0a0   1   SA:B   0   SSD   N/A 95146/194859520   95396/195371568
partner     0a.00.3     0a0   3   SA:B   0   SSD   N/A 95146/194859520   95396/195371568
partner     0b.01.8     0b1   8   SA:A   0   SAS 10000 857000/1755136000 858483/1758174768
partner     0b.01.9     0b1   9   SA:A   0   SAS 10000 857000/1755136000 858483/1758174768
partner     0b.01.10    0b1   10  SA:A   0   SAS 10000 857000/1755136000 858483/1758174768
partner     0a.00.12    0a0   12  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.15    0a0   15  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.14    0a0   14  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.11    0a0   11  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.23    0a0   23  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.20    0a0   20  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.17    0a0   17  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.13    0a0   13  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.21    0a0   21  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.16    0a0   16  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.22    0a0   22  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.18    0a0   18  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.19    0a0   19  SA:B   0  BSAS  7200 847555/1735794176 847884/1736466816
partner     0a.00.10    0a0   10  SA:B   0  BSAS  7200 0/0           847884/1736466816
partner     0a.00.8     0a0   8   SA:B   0  BSAS  7200 0/0           847884/1736466816
partner     0a.00.6     0a0   6   SA:B   0  BSAS  7200 0/0           847884/1736466816
partner     0b.01.11    0b1   11  SA:A   0   SAS 10000 0/0           858483/1758174768
partner     0b.01.7     0b1   7   SA:A   0   SAS 10000 0/0           858483/1758174768
partner     0b.01.5     0b1   5   SA:A   0   SAS 10000 0/0           858483/1758174768
partner     0b.01.6     0b1   6   SA:A   0   SAS 10000 0/0           858483/1758174768
partner     0b.01.3     0b1   3   SA:A   0   SAS 10000 0/0           858483/1758174768
partner     0b.01.4     0b1   4   SA:A   0   SAS 10000 0/0           858483/1758174768
partner     0b.01.2     0b1   2   SA:A   0   SAS 10000 0/0           858483/1758174768
partner     0b.01.1     0b1   1   SA:A   0   SAS 10000 0/0           858483/1758174768
partner     0b.01.0     0b1   0   SA:A   0   SAS 10000 0/0           858483/1758174768
partner     0a.00.2     0a0   2   SA:B   0   SSD   N/A 0/0           95396/195371568
partner     0a.00.0     0a0   0   SA:B   0   SSD   N/A 0/0           95396/195371568
partner     0a.00.4     0a0   4   SA:B   0   SSD   N/A 0/0           95396/195371568

On the other I haven't saw this message. I'm pretty sure the setup of the disks aren't done that well, but now it is like it is :-(

Here's the output of the second filer:

vol status -r  

Aggregate aggrsata (online, raid_dp, hybrid_enabled) (block checksums)

  Plex /aggrsata/plex0 (online, normal, active)

    RAID group /aggrsata/plex0/rg0 (normal, block checksums)

      RAID Disk    Device      HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------    ------      ------------- ---- ---- ---- ----- --------------    --------------

      dparity     0a.00.6     0a    0   6   SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      parity      0a.00.11    0a    0   11  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.12    0a    0   12  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.13    0a    0   13  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.14    0a    0   14  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.15    0a    0   15  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.16    0a    0   16  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.17    0a    0   17  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.18    0a    0   18  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.19    0a    0   19  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.20    0a    0   20  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.21    0a    0   21  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.22    0a    0   22  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

Aggregate aggrsas (online, raid_dp, hybrid) (block checksums)

  Plex /aggrsas/plex0 (online, normal, active)

    RAID group /aggrsas/plex0/rg0 (normal, block checksums)

      RAID Disk    Device      HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------    ------      ------------- ---- ---- ---- ----- --------------    --------------

      dparity     0b.01.0     0b    1   0   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      parity      0b.01.1     0b    1   1   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      data        0b.01.2     0b    1   2   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      data        0b.01.3     0b    1   3   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      data        0b.01.4     0b    1   4   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      data        0b.01.5     0b    1   5   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      data        0b.01.6     0b    1   6   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      data        0b.01.7     0b    1   7   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      data        0b.01.8     0b    1   8   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      data        0b.01.9     0b    1   9   SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

      data        0b.01.10    0b    1   10  SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

    RAID group /aggrsas/plex0/rg1 (normal, block checksums)

      RAID Disk    Device      HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------    ------      ------------- ---- ---- ---- ----- --------------    --------------

      dparity     0a.00.0     0a    0   0   SA:A   0   SSD   N/A 95146/194859520   95396/195371568

      parity      0a.00.1     0a    0   1   SA:A   0   SSD   N/A 95146/194859520   95396/195371568

      data        0a.00.2     0a    0   2   SA:A   0   SSD   N/A 95146/194859520   95396/195371568

Aggregate aggr0 (online, raid4) (block checksums)

  Plex /aggr0/plex0 (online, normal, active)

    RAID group /aggr0/plex0/rg0 (normal, block checksums)

      RAID Disk    Device      HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      ---------    ------      ------------- ---- ---- ---- ----- --------------    --------------

      parity      0a.00.8     0a    0   8   SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

      data        0a.00.10    0a    0   10  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk    Device      HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

---------    ------      ------------- ---- ---- ---- ----- --------------    --------------

Spare disks for block checksum

spare       0b.01.11    0b    1   11  SA:B   0   SAS 10000 857000/1755136000 858483/1758174768

spare       0a.00.23    0a    0   23  SA:A   0  BSAS  7200 847555/1735794176 847884/1736466816 (not zeroed)

spare       0a.00.3     0a    0   3   SA:A   0   SSD   N/A 95146/194859520   95396/195371568

spare       0a.00.4     0a    0   4   SA:A   0   SSD   N/A 95146/194859520   95396/195371568

spare       0a.00.5     0a    0   5   SA:A   0   SSD   N/A 95146/194859520   95396/195371568

Partner disks

RAID Disk    Device      HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

---------    ------      ------------- ---- ---- ---- ----- --------------    --------------

partner     0a.00.7     0a    0   7   SA:A   0  BSAS  7200 0/0               847884/1736466816

partner     0a.00.9     0a    0   9   SA:A   0  BSAS  7200 0/0               847884/1736466816

Any advice how to get solved this issue or any advice what needs to be reconfigured

Any help is highly appreciated

Cheers

Michael

Re: Error message: There are not enough spare disks

I'm trying to figure how you got yourself into this pickle.. You got a little mess on your hands..

It looks like you have three types of disk in this filer, SSD, SAS and SATA.

You have aggr0 as raid-4, with such a small system, you might have wanted to spec this filer differently

You essentially have only spare of each drive and you aren't balanced properly across the cluster.    You should slap your netapp sales rep for specing this system like this.

Re: Error message: There are not enough spare disks

Hi,

that's right. SSD, SAS, SATA.

As far as I know is the SSD used as fast cache. And then there are are dedicated aggregates for SATA and SAS.

The Setup isn't balanced at all, it is more a ACTIVE/PASSIVE setup.

Problem is that I'm not that familiar with NetApp and the guy who did it, isn't here anymore.

Anything you could recommend to change?

Re: Error message: There are not enough spare disks

You're stuck my man.  First, you need to zero the spares (disk zero spares)

Now, I know why the guy who set this up left, he has no clue what he was doing.

I'm counting

18 x sata

6 ssd's

That looks like it's in your internal filer

then you added a shelf for SAS and you have 12 SAS drives

What i would do is roll up aggr0 into the sas aggr, (moving root volume , etc..)

Destroy aggr0, and zero spares, and then move one spare to the other node


Therefore, aggr0 on the other side is raid4 with one spare. (should be using raid-dp, but your stuck)

Also you are burning SSD drives with no use, maybe try to swap two of them out, since you only need one spare for your flashpool.

But, i would call netapp support to verify, don't take just my word for it