ONTAP Hardware

There are not enough spare disks. /aggr0/plex0/rg2:

msantander
13,709 Views

Hi everyone,

 

I'm in a big truoble. I've got 4 spare disks, but my raid i degraded. How can I solve this?

 

I'm using FAS3020 wits system 7.2.6

 

NETAPP> sysconfig -r
Aggregate aggr0 (online, raid_dp, degraded) (block checksums)
  Plex /aggr0/plex0 (online, normal, active)
    RAID group /aggr0/plex0/rg0 (normal)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0c.16   0c    1   0   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      parity    0c.18   0c    1   2   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0a.21   0a    1   5   FC:A   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.17   0c    1   1   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.28   0c    1   12  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0a.26   0a    1   10  FC:A   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.25   0c    1   9   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.24   0c    1   8   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.23   0c    1   7   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.22   0c    1   6   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.20   0c    1   4   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.19   0c    1   3   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0a.29   0a    1   13  FC:A   -  ATA   7200 423111/866531584  423889/868126304

    RAID group /aggr0/plex0/rg1 (normal)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0a.32   0a    2   0   FC:A   -  ATA   7200 423111/866531584  423889/868126304
      parity    0c.33   0c    2   1   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.34   0c    2   2   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0a.35   0a    2   3   FC:A   -  ATA   7200 423111/866531584  423889/868126304
      data      0a.36   0a    2   4   FC:A   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.37   0c    2   5   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.38   0c    2   6   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0a.39   0a    2   7   FC:A   -  ATA   7200 423111/866531584  423889/868126304
      data      0a.40   0a    2   8   FC:A   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.41   0c    2   9   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.42   0c    2   10  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.43   0c    2   11  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.44   0c    2   12  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.58   0c    3   10  FC:B   -  ATA   7200 423111/866531584  423889/868126304

    RAID group /aggr0/plex0/rg2 (degraded)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0c.57   0c    3   9   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      parity    0c.49   0c    3   1   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.27   0c    1   11  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.48   0c    3   0   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.60   0c    3   12  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.45   0c    2   13  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.51   0c    3   3   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.50   0c    3   2   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.56   0c    3   8   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0a.54   0a    3   6   FC:A   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.52   0c    3   4   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      FAILED          N/A                   423111/866531584

Aggregate aggr1 (online, raid_dp) (block checksums)
  Plex /aggr1/plex0 (online, normal, active)
    RAID group /aggr1/plex0/rg0 (normal)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0c.64   0c    4   0   FC:B   -  ATA   7200 847555/1735794176 847827/1736350304
      parity    0a.70   0a    4   6   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0c.75   0c    4   11  FC:B   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0a.68   0a    4   4   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0c.67   0c    4   3   FC:B   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0a.72   0a    4   8   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0c.65   0c    4   1   FC:B   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0c.76   0c    4   12  FC:B   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0a.69   0a    4   5   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0a.74   0a    4   10  FC:A   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0c.73   0c    4   9   FC:B   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0a.66   0a    4   2   FC:A   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0c.71   0c    4   7   FC:B   -  ATA   7200 847555/1735794176 847827/1736350304
      data      0a.61   0a    3   13  FC:A   -  ATA   7200 847555/1735794176 847827/1736350304


Spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block or zoned checksum traditional volumes or aggregates
spare           0a.59   0a    3   11  FC:A   -  ATA   7200 847555/1735794176 847827/1736350304
spare           0a.77   0a    4   13  FC:A   -  ATA   7200 847555/1735794176 847827/1736350304
spare           0c.53   0c    3   5   FC:B   -  ATA   7200 847555/1735794176 847827/1736350304
spare           0c.55   0c    3   7   FC:B   -  ATA   7200 847555/1735794176 847827/1736350304

1 ACCEPTED SOLUTION

aborzenkov
10,690 Views

I believe I found it. Replacing 500GB disk with 1TB disk will cause total physical capacity of aggregate to exceed 16TB and this is not allowed in Data ONTAP 7.2 You actually must have seen "The reconstruction cannot start in RAID group [raid group]: Operation would make volume too big" in logs ...

 

So yes, replacing one of 1TB spare disks with 500GB disk would help in this case. Or you could upgrade to 7.3 which does not count parity disks against total physical capacity ...

View solution in original post

30 REPLIES 30

msantander
5,076 Views

I'm also looking for 500 GB disks on ebay. But seems that depending on size disk, reference are different. Is this correct?

 

NetApp X 267 A-R5 / 500 GB

 

NetApp X 269 A-R5 / 1 Tb

 

If this is correct, I'll buy some x267 disks and I'll change the spare ones.

 

Thanks.

aborzenkov
6,138 Views

Before wastingspending money I'd first try reseating spare disks; may be it will trigger rebuild.

msantander
6,130 Views

Hi,

 

How can I do that?. Can you please tell me the command line for reseating spare disks?

 

 

Thanks.

aborzenkov
6,127 Views

I mean physically reseat drive - pull out, wait a minute, plug in.

gavin_meadows
6,126 Views

You need to physically remove and then insert the failed disk, but as you said there is no amber light on any of the disks we first need to get the disk id of the failed disk which is what most of the latest post have been based around.

 

If you know the disk id you can make the light blink amber and then reseat it:

 

fas01> priv set advanced
Warning: These advanced commands are potentially dangerous; use
them only when directed to do so by NetApp
personnel.
fas01*> blink_on <disk_name>

 

A long winded way to find the disk id would be the process of elimination.  ie list out all of the disk id's that you can see and then identify the missing one....

aborzenkov
6,122 Views

 list out all of the disk id's that you can see and then identify the missing one....

There are 56 disks in 4 shelves. Nothing is missing ...

gavin_meadows
6,110 Views

you are missing the disk id NOT a disk:

 

RAID group /aggr0/plex0/rg2 (degraded)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0c.57   0c    3   9   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      parity    0c.49   0c    3   1   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.27   0c    1   11  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.48   0c    3   0   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.60   0c    3   12  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.45   0c    2   13  FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.51   0c    3   3   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.50   0c    3   2   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.56   0c    3   8   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      0a.54   0a    3   6   FC:A   -  ATA   7200 423111/866531584  423889/868126304
      data      0c.52   0c    3   4   FC:B   -  ATA   7200 423111/866531584  423889/868126304
      data      FAILED          N/A                   423111/866531584

 

 

By listing all the id's you should spot the missing one.  As an example  the sequence might go 0c.50 0c.51 0c.52 0c.54 0c.55 0c.56  - the missing disk id = 0c.53

 

You would need to do this for all your disk id's on the system and familiarise yourself with the disk id ranges for each shelf in a loop

aborzenkov
6,105 Views

By listing all the id's you should spot the missing one.


You may be surprised, but I have actually done it before replying.

msantander
6,098 Views

So...how can i find the failed ones?

 

I'm really worried with this. Seems there's no way of finding failed disks.

 

Thks.

aborzenkov
6,093 Views

So...how can i find the failed ones?

You can find it by comparing ASUP before failure or reading logs. But it is the wrong question - what really matters is not which of spare drives was part of aggregate, but why aggregate reconstruction does not start.

Public