RAID DP Disk Failure

[edlam2000]

I know that RAID_DP allows 2 disk failure.

Let's say there's already a disk with double parity failed, and theretically, it allows another disk to fail, but if another disk fails, will the system shut itself down automatically after 24 hours for its own data-protection?

[radek_kubka]

It will not.

An automatic RAID rebuild will start as soon as the first drive fails using one of the hot-spare disks.

[billshaffer]

Assuming there are spares available.  If the system is unable to rebuild, then yes, it will shut itself down in 24 hours, though this can be altered with the raid.timeout option.

Bill

[radek_kubka]

True. A system with no spares is rather an uncommon view though.

[edlam2000]

Assuming that it's the controller shut down the raid group, so how to restart it after its shutdown?

[saranraj456]

once the disks are replaced , have to boot up the filers.

[edlam2000]

But there is no spare any more and warranty has expired.

I just want to know when one of the controllers shutdown the raid group, how to restart it?

[saranraj456]

pull the failed disks out & start the filer.

have to change the raid.timeout option after starting.

[edlam2000]

Can the filer controller be remotely started or do I have to physically be there to push the button?

[saranraj456]

It can be started from the loader prompt I believe.

[shaunjurr]

Arrange to have someone toss it in the bin or dismantle it for spare parts...  You aren't far away from losing all of your data anyway, so this way you can just eliminate the element of surprise.

[SCOTT_V_RONTALE]

Hi Bill

Is there a document or white report that you can share that if the system is unable to rebuild, it will shut down in 24 hours?

[radek_kubka]

e.g. this one:

https://library.netapp.com/ecmdocs/ECMP1196986/html/GUID-D8B1E3D3-A0DD-4A1D-84EF-DE1DA2B9D3B7.html

[SCOTT_V_RONTALE]

thank you for this one sir! Another is what if 3 failed disk is down with out applying the raid.timeout option what could happend to the system? i know there will be a data loss or 24 hours shut down but is there a TR or white paper that can support this.

Thank You in advance!

[billshaffer]

A few things to consider.  Remember that there can be multiple aggregates on the system, each of which can consist of multiple raid groups - aggr status -r will show the raid groups.

You can have two failed drives in every aggregate and still not lose data, because each aggregate is a seperate entity.  In addition, you can lose two drives in each raid group of a _single_ aggregate without losing data, because each raid group is in its own raid-dp setup.  So, if you have an aggregate with 4 raid-dp raid groups, you could lose 8 drives, as long as two come from each raid group, without losing data.  For the record, I've seen this - an entire shelf powered off, but only two drives from that shelf were in any single raid group, so no data loss.

If you lost a third drive in a raid-dp raid group, that raid group would fail and the aggregate would go offline, and you'd lose data.  Not sure if a failover would happen - assuming dual path HA, if the drive is down on one controller it'll be down on the other also.  Also not sure if raid.timeout would shut the system down - I don't know that an offline aggregate constitutes a degraded state.  Degraded implies that it's still running, which it isn't, technically.

I did a quick look, but couldn't find anything to describe what happens when you lose that third drive.  This link (https://kb.netapp.com/support/index?page=content&id=3013638) hints that the controller will panic, but it is a different situation (media error on rebuild).  One of the links is to article 2014172, which says to call support in the rare event that there was actually another disk failure - so maybe they don't publish what happens in that case.

Bill

[MSTORAGE_1986]

I think you should change degrade options 72 hours however, your filer not panic & with you can raise a case with netapp also.