Noticed two failed drives, so went into the NetApp GUI and "removed" them, which errored out, then physically removed the failed drives. Now all the spare drives have changed names and the spares are in different locations in the appliance. There are twelve spare drives, nine of which are the correct model to replace the failed disks. The system seems ok. The syslog said the failed drives should be removed, so I think pulling them out was the right thing to do. I'm just unsure how to proceed from here. Here are the logs from when I removed the drives...system complaining about the drives at 8AM. I removed the failed drives at 10:25AM.
Wed Sep 19 08:03:43 PDT [raid.fdr.reminder:warning]: Failed Disk 0g.34 Shelf 2 Bay 2 [NETAPP X266_MTOMC320PSX RBTV] S/N [A813LYRE] is still present in the system and should be removed.
Wed Sep 19 08:03:43 PDT [raid.fdr.reminder:warning]: Failed Disk 0f.24 Shelf 1 Bay 8 [NETAPP X266_MTOMC320PSX R5VV] S/N [A82D61TE] is still present in the system and should be removed.
Wed Sep 19 09:00:00 PDT [kern.timed.adjust:info]: server 'time' reports the appliance date is fast by 0.003 seconds. Adjusting date.
Wed Sep 19 09:00:00 PDT [kern.uptime.filer:info]: 9:00am up 22 days, 16:14 5557450451 NFS ops, 352557809 CIFS ops, 67 HTTP ops, 0 FCP ops, 0 iSCSI ops
Wed Sep 19 09:53:48 PDT [HTTPPool04:warning]: HTTP Authentication from 10.96.10.73 using NTLM failed
Wed Sep 19 10:00:00 PDT [kern.uptime.filer:info]: 10:00am up 22 days, 17:14 5567518946 NFS ops, 353141547 CIFS ops, 86 HTTP ops, 0 FCP ops, 0 iSCSI ops
Wed Sep 19 10:25:27 PDT [ispfc_init_6:error]: Failed to build FCAL map on Fibre Channel adapter 0g -- resetting adapter.
Wed Sep 19 10:25:27 PDT [ispfc_init_7:error]: Failed to build FCAL map on Fibre Channel adapter 0h -- resetting adapter.
Wed Sep 19 10:25:38 PDT [raid.disk.missing:info]: Disk 0g.34 Shelf 2 Bay 2 [NETAPP X266_MTOMC320PSX RBTV] S/N [A813LYRE] is missing from the system
Wed Sep 19 10:26:20 PDT [ispfc_init_4:error]: Failed to build FCAL map on Fibre Channel adapter 0e -- resetting adapter.
Wed Sep 19 10:26:20 PDT [ispfc_init_5:error]: Failed to build FCAL map on Fibre Channel adapter 0f -- resetting adapter.
Wed Sep 19 10:26:31 PDT [raid.disk.missing:info]: Disk 0f.24 Shelf 1 Bay 8 [NETAPP X266_MTOMC320PSX R5VV] S/N [A82D61TE] is missing from the system
Wed Sep 19 10:26:40 PDT [ses.channel.rescanInitiated:info]: Initiating rescan on channel 0e.
Wed Sep 19 10:26:44 PDT [ses.channel.rescanInitiated:info]: Initiating rescan on channel 0f.
Wed Sep 19 10:26:54 PDT [raid.disk.offline:notice]: Marking Disk /aggr1/plex0/rg1/0e.19 Shelf 1 Bay 3 [NETAPP X266_MTOMC320PSX R5VV] S/N [A82RLXXE] offline.
Wed Sep 19 10:27:15 PDT [raid.disk.online:notice]: Onlining Disk /aggr1/plex0/rg1/0f.19 Shelf 1 Bay 3 [NETAPP X266_MTOMC320PSX R5VV] S/N [A82RLXXE].
Wed Sep 19 11:00:00 PDT [kern.timed.adjust:info]: server 'time' reports the appliance date is fast by 0.001 seconds. Adjusting date.
Wed Sep 19 11:00:00 PDT [kern.uptime.filer:info]: 11:00am up 22 days, 18:14 5578104722 NFS ops, 353701408 CIFS ops, 87 HTTP ops, 0 FCP ops, 0 iSCSI ops
Wed Sep 19 11:01:10 PDT [raid.disk.offline:notice]: Marking Disk /aggr2/plex0/rg1/0h.40 Shelf 2 Bay 8 [NETAPP X266_MTOMC320PSX R5VV] S/N [A82RW37E] offline.
Wed Sep 19 11:01:40 PDT [raid.disk.online:notice]: Onlining Disk /aggr2/plex0/rg1/0h.40 Shelf 2 Bay 8 [NETAPP X266_MTOMC320PSX R5VV] S/N [A82RW37E].
This is how the failed drives were showing before removal:
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
failed 0f.24 0f 1 8 FC:B - ATA 5400 274400/561971200 274540/562258784
failed 0g.34 0g 2 2 FC:A - ATA 5400 274400/561971200 274540/562258784
These are the current spares:
| RAID Disk | Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) | Phys (MB/blks) |
| --------- | ------ ------------- ---- ---- ---- ----- -------------- | -------------- |
Spare disks for block or zoned checksum traditional volumes or aggregates
| spare | 0e.18 0e | 1 2 FC:A - ATA 5400 274400/561971200 274540/562258784 |
| spare | 0e.21 0e | 1 5 FC:A - ATA 5400 274400/561971200 274540/562258784 |
| spare | 0e.22 0e | 1 6 FC:A - ATA 5400 274400/561971200 274540/562258784 |
| spare | 0e.28 0e | 1 12 FC:A - ATA 5400 274400/561971200 274540/562258784 |
| spare | 0e.29 0e | 1 13 FC:A - ATA 5400 274400/561971200 274540/562258784 |
| spare | 0f.73 0f | 4 9 FC:B - ATA 5400 274400/561971200 274540/562258784 |
| spare | 0g.32 0g | 2 0 FC:A - ATA 5400 274400/561971200 274540/562258784 |
| spare | 0a.73 0a | 4 9 FC:A - ATA 7200 274400/561971200 274540/562258784 |
| spare | 0b.33 0b | 2 1 FC:B - ATA 7200 274400/561971200 274540/562258784 |
| spare | 0c.75 0c | 4 11 FC:A - ATA 7200 635555/1301618176 635858/1302238304 |
| spare | 0c.92 0c | 5 12 FC:A - ATA 7200 847555/1735794176 847827/1736350304 |
| spare | 0c.109 0c | 6 13 FC:A - ATA 7200 847555/1735794176 847827/1736350304 |
