Disk Failure - DS212C (FAS2520)

[Reuben]

I have both the Activity and Attention LED's off in one of the disks - NetApp DS212C disk shelf (FAS2520). The disk is NetApp 108-00270+B0 - 2TB 7.2K RPM 64MB Cache SATA 3.5" encased in a NetApp 111-00734+C0 - 2TB Hard Drive Caddy with NetApp P003464-01a - Emulex SATA - SAS Server Interposer Board.

What could be wrong? The disk or the carrier / caddy. 

[maffo]

What is the status of the disk in ONTAP? Is is failed, unassigned, etc.?

[paul_stejskal]

Event log show say anything? (might need set diag to see anything)

[Reuben]

It is the disk 0a.00.1 in Bay 1 as inserted below 👇

[paul_stejskal]

Those screenshots don't tell me anything.

It might be quicker to open a case.

[Reuben]

Am sorry Paul - am still very green with NetApp and new here. By "open a case" you mean? 

[paul_stejskal]

Do you have an active support contract? Sorry we usually assume you have a slight knowledge of how to work the system, so I apologize for the confusion.

What is your serial number (send PM)? I'll check.

If you need it, you can do:

1. SSH in using PuTTY or other terminal, same login as System Manager.
2. Run "node show -instance"
3. The serial # will be output there.

Also, the support site mysupport.netapp.com should have more info on your system. If you aren't registered on your system you can call our Support line once you get the serial # and have your contact info registered.

Thank you.

[Reuben]

Thankyou. We currently do not have any active support contract. 

Serial Number Inboxed

[AlainTansi]

Have you tried reseating the IOM? 

It may not be making good contact with the mid-plane connectors.

From the screenshots, what i can see is that you  are running out of spare disks which that should be addressed separately. 

Please share your feedback

[Reuben]

Thankyou very much for your advice - will try reseating the IOM tomorrow and send feedback. Only un-plugged the disk n plugged it back.

Correction. The hard Disk is in Bay 0 and is not captured in the screenshot. We cannot see it from ONTAP. We can only view disks in Bay 1 through to 11

[paul_stejskal]

[?] Thu May 21 17:25:54 EAT [nodename: pmcsas_timeout_0: sas.adapter.debug:info]: params: {'debug_string': 'One or more (1) PHYs on expander 5:00a098:005a99c:ff are in a bad state.', 'adapterName': '0a'}

It may be a bad connection, or it may be a bad part.  Also, I noticed SAS 0b wasn't hooked up. This will need to be fixed too.

[amans]

Hello,

If you can let me know the serial number, I can dig further.

Regards,

Aman

[amans]

Hi Reuben,

Thank you for sharing the serial number.

So, I see the node was halted back in July 29,2018 and ever since disk 00.0 is in missing state.

I would suggest to pull out the disk 00.0, wait for 2 mins, insert the disk and see if it comes online. It it does not, then you should be looking for a replacement.

Regards,

Aman

[Reuben]

Hi Amans

Pulling out the disk for 2min and sliding it back in didn't change the missing state. We placed an order for a replacement disk complete with the caddy and emulex board. Scheduled for the replacement yesterday but unfortunately same result. The new disk was also missing in ONTAP and the Activity and Attention LED's are still off. 

Can an updated Disk Qualification Package (DQP) resolve this issue. It has been listed as a risk in Active IQ wellness - same as OS and Firmware Upgrade.

[amans]

Hi Reuben,

Thanks for getting back!

Yes, please  upgrade the DQP and also theIOM6E FW and see if that helps.

If the issue continues, then if there is spare disk on partner node then swap 00.00 with spare disk and see if the problem follows or remains with slot 0.

Regards,

Aman

[amans]

Hi Reuben,

Did you get a chance to upgrade the DQP and shelf FW?

Regards,

Aman

[not_a_Lone_wolf]

Hi,

The fact that none of the LED’s are active on that disk indicates a problem with the hardware and not software. You can try updating DQP and firmware but I don’t think it would be of much help since the disk is not up yet as per ONTAP , and it fails to detect the disk.

Please try below:

1> Swap the disk with a known good spare disk and check if the swapped disk and spare disks are detected in their respective drive slots.

2> If the known good spare drive is not detected in Bay 0, then the drive slot is faulty, maybe broken latch. If the drive is not detected in the swapped slot, then the drive itself is faulty. Replace it( I am assuming it’s an internal disk and hence do not reseat the PCM without expert opinion of Support)

Please let me know if it works.