e5760 -failure ppd pool and volumes

[Dmirti_Mayorov]

Hi

In e5760 (120 hdd - 2 dpp pool -11 volumes
we do not understand  the reason  -at this moment all - ddp pool and volumes failed

in HW -only 1 hdd failed

Is it possible to recover data in this case? 

DISK POOLS------------------------------

SUMMARY

Total Disk Pools: 2
Total Capacity: 1,293.070 TB Usable, 1,276.445 TB Used
Total Free Capacity: 16.625 TB
Status: 0 Optimal, 2 Non Optimal

Name Status Usable Capacity Used Capacity Free Capacity Preservation Capacity Drive/Media Type Volumes DA Capable
vidanl_ddp1 Failed 646.535 TB 630.000 TB (97%) 16.535 TB 42.748 TB(3 Drives) SAS, Hard Disk Drive 6 Yes
vidanl_ddp2 Failed 646.535 TB 646.445 TB (100%) 92.000 GB 42.748 TB(3 Drives) SAS, Hard Disk Drive 5 Yes

SUMMARY

Number of volumes: 11

Name Thin Provisioned SSD Cache Status Capacity Accessible by Source
ddp1-vid3301_0 No No Failed 10.000 TB Host srv-vid3301 Disk Pool vidanl_ddp1
ddp1-vid3302 No No Failed 200.000 TB Host srv-vid3302 Disk Pool vidanl_ddp1
ddp1-vid3302_1 No No Failed 140.000 TB Host srv-vid3302 Disk Pool vidanl_ddp1
ddp1-vid3302_2 No No Failed 40.000 TB Host srv-vid3302 Disk Pool vidanl_ddp1
ddp1-vid3303 No No Failed 200.000 TB Host srv-vid3303 Disk Pool vidanl_ddp1
ddp1-vid3303_2 No No Failed 40.000 TB Host srv-vid3303 Disk Pool vidanl_ddp1
ddp1-vid3304 No No Failed 6,600.000 GB Host srv-vid3304 Disk Pool vidanl_ddp2
ddp2-anl3303 No No Failed 100.000 TB Host srv-anl3303 Disk Pool vidanl_ddp2
ddp2-vid3302 No No Failed 200.000 TB Host srv-vid3302 Disk Pool vidanl_ddp2
ddp2-vid3303 No No Failed 200.000 TB Host srv-vid3303 Disk Pool vidanl_ddp2
ddp2-vid3303_1 No No Failed 140.000 TB Host srv-vid3303 Disk Pool vidanl_ddp2

[elementx]

DDP can tolerate 2 or 3 disks failures depending on config, assuming you have reserve capacity to "restripe" in case of a failure.

I don't often look at CLI output but are these DDPs full (97%, 100%)?

> vidanl_ddp1 Failed 646.535 TB 630.000 TB (97%) 16.535 TB 42.748 TB(3 Drives) SAS, Hard Disk Drive 6 Yes
> vidanl_ddp2 Failed 646.535 TB 646.445 TB (100%) 92.000 GB 42.748 TB(3 Drives) SAS, Hard Disk Drive 5 Yes

I wouldn't reboot controllers or touch anything - get Support to call you and let them figure it out for yu!

[Dmirti_Mayorov]

Yes -  DDPs full (97%, 100%) - but user used only thick volumes and no used snapshot

I don't think it's the biggest problem right now.

Unfortunately the system does not have an active contract  - may be you have any idea how to recover pool?

[elementx]

Well, not all forum members have access to Support KBs, so suggestions may not be based on facts...

If you follow recommendations from the forums you can make things worse - that's why I suggested it's best to not touch anything if you can't know for sure that it will help.

I  found a similar case here, no resolution, but a response said it can be quite complicated to recover:

https://community.netapp.com/t5/EF-E-Series-SANtricity-and-Related-Plug-ins/DDP-FAILED/m-p/145875

You could try random things such as rolling controller restart, but I don't now if that would help or make things worse.

[Dmirti_Mayorov]

Yes - i read this case - but - it's have not solution(