EF & E-Series, SANtricity, and Related Plug-ins

E5700 volume degraded after disk change

jtejada
1,243 Views

Hi all,

 

After drive replacement on a E5700 system, reconstruction finished fine but volume remains "degraded".  As you can see all drives are in "Optimal" state. How to recover volume to optimal state?

 

DETAILS


Name: vg1

Status: Degraded
Capacity: 85.496 TB
Current owner: Controller in bay A

Quality of Service (QoS) Attributes

RAID level: 6
Drive media type: Hard Disk Drive
Drive interface type: Serial Attached SCSI (SAS)
Shelf loss protection: No
Drawer Loss Protection: Yes
Secure-Capable: No
Secure-Enabled No
Data Assurance (DA) capable: Yes
DA enabled volume present: Yes


Total Volumes: 1
Thick volumes: 1
Repository volumes: 0
Free Capacity: 0.000 MB

Associated drives - present (in piece order)
Total drives present: 10

Shelf Drawer Bay
0 1 3
99 2 6
0 3 9
99 4 1
0 5 4
99 1 7
0 2 10
99 3 2
0 4 5
99 5 8


SHELF, DRAWER, BAY STATUS CAPACITY MEDIA TYPE INTERFACE TYPE CURRENT DATA RATE PRODUCT ID FIRMWARE VERSION CAPABILITIES
0, 1, 3 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA
99, 2, 6 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA
0, 3, 9 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA
99, 4, 1 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA
0, 5, 4 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA
99, 1, 7 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA
0, 2, 10 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA
99, 3, 2 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA
0, 4, 5 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA
99, 5, 8 Optimal 10.692 TB Hard Disk Drive SAS 12 Gbps HUH721212AL5204 NE00 DA

 

Regards,

1 ACCEPTED SOLUTION

ahmadm
827 Views

Yes, you would essentially reconstruct the drive. To do that, you can run this command in the controller shell:

 

> startDriveReconstruction_MT 0x10025

View solution in original post

7 REPLIES 7

ahmadm
1,203 Views

Assuming reconstruction has completed for the drive, the volume should go to optimal. However, the fact they are not suggests there could be another failed piece in the RAID group. The failure could be logical, so while the drive is optimal, the logical piece on said drive is failed.

 

I am not sure what level of access you have to the system, but there are some shell commands we could capture to find out which logical piece is failed.

 

 

jtejada
1,169 Views

Please could you let me know these commands and I will verify.

ahmadm
976 Views

If you are familiar with the controller shell, the commands are:

 

  • vdmShowOosPieces
  • evfShowVol 0
  • vdmShowVGInfo

jtejada
915 Views

Here the outputs, I see OOS piece failed.

 

 
-> vdmShowOosPieces
 
*************************** PieceManager OOS Pieces **********************
RAIDVolume:0x1(0x1660b4850), 1 pieces OOS
  Ord:0x0004 Addr:0x00000001656cc3c0 state:FAILED (Active)
  0x0 - Piece Failed Timestamp: 1970 Jan 01  00:00:00
 
 
 
**************************************************************************
value = 76 = 0x4c = 'L'
-> evfShowVol 0
Volume 0x0(RAIDVolume)
 6 VolumeListener(s)
     0xc000(MediaScanAgent)
     N3vdm11VolumeGroupE
     N3vdm17RAIDVolumeManagerE
     N3spm18SymbolEventManagerE
     N3evf10CmdHandlerE
     N3evf22VolumeUserLabelManagerE
 0 Children
Volume 0x0 Attributes:
     Volume Type:       8+2 RAID 6
     User Label:        d0
     WWN:               6d039ea0000201a8000001a05dd3fc69
     Address:           0x0000000165e0f598
     Devnum:            0x10000000
     Capacity:          183601463296 blocks
     BlockSize:         512 bytes
     LargeIoSize:       4096 blocks
     MaxBkgdIoSize:     4096 blocks
     Has Extents:       false
     Ownership:         This controller
     PreferredPath:     This controller
     Transferring:      false
     StopIOInProgress:  false
     Suspended:         0
     Quiesce Count:     0
     Quiesce Both Ctl:  0 (Local: 0/Alt: 0)
     State:             RV_OPTIMAL
     Unreadable Sectors:Not Present
     App Tag:           0x0000
     App Tag Owned:     false
     Protection Type:   1
     PI Reporting:      enabled
     PI Mirroring:      enabled
     isSecure:          false
     isSecureCapable:   false
     isFIPS:            false
     isExopActive:      true
     isPersistent:      true
     isTCGSSupported:   true
     skipped URS:       0
     NonCriticalTransferAllowed: true
     Permissions:       CONFIG=Y            CPYsrc=Y            CPYTGT=Y
                        DRIVESACCESSIBLE=Y  FORMAT=Y            FORMATTING=N
                        IMPORT=Y            MAP=Y               MIRPRI=Y
                        MIRSEC=Y            MISSING=N           MOVE_RELATED=Y
                        NON_RED=Y           OFFLINE=N           READ=Y
                        READY=Y             RECONFG=Y           REP_NOT_ON_PREF=Y
                        SNAP=Y              TRANSFER=Y          WRITE=Y
                        ROLLBACK=Y          MIRACTV=Y           NEWPIT=Y
                        DVE=Y               DSS=Y               CONCAT=N
                        FRC=Y               PITGROUP=Y          ASYNC_MIRPRI=Y
                        ASYNC_MIRSEC=Y      DSP=Y               CPC=Y
                        THIN_MANEXP=Y       THIN_GROWPARAM=Y    XCPY=Y
                        4KDSS=Y             VAULT=Y             RESTORE=Y
                        VAULT_GRP=Y         ALBTRANSFER=Y       PRIORITYXFER=Y
     Activity Count:    1
     QueueDepth:        0
     EUI:               000001a05dd3fc69d039ea00000201a8
 
 
     IAF Aborted:               0
     Segment Size:              256 blocks
     Stripe Size:               2048 blocks
     Drive Recovery Mode:       RECOVERY_INVALID
     PreConfigedPI:             0
     isVolumeCopyActive:        False
     isVolumeCopyInsertPI:      False
     DSS PreAllocation:         Enabled
     AbsoluteMaxDSSSegSize:     4096
     Pre-Read Redundancy Check: Disabled
     Retry on Mismatch:         Enabled
     Repair Writes:             Disabled
     Flash Read Cache:          Disabled
     Atomic Write Support:      Disabled
     Volume Usage:              STANDARD
     Initialization:            Completed
     Offline Type:              0
     NoDemandFlush:             False
     Read Ahead:                1
     Fast Write:                1
     Sync Age:                  2500
     Cache Flush Modifier:      8
     Min Warn Flush Modifier:   8
     Cache Granularity:         64
     Ignore FUA:                Host type(s) - 0 1 2 5 6 7 8 9 10 15 17 22 23 24 25 26 27 28 29
 
     Media Scan Settings: Enabled with Parity Check, but no Repair.
          Period = 30 day(s). Current rate = 88542 blocks/sec.
 
     Background Task:    MEDIA SCAN at block 0x0211a246800 with 77% complete
     Background Delay:   0.1 seconds per 4096 blocks
 
 
 
*** Volume Group Info ***
 
    WWN        : 6d039ea0000201a8000001945dd3fc4a
    RAID Level : RAID 6
    Boundary   : 142172514304 0x211a246800
    Persisted  : 0  0x0
    VG Label   : vg0
    Address    : 0x000000024669b6b0
    Drive Count: 10
    Secure     : False
    PI Capable : True - 2
    Media Type : HDD
 
     Volumes on this Group:
Volume Count:                      1
0x00000 [O]
 
*** Pieces ***
 
    Offset: 0x0                    Length:     0x557f00000 (22950182912 dec)
    Count : 10                     Data Count: 8
 
    Piece     Devnum        Address  Tray/Slot   State
        0  0x00010100    0x00000001656ff850    00,01     PieceOptimalState
        1  0x0001000f    0x0000000165796640    99,16     PieceOptimalState
        2  0x0001011e    0x00000001656e8f58    00,31     PieceOptimalState
        3  0x0001002d    0x00000001656e8e48    99,46     PieceOptimalState
        4  0x00010131    0x0000000165786280    00,50     PieceOptimalState
        5  0x00010004    0x000000016575b9b8    99,05     PieceOptimalState
        6  0x00010113    0x0000000165747760    00,20     PieceOptimalState
        7  0x00010022    0x00000001657475f0    99,35     PieceOptimalState
        8  0x00010126    0x00000001656e0ba0    00,39     PieceOptimalState
        9  0x00010035    0x000000016571a330    99,54     PieceOptimalState
 
StripesPerCachePoolExtent:            16
BlocksPerCachePoolExtent :         32768
 
*** CACHE DEVICE 0 ***
 
     Sector Size     Secs/CacheBlock          Cache Blk Size    Clean Minimum
       512 bytes                  64             32768 bytes 0000208000
Dirty Cache Blks   Dirty Cache High per Pool
            5129   0:12771 1:15775 2:12761
  Hash Data Blks    Hash Parity Blks
          150394                   0
       RD_CACHE       WR_CACHE   WR_CACHE_MIR            FWT
        ENABLED        ENABLED        ENABLED       DISABLED
 
           CWOB  WT_BU_BATT_ON     READ_CACHE    WRITE_CACHE   WRITE_CACHE_MIR
       DISABLED             NO         ACTIVE         ACTIVE            ACTIVE
HIGH_PRIORITY_DEV: NO
Background sync is in progress : NO
CacheBlockMask: ffffffffffffffff
 
Cache Statistics:
                 current       old   current       old
         cache      read      read    parity    parity     disk
 VOL     reads      hits      hits      hits      hits     reads
---- --------- --------- --------- --------- --------- ---------
   0  23089135   3983591       243         0     47308  20092317
 
          read      read
         check     check    insert      full   partial      disk
 VOL      reqs      hits      reqs overwrite overwrite    writes
---- --------- --------- --------- --------- --------- ---------
   0   4014500   2033274    965518      3290    187790         0
 
                            Cache Pool  0       Cache Pool  1       Cache Pool  2  
                         ------------------- ------------------- -------------------
       WB blks   % Dirty   WB blks   % Dirty   WB blks   % Dirty   WB blks   % Dirty
 VOL   current     Limit      high     Limit      high     Limit      high     Limit
---- --------- --------- --------- --------- --------- --------- --------- ---------
   0      5129         2     12771         6     15775         7     12761         6
 
     +---------- cache blocks evicted after ---------+   evicted    blocks
 VOL     fetch       hit  prefetch pftch hit     write     total    in-use
---- --------- --------- --------- --------- --------- --------- ---------
   0  18137518     70172  23524605 29676370  11431433  82840098    150394
 
                prefetch            prefetch  pct
 VOL          hit blocks         miss blocks hits
---- ------------------- ------------------- ----
   0          1896349824          1505574720   55
 
value = 139928058009584 = 0x7f438a3117f0
-> vdmShowVGInfo
Total Volume Groups: ............. 12
 
Seq:1 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:2 / PI Capable:T - 2 / Label:vg0 / VGWwn:6d039ea0000201a8000001945dd3fc4a / RsvdSpace:            5 (GB)
(Active) Drive:0x000000024661e240 devnum:0x00010100 seqNum:1 Tray/Slot:00/01  State:Acc/GrA/Opt
(Active) Drive:0x000000024661e530 devnum:0x0001000f seqNum:2 Tray/Slot:99/16  State:Acc/GrA/Opt
(Active) Drive:0x000000024661e820 devnum:0x0001011e seqNum:3 Tray/Slot:00/31  State:Acc/GrA/Opt
(Active) Drive:0x000000024661eb10 devnum:0x0001002d seqNum:4 Tray/Slot:99/46  State:Acc/GrA/Opt
(Active) Drive:0x00000002466352e0 devnum:0x00010131 seqNum:5 Tray/Slot:00/50  State:Acc/GrA/Opt
(Active) Drive:0x000000024661f0f0 devnum:0x00010004 seqNum:6 Tray/Slot:99/05  State:Acc/GrA/Opt
(Active) Drive:0x000000024661ee00 devnum:0x00010113 seqNum:7 Tray/Slot:00/20  State:Acc/GrA/Opt
(Active) Drive:0x000000024661f6d0 devnum:0x00010022 seqNum:8 Tray/Slot:99/35  State:Acc/GrA/Opt
(Active) Drive:0x000000024661f9c0 devnum:0x00010126 seqNum:9 Tray/Slot:00/39  State:Acc/GrA/Opt
(Active) Drive:0x000000024661fcb0 devnum:0x00010035 seqNum:10 Tray/Slot:99/54  State:Acc/GrA/Opt
Volumes: 0x00000 [O]
 
Seq:2 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:2 / PI Capable:T - 2 / Label:vg1 / VGWwn:6d039ea0000201a8000001955dd3fc4c / RsvdSpace:            5 (GB)
(Active) Drive:0x000000024661ffa0 devnum:0x00010103 seqNum:1 Tray/Slot:00/04  State:Acc/GrA/Opt
(Active) Drive:0x0000000246620290 devnum:0x00010012 seqNum:2 Tray/Slot:99/19  State:Acc/GrA/Opt
(Active) Drive:0x0000000246620580 devnum:0x00010121 seqNum:3 Tray/Slot:00/34  State:Acc/GrA/Opt
(Active) Drive:0x0000000246a3f2d0 devnum:0x00010025 seqNum:4 Tray/Slot:99/38  State:Acc/GrA/Opt
(Active) Drive:0x0000000246620b60 devnum:0x00010134 seqNum:5 Tray/Slot:00/53  State:Acc/GrA/Opt
(Active) Drive:0x000000024683dc48 devnum:0x00010007 seqNum:6 Tray/Slot:99/08  State:Acc/GrA/Opt
(Active) Drive:0x00000002496193c8 devnum:0x00010116 seqNum:7 Tray/Slot:00/23  State:Acc/GrA/Opt
(Active) Drive:0x0000000246621430 devnum:0x0001001a seqNum:8 Tray/Slot:99/27  State:Acc/GrA/Opt
(Active) Drive:0x0000000246621720 devnum:0x00010129 seqNum:9 Tray/Slot:00/42  State:Acc/GrA/Opt
(Active) Drive:0x0000000246621a10 devnum:0x00010038 seqNum:10 Tray/Slot:99/57  State:Acc/GrA/Opt
Volumes: 0x00001 [D]
 
Seq:3 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:3 / PI Capable:T - 2 / Label:vg2 / VGWwn:6d039ea0000201a8000001965dd3fc4d / RsvdSpace:            5 (GB)
(Active) Drive:0x0000000246621d00 devnum:0x00010106 seqNum:1 Tray/Slot:00/07  State:Acc/GrA/Opt
(Active) Drive:0x0000000246621ff0 devnum:0x00010015 seqNum:2 Tray/Slot:99/22  State:Acc/GrA/Opt
(Active) Drive:0x00000002466222e0 devnum:0x00010119 seqNum:3 Tray/Slot:00/26  State:Acc/GrA/Opt
(Active) Drive:0x00000002466225d0 devnum:0x00010028 seqNum:4 Tray/Slot:99/41  State:Acc/GrA/Opt
(Active) Drive:0x00000002466228c0 devnum:0x00010137 seqNum:5 Tray/Slot:00/56  State:Acc/GrA/Opt
(Active) Drive:0x000000024661f3e0 devnum:0x0001000a seqNum:6 Tray/Slot:99/11  State:Acc/GrA/Opt
(Active) Drive:0x0000000246631820 devnum:0x0001010e seqNum:7 Tray/Slot:00/15  State:Acc/GrA/Opt
(Active) Drive:0x0000000246623190 devnum:0x0001001d seqNum:8 Tray/Slot:99/30  State:Acc/GrA/Opt
(Active) Drive:0x0000000246622ea0 devnum:0x0001012c seqNum:9 Tray/Slot:00/45  State:Acc/GrA/Opt
(Active) Drive:0x0000000246623770 devnum:0x0001003b seqNum:10 Tray/Slot:99/60  State:Acc/GrA/Opt
Volumes: 0x00002 [O]
 
Seq:4 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:3 / PI Capable:T - 2 / Label:vg3 / VGWwn:6d039ea0000201a8000001975dd3fc4f / RsvdSpace:            5 (GB)
(Active) Drive:0x0000000246623a60 devnum:0x00010109 seqNum:1 Tray/Slot:00/10  State:Acc/GrA/Opt
(Active) Drive:0x00000002466280e0 devnum:0x0001000d seqNum:2 Tray/Slot:99/14  State:Acc/GrA/Opt
(Active) Drive:0x0000000246624040 devnum:0x0001011c seqNum:3 Tray/Slot:00/29  State:Acc/GrA/Opt
(Active) Drive:0x0000000246624330 devnum:0x0001002b seqNum:4 Tray/Slot:99/44  State:Acc/GrA/Opt
(Active) Drive:0x0000000246624620 devnum:0x0001013a seqNum:5 Tray/Slot:00/59  State:Acc/GrA/Opt
(Active) Drive:0x0000000246624910 devnum:0x00010002 seqNum:6 Tray/Slot:99/03  State:Acc/GrA/Opt
(Active) Drive:0x0000000246624c00 devnum:0x00010111 seqNum:7 Tray/Slot:00/18  State:Acc/GrA/Opt
(Active) Drive:0x0000000246624ef0 devnum:0x00010020 seqNum:8 Tray/Slot:99/33  State:Acc/GrA/Opt
(Active) Drive:0x00000002466251e0 devnum:0x0001012f seqNum:9 Tray/Slot:00/48  State:Acc/GrA/Opt
(Active) Drive:0x00000002466254d0 devnum:0x00010030 seqNum:10 Tray/Slot:99/49  State:Acc/GrA/Opt
Volumes: 0x00003 [O]
 
Seq:5 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:2 / PI Capable:T - 2 / Label:vg4 / VGWwn:6d039ea0000201a8000001985dd3fc50 / RsvdSpace:            5 (GB)
(Active) Drive:0x00000002466257c0 devnum:0x00010101 seqNum:1 Tray/Slot:00/02  State:Acc/GrA/Opt
(Active) Drive:0x0000000246625ab0 devnum:0x00010010 seqNum:2 Tray/Slot:99/17  State:Acc/GrA/Opt
(Active) Drive:0x0000000246625da0 devnum:0x0001011f seqNum:3 Tray/Slot:00/32  State:Acc/GrA/Opt
(Active) Drive:0x0000000246626090 devnum:0x0001002e seqNum:4 Tray/Slot:99/47  State:Acc/GrA/Opt
(Active) Drive:0x0000000246626380 devnum:0x00010132 seqNum:5 Tray/Slot:00/51  State:Acc/GrA/Opt
(Active) Drive:0x0000000246626670 devnum:0x00010005 seqNum:6 Tray/Slot:99/06  State:Acc/GrA/Opt
(Active) Drive:0x0000000246626960 devnum:0x00010114 seqNum:7 Tray/Slot:00/21  State:Acc/GrA/Opt
(Active) Drive:0x0000000246626c50 devnum:0x00010023 seqNum:8 Tray/Slot:99/36  State:Acc/GrA/Opt
(Active) Drive:0x0000000246626f40 devnum:0x00010124 seqNum:9 Tray/Slot:00/37  State:Acc/GrA/Opt
(Active) Drive:0x000000024663a230 devnum:0x00010033 seqNum:10 Tray/Slot:99/52  State:Acc/GrA/Opt
Volumes: 0x00004 [O]
 
Seq:6 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:2 / PI Capable:T - 2 / Label:vg5 / VGWwn:6d039ea0000201a8000001995dd3fc52 / RsvdSpace:            5 (GB)
(Active) Drive:0x0000000246627520 devnum:0x00010104 seqNum:1 Tray/Slot:00/05  State:Acc/GrA/Opt
(Active) Drive:0x0000000246627810 devnum:0x00010013 seqNum:2 Tray/Slot:99/20  State:Acc/GrA/Opt
(Active) Drive:0x0000000246627b00 devnum:0x00010122 seqNum:3 Tray/Slot:00/35  State:Acc/GrA/Opt
(Active) Drive:0x0000000246627df0 devnum:0x00010026 seqNum:4 Tray/Slot:99/39  State:Acc/GrA/Opt
(Active) Drive:0x0000000246632cb0 devnum:0x00010135 seqNum:5 Tray/Slot:00/54  State:Acc/GrA/Opt
(Active) Drive:0x00000002466283d0 devnum:0x00010008 seqNum:6 Tray/Slot:99/09  State:Acc/GrA/Opt
(Active) Drive:0x00000002466286c0 devnum:0x00010117 seqNum:7 Tray/Slot:00/24  State:Acc/GrA/Opt
(Active) Drive:0x00000002466289b0 devnum:0x00010018 seqNum:8 Tray/Slot:99/25  State:Acc/GrA/Opt
(Active) Drive:0x0000000246628ca0 devnum:0x00010127 seqNum:9 Tray/Slot:00/40  State:Acc/GrA/Opt
(Active) Drive:0x0000000246628f90 devnum:0x00010036 seqNum:10 Tray/Slot:99/55  State:Acc/GrA/Opt
Volumes: 0x00005 [O]
 
Seq:7 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:2 / PI Capable:T - 2 / Label:vg6 / VGWwn:6d039ea0000201a80000019a5dd3fc53 / RsvdSpace:            5 (GB)
(Active) Drive:0x000000024662f4e0 devnum:0x00010107 seqNum:1 Tray/Slot:00/08  State:Acc/GrA/Opt
(Active) Drive:0x000000024662f7d0 devnum:0x00010016 seqNum:2 Tray/Slot:99/23  State:Acc/GrA/Opt
(Active) Drive:0x000000024662fac0 devnum:0x0001011a seqNum:3 Tray/Slot:00/27  State:Acc/GrA/Opt
(Active) Drive:0x000000024662fdb0 devnum:0x00010029 seqNum:4 Tray/Slot:99/42  State:Acc/GrA/Opt
(Active) Drive:0x00000002466300a0 devnum:0x00010138 seqNum:5 Tray/Slot:00/57  State:Acc/GrA/Opt
(Active) Drive:0x0000000246630390 devnum:0x0001000b seqNum:6 Tray/Slot:99/12  State:Acc/GrA/Opt
(Active) Drive:0x0000000246630680 devnum:0x0001010c seqNum:7 Tray/Slot:00/13  State:Acc/GrA/Opt
(Active) Drive:0x0000000246630970 devnum:0x0001001b seqNum:8 Tray/Slot:99/28  State:Acc/GrA/Opt
(Active) Drive:0x0000000246630c60 devnum:0x0001012a seqNum:9 Tray/Slot:00/43  State:Acc/GrA/Opt
(Active) Drive:0x0000000246630f50 devnum:0x00010039 seqNum:10 Tray/Slot:99/58  State:Acc/GrA/Opt
Volumes: 0x00006 [O]
 
Seq:8 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:2 / PI Capable:T - 2 / Label:vg7 / VGWwn:6d039ea0000201a80000019b5dd3fc55 / RsvdSpace:            5 (GB)
(Active) Drive:0x0000000246631240 devnum:0x0001010a seqNum:1 Tray/Slot:00/11  State:Acc/GrA/Opt
(Active) Drive:0x0000000246631530 devnum:0x0001000e seqNum:2 Tray/Slot:99/15  State:Acc/GrA/Opt
(Active) Drive:0x0000000246639380 devnum:0x0001011d seqNum:3 Tray/Slot:00/30  State:Acc/GrA/Opt
(Active) Drive:0x00000002492e4468 devnum:0x0001002c seqNum:4 Tray/Slot:99/45  State:Acc/GrA/Opt
(Active) Drive:0x0000000246631e00 devnum:0x0001013b seqNum:5 Tray/Slot:00/60  State:Acc/GrA/Opt
(Active) Drive:0x00000002466320f0 devnum:0x00010000 seqNum:6 Tray/Slot:99/01  State:Acc/GrA/Opt
(Active) Drive:0x00000002466323e0 devnum:0x0001010f seqNum:7 Tray/Slot:00/16  State:Acc/GrA/Opt
(Active) Drive:0x00000002466326d0 devnum:0x0001001e seqNum:8 Tray/Slot:99/31  State:Acc/GrA/Opt
(Active) Drive:0x00000002466329c0 devnum:0x0001012d seqNum:9 Tray/Slot:00/46  State:Acc/GrA/Opt
(Active) Drive:0x0000000246639960 devnum:0x00010031 seqNum:10 Tray/Slot:99/50  State:Acc/GrA/Opt
Volumes: 0x00007 [O]
 
Seq:9 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:2 / PI Capable:T - 2 / Label:vg8 / VGWwn:6d039ea0000201a80000019c5dd3fc56 / RsvdSpace:            5 (GB)
(Active) Drive:0x0000000246632fa0 devnum:0x00010102 seqNum:1 Tray/Slot:00/03  State:Acc/GrA/Opt
(Active) Drive:0x0000000246633290 devnum:0x00010011 seqNum:2 Tray/Slot:99/18  State:Acc/GrA/Opt
(Active) Drive:0x0000000246633580 devnum:0x00010120 seqNum:3 Tray/Slot:00/33  State:Acc/GrA/Opt
(Active) Drive:0x0000000246633870 devnum:0x0001002f seqNum:4 Tray/Slot:99/48  State:Acc/GrA/Opt
(Active) Drive:0x0000000246633b60 devnum:0x00010130 seqNum:5 Tray/Slot:00/49  State:Acc/GrA/Opt
(Active) Drive:0x0000000246633e50 devnum:0x00010003 seqNum:6 Tray/Slot:99/04  State:Acc/GrA/Opt
(Active) Drive:0x0000000246634140 devnum:0x00010112 seqNum:7 Tray/Slot:00/19  State:Acc/GrA/Opt
(Active) Drive:0x0000000246634430 devnum:0x00010021 seqNum:8 Tray/Slot:99/34  State:Acc/GrA/Opt
(Active) Drive:0x0000000246634720 devnum:0x00010125 seqNum:9 Tray/Slot:00/38  State:Acc/GrA/Opt
(Active) Drive:0x0000000246634a10 devnum:0x00010034 seqNum:10 Tray/Slot:99/53  State:Acc/GrA/Opt
Volumes: 0x00008 [O]
 
Seq:10 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:2 / PI Capable:T - 2 / Label:vg9 / VGWwn:6d039ea0000201a80000019d5dd3fc57 / RsvdSpace:            5 (GB)
(Active) Drive:0x0000000246634d00 devnum:0x00010105 seqNum:1 Tray/Slot:00/06  State:Acc/GrA/Opt
(Active) Drive:0x0000000246627230 devnum:0x00010014 seqNum:2 Tray/Slot:99/21  State:Acc/GrA/Opt
(Active) Drive:0x0000000246623d50 devnum:0x00010123 seqNum:3 Tray/Slot:00/36  State:Acc/GrA/Opt
(Active) Drive:0x00000002466355d0 devnum:0x00010024 seqNum:4 Tray/Slot:99/37  State:Acc/GrA/Opt
(Active) Drive:0x00000002466358c0 devnum:0x00010133 seqNum:5 Tray/Slot:00/52  State:Acc/GrA/Opt
(Active) Drive:0x0000000246635bb0 devnum:0x00010006 seqNum:6 Tray/Slot:99/07  State:Acc/GrA/Opt
(Active) Drive:0x0000000246635ea0 devnum:0x00010115 seqNum:7 Tray/Slot:00/22  State:Acc/GrA/Opt
(Active) Drive:0x0000000246636190 devnum:0x00010019 seqNum:8 Tray/Slot:99/26  State:Acc/GrA/Opt
(Active) Drive:0x0000000246636480 devnum:0x00010128 seqNum:9 Tray/Slot:00/41  State:Acc/GrA/Opt
(Active) Drive:0x0000000246636770 devnum:0x00010037 seqNum:10 Tray/Slot:99/56  State:Acc/GrA/Opt
Volumes: 0x00009 [O]
 
Seq:11 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:2 / PI Capable:T - 2 / Label:vg10 / VGWwn:6d039ea0000201a80000019e5dd3fc59 / RsvdSpace:            5 (GB)
(Active) Drive:0x0000000246636a60 devnum:0x00010108 seqNum:1 Tray/Slot:00/09  State:Acc/GrA/Opt
(Active) Drive:0x0000000246636d50 devnum:0x00010017 seqNum:2 Tray/Slot:99/24  State:Acc/GrA/Opt
(Active) Drive:0x0000000246637040 devnum:0x00010118 seqNum:3 Tray/Slot:00/25  State:Acc/GrA/Opt
(Active) Drive:0x0000000246637330 devnum:0x00010027 seqNum:4 Tray/Slot:99/40  State:Acc/GrA/Opt
(Active) Drive:0x0000000246637620 devnum:0x00010136 seqNum:5 Tray/Slot:00/55  State:Acc/GrA/Opt
(Active) Drive:0x0000000246637910 devnum:0x00010009 seqNum:6 Tray/Slot:99/10  State:Acc/GrA/Opt
(Active) Drive:0x0000000246637c00 devnum:0x0001010d seqNum:7 Tray/Slot:00/14  State:Acc/GrA/Opt
(Active) Drive:0x0000000246637ef0 devnum:0x0001001c seqNum:8 Tray/Slot:99/29  State:Acc/GrA/Opt
(Active) Drive:0x00000002466381e0 devnum:0x0001012b seqNum:9 Tray/Slot:00/44  State:Acc/GrA/Opt
(Active) Drive:0x00000002466384d0 devnum:0x0001003a seqNum:10 Tray/Slot:99/59  State:Acc/GrA/Opt
Volumes: 0x0000a [O]
 
Seq:12 / RAID 6 / VGCompleteState / TLP:F / DLP:T / SSM:T / ActDrv:10 / InActDrv:0 / VolCnt:1 / Secure:No
BlockSize:512 / LPE:3 / DBF:3 / PI Capable:T - 2 / Label:vg11 / VGWwn:6d039ea0000201a80000019f5dd3fc5a / RsvdSpace:            5 (GB)
(Active) Drive:0x0000000246622bb0 devnum:0x0001010b seqNum:1 Tray/Slot:00/12  State:Acc/GrA/Opt
(Active) Drive:0x00000002466387c0 devnum:0x0001000c seqNum:2 Tray/Slot:99/13  State:Acc/GrA/Opt
(Active) Drive:0x0000000246638ab0 devnum:0x0001011b seqNum:3 Tray/Slot:00/28  State:Acc/GrA/Opt
(Active) Drive:0x0000000246638da0 devnum:0x0001002a seqNum:4 Tray/Slot:99/43  State:Acc/GrA/Opt
(Active) Drive:0x0000000246639090 devnum:0x00010139 seqNum:5 Tray/Slot:00/58  State:Acc/GrA/Opt
(Active) Drive:0x0000000246634ff0 devnum:0x00010001 seqNum:6 Tray/Slot:99/02  State:Acc/GrA/Opt
(Active) Drive:0x0000000246639670 devnum:0x00010110 seqNum:7 Tray/Slot:00/17  State:Acc/GrA/Opt
(Active) Drive:0x0000000246623480 devnum:0x0001001f seqNum:8 Tray/Slot:99/32  State:Acc/GrA/Opt
(Active) Drive:0x0000000246639c50 devnum:0x0001012e seqNum:9 Tray/Slot:00/47  State:Acc/GrA/Opt
(Active) Drive:0x0000000246639f40 devnum:0x00010032 seqNum:10 Tray/Slot:99/51  State:Acc/GrA/Opt
Volumes: 0x0000b [O]
 
 
value = 139927976262160 = 0x7f438551ba10

ahmadm
911 Views

It appears that drive 99,4,1 is the one with the remaining logical piece failure. If you can show me the output of "evfShowVol 1" command, I can confirm 100%.

 

If that is the case, the safest solution would be to fail the drive then reconstruct it in the UI.

 

jtejada
834 Views

Here the output. Yes, drive 99,4,1 was the drive replaced due to Impending failure status. How to fail the drive? Directly extract it? Or maybe directly making reconstruct in the UI should solve it.

 

-> evfShowVol 1
Volume 0x1(RAIDVolume)
 6 VolumeListener(s)
     0xc001(MediaScanAgent)
     N3vdm11VolumeGroupE
     N3vdm17RAIDVolumeManagerE
     N3spm18SymbolEventManagerE
     N3evf10CmdHandlerE
     N3evf22VolumeUserLabelManagerE
 0 Children
Volume 0x1 Attributes:
     Volume Type:       8+2 RAID 6
     User Label:        d1
     WWN:               6d039ea0000201a8000001a35dd3fc77
     Address:           0x00000001660b4850
     Devnum:            0x10000001
     Capacity:          183601463296 blocks
     BlockSize:         512 bytes
     LargeIoSize:       4096 blocks
     MaxBkgdIoSize:     4096 blocks
     Has Extents:       false
     Ownership:         This controller
     PreferredPath:     This controller
     Transferring:      false
     StopIOInProgress:  false
     Suspended:         0
     Quiesce Count:     0
     Quiesce Both Ctl:  0 (Local: 0/Alt: 0)
     State:             RV_DEGRADED
     Unreadable Sectors:Not Present
     App Tag:           0x0000
     App Tag Owned:     false
     Protection Type:   1
     PI Reporting:      enabled
     PI Mirroring:      enabled
     isSecure:          false
     isSecureCapable:   false
     isFIPS:            false
     isExopActive:      true
     isPersistent:      true
     isTCGSSupported:   true
     skipped URS:       0
     NonCriticalTransferAllowed: true
     Permissions:       CONFIG=Y            CPYsrc=Y            CPYTGT=Y
                        DRIVESACCESSIBLE=Y  FORMAT=Y            FORMATTING=N
                        IMPORT=Y            MAP=Y               MIRPRI=Y
                        MIRSEC=Y            MISSING=N           MOVE_RELATED=Y
                        NON_RED=Y           OFFLINE=N           READ=Y
                        READY=Y             RECONFG=Y           REP_NOT_ON_PREF=Y
                        SNAP=Y              TRANSFER=Y          WRITE=Y
                        ROLLBACK=Y          MIRACTV=Y           NEWPIT=Y
                        DVE=Y               DSS=Y               CONCAT=N
                        FRC=Y               PITGROUP=Y          ASYNC_MIRPRI=Y
                        ASYNC_MIRSEC=Y      DSP=Y               CPC=Y
                        THIN_MANEXP=Y       THIN_GROWPARAM=Y    XCPY=Y
                        4KDSS=Y             VAULT=Y             RESTORE=Y
                        VAULT_GRP=Y         ALBTRANSFER=Y       PRIORITYXFER=Y
     Activity Count:    1
     QueueDepth:        0
     EUI:               000001a35dd3fc77d039ea00000201a8
 
 
     IAF Aborted:               0
     Segment Size:              256 blocks
     Stripe Size:               2048 blocks
     Drive Recovery Mode:       RECOVERY_INVALID
     PreConfigedPI:             0
     isVolumeCopyActive:        False
     isVolumeCopyInsertPI:      False
     DSS PreAllocation:         Enabled
     AbsoluteMaxDSSSegSize:     4096
     Pre-Read Redundancy Check: Disabled
     Retry on Mismatch:         Enabled
     Repair Writes:             Disabled
     Flash Read Cache:          Disabled
     Atomic Write Support:      Disabled
     Volume Usage:              STANDARD
     Initialization:            Completed
     Offline Type:              0
     NoDemandFlush:             False
     Read Ahead:                1
     Fast Write:                1
     Sync Age:                  2500
     Cache Flush Modifier:      8
     Min Warn Flush Modifier:   8
     Cache Granularity:         64
     Ignore FUA:                Host type(s) - 0 1 2 5 6 7 8 9 10 15 17 22 23 24 25 26 27 28 29
 
     Media Scan Settings: Enabled with Parity Check, but no Repair.
          Period = 30 day(s). Current rate = 88542 blocks/sec.
 
     Background Task:    MEDIA SCAN at block 0x027d7ded800 with 93% complete
     Background Delay:   0.1 seconds per 4096 blocks
 
 
 
*** Volume Group Info ***
 
    WWN        : 6d039ea0000201a8000001955dd3fc4c
    RAID Level : RAID 6
    Boundary   : 171125430272 0x27d7ded800
    Persisted  : 0  0x0
    VG Label   : vg1
    Address    : 0x000000024669de00
    Drive Count: 10
    Secure     : False
    PI Capable : True - 2
    Media Type : HDD
 
     Volumes on this Group:
Volume Count:                      1
0x00001 [D]
 
*** Pieces ***
 
    Offset: 0x0                    Length:     0x557f00000 (22950182912 dec)
    Count : 10                     Data Count: 8
 
    Piece     Devnum        Address  Tray/Slot   State
        0  0x00010103    0x000000016575fa40    00,04     PieceOptimalState
        1  0x00010012    0x0000000165732d20    99,19     PieceOptimalState
        2  0x00010121    0x0000000165732c10    00,34     PieceOptimalState
        3  0x00010025    0x00000001656cc3c0    99,38     FAILED
        4  0x00010134    0x00000001656f5408    00,53     PieceOptimalState
        5  0x00010007    0x00000001656f52f8    99,08     PieceOptimalState
        6  0x00010116    0x000000016576bca0    00,23     PieceOptimalState
        7  0x0001001a    0x00000001657a2978    99,27     PieceOptimalState
        8  0x00010129    0x000000016579e910    00,42     PieceOptimalState
        9  0x00010038    0x00000001656ff960    99,57     PieceOptimalState
 
StripesPerCachePoolExtent:            16
BlocksPerCachePoolExtent :         32768
 
*** CACHE DEVICE 1 ***
 
     Sector Size     Secs/CacheBlock          Cache Blk Size    Clean Minimum
       512 bytes                  64             32768 bytes 0000208000
Dirty Cache Blks   Dirty Cache High per Pool
            3896   0:18924 1:16870 2:15379
  Hash Data Blks    Hash Parity Blks
          152029                   0
       RD_CACHE       WR_CACHE   WR_CACHE_MIR            FWT
        ENABLED        ENABLED        ENABLED       DISABLED
 
           CWOB  WT_BU_BATT_ON     READ_CACHE    WRITE_CACHE   WRITE_CACHE_MIR
       DISABLED             NO         ACTIVE         ACTIVE            ACTIVE
HIGH_PRIORITY_DEV: NO
Background sync is in progress : NO
CacheBlockMask: ffffffffffffffff
 
Cache Statistics:
                 current       old   current       old
         cache      read      read    parity    parity     disk
 VOL     reads      hits      hits      hits      hits     reads
---- --------- --------- --------- --------- --------- ---------
   1   8813418   3365511    945332         0     88559   5705963
 
          read      read
         check     check    insert      full   partial      disk
 VOL      reqs      hits      reqs overwrite overwrite    writes
---- --------- --------- --------- --------- --------- ---------
   1   2462860    979939    361100      7364    114205         0
 
                            Cache Pool  0       Cache Pool  1       Cache Pool  2  
                         ------------------- ------------------- -------------------
       WB blks   % Dirty   WB blks   % Dirty   WB blks   % Dirty   WB blks   % Dirty
 VOL   current     Limit      high     Limit      high     Limit      high     Limit
---- --------- --------- --------- --------- --------- --------- --------- ---------
   1      3896         1     18924         9     16870         8     15379         7
 
     +---------- cache blocks evicted after ---------+   evicted    blocks
 VOL     fetch       hit  prefetch pftch hit     write     total    in-use
---- --------- --------- --------- --------- --------- --------- ---------
   1   2839391   2105770   1010947  5949256   4849966  16755330    151999
 
                prefetch            prefetch  pct
 VOL          hit blocks         miss blocks hits
---- ------------------- ------------------- ----
   1           106899520            64700608   62
 
value = 139928058009584 = 0x7f438a3117f0
 
 

ahmadm
828 Views

Yes, you would essentially reconstruct the drive. To do that, you can run this command in the controller shell:

 

> startDriveReconstruction_MT 0x10025

Public