Subscribe

E2600 Disaster Recovery

We have a few E2600s that are now out of support. One of them seemed to have a controller failure recently, and somehow it got fully messed up in that the other controller got locked up, and the unit became unusable. Eventually we got the controller unlocked, but now that controller lost all the configuration and reports the unit as if it had never been configured. Wondering if there's a way to recover the configuration from the disks, before I completely give up and wipe away all the data. We could even move all the disks to a similar unit and try recovering there. But all my attempts have failed so far... Thanks for any pointers...

 

Jorge

Re: E2600 Disaster Recovery

Hi jorge,

 

The configuration information is stored in the DACstore which live in each drive depending on your firmware version

 

How did you unlock the controller ?

How do you see you lost all configuration ? Santricity ?

 

"reports the unit as if it had never been configured" --> what message do you have exactly (error or warning or information) ?

 

Do you have an old support bundle ?

Re: E2600 Disaster Recovery

After a long time looking around, I unlocked the controller by going in through the serial console and doing lemClearLockdown. Before doing that, I couldn't really do anything else. 

 

When I see the unit now in Santricity, it gives me the default configuration screen, asking if I want to create a disk pool and saying I have 60 disks available. I also am able to connect using SMcli, and do a show storageArray healthStatus; which returns the following:

 

show storageArray healthStatus
;

The controller clocks in the storage array are out of synchronization with the storage management station.

Controller in Slot A: Unknown
Controller in Slot B: Wed Mar 30 08:53:28 PDT 2016
Storage Management Station: Wed Mar 30 16:09:19 PDT 2016

 

The following failures have been found:
Offline Controller
Storage array: Unnamed
Component reporting problem: Controller in slot A
  Status: Failed
  Location: Controller/Drive tray 0, Controller in slot A
  Replacement part number: 46482-00
  Board ID: 2660
  Submodel ID: 138
  Serial number: SV11813489
Component requiring service: Controller in slot A
  Service action (removal) allowed: Yes
  Service action LED on component: Yes

 

Storage Array in Recovery Mode
Storage array: Unnamed
  Database location: On drives
    Status: Invalid
    Revision number: 1.0
    Identification number: 0000000000000000000000000000000000000000
    Generation number: 0
    Accessibility state: Offline
  Database location: Onboard (Controller B)
    Status: Frozen
    Revision number: Unknown
    Identification number: Unknown
    Generation number: 0
    Accessibility state: Recovery Mode
  Database location: Onboard (Controller A)
    Status: Unknown
    Revision number: Unknown
    Identification number: Unknown
    Generation number: 0
    Accessibility state: Unknown

 

 

I'm worried about the fact that it says that the database on the drives is invalid...

Re: E2600 Disaster Recovery

Okayyyyy

 

When controllers detect corruption in the primary database, they reboot and enter a Lockdown state, which is the Recovery Mode.

 

Just to confirm. What does the seven segment display shows you ?

 

1- You should have on the santricity host the backup configuration file. In Windows, this is under C:\Program Files (x86)\StorageManager\client\data\monitor\dbcapture.

 

2- Go on the enterprise management window > right-click > Execute script and type the following command

(Check the help about this command by going in the menu Help > commande reference)

 

load storageArray dbmDatabase file=”<configuration file name>"

 

3- Check the syntax and Execute the command

 

Controler should exit the recovery mode afterwards, if not clear it.

Re: E2600 Disaster Recovery

Hmmm. What is "the seven segment display"?

 

Our Santricity is running on linux, not Windows, and it is managing several hosts. Not sure how we could find the backup configuration file there...

 

Re: E2600 Disaster Recovery

The seven segment display is a small LED screen on the back of the controller.

 

 

Try in /opt or /bin

Re: E2600 Disaster Recovery

Hi,

 

Please refer Kb 3014685 What procedure needs to be followed if a controller enters lockdown?

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Re: E2600 Disaster Recovery

Hello Jorge,

 

Did you find any backup file ?

 

If not, I have an alternative but I would need a recent support bundle. I can try to generate a recovery script

 

or

 

You can clear the recovery mode but with no garanty that the controller will like it.

 

 

Re: E2600 Disaster Recovery

Hello again,

 

Sorry about the no reply yesterday, other crisis took precedence. Anyway, here are the answers to your questions:

 

The seven segment display is showing 00 for the non-faulty controller, and 0E, then Lt for the faulty controller.

 

I didn't find a backup file, but I'm not sure what I'm looking for. Any particular name for the file I'm looking for? Hmmm. Maybe I did find something. I looked for anything called .cfg, and there's this candidate:

 

   ./opt/SM/FirmwareUpgradeReports/genome-cloudstore/genome-cloudstore_60080E500023B966000000004F8805F3_Configuration.cfg

 

I found a couple of old support bundles, but they are for a different array. We have other arrays with similar configurations, if that would help.

 

 

 

Re: E2600 Disaster Recovery

Welcome back.

 

Haha no worries.

 

1- 0E + Lt means that he is in recovery mode.

 

2- I know that in windows it lives in a zip file

 

3- No, you can't take support bundle from other array, there is some specific values volume wwn or capacity or assigned drives that would not match.

 

Let's try something else then. Type the following command on both controllers and send me the output please

bdbmShowBackupDatabaseInfo