topic Re: Sim 9.5 root volume trashed after power failure in Simulator Discussions

Sim 9.5 root volume trashed after power failure

peterjumper — Fri, 24 May 2019 18:22:09 GMT

A few times now we've had power failures and the 9.5 vsim root vol gest trashed. Logging in, I see

***********************
** SYSTEM MESSAGES **
***********************

Internal error: Cannot open corrupt replicated database. Automatic recovery
attempt has failed or is disabled. Check the event logs for details. This node
is not fully operational. Contact support personnel for the root volume recovery
procedures.

(Or the other one about root vol corrupted.)

I've read the articles that say to use

VLOADER> unsetenv bootarg.init.boot_recovery

... but I get:

no such file or directory

Others have noted it's not there in 9.5 and possibly earlier versions.

Is there a way to recover in 9.5?

I also read there is a way to use fast persistent storage for NVRAM. We have NVME and SSD. Is there a how-to on that?
Thanks!
Peter

Re: Sim 9.5 root volume trashed after power failure

emalins — Tue, 16 Jul 2019 10:31:14 GMT

I've had exactly the same issue. It's making me reluctant to move our FAS to 9.5 if recovering the simulator doesn't look posisble without a rebuild. If anyone's got a howto for recovering it, I'd love to know.

Re: Sim 9.5 root volume trashed after power failure

peterjumper — Fri, 19 Jul 2019 00:04:01 GMT

I'm fairly certain that this Sim issue does not affect actual hardware. This happens on Sim because the NVRAM is faked in system memory. Power loss --> NVRAM loss. That wouldn't happen on hardware unless you lost power and then disssambled your controller at least to the point of removing NVRAM battery / memory, then put it back together and powered on. Or left it powered off until the NVRAM bettery discharged. Obviously unnatural acts.

Still, it makes it difficult for people inside NetApp, partners, and customers to dev and test when the Sim looses it's mind after a mere power glitch.

Re: Sim 9.5 root volume trashed after power failure

Jim_Robertson — Tue, 15 Oct 2019 17:00:23 GMT

@peterjumper wrote:

A few times now we've had power failures and the 9.5 vsim root vol gest trashed. Logging in, I see

***********************
** SYSTEM MESSAGES **
***********************

Internal error: Cannot open corrupt replicated database. Automatic recovery
attempt has failed or is disabled. Check the event logs for details. This node
is not fully operational. Contact support personnel for the root volume recovery
procedures.

Has anyone found a work around for this? I ran into this today after a long weekend. There were no power outages that I'm aware of, and this was on a simulator that I had just rebuilt last week. I had made sure that there was enough space on the root aggregate, and on vol0, so I don't think it was a matter of it running out of space (the root vols on the two SVMs did go over their snapshot reserve, but there was still space in the SVM root volumes).

I also checked the VLOADER prompt, and there is no option for bootarg.init.boot_recovery.

Re: Sim 9.5 root volume trashed after power failure

Taiwo — Fri, 10 Apr 2020 06:02:11 GMT

unsetenv bootarg.rdb_corrupt

Re: Sim 9.5 root volume trashed after power failure

alanarh — Tue, 02 Jun 2020 22:42:08 GMT

Not sure if it's related, but I think it is. I just updated the VMware Player to version 15.5.5 and now when I run the unsetenv bootarg.rdb_corrupt command it says no such file or directory.