question regarding Netapp FAS-2040 errors

JOEOFFORTLEWIS · ‎2013-04-08

Recently we have obtained a few FAS-2040's for a classified and unclassified private network. They were configured and installed properly until one day I recieved an error light on the FAS and it would simply not allow me to connect to any virutal machines that were on it. Long story short, I did a lot of troubleshooting on this machine to see why it would provide an error light and stop working there after. It was decided to send it to the unit that supports us. They sent it back stating that they installed a new raid controller on the device.

Once I set it back up into the server rack, ensured all drives were in the right areas, booted it on, it started to go into a boot loop with the following three statement/errors that I saw during the boot up process.

1. No Root Volume Found

2. Warning there do not appear to be any disks attached to the system

3. The platform Doesnt support service processor

With those errors I would assume that I would need to rebuild and configure the entire FAS again but the second error throws me off. If it does not see any disks at all, then of course it would throw a no root volume found error. As for the third error I am not sure what that means.

Also I am unsure, as I have been thrown into the Netapp storage world, on how to see if the harddrives are seen on it, or if there is a bios, etc etc.

Any help with commands and input on what this means would be really appreciated.

Thank you very much,

Joe

pedro_rocha · ‎2013-04-08

Hello Joe,

Are you able to enter maintenance mode? Normally you would hit ctrl+c during boot and then select from the menu that appears, the maintenance mode. From there you are able to check for the disk drives this controller is able to see or own.

One question: did your suppport folks only replaced the controller? I mean, disks stayed with you?

If you want to recover the data, you will need to assign all the disks that were previously owned by the replaced controller to the new one (this happens because you are using software based ownership, which labels the disks using the controllers system id (which has changed if the controller was replaced)).

Please verify if this helps you in any way so I can provide you with more stuff.

Regards,

Pedro Rocha.

JOEOFFORTLEWIS · ‎2013-04-09

I am able to enter maintenance mode no problem. From what I was told, only the controller was changed and I still have the original disks placed back in the order they were previously.

Data recovery would be optimal and if worse comes to worse I can rebuild the entire system.

Thank you very much for your input and help, its greatly appreciated.

Joe

pedro_rocha · ‎2013-04-09

Joseph,

You do not need to re-install the system from ground zero (if disk drives are intact). The procedure you need is the motherboard replacement one, which you can find in here: https://library.netapp.com/ecm/ecm_download_file/ECMM1280334

All the data and configurations that you have done previously will be available again.

Kind regards,

Pedro Rocha.

JOEOFFORTLEWIS · ‎2013-04-10

Pedro,

It seems that some of these commands listed in the directions you sent me are not available. As a result I am unable to further proceed using this guide. Not sure if this helps any but my Bios Ver is 6.1 and the loader ver is 2.1. Could it be that I have an older bios/loader?

Its a tad frustrating working in a secure area and on a secure network as I am unable to find the documents I need due to the firewall/internet filters we have here. It blocks me from viewing parts of the netapp website and viewing support docs. But I know ways around it, just more frustrating in the long run. Regardless let me know

Thank you again for all your help.

Joe

pedro_rocha · ‎2013-04-10

Hello Joe,

I would suggest you to start from page 10 at "Setting the time after replacing the controller module". Please check that and tell me if there is anything from there that you cannot execute.

Is it an HA pair or you only have one controller module (not sure if you mentioned that before)? If you have two, just make sure you select the correct disk drives when reassigning disks to the replaced controller.

Kind regards,

Pedro Rocha

JOEOFFORTLEWIS · ‎2013-04-11

Good morning,

Well I did start on page 10 exactly where you mentioned and was able to set the date and time with no problem. Reassigning the Disks is where I run into a snag. The command "disk show -v" is not a valid command and I verified if it was in the list of available commands by typing help. As a result I did not attempt to perform any other commands seeing as I did not have vital information needed by the disk show command. Comparing the command list I did not even see other ones like "boot_ontap" found at the bottom of pg 12 for example. Tried to enter some and still recieved the invalid command error.

Are there commands that can be entered even though they do not appear on the help command list?

I apologize for the constant problems on my behalf but again I do really appreciate everything you are offering.

Let me know when you get time

Thank you Pedro

Joe.

pedro_rocha · ‎2013-04-11

Hello Joe,

Even in maitenance mode you were not able to execute disk show?

"boot_ontap" is available in the Loader prompt. You don't see it also?

Can you capture some CLI outputs and attatch them here so I can take a look?

Regards,

Pedro Rocha

+55 61 8203-5800

Enviado pelo meu BlackBerry®

JOEOFFORTLEWIS · ‎2013-04-11

Absolutely, I will do it as soon as I am able.

JOEOFFORTLEWIS · ‎2013-04-12

Below is the list of commands available to me. I tested out the "Disk Show" command among others like "Disk reassign", "System Node Run", "Storage Failover Giveback", "Partner" and "Run Partner" and all came back with the error of Invalid command.

LOADER-B>help

Available commands:

version Print loader and BIOS version.

update_flash Updates the boot flash with the firmware image on the PC-Car

netboot Boots the supplied URL off the network.

boot_diags Boots the diagnostic image off of the PC-Card.

boot_backup Boots the backup image of Data ONTAP off of the PC-Card.

boot_primary Boots the primary image of Data ONTAP off of the PC-Card.

boot_ontap Boots the correct image of Data ONTAP

update_bmc Updates the BMC firmware

bmc reboot Reboot the BMC

bmc flash Update the BMC firmware

flash Update a flash memory device

bye Reset the system.

set date Set current date

set time Set current time

set console Change the active console device

lsmod Per-file metadata.

autoboot Automatic system bootstrap.

go Start a previously loaded program.

boot Load an executable file into memory and execute it

load Load an executable file into memory without executing it

ping Ping a remote IP host.

arp Display or modify the ARP Table

ifconfig Configure the Ethernet interface

show date Display current time according to RTC

show time Display current time according to RTC

show devices Display information about the installed devices.

savenv Save the environmental variables.

saveenv Save the environmental variables.

unsetenv Delete an environment variable.

set-defaults Reset all system environmental variables to default values.

setenv Set an environment variable.

printenv Display the environment variables

help Obtain help for loader commands

For more information about a command, enter 'help command-name'

LOADER-B>

LOADER-B> disk show

Invalid command: "disk"

Available commands: version, update_flash, netboot, boot_diags, boot_backup, boot_primary, boot_ontap, update_bmc, bmc, flash, bye, set, lsmod, autoboot, go, boot, load, ping, arp, ifconfig, show, savenv, saveenv, unsetenv, set-defaults, setenv, printenv, help

AGUMADAVALLI · ‎2013-04-12

i think you neeed to hire a professional services to install further, because communities will not tell each and every step.

thank you

AK G

JOEOFFORTLEWIS · ‎2013-04-12

I thank you and its quite obvious something is different/wrong here thus I am here to ask. I do not expect people here to fix my issue 100% but the opinions and help of others still helps guide one in the right direction. Seeing as this does not have commands that it 'should' than something can be hopefully pointed out.

Again I am not looking for the world here but I thank you for your concern.

HENRYPAN2 · ‎2013-04-12

Poor Joseph,

You may wish to try boot_diags before hiring a pro.

Good luck

Henry

JOEOFFORTLEWIS · ‎2013-04-12

heh, way ahead of ya, Henry. It came back clean, thank you for the idea.

have a good weekend!

pedro_rocha · ‎2013-04-12

Joe,

You have to get into the maintenance mode before using "disk" commands. On the LOADER prompt you should enter boot_ontap, and interrupt it with Ctrl+C. Then choose the right option to get into the maintenance mode. Once there, you should be able to use disk assign.

Try that and let me know. Also, follow the guide I sent you before.

Regards,

Pedro Rocha

JOEOFFORTLEWIS · ‎2013-04-18

Pedro,

Upon following the documentation you provided, I recieved a couple new errors. Still based on what I'm reading it leads me to beleive that everything has been wiped on these disks, but again I could and am probably wrong. Here is what I got:

[raid.assim.disk.badlabelversion:error]: Disk 0C.00.1 Shelf 0 Bay 1 [NETAPP x290_s15K7560A15 NA00] has raid label with version (10), which is not within the currently supported range (5 - 9). Please contact NetApp Global Services.

The above error is given for each drive. How would I go about checking what raid level is set on another FAS and what command would I use exactly to change this? I went through all the commands and while I found the Raid command, I did not see any options for that command to change a raid level. Also I recieved the following:

[coredump.spare.none:info]: No sparecore disk was found.

[raid.assim.tree.noRootVol:error]: No usable root volume was found!

PANIC: raid: there are no data or parity disks in process rc on release NetApp Release 7.3.7

Thoughts?

pedro_rocha · ‎2013-04-19

Joe,

It seems that some disks were swapped. Those RAID LABEL errors state that probably the associated disks were labeled with an ONTAP version that is not supported by your original ONTAP version. This situation is a little complicated because you would need to re-labeled those disks with a supported version.

Are you able to contact NetApp supporte?

Regards,

Pedro Rocha.

AGUMADAVALLI · ‎2013-04-09

Hi there,

Basic thing: you have configure and install the system again. Allocate 3 disks, initialize and install the OS on it via maintenance mode.

In FAS 2040, instead of SP, they may have called it BMC or RLM - Please verify the manual supplied with it.

thank you,

AK G

JOEOFFORTLEWIS · ‎2013-04-09

Thanks AK, I just need to locate a digital copy of the manual. The people signed and responsible for these FAS's sadly did not provide or even receive any documentation/manual. The place I work for has some very bad business practices and I am left to my own devices. Talk about being set up for difficult times. Regardless, if what the previous user said doesnt work, I know that it will have to be rebuilt.

Thank you for your input.

Joe

AGUMADAVALLI · ‎2013-04-09

just download from below link:

http://support.netapp.com/documentation/productlibrary/index.html?productID=30421

thank you,

AK G

echen · ‎2013-10-25

To reinitialize the array with faulty disk/hardware, you need to power cycle controller/DAE, take out bad disk, power it back on, then maintenance mode then initialize it. Power cycling will remove old error msg