Solved: Re: 270c controller not booting, "No root volume found".

CLUSTERMAGNET · ‎2011-05-09

Guys, our 2nd NetApp 270c controller is not booting. Can you please help?

Thanks

CFE version 1.2.0 based on Broadcom CFE: 1.0.35

CPU type 0x1040102: 650MHz

Total memory: 0x40000000 bytes (1024MB)

Starting AUTOBOOT press any key to abort...

Loading: 0xffffffff80001000/25888 0xffffffff80007520/13053911 Entry at 0xffffffff80001000

Starting program at 0xffffffff80001000

Press CTRL-C for special boot menu

............................................Interconnect based upon M-VIA ERing Support

http://www.nersc.gov/research/FTG/via

Tue Jul 20 06:36:28 GMT [cf.nm.nicTransitionUp:info]: Interconnect link 0 is UP

Tue Jul 20 06:36:29 GMT [cf.nm.nicReset:warning]: Initiating soft reset on Cluster Interconnect card 0 due to rendezvous jammed

Thu Jan 1 00:00:27 GMT [fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 0b.

NetApp Release 7.2.2: Sat Mar 24 20:52:58 PDT 2007

Starting boot on Wed Dec 31 23:59:58 GMT 1969

Thu Jan 1 00:00:27 GMT [nvram.battery.state:info]: The NVRAM battery is currently ON.

Thu Jan 1 00:00:32 GMT [diskown.isEnabled:info]: software ownership has been enabled for this system

WARNING: NVLOG: nvlog magic was initially found ZEROed (by CFE ?) !!

WARNING: nvlog_test: Unexpected nvlog contents. Will discard contents of nvlog, USER DATA MAY BE LOST

Thu Jan 1 00:00:33 GMT [fmmbx_instanceWorke:info]: no mailbox instance on local side

Thu Jan 1 00:00:33 GMT [cf.noDiskownShelfCount:info]: Disk shelf count functionality is not supported on software based disk ownership configurations.

Thu Jan 1 00:00:36 GMT [coredump.spare.none:info]: No sparecore disk was found.

WARNING: 0 disks found!

Storage Adapters found:

1 Fibre Channel Storage Adapters found!

0 SAS Adapters found!

0 Parallel SCSI Storage Adapters found!

1 ATA Adapters found!

Target Adapters found:

1 Fibre Channel Target Adapters found!

2 iSCSI Target Adapters found!

1 Unknown Target Adapters found!

WARNING: there do not appear to be any disks attached to the system.

Check that disks have been assigned ownership to this

system (ID 84178826) using the 'disk show' and 'disk assign'

commands from maintenace mode.

No root volume found.

Rebooting... (ctrl-c to bre#

bsti · ‎2011-05-11

Can you show us the output from a disk show -v? I believe you can run this from the maintenance mode menu.

View solution in original post

aborzenkov · ‎2011-05-09

I would check speed setting on shelf (should be 1Gb/s) and terminator setting on FAS270 head (should be on if no external shelf is connected).

CLUSTERMAGNET · ‎2011-05-09

Thanks for help! Quick question, why shouldnt the setting be at 2 Gbps... Also, yes, there is a shelf attached.

I am instructing the guys via remote support... How is the terminator setting set?

Thanks!!

CLUSTERMAGNET · ‎2011-05-09

The setting is 1 Gb, and there is no need for terminator setting since the shelf is attached.

Any other clues?

Thanks!

aborzenkov · ‎2011-05-09

0b is disk adapter on FAS270. If its initialization failed, it usually means open loop. Depending on which ESH module is installed your shelf may need termination too. It has to be set to 1Gb/s as well.

So far it is hardware problem which in almost all cases I have seen was caused by incorrect loop termination setting.

CLUSTERMAGNET · ‎2011-05-09

Thanks! Ill have the tech snap a picture of the settings.

Have a look at this thread: http://communities.netapp.com/message/53762#53762

Also, I am very suspicious that the above thread has a correct answer.

aborzenkov · ‎2011-05-09

Boot into maintenance mode and check whether filer sees any disk at all.

CLUSTERMAGNET · ‎2011-05-09

Thanks for your time aborzenkov!

Filer is serving production luns via controller A.. So yes, all disks are assigned to controller A.

controller B is stuck in this infinite boot loop i posted above... I hope that will explain what you were asking.

shaunjurr · ‎2011-05-10

Hi,

The FAS270c was the first system to use disk ownership. If your "B" controller doesn't see any disks it's probably because none are assigned. You will need to assign at least 2 disks (it'll probably complain about spares if you don't assign 3) to get a raid4 root aggregate and volume up. 'disk show' is your friend here.... at least to see if you have disks assigned.

Good luck.

CLUSTERMAGNET · ‎2011-05-09

Oh, sorry, you mean Ctrl+C and choose option 5 on controller B?

aborzenkov · ‎2011-05-09

Yes.

CLUSTERMAGNET · ‎2011-05-11

I have unassigned 7 disks from ControllerA, and can verify that they are listed as unassigned,however, when booting controller B, it is not able to see any unassigned disks.

Should I have the guys tripple check the termination settings? There is a shelf attached as well, which is perfectly visible by controllerA...

Thanks for all your help!!!

PUNKGENIUS · ‎2011-05-23

how do you check the speed setting on the shelf?

CLUSTERMAGNET · ‎2011-05-23

Have a look on the back of the unit.

bsti · ‎2011-05-11

Can you show us the output from a disk show -v? I believe you can run this from the maintenance mode menu.

CLUSTERMAGNET · ‎2011-05-11

Certainly, controller A:

(from booted ontap 7.2.2)

fas270> disk show -v

DISK OWNER POOL SERIAL NUMBER

------------ ------------- ----- -------------

0b.27 fas270 (84179494) Pool0 3KS65WYF00009732Q7GG

0b.18 Not Owned NONE 3HY8HGV5000074528JUE

0b.44 Not Owned NONE ZAJVV63G

0b.41 fas270 (84179494) Pool0 ZAJVV1WG

0b.43 fas270 (84179494) Pool0 ZAJEVYNG

0b.42 fas270 (84179494) Pool0 ZAJVV98G

0b.35 fas270 (84179494) Pool0 ZAJW112F

0b.36 fas270 (84179494) Pool0 ZAJKDNZG

0b.33 fas270 (84179494) Pool0 ZAJX399F

0b.32 fas270 (84179494) Pool0 ZAJWPZ9F

0b.45 Not Owned NONE ZAJVV26G

0b.40 fas270 (84179494) Pool0 ZAJETTNG

0b.38 fas270 (84179494) Pool0 ZAJKDJ6G

0b.22 fas270 (84179494) Pool0 3HY9ATW0000075163TBE

0b.21 fas270 (84179494) Pool0 3HY8GWPP00007451Z8GV

0b.28 Not Owned NONE 3HY8HLWM00007453RGJ3

0b.19 Not Owned NONE 3HY8HM6V000074528H5N

0b.24 fas270 (84179494) Pool0 3HY8HFMV000074527D4E

0b.23 fas270 (84179494) Pool0 3HY8HELC000074527CVE

0b.26 fas270 (84179494) Pool0 3HY8HFRP000074527CPE

0b.25 fas270 (84179494) Pool0 3HY8HM5D00007451KKDY

0b.29 Not Owned NONE 3KS5RFK000009717X0V0

0b.20 Not Owned NONE 3HY8HE29000074525ZRD

0b.39 fas270 (84179494) Pool0 ZAJPDSXF

0b.34 fas270 (84179494) Pool0 ZAJX4WUF

0b.17 Not Owned NONE 3KS5QZB400009717WFXH

0b.37 fas270 (84179494) Pool0 ZAKAJ7PF

0b.16 Not Owned NONE 3HY8HK55000074528HC9

controllerB from maintenance:

*> disk show -v

Local System ID: 84178826

disk show: No disks attached to system.

*>

aborzenkov · ‎2011-05-11

Why do you ask questions if you do not listen to answers ...

You have hardware problem. Disk controller initialization fails. Before you can start with disk assignment you need to resolve this.

It could be open loop. It could be as simple as failure on controller B. It could be failure on connecting cable or ESH module in shelf.

Disconnect everything from controller B. Set it correctly (read hardware documentation on NOW). Check whether it sees any disk. If yes - troubleshoot issue between controller and shelf. If no - controller is broken.

CLUSTERMAGNET · ‎2011-05-11

My fault, catching up to all the possibilities. You're absolutely right, the problems must be scsi termination related. I will get the guys onsite to take a look.

What if there is no termination switch on the controller?

Thanks!

CLUSTERMAGNET · ‎2011-05-11

I've found the image of the back of the 270c controllers, they both definitely have a termination switch. Thanks a ton for all your help! Will get the guys to take a look.