Failed disks (back-end LUNs) on v-series with ONTAP 8.1

[onavatte]

Hi,

I've installed a couple of v-series (3240C) controllers at one of my client and we have issues with back-end disks.

The back-end storage is an IBM DS5300 on Cisco 9148 switches.

ONTAP 8.1 has been installed on all of them.

- On them, when new disks are presented to the pair, one of the controller is seing the disk correctly but the other is showing messages like:

Tue Jun 26 16:23:27 EDT [toaster:disk.init.failure.error:warning]: Disk m134A02:1-41.126L222 failed initialization due to error 5.

Tue Jun 26 16:23:30 EDT [toaster:disk.senseError:error]: Disk m134A02:1-41.126L222: op 0x28:0000a3e8:0018 sector 0 SCSI:illegal request -  (5 94 1 0)

Tue Jun 26 16:23:30 EDT [toaster:diskown.errorReadingOwnership:warning]: error 23 (adapter error prevents command from being sent to device) while reading ownership on disk m134A02:1-41.126L222 (S/N 600A0B800047CEE20000010D4FD7A805)

- The sysconfig -v command is showing a (Failed) or (Unitialized) state only on the problematic node,

- The disk show command hangs the ssh session (wich cannot be reseted by logout telnet or rshkill...) => a bug has be logged for that issue which is supposed to be fixed for a while.

The workaround is to perform a takeover / giveback: disks are showing correctly on both nodes after that.

But, as controllers are going into production and we plan to present many other disks afterwards, this is not the better solution.

Recently, I've had to setup a new pair of V3240 (same config for the back-end).

Controllers were delivered with ONTAP 7.3.6.

Disks presentation has happened flawlessly. I've then upgraded to ONTAP 8.1 and rebooted..

- One of the controller is seing the disks correctly

- The other one haven't even booted !

In maintenance mode, disk are showing as (Failed) in sysconfig -v and not any disk commands are working.

Logs are showing the same type of errors, with a little difference:

Jul 04 15:57:58 [localhost:disk.encryptSupportReqd:error]: Detected nonencrypting disk m500A01:1-1.126L1 but bootarg.storageencryption.support indicates that only encrypting disks should be present. 0 of 4 disks checked are encrypting disks.

Jul 04 15:57:58 [localhost:diskown.disabled:info]: software ownership has been enabled for this system

Jul 04 15:57:58 [localhost:DuringIO:error]: error 3 (disk failed) on disk 600A0B80:006E473E:00000090:4FE8EEAF:00000000:00000000:00000000:00000000:00000000:00000000 (S/N N/A) while opening disk

Jul 04 15:57:58 [localhost:diskown.errorDuringIO:error]: error 3 (disk failed) on disk 600A0B80:006E8714:0000009B:4FE8EE1E:00000000:00000000:00000000:00000000:00000000:00000000 (S/N N/A) while opening disk

Jul 04 15:57:58 [localhost:diskown.errorDuringIO:error]: error 3 (disk failed) on disk 600A0B80:006E473E:00000094:4FE8EF60:00000000:00000000:00000000:00000000:00000000:00000000 (S/N N/A) while opening disk

...

What does mean "bootarg.storageencryption.support indicates that only encrypting disks should be present" ??

It's a basic installation, from scratch, with no fancyful configuration and certainly not with disk encryption.

- Note that if I perform a boot_backup, (in 7.3.6), disk are showing correctly !.. whitch make me think about a ONTAP 8.1 related issue.

Off course, I found no entries on thoses issues on NOW, on forums or elsewhere on internet.

Have anyone ever seen something like that ? Any idea ?

For the first issue, does anyone know a magic command to force a (back-end) disk rescan without the need to do a takeover / giveback ?

Thanks a lot,

Olivier

[thomas_glodde]

You tried a "set defaults" in the LOADER prompt (means the machine is stopped using the "halt" command) and then "bye"?

[onavatte]

It works ! ("set-defaults" with a dash).

As I understand, this command set all the environment variables to default.

So in my case, it seemed that the ONTAP 8.1 upgrade has set strange values to some variables.

Thank a lot for the quick answer.

Any ideas for my first issue ? Should I use also a "set-default" command as a workaround ? I could suppose that this issue is related to some env variables specific to v-series to set. If it's the case, I would prefer to go in a more granular way than resetting all to default.

As one of those filers is in production, I obviously have to plan a maintenance windows to reboot it. I also wondering what is the impact of running this command.

Do you have some documentation about environment variables and "set-defaults" ?

[christin]

This sounds like a support related question. If you have an active NetApp Support login, there are subject matter experts in the NetApp Support Community that may help answer your questions.

If this is an urgent issue please open a case with NetApp Technical Support.

Regards,

Christine

[thurinllenwe · Accepted solution]

The issue arose because ONTAP booted with a boot option that specified the system contained encrypted disks.  This option can be enabled or disabled.

If enabled, and no encrypted disks are present, you will see errors like the one that OP had.  The reverse is also true.

To disable:

setenv bootarg.storageencryption.support false

To enable:

setenv bootarg.storageencryption.support true

Some may call me a "necro-poster", but, since the specific command was never offered, I offer it here for any that come across the post.  Incidentally, I came across this post during a search.  I'm reminded of the following: