LUN Serial Numbers and physically moving aggregates.

[peiffer83]

Hello,

We are currently planning migrating our old disk shelfs that are connected to our FAS3210 to our new FAS8040.  We plan to have an outage and take all the aggregates offline following best practices.  Then move the shelfs to the new location and attach to the FAS8040.  My question is will the LUN serials change when the aggregates are brought online in the new controller.  We have many Oracle ASM Linux systems that are mapped by wwid or lun serial numbers.  If that changes they loose access to their LUN's and everything needs to be remounted manually.  I am trying to avoid this.  Windows doesn't seem to care if LUN's serial numbers change because its mounted by NTFS UUID which doesnt change.

For example in another system I am using the 7 mode tranistion tool to move volumes from an old 7mode system to another new 8040 running cluster mode.  I tested today moving the LUN's and they moved without issues, but the linux system didn't mount them.  I know this is normal practice because its a new LUN.  I can either change the LUN serial or update the mount information to fix this issue.

Old FAS3210 LUN serial 360a98000646f4c4a474a6a6c385a7755 Viewed from Linux system

New FAS8040 LUN serial 3600a098038303747695d495251426c36 Viewed from Linux system

On the other site we are moving a huge amount of Oracle Systems and it would be a lot more work to change all the serial numbers and remount during our planned outage.  So my main question is when you attach older disk shelfs to a new controller do all the LUN serials change.  If so what is the best practice to get around this issue.

My options so far:

1. Export all the serial numbers and lun information before I move the disk shelfs and if the serial numbers change, revert them back on the new controller.

2. Follow this KB to update the mounting information on the Linux Hosts. https://kb.netapp.com/support/index?page=content&id=2012720&actp=LIST_RECENT&viewlocale=en_US&searchid=1460739986114

Looking forward to feedback thank you.

[SeanHatfield · Accepted solution]

I have the impression your new 8040 is running 7mode.  If so, then the lun serials persist if you are doing a "head swap" AND you are keeping the old root volume from the 3140.  If the 8040 is already in service it has new WWNNs, so the lun serials will change when you bring them online.  When going 7->7 it was common to reserial the luns back to their original state, and that should be covered in the controller hardware upgrade guide.

If you are going 7->C, then instead you should follow the SAN Host Transition and Remediation Guide included in the 7MTT documentation.  

[peiffer83]

The 8040 is already up and running 7 mode with its own aggregate and new wwns. Can you link me to that guide to reserial the luns?

[SeanHatfield]

It was a common issue with VMFS datastores during controller upgrades, so the Headswap guide calls out this KB:

https://kb.netapp.com/support/index?page=content&id=3011208

[peiffer83]

Thanks Sean,

I am just going to summerize what I learned.

When we first transitioned the server from 7mode to cluster mode it came up with a new disk signature.  A disk signature is like a MAC address for a LUN. 

For example here is a disk signature for the LUN Oracle sits on:

360a98000 + 646f4c4a474a6a6c385a7755

Controller Type ID + LUN Identification

When I first copied the LUN to the 8040 running cluster Mode it changed its LUN Serial Number from  360a98000646f4c4a474a6a6c385a7755 to 3600a098038303747695d495251426c36.  Netapp has two different operating systems (7mode/Cluster Mode).  The new Netapp runs Cluster mode.  7MODE prefix = 360a98000 and Cluster Mode prefix = 3600a0980.   So that explains the new prefix and since the LUN was copied it generated a new LUN serial number in blue 38303747695d495251426c36.

Oracle Linux is expecting the LUN Serial to never change and if it does it cannot mount the file system.  After a LUN is copied the server is shut down to transition to the new array running cluster mode.  While the server was down I brought the new LUNs offline and changed their serial numbers back to the originals on the old Netapp 646f4c4a474a6a6c385a7755. 

Netapp Commands example for one LUN:

lun offline -vserver  -path /vol/LUN

lun modify -vserver -path /vol/LUN -serial doLJGJjl8ZwU

lun online -vserver -path /vol/LUN

doLJGJjl8ZwU = 0x646f4c4a474a6a6c385a7755 in HEX which matches the number above in blue. 

After the serial was changed the Linux system was brought back online and saw this serial number 3600a0980646f4c4a474a6a6c385a7755.

Then our LINUX admins had to edit the Multipath.conf file and remove the OLD WWID:

multipath {

                wwid                     360a98000646f4c4a474a6a6c385a7755 (FAS3210 7-MODE) OLD

                                             3600a0980646f4c4a474a6a6c385a7755 (FAS8040 C-MODE) NEW

                alias                   ORACLE LUN1

        }

After that was done and old paths were removed from the multipath bindings file all was well and the Linux server was happy.  I have verified with another linux server already connected to our other site that runs 7mode uses the 360a98000 prefix.  This means that if the serial numbers change on the LUN’s all I have to do is document them before to move and change them after.  Then power on the Linux servers and they won’t know any difference.

Steps to follow for 7mode-C-Mode site migrations.

1. Document all LUN serial numbers before disk shelf move in a spreadsheet
2. Document current Multipath configs on all Linux servers.
3. Change Oracle Databases to disabled on Reboot.
4. Change Grub.conf to boot into single user mode.
5. Shut down Linux Server.
6. Verify Netapp LUN S/Ns after Disk Shelfs are attached.
7. Power On server and boot into single user mode.
8. Check if prefix has changed (if no change and serial is the same) Try to start system normally.
9. If Prefix has changed:

Change Multipath.conf file to new Netapp Prefix 3600a098….

EXAMPLE:

multipath {

                wwid                     360a98000646f4c4a474a6a6c385a7755 (FAS3210)

3600a0980646f4c4a474a6a6c385a7755 (FAS8040)

                alias                   ORACLE LUN1

        }

1. Exit single user shell.
2. Verify all LUN’s are connected and mounted.
3. Verify ASM can access LUNs.
4. Remove Single User mode from Grub.
5. Add Oracle Databases to start automatically.
6. Remove extra multipath bindings if needed. 

-https://kb.netapp.com/support/index?page=content&id=2012720&actp=LIST_RECENT&viewlocale=en_US&searchid=1460739986114

I'll update this thread when I complete the move to explain what happened.

[MikeSchaus]

How (which toll etc) did you convert the LUN Serial in hex format?

[peiffer83]

http://www.voleg.info/netapp_serial_to_wwid_converter.html