Thanks Sean,
I am just going to summerize what I learned.
When we first transitioned the server from 7mode to cluster mode it came up with a new disk signature. A disk signature is like a MAC address for a LUN.
For example here is a disk signature for the LUN Oracle sits on:
360a98000 + 646f4c4a474a6a6c385a7755
Controller Type ID + LUN Identification
When I first copied the LUN to the 8040 running cluster Mode it changed its LUN Serial Number from 360a98000646f4c4a474a6a6c385a7755 to 3600a098038303747695d495251426c36. Netapp has two different operating systems (7mode/Cluster Mode). The new Netapp runs Cluster mode. 7MODE prefix = 360a98000 and Cluster Mode prefix = 3600a0980. So that explains the new prefix and since the LUN was copied it generated a new LUN serial number in blue 38303747695d495251426c36.
Oracle Linux is expecting the LUN Serial to never change and if it does it cannot mount the file system. After a LUN is copied the server is shut down to transition to the new array running cluster mode. While the server was down I brought the new LUNs offline and changed their serial numbers back to the originals on the old Netapp 646f4c4a474a6a6c385a7755.
Netapp Commands example for one LUN:
lun offline -vserver -path /vol/LUN
lun modify -vserver -path /vol/LUN -serial doLJGJjl8ZwU
lun online -vserver -path /vol/LUN
doLJGJjl8ZwU = 0x646f4c4a474a6a6c385a7755 in HEX which matches the number above in blue.
After the serial was changed the Linux system was brought back online and saw this serial number 3600a0980646f4c4a474a6a6c385a7755.
Then our LINUX admins had to edit the Multipath.conf file and remove the OLD WWID:
multipath {
wwid 360a98000646f4c4a474a6a6c385a7755 (FAS3210 7-MODE) OLD
3600a0980646f4c4a474a6a6c385a7755 (FAS8040 C-MODE) NEW
alias ORACLE LUN1
}
After that was done and old paths were removed from the multipath bindings file all was well and the Linux server was happy. I have verified with another linux server already connected to our other site that runs 7mode uses the 360a98000 prefix. This means that if the serial numbers change on the LUN’s all I have to do is document them before to move and change them after. Then power on the Linux servers and they won’t know any difference.
Steps to follow for 7mode-C-Mode site migrations.
- Document all LUN serial numbers before disk shelf move in a spreadsheet
- Document current Multipath configs on all Linux servers.
- Change Oracle Databases to disabled on Reboot.
- Change Grub.conf to boot into single user mode.
- Shut down Linux Server.
- Verify Netapp LUN S/Ns after Disk Shelfs are attached.
- Power On server and boot into single user mode.
- Check if prefix has changed (if no change and serial is the same) Try to start system normally.
- If Prefix has changed:
Change Multipath.conf file to new Netapp Prefix 3600a098….
EXAMPLE:
multipath {
wwid 360a98000646f4c4a474a6a6c385a7755 (FAS3210)
3600a0980646f4c4a474a6a6c385a7755 (FAS8040)
alias ORACLE LUN1
}
- Exit single user shell.
- Verify all LUN’s are connected and mounted.
- Verify ASM can access LUNs.
- Remove Single User mode from Grub.
- Add Oracle Databases to start automatically.
- Remove extra multipath bindings if needed.
-https://kb.netapp.com/support/index?page=content&id=2012720&actp=LIST_RECENT&viewlocale=en_US&searchid=1460739986114
I'll update this thread when I complete the move to explain what happened.