ONTAP upgrade from 9.12.1 to 9.14.1 failed on one of two node cluster

[MarcusSerrao]

Hey all.

In the middle of an upgrade of new FAS2750 via the ONTAP manager web portal. Upgrade is from 9.12.1 P2 -> 9.14.1 P6.

The 1st node in the 2-node cluster could not be upgraded to 9.14.1 P6 while the 2nd one upgraded successfully.

Error indicated in the console:

cluster2::cluster image show-update-progress

cluster2-01

status: failed

error: Failed to migrate data LIFs from node "cluster2-01". Action Migrate all of the data LIFs using the "network interface migrate-all -node cluster2-01" command

cluster2-02

status: completed

Status: Paused - An error occurred in the "ONTAP updates" phase. The update cannot continue until the error has been resolved. Solve all issues, then use the "cluster image resume-update" command to resume the update.

Any help? 

[SpindleNinja]

What LIFs do you have on that node?  and have you tried to migrate the lifs?      

[MarcusSerrao · Accepted solution]

Hi. Thanks for the reply. I figured out the issue.

Since this is a new device, I initialized the management ports, provisioned the storage and created a data SVM with iSCSI data LIFs before doing the ONTAP upgrade. I wasn't sure how much had to be initially configured to upgrade the device. 

By process of elimination, removing the data LIFs and leaving only the management LIFS, I realized that the management ports (e0M) for each node were plugged into different VLANs by mistake. This put them in two different broadcast domains and as a result, the floating cluster management was bound to the 1st node and had no failover port in it's policy. During the upgrade of the 1st node, it couldn't fail this port over to the 2nd node so it errored out with the "Failed to migrate data LIFs" error.  Once I plugged both management ports into the same VLAN and flipped the cluster management IP to the 2nd node management port via command line, the upgrade of the 1st node went through. 

Learned a bit there 🙂