Upgrading Clustered Data ONTAP 8.3x to ONTAP 9.1 Using Automated Nondisruptive Upgrade Method

[mcgue]

One of the methods available for upgrading between 8.3x to 9.1 is the Automated Nondisuptive Upgrade (ANDU) method.  This method simplifies the upgrade process and removes much of the burden of the process.  There still remains a few items that need to be reviewed and performed prior to using the ANDU method.  Following is an outline of the manual checks that the ANDU requires.  For the complete documentation on the process use the Upgrade Express Guide.

1. Decide if the ANDU method is the correct method to use for your cluster.  Follow this flowchart.
2. Watch this video on the process.
3. Ensure compatibility of your cluster and infrastructure with ONTAP 9.1.
   • Check that the model of the cluster nodes are supported in ONTAP 9.1 and verify limits using the Hardware Universe
   • Review the ONTAP 9.1 Release Notes.  In particular review the Upgrade Cautions.
   • Verify compatibility of cluster switches, external clients, infrastructure, software, etc. using the Interoperability Matrix.
4. Confirm the cluster is ready for ANDU.
   • Verify that NFS clients are using hard mounts.
   • Do not issue commands to alter configurations while in mixed version state.
5. If your cluster contains at least 8 nodes then disable automatic LIF rebalancing.
6. Use a PowerShell script to verify additional cluster health.  Add the following parameter to the script to perform the following checks:  "-RunSection Sysstat,SAN,Jobs"
   
   • CPU and disk.
   • SAN configuration.
   • Running jobs.

Remember you can also create an Upgrade Advisor report at the My AutoSupport Site.

[hariprak]

Hi,

Thanks for sharing this information.

[mcgue]

Added a step about LIF rebalancing as ANDU can perform a batch update for larger clusters. 

[its-libertybell]

SnapMirror and Node Upgrade Order

tl-dr ( how do I upgrade ONTAP when clusters are both a source and a destination for SnapMirrors. Instructions say destination first. Whaaa? )

We are planning upgrade from 8.3.1 to 9.1P5 . The AutoSupport Upgrade Advisor Plan generated for this task, gives the following advice for suspending SnapMirror replication, and the correct order in which to upgrade the Peers of the SnapMirror relationship:

Suspending SnapMirror operations

WARNING

Cluster mycluster2 is running SnapMirror.

To prevent SnapMirror transfers from failing, you must suspend SnapMirror operations and upgrade destination nodes before upgrading source nodes.

(i) Suspend SnapMirror transfers for a destination volume

(ii) Upgrade the node that contains the destination volume

(iii) Upgrade the node that contains the source volume

(iv) Resume the SnapMirror transfers for the destination volume.

Note: SnapMirror transfers for all other destination volumes can continue while the nodes that contain the original destination and source volumes are upgraded.

Our problem is that mycluster2 and the Cluster it is SnapMirroring to, mycluster1 have Volumes where SnapMirror replication is going in both directions:

mycluster1 - svm1 - vol1     SnapMirror--> mycluster2 - svm2 - vol1_dr
mycluster1 - svm1 - vol2_dr  SnapMirror<-- mycluster2 - svm2 - vol2

So, the question is, which cluster should I upgrade to 9.1P5 first? Each cluster contains both source, and destination volumes.

[mcgue]

That certainly makes a tough decision.  As you have it you would need to suspend SnapMirrors originating from the cluster you upgrade first.  The alternative though to this problem is configuring version-flexible SnapMirror relationships.  That is described starting on page 99 here: 

https://library.netapp.com/ecm/ecm_download_file/ECMP1610205

[gpallis]

Hello all. I am planning to do a similar upgrade for our AFF8080 currently running on 8.3.2P5 to 9.1P5 (or P6).

I have checked all the advisories, bugs, etc. and I can’t see any show stoppers for going ahead. Did anyone experienced any issues after performing this upgrade? Thanks

[Amandeep]

Make sure do this before upgrade.

system node run -node node0* options wafl.deswizzle.enable off

else cluster runs into performance issues after ugrade as inode upgrade starts.

[gpallis]

Thanks for the tip Amandeep. I 'm not running any Snapmirroring on this filer at the moment and to be honest disabling deswizzle is not something I did in the past before an OS upgrade. Unless if things have changed with the latest CDOT versions.

[netappmagic]

- For 3 HA pairs (6 nodes) how System Manager Upgrade determine which HA will  be upgraded first?

- Will it be automatically go to next HA pair after this pair is done?

- Are there any human intervention required?

Thanks!

[gpallis]

Upgrading to 9.1P6 from 8.3.2P5 went fine with absolutely no problems for the AFF8080 here. Couple of weeks now and it's running fine so far

[LionO]

Upgraded from 8.3.1 to 9.1P5 using this method, just before 9.1P6 came out. Appeared to all be fine, but actually tanked performance on one of our NLSAS with FlashPool Aggregates within an hour of the upgrade. Support have been unable to identify why, and we have had to move high workload volumes off the aggregate to return to acceptable perfomance. Recommmend you collect perfstat using GUIPerfstat on your systems both before and after the upgrade, covering periods of low load and high load (e.g. Full Backups of VMs)

We have seen performance drop and not increase. Case has been open with support since 9th August 2017. Be smart, and be ready to be able to back up any claim that an upgrade has reduced perfomance, and to be able to give infomation so that NetApp support can find the cause and solve it. In the graph below of Aggregate Latency, where the line touches the x-axis is the upgrade. Line of best fit jumps 5ms up in latency within 20mins of the upgrade completing, permanently. This caused dropouts on VMs during peak load, and we were unable to complete Full Backups in a timely fasion. 

NetApp Support are doing what they can, but without perfstat data from just before the upgrade, they are having a tough time finding the source of the loss in performance. Get perfstat data people. GET IT.

[ubimet]

Hi LionO,

we also want to upgrade, but before, I wanted to ask you, if NetApp was able to resolve your perfromance issues?

BR Florian

[qw]

Hi,

  Please find latest ANDU demo.

  OnCommand System Manager 9.3, bundled with ONTAP 9.3 allows the user to upgrade easily between LTS release as Direct one Stage LTS upgrade. 

  This demo describes how the end-user can easily upgrade from one LTS (Long Term Service) release say 9.1 to its immediate LTS release 9.3.