ONTAP Discussions
ONTAP Discussions
When upgrading OnTap minor release versions (P1 -> P5), the following cmdlet works as expected:
Invoke-NcClusterHaTakeover -Controller $connectedController -Node $controllerBNode
When upgrading OnTap major release versions (9.2 -> 9.3), the following cmdlet produces the following error message:
Failed to initiate takeover. Reason: version mismatch. Note: If you are performing a Data ONTAP non-disruptive upgrade or downgrade, use the (privilege: advanced) "-option allow-version-mismatch" parameter.
This is all well and good and so I add AllowVersionMismatch switch parameter as follows:
Invoke-NcClusterHaTakeover -Controller $connectedController -Node $controllerBNode -AllowVersionMismatch
But receive the same error message:
Failed to initiate takeover. Reason: version mismatch. Note: If you are performing a Data ONTAP non-disruptive upgrade or downgrade, use the (privilege: advanced) "-option allow-version-mismatch" parameter.
I'm hoping it is obvious what I missing and/or how I elevate to "set -privilege advanced" from the Netapp Powershell CLI. I have considered performing this takeover step with Invoke-NcSSH, but I am hoping there is a more elegant way and that I am just missing a step or something obvious. (Invoke-NcSsh option mentioned here: https://community.netapp.com/t5/Software-Development-Kit-SDK-and-API-Discussions/Do-we-have-any-DataONTAP-powershell-toolkit-cmdlet-to-set-privilege-l...)
The parameter help talks about cleanly halting the node to be taken over but this doesn't quite make sense to me - is it a clue to someone else? I am not familiar with proatively halting a node during a NDU:
-AllowVersionMismatch If specified, allow takeover if the partner was running an incompatible operating system version and was cleanly halted. This option is used for non-disruptive upgrade (NDU).
Thanks in advance for your insight!
Solved! See The Solution
A quick follow-up to this issue.
I finally have the zapi call workarounds in place to get around the broken cmdlets/broken support on ONTAP. Included below are the various zapi blocks I have to workaround broken Invoke-NcSSH, and broken Invoke-NcClusterHATakeover cmdlets.
Disable AutoGiveback as suggested during an OnTAP NDU
$disableAutoGiveBackANodeApiCall = @" <cf-modify-iter> <query> <options-related-info-modify> <node>$controllerANode</node> </options-related-info-modify> </query> <attributes> <options-related-info-modify> <auto-giveback-enabled>false</auto-giveback-enabled> </options-related-info-modify> </attributes> </cf-modify-iter> "@ Invoke-NcSystemApi -Controller $connectedController -Request $disableAutoGiveBackANodeApiCall -ErrorAction Stop | Out-Null $disableAutoGiveBackBNodeApiCall = @" <cf-modify-iter> <query> <options-related-info-modify> <node>$controllerBNode</node> </options-related-info-modify> </query> <attributes> <options-related-info-modify> <auto-giveback-enabled>false</auto-giveback-enabled> </options-related-info-modify> </attributes> </cf-modify-iter> "@ Invoke-NcSystemApi -Controller $connectedController -Request $disableAutoGiveBackBNodeApiCall -ErrorAction Stop | Out-Null
Invoke Takeover during an NDU with version mismatch
$Request = @" <cf-takeover> <node>$controllerBNode</node> <version-mismatch-ok>true</version-mismatch-ok> </cf-takeover> "@ Invoke-NcSystemApi -Controller $connectedController -Request $Request -ErrorAction Stop | Out-Null
Revert LIFs not at home after an NDU
ForEach ( $vServer in $vServers ) { $lifsNotHome = Get-NcNetInterface -Controller $connectedController -Vserver $vServer.Vserver | Where-Object { $_.IsHome -eq $FALSE } | Select-Object -ExpandProperty InterfaceName ForEach ( $lif in $lifsNotHome ) { $revertLifApiCall = @" <net-interface-revert> <vserver>$($vServer.Vserver)</vserver> <interface-name>$lif</interface-name> </net-interface-revert> "@ try { if ( $vServer.VserverType -like "admin" ) { Invoke-NcSystemApi -Controller $connectedController -Request $revertLifApiCall -ErrorAction SilentlyContinue | Out-Null } else { Invoke-NcSystemApi -Controller $connectedController -Request $revertLifApiCall -ErrorAction Stop | Out-Null } } catch { Write-Warning "$controller`: LIF revert was not successful" } } }
I hope this helps out my fellow Netapp wrangers.
Best
n
Hi,
Did you try using "Invoke-NcSystemApi" using the "cf-failover" ZAPI?
Alternately you could use Invoke-NcSsh and pass mulitple commands using ampersand...EG:
[String]$command = "set -privilege advanced`;storage failover takeover -ofnode cluster1-02" Try{ Invoke-NcSsh -Command $command -Credential $credentials -ErrorAction Stop Write-Host "Executed Command`: $command" }Catch{ Write-Warning -Message $("Failed Executing Command`: $command. Error " + $_.Exception.Message) Break; }
Whilst that might be a workaround it's not really a solution and it does sound like a bug to me. I'll raise the issue with our developers for you. Thanks
/Matt
Thanks for the quick reply Matt!
I should have mentioned in the first place I was using the Powershell Toolkit 4.3 and the issue was previously present in 4.0 -- I will see if 4.5 fixes the issue and report back when our next set of controllers comes up for upgrades (it will be a few weeks).
I'm not terribly familiar with the ZAPI but I came up with the following block - I won't be able to test on an HA cluster for a few weeks:
$Request = @" <cf-takeover> <node>$controllerBNode</node> <version-mismatch-ok>true</version-mismatch-ok> </cf-takeover> "@ Invoke-NcSystemApi -Controller $connectedController -Request $Request
Does that look reasonable?
Unfortunately, Invoke-NcSsh isn't the best solution at the moment as it appears cDot 9.3 may affected by the same bug as 7mode 8.2.5 as discussed here here: http://community.netapp.com/t5/Microsoft-Cloud-and-Virtualization-Discussions/Invoke-NaSsh-not-working-anymore-with-DOT-8-2-5-7-mode/m-p/135824/highli...
(I replied to the Netapp empoyee mentioning the bug about this detail in case the bug should be expanded to include cDot - I upgraded to PSTK 4.5 and the issue persists)
The upgrade script we built uses Invoke-Ssh at the start and end to invoke autosupport and disable autogiveback. Immediately after upgrading to 9.3, Invoke-NcSsh started returning:
Invoke-NcSsh : An established connection was aborted by the software in your host machine.
Best
n
A quick follow-up to this issue.
I finally have the zapi call workarounds in place to get around the broken cmdlets/broken support on ONTAP. Included below are the various zapi blocks I have to workaround broken Invoke-NcSSH, and broken Invoke-NcClusterHATakeover cmdlets.
Disable AutoGiveback as suggested during an OnTAP NDU
$disableAutoGiveBackANodeApiCall = @" <cf-modify-iter> <query> <options-related-info-modify> <node>$controllerANode</node> </options-related-info-modify> </query> <attributes> <options-related-info-modify> <auto-giveback-enabled>false</auto-giveback-enabled> </options-related-info-modify> </attributes> </cf-modify-iter> "@ Invoke-NcSystemApi -Controller $connectedController -Request $disableAutoGiveBackANodeApiCall -ErrorAction Stop | Out-Null $disableAutoGiveBackBNodeApiCall = @" <cf-modify-iter> <query> <options-related-info-modify> <node>$controllerBNode</node> </options-related-info-modify> </query> <attributes> <options-related-info-modify> <auto-giveback-enabled>false</auto-giveback-enabled> </options-related-info-modify> </attributes> </cf-modify-iter> "@ Invoke-NcSystemApi -Controller $connectedController -Request $disableAutoGiveBackBNodeApiCall -ErrorAction Stop | Out-Null
Invoke Takeover during an NDU with version mismatch
$Request = @" <cf-takeover> <node>$controllerBNode</node> <version-mismatch-ok>true</version-mismatch-ok> </cf-takeover> "@ Invoke-NcSystemApi -Controller $connectedController -Request $Request -ErrorAction Stop | Out-Null
Revert LIFs not at home after an NDU
ForEach ( $vServer in $vServers ) { $lifsNotHome = Get-NcNetInterface -Controller $connectedController -Vserver $vServer.Vserver | Where-Object { $_.IsHome -eq $FALSE } | Select-Object -ExpandProperty InterfaceName ForEach ( $lif in $lifsNotHome ) { $revertLifApiCall = @" <net-interface-revert> <vserver>$($vServer.Vserver)</vserver> <interface-name>$lif</interface-name> </net-interface-revert> "@ try { if ( $vServer.VserverType -like "admin" ) { Invoke-NcSystemApi -Controller $connectedController -Request $revertLifApiCall -ErrorAction SilentlyContinue | Out-Null } else { Invoke-NcSystemApi -Controller $connectedController -Request $revertLifApiCall -ErrorAction Stop | Out-Null } } catch { Write-Warning "$controller`: LIF revert was not successful" } } }
I hope this helps out my fellow Netapp wrangers.
Best
n