topic Re: Invoke-NcSnapMirrorBreak - Another SnapMirror operation is in progress. in Microsoft Virtualization Discussions

Invoke-NcSnapMirrorBreak - Another SnapMirror operation is in progress.

MRoney — Fri, 27 Aug 2021 05:07:02 GMT

I am trying to script the snapmirror failover process using PTK.

When I run the following set of commands,

# Update Existing SnapMirror and then Break it off
Invoke-NcSnapMirrorUpdate -Controller $DstCtrl -DestinationVserver $DstSVM -DestinationVolume $DstVol -ErrorAction stop
Watch-Command { Get-NcSnapMirror -Controller $DstCtrl -DestinationVserver $DstSVM -DestinationVolume $DstVol | Select Status } -Until "idle"
Invoke-NcSnapMirrorQuiesce -Controller $DstCtrl -DestinationVserver $DstSVM -DestinationVolume $DstVol -ErrorAction stop
Watch-Command { Get-NcSnapMirror -Controller $DstCtrl -DestinationVserver $DstSVM -DestinationVolume $DstVol | Select Status } -Until "quiesced"
Invoke-NcSnapMirrorBreak -Controller $DstCtrl -DestinationVserver $DstSVM -DestinationVolume $DstVol -Confirm:$false -ErrorAction stop

the Invoke-NcSnapMirrorBreak sometime returns with an error:

Invoke-NcSnapMirrorBreak : Another SnapMirror operation is in progress.
At Z:\Software\NetApp\Powershell Toolkit\Scripts\SnapMirror-Reverse-v3.ps1:119 char:1
+ Invoke-NcSnapMirrorBreak -Controller $DstCtrl -DestinationVserver $Ds ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (apfas8200:NcController) [Invoke-NcSnapmirrorBreak], EANOTHER_OP_ACTIVE
+ FullyQualifiedErrorId : ApiException,DataONTAP.C.PowerShell.SDK.Cmdlets.Snapmirror.InvokeNcSnapmirrorBreak

Why does it report another operation is in progress? I just checked the status, and it was quiesced.

It does not fail every time it runs, just some of the time. I have many snapmirrors where I perform this operation repeatedly. Many work just fine and then others seem to fail nearly all the time. I can sometimes just resume the snapmirror and start the script again and it runs without any issues on the same snapmirror.

Is there some code I can add to check and try again... like maybe 3 times before aborting?

Or better yet, does anyone know what I am not doing correctly to make sure the snapmirror is not performing another operation before I request the snapmirror break?

Re: Invoke-NcSnapMirrorBreak - Another SnapMirror operation is in progress.

aladd — Tue, 31 Aug 2021 04:23:16 GMT

I'm not seeing anything wrong with the script. But adding a 3 second wait time following each watch-command may be helpful.

incorporating Python may help.

https://docs.netapp.com/us-en/ontap-select/concept_api_before_python.html

and I found the below on creating a time delay in Python.

https://realpython.com/python-sleep/#adding-a-python-sleep-call-with-timesleep

Re: Invoke-NcSnapMirrorBreak - Another SnapMirror operation is in progress.

NetApp_SR — Tue, 31 Aug 2021 05:25:56 GMT

I agree with aladd that adding a delay after the "Watch-Command" lines is good practice. Power shell has a built in command to add the delay. The scripts I see add 15-30 seconds.

Start-Sleep -s 15

Re: Invoke-NcSnapMirrorBreak - Another SnapMirror operation is in progress.

MRoney — Thu, 09 Sep 2021 16:50:37 GMT

Thank you for the feedback. I am adding the delay and will let you know how that worked out.

Re: Invoke-NcSnapMirrorBreak - Another SnapMirror operation is in progress.

Luis_KawaZ750 — Fri, 27 Dec 2024 18:34:02 GMT

Hi MRoney,

did you resolve with delay?

Thanks a lot.

Bye

Re: Invoke-NcSnapMirrorBreak - Another SnapMirror operation is in progress.

MRoney — Fri, 27 Dec 2024 18:53:57 GMT

Unfortunately, the delay did not resolve the issue. I ended up re-writing the code. Below is the replacement code I used. Note: I am not a great PowerShell programmer, so I am sure there is a bunch of this that could be improved, but we have not had any EANOTHER_OP_ACTIVE issues with this new code..... basically, it just keeps retrying until success.

# Update Existing SnapMirror and then Break it off
Wr G "";Wr G "Updating Existing Snapmirror:"
do {
$jobstatus = Invoke-NcSnapMirrorUpdate -Controller $DstCtrl -Destination $DstLoc -Source $SrcLoc -ONTAPI
sleep 5
Write-Host ($jobstatus | Format-Table | Out-String)
} until ($jobstatus.Status -eq 'succeeded')

do {
$smstatus = Get-NcSnapMirror -Controller $DstCtrl -Destination $DstLoc -ontapi
sleep 5
Write-Host ($smstatus | Format-Table | Out-String)
} until ($smstatus.MirrorState -eq 'snapmirrored' -and $smstatus.Status -eq 'idle')

Wr G "";Wr G "SnapMirror Status after Update:"

$smstatus = Get-NcSnapMirror -Controller $DstCtrl -Destination $DstLoc -ontapi
Write-Host ($smstatus | Format-Table | Out-String)

$worked = $false
while (-not $worked) {
try {
Invoke-NcSnapMirrorQuiesce -Controller $DstCtrl -Destination $DstLoc -ONTAPI -ErrorAction Stop
$worked = $true # An exception will skip this
} catch {
Write-Host ("Error:", $_)
Write-Host ("Retrying...")
sleep 5
}
}
do {
$smstatus = Get-NcSnapMirror -Controller $DstCtrl -Destination $DstLoc -ontapi
sleep 5
Write-Host ($smstatus | Format-Table | Out-String)
} until ($smstatus.MirrorState -eq 'snapmirrored' -and $smstatus.Status -eq 'quiesced')

Wr G "";Wr G "SnapMirror Status after Quiesce:"

Get-NcSnapMirror -Controller $DstCtrl -Destination $DstLoc -ontapi

$worked = $false
while (-not $worked) {
try {
Invoke-NcSnapMirrorBreak -Controller $DstCtrl -Destination $DstLoc -Confirm:$false -ONTAPI -ErrorAction Stop
$worked = $true # An exception will skip this
} catch {
Write-Host ("Error:", $_)
Write-Host ("Retrying...")
sleep 5
}
}
do {
$smstatus = Get-NcSnapMirror -Controller $DstCtrl -Destination $DstLoc -ontapi
sleep 5
Write-Host ($smstatus | Format-Table | Out-String)
} until ($smstatus.MirrorState -eq 'broken-off' -and $smstatus.Status -eq 'idle')

do {
$smstatus = Get-NcSnapMirror -Controller $DstCtrl -Destination $DstLoc -ontapi
sleep 5
Write-Host ($smstatus | Format-Table | Out-String)
} until ($smstatus.MirrorState -eq 'broken-off' -and $smstatus.Status -eq 'idle')
Wr G "";Wr G "SnapMirror Status after Break:"

Get-NcSnapMirror -Controller $DstCtrl -Destination $DstLoc -ontapi

Re: Invoke-NcSnapMirrorBreak - Another SnapMirror operation is in progress.

Luis_KawaZ750 — Sat, 28 Dec 2024 18:39:39 GMT

Hi MRoney,

I'll try your solution!

Thanks a lot and happy new year!

Bye