2010-11-12 07:07 AM
Hi, Provisioning Manager is being used to migrated several vFilers, but this one is repeatedly failing - can someone decipher why?
Another issue is when this failure occurs, it disables snapmirror on the head - killing other migrations and snapmirrors! (I end up having to issue an options snapmirror.enable on)
Here is the excerpt from the vfiler_trans_migrate_log – can you tell me why the cutover of app-vf-01 repeatedly fails and why it sets options snapmirror.enable off causing all other snapmirrors to abort ?
--------------------START OF NON-DISRUPTIVE MIGRATION------------------------------
THE VFILER UNDER MIGRATION: app-vf-01
Snapshot create started
Snapshot create completed took:2482 msecs
Disable visibility started
Disable visibility snapshot took:1252 msecs
Snapshot disable started
Snapshot disable completed took:0 msecs
CUTOVER started : Thu Nov 11 20:49:24 PST 2010
Step:'Shutdown iscsi sessions running in vfiler:app-vf-01 context' took:0 seconds
WAFL:Ref counts for volume 'appdata'
WAFL:vol offline on 'appdata' failed error: CR_BEING_USED
Multi-volume offline stats: Total volumes = 2, Volumes with errors = 1, Offline async msg complete = 312, Phase1 = 0, CP1 = 1166, Phase2 = 0, Phase3 = 0, CP2 = 468, Phase4 = 0, Name cache = 91, Phase5 = 3, Inode cache = 151, Phase6 = 21, CP switch = 1530, Phase7 = 121, Total Time = 3863
WAFL:vol online on 'appdata' failed error: CR_ALREADY_ONLINE
Multi-volume online stats: Total volumes = 1, Volumes with Errors = 1, Time to online = 0
Multi-volume online stats: Total volumes = 1, Volumes with Errors = 0, Time to online = 1346
Snapshot deletion took :6073 msec
Total Heartbeats - 1
Frequency - 2500 millisecond
[MON] WatchDog = 120sec
[MON] Fallback Triggered on:app-vf-01
[FALL] Fallback Success
[MON] Sleep to MEngine
2010-11-18 10:20 PM
Hi, with netapp support's help, we determined there was a stale /etc/sm/notify entry preventing the offlining of the old volume.
cp /dev/null /etc/sm/notify resolved this