Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
ERROR netapp.manage.NaAPIFailedException: (errno=16)
2013-08-07
08:36 AM
6,634 Views
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Error when trying to delete snapshot, the snapshot is never deleted
I can manually delete the snapshot
SnapCreator debug log
########## Running Snapshot copy Delete on Primary ##########
[2013-08-07 17:05:36,963] INFO: STORAGE-02090: Listing Snapshot copies on volume [wy1_vsphere_nfs_dev_linux].
[2013-08-07 17:05:36,963] INFO: STORAGE-02091: Listing Snapshot copies on volume [wy1_vsphere_nfs_dev_linux] finished successfully.
[2013-08-07 17:05:37,884] INFO: STORAGE-02013: Removing Snapshot copy [sc-hourly_20130803110100] on volume [wy1_vsphere_nfs_dev_linux].
[2013-08-07 17:05:37,884] ERROR: com.netapp.snapcreator.storage.executor.ZapiExecutorException: netapp.manage.NaAPIFailedException: (errno=16)
at com.netapp.snapcreator.storage.executor.ZapiExecutorImpl.run(ZapiExecutorImpl.java:62)
at com.netapp.snapcreator.storage.api.ontap.Ontap7ModeApi.snapshotDelete(Ontap7ModeApi.java:228)
at com.netapp.snapcreator.storage.StorageCoreImpl.snapshotDelete(StorageCoreImpl.java:831)
at com.netapp.snapcreator.workflow.task.ZAPITask.deleteSnapshot(ZAPITask.java:908)
at com.netapp.snapcreator.workflow.task.SnapDelete.execute(SnapDelete.java:99)
at com.netapp.snapcreator.workflow.impl.SCTaskCallable.call(SCTaskCallable.java:48)
at com.netapp.snapcreator.workflow.impl.SCTaskCallable.call(SCTaskCallable.java:20)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: netapp.manage.NaAPIFailedException: (errno=16)
at netapp.manage.NaServer.invokeElem(NaServer.java:671)
at com.netapp.snapcreator.storage.executor.ZapiExecutorImpl.run(ZapiExecutorImpl.java:53)
... 11 more
[2013-08-07 17:05:37,884] ERROR: STORAGE-02015: Removing Snapshot copy [sc-hourly_20130803110100] on volume [wy1_vsphere_nfs_dev_linux] failed with werror [netapp.manage.NaAPIFailedException: (errno=16)].
########## Snap Creator Framework 4.0p1 failed ##########
[2013-08-07 17:05:37,884] INFO: Pre Exit commands are not defined. Skipping !
[2013-08-07 17:05:38,072] INFO: STORAGE-05016: Creating OM Event [script:critical-event] on [beoncommand].
[2013-08-07 17:05:38,072] INFO: STORAGE-05017: OM Event [script:critical-event] on [beoncommand] created successfully.
Netapp Filer log
BEONCOMMAND [bevcenter5.cmceu.chicago-metallic.ad:9091]: SNAPCREATOR Snap Creator Framework 4.0p1: (0) Backup Started: INFO: Snap Creator Framework 4.0p1 Backup for sc ACTION: backup POLICY: hourly Plugin: vibe Supported Volumes: wy1_vsphere_nfs_dev_linux, Started
BEONCOMMAND [bevcenter5.cmceu.chicago-metallic.ad:9091]: SNAPCREATOR Snap Creator Framework 4.0p1: (0) Backup Completed: INFO: Snap Creator Framework 4.0p1 Backup for sc ACTION: backup POLICY: hourly Plugin: vibe Supported Volumes: wy1_vsphere_nfs_dev_linux, finished
BEONCOMMAND [bevcenter5.cmceu.chicago-metallic.ad:9091]: SNAPCREATOR Snap Creator Framework 4.0p1: (0) Snapmirror Backup of bebrfiler1:snapmirror_wy1_vsphere_nfs_dev_linux Started: INFO: Snap Creator Framework 4.0p1 Snapmirror Backup of bebrfiler1:snapmirror_wy1_vsphere_nfs_dev_linux for sc ACTION: backup POLICY: hourly Plugin: vibe Supported Volumes: wy1_vsphere_nfs_dev_linux, Started
8 REPLIES 8
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
SC is getting error from ONTAP. What could be happening is that you are using snapmirror and what happens in this case is snapshots get locked for a little while. Usually folks add sleep NTAP_POST_DATA_TRANSFER_CMD01=sleep 60. If that isnt case there may be some other issue, I would open an NGS support case as there could be something else non-SC related happening.
Regards,
Keith
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Keith,
I can't find any information on NTAP_POST_DATA_TRANSFER_CMD01
So I have tried to add POST_NTAP_DATA_TRANSFER_CMD01=sleep 60 to the snapcreator cinfig file.
But I get the following error message:
[2013-08-08 09:32:11,056] INFO: [bevcenter5.cmceu.chicago-metallic.ad:9091 (4.0.0.1)] Executing Post data transfer command [sleep 60] on bevcenter5.cmceu.chicago-metallic.ad
[2013-08-08 09:32:11,056] ERROR: [bevcenter5.cmceu.chicago-metallic.ad:9091 (4.0.0.1)] [agt-00011] Command [sleep 60] is not allowed
or i'm i doing something wrong? probably 🙂
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sorry didnt know you had agent, you should read about commands in AIG
POST_NTAP_DATA_TRANSFER_CMD01=SERVER: sleep 60
That should work
Keith
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
AIG = ?
The only explanation I can find on the command is:
POST_NTAP_DATA_TRANSFER_CMD<##>
This is a post data transfer command that runs after SnapVault or SnapMirror transfer, where ## is a number from 01-99.
The SERVER: sleep 60 command
SERVER = just SERVER or the Netapp Filer name, ...?
Just using SERVER
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 13:34:30,709] INFO: Executing Post data transfer command sleep 60 on server
[2013-08-08 13:34:30,709] ERROR: commands failed with following error
[2013-08-08 13:34:30,709] ERROR: Cannot run program "sleep": CreateProcess error=2, The system cannot find the file specified
########## Snap Creator Framework 4.0p1 failed ##########
Using netapp filer name
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 13:43:20,134] INFO: [bevcenter5.cmceu.chicago-metallic.ad:9091 (4.0.0.1)] Executing Post data transfer command [BEWYFILER1: sleep 60] on bevcenter5.cmceu.chicago-metallic.ad
[2013-08-08 13:43:20,134] ERROR: [bevcenter5.cmceu.chicago-metallic.ad:9091 (4.0.0.1)] [agt-00011] Command [BEWYFILER1: sleep 60] is not allowed
########## Snap Creator Framework 4.0p1 failed ##########
SnapCreator server = Windows 2008 R2
SnapCreator framework= 4.0p1
Agent server= Windows 2008 R2 (vCenter 5.1U1)
NetApp filers: NetApp Release 8.1 7-Mode
Grz,
Geert
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
AIG is the admin guide
I guess this isnt documented there...my apologies
Anyway for all pre / post cmds default behavior is send cmds to SC Agent. If you want to run them on server you need to specify SERVER: or you can specify AGENT: for agent. There is also CLONETARGET: if you are using a second agent for cloning.
You are also using windows so I think you need POST_NTAP_DATA_TRANSFER_CMD01=SERVER: cmd /c sleep 60 OR POST_NTAP_DATA_TRANSFER_CMD01=SERVER: "cmd /c sleep 60" OR POST_NTAP_DATA_TRANSFER_CMD01=SERVER: "sleep 60"
One or all of those should work
Keith
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ok nevermind obviously windows doesnt have a sleep command...sorry I am a unix guy.
Point is you need to try and have SC wait and you do this by sleeping. I see nothing really that useful on windows so most people say to use ping
ping localhost -n 1 -w 60000 > nul
This should ping for 60 seconds
POST_NTAP_DATA_TRANSFER_CMD01=SERVER: ping localhost -n 1 -w 60000 > nul
Hope this helps, if not I am afraid someone else who knows windows better will have to jump in
Keith
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Depending on the version of Windows the TIMEOUT command should work for you.
All we're doing is sending the command to introduce a pause.
Open a command line on your Windows system and type:
TIMEOUT /T 60
You should get a countdown on screen that reads : Waiting for 60 seconds, press a key to continue ...
If this works add it into your config file:
POST_NTAP_DATA_TRANSFER_CMD01=SERVER: "TIMEOUT /T 60"
Don't forget the space after the ":" in SERVER:
If it doesn't work you may need to specify that this is a command line call by using:
POST_NTAP_DATA_TRANSFER_CMD01=SERVER: cmd.exe /c "TIMEOUT /T 60"
Hope this helps,
John
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Little update (going crazy here)
Some information I found:
http://communities.quest.com/thread/17401
The last 2 work
SERVER: timeout 30
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 14:47:36,455] INFO: Executing Post data transfer command timeout 30 on server
[2013-08-08 14:47:36,455] INFO: stdout:
[2013-08-08 14:47:36,455] ERROR: stderr:ERROR: Input redirection is not supported, exiting the process immediately.
########## Snap Creator Framework 4.0p1 failed ##########
SERVER: "TIMEOUT /T 60"
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 14:52:04,629] INFO: Executing Post data transfer command "TIMEOUT /T 60" on server
[2013-08-08 14:52:04,645] ERROR: commands failed with following error
[2013-08-08 14:52:04,645] ERROR: Cannot run program ""TIMEOUT": CreateProcess error=2, The system cannot find the file specified
########## Snap Creator Framework 4.0p1 failed ##########
SERVER: C:\Windows\System32\timeout /T 5
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 15:01:24,851] INFO: Executing Post data transfer command C:\Windows\System32\timeout /T 5 on server
[2013-08-08 15:01:24,851] INFO: stdout:
[2013-08-08 15:01:24,851] ERROR: stderr:ERROR: Input redirection is not supported, exiting the process immediately.
########## Snap Creator Framework 4.0p1 failed ##########
SERVER: C:\Windows\System32\timeout /T 5 1>nul
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 15:05:42,869] INFO: Executing Post data transfer command C:\Windows\System32\timeout /T 5 1>nul on server
[2013-08-08 15:05:42,869] INFO: stdout:
[2013-08-08 15:05:42,869] ERROR: stderr:ERROR: Invalid syntax. Default option is not allowed more than '1' time(s).
Type "TIMEOUT /?" for usage.
########## Snap Creator Framework 4.0p1 failed ##########
c:\temp\test.cmd = C:\Windows\System32\timeout /T 5 1>nul
SERVER: c:\temp\test.cmd
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 15:13:07,137] INFO: Executing Post data transfer command c:\temp\test.cmd on server
[2013-08-08 15:13:07,137] INFO: stdout:
C:\Windows\system32>C:\Windows\System32\timeout /T 5 1>nul
[2013-08-08 15:13:07,137] ERROR: stderr:ERROR: Input redirection is not supported, exiting the process immediately.
########## Snap Creator Framework 4.0p1 failed ##########
SERVER: ping -n 5 127.0.0.1 1>nul
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 15:25:30,454] INFO: Executing Post data transfer command ping -n 5 127.0.0.1 1>nul on server
[2013-08-08 15:25:30,454] INFO: stdout:Bad parameter 1>nul.
[2013-08-08 15:25:30,454] ERROR: stderr:
########## Snap Creator Framework 4.0p1 failed ##########
SERVER: ping -n 5 127.0.0.1
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 15:20:03,624] INFO: Executing Post data transfer command ping -n 5 127.0.0.1 on server
[2013-08-08 15:20:03,624] INFO: stdout:
Pinging 127.0.0.1 with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Ping statistics for 127.0.0.1:
Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
[2013-08-08 15:20:03,624] INFO: Post data transfer completed successfully
########## POST DATA TRANSFER COMMANDS FINISHED SUCCESSFULLY ##########
c:\temp\test.cmd = ping -n 5 127.0.0.1 1>nul
SERVER: c:\temp\test.cmd
########## POST DATA TRANSFER COMMANDS ##########
[2013-08-08 15:28:38,877] INFO: Executing Post data transfer command c:\temp\test.cmd on server
[2013-08-08 15:28:38,877] INFO: stdout:
C:\Windows\system32>ping -n 5 127.0.0.1 1>nul
[2013-08-08 15:28:38,877] INFO: Post data transfer completed successfully
########## POST DATA TRANSFER COMMANDS FINISHED SUCCESSFULLY ##########
![](/skins/images/094DD352636536671740FB44AC67F2EA/responsive_peak/images/icon_anonymous_message.png)