SnapVault for SMVI 3.0 (SV-SMVI)

by Frequent Contributor on ‎2010-11-23 02:00 PM - edited on ‎2014-10-07 09:48 AM by allison Former NetApp Employee
NOTE: This version supercedes SV-SMVI 1.0 through 2.X.  Please use this version moving forward.  Features from SV-SMVI 1.0 and 2.X may not exist in SV-SMVI 3.X as they are no longer needed.
NOTE: Please read the documentation on setup of SV-SMVI, especially regarding configuration of sv-smvi.cmd script files.

 

Problem Statement

Some users would like to  use SMVI in combination with SnapVault environments.

 

Solution Description

SV-SMVI  3.0 is a new version of SV-SMVI, and now runs as a post-script job through the SMVI 2.0+ or VSC 2.0+ script process.  SV-SMVI determines the most recent  SMVI backup snapshot information based on output from SMVI/VSC, and then tells SnapVault secondary  controllers to pull the most recent SMVI backup to the SnapVault  secondary and take a SnapVault snapshot.  It is modeled after kb3990 (https://now.netapp.com/Knowledgebase/solutionarea.asp?id=kb3990)  which describes how to perform similar operations for older versions of  SnapManager for Exchange.  Features have been added to SV-SMVI 3.0 to allow for fan-in SnapVault secondary relationships, specifying multiple SnapVault snapshot schedules (or no schedule) and remapping of names to IP addresses in complex subnet configurations.

Category

Tool

 

Documentation

Included in the ZIP file.

 

Version

3.0.3       Added -https option and corrected documentation based on version updates.  NOTE: 07/15/2012 - updated documentation ONLY to correct -svschednames option text.

3.0.2       Corrected -nosvsnap option with command line argument conditional -- works now.

3.0.1       Added many:many relationship functionality for users who have NFS datastores but multiple qtrees inside of the datastore and separate SnapVault relationships.

3.0          *** NEW VERSION, FIRST RELEASE ***

 

License

SV-SMVI uses the NOW (NetApp On the Web) ToolChest license, available at http://now.netapp.com/NOW/cgi-bin/toolchestlic.cgi.

 

Example Output

runOpenReport(): LOG REPORT FOR SV-SMVI
runOpenReport(): -----------------------------------------------------
[14:48:14] runOpenReport(): SV-SMVI Version: 3.0
[14:48:14] runOpenReport(): Log Filename: C:\Program Files (x86)\NetApp\SV-SMVI\report\SV-SMVI_20101110_144814.log
[14:48:14] runOpenReport(): Start Time: Wed Nov 10 20:48:14 2010
[14:48:14] runCheckArgs(): Found backup: HOST = 'FAS-A.myserver.com', VOLUME = 'Datastore1', SNAPSHOT = 'smvi_MyBackup_recent' ...
[14:48:14] runCheckArgs(): Preserving SMVI backup with storage controller FAS-A.myserver.com, volume Datastore1, snapshot named smvi_MyBackup_recent ...
[14:48:14] runLogin(): Initializing connectivity to storage controller ...
[14:48:14] runLogin(): Logging into to storage controller FAS-A.myserver.com ...
[14:48:14] runLogin(): Testing login by ONTAP version from storage controller FAS-A.myserver.com ...
[14:48:14] runLogin(): ONTAP version: NetApp Release 7.3.3: Thu May 13 09:49:11 PDT 2010
[14:48:14] runLogin(): Storage appliance login successful.
[14:48:15] runCollectVolumeNameData(): Looking for snapshot smvi_MyBackup_recent on controller FAS-A.myserver.com ...
[14:48:15] runCollectVolumeNameData(): Snapshot smvi_MyBackup_recent was found in volume vol_Datastore1.
[14:48:16] runCollectSVData(): Running ZAPI snapvault-primary-relationship-status-list-iter-start on storage controller FAS-A.myserver.com ...
[14:48:16] runCollectSVData(): Running ZAPI snapvault-primary-relationship-status-list-iter-next on FAS-A.myserver.com ...
[14:48:17] runCollectSVData(): SnapVault relationship found (primary = FAS-A.myserver.com:/vol/vol_Datastore1/qt_Datastore1, secondary = FAS-C.myserver.com:/vol/vol_Datastore_Archive/Datastore1) ...
[14:48:17] runCollectSVData(): Running ZAPI snapvault-primary-relationship-status-list-iter-end on FAS-A.myserver.com ...
[14:48:18] runLogin(): Initializing connectivity to storage controller ...
[14:48:18] runLogin(): Logging into to storage controller FAS-C.myserver.com ...
[14:48:18] runLogin(): Testing login by ONTAP version from storage controller FAS-C.myserver.com ...
[14:48:18] runLogin(): ONTAP version: NetApp Release 7.3.3: Thu May 13 09:49:11 PDT 2010
[14:48:18] runLogin(): Storage appliance login successful.
[14:48:18] runSVUpdateRelationship(): Running ZAPI snapvault-secondary-initiate-incremental-transfer on storage controller FAS-C.myserver.com, snapshot smvi_MyBackup_recent, secondary path /vol/vol_Datastore_Archive/Datastore1 ...
[14:48:18] runSVUpdate(): SnapVault incremental transfer started successfully.
[14:48:20] runSVCheckRelationships(): Running ZAPI snapvault-secondary-get-relationship-status on storage controller FAS-C.myserver.com, path /vol/vol_Datastore_Archive/Datastore1 ...
[14:48:20] runSVCheckRelationships(): Relationship for path /vol/vol_Datastore_Archive/Datastore1 is still running ...
[14:48:20] runSVCheckRelationships(): More relationships need to be updated, sleeping for 30 seconds ...
[14:48:50] runSVCheckRelationships(): Running ZAPI snapvault-secondary-get-relationship-status on storage controller FAS-C.myserver.com, path /vol/vol_Datastore_Archive/Datastore1 ...
[14:48:51] runSVCheckRelationships(): Relationship for path /vol/vol_Datastore_Archive/Datastore1 is idle, removing from the list to check.
[14:48:51] runSVCheckRelationships(): All relationships updated.
[14:48:51] runSVSnapCreate(): SnapVault secondary snapshot(s) to be taken on FAS-C.myserver.com:/vol/vol_Datastore_Archive/Datastore.
[14:48:51] runSVSnapCreate(): Creating a SnapVault secondary snapshot for volume vol_Datastore_Archive using schedule Daily_Datastore_Snapshots ...
[14:48:52] runSVSnapCreate(): SnapVault secondary snapshot created successfully.
[14:48:52] runSVUpdate(): A total of 1 SnapVault relationship update(s) and 1 SnapVault snapshot creation(s) successful.
[14:48:52] runCmd(): Command completed successfully.
[14:48:52] runPrintReport(): End Time: Wed Nov 10 21:15:52 2010
runExit(): -----------------------------------------------------
runExit(): Exiting with return code: 0

Comments

I'm not entirely familiar with using post-backup scripts in SMVI/VSC.. but I'm assuming this means I now have to create extra SMVI jobs specifically to snapvault when I want them to instead of snapvaulting every smvi snapshot?

Frequent Contributor

Hey, Doug.  When you create an SMVI backup job, you have the ability to add a script to it (you'll see it in the SMVI/VSC interface).  That script would become the sv-smvi.cmd file included in the zip image I distribute.  In the sv-smvi.cmd batch job, you'll need to add in the options you require and a path to where you have stored sv-smvi.exe.  So, if you put sv-smvi.exe in C:\Temp, you would make sure the path to sv-smvi.exe (including quotes if necessary - remember that Windows will require quotes to paths with spaces in them) points to the right location.  The options -svuser and -svpasswd (or -svcryptpasswd) are the only ones needed.  You don't even need to specify -backupname, as that comes when you run it as a post-script backup operation.  The environment variables from SMVI tell SV-SMVI which snapshot, which controller and which volume.  Everything after that is handled by SV-SMVI.

It reduced the code by about 40% in not having to do 'smvi backup list' parsing, which worked in most cases but could break due to I18N issues or other space parsing problems.  I also added in the ability to do updates to multiple qtrees in a single volume on the SV secondary so you can consolidate now, and also specify your own set of SV schedule names if you choose.

So the steps would be:

1) Put sv-smvi.exe into a path where you want to run it from.

2) Modify sv-smvi.cmd so that it calls the new location of sv-smvi.exe (remember to use quotes for paths with spaces!)

3) Place sv-smvi.cmd in your smvi\server\scripts folder.

4) Modify your SMVI backup jobs, and specify sv-smvi.cmd as the script to run -- it will only run on POST_PHASE backup jobs.

5) Run your SMVI backup.

If you want to see exactly what is happening (and I encourage users to try this), save the log output by adding:

-verbose -report -reportdir C:\Temp

to the command line arguments in sv-smvi.cmd.  This will save a log file in C:\Temp which you can read to see what is happening with your backup job.  By default, SMVI doesn't report back the log output from SV-SMVI even if you specify -verbose.

Let me know if this helps.  Thanks,

--Matt

Frequent Contributor

Just a heads-up, I'll be adding a 3.0.1 tomorrow or Friday.  One customer came up with a new test case (many:many relationships) and I wanted to provide that enhancement to SV-SMVI users.  It would only be needed if you use qtrees inside of datastores and want replication of each qtree.  Thanks,

--Matt

Matt,

  The way we use our NetApps currently is that all of our NFS volumes are mirrored offsite (for DR) and then SMVI runs every 4 hours (for backup) and then is SV-SMVI'd offsite once a week (for long term backup/archiving).

  Using the post-backup script, we will have to create a sibling job that only runs weekly for every current SMVI job, is that correct?

Frequent Contributor

Hi, Doug.  That's correct -- you will duplicate the backup job, but only have it run once a week with the script specified.  Be sure to take a look at the log output when you're testing it out.  One quick question -- are your SV-SMVI backups taking longer than 30 minutes at this point?  If so, I'm going to update the documentation on a smvi.override change to allow SMVI to let scripts run longer.  It is separate from SV-SMVI but may help some environments.

Thanks again,

--Matt

Matt,

  One of the jobs takes ~an hour and another one takes ~35 minutes.

-Doug

Frequent Contributor

For jobs that last longer than 30 minutes, you'll need to update your smvi.override file (in smvi\etc) with the following value:

smvi.script.timeout.seconds=XXXX

Where XXXX = 6000 (6000 seconds, for example).  This will let the job continue to run rather than SMVI killing it after 30 minutes.  It's not part of what SV-SMVI has any control over, purely a function of SMVI.

Thanks, Doug!

--Matt

jwhelan31

I notice SV_SMVI User Guide on page 7 states the SnapVault primary source (regardless if VMFS or NFS) is a qtree.  I currently have my datastores setup as volumes per NetApp's recommendation for proper integration with RCU (and something else I can't remember).  Will this still work if I initialize my snapvault relationship as such:  fas_sec> snapvault start -S fas_pri:/vol/nfs_ds1/- fas_sec:/vol/vol_file_sv/nfs_ds1.  I think page 8 answers this question but I just wanted to confirm.

Frequent Contributor

Hi, jwhelan31.  Yes, it should work in the example you give (where you're replicating everything that isn't in a qtree by using the dash option).  The root of a volume is a qtree itself, and the dash option in your relationship line takes care of that.  The only thing to be aware of in that situation is that you are relocating the data to a new path on the secondary and that if you ever put data in a qtree inside that volume that it won't be replicated.  If you're ever unsure, you can use the "-report -reportdir C:\Temp -verbose" options in your sv-smvi.cmd script (make sure C:\Temp exists) and look at the log file to see exactly what/how your qtree data (or non-isolated qtree data in your case) is being transferred.

Thanks,

--Matt

jwhelan31

We would not be putting any other qtrees in that source volume that we are using for the NFS datastore so I think we are good.  Thanks for the speedy reply!!!

jwhelan31

Hi Matt,

I am having an issue getting the script to run in Windows 2008 R2 and wonder if I am hitting an issue with UAC or something quirky like that with R2.  Have you come across anything like this yourself?

2010-11-30 13:25:07,730 PROGRESS - Starting backup request
2010-11-30 13:25:07,792 INFO - Generating backupName for the scheduleJob 6d1b6180-94ac-43d0-b361-d5f2ba4e0d5d is backup_vm_her5_dd_01_sata_01_infra_sv_20101130132507
2010-11-30 13:25:09,276 INFO - A VMware consistency snapshot will be performed on Virtual Machine ch02pd013.
2010-11-30 13:25:10,073 INFO - A VMware consistency snapshot will be performed on Virtual Machine ch02pd003.
2010-11-30 13:25:10,636 INFO - A VMware consistency snapshot will be performed on Virtual Machine vMA.
2010-11-30 13:25:11,277 INFO - A VMware consistency snapshot will be performed on Virtual Machine pp-htc.
2010-11-30 13:25:11,870 INFO - A VMware consistency snapshot will be performed on Virtual Machine ch02pd015.
2010-11-30 13:25:12,448 INFO - A VMware consistency snapshot will be performed on Virtual Machine fqns2.
2010-11-30 13:25:13,027 INFO - A VMware consistency snapshot will be performed on Virtual Machine ch02pd007.
2010-11-30 13:25:13,605 INFO - A VMware consistency snapshot will be performed on Virtual Machine ch02pd006.
2010-11-30 13:25:14,183 INFO - A VMware consistency snapshot will be performed on Virtual Machine ch02pd005.
2010-11-30 13:25:14,745 INFO - A VMware consistency snapshot will be performed on Virtual Machine ch02pd001.
2010-11-30 13:25:15,308 INFO - A VMware consistency snapshot will be performed on Virtual Machine ch02pd002.
2010-11-30 13:25:15,308 PROGRESS - Backing up datastore(s) ([vm_her5_dd_01_sata_01_infra (netfs://10.200.8.135//vol/vm_her5_dd_01_sata_01_infra/)])
2010-11-30 13:25:15,308 PROGRESS - Backing up the following virtual machine(s) ([ch02pd013, ch02pd003, vMA, pp-htc, ch02pd015, fqns2, ch02pd007, ch02pd006, ch02pd005, ch02pd001, ch02pd002])
2010-11-30 13:25:15,308 PROGRESS - The following virtual machines will have a VMware snapshot taken for consistency ([ch02pd013, ch02pd003, vMA, pp-htc, ch02pd015, fqns2, ch02pd007, ch02pd006, ch02pd005, ch02pd001, ch02pd002])
2010-11-30 13:25:15,636 PROGRESS - Script sv-smvi.cmd is starting in phase PRE_BACKUP
2010-11-30 13:25:15,667 INFO - Script sv-smvi.cmd completed with output: C:\Program Files\NetApp\Virtual Storage Console\smvi\server\log\scripts>"C:\Program Files\NetApp\SV-SMVI\sv-smvi.exe" -svuser root -svpasswd password -verbose -report mkdir("C:\Windows\system32\config\systemprofile\AppData\Local\Temp\pdk-SYSTEM\") failed, errno=2 Panic: Can't find temp dir for perlapp
2010-11-30 13:25:15,667 ERROR - Script sv-smvi.cmd failed with exit code 1
2010-11-30 13:25:15,667 ERROR - FLOW-11019: Failure in ScriptExecutionTask: Script sv-smvi.cmd failed with exit code 1
2010-11-30 13:25:15,667 ERROR - FLOW-11010: Operation transitioning to abort due to prior failure.
2010-11-30 13:25:15,714 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine ch02pd001
2010-11-30 13:25:15,714 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine fqns2
2010-11-30 13:25:15,714 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine ch02pd015
2010-11-30 13:25:15,730 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine ch02pd006
2010-11-30 13:25:15,730 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine pp-htc
2010-11-30 13:25:15,730 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine ch02pd005
2010-11-30 13:25:15,745 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine ch02pd013
2010-11-30 13:25:15,745 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine ch02pd007
2010-11-30 13:25:15,745 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine ch02pd003
2010-11-30 13:25:15,761 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine ch02pd002
2010-11-30 13:25:15,761 INFO - Failed to find VMWare snapshot smvi_906afb22-c8a1-466a-ba41-98cf0a606cfd for virtual machine vMA
2010-11-30 13:25:16,011 PROGRESS - Script sv-smvi.cmd is starting in phase FAILED_BACKUP
2010-11-30 13:25:16,027 INFO - Script sv-smvi.cmd completed with output: C:\Program Files\NetApp\Virtual Storage Console\smvi\server\log\scripts>"C:\Program Files\NetApp\SV-SMVI\sv-smvi.exe" -svuser root -svpasswd password -verbose -report mkdir("C:\Windows\system32\config\systemprofile\AppData\Local\Temp\pdk-SYSTEM\") failed, errno=2 Panic: Can't find temp dir for perlapp
2010-11-30 13:25:16,058 PROGRESS - Script sv-smvi.cmd failed with code 1 in phase FAILED_BACKUP
2010-11-30 13:25:16,496 PROGRESS - Sent status notification message to [jwhelan@firstquality.com]
2010-11-30 13:25:16,496 PROGRESS - Storing logs for backup_vm_her5_dd_01_sata_01_infra_sv_20101130132507 in file C:\Program Files\NetApp\Virtual Storage Console\smvi\server\repository\logs\6d1b6180-94ac-43d0-b361-d5f2ba4e0d5d\backup_backup_vm_her5_dd_01_sata_01_infra_sv_20101130132507.xml

Frequent Contributor

Hi, jwhelan31.  Two things:

1) Your backup above wouldn't do anything with sv-smvi.exe primarily because it is in FAILED_BACKUP phase.  sv-smvi.exe will only run in POST_BACKUP phase, which means you had a successful backup.

2) The "Can't find temp dir for perlapp" message generally happens because the user you are executing as has a TEMP or TMP environment variable defined that points to a directory that doesn't exist.  You'll need to ensure the user executing sv-smvi.cmd/sv-smvi.exe has a TEMP/TMP environment variable with a legitimate path defined.  This can be reviewed/set through right-clicking on My Computer->Properties->Advanced->Environment Variables.

Thanks,

--Matt

jwhelan31

Is there a way for me in VSC 2.x to choose to run this script in a POST_BACKUP phase?  From what I can tell, I only have the ability to choose the script but not when it should run.  I will try to figure out the environment variable thing.  Thanks again for your help.

Frequent Contributor

Hey, jwhelan31.  The POST_BACKUP phase is automatic; if your back-up with SMVI is successful, it will automatically return POST_BACKUP.  If it wasn't successful, it will be FAILED_BACKUP.  If you're getting the latter you need to address why the backup isn't completing or why it is failing.  On the environment variable issue, try adding this to the sv-smvi.cmd file, right before the sv-smvi.exe line:

set TEMP=C:\Temp

set TMP=C:\Temp

Make sure C:\Temp exists on your server (and is writable) and give it another go.  Another debug point is that you can add:

set > C:\foo.txt

Right before the sv-smvi.exe entry as well and see what the script things TEMP and/or TMP are set to.

Thanks again,

--Matt

j_samarin

we are testing a similar scenario with VSC 2.0.1 & the newest SV-SMVI 3.0.1. a problem exists where if you run SMVI/VSC and obtain say 10 backups, and then run a SMVI backup with the SV-SMVI script it will only update the latest backup and the previous 10 backups are not transferred over as in previous version.

Frequent Contributor

Hi, j.samarin.  The SV-SMVI is only going to execute against the STORAGE_SNAPSHOTS indicated set in the environment variables passed by SMVI (you can make a .cmd script that just runs "set > foo.txt" and look at the values in the foo.txt file).  If you want every backup to be replicated, you'll put the SV-SMVI job on every backup job.  If you don't see the volumes indicated in the foo.txt, it's likely not specified as part of the backup.  The example that Doug was talking about was specific to only replicating once a week with SV-SMVI, which is why he needed a second job.

If you're still having an issue, please send the SV-SMVI log output (use the -report -reportdir <dir> -verbose options) through a Communities message.  Thanks,

--Matt

j_samarin

and that makes sense, in the past we would archive all the previous retained backups in one shot but appears that option is not possible. one bug we have noticed, is if you create a backup job for a VM through the VSC the SV-SMVI via "host and clusters" or "vms and templates" it throws an error in the windows event log saying "can't find datastore <vm name>", where <vm name> is the virtual machine name. but if you create the backup job via the "datastore" section of vcenter the error does not occur. also the "-nosvsnap" options gives a return code 14.

Frequent Contributor

Hey, j.samarin.  If you can send me the log with the error code 14 with -nosvsnap, that would be excellent.  I need to validate that you have an issue there and what it might be.  I'm not sure why the VM/datastore job isn't working, but that would be part of VSC/SMVI and outside the scope of SV-SMVI (if you have the full SMVI logs for that as well as SV-SMVI I can validate that for you).

Thanks,

--Matt

New Contributor

pls be aware that you need to set the option

httpd.admin.enable on

otherwise, you will get a ZAP::invoke - error...

@Matt: Can you pls add HTTPS support (-poto http/https) to the sv-smvi.pl/sv-smvi.exe script...

marc

j_samarin

@matt

here is the SMVI log for the -nosvsnap issue

2010-12-09 16:02:43,149 PROGRESS - Starting backup request
2010-12-09 16:02:43,259 INFO - Generating backupName for the scheduleJob 977b73a0-f350-40c7-9d7c-b3187bebfe90 is backup_sv-smvi_testbackup_20101209160243
2010-12-09 16:02:46,274 INFO - A VMware consistency snapshot will be performed on Virtual Machine nfs_test1.
2010-12-09 16:02:46,274 PROGRESS - Backing up datastore(s) ([backup_test (4cc892fe-05384e82-ccbd-001e0b21929c)])
2010-12-09 16:02:46,274 PROGRESS - Backing up the following virtual machine(s) ([nfs_test1])
2010-12-09 16:02:46,274 PROGRESS - The following virtual machines will have a VMware snapshot taken for consistency ([nfs_test1])
2010-12-09 16:02:46,446 PROGRESS - Script sv-smvi.cmd is starting in phase PRE_BACKUP
2010-12-09 16:02:47,415 INFO - Script sv-smvi.cmd completed with output: C:\Program Files\NetApp\Virtual Storage Console\smvi\server\log\scripts>"C:\Program Files (x86)\NetApp\SV-SMVI\sv-smvi.exe" -svuser <removed> -svcryptpasswd <removed> -nosvsnap -report -verbose -reportdir C:\Temp ERROR: Command line argument (-nosvsnap) invalid! ERROR: Check command line for spaces in arguments without quotes and missing argument values! ERROR: Run C:\Program Files (x86)\NetApp\SV-SMVI\sv-smvi.exe -help for more information on command-line options.
2010-12-09 16:02:47,415 ERROR - Script sv-smvi.cmd failed with exit code 14
2010-12-09 16:02:47,415 ERROR - FLOW-11019: Failure in ScriptExecutionTask: Script sv-smvi.cmd failed with exit code 14
2010-12-09 16:02:47,431 ERROR - FLOW-11010: Operation transitioning to abort due to prior failure.
2010-12-09 16:02:47,509 INFO - Failed to find VMWare snapshot smvi_8970146d-6a2e-420f-8c17-1a05c94fe233 for virtual machine nfs_test1
2010-12-09 16:02:47,556 PROGRESS - Script sv-smvi.cmd is starting in phase FAILED_BACKUP
2010-12-09 16:02:48,134 INFO - Script sv-smvi.cmd completed with output: C:\Program Files\NetApp\Virtual Storage Console\smvi\server\log\scripts>"C:\Program Files (x86)\NetApp\SV-SMVI\sv-smvi.exe" -svuser <removed> -svcryptpasswd <removed> -nosvsnap -report -verbose -reportdir C:\Temp ERROR: Command line argument (-nosvsnap) invalid! ERROR: Check command line for spaces in arguments without quotes and missing argument values! ERROR: Run C:\Program Files (x86)\NetApp\SV-SMVI\sv-smvi.exe -help for more information on command-line options.
2010-12-09 16:02:48,134 PROGRESS - Script sv-smvi.cmd failed with code 14 in phase FAILED_BACKUP
2010-12-09 16:02:48,181 PROGRESS - Sent status notification message to <removed>
2010-12-09 16:02:48,181 PROGRESS - Storing logs for backup_sv-smvi_testbackup_20101209160243 in file C:\Program Files\NetApp\Virtual Storage Console\smvi\server\repository\logs\977b73a0-f350-40c7-9d7c-b3187bebfe90\backup_backup_sv-smvi_testbackup_20101209160243.xml

running the same backup minus the "-nosvsnap" switch in the script completes sucsefully everytime. also i found that the errors reported in event viewer when running the backup from the "host and clusters" or "vms and templates" is a bug in SMVI not SV-SMVI. also some vsphere "authentication" errors in the event viewer on the vSphere/VSC server are also bugs in SMVI.

@everyone

if anyone else can verify these problems by creating a manual backup of a single VM, then running it via the "NetApp" tab in "Hosts and Clusters" or "VMs and Templates" vs the "NetApp" tab under "Datastores", would be greatly appriciated. netapp seems to be stumped at the moment.

Frequent Contributor

Hey, Marc -- I can write it but note that I'll need someone to test it.  Can you try it out in your environment?

Thanks,

--Matt

Frequent Contributor

Hey, j.samarin.  I verified the issue you're seeing -- I have a test binary for you to try out which should address the issue.  I can spin a 3.0.2 as soon as Marc lets me know how testing goes for HTTPS support.

Let me know, thanks!

--Matt

Hi Matt

I have installed and configured this just as described and everything seems fine, BUT i have confirgued a daily and a weekly snapvault snap schedule on the secondary and these schedules are working fine besides that i also get new snapshots whenever the SMVI backup runs from the primary (whenever the script runs)

Please advice

Kevin

Frequent Contributor

Can you use the -svschednames and specify just the daily or weekly?  You'll need separate SMVI backup jobs with the different command line options (so that you vault at different times) but would that help?

Thanks,

--Matt

P.S.  I'm still working through the HTTPS testing with a customer but I'll spin a 3.0.2 with the -nosvsnap fix and make 3.0.3 for the HTTPS feature.

If anyone is interested, here is a list of role capabilities I used for the svuser:

login-http-admin,api-system-get-version,api-volume-list-info,api-snapshot-list-info,api-snapvault-*

-Dan

Hi j.samarin, rmatt

need a little hint:

Snapshots on secondary are not made. Debug log shows this..

[14:28:24] runSVCheckRelationships(): Relationship for path /vol/SMVI_backup_1/na02_fc_nfs2_q1 is idle, removing from the list to check.
[14:28:24] runSVCheckRelationships(): Running ZAPI snapvault-secondary-get-relationship-status on storage controller mssna03, path /vol/SMVI_backup_1/na02_fc_nfs2_q2 ...
[14:28:24] runSVCheckRelationships(): Relationship for path /vol/SMVI_backup_1/na02_fc_nfs2_q2 is idle, removing from the list to check.
[14:28:24] runSVCheckRelationships(): All relationships updated.
[14:28:24] runSVSnapCreate(): Duplicate SnapVault secondary controller/volume found (mssna03:/vol/SMVI_backup_1), removing duplicate from list.
[14:28:24] runSVSnapCreate(): Duplicate SnapVault secondary controller/volume found (mssna03:/vol/SMVI_backup_1), removing duplicate from list.
[14:28:24] runSVSnapCreate(): SnapVault secondary snapshot(s) to be taken on mssna03:/vol/SMVI_backup_1/SMVI_mssna02_na02_fc_nfs2.
[14:28:25] runSVUpdate(): A total of 3 SnapVault relationship update(s) and 0 SnapVault snapshot creation(s) successful.

Update: found that the Protection Manager didn´t create schedules.

create SMVI_backup_1  0@-@0 preserve=default
Changed this to sth else....now snap are created...

Any suggestions how to configure a DFPM snapvault schedule with this correctly?

--Thorsten

j_samarin

@thorstenschmidt

yes, my suggestions is to not use dpm. i know that sounds pretty pompous, but the cli is actually a better way to configure everything. it's not intuitive in anyway shape or form, netapp really needs to hire some apple engineers to help fix that. but it is pretty simple to configure once you understand how it all works. the scheduling/retention is the key component for archiving as you found. use the "snapvault snap sched" command to set your retention, and you don't even need to worry about the "schedule" part because the sv-smvi is going to manually push the update. for instance, our snapvaults are setup with "254@-" meaning it will keep 254 snaps all the time via the "-" switch. then when using sv-smvi script, the snapvault schedule will simply keep a rolling set of snapshots based on the retention. my suggestion would be to setup a separate test backup that kicks off ever 30 or 60 minutes so you can see a the processes in action. having say 4 smvi backups and maybe keep 6 or 8 snapvault snaps you will see the updating and creation/removal and how they tie together. my apologies if this is basic or command knowledge to you already.

jstegema Former NetApp Employee

Hi there,

Thank you for a great script, and after some trail and error, got it work great in the lab.

I have a customer whose Primary site (NetApp 3040HA SAS) with all Prod, Dev and Test VMs on ESX4.0 (approximately 11.5TB of VM data) is replicated, via SnapMirror to DR site (NetApp 3040HA SATA). The issue with this process is that the VMs at the DR site are not consistent. This replicated data/or at very least Prod and selected Dev should then be backed up to additional disk on the DR site using SnapVault with retention policies. It is also proposed that the DR VM data (only a handful of critical servers) will also be included in the SnapVault process. They have 2 x vCentre servers running in Linked mode and VSC 2.0.1 is installed on the Primary parent vCentre but not configured.

The desired outcome is to have the SnapMirror and SnapVault "run" from the VSC vCentre plugin, in conjunction with SV-SMVI and associated scripts.

In the lab I have successfully managed to process a SMVI backup job with VM consistency, also updating SnapMirror from Prod to DR, and then using the SV-SMVI scripts to update SnapVault secondary using Prod as the source; however I want to use the SnapMirror destination as the Snapvault source - is there an option or syntax that would allow me to do this?

--

Cheers,

Jason

hongwei Former NetApp Employee

Hi guys,

I've tried using the SV-SMVI tool but the job always failed at the script portion...

I've copy the exe file and placed it in the programs\netapp\SV-SMVI directory which i created

I've copy the .cmd file and placed it in the programs\netapp\smvi\server\scripts directory

Configured the snapvault relationship of the datastore LUNs

Configured the cmd file as

"C:\Program Files\NetApp\SV-SMVI\sv-smvi.exe –svuser root -svpasswd <passwd for root>"

but still it outputs error 1 on the script...

Worst when i try to excute the  -verbose -report -reportdir C:\Temp, it doesn't churn out anything...

What could have i missed out or done wrong ?

Regards,

HongWee

Frequent Contributor

Hi, hongwei.  Regarding the lack of output, are you running this as a script job through your SMVI backup, or on the command line?  You want to place the .cmd as a script to run (make sure it is listed under the Scripts tab on the right) and see if it still fails.  If you think there's an error, you can create a 'test.cmd' file and simply put in a 'set > C:\test.log' command in it and run that as a script.  If you don't get output from that, then check to make sure you are running the latest VSC (or SMVI 2.0p1).  If you do with the test.cmd but not with the sv-smvi.cmd (make sure to use -verbose -report -reportdir C:\Temp and make sure C:\Temp exists), let me know what Event Viewer states.

Thanks,

--Matt

I ran into an issue where I had templates in the entity list for SMVI.  When I ran the backup the SV-SMVI script would error out even before PRE_BACKUP phase and no debug logs were generated.  Note to all, don't backup VM templates and use SV-SMVI

-Dan

Frequent Contributor

Hey, Dan.  Thanks for sending this in.  Note that SV-SMVI only works with post-backup (POST_BACKUP environment variable) operations.  Is there a reason why you're trying to do it pre-backup?

Thanks,

--Matt

Pretty interesting. I'm not sure if i need this script.  I've searched  hours for a similar solution. But i would prefer to use Snapvault via the Net App Management Console. So i've tried to SnapVault the Vmware Volumes to the secondary Storage system. It works but on the snapvault destination i only have the volume without the VSC (SMVI) generated snapshots (what you are seeing under .snapshot directory when connecting via NFS). Isnt it possible to snapvault a volume with its snapshots (SMVI generated snapshots in this case)  to a snapvault destination?

Here's what i want to do:

1) Filer1:VSC generates Vmware consistent snapshots

2) Snapvault job (created through NetApp Management Console/DPM) is transfering the Snapshots of this volume (.snapshot dir) to snapvault destination Filer2 where i can configure special retention time

3) SnapMirror syncs the Filer2 volume to Filer3

Step 1) and 3) are no problem. Step 3) gives me a headache since i dont find a solution to configure snapvault the way that it's transfering the Snapshots of the Volumes on Filer1 (the vmware consistent snapshots) and not only the volume with its running VM (and therefore not VM consistent on snapvault destination). Is your script the only solution to my problem or am i missing some major knowledge about snapvault?

Thanks

j_samarin

@begas

dpm blows...the cli and sv-smvi are your best bet. we are doing exactly what you are describing and except for some intermittent hangups deleting / renaming older snapvault snapshots its all working like a champ.

Frequent Contributor

Hi, begasoftfiler.  SV-SMVI 3.0 addresses the issue around (2) and is the main reason for its existence.  There's not a great answer in the product for now which is the reason why I wrote this (and then re-wrote it for VSC and SMVI 2.0).  Hopefully it helps our in your environment.

Thanks,

--Matt

Frequent Contributor

Hi, j.samarin.  Let me know if you get any closer to the intermittent hang-up issues.  You might want to increase your timeout values for SMVI to run the job just in case that's the timeout issue (smvi.script.timeout.seconds can be set to a larger value).

Thanks again,

--Matt

j_samarin

@rmatt

im not sure if the problem was sv-smvi related, but we were getting errors "Snapshot 'snapshot_name' was not deleted as it is currently being used by application(s) 'snapvault'. Please retry after sometime." after deleting the snapshots manually we are no longer getting the errors but will get an occasional "WARN com.netapp.smvi.task.scripting.ScriptExecutionTask - Script sv-smvi.cmd timed out after 30 minutes", plus a couple other menacing timoeout errors.

Hi Matt,

I'll get these errors:

81792188 [backup1 6916b3913e64aca6a61721c619b45ee9] WARN com.netapp.common.flow.Operation - FLOW-11011: Operation aborted

81792008 [backup1 6916b3913e64aca6a61721c619b45ee9] ERROR com.netapp.common.flow.TaskInstanceTemplate - FLOW-11019: Failure in ScriptExecutionTask: Script sv-smvi.cmd failed with exit code 1 java.lang.Exception: Script sv-smvi.cmd failed with exit code 1 at com.netapp.smvi.task.scripting.ScriptExecutionTask.execute(ScriptExecutionTask.java:154) at com.netapp.common.flow.TaskInstanceTemplate.execute(TaskInstanceTemplate.java:336) at com.netapp.common.flow.ForLoopTemplate.execute(ForLoopTemplate.java:141) at com.netapp.common.flow.Operation.executeCurrentStack(Operation.java:146) at com.netapp.common.flow.Operation.execute(Operation.java:61) at com.netapp.common.flow.Threadpool$OperationThread.run(Threadpool.java:257)

81792007 [backup1 6916b3913e64aca6a61721c619b45ee9] ERROR com.netapp.smvi.task.scripting.ScriptExecutionTask - Script sv-smvi.cmd failed with exit code 1

If i try with test.cmd (set > test.log) i'll get the output.

verbose logging en reporting doesn't work on the sv-smvi.cmd

Environment:

Vsphere 4.1

Vcenter 4.1

VSC 2.01p1

LatestSV-SMVI

Running on Windows 2008R2 64bits

Thanks,

JJ

Hi Matt

In version 2.something there was some issues regarding snapvault updates if you had more than 1 entity in you backup (sv-smvi did only see the first entity), do you also have this limitation in 3.x?

We have a customer with a SMVI backup with 2 entities, and when they run a backup, SV-SMVI is not able to find the snapshot name on one of the entities.

/Bo

Hi

I want to use the SV-SMVI executable to initiate a snapvault backup. 

Altough I have created a user with all the api* capabilities an error is generated and no snapvault updates are being run?
==============
[15:33:42] ERROR: ZAPI connection to 134.188.143.11 failed: User smvi-user does not have capability to invoke API system-get-version.
==============
What could be the problem?
 
User/group/roles created:
==================
 
On Source-filer Ontap 7.3.2
Role:
========
Name:    api-access
Info:
Allowed Capabilities: api-*,login-http-admin,cli-ifconfig
Group:
========
Name: api-group
Info:
Rid: 131081
Roles: api-access
User:
========
Name: smvi-user
Info:
Rid: 131082
Groups: api-group
On Destination-filer Ontap7.3.2.
Role:
=========
Name:    api-access
Info:
Allowed Capabilities: api-*,login-http-admin,cli-ifconfig
Group
=========
Name: api-group
Info:
Rid: 131085
Roles: api-access
User:
=========
Name: smvi-user
Info:
Rid: 131086
Groups: api-group

If someone has any idea's would be great.

New Contributor

matt,

I am using VMFS datastores with luns in qtrees, each on its own volume.  I would like to minimize the snapshots on the primary volume as much as possible.

On the secondary, I would like to keep 24 hourlys, 30 dailys and 12 monthlys.

What would be the best way to set this up to minimize the local snapshots, ideally I would like to not keep any snapshots on the primary more than 1 day.

j_samarin

@bever

im not sure what you mean by "primary volume". if you mean the primary filer, then thats just a matter of setting your retention in smvi to "1" if thats all you want to keep. as far as the secondary filer you just need to set your retention policies for the archives you want to keep. personally i find it much more efficient and less time consuming to keep a handful of backups on the primary filer for restoring. the restore process for either a single file or an entire vm is much simpler using the native smvi tools then recovering from an archive. it can be done and that is the point of snapvault, but depending on your rate of change there is very little impact on having multiple smvi backups online. plus it shortens the restore time not having to migrate data from an archive which may be located at a dr site or some other wan connection.

Frequent Contributor

It's definitely an API issue, but I'm not sure why api-* wouldn't pick it up.  You can explicitly add api-system-get-version to see if that helps resolve the role/group issue.  You can also test it by logging in and running 'ontapi execute <system-get-version></system-get-version>' from advanced mode.

Thanks,

--Matt

Frequent Contributor

Hi, bever.  If you really want that kind of relationship, you'd have to have three different backup jobs (so three snapshots on the SV primary):

Job 1: SV primary -> SV secondary using -svschednames sv_hourly

Job 2: SV primary -> SV secondary using -svschednames sv_daily

Job 3: SV primary -> SV secondary using -svschednames sv_monthly

You can take 23 hourly snapshots, and use the 0 hour as your daily snapshot for every day except the 1st of the month, in which case you'd run an sv_monthly.

Would that cover what you're trying to do?  Thanks,

--Matt

justinryan

Hi, sorry if this is obvious, but what is the restore procedure for this setup? I know you can restore at the VM, VMDK, file, etc level from a backup that's in SMVI/VSC, but given that I have a limited retention schedule on the production filer and am pushing backups over to the SnapVault using the SV-SMVI script, how do I get those backups back into SMVI/VSC from the snapvault?

Frequent Contributor

It depends -- if you are vaulting, you can use SnapVault to restore ('snapvault restore') the vaulted contents to the primary, but note that in some cases (VMFS) you would be pulling everything over.  NFS datastores might be best addressed with 'ndmpcopy'.  The solution depends on the kind of datastore, the contents in the vault and whether you can copy directly (or indirectly) and whether you need to do a partial or full 'snapvault restore' operation.  It's not directly integrated so the process will depend on your environment and your own defined process.

justinryan

@rmatt, thanks for the response. Our setup looks like this:

primary, many 2.3T volumes > one qtree > one 2T LUN (FC) > VMFS Datastore > many VMs

secondary, a few 16T volumes > snapvaulted qtrees

Our recovery strategy would mainly be to restore an individual VMDK, ideally to a different datastore as a new read-write VMDK that we keep using, not just a temporarily-mounted backup.  We would also like to be able to restore an entire VM which has several disks on different datastores, which may or may not be on the same snapvault volume.

any ideas? thanks!!

j_samarin

@justinryan

what works for us, is if we need to recover any archived data we simply use the lun clone or flexclone features of the filer. as you may or may not know, this creates a "zero" sized copy of the original lun / volume and only grows if data gets written. from there we simply map the lun to our esx servers. from there we can copy or restore single vms, vmdks, or files similarly to how smvi does it. we were going to write a scripted front end for doing these types of things, but its such a simple process there wasn't really any point. for you scenario since you want to keep the restored data online and sounds like you don't want to replace the original data. i would simply create a new volume / lun with the storage requirements you need as a "landing zone" for the recovery. setup your lun or nfs share as a new datastore in esx and simple follow the previous process as i suggested. if using luns w/ fibre channel you would need to copy the data via the esx service console or vic. if using nfs, you could just use ndmp copy to move the data to the new location you created. once this is done simple destroy to flexclone or lun clone and you're back to your regular production environment with your new datastore that contains your recovered data...if im missing something, or that doesn't quite make sense let me know.

ZAPI error resolved:

The problem was that we changed the passwords. After changing these by root the smvi-user has to logon and change them again (console doesn't ask for this?)

If SV-SMVI commands are run before changing the passwords (altough using the correct passwords) Authentication goes well but no commands can be run.

=> and logs error: ERROR: ZAPI connection to xxx.xxx.xxx.xxx failed: User smvi-user does not have capability to invoke API system-get-version.

Warning!

This NetApp Community is public and open website that is indexed by search engines such as Google. Participation in the NetApp Community is voluntary. All content posted on the NetApp Community is publicly viewable and available. This includes the rich text editor which is not encrypted for https.

In accordance to our Code of Conduct and Community Terms of Use DO NOT post or attach the following:

  • Software files (compressed or uncompressed)
  • Files that require an End User License Agreement (EULA)
  • Confidential information
  • Personal data you do not want publicly available
  • Another’s personally identifiable information
  • Copyrighted materials without the permission of the copyright owner

Files and content that do not abide by the Community Terms of Use or Code of Conduct will be removed. Continued non-compliance may result in NetApp Community account restrictions or termination.