What is Snap Creator?

by quiroga Former NetApp Employee on ‎2009-11-11 09:03 AM

Summary


Tired of hard-to-maintain scripts or multiple tools to manage multiple backup applications? NetApp Snap Creator Framework is OS-independent software that integrates with NetApp data protection technologies to provide storage management for your customized applications.

Snap Creator Framework leverages proven NetApp technologies to support multiple applications using a single interface. 

  • Data protection: The NetApp Snap Creator Framework addresses the needs and challenges of administrators and developers by providing a centralized, consistent solution for backing up critical information: one that integrates with existing application architectures to reduce costs and enable a higher ROI.
  • OS independent: Our design approach delivers functionality that is OS independent, to help alleviate the need to manage interoperability issues.
  • Extensible: Snap Creator offers fast integration and greater storage abstraction to leverage our policy-based automation features and better integrate with your current and future application needs.

Snap Creator Framework provides plug-ins to integrate NetApp features with popular third-party applications such as Oracle®, MySQL, and DB2. And if you have special applications that are not part of our supported offering, it's no problem. Snap Creator Framework allows the development of customer plug-ins to provide backup procedures for applications not covered by existing supported applications.

Let our Professional Services experts help you. We have real-world expertise using Snap Creator Framework that we can apply to your key business requirements and that can help minimize operational impact on your organization.

To learn about NetApp Snap Creator Framework, go to NetApp Snap Creator Framework datasheet (PDF) or contact your local NetApp account team today.

History

Snap Creator has it's roots in the field. It was developed starting in Oct 2007 from NetApp employees in Professional Services and Rapid Response Engineering (RRE). It was originally developed to reduce and elliminate scripting for Applications where NetApp did not have SnapManager coverage. Snap Creator filled a ton of gaps in the early days. Professional Services was easily and quickly able to integrate application with NetApp's leading storage technologies such as snapshot, cloning, snapmirror, and snapvault.

Over the years Snap Creator grew from a Professional Services solution, to a PVR solution, and finally the fully supported software distribution available on the NOW site.

Snap Creator is a critical piece of OnCommand Software

Features

Snap Creator has the following features:

  • Supported plug-ins for Oracle, DB2, MySQL,Domino, Sybase ASE, VMware, SME, and SMSQL
  • Supports Windows (32/64-bit), RedHat(32/64-bit), SuSE (32/64-bit), HP-UX (PA-RISC/Itanium), AIX, Solaris(Sparc/x86)
  • Compatible with Data ONTAP 8 (7 mode)
  • Brower-based GUI on NetApp Web Framework (NWF)
  • GUI Backup Scheduler
  • Integration with SnapDrive
  • Integration with storage controller SnapVaultscheduler
  • Integration with the following NetApp technologiesthrough ZAPI: Snapshot, SnapVault, SnapMirror, LUN cloning, volume cloning, andIGROUP mapping
  • Integration with any application or databasethat runs in an open systems environment (you can write the application backupscript or plug-in module if one does not exist)
  • Integration with NetBackup™, CommVault, or anybackup software with CLI commands
  • Ability to configure multiple Snapshot orSnapVault policies with different retentions
  • Both Snapshot and SnapVault policies are managedfrom Snap Creator
  • Ability to create and manage Snapshot copies andSnapVault retentions across multiple volumes and storage controllers
  • Support for volume and qtree SnapMirror
  • Support for FAS and vFiler® units
  • Support for consistency groups (consistentSnapshot copies across multiple volumes or appliances)
  • Support for 1–to-1, fan-in, and fan-outSnapMirror or SnapVault relationships
  • Support for cascading SnapMirrorSnapshot->SnapVault->SnapMirror
  • A dynamic configuration file in which users canpass their own user-defined values
  • Granular error logging and the ability to senderror messages via e-mail or other tools
  • Integration with Operations Manager (the abilityto create events in Operations Manager)
  • Integration with Protection Manager (SnapCreator backups can be registered in Protection Manager)
  • Integration with Open Systems SnapVault (OSSV)
  • Support for password encryption so clear-textpasswords are not saved in the configuration file
  • Support for both volume, file, and SnapVaultrestore but without direct application integration
  • Agent for central backup management
  • Global configuration files
  • API supporting plug-ins development for variousscripting languages such as Perl, Phython, Powershell .

What is Snap Creator used for?

Snap Creator is a backup framework for integrating virtually any application with NetApp snapshot technology. Applications are integrated through plug-ins. Snap Creator should be used to provide a customer with backup, restore, and cloning solutions when it is not possible to use SnapManager products.

What Operating systems are supported?

Snap Creator is written in perl and java so it is platform independent, any open systems operating system is supported. For official support details please refer to the IMT: http://now.netapp.com/matrix

What about application and database Support?

NetApp Snap Creator Framework supports VMware, Oracle, MySQL, Sybase, DB2 and Lotus Notes.  If you are interested in support for another application or database, please contact Rob O'Brien for more information on how to proceed.

There is also a complimentary NetApp Professional Service offering.  The Snap Creator Implementation service can be purchased by our customers if they do not have the expertise required to deploy Snap Creator.

How does Snap Creator communicate with a NetApp Controller/Filer?

Snap Creator uses ZAPI so communications occur through http or https, there is no support for rsh or ssh

Is there a binary of Snap Creator available?

Yes, currently the Snap Creator distribution comes with Linux (Red Hat and SUSE), Windows, AIX, HP-UX, and Solaris (Sparc) 32-bit or 64-bit binarys.

What are the requirements for Snap Creator?

  • Data ONTAP 7.3.x
  • Data ONTAP 8.0.x
  • FlexClone license required for Volume Cloning
  • Snap Restore license required for restore

What is a Snap Creator Plug-in?

A Snap Creator plug-in can be written in virtually any language: java, perl, shell script, powershell, c / c++, python. A plug-in instructs Snap Creator how to handle a specific application. When Snap Creator needs to do something with an application it sends the request to the plug-in.

Is there a GUI?

Yes the Snap Creator GUI is based on the NetApp Web Framework. The GUI is not required but rather provided as a tool to help create and manage snapcreator configuration files.

What about Restores?

Restores are supported.  They are interactive and you are able to do a volume restore or a file restore. Most applications will not be automatically handled but you can automate the application piece with custom scripts or commands vie PRE and POST Restore commands. Currently the only plug-ins which handle application recovery are VMware and Lotus Domino.

What about VFilers?

Full VFiler support is available

Does Snap Creator support RBAC?

Yes, that is one of the strengths of using ZAPI. It is strongly recommended to create a special Snap Creator user with only the minimum priviledges. Please refer to Snap Creator Administration and Installation guide.

Does Snap Creator store passwords in clear text?

By default yes, but you can also encrypt the passwords so they are not stored in clear text. Please refer to Snap Creator Administration and Installation guide.

How do I create a configuration file for Snap Creator?

The best way is by using the GUI.

The default configuration file should be used as a template and can be found under /snapcreator/configs/default/default.conf

If you dont want to use the GUI, you can simple copy the default template to your desired configuration /snapcreator/configs/mydir/myconf.conf

The you can call Snap Creator and pass it --profile myconf to tell Snap Creator to use your configuration file (myconf.conf)

Are multiple configuration files per Snap Creator config supported?

Yes, you can create more than one configuration under a configuration directory but snapcreator can only still use one configuration at a time. To do this use the --profile <Profile> and --config <Config> options together.

What are the required parameters in the Snap Creator configuration file?

At a minimum to just perform snapshots on a volume or volumes you need to configure the following:

CONFIG_TYPE=STANDARD

SNAME=oracledb01

SNAP_TIMESTAMP_ONLY=Y

VOLUMES=filer1:vol1;filer2:vol2

NTAP_USERS=file1:user1/password;filer2:user2/password

LOG_NUM=10

NTAP_SNAPSHOT_RETENTIONS=daily:7,weekly:4,monthly:1

NTAP_PWD_PROTECTION=N

TRANSPORT=HTTP

PORT=80

LOG_NUM=30

LOG_TRACE_ENABLE=Y

SC_TMP_DIR=

For more information about the configuration parameters see the Please refer to Snap Creator Administration and Installation guide.

Do I enter the filer short hostname or FQDN?

Always use the filer short hostname (what appears on the command prompt of your filer) not the FQDN. Also please make sure you can ping the short hostname, you made need to add the short hostname to your hosts file. If you can't resolve the short hostname and run Snap Creator you will get a "in Zapi::invoke, cannot connect to socket" error.

The reason is, protection manager, snapvault and snapmirror will throw error if hostname does not match short hostname

Data Protection

What gets backed up by Snap Creator

Snap Creator creates and manages snapshots on one or more volumes, so anything in your volume gets backed up.

Can I backup more than one Volume?

Yes, Snap Creator can take and manage snapshots on multiple volumes across multiple filers.

Are Consistency Groups Supported?

Yes, Consistency Groups (CGs) are supported for multiple volumes on a single filer and multiple volumes across multiple filers. Snap Creator will do a cg-start on all filers and once the cg-start's are done it will do cg-comitts on all filers.

Can I use SnapDrive instead of ZAPI to take the snapshot?

Yes, Snap Creator integrates with SnapDrive so you can use SnapDrive to create the snapshots and let Snap Creator do the rest. The advantage of using SnapDrive with Snap Creator is that the file system will be consistent.

What about snapshot names?

There are two snapshot naming conventions, _recent or timestamp. Snapshot names will be called either %SNAME_%SNAP_TYPE_recent (recent snapshot) or %SNAME_%SNAP_TYPE-timestamp (timestamp snapshot) where timestamp YYYYMMDDHHMMSS.

If you specify SNAME to be "oracledb01" and define a snapshot policy "daily" then the snapshot will be called either oracledb01-daily_recent or oracledb01-daily_20090130173000.

If you are using the _recent naming convention, recent snapshots are always renamed using the timestamp (human readable) of when the snapshot was taken ie: oracledb01-daily_20100425144055, the rename process happens before a new snapshot is taken

Note: Only snapshots that match %SNAME-%SNAP_TYPE can be managed by snapcreator

What about snapshot policies?

You can create one or more snapshot policy per configuration. Each snapshot policy must specify how many snapshot snapshots (primary)and optionally snapvault snapshots (secondary) are allowed. Additionally you can tell Snap Creator to only delete a snapshot when it is older than X days. So you can configure how many snapshots to keep and do a second check based on how old they are. To call snapcreator and specify a policy use the --policy <Policy> option.

How is SnapVault integrated?

Snap Creator has complete snapvault integration, no schedules are used on the filer. You can specify a snapvault retention in Snap Creator and snapcreator will take care of your snapvault update, create snapvault snapshot, and manage snapvault snapshots. All you need to do is create the snapvault relationship, Snap Creator does not create or manage snapvault relationships. The snapvault update is also done after we return the Application into normal mode so minimal App impact occurs. Snap Creator will wait a defined amount of time for snapvault to finish, it checks every minute if the snapvault transfer is finished.

Are there restrictions on SnapVault?

No none that we know but feel free to try and find some

Is volume to qtree SnapVault supported?

Yes you can snapvault a volume (source) to a qtree (destination) with Snap Creator

How is SnapMirror integrated?

Snap Creator can perform a snapmirror update after a snapshot has occurred. You need to make sure the snapmirror relationship exists since Snap Creator will not manage snapmirror relationships.

Are there restrictions on SnapMirror?

Qtree snapmirror will work but we do not manage snapshots so it is not nrecommended

Can you mix snapvault and snapmirror?

Yes, you can mix snapmirror and snapvault within a source volume or have them separated. Snap Creator supports doing both snapmirror and snapvault within the same backup.

Is both QSM and VSM supported?

Yes

What about Cascading SnapMirror?

Cascading SnapMirror is fully supported as of SnapCreator v3.0. Cascading snapmirror is the ability to snapmirror a snapvault destination (snapshot->snapvault->snapmirror), in this situation SnapCreator performs a snapshot, does a snapvault transfer of that snapshot, takes a snapvault snapshot, and then performs a snapmirror update from the snapvault destination.

How is Lun and Volume Cloning Integrated?

Snap Creator can create an optional volume or lun clone from all luns in a volume. It will then also map all those luns to an igroup you define in the configuration file. LUN clones will be named <NAME>_CLONE; where <NAME> is what the lun is called.

Volume clones will be named cl_<NAME>_<TIMESTAMP>;where <NAME> is what the volume is called and <TIMESTAMP> is YYYYMMDDhhmmss

In Snap Creator you can have as many volume clones as you want but snapcreator is limited to only one lun clone. We recommend using volume clones, lun clones is really only a viable solution in the context of backing a lun up to tape.

Note: Volume Clones require a FlexClone license

Does Snap Creator Integrate with Protection Manager?

Yes in the same way our SnapManager products do. Snap Creator will take a snapshot and then register the contents of that snapshot with Protection Manager which can than do snapmirror/snapvault.

This is very powerful because it allows us to integrate any application with PM and we really need to start using PM everywhere!

How do I schedule Snap Creator?

Snap Creator has a built in scheduler in GUI or you can use cron (UNIX) or Windows Task Manager

How do I run Snap Creator?

This depends on what you want to do but assuming the configuration I want to use is called "myconf.conf" located under /path/to/snapcreator/configs/myconf/myconf.conf, I want to use the "daily" snapshot policy defined in myconf.conf, I want to only create a snapshot (no cloning), and I want to see output displayed to screen the command would look like this:

./snapcreator --profile myconf --action snap --policy daily --verbose

NOTE: The --verbose is optional and only useful for testing or troubleshooting, everything that gets saved to backup report under /snapcreator/logs/logname.out is printed to screen

Integrating Snap Creator

What are PRE/POST Commands?

In Snap Creator at every phase of the backup process you can run PRE or POST commands. They can be scripts or commands. You can have between 1 and 9 of every type of PRE and POST command (with exception to PRE_EXIT_CMD) . Below is an explanation of the different types:

PRE COMMANDS:

PRE_APP_QUIESCE_CMD01 - Can have between 01 and 99, these commands/scripts are run before the Application is placed into backup mode

PRE_NTAP_CMD01 - Can have between 01 and 99, these commands/scripts are run before any NetApp magic happens

PRE_NTAP_CLONE_DELETE_CMD01 - Can have between 01 and 99, these commands/scripts are run before the clones are deleted

PRE_APP_UNQUIESCE_CMD01 - Can have between 01 and 99, these commands/scripts are run before the Application is taken out of backup mode

PRE_RESTORE_CMD01 - Can have between 01 and 99, these commands/scripts are run before your restore occurs

PRE_EXIT_CMD01 - Can only have one command, this command/script is run when an error is encountered

POST COMMANDS:

POST_APP_QUIESCE_CMD01 - Can have between 01 and 99, these commands/scripts are run after the Application is placed into backup mode

POST_NTAP_CMD01 - Can have between 01 and 99, these commands/scripts are run after all the NetApp magic happens

POST_NTAP_DATA_TRANSFER_CMD01 - Can have between 01 and 99, these commands/scripts run after snapvault or snapmirror Update

POST_APP_UNQUIESCE_CMD01 - Can have between 01 and 99, these commands/scripts are run after Application is taken out of backup mode

POST_RESTORE_CMD01 - Can have between 01 and 99, these commands/scripts are run after your restore occurs


A good example is you want to mount a clone Snap Creator made from a snapshot, you can call a mount script or snapdrive for example to mount your clone after the CLONE is made via the PRE command, PRE_NTAP_CLONE_DELETE_CMD01. Then you could make a PRE_NTAP_CLONE_DELETE_CMD02 which calls netbackup or whatever to backup your mountpoint.

I heard the Snap Creator Configuration file is Dynamic, what does that mean?

There are variables you can define in the configuration file which are not static, they are set when the script runs dynamically. All dynamic values start with "%" for example %MSG. You can also pass in one user defined value which gives you even more flexibility. Below are the available dynamic variables:


%SNAP_TYPE - The Snapshot Schedule: daily, monthly, etc

%SNAP_TIME - An epoch timestamp associated with the snapshot

%ACTION - Could be -snap,-clone_vol, -clone_lun, or -arch depending on wich Action you used

%MSG - Used to send the error message to other monitoring tools or email. Can only be used with SENDTRAP function.

%USER_DEFINED - Pass a usre defined argument to the script

Can I have Snap Creator call Symantec NetBackup or EMC Legato NetWorker after the snapshot and/or clone is done?

Absolutely, if you are using clones mount the clone via PRE_NTAP_CLONE_DELETE_CMD01 and call your backup software via PRE_NTAP_CLONE_DELETE_CMD02. If you are not using clones then call your backup software via POST_NTAP_CMD01

Where do I call my Snap Creator plug-in for putting my Application into backup mode?

To put an application into backup mode using commands or scripts, use APP_QUIESCE_CMD and to take the backup out of backup mode use APP_UNQUIESCE_CMD


To put an application into backup mode using built-in Snap Creator modules, use APP_NAME=<plug-in>

I want to use snapdrive to create snapshots, where do I put my snapdrive command?

You can have one or more snapdrive commands, use the following config setting: NTAP_SNAPSHOT_CREATE_CMD01, NTAP_SNAPSHOT_CREATE_CMD02, etc

NOTE: for the snapshot name please use %SNAME-%SNAP_TYPE_recent

%SNAME is a name you give the snapshot in the config file, can be whatever

%SNAP_TYPE is the snapshot policy name something like daily, weekly, monthly, or yearly. This is a value you pass into snapcreator when running it and also a value you configure under NTAP_SNAPSHOT_RETENTIONS and NTAP_SNAPVAULT_RETENTIONS in the configuration file.

Error Handling / Reporting

What type of logs or reports are available?

There are three types of logs: debug log, error log, and the out report.

The stderr log captures STDOUT and STDERR from any command or script snapcreator calls as well as ZAPI return messages (includes all PRE/POST Commands). Debug logs end with .debug. Every backup has a debug log and a backup out report.

The error log is a history of every time Snap Creator encountered an error, so every time an error occurs it is written and saved here. Error logs end with .errror.

The out Report is a record of verbose output that Snap Creator did during a particular backup. OUT Reports end with .out.

The debug log contains verbose + debug + trace (if enabled). The debug reports end with .debug

The debug log, stderr log, and out report are named with the configuration name and timestamp so you can easily identify them ie: myconf.debug.<timestamp> and myconf.out.<timestamp>

How long are logs and reports kept?

The error log is NEVER overwritten or deleted. The debug log, stderr log, and out logs exist for every backup and it is user defined with how many you would like to keep, default is 10.

Does Snap Creator Integrate with Operations Manager?

Yes, Snap Creator can create Operations Manager events for both failure and success

If Snap Creator has an error how do I know?

Many options here. You can monitor the error log yourself, you can have Snap Creator send an email, you can have Snap Creator send a message to tivoli or anything that excepts cli input, and you can have Snap Creator create an Operations Manager event.

Can I send errors to email for notification?

Yes, you can use mailx or any email client that accepts cli input

Can I send a message when SnapCreator completes successfully?

Yes, upon a successful backup you can send a message which can go to email, Operations Manager, or anything that accepts cli input

Warning!

This NetApp Community is public and open website that is indexed by search engines such as Google. Participation in the NetApp Community is voluntary. All content posted on the NetApp Community is publicly viewable and available. This includes the rich text editor which is not encrypted for https.

In accordance to our Code of Conduct and Community Terms of Use DO NOT post or attach the following:

  • Software files (compressed or uncompressed)
  • Files that require an End User License Agreement (EULA)
  • Confidential information
  • Personal data you do not want publicly available
  • Another’s personally identifiable information
  • Copyrighted materials without the permission of the copyright owner

Files and content that do not abide by the Community Terms of Use or Code of Conduct will be removed. Continued non-compliance may result in NetApp Community account restrictions or termination.