Bunch of newbie questions

[uptimenow]

Hi, I played around with SnapCreator 3.5 this afternoon. I'm pleasantly surprised by the functionality it offers, and I will definitely invest more time in it.

However, I've got a couple of beginner questions I'd like to ask:

- I am in the CET timezone. In the "Schedules" and "Job Monitor" section of the GUI, under "Management", I see the times indicated as GMT with a 2-hour difference. However, the logfiles have timestampts in CET, as do the messages inside them. Can SnapCreator be configured to display all time information as CET (in my case) ?

- The first thing I tried to configure was a profile and a schedule for a Windows server with a couple of SnapDrive LUNs. Those LUNs needed basic consistent snashotting. I was able to do this by running sdcli.exe the NTAP_SNAPSHOT_CREATE_CMD1 setting, as explained in the docs. Now, I have a bunch of standard application servers, each of them with a number of LUNs. After I got my first configuration up and running via the GUI, I created a new configuration for another server manually by copy-pasting the first server's configuration. I really like the fact that everying is saved in a simple text file and that the GUI can reload this at any time! (I am also under the impression that the agent rereads its configfile after it has been changed without having to restart the service - cool!) Ideally, I would like to be able to copy-paste configs for new servers and limit the effort to just changing the SC_AGENT setting in the config file. However, for new servers I also had to change the following settings:

VOLUMES

SNAPVAULT_VOLUMES

SNAPMIRROR_VOLUMES

NTAP_SNAPSHOT_CREATE_CMD1

How do you go about creating quick configuration for different servers based on the same "template server". Ideally I want to configure SC to "snapshot all the LUNs and mountpoints via sdcli" and then trigger SnapMirror/SnapVault regardless of what volumes the LUNs are on. What happens when you create an additional LUN and drive letter for a server  ? Do you need to add the new volume to the configfile, and change the sdcli.exe command to include the new driveletter ?

In the GUI, one can list the volumes for a certain config under "Data"/"Volumes", but this is basically read-only information, no way to add or remove volumes to a configuration from there.

(on an related note, although this is not SC-specific, how can you use sdcli.exe and tell it to snapshot all the NetApp LUNs without providing an explicit list via the "-D" option - that way one could just use the same NTAP_SNAPSHOT_CREATE_CMD1 for all configurations ?)

Maybe I need to read up on the discovery feature of SC. I enabled snapdrive discovery in the configuration, but I am not sure what it does exactly ...

- The Installation and Administration Guide mentions that SC can be used with SMSQL 6.0. Is this a typo ? The highest version of SMSQL I can find on the NOW site is 5.1.

- When configuring a backup for SMSQL or SME, do you have to enable the "consitency group" feature, or will SMSQL/SME take care of that when it is called (I would assume the setting doesn't matter in SC and SMSQL/SME will handle the snapshotting anyway) ?

- At the moment I am getting this error when running a backup for an SMSQL server - I think it is SMSQL 5.0. Am I getting this error because SMSQL 5.0 is not supported with SC ?

[Wed Apr  4 17:09:27 2012] [vmwk30047:9090(3.5.0.1)] ERROR: [sms-00005] Command [%SystemRoot%\Sysnative\WindowsPowerShell\v1.0\powershell.exe -psconsolefile C:\Program Files\NetApp\SnapManager for SQLServer\smsqlShell.psc1 -command "new-backup -svr vmwk30047 -RetainBackups 24 -lb -bksif -RetainSnapofSnapInfo 24 -trlog  -mgmt standard"] failed with return code 256 and message

[Wed Apr  4 17:09:27 2012] [vmwk30047:9090(3.5.0.1)] ERROR: [sms-00002] Quiescing databases failed

[Wed Apr  4 17:09:23 2012] [vmwk30047:9090(3.5.0.1)] ERROR: [scf-00053] Application quiesce for plugin smsql failed with exit code 256, Exiting!

- I am still a little confused about the difference between a schedule and a policy. Right now, I have one backup profile per server I need to backup (this corresponds to a directory under the configs directory). Then, for a particular server, I created multiple configuration, eg. "servername-hourly", "servername-daily", etc. As an example of some differences between the configurations, a "-daily" configuration will use SnapMirror to replicate volumes, whereas an "-hourly" configuration will not update SnapMirrors. In the configuration, there are four policies to choose from: hourly, daily, weekly and monthly.

Then, in the schedules, I created a schedule per configuration. What is the point of selecting a pilcy there ? The dropdown also has those four policies. And then one can select a frequency again, where you will set the actual frequency for the snapshots. I'm not entirely sure how one is supposed to configure things ...

Let me just reiterate that this product looks very nice and powerful, great work so far guys ! I think this product is very powerful. For beginners, I think it could use a couple of step-by-step guides on how to create simple configurations like a SnapDrive backup of a server, an example of SMSQL integration, how discovery works, etc. Perhaps a couple of YouTube movies would be really cool !

Best regards,

Filip Sneppe

[spinks]

Filip,

I'll answer what questions I can.

I'll start at the bottom and work my way back up as the bottom questions are the ones I can answer.

We do have some introductory videos here on the communities site.  Check here:  https://communities.netapp.com/community/products_and_solutions/databases_and_enterprise_apps/snapcreator/content?filterID=content~objecttype~objectty...

You can also check out TR:4009: NetApp Snap Creator Framework with IBM Lotus Domino Plug-In Deployment and Configuration Guide - http://media.netapp.com/documents/tr-4009.pdf

Even though this TR is focused on the Domino plug-in, most of the information is applicable to Snap Creator as a whole.

In regards to the schedule vs policy - not all customers use the Snap Creator GUI (even though most do).  Snap Creator expects certain parameters to be passed to it including profile (folder) name, action (snapshot, restore, clone, etc), and policy (daily, weekly, etc).  Policy does have 4 predefined settings as you mentioned, but you can name a policy whatever you want as long as you define it.  This policy setting is part of the Snap Creator binary.

The scheduler is part of the Snap Creator GUI, but is not part of the Snap Creator binary - it is a separate piece.  When Snap Creator is executed by the scheduler the same required parameters as mentioned earlier are passed to Snap Creator - this includes the profile, the action, and even the policy.  Even though it seems redundant it is necessary.

The policy tells Snap Creator which policy to use, but the frequency is how often you want the schedule to run.

The cron option allows for some serious flexibility.  Documentation on how to use cron can be found here:  http://www.quartz-scheduler.org/documentation/quartz-1.x/tutorials/crontrigger

I'll have to pass on the SMSQL questions - I don't have enough experience with those plug-ins to give a definitive answer.  If you haven't already done so check the debug logs for detailed information - this might help clarify what is happening.

For the configs, you might want to check out the global config option.  This will allow you to provide the information and credentials for the storage system - config files will gather this information from the global config.

You will still need to define which volumes are needed for each config file.

Typically this is used when server A uses volume 1-3 while server B users volume 4&5.  You wouldn't want to take snapshots on all 5 volumes when you go to backup server A.

If there is a different use case that you have in mind please clarify it and we will do what we can to help.

I think in this version of the GUI the time zone is fixed to GMT - I'm not sure if customizing the time zone is currently on the features list.  I'm sure someone else will comment and clarify.

Hopefully this is helpful - please let us know if you have additional questions.

Thanks,

John

[ktenzer]

Hi Filip,

Here are some more answers

How do you go about creating quick configuration for different servers based on the same "template server". Ideally I want to configure SC to "snapshot all the LUNs and mountpoints via sdcli" and then trigger SnapMirror/SnapVault regardless of what volumes the LUNs are on. What happens when you create an additional LUN and drive letter for a server  ? Do you need to add the new volume to the configfile, and change the sdcli.exe command to include the new driveletter ?

This is done by downloading config. If you rt click config in GUI there is download option. You can download it once, then change it locally and then rt click on profile. There is upload option. This is almost as easy as a copy/paste feature. We have been debating the copy/paste but the download/upload is almost the same thing and being able to download config is required for many reasons so we felt we met both needs. Let us know thoughts on this?

- At the moment I am getting this error when running a backup for an SMSQL server - I think it is SMSQL 5.0. Am I getting this error because SMSQL 5.0 is not supported with SC ?

[Wed Apr  4 17:09:27 2012] [vmwk30047:9090(3.5.0.1)] ERROR: [sms-00005] Command [%SystemRoot%\Sysnative\WindowsPowerShell\v1.0\powershell.exe -psconsolefile C:\Program Files\NetApp\SnapManager for SQLServer\smsqlShell.psc1 -command "new-backup -svr vmwk30047 -RetainBackups 24 -lb -bksif -RetainSnapofSnapInfo 24 -trlog  -mgmt standard"] failed with return code 256 and message

[Wed Apr  4 17:09:27 2012] [vmwk30047:9090(3.5.0.1)] ERROR: [sms-00002] Quiescing databases failed

[Wed Apr  4 17:09:23 2012] [vmwk30047:9090(3.5.0.1)] ERROR: [scf-00053] Application quiesce for plugin smsql failed with exit code 256, Exiting!

The 256 error means command not found, so either PATH or command. Please play with the SMSQL_32bit option if it is set to Y then change to N or visa versa. This changes the powershell command path used, seems powerhshell isnt found under: %SystemRoot%\Sysnative\WindowsPowerShell\v1.0\powershell.exe

- I am in the CET timezone. In the "Schedules" and "Job Monitor" section of the GUI, under "Management", I see the times indicated as GMT with a 2-hour difference. However, the logfiles have timestampts in CET, as do the messages inside them. Can SnapCreator be configured to display all time information as CET (in my case) ?

This we are working on but it is a bit complex since the scheduler has different requirements than everything else in SC. We use GMT there so we are 100% sure of timing and we dont have bugs. We used to use localtime but there were various bugs so we switched to GMT for scheduler. Again everything else is localtime just scheduler is GMT.

Regards,

Keith

[uptimenow]

Thanks for the helpful answers!

[uptimenow]

Hi Keith,

Thanks for the quick and helpful reply !

This is done by downloading config. If you rt click config in GUI there is download option. You can download it once, then change it locally and then rt click on profile. There is upload option. This is almost as easy as a copy/paste feature. We have been debating the copy/paste but the download/upload is almost the same thing and being able to download config is required for many reasons so we felt we met both needs. Let us know thoughts on this?

My question is not so much about how easy it is to create a config, but more about being able to create a config that is (1) as generic as possible so that it can be applied to multiple servers without any adjustments, and (2) that doesn't require an administrator to change the SC config when an aspect of the server configuration changes. (eg. A LUN residing on a new volume is added to a server and needs to be backed up.) Is there any way to create such configurations, now or in the future ? I thought that SNAPDRIVE_DISCOVERY would have something to do with this, but I'm a little unclear as to how this setting can be used...

I must say, I am trying out SnapCreator at a customer where the current backup & replication solution involves a lot of custom scripting (MySQL, SMSQL, SME, SMO, SnapDrive Snapshots, DB2, SnapVault & SnapMirror replication). We are trying out SnapCreator in the hope of reducing/eliminating some or most of that custom scripting. We did not want to switch to Protection Manager because it wouldn't allow the flexibility we sometimes require. So we're planning on using SC as a "standalone' solution. So far, it does look like a very flexible piece of software. Still, looking at this from the perspective as someone who has worked with PM before, it appears to me that SC does not know a concept similar to a Dataset in PM, and sort of treats every "application server" as a separate backup unit. Is that assumption correct ?

Anyway, I still have a lot of reading and experimenting to do.

The 256 error means command not found, so either PATH or command. Please play with the SMSQL_32bit option if it is set to Y then change to N or visa versa. This changes the powershell command path used, seems powerhshell isnt found under: %SystemRoot%\Sysnative\WindowsPowerShell\v1.0\powershell.exe

Yes, it was indeed a problem with the path to the powershell command, it's working correctly now, thanks.

This we are working on but it is a bit complex since the scheduler has different requirements than everything else in SC. We use GMT there so we are 100% sure of timing and we dont have bugs. We used to use localtime but there were various bugs so we switched to GMT for scheduler. Again everything else is localtime just scheduler is GMT.

OK, I can live with this, and it's good to know that this may change in the future.

Best regards,

Filip Sneppe

[ktenzer · Accepted solution]

Hi Filip,

More answers for you

My question is not so much about how easy it is to create a config, but more about being able to create a config that is (1) as generic as possible so that it can be applied to multiple servers without any adjustments, and (2) that doesn't require an administrator to change the SC config when an aspect of the server configuration changes. (eg. A LUN residing on a new volume is added to a server and needs to be backed up.) Is there any way to create such configurations, now or in the future ?

What you are referring to here is auto-discovery. The SC-Framework has two methods for plug-ins to perform discovery 1) auto-discovery where plugin determines all the proper settings ie: VOLUMES, etc and updates the SC config file automatically 2) Volume validation where plugin discovery volumes and sends them back to scServer. If the list of volumes differs than what is saved in configuration file you get an error. Most application plugins: oracle, mysql, db2, maxdb, etc use volume validation. Volume validation is only supported for NFS today, we are looking to expand it to LUNs in future but this requires SnapDrive which adds lots of complexity. The only plug-ins that support auto-discovery are the virtualization plugins. Again SC is a framework so plug-ins have many options and the nice thing is customers can create their own plugins to do exactly what they want to do. Most of our plugins are created in perl but you can use even scripting languages like powershell.

To enable volume validate, set VOLUME_VALIDATION=DATA. SC again if it is NFS will discover volumes where data files are located for application and send them back to scServer for validation.

Again anything is possible but out-of-the-box different plugins support different things since all applications are NOT created equally.

Let me know if this answers your question

Keith

[uptimenow]

Hi Keith,

Thanks for the quick reply. I understand from the FAQ at:

https://communities.netapp.com/docs/DOC-4572/version/12

that I can mail dl-snapcreator@netapp.com to get access to

the source code. Eventually, I'd like to have a look at the

MySQL (& possibly VMware) plugin(s) that are builtin, I guess.

I am wondering if there is a possibility for a non-NetApp enmployee

to subscribe to a mailing list to get some more info on SC development.

I'd like to lurk on it for a while to get a feel for what's going on.

Also, I see that this forum contains the occasional upload of some

plugins for specific applications. Is there some sort of central repository

with easy access to all the plugins that have been written so far ?

I'd like to look at some of that code if possible.

(More specifically, the MySQL snapshot solution that is in place at

the customer I talked about is a scripted solution where every database

has a MySQL replica, and the replica is stopped and then a snapshot

operation takes place to get a consistent DB snapshot. I'm interested

in how the SC built-in MySQL plugin works)

Thanks in advance,

Filip

[ktenzer]

Hi Filip,

Snap Creator has a community development model. This means source code, bug tracking, wikis, are open to any NetApp customers. You need to signup for this community it involves.

1. goto www.github.com and creating an account

2. goto www.snapcreator.com and signup with your github user

After completing this you will receive an email and access to the Snap Creator Developer Community. You also get access to the mailing list.

If you cannot access www.snapcreator.com it means your NOW account does not have correct permission. There is a guide in this community under documents to troubleshoot SSO issues.

Regards,

Keith