When to use SME and SMSQL in Vmware environment

[aleskosek]

Hi!

We have a customer who have a Netapp solution in a VMWare environment. It is smaller installation so they also have Exchange and SQL in VMWare servers. We will implement SMVI for backup but we are wondering when to use SM for Exchange and SM for SQL. We have follwoing concerns:

1. Can SME and SMSQL work with SMVI? Is there a need to use both solutions?

2. What advantages will bring for example SME if we are already using SMVI?

3. Will SME or SMSQL work on the same virtual machine as SMVI or those products do not work together?

4. Does SME and SMSQL work at all with VMDKs?

5. Is there any whitepaper how to use SME or SMSQL in VM environment?

Thanks for all your help, regards, Ales

[bjornkoopmans · Accepted solution]

Hi Ales,

Any backup (snapshot or traditional) needs to contain a complete set of related data to be useful for restore. So to properly back up your database application (Exchange, SQL, Oracle, etc.), you should always use some sort of agent to bring the database in a consistent state. Meaning that any data in the RAM is written to disk, the logfiles are up to date and the database is quiesced.

You use SME or SMSQL to bring your database in that consistent state. SMVI doesn't bring your application in a consistent state, only the OS (if you incorporate VMware snapshots). So SMVI is great for static data, such as application binaries, OS and webservers (without database), but not for databases. On the other hand, SME and SMSQL don't backup the OS and application binaries.

1 + 2) So in short: you'll need both solutions. 🙂

3) They'll work perfectly together, but you might want to consider not scheduling snapshots with both solutions on the exact same time.

4) Nope. Think about it: you use SME/SMSQL to create consistent snapshots of your databases. You can't (storage)snapshot a single vmdk. You'll need RDM's for that.

5) Not sure here, but there really is hardly any difference with a physical environment.

Hope it helps.

Bjorn

[aleskosek]

Hi!

Thanks for good explanation. We will try to install it and we will see.

It looks a bit complicated because there is SMVI and SMSQL. Hope it will not be too toug installation.

There is another question. We will run HA configuration.

Will SMSQL work in HA configuration also? Will secondary server automatically took over the primary configuration?

Lep pozdrav / Best regards, Aleš

[bjornkoopmans]

Not sure what you mean exactly by 'HA configuration'. Are you talking about your SQL servers, your filers, your disks or perhaps you're talking about SnapMirror or such?

Bjorn

[aleskosek]

Hi!

With HA I mean VMWare HA configuration. Since this is not typical cluster I am not sure how it works with SMSQL.

Thanks.

[radek_kubka]

VMware HA operates on a different layer & both guest OS and application (SQL) are totally unaware of it - so there are no issues with running SMSQL in this environment.

Having said that, in theory you can backup SQL with SMVI only via VSS integration, but this arguably has some drawbacks (more details e.g. here: http://communities.netapp.com/message/8132#8132)

Regards,
Radek

[joostvandrenth]

SMVI leverages vmware snapshots, and they can 'activate' VSS inside the vm guest. I have read (trying to find out where it was) that vss in this way actually leverages the vss writers - including the application writers present. So while it is true SMVI does not call VSS, the VMware triggering of a snapshot does do so. In that way SMVI gives us a consistent snapshot.

I find this point interesting but am not sure I am correct.

[amritad]

Hi,

NetApp offers two solutions today for addressing application-consistent data protection in a VMware environment:

·         SMVI, working through the VMware guest VSS stack, provides application-consistent backup and recovery for applications that have VSS writers and store their data on virtual disks (VMDKs). Recovery, in this scenario, is at the full VM level only.

·         SnapDrive® and application-specific SnapManager products such as SnapManager for Exchange (SME) and SnapManager for SQL (SMSQL) running in the guest OS, provide application-consistent backup and fine-grained recovery for applications whose data is stored using Microsoft iSCSI Software Initiator LUNs or RDMs.

The critical difference in the level of protection provided by each of these solutions is in the granularity of recovery. To understand the significance of this, you need to understand the concept of roll-forward recovery. Roll-forward recovery replays information stored in transaction log files to return a database to the state it was in at an exact point in time. In order to perform a roll-forward recovery, archival logging must be enabled, a full backup image of the database must be available, and there must be access to all logged files created since the last successful backup.

Since the only recovery mode available today for applications backed up using SMVI (using the VMware built-in VSS support) is recovery to the point of the last backup, customers must use the relevant SnapManager application if more fine-grained, roll-forward recovery is required.

Today, both solutions can be used together—SMVI to back up/recover the system data and a SnapDrive and application SnapManager combination to back up/recover the mission-critical application data—to get the desired level of data protection required.

Regards

Amrita

[bjornkoopmans]

Hi Amrita,

Thanks for your clarification. However, I'm definitely no DBA, but something tells me MS SQL or Oracle won't like it being restored when the backup was made using just VSS. I'd think an Oracle would have to be in a back-up state before a proper and consistent back-up can be made. Likewise, I imagine MS SQL would be even more difficult about it.

Does SMVI, when configured to make VMware snapshots, also snapshots the memory of the vm? If not, the SMVI-snapshot s not guaranteed to be application-consistent.

By the way, I never worked with SMVI 2.0, but previous releases had great difficulty created VMware snapshots, even resulting (in our case at least) in frozen vm's untill the process timed-out!

[radek_kubka]

something tells me MS SQL or Oracle won't like it being restored when the backup was made using just VSS.

I am not 100% sure about Oracle on Windows, but MS SQL will be fine - VSS process puts in fact SQL in a hot-backup mode. As already mentioned (here, or in a similar thread), VSS-based backup won't do log truncating, so you really should either use simple recovery mode (whenever the database performs a checkpoint, space in the log is reclaimed), or manually truncate the log.

Regards,
Radek

[amritad]

Hi

The SMVI BPG has a section on data consistency maybe that will help.

http://www.netapp.com/us/library/technical-reports/tr-3737.html

We haven't heard about frozen snapshots in 2.0.The miscsllaneous section talks about the best practices for VMware snapshots.

REgards

Amrita

[mearsrich]

Hi Amrita,

We use SMVI 1.2R1 and occasionally get stuck Snapshots. I would like to upgrade to SMVI 2.0 as it seems to be better in this respect, but as we use NFS I understand that this would actually mean we would be unable to restore without a FlexClone license. Is this ture? What would you advise we do?

Thanks,

Rich.

[amritad]

Hi Rich,

Flexclone is required in 2.0 for:

1)single File restore (i.e. the granularity of restoring just a single guest file)

2)NFS datastore mounts (flexclone is optional for VMFS mounts as SMVI uses LUNClone if Flexclone is not available in this case)

3)out-of place NFS VMDK restores (when you're restoring an NFS VMDK to an alternate datastore)

It is not required for other operations.

Regards

Amrita

[mearsrich]

Hi,

Thank you for that. Just to confirm, if I upgrade to SMVI 2.0 I will still be able to restore a single VM (vmdk and/or vmx) to the original datastore only. That is core functionality so I would expect this to be the case.

I see that to restore to a different datastore requires FlexClone, how about restore to the same datastore but a different path. I assume I wouldn't need a license for that as there would be no additional mounting or anything involved.

My final question Does the SMVI 2.0 GUI now have the compression option? I use snapmirror.conf at the moment to snapmirror after the SMVI 1.2R1 backups. To carry on using compression must I keep doing this and are there any downsides to not using SMVI to start the SnapMirror ???

Thanks again,

Rich.

[amritad]

Hi Rich

if I upgrade to SMVI 2.0 I will still be able to restore a single VM (vmdk and/or vmx) to the original datastore only. That is core functionality so I would expect this to be the case

Yes that is correct

I see that to restore to a different datastore requires FlexClone, how about restore to the same datastore but a different path. I assume I wouldn't need a license for that as there would be no additional mounting or anything involved.

USing the GUI you can only restore to the same datastore there is no option to specify a different path.

SMVI 2.0 GUI now have the compression option?

The GUI doesn't have the compression option. If you set the policy to update snapmirror using SMVI the policy takes care of replication automatically for each backup. Apart from that I am not sure if there are any downsides.

Regards

Amrita

[gorazdbabic]

Since it`s already been debated about SMSQL on this site, can someone please clarify (I am an SQL newbie) how to configure db migration in the 4th step of a configuration wizard.

First of all: should all the system databases be migrated to lun? As I read tempdb is recreated everytime the server starts... So, should I move just master, model and msdb (besides the production databases) to the new lun?

And another thing: should I move the above mentioned system db all to the same lun (db+logs) or should db be on one lun and the logs on another? For production db it is clear they should be separated. Master db is required to be on one lun (wizard suggests that).

regards to all,

gorazd

[bjornkoopmans]

Hi Gorazd,

I think this is a little off topic, but to answer your questions:

How you distribute your databases and logs onto LUNs is really up to you, as long as they are ALL on LUNs. Ofcourse it all depends on business requirements, licenses, level of complexity, etc. There is great and elleborate documentation about this subject on NOW, along with a lot of (sometimes conflicting) Best Practices. I don't have the numbers at hand, but I suggest you try the search module on NOW. These documents should give you a really good idea on how to set things up properly.

At the very minimum, you should have:

- LUN 1 for all the systemdb's and system transactionlogs

- LUN 2 for all your user db's and transaction logs

- LUN 3 for SnapInfo data.

Tempdb is recreated at each reboot, that's true, but if you don't change the location of the tempdb, it will be created on a local drive every time.

Regards, Bjorn

[gorazdbabic]

Bjorn, thank you for the answer. I read the documents you mentioned and as you said they add to confusion ..

OK, I agree there should be 3 luns. But let me be mores specific: lets`s say I move move model on 2 luns (db and logs separately); msdb and tempdb the same story; when I try to move master the wizard window says: "you cannot migrate all the files to the specified lun, as it`s already shared by multiple databases.... you must use both of the shared luns" - which implies I should move db and logs separately, I presume. But when I do that another window says: "select all of the database files in the master database for migration to a lun", which is no go.

And even worse when I select the default Northwind, when the wizards issues a warning:"Do not migrate to a lun which contains system database..."

Regarding to that I should have separate pair of luns for system and separate pair of luns for production databases + snapinfo lun; which makes 5 luns. I haven`t found the answer to that anywhere and I`ve been searching for 3 days already. Should I just transfer the production db an leave the system db alone - as I saw on one demo video?

thank you all; Gorazd

[bjornkoopmans]

Hi,

First of all: dumb the Northwind database ASAP! 🙂 It's a demo database that serves no purpose other then that and forms a huge security issue to your SQL server(s).

having said that, as Northwind is a demo database, it's by definition a user database and cannot be placed on the same LUN as system dabatases. That's what the wizard was telling you.

Not sure about the other messages you get, but I do know SMSQL definitely doesn't need your db-files and translog-files to be on separate LUNs. Are you sure you selected all the files, associated with master? Don't forget about full text indexes and such.