NDMP and lots of small files...

tyeoman_swinton · ‎2008-12-10

Hi,

We currently have a need to backup our SnapVault (OSSV) repositories to tape. Unfortunately the dataset is made up of lots of little files and of course this is bad news for tape drives (especially LTO4 which we are using in this case).

We have the option of direct connection of a tape drive to the filer or using 3-way NDMP. Our backup software is Netbackup v6.0.

Does anyone have any neat ideas on how to backup the data in a 'block level way' or something faster than accessing all the individual files using NDMP or otherwise?

The tape backup is 'belt and braces' so we are happy to have a volume level backup and not be worried about restoring individual files.

Of course the ultimate way will be to snapmirror off site somewhere but that is not currently an option for us.

Any ideas will be gratefully received!

Cheers,

Trevor.

radek_kubka · ‎2008-12-10

Hi Trevor,

You may check out SnapMiror to tape:

http://now.netapp.com/NOW/knowledge/docs/ontap/rel7251/html/ontap/onlinebk/4mirro35.htm

I don't think NetBackup can utilise this though...

Have a look at this thread:

http://communities.netapp.com/thread/2471?tstart=0

Regards,

Radek

tyeoman_swinton · ‎2008-12-10

Hi Radek,

Thanks for reminding me of that option. I've used that before to set up snapmirror relationships for remote filers. Unfortunately NetBackup of integration is required for the guys to manage via a 'single pain of glass'.

I'm going to proxy the backup using a RedHat server and backing up the NFS share, at least that way I can take advantage of the filer cache/read ahead mechanisms to improve the stream to tape, not ideal but it is belt and braces as mentioned before.

Many Thanks,

Trevor

BrendonHiggins · ‎2008-12-10

The snapmirror to tape is a pig and should only be used for seeding remote snapmirror targets. We have just started using snapvault and a remote server for D2D backup. Works great!

The backup of many small files to tape will always be an issue, as tape likes large files. You could snapvault to tape

http://now.netapp.com/NOW/knowledge/docs/ontap/rel73_vs/html/ontap/onlinebk/protecting/reference/r_oc_prot_sm-snapvault-to-tape-backup-scenario.html#r...

This is fast as it streams the block from the volume to tape and does not have files to work with. Can only restore to a filer however.

Brendon

radek_kubka · ‎2008-12-10

Brendon,

Isn't the thing you are pitching just SnapMirror to tape?

The very first sentence on the page you are pointing to says: "In this scenario, you use SnapMirror to replicate Snapshot copies from a SnapVault secondary storage system to an attached local tape drive for the purposes of backup"

I mean the underlying mechanism/command is SnapMirror (good for gazillions of files) vs. NDMP (bad for the same task).

Regards,

Radek

BrendonHiggins · ‎2008-12-10

That is what I thought when I first read it but when you do it, it locks the volume and stops the snapmirror while it is writing to table. We mirror every 30 minutes and 5 TB to tape takes a couple of days longer.

SnapVault to tape does not cause this issue.

Do not know why, I am just an end user who had his fingers burnt with this.

Bren

mwalters · ‎2008-12-11

Interesting!

I am fairly convinced this is indeed SM2T (SnapMirror to Tape).

The data being chucked to tape has to be configured as a SnapMirror source, and the command it uses is "snapmirror store".

Definitely looks, smells & feels like SM2T to me

Could your use of this have coincided with some ONTAP upgrade that perhaps handles snapshots slightly better ? Not that I can specifically think of any recent versions that might change the snapshot locking behaviour.....

cheers

Mike

tyeoman_swinton · ‎2008-12-12

Cheers Mike,

Sounds like SM2T is a plan. I've seen this article http://partners.netapp.com/go/techontap/matl/top10_NetBackup.html#5 that mentions that NetBackup can manage the process. Unfortunately we are running NB 6.0 MP4 so we need a magical add on. Do you know what this is? We will upgrade to MP5 or 6.5 but I need to prove the backup and more importantly the restore works before proceeding with any upgrade.

Thanks,

Trevor

radek_kubka · ‎2008-12-12

Hi Trevor,

Sounds like SM2T is a plan. I've seen this article http://partners.netapp.com/go/techontap/matl/top10_NetBackup.html#5 that mentions that NetBackup can manage the process.

I think (correct me if I am wrong) that they are talking there about replicating/cascading SnapVault target volumes to a tertiary NetApp device - i.e. this is SM to disk, not to tape.

As said before, I don't think NetBackup can do SM2T - have seen a lot of their technical presentations, whitepapers, etc. & never came across this.

Regards,

Radek

mwalters · ‎2008-12-13

This is odd: I cannot reply to the original post, it appears to be marked "answered" and there is no reply option ???

Odd, because the replies are NOT complete, I think !!

For Trevor, actually NBU has supported SM2T since version 4.5....including version 5 and versions 6: SM2T use with NetBackup (4.5 and later) is described within the NBU for NDMP administration guide (as the NBU for NDMP option is required for SM2T use). You should be able to find this guide at support.veritas.com.

Be aware of all the usual SM2T caveats (see the NetApp online backup and recovery guide for your ONTAP, on the NOW site, in the SnapMirror chapter)

Cheers

Mike

tyeoman_swinton · ‎2008-12-15

Mike,

Thanks loads for the further information! I'll start reading the docs. On the 'answered' thread thang, I've been a bit excited on clicking on things again, sorry!

I'll let you know how we get on.

Cheers,

Trevor

tyeoman_swinton · ‎2008-12-15

Hi Mike,

I've had a look around at the suggested docs but to be brutally honest I'm struggling to find anything on SM2T and how I can get NetBackup (6.0) to control this. I've had a look at the NBU and NDMP Symantec doc but it doesn't mention SM2T at all.

Sorry to be a pain but is there anything more specific that you can post to point me in the right direction?

Many Thanks,

Trevor

tyeoman_swinton · ‎2008-12-16

Hi Mike,

Is the NearStore storage unit the key to this in NetBackup?

SM2T from the filer directly to tape is easy but it would be great to manage this from a NetBackup policy, I guess I need the 'Open Storage License' to use SM2T under NetBackup control?

Thanks,

Trevor

tyeoman_swinton · ‎2008-12-16

This looks like it:

Solution ID: kb22417

Last updated: 3 MAR 2008

Using Veritas NetBackup to Perform a SnapMirror to Tape

Symptoms

Can I use NetBackup to control a SnapMirror to Tape?

Solution

SnapMirror to Tapecan be used as a method to initialize volume SnapMirror relationships across slow WAN links. The SnapMirror to Tape can be performed from the NetApp filer's command line or be scheduled using Veritas NetBackup.

NOTE: SnapMirror to Tape is meant to be used as an initialization process for volume SnapMirror relationships, not as a backup method. Be sure to review the limitations with SnapMirror to Tape in the Data Protection Online Backup and Recovery Guide for your version of Data ONTAP.

SnapMirror to Tape has two steps: snapmirror store and snapmirror retrieve. The snapmirror store is run on the source filer to snapmirror the source volume to tape. The snapmirror retrieve command is run on the destination filer to snapmirror the volume from tape to the destination volume. Once the snapmirror retrieve completes, a snapmirror updatecan be run on the destination appliance to start the volume snapmirror relationship between the source and destination. Regular updates can then be scheduled using the snapmirror.conf file on the destination.

If the SnapMirror to Tape is performed using the filer's command line (CLI), the administrator must manually insert new tapes as necessary during the snapmirror store on the source filer. For the snapmirror retrieve, the administrator must insert the tapes in the correct order. More details on performing the SnapMirror to Tape via the filer's CLI can be found in the Data ONTAP Data Protection Online Backup and Recovery Guide: Chapter 4.

Veritas NetBackup 4.5 and later can also be used to control the SnapMirror to Tape operations. Using NDMP, the NetBackup master server can instruct the filer to perform the snapmirror store and snapmirror retrieve. The benefit of using NetBackup to control the SnapMirror to Tape is its ability to control the tape robot. NetBackup can automatically add new tapes as needed for the snapmirror store on the source and ensure the proper tapes are loaded in order for the snapmirror retrieve on the destination. This article will describe the steps to use NetBackup to perform the SnapMirror to Tape.

To perform the snapmirror store on the NetApp source filer:

Enable NDMP authentication between the NetBackup server and the NetApp filer.

For more information on configuring NDMP security features on the NetApp filer, refer to the Data ONTAP Tape Backup and Recovery Guide: Chapter 5.

A backup policy must be created in NetBackup. When creating this policy, the following directive must be set:
set type = smtape
/vol/source_volname_to_snapmirror
Start the backup.

Once the snapmirror store completes, the tapes can be shipped to the destination and the data can be loaded on the destination NetApp filer.

To perform the snapmirror retrieve on the NetApp destination filer:

Configure NDMP authentication between the NetBackup master server and the destination filer.
In NetBackup, load the tapes.
On the destination filer, create a volume for the restore.

The destination volume must be the same type of volume as the source volume. If the source was a FlexVol, the destination must also be a FlexVol.
Refer to the Data ONTAP Data Protection Online Backup and Recovery Guide: Chapter 4 for details on the volume type restrictions for Volume SnapMirror.

Restrict the destination volume.

Command: vol restrict <volname>

Perform a normal NDMP restore from NetBackup.
On the destination filer, perform a snapmirror update using the following command:
snapmirror update -S source:source_vol destination:dest_vol
If regular snapmirror updates are desired, configure the update schedule in the destination filer's snapmirror.conf file.

Note: It might be necessary to add the relationship to the snapmirror.conf file prior to starting the restore.

Related links:
Data ONTAP Online Backup and Recovery Guide

Environment

All Filer models

Data ONTAP

SnapMirror

Tape

Veritas NetBackup 6.0

mwalters · ‎2008-12-16

Hi there,

Sorry, I thought there was some formal docs but it looks like I was wrong.

Yes, you have found what I suspect may be the only documentation for this feature. I believe you should speak with your friendly local SE to get approval: we generally do not recommend SM2T as a "backup mechanism" due to the caveats I pointed you to earlier: if you are comfortable with those still meeting the needs for your customer, then I would ask you to contact your NetApp backup specialist SE/CSE to "sanity" check this with Prod Mgmt and engineering.

Thanks

Mike

tyeoman_swinton · ‎2008-12-16

Hi Mike,

No probs. The SM2T process is SO MUCH faster, so for us at the moment it will be our only option. I've passed this over to our friendly neighbourhood SE for his thoughts as you recommended.

Many Thanks,

Trevor

rmharwood · ‎2009-06-11

Hello guys,

I hate to have to drag an old(ish) thread back into play, but I haven't been able to locate anything else.

We're backing up a volume with around 12 million files using SM2T and it works fine - what would have taken 4 days is now taking 5 hours. We can live without being able to do granular restores, since this is for an offsite DR copy only.

Has anyone tried to a restore from a SM2T image? The documentation just says "perform a normal NDMP restore from NetBackup". Well, I tried selecting /vol/volname to restore and NetBackup says "NDMP restore failed from path /vol/volname". I've tried specifying an alternate restore path but every combination I've tried gives the same result. I am trying to restore to a filer different from the original.

I will go ahead and open a case with NetApp if nobody has any ideas...

Cheers,

Richard

radek_kubka · ‎2009-06-11

Scary...

As wise man said backups are useless without restores.

It may be actually an issue with NetBackup, so worth to open the case with them, too.

That bit looks suspicious to me though: "perform a normal NDMP restore" when the image was created using SM2T

rmharwood · ‎2009-06-17

For anyone remotely interested or still following this thread, I have managed to get a SM2T restore working. It appeared as though I didn't make the (restricted) restore volume big enough. The volume originally had ASIS switched on and I had made the target volume big enough to hold the contents but it seems that it needs to be at least as big as the size of the volume, not its contents. Can't be sure, but I made sure to make the target volume at least as large as the original and the restore works!

Cheers,

Richard

radek_kubka · ‎2009-06-17

Good news I think

Re NDMP - I remember when using ndmpcopy that the process is very picky about the size of a target volume: everything is checked upfront & if criteria isn't met the process immediately fails.

Regards,
Radek