Subscribe

NDMP and lots of small files...

[ Edited ]

Hi,

We currently have a need to backup our SnapVault (OSSV) repositories to tape. Unfortunately the dataset is made up of lots of little files and of course this is bad news for tape drives (especially LTO4 which we are using in this case).

We have the option of direct connection of a tape drive to the filer or using 3-way NDMP. Our backup software is Netbackup v6.0.

Does anyone have any neat ideas on how to backup the data in a 'block level way' or something faster than accessing all the individual files using NDMP or otherwise?

The tape backup is 'belt and braces' so we are happy to have a volume level backup and not be worried about restoring individual files.

Of course the ultimate way will be to snapmirror off site somewhere but that is not currently an option for us.

Any ideas will be gratefully received!

Cheers,

Trevor.

Re: NDMP and lots of small files...

Hi Trevor,

You may check out SnapMiror to tape:

http://now.netapp.com/NOW/knowledge/docs/ontap/rel7251/html/ontap/onlinebk/4mirro35.htm

I don't think NetBackup can utilise this though...

Have a look at this thread:

http://communities.netapp.com/thread/2471?tstart=0

Regards,

Radek

Re: NDMP and lots of small files...

Hi Radek,

Thanks for reminding me of that option. I've used that before to set up snapmirror relationships for remote filers. Unfortunately NetBackup of integration is required for the guys to manage via a 'single pain of glass'.

I'm going to proxy the backup using a RedHat server and backing up the NFS share, at least that way I can take advantage of the filer cache/read ahead mechanisms to improve the stream to tape, not ideal but it is belt and braces as mentioned before.

Many Thanks,

Trevor

Re: NDMP and lots of small files...

The snapmirror to tape is a pig and should only be used for seeding remote snapmirror targets. We have just started using snapvault and a remote server for D2D backup. Works great!

The backup of many small files to tape will always be an issue, as tape likes large files. You could snapvault to tape

http://now.netapp.com/NOW/knowledge/docs/ontap/rel73_vs/html/ontap/onlinebk/protecting/reference/r_oc_prot_sm-snapvault-to-tape-backup-scenario.html#r...

This is fast as it streams the block from the volume to tape and does not have files to work with. Can only restore to a filer however.

Brendon

Re: NDMP and lots of small files...

Brendon,

Isn't the thing you are pitching just SnapMirror to tape?

The very first sentence on the page you are pointing to says: "In this scenario, you use SnapMirror to replicate Snapshot copies from a SnapVault secondary storage system to an attached local tape drive for the purposes of backup"

I mean the underlying mechanism/command is SnapMirror (good for gazillions of files) vs. NDMP (bad for the same task).

Regards,

Radek

Re: NDMP and lots of small files...

That is what I thought when I first read it but when you do it, it locks the volume and stops the snapmirror while it is writing to table. We mirror every 30 minutes and 5 TB to tape takes a couple of days longer.

SnapVault to tape does not cause this issue.

Do not know why, I am just an end user who had his fingers burnt with this.

Bren

Re: NDMP and lots of small files...

Interesting!

I am fairly convinced this is indeed SM2T (SnapMirror to Tape).

The data being chucked to tape has to be configured as a SnapMirror source, and the command it uses is "snapmirror store".

Definitely looks, smells & feels like SM2T to me

Could your use of this have coincided with some ONTAP upgrade that perhaps handles snapshots slightly better ? Not that I can specifically think of any recent versions that might change the snapshot locking behaviour.....

cheers

Mike

Re: NDMP and lots of small files...

Cheers Mike,

Sounds like SM2T is a plan. I've seen this article http://partners.netapp.com/go/techontap/matl/top10_NetBackup.html#5 that mentions that NetBackup can manage the process. Unfortunately we are running NB 6.0 MP4 so we need a magical add on. Do you know what this is? We will upgrade to MP5 or 6.5 but I need to prove the backup and more importantly the restore works before proceeding with any upgrade.

Thanks,

Trevor

Re: NDMP and lots of small files...

Hi Trevor,

Sounds like SM2T is a plan. I've seen this article http://partners.netapp.com/go/techontap/matl/top10_NetBackup.html#5 that mentions that NetBackup can manage the process.

I think (correct me if I am wrong) that they are talking there about replicating/cascading SnapVault target volumes to a tertiary NetApp device - i.e. this is SM to disk, not to tape.

As said before, I don't think NetBackup can do SM2T - have seen a lot of their technical presentations, whitepapers, etc. & never came across this.

Regards,

Radek

Re: NDMP and lots of small files...

This is odd: I cannot reply to the original post, it appears to be marked "answered" and there is no reply option ???

Odd, because the replies are NOT complete, I think !!

For Trevor, actually NBU has supported SM2T since version 4.5....including version 5 and versions 6: SM2T use with NetBackup (4.5 and later) is described within the NBU for NDMP administration guide (as the NBU for NDMP option is required for SM2T use). You should be able to find this guide at support.veritas.com.

Be aware of all the usual SM2T caveats (see the NetApp online backup and recovery guide for your ONTAP, on the NOW site, in the SnapMirror chapter)

Cheers

Mike