Backup with NDMP to tape

[brendanheading]

Hi guys,

I've been busy talking to my reseller about acquiring a FAS2020A (to replace our existing file servers, local storage and iSCSI software terminators) following my plea for help here last week. It's going pretty well. The area I'm having difficulty tying down is backup, and it will be difficult for me to get approval to move to the FAS without being able to show how backup will work. The reseller has done their best to deal with this question, but it appears that the world is moving away from what we're trying to do, which is back up the filer directly to tape.

Our options are :

- continue doing what we're doing right now, which is to run a backup client on each server, with the data stored on the filer instead of on the local disk. This could be done but I'd much rather back up the filer.

- purchase a second identical filer for use as a backup destination. That would be my first choice but I doubt I'll have the budget.

- stick with what we have and try to get NDMP working.

We currently have an ADIC i500 tape library (with parallel SCSI connection only) and EMC Networker. I'm not above moving to Backup Exec or another backup tool if this will work better than Networker. The tape library doesn't have a fibre channel connection and I'd prefer to avoid adding one, as they're very expensive. I've also got a bunch of Apple XServe RAID arrays (don't laugh) which will become redundant if this solution is in place. I can easily put those behind a software iSCSI terminator and make them available as network block devices.

I understand that the way this is done is to use NDMP which allows the filer to dump volumes directly to tape under the direction of Networker. My questions are :

- does the filer have to be connected directly to the tape drive or can it stream data through the server ?

- what plugins do I need to add for Networker to do this ? My reseller indicates I'll need Networker's NDMP and SnapImage plugins. What role do both of these take ?

- any comments on the suitability of Backup Exec ?

- is there any way I can backup both to an attached local disk (ie the XServes) as well as to tape ?

- does NDMP on the Filer allow me to do incremental backups or do I have to dump the whole volume each time ?

I am aware that I will lose deduplication. One way out of that problem is to go to Backup Exec which incorporates deduplication support.

Thanks in advance for any pointers!

Brendan

[rmharwood]

Hi Brendan,

I make use of NDMP for backup a lot. I don't think I can answer all of your questions but hopefully I can give some insights.

We use Symantec NetBackup (6.5) for our tape operations. We have used EMC Networker in the past but not with NDMP backups.

With NetBackup you can stream NDMP through the media servers to tape, rather than directly. I don't see why this would be different with Networker but you should check on this.

I'm not too familiar with the technical differences between NetBackup and BackupExec. I'm quite sure that all of these backup programs will backup to disk rather than to tape - with the correct set of licensed features of course.

You can do incremental NDMP dumps but we rely upon on-disk snapshots instead. Most of our stuff gets to tape only once a week and that's a full backup each time. With NetBackup you cannot simultaneously stream an NDMP backup to more than one destination ("in-line copy"). We make use of NetBackup's duplication features to make second copies, however, and I believe you can use this method to make a tape copy from a disk copy.

I think you'll find that none of these solutions will give you any kind of deduplication benefit on tape. Because tape is linear rather than random access there is no equivalent to block level duplication. I believe there was one backup vendor who was advertising some kind of tape level deduplication mechanism but I can't remember who that was or how it was achieved.

Hope this helps some.

Cheers,

Richard

[brendanheading]

Richard, thanks, that is interesting. It is useful to know that there are people out there doing this, and that it works.

I've been doing some more googling on this subject (the usefulness of the results you get is quite sensitive to the search terms you use). Some answers are :

- the NDMP protocol supports the idea of incremental backups, using backup levels. The Filer creates a snapshot automatically when asked to do an NDMP backup, and normally you'd call this level 0. When you request a level 1 backup, it gives you the deltas that have been added to the disk since level 0. A level 2 backup is the deltas since level 1 .. and so on. You can keep requesting a level 1 backup if you want to do a cumulative incremental backup.

- the concept of "NDMP remote" is used where the NDMP backup is passed to the backup server, rather than directly to the tape drive. The term "three way NDMP backup" appears to basically refer to the same thing.

Netbackup looks interesting (and it looks like they are in the process of consolidating this with the Backup Exec product), however the fact that they don't quote a price suggests that I'm likely to get gouged Now that I understand a bit about what goes on in NDMP backups, and that it should work well, I think I'll talk a bit more to EMC about whether or not Networker will work for us as required.

Brendan

[aborzenkov]

- does the filer have to be connected directly to the tape drive or can it stream data through the server 

You can stream data to another server

- what plugins do I need to add for Networker to do this ? My reseller indicates I'll need Networker's NDMP and SnapImage plugins. What role do both of these take ?

Recent enough Networker includes all needed functionality. You will need NDMP license (it is per head and tiered; I'd expect FAS2020 to be Tier 1). You may need DSA license as well for EMC NetWorker, I do not remember exact license structure; Fujitsu NetWorker does not require it. NDMP license enables NetWorker to talk to filer and initiate backups; DSA functionality accepts data stream from filer and saves it to NetWorker device. IIRC NetWorker supports DSA since 7.2 or 7.3.

Correction: NetWorker feature is called DSA (Data Server Agent) and seems to be included in base funcionality; there is separate license for NDMP Tape Server indeed but as far as I can tell it is not required in this case.

SnapImage has absolutely nothing to do with your case.

- is there any way I can backup both to an attached local disk (ie the XServes) as well as to tape ?

Not sure I understand what you mean. Using NetWorker you sure can do it as long as you have needed licenses (e.g. backup to disk)

- does NDMP on the Filer allow me to do incremental backups or do I have to dump the whole volume each time ?

Yes, you can do incremental backups.

Keep in mind that NDMP is (near to) useless for LUNs (you mentioned iSCSI ...). It is only really useful for file system data, i.e. volumes exported via NFS/CIFS.

Message was edited by: aborzenkov

[brendanheading]

Thank you for that.

By "backup to attached local disk" I mean backup to a pool of cheap disk storage attached to the same server as my tape drive; this will be my tape server. It sounds like it should work fine. I expect we have those licenses already as we already backup to tape and are already looking at backup to disk. We'll just need the NDMP license, and make sure we are properly set up (tier wise) to deal with the FAS2020.

What are the limitations of doing the backup with respect to iSCSI ? That comes as news to me.

[aborzenkov]

You can only save LUN as a whole using NDMP, meaning - no incremental

backups, no per-file restore using NetWorker.

[brendanheading]

I see. That's useful to know. I'm not expecting to have massive numbers of iSCSI LUNs, so perhaps full backups will be acceptable here.

I'm curious to know why this limitation is present. If the filer is creating a snapshot of the volume under the hood in order to do NDMP, I'd not expect that it would make any difference what filesystem was actually on the volume's blocks.

[radek_kubka]

If the filer is creating a snapshot of the volume under the hood in order to do NDMP, I'd not expect that it would make any difference what filesystem was actually on the volume's blocks.

The thing is that from an external point of view (i.e. host side) any snapshot is a fully blown, point-in-time copy of an entire volume. So NDMP-capable software sees that snapshot as a volume & copies it to tape.

Regards,

Radek