ONTAP Discussions

NDMP Backup 2MB/min

oweinmann
20,572 Views

Hi,

is there a limitation (file count) for NDMP? We see a very very poor Performance with NDMP Backups of a 2040c with BackupExec 2010r3. We have not evaluated a different backup software yet so we are not sure if this is the bottle neck.

The volume we are trying to backup consists of several qtree's and has a total around 16 Mio Files.

Regards,

Oliver

1 ACCEPTED SOLUTION

oweinmann
14,202 Views

Hi,

we finally have backup jobs running with backup exec at a very good average speed of 3.8GB/min. So we need 13 hours for around 3.1TB. We have two tape drives so we could even split the jobs.

The key was to restructure the folder structure. Before we had just one large volume Projects with up to 14 million files. I have now started to break it down in ProjectsAC to ProjectsXZ and used snapmirror to move the qtrees. This had a signifficant impact on the backup speed.

Problem solved!

View solution in original post

25 REPLIES 25

gary_thomas
5,930 Views

We had a similar issue and now use smtape.

We get about 280-350mb/s on volumes that have a millions of small files, restore is significantly faster.

We are doing it via ethernet and not fc, so its probibly even faster yet again.

For us we just keep enough disks spare to do a out of store replace and just q a job overnight to restore if required.

smtape has the advantage of having all the snapshots in the backup and we keep 6 months of snapvaults so while you can't do a single file restore without restoring the volume, once you have done the restore its very easy to work with the client to get the exact file/s they require as 9 times out of 10 they can't get a accurate date or filename for stuff thats older then a couple of weeks.

Oh i forgot, usually takes a couple of minutes for smtape to start, but once it starts it flys.

oweinmann
5,930 Views

Hi,

sm2tape is not an option for us. I will now try to create smaller volumes like VolAC, VolDF and so on. Hopefully this makes backup faster. We have two tape drives so we can run the jobs parallel.

oweinmann
14,203 Views

Hi,

we finally have backup jobs running with backup exec at a very good average speed of 3.8GB/min. So we need 13 hours for around 3.1TB. We have two tape drives so we could even split the jobs.

The key was to restructure the folder structure. Before we had just one large volume Projects with up to 14 million files. I have now started to break it down in ProjectsAC to ProjectsXZ and used snapmirror to move the qtrees. This had a signifficant impact on the backup speed.

Problem solved!

milohuang
5,929 Views

I use direct connect 1xLTO5 to filer, we got close to theoritical value of 500GB/Hr

Using NDMP over TCP/IP where tape is connect to backup server, we get only 300GB/Hr

GABRIEL_CARMO
5,929 Views

You could post how is your topology between SAN storage, Tape Library and backup server?

My problem seems to be in the topology.

Public