is there a limitation (file count) for NDMP? We see a very very poor Performance with NDMP Backups of a 2040c with BackupExec 2010r3. We have not evaluated a different backup software yet so we are not sure if this is the bottle neck.
The volume we are trying to backup consists of several qtree's and has a total around 16 Mio Files.
We've had problems similar to this when we back up volumes that contain large numbers of files. It has to crawl the file list before it actually starts backing up, and sometimes that can take hours before it actually starts moving any data. Commvault supposedly now has a solution that allows it to skip that crawl by using snapshots and only looking at changed data. Another option that NetApp presented to us is doing a volume based backup. That avoids the file crawl, but doesn't allow individual file restores... you have to restore the entire volume. Might be an option if you can use snapshots for individual restores and the volume backup for major disasters.
thanks for the recommendations. Is there a trial version of commvault available? I had recommended commvault in the past to our boss but we had backup exec for several years now and it is quite ok. I will now evacuate all qtrees wth more than 600.000 files to a new volume and create a separate backup job for it. We have two LTO 5 FC Tape drives, so we can split it up.
We are using EMC Networker at are seeing extremely slow thoughput as well. We get 7Mb/s regardless of whether the volume has many files or one large file of a similar size. A CIFS mount to the same volume and we get 70Mb/s. There is something extremely sucky about NDMP. To date we have had no suggestions from either EMC or IBM (N-Eries) that has made any difference at all.
I had several installations with NetWorker and backup speed was OK. Restore was different matter though ☹
Incidentally, all installations were using 3-way backup (via NetWorker DSA) and not direct backup to tape. I constantly observed full wire speed (120MB/s for 1Gb/s interface). But these all were database installations with relatively small number of large files, 10-20GB each.