Backup is running slow to AltaVault

[netappmagic]

We have virtual appliance, and currently only have a few TB on the cache. Users are complaining the backup is running slower on NFS than NFS on the filer.

There are not too manu attributes I can set on the data interface, for instance, full/half duplex, speed.

Please help, where can I start with to check out the performance issue like this?

[chriswong]

Hi,

If you have a paid appliance you should contact NetaApp support for resolution. If you are using an evalulation AltaVault, then you should start with basic transmission tests (single file over 1gbE or 10gbE), and provide more context about the environment it is in (how did you mount the NFS share, what options, how many users, etc). AltaVault is designed for sequential workloads (large objects of several GB in size minimum), so if you're using this as a typical file share for users to access, then the degraded performance would not be unexpected.

Regards,

Christopher

[heightsnj]

@chriswong

We are using it for backups only.

When you are saying "AltaVault is designed for sequential workloads ", do you mean that we should not run too many backups around the same time? Are there any optimal number or preferred number of backup sessions can be run to backup data (via NFS over 10GbE) simultaneusly over to ALV from different clients?

[chriswong]

Hi,

There are a number of TR (technical reports) which provide guidance for configuring backups to AltaVault, depending on which backup application you're using. There is also a performance tuning guide TR available as well. This has a section which provides performance based on certain controlled tests to an AltaVault AVA800. There are no performance performance results that I can share for AVA400 or virtual models, but you can estimate the performance based on the single and multi-stream backups. Performance typically will peak between 16-20 sessions, and less for virtual appliances. This of course is dependent on aspects of the backup application as noted in the TRs.

Weblink: http://www.netapp.com/us/library/index.aspx (search AltaVault TR)

Regards,

Christopher

[heightsnj]

@chriswong

In AltaVault Performance Tuning document, page 7, assuming an application bckup can back up a job in 50MB/s rate, then for an 10Gb interface, the backup can handle 10-15 jobs. I thought it should be able to handle more, because 10-15 jobs only use 500-750MB/s, and 10Gb is equivalent to 1250MB/s.

Am I missing anything here?

[chriswong]

In theory, the scale should be linear until there is a bottleneck introduced. For virtual appliances, this is typically at the CPU, memory, disk, or disksubsystem level. Networking at 10gbe tends not to be the bottleneck in both virtual and physical AltaVault appliances.

Regards,

Christopher

[heightsnj]

@chriswong

Understood.

However, I am trying to make sure I understand  the concept in the scenerio as described in the document, page7

How did the document (Altavault Performance Tuning, page7 ) come to the conclusion that the backup application can handle 10-15 jobs? If we devide 1250 MB/s (=1GbE) by 50MB/s, 1GbE can handle 25 jobs, assuming the application backup can backup a job in the rate of 50MB/s. Am I correct here?

[chriswong]

This is an example. An AltaVault appliance (400/800) can typically saturate at up to that number of sessions before a bottleneck begins to appear (not related to the 10gbe interface, but somewhere else in the system, i.e. disk/cpu, etc as previously mentioned) depending on workload and other factors. I could understand how it might be misleading to say "saturate 1 of its 4 10gbe" here, but the saturation is of the appliance resources, not the interface directly.  Ch 5 provides the actual performance guidelines based on actual testing done to an AltaVault appliance and is more accurate relative to determining measurements of sessions and throughput.

[heightsnj]

@chriswong  Thanks!

Two more following-up.

Will using 1 x 10GbE or using 4 x 10GbE simultaneously make any performance differences and why? Assuming one 10GbE is not a bottleneck here.

Also, in the example of 5.1 Basic Cold Ingest Performance, how did you come to throughput rate of 5.5TB/hr, if one Linux system can deliver 725MB/s? could you please list the calculation here in detail?

[chriswong]

Having additional interfaces will provide failover connectivity, but likely not present more performance. Most users will put the AltaVault interfaces into a VIF (virtual interface, 802.3ad).

To address your 5.1 comment, the read throughput is to show that there was no bottleneck on the client side (aka the 4 linux systems could deliver in excess of capacity over the network to AltaVault). In this case, they could deliver in excess of 2.9GB/s, which is greater than 1.5GB/s (which is the translation from 5.5TB/hr) to AltaVault. Again, these are examples from synthetic workload testing of data sent to AltaVault.

Regards,

Christopher