Throughput Expectations

[dfamiglietti]

All,

I have been having issues with my backup speeds. I talked to my backup vendor and they tracked it down to being a throughput issue.

I want to know if what I am experiencing with my NetApp 2220 is to be expected and if so what I can do to speed things up.

Bellow I will explain the environment, what we would like to know is this:

• Is the networkthroughput we are seeing look low?
• If it looks what is to be expected what could we go to maybe speed this up (thinking of getting the 2220 10 GBE card).

Here is a little background:

We have been using NetApp here for quite some time our production storage is a pair of 2040's.

We went to a new backup solution at the start of the year (CommVault utilizing NetApp SnapProtect) we wanted varying retention so we are using SnapProtect for snapshots (7 day retention) on our production 2040's. We also purchased a 2220 so we could mount those snapshots and backup to disk for longer retention (30 day retention) then after 30 days CommVault would push this data to tape (7 year retention).

Phase 1 and 2 (using SnapProtect for snapshots and then mounting those snaps and backing up to disk) was just completed so we went on our way to setup the tape backup. This is where we ran into an issue, how the tapes work in CommVault is it uses a full backup from disk (which was deduped when the data was backed up) rehydrates that data and copies it back from the SAN to the tape. it seems as if the NetApp isn’t serving this data very quickly.

Here are the Throughput numbers we are seeing:

From the 2220 GUI:

I/O Throughput (Network Sent) ~12 - 24k kB/sec

IOPS: ~220 - 320

From the backup Server Teaming software:

Average RX Traffic:150 Mbps

Peak RX Traffic: 270 Mbps

Here is a little viewon our architecture:

NetApp 2220 Backup Storage SAN:

• iSCSI traffic
• Raid DP with 12 SATA Disks
• 4 1GB ports from the san trunked into my pair of stacked 2960S's with a corresponding port channel group setup (Channel-Group 4) carrying the storage vlan (vlan 220) and the management vlan (vlan 10).

Backup Server:

• 8 NICs Total
• Team1: 2 NICs (set in failover) on our server management network (vlan 10)
• Team 2: 3 NICs set in 802.3ad Dynamic with FT balanced by Destination IP with a corresponding port channel group setup on the 2960S's (Channel-Group 5) on our production storage vlan (vlan 200) These are the 2 2040's that runs our server environment.
• Team 3: 3 NICs set in 802.3ad Dynamic with FT balanced by Destination IP with a corresponding port channel group setup on the 2960S's (Channel-Group 6) on our production storage vlan (vlan 220) This is the 2220 that holds our backup to disk data.

Tape Drive:

• HP LTO 6 Tape Drive connected via Fiber Channel (already tested and working perfectly)

[jskinner]

There is some tuning that may be done in multiple places, including the disk library level. I would suggest a support call to CommVault & NetApp with the details above to start with and the Simpana version and ONTAP version information to both parties.

You may be asked to run some additional reports/data pulls from the Commserve or NetApp controllers to pinpoint something as well.

[dfamiglietti]

I have an open case with NetApp, although I have not gotten any information at all from level 1. I can also reopen my CommVault case if needed. I noticed your a NetApp employee, are you in support? is there any way I can send you the NetApp case number so you can take a look or move the case along to the correct party. I ask as you seem to have an idea from both the CommVault and NetApp perspective.

FYI

we are on OnTap 8.1.2

and Simpana 9R2 service back 9b

[jskinner]

Hi Daniel,

I found your case. I've added the details you posted above from the storage/network perspective to our support case.

I would definitely either re-open or open a new case with CommVault and relay the case numbers to both parties in both cases.

[dfamiglietti]

Just opened a new case with CommVault and explained that NetApp asked me to open the issue. I sent you a PM but is there any way you can take ownership of the case or give it to someone who is fimilar with both sides of the issue liek yourself as CommVault is requesting a NetApp be setup so we can have a conference call with both vendors involved.