Snapmirror design question...

pash_hrnic · ‎2013-06-06

Hi everyone,

We've run into some issues with snapmirrors consuming too much bandwidth, and need to re-think the way we do them. A recent rip-and-replace of our backup environment caused us to destroy, and recreate our snapmirror relationships. In addtion to what was previously mirrored, we are now mirroring approx double the data and wish to isolate this traffic from the production network (which are only 1Gb/s)

Our setup:

- Sources are a clustered Pair of FAS3140 running ONTAP 8.1.2 7-mode (mix of SATA and FC...node 1 = controls FC & node 2 = controls SATA (appropriately cabled for MPHA), node1's disks/vols are only available to node 2 during a failover...and vice versa)

- Destination is a single head FAS3140 running ONTAP 8.1.2 7-mode

- There are 4 x Gig NICs in each device that are not being used.

There's 25-30' between racks; the idea is to use a direct cable connections from each node in the clustered pair, to the destination node (total of 4 eth cables). My question about this design...is it possible & has anyone done it? If so...What vif modes do we use to at each end to establish a successful connection? Ideally, we would like to aggregate the remaining NIC's at the destination without having to use 2 IP's (while maximizing throughput...redundant cabling for mirrors not an issue ATM)

Any insight & information is appreciated

Thanks.

bbjholcomb · ‎2013-06-06

I suggest a multimode vif with LACP (if your switch supports it). I would also control the bandwidth in the snapmirror.conf file, kbs=. You will have make sure the destination IP address is routed through this vif.

SNETAPPUSER · ‎2013-06-06

If my understanding is correct, your setup is as follows..

Primary site: You have FAS3140 Standard HA Pair and each node has four 1GbE NICs and both nodes have volumes that are own by one particular node at any given time

Secondary / DR Site : FAS3140 Single Controller and has 4GbE NICs

What you want to achieve is Snapmirror from both primary site controllers to DR site controller

PR NODE01 -

-

Snapmirror - ----> DR NODE01

-

PR NODE02 -

This is possible and a working solution (one of our customer had the same exact setup until very recently)

From design point of view what you want to do is...

1. Create a interface group (VIF) using two free 1GbE NICs for each FAS3140 standard HA pair; use LACP (if your switch supports it) and IP based load balancing

2. Assign relevant IP addresses and subnetmask details for VIFs along with VLAN ID and Gateway (if in used)

3. In the DR Node, create another interface group (VIF) using freely available four 1GbE NICs (or you may use two of them; it replay depends on the bandwidth you want for the SnapMirror)

4. Assign relevant IP address and subnetmask details for the VIF along with VLAN ID and Gateway (if in used) - for DR NODE

5. Configure your Snapmirror options and /etc/snapmirror.conf file to suit your requirements. As bbjholcomb has mentioned, you can use " kbs=" in snapmirror.conf file to throttle the bandwidth use by Snapmirror transfers.

Hope above information and guides will helpful in designing your Snapmirror environment.

pash_hrnic · ‎2013-06-10

SNETAPPUSER wrote:

.

3. In the DR Node, create another interface group (VIF) using freely available four 1GbE NICs (or you may use two of them; it replay depends on the bandwidth you want for the SnapMirror)

Hope above information and guides will helpful in designing your Snapmirror environment.

This is an on-site mirror and not a DR replica

From a networking perspective I believe that we are on the same page...my wishes are to eliminate the switch infrastructure altogether, the idea is to use a direct cable connection (our replica destination is at the other end of the data center, on a separate ups circuit).

Ideally would like to load balance all traffic across available NICS...however on the destination side, I am not sure if LACP will work without a switch. My main question is what vif modes do we need to use to allow for 2 x load balanced OUTGOING mirrors to make a connection to a vif group of 4 interfaces at the destination?

bbjholcomb · ‎2013-06-13

I don't think you can use LACP without a switch. You could setup two direct connections from the controller to controller, use two IP addresses. Chang the snapmirror.conf file to send half the volumes over one IP address and the other volumes would go over the 2nd IP address.

pash_hrnic · ‎2013-06-13

You're right LACP requires connectivity to swtiches that support the protocol.

I have done some testing. I can confirm that a direct cable connection can be done, using static multimode vifs at either end

CLI based snapmirror is pretty straightforward:

Edit snapmirror.allow on source to allow newly created destination IP's
Edit snapmirron.conf file, when specifying source, ensure that you use the IP or hostname-vif (ir filerA-vif2) assigned to mirrors. The Destination field should match the primary hostname.

DOUGLASSIGGINS · ‎2013-06-07

One other suggestion might be to use snapmirror compression:

http://mtellin.com/2009/11/23/using-compression-with-snapmirror-transfers-in-ontap-7-3-2/

Read about the caveats

bbjholcomb · ‎2013-06-10

According to the NetApp documents, if you compress all volumes it can cause an additional 35% CPU utilization.

HENRYPAN2 · ‎2013-06-10

Hi bbjholcomb,

Are you ref to <Using Compression with SnapMirror transfers in ONTAP 7.3.2>

November 23, 2009 By mike 11 Comments

I was wonder what the recent NetApp docs saying the compress all volumes cause an additional ??% CPU utilization.

Cheers

Henry

bbjholcomb · ‎2013-06-13

My last comment was about using direct cables from one controller to another.

tr-3446