Predicting SnapMirror transfer size / required bandwidth

[strattonfinance]

Hi all,

What is the easiest way to predict the amount of data that would be transferred daily between two filers using SnapMirror?

Is it as simple as adding up the current snapshot deltas for 24 hours for volumes that would be SnapMirrored, assuming our current snap schedule matches up with the schedule we'd use for SnapMirror?

e.g.

On volume X we snapshot hourly for 24 hours, and would be happy with SnapMirror replicating data every hour (so max 1 hour data loss in a disaster).

Therefore amount of data that would be transferred by SnapMirror in a day = (hourly.0 -> hourly.1 delta) + (hourly.1 -> hourly.2 delta) + ... + (hourly.23 -> hourly.24 delta)?

Is that correct? Or is there a simpler method?

Thanks all.

[scottgelb · Accepted solution]

The easiest method is to use the "snap delta" command.  You can specify a volume and snapshot names... so you can specify vol1 and nightly.0 and nightly.1 for example.  I would do this over a period of time that is a good sample of daily change rates to get an average daily change.  Maybe run it for a few days, a week, month, whatever makes sense and take the totals.  Also, 7.3.2 officially supports snapmirror compression (LZ) so you can get a smaller transfer than you calculate if the data compresses well.

[strattonfinance]

Thanks for the info Scott - what you've suggested is what I was trying to describe, albeit a lot less eloquently 🙂

Assuming we would use hourly replication, would the delta between two nightly snapshots give us correct figures for a 24 hour period? Or do we need to add up the deltas between 24 hourly snapshots to get correct figures?

Thanks for the info re compression in 7.3.2 as well, sounds interesting - are there any tools to estimate how compressable our data would be?

[scottgelb]

There could be variances between the hourly and nightly... depending on changes between each snap.. The nightly delta will tell the daily change rate, but if high changes and deletions or something between hourly it could be different for each hourly run... You could run snap delta every hour then take the hourly change rate for each hour over several days and put in a spreadsheet to calculate the max and average over that time.  You could also do the same with the nightly for comparison.

I don't know of a way to estimate the compression... but since it's LZ you can estimate similar to the tape backup if you do ndmp to tape (although the snapmirror will be block changes, not file changes).  2:1 is what we can hope for although it could be less or more depending on the data...if all 0s for example...not likely but you get the picture.

You could also setup the D.R. system side-by-side on the LAN or direct attached then just run the mirrors...then you can use the snapmirror logs to see what is happening and size the line accordingly.

[strattonfinance]

> There could be variances between  the hourly and nightly... depending on changes between each snap.. The  nightly delta will tell the daily change rate, but if high changes and  deletions or something between hourly it could be different for each  hourly run... You could run snap delta every hour then take the hourly  change rate for each hour over several days and put in a spreadsheet to  calculate the max and average over that time.  You could also do the  same with the nightly for comparison.

Thanks, will do this as part of our analysis.

> I don't know of a way to estimate  the compression... but since it's LZ you can estimate similar to the  tape backup if you do ndmp to tape (although the snapmirror will be  block changes, not file changes).  2:1 is what we can hope for although  it could be less or more depending on the data...if all 0s for  example...not likely but you get the picture.

Well, we're already using de-dupe so most of our zeros won't be transferred anyway 😉 Not really sure how compressible our data will be - certainly not highly compressible. It's all VMware, a mix of OSes, apps and user data. GUess we'll just have to assume no compression and regard it as a potential bonus.

> You could also setup the D.R.  system side-by-side on the LAN or direct attached then just run the  mirrors...then you can use the snapmirror logs to see what is happening  and size the line accordingly.

Would love to do this, but this process is to determine if the bandwidth required to use SnapMirror is even feasible for us - we won't be buying the second filer unless it is.

Thanks for your help.

[jsykora]

I would definately recommend a few weeks of trial SnapMirror with your DR filer sitting off your LAN and then monitor the reports to get a better indication of what's going on.  The "snap delta" command is definately your friend when doing SnapMirror.

One thing you might want to look into if you're squeezed on WAN bandwidth is to look into a pair of Riverbed Steelhead WAN accelerators.  These things are pretty much amazing.  I use ASIS on all of my SnapMirror volumes, but even after that the Steelheads are usually able to reduce the SnapMirror traffic by 75-95% on average.  I'm replicating a FAS2020 a/a with an additional SATA shelf over a 1.5mb/s MPLS connection using two Steelhead 250Ms.  I'm in the process of swapping the Steelhead 250Ms for 550Ms so that I can go up to a 3mb/s WAN link at my primary and DR sites.  The 250Ms will go to other remote sites since the job they do on MAPI, CIFs, etc are amazing as well.

I'd say SnapMirror and Steelheads were made for one another!

[alapati]

I think everything has been said. However, I want to point out there is a tool on the NOW Utility ToolChest that estimates the transfer time and how much data is to be transferred. You might want to take a look here. I also attached a screen shot.

SnapMirror Progress Monitor v2.1

http://now.netapp.com/NOW/download/tools/smpm/

[strattonfinance]

> However, I want to point out there is a tool on the NOW Utility  ToolChest that estimates the transfer time and how much data is to be  transferred

Thanks, looks to be a useful tool.