2009-11-21 08:10 PM
We're running a particularly nasty nightly "rsync" job on a Debian box connected to a NetApp FAS2020.
The changed data far and wide between on the NetApp, and just the scan part of the rsync is taking 16-20 hours a night.
For better speed, is it possible to run "rsync" directly on the OS of the netapp?
In this case it is a requirement that the far side of the rsync result in a working Unix filesystem, with no NetApp involved.
As far as I can tell so from from the documentation, NetApp ignores rsync in favor of the proprietary SnapVault and
SnapMirror solutions. In this case those are non-starters.
2009-11-22 01:04 PM
Short answer = no.
Longer answer = SnapMirror/SnapVault are designed basically for cases where rsync won't cut it (I like rsync a lot and have written quite a few scripts using....but when you get into the millions of files the initial scan time just kills you). SnapMirror/SnapVault hook into the file system at a pretty low level in order to already know what's just changed and just transmit the differences without an intensive scan.
Cost-wise, a FAS2020 isn't all that expensive now and does include NFS so might be an option.
2009-11-22 06:23 PM
Thanks for the answer: as I suspected/feared.
I've experimented ionotify to help direct rsync to the directories that have changed content. It works ok, but with limitations. ionotify has to set up a watch on each directory, and can't just watch an entire heirarchy. And you have to duplicate the notify on each connected host.
2009-11-23 02:44 AM
No offence but if you dont need a netapp controller buy some disks and use those. Buying a controller and using it as dumb disk is a bit
of a waste of money I reckon.
On the other hand you could maybe use ndmpcopy and/or "vol copy" command that run in Ontap.
2009-11-28 11:12 PM
Good info. How would ndmpcopy apply, if the destination is a Debian host?
While I see lots of implementors over at http://www.ndmp.org/ it is a lot to adsorb. Which product would be right for remote tape or disk mirror backups (assuming no NetApp hardware offsite)?
2009-11-29 10:18 AM
If you don't have any NetApp gear on the remote side, you will have to do host-based replication. All on-board NetApp replication solutions require a NetApp controller at the remote end. And you can't run external software on the controller (like rsync) as there is no mechanism to do so at this time.
-- Adam Fox