Running rsync directly on a NetApp (e.g. FAS2020)

[bepresseditor]

We're running a particularly nasty nightly "rsync" job on a Debian box connected to a NetApp FAS2020.

The changed data far and wide between on the NetApp, and just the scan part of the rsync is taking 16-20 hours a night.

For better speed, is it possible to run "rsync" directly on the OS of the netapp?

In this case it is a requirement that the far side of the rsync result in a working Unix filesystem, with no NetApp involved.

As far as I can tell so from from the documentation, NetApp ignores rsync in favor of the proprietary SnapVault and

SnapMirror solutions.  In this case those are non-starters.

[amiller_1]

Short answer = no.

Longer answer = SnapMirror/SnapVault are designed basically for cases where rsync won't cut it (I like rsync a lot and have written quite a few scripts using....but when you get into the millions of files the initial scan time just kills you). SnapMirror/SnapVault hook into the file system at a pretty low level in order to already know what's just changed and just transmit the differences without an intensive scan.

Cost-wise, a FAS2020 isn't all that expensive now and does include NFS so might be an option.

[bepresseditor]

Thanks for the answer: as I suspected/feared.

I've experimented ionotify to help direct rsync to the directories that have changed content.  It works ok, but with limitations.  ionotify has to set up a watch on each directory, and can't just watch an entire heirarchy.  And you have to duplicate the notify on each connected host.

[eric_barlier]

No offence but if you dont need a netapp controller buy some disks and use those. Buying a controller and using it as dumb disk is a bit

of a waste of money I reckon.

On the other hand you could maybe use ndmpcopy and/or "vol copy" command that run in Ontap.

Cheers,

Eric

[amiller_1]

Ah....ndmp is a very good idea.....should have remembered that (especially with the multiple levels can approximate an incremental/full cycle at least).

[bepresseditor]

Good info.  How would ndmpcopy apply, if the destination is a Debian host?

While I see lots of implementors over at http://www.ndmp.org/ it is a lot to adsorb.  Which product would be right for remote tape or disk mirror backups (assuming no NetApp hardware offsite)?

[adamfox]

If you don't have any NetApp gear on the remote side, you will have to do host-based replication. All on-board NetApp replication solutions require a NetApp controller at the remote end. And you can't run external software on the controller (like rsync) as there is no mechanism to do so at this time.

-- Adam Fox