ONTAP Discussions

Running rsync directly on a NetApp (e.g. FAS2020)

bepresseditor
11,198 Views

We're running a particularly nasty nightly "rsync" job on a Debian box connected to a NetApp FAS2020.

The changed data far and wide between on the NetApp, and just the scan part of the rsync is taking 16-20 hours a night.

For better speed, is it possible to run "rsync" directly on the OS of the netapp?

In this case it is a requirement that the far side of the rsync result in a working Unix filesystem, with no NetApp involved.

As far as I can tell so from from the documentation, NetApp ignores rsync in favor of the proprietary SnapVault and

SnapMirror solutions.  In this case those are non-starters.

6 REPLIES 6

amiller_1
11,198 Views

Short answer = no.

Longer answer = SnapMirror/SnapVault are designed basically for cases where rsync won't cut it (I like rsync a lot and have written quite a few scripts using....but when you get into the millions of files the initial scan time just kills you). SnapMirror/SnapVault hook into the file system at a pretty low level in order to already know what's just changed and just transmit the differences without an intensive scan.

Cost-wise, a FAS2020 isn't all that expensive now and does include NFS so might be an option.

bepresseditor
11,198 Views

Thanks for the answer: as I suspected/feared.

I've experimented ionotify to help direct rsync to the directories that have changed content.  It works ok, but with limitations.  ionotify has to set up a watch on each directory, and can't just watch an entire heirarchy.  And you have to duplicate the notify on each connected host.

eric_barlier
11,198 Views

No offence but if you dont need a netapp controller buy some disks and use those. Buying a controller and using it as dumb disk is a bit

of a waste of money I reckon.

On the other hand you could maybe use ndmpcopy and/or "vol copy" command that run in Ontap.

Cheers,

Eric

amiller_1
11,198 Views

Ah....ndmp is a very good idea.....should have remembered that (especially with the multiple levels can approximate an incremental/full cycle at least).

bepresseditor
11,198 Views

Good info.  How would ndmpcopy apply, if the destination is a Debian host?

While I see lots of implementors over at http://www.ndmp.org/ it is a lot to adsorb.  Which product would be right for remote tape or disk mirror backups (assuming no NetApp hardware offsite)?

adamfox
11,198 Views

If you don't have any NetApp gear on the remote side, you will have to do host-based replication. All on-board NetApp replication solutions require a NetApp controller at the remote end. And you can't run external software on the controller (like rsync) as there is no mechanism to do so at this time.

-- Adam Fox

Public