Subscribe

Any way to increase the rate of deduplication

Are there any tunable parameters to increase/decrease the rate at which the data deduplication process runs?

I am currently un deduping a volume and I would like to increase the rate at which that is being done

Re: Any way to increase the rate of deduplication

From the NetApp Deduplication for FAS and V-Series Deployment and Implementation Guide:

THE PERFORMANCE OF THE DEDUPLICATION OPERATION
The performance of the deduplication operation itself varies widely depending on the factors listed above, and this determines how long it takes this background process to finish running. On a FAS6080 with no other load on the system, we have seen deduplication performances of up to 120 MBytes/sec (running a single deduplication session). If multiple deduplication streams are running, this total bandwidth gets divided evenly into the number of streams.
To get an idea of how long it takes for a deduplication process to complete, let’s say that the deduplication process is running on a flexible volume at 25MB/sec. If 1TB of new data has been added to the volume since the last deduplication update, this deduplication operation takes about 10 to 12 hours to complete. (There are no configurable parameters that can tune the deduplication process; that is, the priority of this background process in Data ONTAP is fixed.)

Although no system-wide control is available you would be able to prioritize dedupe activity (using 'system' as a proxy) on a given flexvol through the use of FlexShare.