Limit the amount of SIS or Dedupe jobs run at one

[davidrnexon]

Hi, is there a way in c-mode to limit the amount of dedupe jobs that can run at one time ?

[bbjholcomb]

Yes, you can setup a schedule. In 7DOT/7mode you could only run them on the hour and the limit was 8 concurrent.

You can create new schedule or use existing ones. We setup a bunch of them, every two minutes for a few hours. Most run in a few seconds, we have been this in 7mode for 2 years, 400 volumes, it has saved us about 252TB. We only run this on volumes that don't contain DB, all or NFS.

To create a new efficiently policy:

     volume efficiency policy create -vserver xxxx -schedule xxxx (you can -duration if you want it stop if it's been running too long).

We have it working in cDOT for a new months on 8.1 no problems so far.

[davidrnexon]

thanks for your reply, so based on your schedule 1 dedupe job can not run for more than 2 minutes ? Is that a bit of a nightmare to administer a schedule for 400 volumes ?

There must be a setting somewhere that will limit the amount of concurrent dedupe job running, that way just use the default midnight schedule and let the system work out the queues.

[bbjholcomb]

The dedupe will run as longs as takes, it depends on how much was changed and is duplicated.

You can set the duration so that it will stop the dedupe if it runs too long.