Subscribe

Huge CIFS snapshots after dedupe

Somehow our CIFS snapshots get huge after dedupe.

For example, if i dont run dedpue for a week, my snaps are few hundred mb each, i take them twice a day, noon and midnight.

However, when i run dedupe, it somehow increases the size of a last snap. i just run full dedupe (took 6 h), my last snap was 230mb befrore the dedupe, the same snap is now 70Gb, after the dedupe had run.

Also, if i schedule dedupe to run at 11 and then run snap at 12 the snap is way bigger than if there is no dedupe scheduled at all.

Whats going on??

The same things happens on both controllers.

BTW, im running full dedupe on another CIFS volume now and i can actually see the last snap growing as dedupe progresses. It was 124Mb before i started, then i checked it at 39% of dedupe, it was 21Gb, now at 45% of dedupe  its grew to 26Gb.

what the ???????

Re: Huge CIFS snapshots after dedupe

Yeah, unfortunately snapshots are not de-dupe-aware (yet)...

So the thing is your snapshot will normally grow by exactly as much as your de-dupe savings, because from ONTAP perspective de-duped block have changed, hence they must be protected by writing the changes to disks

Although not surprising to me, it's arguably very awkward. The only workaround I know of is to run de-dupe first on a volume with no snapshots & then take a snapshot(s). Obviously it doesn't help much if de-dupe is meant to be run regularly, not as a one off.

Anyone more knowledgeable with better current (coming?) workarounds / resolutions?

Regards,
Radek

Re: Huge CIFS snapshots after dedupe

So does that mean that dedupe is really usless if using snaps?

Because all the data saved by dedupe will be taken by a large snap?

Also, what is file folding, would any of that help?

i have found this command: options cifs.snapshot_file_folding.enable on     -    what would that do? are there any downsides, issues with using it?

Thanks

Re: Huge CIFS snapshots after dedupe

I am not aware of anything more complete & current than the TR-3505 (a.k.a. De-dupe Bible )

http://communities.netapp.com/docs/DOC-1642

Page 20 says:

NETAPP WHITE PAPER

For deduplication to provide themost benefit when used in conjunction with Snapshot copies, the following bestpractices should be considered:

·         Run deduplication before creating new Snapshot copies.

·         Remove unnecessarySnapshot copies maintained in deduplicated volumes.

·         If possible, reduce the retention time of Snapshot copies maintained in deduplicated volumes.

·         Schedule deduplication only after significant new data has been written to the volume.

·         Configureappropriate reserve space for the Snapshot copies.

·         If the space used by Snapshot copies grows to more than 100%, it will cause df –s to report incorrect results, because some space fromthe active file system is being taken away by Snapshot, and therefore actualsavings from deduplication aren’t reported.

·         If snap reserve is0, you should turn off the Snapshot auto-create schedule (this is the case inmost LUN deployments).

So nothing very surprising or massively helpful I am afraid...

And here is the explanation of file folding:

http://now.netapp.com/NOW/knowledge/docs/ontap/rel732_vs/html/ontap/onlinebk/GUID-8E8AEEB7-4E25-4A78-ACCB-882A91B1D61C.html

File folding describes the process of checking the data in the most recent Snapshot copy, and, if it is identical to the Snapshot copy currently being created, just referencing the previous Snapshot copy instead of taking up disk space writing the same data in the new Snapshot copy.

Without file folding everything would be only worse as multiple snaps would get ballooned, not just one!

Regards,

Radek

Re: Huge CIFS snapshots after dedupe

Thanks for the info.

I have turned file folding on, i am also following dedupe best practices, it would be nice if they ever make possible to dedupe snaps.

Thanks

Re: Huge CIFS snapshots after dedupe

So what will happen if i delete the large snap that grew because of the dedupe? will the previous one increase in size?

Attached is my snap pic, the 2nd from the top is the snap that increased in size after the dedupe, can i delete it without  making another one grow?

Re: Huge CIFS snapshots after dedupe

So what will happen if i delete the large snap that grew because of the dedupe? will the previous one increase in size?

Yes, this is exactly what will happen. I've tried this the other day in a lab environment & this is one of the most frustrating things I've ever witnessed - you won't get any actual savings, unless you get rid of all snapshots (taken prior to de-dupe scan).

Sweet...

Re: Huge CIFS snapshots after dedupe

The trick as Radek mentioned it to dedupe before snapshotting. If you are snapping daily dedupe daily. If you snapshot hourly, retain a limited number of those type of snaps, the blocks will only be trapped untill the snapshot expires. So in you current model, you just have to be patient until your snapshot schedule rotates through and you no longer have any snaps from before dedupe. From then out try to dedupe prior to any long term snaps being taken.

Keith

Re: Huge CIFS snapshots after dedupe

So i take 2 snaps a day, at 12AM and 12PM, and keep them for 7 days, my dedupe is set to AUTO, so it runs very rarely, if it does it creates huge snap.

What would be the best dedupe options then? Every day at 11PM?, an hour before snap? I have tried that, but then the midnight snaps are big.

What about running dedupe manually only, like once every month, and deleting all snaps before i do that, would that be the best solution?

Thanks

Re: Huge CIFS snapshots after dedupe

Depending on the type of data you have, you may also want to make sure that you have the cifs file folding set to on: options cifs.snapshot_file_folding.enable on.

Thanks,

Mitchell