2011-12-02 06:17 AM
We have many existing volumes which seem to be good candidates for dedupe. These volumes have regular snapshots and snapmirror snapshots.
We have some internal confusion on the issue with deduplication and snapshots. The guides I have read say something along the lines of, a snapshot "locks" blocks which are not available to be deduped. Can anybody explain what this actually means? As in, if I take a snapshot of a volume with many duplicate blocks, make no changes to the active file system (i.e. snapshot doesn't grow) and then run dedupe, does this mean that I will see zero savings on it? When the snapshot expires, will I then instantly see dedupe savings? Or am I required to re-run dedupe with no existing snapshots in order to see savings? Or does it mean something else entirely?! Does it mean that if I take that snapshot, then change blocks (i.e. snapshot grows to reference those blocks not in the active file system), that those blocks are locked and unable to be deduped, but the active file system can be deduped entirely?
I feel like I have a pretty good understanding of the snapshot process, but as it stands I can't figure out how we would ever get any meaningful performance out of dedupe on some of these volumes which are either snapshotted or snapmirrored fairly frequently. My understanding is that any block which is referenced by a snapshot does not get deduped. The guides and explanations I've read make sense for snapmirroring infrequently (weekly or even daily). But if I am snapmirroring once every 30 minutes, virtually all blocks at all times will be "locked" in snapshots.
Hopefully I am reading this wrong! We would like to use dedupe on some of these volumes but it seems like this would have to be a manual process...for instance, once a month, manually clear all snapshots and break snapmirrors, dedupe, then re-establish snapmirror and snapshot schedules.
2011-12-02 06:59 AM
As in, if I take a snapshot of a volume with many duplicate blocks, make no changes to the active file system (i.e. snapshot doesn't grow) and then run dedupe, does this mean that I will see zero savings on it? When the snapshot expires, will I then instantly see dedupe savings?
This is *almost* correct. In fact it will look as follows:
- existing snapshot, 0GB in size
- dedupe is run and 'saves', say, 50GB of space
- snapshot grows at the same time by the same value
- snapshot expires and at that point in time your savings become 'real'