Is deduplication safe for my data?

[lfreeman]

Deduplication eliminates redundant files stored on disk. Since you are only storing a single copy of the data, this raises the issue of the reliabilty of storage systems that offer dedupication. Here is an article that digs deeper into the subject:

Anyone care to comment on steps they take to protect their deduplicated data?

[sorenmikkelsen]

Dear Larry,

personally I believe that NetApp deduplication pose less than a "disappearing risk" of data loss, through the algortihms used for identification of duplicate data/blocks.

As with protection against data loss caused by disk failure, it's a matter of the cost (e.g. RAID4 vs. RAID-DP).

Of course the algorithms can be made less prone to false recognition, but that will cost performance. Eventually that could be overcoem through a dedicated dedupliation controller (something similar to the deduplication in VTL running on a dedicated CPU).

I guess the bigger concern for data loss to companies today (compared to data loss which may be caused by false-recognition in deduplication), is the (un-)intended deletion by user, prior to having the data secured (e.g. via snapshot or backup).

In my honest opinion, I'd not implement (nor even further consider) any specific measures for protecting data, only because of deduplication may falsely recognize/delete data.

Kind regards,

Soren