Data ONTAP Discussions

Different types of Deduplication

Hi Guys,

   I am new to this stuff and I always keep on hearing the term inline dedup which tells me that we have something like inline and out of line dedup so I was wondering what are the differences between them??



Re: Different types of Deduplication

To my knowledge in Inline deduplication as data comes to system it looks at it and if found matching with other block within the volume it just updates the pointer rather than writing the data, which makes system bit slow if it is having high data flow and performance suffers (BTW it is available only in VTL) while in other type of deduplication it writes all the data to disk and at defined time or % new data it starts comparing for duplicate blocks and then removes it.

In short it's just the matter of when duplicate blocks blocks are getting removed, before they are written to disk or after they are written to disk.

Re: Different types of Deduplication


     Thanks a lot for the reply....  so only VTL supports inline dedup what about normal disks? shall we use a third party software for example??


Re: Different types of Deduplication

I am not sure what do you mean by normal disk and third party software, can you be more specific please?

Re: Different types of Deduplication

I mean, a DS-14 shelf connected to FAS 3000 for example can't

i configure inline dedup for such setup? and third part software I heard that some softwares can

can perform the inline deduplication but I do not know how...

Thanks for being keen in helping me out..

Re: Different types of Deduplication

Yes you are right you can't use inline deduplication in FAS or V series and rest about third party softwares I am not sure who offers it but as my general knowledge goes the lun/volume needs to be in their control to have inline deduplication offered because they can't simply ask ontap to do duplication or teach that it's a duplicate block reaching to you buddy so remove it.

Re: Different types of Deduplication


thanks for this but it maybe a client software that resides on the host as an agent or something...

Thanks again for the professional reply

Re: Different types of Deduplication

There isn't any client software from NetApp which allows to do this and still I have to hear for this type of client application from NetApp or someone else for this job.

Re: Different types of Deduplication

In-line Dedupe is offered by CommVault ( 3rd Party Software).

What it does is, before committing to the Storage it scans the files as they leave the client/server where it's installed and removes the redundant files right from the source before committing to the storage.

Hope this helps.

Re: Different types of Deduplication

Thanks a lot, that was helpful and thats what I thought ... thanks again.