Volume efficiency and deduplication

[sakula43]

Hello Team,

I want to understand if there is difference between "volume efficiency and deduplicaiton " in netapp. When i search for deduplication, i find articles on how to set volume efficiency for the volume. 

But,

We have 2 volumes, "1 have volume efficiency on" and another volume dont.

But when i give the below command on both the volumes, i see dedupe-space-saved for both the volumes.

"volume show volumename -fields dedupe-space-saved"

So, does that mean deduplication is auto enabled on all the volumes by default ? If so, what is the use of enabling "volume efficiency "? 

My goal is to free some space on the aggregate. I have already deleted unwanted snapshots and unused volumes. 

[Alexvasile]

Hello ,

Deduplication is one of the space efficiency features .

You can have a volume with only some of the space efficiency features on ;  ( if you have a AFF device then by default all the features are enabled ; for FAS you need to enable space efficiency support.) .

Space efficiency has :  compression , inline compression , inline deduplication , Corss Volume  Deduplication , Cross Volume Background Deduplication , Compaction .

Check : https://docs.netapp.com/us-en/ontap-cli-991/volume-efficiency-show.html#examples

For FAS : https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/How_to_enable_volume_inline_deduplication_on_Non-AFF_(All_Flash_FAS)_a...

Hope this helps,

Alex

[sakula43]

Hello Alex,

Thanks for your response. 

The 2 volumes in discussion here are both from FAS devices. 

1st volume -- 4 TB of size, 

When i give the below command, it says 3.7 TB is dedupe-space-saved.

vol show -volume volname -fields dedupe-space-saved

When i list volume efficiency for this volume, it says volume efficiency is enabled but all the storage efficiency values are false.

Compression: false
Inline Compression: false
Constituent Volume: false
Inline Dedupe: false
Data Compaction: false
Cross Volume Inline Deduplication: false
Cross Volume Background Deduplication: false
Extended Compressed Data: false

2nd volume : This is 40 TB and volume efficiency is not enabled. Still when i use volume show, this says only 110 MB is dedupe-space-saved.

Does that mean deduplication is enabled on this volume? doesn't volume effic should be on to have deduplication?

Also, if i enable volume efficiency on this 40TB volume now, how much time do you think it will take to complete the action? 

[Alexvasile]

Best place to do a research is inhere : https://docs.netapp.com/us-en/ontap/pdfs/sidebar/Use_deduplication__data_compression__and_data_compaction_to_increase_storage_efficiency.pdf

As a note :

Cross volume deduplication is supported only on volumes that are owned by nodes that are All-Flash optimized personality enabled.
Inline deduplication is supported only for volumes that are either owned by nodes that are All-Flash optimized personality enabled or on hybrid aggregates.

You can enable compression out of the bat  via vol efficiency modify ***** -compression true.

PS : on FAS volume efficiency is enabled means you are eligible ( but most of them have strings attached - manual work to enable  ) 

Normal deduplication as post process can be achieved with :

Ex : vol efficiency start -vserver vs1 -volume sis1 -dedupe true -scan-old-data true

Also check this : https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/How_to_set_up_deduplication%2F%2Fcompression_in_ONTAP_9

Alex.

[paul_stejskal]

There may have been a dedupe scan at one point and only 110 MB of data is still deduped, and the other 39+ TB isn't. Dedupe being off means a scan won't run. The data state is not impacted because that is a separate and more dangerous process to undo and not recommended without NetApp supervision.

I hope this helps.

Also on how long? It could be a while as you have spinning disks. Change rate will also come into play. If the data is static the initial scan may take hours or even up to days, but subsequent scans will be quicker.