LUN Dedupe and space usage

[tomatthe]

Working with a small FAS2750, 2 x ~10tb Aggregates.   The only intention for this system is to use for VMware datastores via iscsi.

It seems thin volumes, and unreserved space luns are the general recommendation for this setup.  What isn't clear to me is how to take advantage of the space savings from dedupe.  As far as I can tell the only way to access this saved space would be to over provision luns from a volume which just feels kind of strange.  I'd basically be taking a guess at what savings I might get and would need to watch space fairly closely.

Looking for suggestions or just confirmation that basically over provisioning luns is the only way to use the space savings in this scenario.

[ttran · Accepted solution]

Hello Tomatthe,

Deduplication provides significant physical space savings for datastores as common data across the datastores will only use one set of physical blocks. Please keep in mind that it will take time for all the scanning of all the blocks to determine a common footprint before you start seeing the space savings. It is common to over-provision the LUN(s); just keep in mind to have volume autosize/autogrow enabled and there's sufficient space in the aggregate for the volume. Where data access can be interrupted is where the aggregate runs out of space to sufficiently grow the volume(s).

You can find more information and the four different configuration options that explains any space savings on the bottom of page 69 of the following document:

NetApp Data Compression, Deduplication, and Data Compaction 

Regards,

Team NetApp

[tomatthe]

Thanks for the info, I'd read that doc along with quite a few others on this topic.  This system has a single use case which is VMware datastores.  I had generally hoped to be able to setup volumes/luns and basically leave it alone.  The volumes size is probably 95% of the aggregate, and there is no room for it to expand if needed so not sure the autogrow would really do much here.

It sounds like the only thing I can do is to over provision, and just keep a close watch on the used volume space before it becomes a problem.  I had a support ticket on this and asked repeatedly for someone to just please verify that, yes you had to overprovision luns to actually gain the benefit of dedupe but could never get anyone to actually say that.  It has a been a rather frustrating experience for what I feel like is a pretty straightforward and simple use case.  Thanks for verifying what seems pretty obvious, but is kind of a hassle to work with imo.

[keitha]

Hey,

I don't want to start a protocol war BUT you might want to try some NFS datastores (since you are already using IP. With NFS, there are no LUNs in play so as Dedupe occurs you will see the space savings in the datastores right away. 

If however you MUST use ISCSI then yes you do have to Over Provision. Thats the same with every platform that uses LUNs with storage efficiencies such as dedupe. You can also install the VSC (Virtual Storage Console) which will allow you to monitor the actual space usage from within vCentre  

[tomatthe]

Agreed on NFS, but basically wasn't an option for this setup.  I was using a Nimble array prior to this which handles things a little differently and that is the functionality I was looking for.  I wasn't used to the over provisioning, and really just wanted some verification that yes that is the proper way to do it with this setup.  Thanks for the added info.