ONTAP Discussions

Send in your NetApp dedupe story and get a coveted "We Got Your Back" t-shirt from DrDedupe

lfreeman
7,748 Views

With thousands of people using NetApp deduplication today, there are sure there are some interesting stories out there-

What surprised you most about NetApp deduplication?

What applications are you deduplicating today?

How did deduplication change the way you think about storage utilization?

What did deduplication allow you to do with your new-found storage capacity?

We'll review all stories and award our popular "We Got Your Back" NetApp dedupe t-shirts to the writers and teams that send them in. Post your stories here on the dedupe community or send an email to |wegotyourback@netapp.com] for consideration.

DrDedupe

wegotyourback4.jpg

18 REPLIES 18

BrendonHiggins
7,710 Views

I have just purchased a FAS3140 with 5 shelves of 1Tb SATA drives. We are using it as a snapvault target.

Have just set sis running against the 1,433Gb of office documents in a CIFS volume used by our document managment system. There are multiple versions of the same document in the volume. Ran a test in June 08 using our FAS3070 and the 'large' CIFS volume withover 4Tb of office docs of the same type in it. Result was 14% saving on a volume with no snapshots.

Now I are running SIS on the none deduped 'small' volume.

Have an office swipe stake were a £ lets you pick your %. Closest to correct % wins. SIS is running now and results due in a day or two. Anyone want to play?

{Paypal link} ~ No just kidding

Bren

lfreeman
7,709 Views

Tell you what Brendon, when you hit 20% I'll send your entire team NetApp We Got Your Back t-shirts. But you'll need to promise to send me a picture of everyone wearing their shirts with pride...

DrDedupe

BrendonHiggins
7,709 Views

Results are in ~ I lost 😞

Filesystem used saved %saved
/vol/sv_dmfil01wsg/ 1132GB 236GB 17%

Last Operation Begin: Wed Dec 10 14:34:15 GMT 2008
Last Operation End: Thu Dec 11 19:27:14 GMT 2008

Was run on a FAS3140 in a aggregate made up from 12x 1Tb SATA drives. No other load on the filer as it is still in test. Noticed a CPU hit of 2 - 3% {very light}

lfreeman
7,709 Views

Tell you what Brendon, we'll round that 17% saving up to 20% and send your team the shirts. Send me an email larry.freeman@netapp.com with the sizes and mailing address and I'll send them out!

DrDedupe

rickscherer
7,709 Views

All right so for the most part we're only running A-SIS on our VMware NFS volumes. We are seeing around 61% savings over our four datastores.

root@dpcrcvmesx1 root# esxcfg-nas -l
vmstore1 is /vol/vol501/dpcrcvmstore1 from sfas8 mounted
vmstore2 is /vol/vol507/dpcrcvmstore2 from sfas8a mounted
vmstore3 is /vol/vol154/dpcrcvmstore3 from sfas7 mounted
vmstore4 is /vol/vol155/dpcrcvmstore4 from sfas7a mounted
root@dpcrcvmesx1 root# ssh fas7 df -khs vol154
Filesystem used saved %saved
/vol/vol154/ 137GB 208GB 60%
root@dpcrcvmesx1 root# ssh fas7 df -khs vol155
Filesystem used saved %saved
/vol/vol155/ 159GB 258GB 62%
root@dpcrcvmesx1 root# ssh fas8 df -khs vol501
^[[AFilesystem used saved %saved
/vol/vol501/ 345GB 460GB 57%
root@dpcrcvmesx1 root# ssh fas8 df -khs vol507
Filesystem used saved %saved
/vol/vol507/ 316GB 566GB 64%

Proof is in the pudding I was really surprised how easy it was to enable on our 6030, just enabled it and kicked it off and in a few hours we were saving well over 1TB of data For 101 virtual machines on this particular cluster I am extremely happy! This definitely allows me to worry less about how much storage our VM environment is taking up, and focus more on maintaining it (and not my storage). I am also enjoying how dedup is inherited through our SnapMirror replication, that puts a lot less stress on our WAN (even though we're fibre connected between the sites, its still nice to know we're not throwing too much over it). In all, this was a zero effort way to save a significant amount of storage and defer potential storage fees! Thanks NetApp!

lfreeman
7,710 Views

Rick that is one great story. Please enjoy your 1TB storage rebate courtesy of NetApp deduplication (actually 2TB with your replicated copy.) Go ahead and send your shirt size and mailing address to me larry.freeman@netapp.com and your prized shirt is as good as there!

DrDedupe

cebulrdcis
7,708 Views

We have implement de-dup in our NetApp FAS3070C Primary Storage and to our FAS3020 our Secondary storage (Snapvault), We do dedup on our NFS - VMWare and Xen Virtual Servers, CIFS - User Home drives, Department share drive - office documents, pdfs, htmls, manuals, installers and drivers for our development and so with Lotus Notes Database archives. It really gives us a lot of savings in space where-in we can allocate it with new requirements and increase in quota. Below are some of our dedup savings highlights.

filer01-3070> df -g -s
Filesystem used saved %saved

/vol/v_cifs01/ 370GB 98GB 21%
/vol/v_nfs0/ 42GB 12GB 23%
/vol/v_user0/ 138GB 41GB 23%

/vol/v_cifs0/ 572GB 224GB 28%

/vol/vol_bpdimage/ 475GB 449GB 49%
/vol/v_user1/ 479GB 158GB 25%

/vol/v_vmware01/ 35GB 87GB 71%

Filer02-3070> df -g -s

Filesystem used saved %saved


/vol/v_csscnfs01/ 1GB 0GB 14%
/vol/v_cifs02/ 1075GB 274GB 20%
/vol/v_cssccifs01/ 515GB 63GB 11%
/vol/v_cifs03/ 3401GB 239GB 7%
/vol/v_cifs04/ 1102GB 217GB 16%

/vol/v_cifsbkp01/ 289GB 88GB 23%

/vol/v_cifs07/ 38GB 9GB 20%

/vol/v_csscnfs02/ 0GB 0GB 23%
/vol/v_cssccifs03/ 282GB 64GB 19%

We have not implemented dedup on high-io data, but almost all of our data in our primary storage were snapvaulted I have implemented dedup on my snapvault target, below are the results, and it is really amazing.

Secondary-3020> df -g -s
Filesystem used saved %saved

/vol/v_csscnfs01/ 1GB 1GB 43%

/vol/v_mlibtl0/ 34GB 62GB 64%

/vol/v_tcases/ 31GB 35GB 54%
/vol/v_user0/ 139GB 62GB 31%

If you will give us shirts for all of our team, I'am very willing to send you a picture of our team wearing the dedup shirt.

Thanks and Best Regards,

tons

tomas_netherlands
7,709 Views

A customer called that he was running out of space on his Netapp. He had a fas2020c. Asked him if he was using De-duplication. He didn't know what is was exactly.

Heard if it, but did not used it.

So i moved all simular VM's to one volume and enabled SIS on the volume, started a de-duplication job. The next morning he called and was very suprised. Het got 100 GB of free space without buying storage. got 63% on 160 gb of space 😉

Tomas

lfreeman
7,296 Views

Hi Tomas, you might say that NetApp dedupe was just the medicine your customer needed. I love it when we make our customers feel better. "Just dedupe and call me in the morning", I'll have to remember that one. Oh and you've just won a t-shirt. Drop me a line larry.freeman@netapp.com with your size and mailing address and the Dr will do the rest-

DrDedupe

lfreeman
7,708 Views

Hi Antoni, thanks for sharing, and welcome to the "I saved over a TB with NetApp dedupe" club. But here's the line that caught my attention:

/vol/vol_bpdimage/ 475GB 449GB 49%

Are these image files? If so, what kind of images? We often hear from other dedupe vendors that images do not dedupe well, but I've seen otherwise in many cases.

As far as the shirts, they are as good as yours. Please send me a note larry.freeman@netapp.com and give all the shirt sizes you need and your mailing address - we look forward to posting a picture of your team on this community site.

DrDedupe

cebulrdcis
7,296 Views

the files in /vol/vol_bpdimage/ is not images, but a installers and drivers

Will wait for the shirts then...

mpitech_com
7,295 Views

We are using a FAS2020 with SAS disks as primary storage for our ESX cluster

The NearStore option was worth the small license fee.

Filesystem used saved %saved
/vol/vol1/ 69GB 89GB 56%
/vol/vol2/ 257GB 338GB 57%
/vol/vol3/ 95GB 207GB 68%
These volumes contains VMDK files for virtual machines. It splitted into 3 partitions due to the 500GB A-SIS volume limit on FAS2020

/vol/vol4/ 227GB 192GB 46%

This volume contains Lotus Notes databases in VMDK files and in NFS shares mounted from Linux based Notes servers


/vol/vol9/ 151GB 35GB 18%

ISO Images and other installation media


/vol/DC1_Swap/ 12GB 28GB 70%

This volume contains VMDK files with partitions for pagefiles


/vol/fs01/ 203GB 130GB 39%

Contains a volume shared via CIFS for Windows clients used for home and department shares

Total is around 1 TB saved or 50% saved due to dedup.

We are a team of 2 managing the complete infrastructure and the ease of use is just amazing.

The space saved is instantly available because we mount the ESX volumes via NFS.

We expected to migrate most of the physical servers to ESX and keep the file server as a physical server.

Deduplication made it possible to migrate our fileserver to the filer as well.

--

Brian Albertsen

coleman_greene
7,295 Views

I snapmirrored a 500GB volume used for NAS VMware images, set up the test copy for sis deduplication and am seeing an unbelieveable 76% data savings!.

df -s /vol/dedup_test
Filesystem                used      saved       %saved
/vol/dedup_test/      95321964  299076676          76%

We're so happy, we've quoted out and are ordering dedicated filers for our VM environments!

ayyappa_b
7,295 Views

Hi,

Recently i have done the de-duplication on FAS3070, and the space savings are amazing and al the volumes are more than 95% full now came down to less than 75% and on average we saved more than 23% space on filers and on the gr8 thing i am now tension free about the space issues.

Regards

Ayyappa

__PLURALNETAPP_19478
7,295 Views

Here is our DeDup savings.  All these vols are used for ESX via NFS.

/vol/haw_esx_vmlocal1/   15531852  161898592          91%
/vol/haw_esx_vmlocal2/   24805564  125892216          84%

/vol/nyc_esx_vmlocal1/   35803500  190186356          84%

/vol/nyc_esx_vmlocal2/   35825684   93779404          72%

we went from almost 2TB of allocated space to 600GB and have plenty of space to grow

semionkras
5,556 Views

/vol/root/                  1436336     105088              7%

/vol/WsData/            206676       257588             55%   -- .NET  Code

/vol/GRFSystem/      25674436   1748280            6%    --  Images

/vol/OraRacVol1/      14208620   2160960            13%  -- Oracle Database (for testing)

/vol/Software/           83636724   19807324          19%  -- Software

/vol/FaxIn/                616232      14756                2%   -- Faxes in gif format

/vol/ApollogVol/        11537072   4261712            27%  -- Oracle binaries

What surprised you most about NetApp deduplication?

No surprises all works as expected .

What applications are you deduplicating today?

See table.

How did deduplication change the way you think about storage utilization?

Great way to get some aditional free storage space .

Becouse of 2 TB boundary for dedup i need to reconsider all my volume sizing and break my big volume to some smaller pieces.

Cant wait to integrate ESX on NETAPP so i can see very nice numbers saved

What did deduplication allow you to do with your new-found storage capacity?

I saved some space and it gave me chance not to buy additional shielf

jb2cool01
5,556 Views

DrDedupe,

We have revently upgraded all of our systems to take advantage of de-dupe technology. We were using FAS940's and unfortunately these do not have the horsepower to de-dupe so the time came to have a hardware refresh and upgrade our systems.

We now have the following;

FAS3140 with 4 shelves of 300FC disk primary system (Not quite in place yet as we are still in the middle of the upgrade)

FAS3140 with 4 shelves of 300FC disk secondary system used as a snapmirror destination

FAS2050 with 20 x 1Tb internal disks as snapvault destination

I have set SIS running against a 1Tb snapvault destination volume and a 370Gb snapvault destination volume on our FAS2050. These CIFS volumes contain office documents, databases, pictures and standard stuff that users keep in their home drives we know there is a fair bit of file duplication and were eagre to see the dedupe results.

Results in so far are;

Filesystem                used      saved       %saved

/vol/sv_volshared/  1026389256  307972476          23%

/vol/sv_vol1/        396317464   63516644          14%

I still have a few more volumes to do on the FAS2050. Just have to wait for the final steps of this upgrade to go through so that i can use SIS on my primary system to reap the benefits there.

What surprised me was how simple it was to use. Just a case of turning it on and starting it (Or scheduling it). What am i de-duping? well just CIFS home drives and large file storage areas right now but we are about to start using VMware in anger and that's where de-dupe is really meant to shine.

Nothing like free storage.

**UPDATE**

Done a few more volumes and now looking like this

Filesystem                used      saved       %saved
/vol/sv_vlafiler1/  2537566552          0           0%
/vol/vol0/              608580          0           0%
/vol/sv_excharchive1/  476815288          0           0%
/vol/volarchive/     581475004  187957228          24%
/vol/sv_vol1/        398566316   63534176          14%
/vol/sv_vollabdata/  629191184  226191208          26%
/vol/sv_volbackup/   343354928   24972968           7%
/vol/sv_volrestricted/     225648          0           0%
/vol/sv_volshared/  1021872416  309931988          23%

Getting up towards the 1Tb mark and this is only on my SnapVault destination. Will have more savings when this is implemented on my primary and SnapMirror systems.

Also, i found out that a DeDupe scan will automatically run every time a SnapVault destination gets updated. That's great as there is less to set up.

jb2cool01
5,556 Views

I have dedupe running on all three of my Netapp devices not and here are my results;

Filesystemusedsaved%saved
/vol/vol1/3662119726637705215%
/vol/volbackup/1542352842464075614%
/vol/vollabdata/62229552823286806427%
/vol/volshared/120791965231902388021%
Filesystemusedsaved%saved
/vol/volbackup/1541528522464075614%
/vol/vol1/3660672326637832815%
/vol/vollabdata/62229527623286806427%
/vol/volshared/120770211631907104021%
Filesystemusedsaved%saved
/vol/volarchive/59274464420468053226%
/vol/sv_vollabdata/51821295618661252826%
/vol/sv_volbackup/1060699788171485842%
/vol/sv_volshared/126671123623235200815%
/vol/sv_vol1/3472362486616036816%

Average savings so far are 18% and i've saved almost 2Tb

Public