2009-08-24 02:38 PM
I've been trying to find any documentation here and on NOW stating that Oracle (NFS or FC) is supported with dedup. What kind of savings can be expected?
My understanding is that Oracle will not work well with dedup because of the unique page IDs that it uses internally. Can someone help me find the truth of the matter? I am not a DBA.
2009-08-24 09:34 PM
Dedupe is not dependant on FC or NFS or what have you as its done at WAFL layer hence you will be in a supported configuration
if turning it on. Deduplication by nature has got its limitations and some technologies dont lend themselves to being deduped
or gain a lot from it. structured data is one of these technologies I believe, same as sequential writes. Dedupe will rearrange block layout and thus
what used to be a sequential read could be spread out after a dedupe. Hence apps that needs sequential reads may not be suited for dedupe if they are
The good thing about NTAP dedupe is that its free, its low/no impact and it can be turned off/reverted. All this to tell you that I dont
think any of us can guess how much savings you will get but feel free to try it out.
Have you got a backup of your LUNs on netapp? if so you could clone the LUN and test dedupe on something different than prod. controllers.
Let us know your thougths.
Message was edited by: eric barlier
2009-08-25 06:38 AM
Hmm, not quite the answer I was looking for. Thanks though Eric.
Let me re-phrase...
I am unable to find any published results or recommendations regarding Oracle databases and NetApp dedupe. Has anyone tried this and if so, what results did you see? Were there any negative impacts?
We use NFS for all of our Oracle and Sybase databases.
2009-08-25 06:47 AM
Our performance engineering team has done some basic benchmarking with Oracle and De-Dupe in OLTP type workloads. I have asked the engineer who was responsible to comment here if he is able, so I won't steal his "thunder". To date, I don't believe the results have been published because they aren't really all that interesting. If the engineer doesn't respond, please let me know and I can discuss more offline.
2009-08-25 07:43 AM
I work in NetApp's workload performance engineering area and have tested Oracle 11g with and without dedupe using an industry standard OLTP benchmark. The database block size used for this testing was 8K (remember dedupe works on 4K blocks at the WAFL level). Here's is a summary:
1) Impact of dedup on transactional throughput: For both FCP and NFS attached storage, enabling dedupe had very little impact on OET (order entry transactions-per-minute) application performance. Over a 2-hour workload run the average OET rate was nearly identical (< 1% variation) for the baseline (no dedupe) versus dedupe enabled.
2) Impact of running dedupe processing on OLTP throughput: Executing the deduplication process during peak workload had a noticeable negative impact on transactional throughput. Definitely not a good practice to do the deduplication during peak workload ---- i.e. find a quiet window or run overnight as a batch process.
3) Impact of dedupe on table scan rates: For large table scans (200,000,000 rows in this test case), the additional indirection introduced by deduplication resulted in less efficient sequential I/O processing. Depending on several factors, the impact can range from 3-16% reduction in throughput.
4) Storage savings: The initial dedupe activation and deduplication process reduced actual storage used by 17% and 15% respectively for FCP and NFS attached storage. Over time some of this savings was lost due to workload execution causing fewer blocks to be duplicates.
I was surprised at the amount of storage savings. You are correct in that one would expect minimal (if any) block duplication because of the unique SCN header and somewhat unique footer (trailor) information stored in each database block. In this test case I believe most of the savings came from allocated but never used ("zeroed") database blocks. Also, since the db block is 8K, only the 1st 4K piece has the unique identifier info. The 2nd 4K piece has the trailor, but trailor content may not be completely unique across all database blocks. In a datawarehousing type database with 16K blocks size, the middle two 4K blocks could very well contain duplicate information (especially if the DBA set up the database to allow for lots of free space.
As usual the final answer is "it depends", but IMO a 15% storage savings isn't enough to justify using dedupe on an Oracle database.
Hope this information is helpful, Dean
2010-08-13 09:27 AM
This is excellent info for me. We recently had problems with a volume using dedup. The volume (720GB), contained a lun (700GB) that was ocfs mounted on two hosts in a rac cluster used for backups. The volume hit 100% full , the filer offlined the volume, and since ocfs was running on the hosts, the ocfs services caused both hosts to fence and reboot. Obviously this crashed the whole rac environment!! My understanding is that the dedupe cannot be turned off on a volume once it has been turned on. If this is true, how much space is requried in a volume to handle a 100% unqiue block level lun. In my case abve, the 700GB lun contains ZERO ducplicate blocks. If the volume has dedupe on, how big should the volume be? lun size + dedupe overhead = volume size.
2010-08-15 03:14 PM
You mentioned: "My understanding is that the dedupe cannot be turned off on a volume once it has been turned on" > dedupe can indeed be turned off at any time.
you need to make sure you have enough space in the aggregate to be able to cater for the saved blocks though. So if your dedupe savings are 100GB then you
need at least 100GB free to be able to undedupe.
the problem where your volume went offline is likely not linked to dedupe at all. Its probably linked to snapshots being enabled on the volume
whilst the volume is only 20GB bigger than the LUN. NOTE: this is just a guess of course.
If there is a 30GB change inside the LUN at the next snapshot your volume will be full and it ll go offline. to get around this you need to
do an assessment of your rate of change and configure a few options.
In my environment we have turned snapshot reserve off (hands 20% space to volume), we ve got snap autodelete on and we allow more
space for snapshots.