2010-04-23 02:00 PM
Is it advisable to run Windows OS level defrags on vmdks within a FC LUN? In researching the problem, I have found a mixed bag of answers to the question - some examples here:
"I spoke to one of the vmw. designers/architectures here, he said yes"
"ABSOLUTELY, DO NOT RUN DEGRAG ON AN iSCSI or FIBRE CHANNEL LUN. ESPECIALLY IF YOU ARE TAKING SNAPSHOTS"
"He was told by a VMWare engineer that if you run Windows defragmenation on the virtual server while also are using de-duplication on the storage, it can cause corruption of data on the virtual server and/or the VMDK"
Can anyone provide some insight to the risks/benefits of OS level defrag vs. reallocation vs. other ?
Can anybody point me to any relevant documentation on this ? Official NetApp position?
If you have virtualized any servers in your shop, are your sysadmins performing OS level defrags on their mapped 'drives' ? Is the practice useful, useless or dangerous?
Single FC LUN containing multiple VMDKs
Deduplication turned on
Snapshots (via script)
VM's w/ Windows2003 OS
(VMWare ESX 4.0 VSphere)
Solved! SEE THE SOLUTION
2010-04-23 10:26 PM
Grr....I had a super long response worked up and the Communities timed out my session (I'd been active just 5 minutes ago so not sure what happened).
In short, you don't want to do this....has to do with temporal locality....here's an explanation I sent to a customer recently.
2010-04-26 09:09 AM
According to TR-3749 (jan 2010), Netapp and vSphere storage best practices:
“DISK DEFRAGMENTATION UTILITIES
Virtual machines stored on NetApp storage arrays should not use disk defragmentation utilities as
the WAFL file system is designed to optimally place and access data at level below the GOS file
system. Should you be advised by a software vendor to run disk defragmentation utilities inside of
a VM, please contact the NetApp Global Support Center prior to initiating this activity.
I would tend to think this applies to non - VMs on Netapp storage as well.
We have an ongoing debate in our office about if we should do this or not. Seems like anyone you talk to will give you a different answer. I argue NO, you should not. A few of the cons:
The question is, is there a measurable overhead within the Windows guest that would be reduced by defrag, and does the benefit outweigh the chaos on the storage side caused by deffraging?
2010-04-29 06:29 PM
Yes....great data there. I've seen reallocate really help as well (the most dramatic example was a bunch of Groupwise servers on FC disk where reallocate cut latencies more than in half). I'm still wrestling between when it makes sense to use "reallocate" vs. "reallocate -p" (if there are any difference in how long it takes to run, how it impacts speed, exactly how much -p helps with snapshot deltas, etc.).
2010-04-29 09:59 PM
exactly how much -p helps with snapshot deltas
That I can answer reallocate -p has zero impact on volume snapshot size.
I'd add to your list interaction with VSM. Data transferred after reallocate will be defragmented; but after reallocate -p - not (at least if I correctly understand how it works). This may need to be taken in account if destination is often used for tasks like backup verification.
2012-04-12 07:57 AM
I'm not a storage expert. I just maintain a set of virtualized sql servers using Netapp luns (iscsi/snapdrive/snapmanager;...).
I noticed that data updates are written to free blocks, meaning the original block is not updated, but kept, since referenced by snapshots earlier made. So, may I conclude fragmentation is inherent to Netapp? May I conclude windows defrag might cause volumes running out of space? May I conclude that (in case we would have enough free space in the volume) the chance that less physical IO is initiated after defrag is negligible or even that in some cases the number of physical IO's might increase? May I conclude Windows will initiate less IO's since it thinks data is sequentialized, but the consequential number of IO's on Netapp is unpredicatable? May I conclude that the sql command "set statistics io on" does not tell me the truth about the number of physical reads executed on Netapp (or any other disk virtualisation/SAN system), only the number of physical IO windows or SQL thinks that have to be done?
Anyway I wonder what defrag means in RAID setup. Isn't it so that 1 byte might be spread over 8 physical disks ... or that 8 bytes might be spread over 8 physical disk?
When I read this, I start to wonder whether sql server index rebuilds might no longer be best practice, since this will have the same effect on snapshots as windows defrag? May I conclude we benefit HA, DR and fast restore, but that we should review best practices regarding IO optimisation?