Need insight to pros and cons of running Diskkeeper 2010 on Virtual servers

[celliott3535]

The discussion of defragmenting VM's has come up at our organization.  We have licenses for Diskkeeper 2010 for our physical servers and after talking with Diskkeeper, they are claiming to be able to defrag our VM’s and in the process, improve performance.  We are very leery of these claims.  It has been my understanding that running any type of defragmentation tool on the VM will not be of any benefit and may in fact cause a performance issue.  It is also my understanding that running a client side defrag tool will cause the snapshots to increase in size.  Based on my understanding, the snapshots space requirements could actually double in size or more.  We have been using the reallocate command on our aggregates and it has made a big difference in decreasing latency.  We have not used any other filer commands on the aggregates.  Any insights would be greatly appreciated.

[datamanaged]

Hi Celliott3535,

I would say this is a *bad* idea if you're using a netapp filer as storage for your VMs. WAFL (the underlying Netapp Filesystem) does not arrange data sequentially on disks like most filesystems. It essentially writes all data to the end of the aggregate, including rewritten data, even after running the 'defrag' utility the data on the physical disks would still be "fragmented". You also have dedupe to consider, if you run a defrag on your VM disks, then all the data that diskkeeper moves around will become re-duplicated until the dedupe process runs (which will essentially undefragment the data).

If you're worried about fragmentation affecting performance, I wouldn't from a VM level. The simple fact of the matter is that the VM's Filesystem will 99.9999% of the time not correspond to the underlying disk arrangement, you'll simply be wasting I/O and CPU resources running diskkeeper.

If you're still worried about fragmentation you should take a look at the netapp reallocate command, and a few of the following links:

http://www.las-solanas.com/storage_virtualization/san_volume_defragmentation.php

http://www.storagemonkeys.com/index.php?option=com_community&view=groups&task=viewdiscussion&groupid=45&topicid=33&Itemid=138

https://communities.netapp.com/message/28153   (Andrew miller's response is much better than mine, lol)

[chriskranz]

I second this opinion. The read-ahead algorithms and techniques of NetApp WAFL make the benefit of filesystem defragmentation really minimal. What areas of the filesystem are you looking to benefit? If you're talking Exchange or SQL, then arguable the data wouldn't be in a VMDK, but a database defrag from the application may benefit as it also rebuilds the indexes. A filesystem defrag will have no benefit, and perhaps may further fragment a large database file.

Running a LUN reallocate directly on the NetApp may have some good benefits to performance however as this will optimise the data layout so that the read patterns can be more efficient and use more contiguous blocks for the corresponding LUN.

You'd need specific use cases to verify the benefit of doing any defragmentation. I've asked this question a couple of times of VMware professionals and always got a similar response to the above. Any sort of simple filesystem reallocation on the VM will have little actual benefit. In fact as it'll reallocate all the data, it'll probably have a detrimental effect on the NetApp storage system, the snapshots and any replication you have.

[danielmorgenstern]

Just chiming in since this has been a topic in our environment, too.  We avoid defrag of VMs and have standing requests to have it turned off via GPO for our Windows servers.

We have also heard claims (from defrag software vendors) of performance benefits and there is even a "whitepaper" or two floating around with such claims but I don't put much stock in them.

Our practical thinking is that we don't generally see the kinds of poor performance and degradation that call for having a "standard operating procedure" to do "preventative maintenance" such as defragging.  If *a* server is having performance problems, we look into it...configs, OS, apps, hardware, etc.  In my experience, there's never a magic bullet.

On the flipside, just to give the idea the benefit of a very large doubt...even if there was some performance bump from defragging VMs, is it worth a relatively large i/o spike on the storage system if something like a preventative defrag for all VMs was scheduled?  Doesn't make sense to me.

(The only time we run defrag is when creaing a new server  image from scratch that will be used to deploy multiple VMs.  After  cleanups and configuration, we run it on the base image before it is  templated.  We haven't done any measurements to see if this is actually  worth it but it's an image prep "best practice" that has been carried  along and gives various folks on our teams a "warm fuzzy feeling"   )

[celliott3535]

Just wanting to post a follow-up.  I have found all three posts to be very helpful in looking into this topic.  I have spoke with a couple of our vendors before and since this post and I am getting the same advice.   Unfortunately, I was really looking for a clear cut, black & white answer of either yes you definitely need to defrag or no don't defrag because it will cause a X% decrease in filer performance.  But as we often find in life, this appears to be a gray area instead.  It looks like our company may need to perform some actual testing to see if we can tell a difference.  If we move forward with the testing, I will post the results.