Subscribe

Defragment VMs, should it be done?

I am aware of at least two disk defragmenting solutions aimed directly at VMs (Diskeeper and Perfectdisk). Both claim increased IO and reduced latency for VMs that are defragmented. Has anyone tried these products? Is it important to defrag a VMs drives? Does running defrag on a VM affect its alignment? All of my VMs are aligned and I certainly do not want to change this.

Re: Defragment VMs, should it be done?

Hi,

The short answer is: no.

Plenty of detailed info could be found here: https://communities.netapp.com/message/27821.

This quote is probably key:

"Virtual machines stored on NetApp storage arrays should not use disk defragmentation utilities as the WAFL file system is designed to optimally place and access data at level below the GOS file system. Should you be advised by a software vendor to run disk defragmentation utilities inside of a VM, please contact the NetApp Global Support Center prior to initiating this activity."

Regards,

Radek

Re: Defragment VMs, should it be done?

I have seen V-Locity (Diskeeper's VM Optimizing Product) make a very big difference on VMs.

It is true that you should not run a traditional defrag program on a VM, as it is not designed to address the virtual environment.

There's a lot of information on this.

Here's an Osterman Research White Paper on The Importance of Defragmentation in Virtualized Envelironments:

http://downloads.diskeeper.com/pdf/The_Importance_of_Defragmentation_in_Virtualized_Enviroments.pdf

And here's another White Paper by Topix Media on Best Practices for Optimising Virtual Disks: http://downloads.diskeeper.com/pdf/Topix_WP_en_DOLLAR.pdf

And here's a bunch of testimonials from IT managers running VMs of various sizes and configurations at their sites who have also actually used V-Locity:

http://downloads.diskeeper.com/pdf/V-locity_SuccessStories.pdf

So the more accurate answer would be Yes, one should defragment VMs, but only if one uses the right tool to get it done properly.

Check out these white papers and see what you think.

Re: Defragment VMs, should it be done?

I have seen V-Locity (Diskeeper's VM Optimizing Product) make a very big difference on VMs.

It is true that you should not run a traditional defrag program on a VM, as it is not designed to address the virtual environment.

Okay, this is not the point whether a defrag program understands VMs - the point is there is no defrag program (yet) which understands NetApp WAFL file system & that is also true about Diskeeper.

The thread I am referring to in my previous post gives more details why running OS/VM level defrag on NetApp is a bad idea.

In particular this is NetApp official best practices document:

http://www.netapp.com/us/library/technical-reports/tr-3749.html

You will find the relevant paragraph on page 78.

Regards,
Radek

Re: Defragment VMs, should it be done?

Radek is 100% correct here! Regardless of the tool a defrag of the VMs will result in temporary loss of dedupe on the NetApp, huge snapshots and likely a decrease in vm performance.

Keith

Re: Defragment VMs, should it be done?

Running a defragmentation job on a production server would be tantamount to an act of suicide.

And, leaving aside the fact that defragmenting a logical volume on WAFL is a waste of time, I'm not convinced that file fragmentation is the issue that it once was. Fragmentation isn't an issue on SSDs (there is no distinction in average access times between random or sequential I/O) and even on traditional disks, with all the clever caching that modern OS's do, any frequently-accessed parts of a volume are going to be kept in the OS page cache anyway - provided there is sufficient RAM - irrespective of whether they are fragmented or not. RAM is cheap these days so it's a lot easier to add more rather than take massive risks with data.

The whole notion of defragmentation strikes me as a bit like colonic irrigation. The concept makes sense in theory but in practice there is little or no justification for it.

The claims that VMs in particular "need" defragmentation are very suspicious, and the erroneous statement "VMs usually run multiple OS's simultaneously" suggests that we are benefitting from the contributions of an individual who lacks technical background.

Re: Defragment VMs, should it be done?

The whole notion of defragmentation strikes me as a bit like colonic irrigation. The concept makes sense in theory but in practice there is little or no justification for it.

It starts to get interesting . Whilst we both agree that using VM/OS level defrag tools doesn't make sense at all, fragmentation as such on a NetApp system can impact the performance & can be rectified, albeit using storage-level tools (namely reallocate). There is a lot of discussions around that on Communities with the most notable thread: http://communities.netapp.com/thread/6530.

As for the practical impact of reallocation, this is a nice example: http://communities.netapp.com/message/26889#26889.

Regards,
Radek

Re: Defragment VMs, should it be done?

Radek,

I read through both of the links you provided about Netapp's reallocate. It was good reading but generated more questions for me than answers:

- What volumes should I run reallocate on? I have an NFS volume that holds my VMs, I have NFS volumes that hold my SQL databases and log files, I have LUNs for my Exchange database and logs. Are they all candidates?

- What if the volume is deduped already, should reallocate be run? What is the affect?

I agree with the sentiments of people in both threads that it would be nice to have a TR on this subject. Is there any documentation or white papers on reallocate?

Thanks to everyone who responding to my post, I certainly will not be using any type of defragment utility on my VMs but my curiosity is peaked about possible performance improvements with reallocate.

Dave

Re: Defragment VMs, should it be done?

Hi Dave,

Reallocation seems to be a prime example of a dark art in the NetApp world, indeed!

- What volumes should I run reallocate on? I have an NFS volume that holds my VMs, I have NFS volumes that hold my SQL databases and log files, I have LUNs for my Exchange database and logs. Are they all candidates?

The only answer can be - it depends! The old rule applies here: if it ain't broke, don't fix it. Is there a performance problem? Can it be related to fragmentation? The common suspects are slow sequential reads, as this is when fragmentation bites. Reallocate can be first run to measure the level of fragmentation, then the actual reallocation can be done if really needed.

- What if the volume is deduped already, should reallocate be run? What is the affect?

We have a neighbouring, fresh thread (http://communities.netapp.com/message/51653#51653) with the firm answer - no. Just common sense says if a block is shared between multiple files it is really impossible to tell where is its optimal location (you can't please them all )

Regards,
Radek

Re: Defragment VMs, should it be done?

I have 4 NFS volumes for my SQL Server. Below are these volumes along with information I gathered after running "reallocate measure /vol/volname" on each one

SQLSystemDB, Optimization: 4 [hot-spots: 44]

SQLUserDB, Optimization: 4

SQLLogs, Optimization: 7

SQLSnapInfo, Optimization: 1

Four quick questions:

1. It appears that I should run a reallocate on three of these volumes since their optimization level is 4 or above, is this correct?

2. I am going to run "reallocate start -f -p /vol/volname" to actually carry out the reallocation. Is this the correct command? It appears to be after reading the man pages

3. Should I run this off hours?

4. And the most important question - this is a production SQL 2008 R2 server, is this safe to run? The first rule of medicine and IT, do no harm!

Dave