Hi Dave,
I'm the Director of Product Management at Diskeeper Corporation, the vendor that makes the V-locity product for VMware. I hope what I’m writing is helpful and does not come across as a sales pitch, that is not my intent. I feel it’s important to substantiate statements (for or against) with data.
As you noted the purpose of defrag is to increase IO throughput / reduce latency – at the GOS. Windows-based file optimization tools are not going to affect physical placement in a NetApp FAS. So, the comment in the NetApp paper is correct in this regard. However, it does not speak to the fact that the IO overhead in Windows is negatively impacted. Defragmentation does not alter alignment whatsoever, but the unwanted effect of misaligned partitions is the same as from NTFS fragmentation – “…results in a dramatic increase of I/O load on a storage array and negatively affects the performance of all virtual machines being served on the array”; pulled from page 79 of the NetApp Best Practices paper referenced earlier in this thread.
We recently completed tests on a NetApp FAS3140 at a Microsoft lab in Washington. Needless to say we didn’t use ESX, but the results would be similar. Testing included Jetstress, Iometer, SQLIO and SQLIOsim, as well as Perfmon to collect additional data. Specifically, benchmarks were run on fragmented test files (starting at around 60,000 fragments and up) that are used by those apps (e.g. iobw.tst, Jetstress*.edb, Sqliosim.mdx, etc…) and non-fragmented test files. In every case, fragmentation in the GOS resulted in more IOs and latencies for those benchmark apps. Defragmented files resulted in less total IO required and faster response times. As one example with SQLIO, 8KB Random Writes IOs/Sec improved from 486.15 to 1274.78 with the Max_Latency(ms) dropping from 4490 to 161.
Perfmon also indicated IO issues (the Perfmon screenshot in this post is from those FAS tests: http://www.diskeeper.com/blog/post/2011/03/29/Best-Practices-SAN-defrag.aspx). The split I/Os reported in Perfmon are indicative of fragmentation and/or misaligned partitions.
We ran the reallocate command as part of the tests, which also helped in most cases.
I understand the “do no harm” viewpoint for any evaluation, and there are caveats; which is why we have special technology for VMs/SANs. If you can test on non-production LUNs, that’s a great start. Perhaps the most readily available option is to monitor Perfmon stats in the VMs for the presence of potential logical disk bottlenecks. Ultimately theory always has to be validated with scientific process, and we have support engineers, methodology, and software tools available to assist anyone with testing.
As to the comment on caching, yes if you can cache everything you don’t have to be as involved in disk bottlenecks, but where it is cached is relevant. If you can cache in Windows, you can leverage Fast IO, and don’t even need to generate IRPs to pass into the storage subsystem. I've also tested on many SSDs (http://www.diskeeper.com/blog/post/2010/04/02/Do-you-still-need-HyperFast-if-you-have-TRIM.aspx), which mitigate the impact of fragmentation on reads (Intel X25-E being the best I tested – had to get to 200k free space fragments before write performance dropped), but can still be impeded. Consolidating free space, not file fragments, is the principal need on SSDs. That said, better controllers and more channels (and PCM memory in the future) continue to improve SSD overall performance, so it's reasonable to expect fragmentation to be a relatively moot issue in the future.
Michael Materie
Diskeeper Corporation
