Just chiming in since this has been a topic in our environment, too. We avoid defrag of VMs and have standing requests to have it turned off via GPO for our Windows servers.
We have also heard claims (from defrag software vendors) of performance benefits and there is even a "whitepaper" or two floating around with such claims but I don't put much stock in them.
Our practical thinking is that we don't generally see the kinds of poor performance and degradation that call for having a "standard operating procedure" to do "preventative maintenance" such as defragging. If *a* server is having performance problems, we look into it...configs, OS, apps, hardware, etc. In my experience, there's never a magic bullet.
On the flipside, just to give the idea the benefit of a very large doubt...even if there was some performance bump from defragging VMs, is it worth a relatively large i/o spike on the storage system if something like a preventative defrag for all VMs was scheduled? Doesn't make sense to me.
(The only time we run defrag is when creaing a new server image from scratch that will be used to deploy multiple VMs. After cleanups and configuration, we run it on the base image before it is templated. We haven't done any measurements to see if this is actually worth it but it's an image prep "best practice" that has been carried along and gives various folks on our teams a "warm fuzzy feeling" )