Performance windows 2012 SLOW, why?

[THEFONS01]

We recently made a Windows 2012 server VM on our ESXi5.1 Cluster, with FC Datastores on FAS3210 with BSAS 1.6 TB disks 7200rpm (our SLOW disks). and it performs very bad

To resolve this problem we had te relocate the VM to the other aggr with SAS 500GB 10000rpf (out FAST disks).

Alle other VM's running on the SLOW disks have no performance problems, only windows 2012. The other VM's consist of mostly windows 2008R2 servers.

The creation of the Datastore on boths filers (SLOW and FAST) was identical and like this:

created a flexvol on the aggr of 525GB

created a lun of 500GB on that flexvol and assigned the initiater group from the VMware ESXi5.1 cluster where the new windows 2012 server would run.

we used thin provisioning where possible

on the VCentre server we had the clusterservers do a rescan for new devices, created a new datastore on the discoverd device and let VMware format the disks as VMFS5.54

then we created a brand new windows 2012 VM on that datastore, using a 40GB c: and 400Gb d: drive. again with thin provisioning

the installation had no issies.

But logging in on the console and just using windows was verry irritating.

after installing Sharepoint 2013 thing were even worse, resulting in typing en watching the letter appear on by one in windows.

To see where the sluggishness came from we tried several scenarios, and only vmotioning the server to a identical volume on the FAST disk resulted in a GOOD performing server.

The windows 2012 server remaind the same: the VMware servers did not change, the same initiators were used. Only the DATASTORE was connected to FAST disk instead of the SLOW disk.

doing the same with windows 2008R2, with identical hardware assignments and the same SLOW and FAST disk assignments in this test gave a GOOD performing server on BOTH SLOW and FAST disk

de filers are like this:

Filer 1 and filer 2 in HA mode

NetApp Release 8.1.1 7-Mode

Filer 1 has 24 disks SAS 546.88GB 10000RPM

Aggr is a raid_dp with one Spare disk

Filer 2 has 12 disks BSAS 1,62TB 7200RPM

Aggr is a raid_dp with one Spare disk

Has anyone seen the same slow performance on BSAS disk for windows 2012?

I have searched on google and havent found any conclusive articles

Any thoughts? I do not want my test servers using FAST disks...

regards Fons

[THEFONS01]

Are we really the only ones with this problem?

That would mean we did something wrong in our configuration?

Again: could you please share with me if you are running windows 2012 on ESXi5.1 with Netapp filers and BSAS disks?

If so, and it is running performant, what settings dit you use?

I would greatly appreciate any feedback

thx in advance

Fons

[keitha]

Sorry, I haven't seen any issues like this at any of my clients. I'm also not sure what you mean by BSAS disks? SATA? 1.5 TB 7200 RPM right?

[THEFONS01]

Hey Keitha,

thnx for the responce,

BSAS disks are SATA disks with added hardware (the bridge) that allows them to appear as SAS drives to the disk shelf

BSAS disk are considdered to be the same as SATA disks when mixing disk in a filer. (we didnot mix them, on filer has SAS drives, the other has BSAS disks)

Are you running windows 2012 server as a VM on SATA disks?

grtz Fons

[keitha]

Gotcha...I'm a NetApp Employee (CSE) so I don't have any VMs of my own but do have clients running 2012. I'm just not sure of the disk type. The Disk type should not however matter as long as it can keep up with the IO demands on it. I would be surprised if a single 2012 server not running anything was maxing out a SATA Aggr. If it did, everything on the aggr should be slow which you did not indicate was the case.

I'm not sure where to start with this though. VMware tools?

Do you know the average latency the 2 different Aggrs are providing?

Keith

[THEFONS01]

We performed some performance and IO tests on two excactly the same configurations: one windows 2008 R2 latest SP's, one with windows 2012.

VMware is on ESXi 5.1.0 build 799733, vm-hardware both on vmx-09 and the latest VMtools running (Current)

When both were given their drives form the FAST disks (a VMware thin provisioned Datastore) there was no real performance differance.

When both were given their drives form the SLOW disks (also a thin provisioned VMware datastore) windows 2012 was sometimes almost unresponsive, the w2k8 server ran witrh no probs.

Both test were performed on the same VMware servers.

We noticed that the IO tests were NOT consistant: some were slow, some were faster but stil slower than the w2k8 server.

the average latency on both aggrs were low.

I'll come back with some numbers if you like?

grtz Fons

[rb1kenobi]

Hi Fons, just thought I'd add my two cents here:

I am NOT using Netapp - but a competitors gear which would be a little faster than your "SLOW" disks, but not as fast as your "FAST" disks.

It's a Nexsan NST5100 hybrid with SSD  caching drives fronting a 8 X 1TB 7200 SATA array. 1Gb iSCSI.

I am however seeing similar issues as you describe.

VMWARE ESXi  5.1.0 build 799733 on HP DL380p Gen8  (using HP custom iso image to ensure drivers are appropriate)

Windows 2012 boxes are roughly 30-50% slower than "Vmware Converted" 2003 boxes running similar applications.

I haven't had the luxury of creating a physical windows 2012 box to compare to the virtual, but its either something with the way VMware and 2012 are talking, or something in 2012 itself.

If you stumble across a solution, I'd appreciate hearing about it.

Regards.

[WILLIAM_LORENZO]

Fons,

We had similar issues in our environment running all 15k SAS shelves. We had to make a few tweaks that helped a little but not a lot thus far. Some things I found were the vmxnet3 adapter performs horribly on server 2012 still; We are 100% patched up to the latest 5.1 code and still saw a minor difference going to E1000E adapters. Another fix we found to help with display sluggishness was with the WDDM driver. This was supposed to be fixed in 5.1 and it is to an extent but not fully functional. Check out http://ryanmangansitblog.wordpress.com/2012/09/17/vmware-fix-for-server-2012-virtual-machines-running-on-esxi-5-0/ . I know this is reference to 5.0u1 but it still applied to an extent for us.

That's all we have found thus far. From what I can tell running performance stats the NetApp on the back-end is not the limiter it still seems to be VMware getting the code base optimized. Sorry I couldn't be of more help but rest assured you aren't the only one with issues in a VMware/NetApp/2012 environment.

Bill

[THEFONS01]

Hy Bill,

thanks for the reply, i checked out the link and your tips.

Alas.the WDDM driver is not installed, we use the Vesa driver (default), and we are using the E1000E NIC

But stil: any help is appreciated, and knowing we're not alone out there helps

grtz Fons

[bailibr]

Any updates on your issue? We are having a similar issue with vmware 5.1 and Windows 2012. Its only pushing about 60MB in the network

[ralfaro01zero]

Have you disabled the Disk Defrag. service on Win2012? No need to defrag on a virtual environment, it is a waste of resources.

Cheers.

[THEFONS01]

Hey Ricardo

Checked the use of defrag: is was disabled already, thnx for your tip

grtz Fons