vSphere ESXi5 and NetApp NFS datastore best practices

[adnorton1255]

I just got a couple of new NetApp 3270 filers running 8.1RC2 and I am building a new vSphere 5 environment but I want to make sure I am following all the best practices and best configuration for this setup. The Filers are also running some LUN's for my Exchange environment (will be) and our CIFS share. We are using 10GBe connections to and from the filers

I have read through the guide somewhat and some things just don't add up or confuse me a bit.

We have the X1140A cards so according to the guide these cards are Full Flow Control but should I still set my network flow control to off even if these cards are set to full?

Should we enable jumbo frames on everything? I had a technician say it wasn't necessary so I want to verify.

Any other information people can provide on setting this up to optimize my vSphere environment including SDRS and using snapdrive in a VM would be greatly appreciated.

Thanks

[AGUMADAVALLI]

Hi there,

use the below link for implementing NFS with vmware.

http://media.netapp.com/documents/tr-3749.pdf

Thank you,

AK G

[arunchak]

HI,

Also check this KB:

https://kb.netapp.com/support/index?page=content&id=1013275

It talks about the best practices for vSphere5 with NetApp.

Hope this is useful.

-Arun

[adnorton1255]

Thanks for the links gang. Read through the document already. I was curious about the X1140A card the document says it is always full Flow Control so do I set my servers and network ports to full or to off per the recommendation?

Thanks

[BrettEdelen]

leave flow control off for the 10GB stuff.

Not only is it in the best practices guides, but we had it on, and ran into pretty bad latency issues.

[WillFulmer]

On the X1140 cards, you cannot turn Flow Control off.

Unable to configure flow control on CNA card X1140A: ifconfig: (e4a) : SIOCSFLOWCONTROL - Operation not supported

https://kb.netapp.com/support/index?page=content&id=2016978

In this case, what is the configuration on the physical switchport?

[LUBEL_ANDY]

Did you ever figure out how best to achieve this?  I apparently have issues, NetApp advised me to disable flow control however also have the 1140A which it cannot be disabled on.

[ericgolan]

cDot

net port modify -node <nodename> -port e* -flowcontrol-admin none

can use * at the node to run all all nodes

also you can run

net port show -fields flowcontrol-admin      <--- this will show status on all ports in the cluster

7mode

ifconfig e0a flowcontrol send

if i am not mistaken

can put in the /etc/rc at the end of each ifconfig

if you use ifgrp or vif make sure you run the command in seperate lines in a make sense order

ea

ifconfig e0a flowcontrol send

vif create vif1 e0a e0b

[LUBEL_ANDY]

flow control cant be modified at all for these CNA cards.  ugh.. just had this make all my vm's read only.

Shutting down idle connection to client (xx.xx.xx.xx:833), where receive side flow control has been enabled. There are 62100 bytes in the receive buffer. This socket is being closed from the deferred queue.

just trying to set it as you stated also is "not supported"

toaster-7-mode> ifconfig e5a flowcontrol send

ifconfig: (e5a) : SIOCSFLOWCONTROL - Operation not supported

[parityerror]

We face the same issue with these CNA cards (unable to modify flow control). The way it was explained to me by NetApp was that the X1140 cards don't have enough onboard memory to support appropriate buffering without flow control.

Currently we have it working nicely with Flow Control full on the 7-mode NetApp filers, and flow control off for the switch ports. Not entirely sure how this works so well (given that the switches shouldn't be expecting flow control messages) but there you go, somehow it works.

It is very strange that when troubleshooting some earlier faults, NetApp seemed adamant that we had incorrectly configured flow control (i.e. they instructed us to switch it off), and they were quite surprised when it wouldn't allow us to disable flow control.

If it continues to be a problem you can speak to NetApp rep and try to arrange some different CNA cards which do support disabling flow control (might require some commercial discussions too) - we had to do this for our Flexpod environment (C-DOT FAS3240s connected to Nexus 5Ks) as we couldn't get it running in a stable manner with flow control enabled on those cards.

It has been helpful hearing from others fcing these challenges too - nice to know we aren't alone