Jumbo Frames NFS/CIFS

[JEARLY_1616]

I am currently running a FAS3240 pair (8.0.2P6 7-Mode)  in a Flexpod setup (UCS B-Series, Nexus 7010).  The VM hosts are currently accessing the NetApp via 4G FC.  The NetApp is sharing most of the office's file shares via CIFS.  I have a pair of 10G interfaces in the filers, currently these are the interfaces running both NFS and CIFS traffic.  There is very little NFS traffic at the moment but I am planning to change that soon.  My question is that currently everything runs at 1500 MTU, if I increase the VM hosts/Nexus/NetApp to 9000 MTU will the Desktop machines accessing the NetApp via CIFS have a problem with the NetApp being configured for 9000??  I have read about option ip.path_mtu_discovery.enable but wanted to bounce this plan off the community to see if I was headed in the right direction or about to shoot myself in the foot.

Thanks,

John

[scottgelb]

With this many clients you can't modify, don't do it...Jumbo needs all devices at jumbo or it can cause issues.  I saw one host come in at 1500 on a jumbo network and it took down iscsi luns on one of the servers.  If you can control every device on the segment then worth looking at but otherwise I don't recommend it (after being bit when a customer enabled on the netapp side and thought all was jumbo on all devices).

[JEARLY_1616]

So you are saying that unless I can segment the CIFS shares for the users and the NFS Shares for the ESX hosts on seperate physical interfaces then I shouldn't enable Jumbo Frames at all. 

That being said, what if I move all of my CIFS shares onto a Windows box and only allow the users to get to those via Windows and not directly to the NetApp, or would the benefit of Jumbo Frames not be worth the work to put this in place??  If I don't enable Jumbo Frames, do you see any problems with switching from 4G FC LUNs to 10G 1500 MTU NFS Shares??

Thanks.

[scottgelb]

A VLAN is also an interface, so you could use the same physical interface but have one VLAN at 1500 and another VLAN at 9000 on the same physical port.  Unless you can guarantee all devices are 9000 on the segment (and some devices need larger if they don't include header/trailer but documented by the switch or nic vendor for that). 

At 10G Jumbo would be nice to have but might not be worth the hassle...for 1G I don't recommend it anymore since we didn't see a big return on it...but for 10G it can be a good boost depending on what you are doing...but if small random I/O it might be diminishing returns. 

If you can create a temporary VLAN and run it jumbo then do some testing compared to 1500 you can see or use SIO or iometer to simulate the workload.

[JEARLY_1616]

I thought Jumbo Frames were at the physical layer, not logical.  The 10G interfaces in my filers are broke out into 2 VLANs, one the users use for CIFS and one the hosts use for NFS.  The 2 10G interfaces are combined into a single IFGRP and that is broke out into 2 VLAN interfaces, so which interface on the filer should I enable Jumbo Frames, just the IFGRP-VLAN NFS interface??  Do I still enable it on the physical interfaces going into the filers on the Nexus 7K?

[scottgelb]

MTU can be set on either... 7-Mode on the ifconfig which can be ifgrp or ifgrp-vlan so you could seat the base ifgrp at 1500 and ifgrp-21 at 9000 or any mix of them.  cDOT is similar where the setting is in the network port modify command which can be ifgrp or ifgrp-vlan similar to ifconfig in 7-Mode.  Enable it on the specific interface (untagged ifgrp or tagged ifgrp) where you want it set, then match on the Nexus for that trunk.