Nexus 3000 and cDOT using LACP

JKeicher · ‎2014-11-04

I have two different cDOT clusters configured with LACP. One wired to Nexus 7000 switches and everything works fine. However, the cluster wired to the Nexus 3000 switches are experiencing strange behavior. NFS mount issues.

Nodes 3 and 4 were added to an existing cDOT. Nodes 1 and 2 do not use LACP.

From the host I try to mount two different volumes. Should be simple...right? Mount vol3 to /mnt/vol3 and vol4 to /mnt/vol4. However, since the vols are on different filers, we need to mount them from the source LIF.

filerc01-03:/vol/vol3 /mnt/vol3

filerc01-04:/vol/vol4 /mnt/vol4

I can mount /mnt/vol3 but when I try to mount /mnt/vol4 it will timeout and not mount. However, if I change the mount IP, to this it mounts just fine.

filerc01-03:/vol/vol4 /mnt/vol4

OK, so if I go back to try and mount it the correct way....

filerc01-04:/vol/vol4 /mnt/vol4

....while it is timing out...and trying again...I go to the Filer and migrate the LIF from na04 to NA03 it mounts. I can then revert all back to it's home position, then the mount will hang.

So, one would think this is a host issue. Not so, I have multiple servers that have the same problem. However, on some of these servers the mount issue is exactly the opposite.

/mnt/vol4 mounts just fine using filerc01-04:/vol/vol4 /mnt/vol4, and vol3 will not mount unless you use NA04's IP or migrate the LIF to NA04.

I have validated the ports on the switches are configured correctly, and validated the ports on the switches are cabled to the Filer correctly. The Filer filerc01-03 goes to switch one and both ports (e0d and e0f) go to ports 1/20 and 1/21. The Filer filerc01-04 goes to switch two and both ports (e0d and e0f) go to ports 1/20 and 1/21.

The host side

=========

# cat pprhel-upgrade

201401171418.x86_64

Red Hat Enterprise Linux Server

5.10.0.2

Server

(Tikanga)

http://bugzilla.redhat.com

# rpm -qa | grep nfs

nfs-utils-lib-1.0.8-7.9.el5

nfs-utils-1.0.9-70.el5