vPC and LACP configuration

[cpuetzch2m]

Having some problems with our environment and trying to rule out configuration issues with our Netapp. I read some older documentation or blog postings and after looking at my rc file I want to confirm that indeed this is configured correctly. The reason I am asking is to make sure our network infrastructure is setup correctly as well. We got v3140 Netapp connected to the network thru Cisco Nexus 2000 switches. We use port channels/LACP, but ran into issues when moving ports between physical switches. The networking group had problems bringing the port channels back up saying their switches would need to see LACP traffic first. However, with the Netapp in failover state there would be no LACP traffic coming naturally until a giveback operation would be initiated. However, that felt like flying blind a bit not knowing if those ports would actually come up. Here is what I have in my rc file:

hostname FILER-NET-1

#

ifconfig e1a flowcontrol send

ifconfig e1b flowcontrol send

#

ifgrp create lacp VIF_10g -b ip e1a e1b

#

vlan create VIF_10g 173 174 175 176 111

#

ifconfig VIF_10g-173 `hostname`-VIF_10g-173 netmask 255.255.255.0 partner VIF_10g-173 mtusize 9000 trusted -wins

ifconfig VIF_10g-174 `hostname`-VIF_10g-174 netmask 255.255.255.0 partner VIF_10g-174 mtusize 9000 trusted -wins

ifconfig VIF_10g-175 `hostname`-VIF_10g-175 netmask 255.255.255.0 partner VIF_10g-175 mtusize 9000 trusted -wins

ifconfig VIF_10g-176 `hostname`-VIF_10g-176 netmask 255.255.255.0 partner VIF_10g-176 mtusize 9000 trusted -wins

ifconfig VIF_10g-101 `hostname`-VIF_10g-111 netmask 255.255.255.0 partner VIF_10g-111 mtusize 9000 trusted -wins

#

route add default 10.10.200.254 1

routed on

#

ifconfig e0a `hostname`-e0a mediatype auto flowcontrol full netmask 255.255.255.0 partner e0a -wins

#

options dns.domainname example.com

options dns.enable on

options nis.enable off

#

savecore

From my perspective I see LACP configured, but are the flow control settings correct? The switch is supposed to support jumbo frames

[aborzenkov]

You interface group is set to LACP. It is impossible to answer whether "settings are correct" without knowing the whole environment; in particular, Jumbo frames have to enabled on switches and communication partners as well.

There are contradictory theories about flow control. I did see some strange phenomenon which (in hindsight) could be explained by lack of flow control - disruption of NFS over UDP. Problem went away after we switched to TCP ... which incidentally has own flow control and reliable delivery. So I would enable flow control (again do not forget to enable it on switches too ...) until some evidence that it is causing issues.

I do not understand what do you mean under "moving ports between physical switches". Could you elaborate?

[cpuetzch2m]

We were told that the ports on the Nexus 2Ks would not be "as equal" as the ports on the 5Ks and so it was decided to move the Netapp ports to the 5Ks accordingly.

Jumbo frames are configured as a global setting on the physical switches (according to my networking group). They are also configured in VMware (which is housed on this particular environment) and vmkping's with the appropriate frame size are passing through fine.

Unfortunately I do not have access to the port configuration on the Nexus switches.

[radek_kubka]

Hi and welcome to the Community!

Be careful with Nexus 2000, as these are not switches as such, but fabric extenders, i.e. remote line cards connected to another Nexus switches.

There is a number of gotchas around this setup & e.g. until the very latest versions of firmware so called 'dual homing' was not supported (i.e. connecting single N2k to two N5k).

Regards,

Radek

[cpuetzch2m]

Actually got the configuration for the port channel on the 5Ks now.

interface port-channel200

  description NetApp

  switchport mode trunk

  vpc 200

  switchport trunk allowed vlan 111,173-176

  flowcontrol receive on

  flowcontrol send off

interface port-channel201

  description NetApp

  switchport mode trunk

  vpc 201

  switchport trunk allowed vlan 111,173-176

  spanning-tree port type edge trunk

  flowcontrol receive on

Now in the Netapp configuration I posted I have this:

ifconfig e1a flowcontrol send

ifconfig e1b flowcontrol send

But the Nexus 5K shows flowcontrol send off. I am not an expert on the Nexus side of things, but is that combination correct?

[radek_kubka]

Using just common sense, there is no conflict:

On N5k you have flowcontrol send off, but flowcontrol receive is on - so it matches NetApp setting flowcontrol send (in other words, flow control is on when NetApp is sending & 5k receiving, not the other way around)

It is a bit hairy subject with very sparse documentation, but you can have a look at this TR, page 27:

http://media.netapp.com/documents/tr-3749.pdf

[jake_struve]

So you posted the config for the portchannels but what about the config for each port.  You need to tell the port it belongs to that channel and what mode to work on.

For a port that belongs to portchannel 200 you should see a command that looks like:

channel-group 200 mode active (active enable LACP, ON is simple ether channel)