I was trying to configure a FAS2040 as per the best practices outlined in http://media.netapp.com/documents/tr-3749.pdf. It mentions the use of a VIF with all 4 available NICs using LACP, VLAN tagging, enabling jumo frames and setting flow control to Send.
All seemed fine until I set flowcontrol to Send, after which I completely lost communications with the filer. The switch was configured correctly and listed the interfaces as up but I couldn't connect to or ping any of the IPs.
Here's the jist of the config (names and IPs are different, just keeping it simple)
vif create lacp vif0a -b ip e0a e0b e0c e0d
vlan create vif0a 101 102 103
ifconfig vif0a-101 10.0.1.1 netmask 255.255.255.0
ifconfig vif0a-102 10.0.2.1 netmask 255.255.255.0
ifconfig vif0a-103 10.0.3.1 netmask 255.255.255.0
ifconfig vif0a-102 mtusize 9000
ifconfig e0a flowcontrol send
ifconfig e0b flowcontrol send
ifconfig e0c flowcontrol send
ifconfig e0d flowcontrol send
After the restart, the filer was completely unreachable. When I removed the flowcontrol send from each interface connectivity was restored.
Am I trying to do something that is not supported or are the problems likely to be elsewhere?
i had a similiar event, and we run in a problem between netapp and cisco. Thats the handshake over the flowcontrol take to long, so the cisco take the ports down and the vif dont work anymore.
I cant find the KB entry now, but we changed the flowcontrol for these Filers on receive had we had zero problems from this time. Normal i let the Fullcontrol on full only with these Problems i changed it to receive.
It seems my issue was more to do with the VIF and VLAN configuration. When I set each interface to Send it works fine. When I rebooted, thats when I lost the communication.
I've logged a call with support who asked whether I'd updated the /etc/rc file, which I hadn't. I've never configured a VLAN before which is why I've never come across it. I've set up VIFs before but through the setup command (which I've now discovered) adds the entris to the /etc/rc file for you.
So, I'm now struggling to get my head around the the /etc/rc configuration. I've manually configured everything, got it working, added what I assumed was the right config into the /etc/rc file, rebooted and have a perfectly useless NetApp filer - completely inaccessible.
Can someone give me a little help with it, it is *hugely* frustrating. I'm not a CLI guy at all.
For a start, I can't edit the file from the CIFS share so am resorting to the wrfile command through putty which is even more annoying.
At the moment I'm dumbing things down a little and avoiding the use of VLANs but I still can't get it right:
#Auto-generated by setup Tue May 3 15:31:57 GMT 2011 hostname isnldnn07 vif create vif0a e0b e0c e0d ifconfig e0b `hostname`-e0a 10.254.114.7 netmask 255.255.0.0 mtusize 1500 trusted -wins up ifconfig e0c `hostname`-e0b mediatype auto flowcontrol send ifconfig e0d `hostname`-e0c mediatype auto flowcontrol send ifconfig e0e `hostname`-e0d mediatype auto flowcontrol send ifconfig vif0a `hostname`-vif0a 10.254.114.8 netmask 255.255.0.0 mtusize 9000 trusted up route add default 10.254.251.1 1 routed on options dns.domainname <mydomain> options dns.enable on options nis.enable off savecore
You just need to remember that the rc file is just a batch file. All you need to do is add valid commands in the correct logical order.
You might start out initializing your interfaces first. The media option isn't really necessary. I'm not sure how the flow controll "send" recommendation crept into best practices, but setting it to "full" never caused me any problems because it is a matter of negotiation with the switch anyway. I don't have a lot of setups where flow control actually is used on a regular basis.
This just seems confused and it seems you don't know what the commands are supposed to do:
ifconfig e0b `hostname`-e0a 10.254.114.7 netmask 255.255.0.0 mtusize 1500 trusted -wins up <--- This is already in the vif and can't work
ifconfig e0c `hostname`-e0b mediatype auto flowcontrol send <---- You can't and don't need to assign IP's to interfaces already in a vif ifconfig e0d `hostname`-e0c mediatype auto flowcontrol send <---- ifconfig e0e `hostname`-e0d mediatype auto flowcontrol send <----
The "hostname" in backticks is doing a hostname lookup (it looks in /etc/hosts for isnldnn07-e0N ) to get an IP to be assigned to the interface, which you don't need. You shouldn't be trying to assign IP addresses to interfaces which are already in a vif. Just run
ifconfig e0b flowcontrol send mtu 9000<--- for whichever interfaces you want in your vif with mtu 9000
After you set some basics on your interfaces, create the vif:
vif create vif0a e0b e0c e0d <--- It really seems like you can't decide which interfaces are supposed to be in this vif. It seems like you just edited "e0a" to be "vif0a". You can call a vif interface whatever you want up to a certain length.
This is broken too:
ifconfig vif0a `hostname`-vif0a 10.254.114.8 netmask 255.255.0.0 mtusize 9000 trusted up
Again, you don't need the `hostname` part if you already have the IP there (that is a pretty big netmask too, by the way 65k hosts in your net?). You really need to decide which interfaces are going to be in the vif and concentrate on getting the rc file exactly right.
The "routed on" is probably not needed in 99% of cases, especially if you only have one gateway and no RIP services in your net. If you are running a cluster (HA) setup it gets disabled anyway. You can normally safely just leave this out. More often than not, it just causes strange problems in nets with crappy old RIP services running.
There is a lot of brokeness here. I'd like to hand out reading glasses to some of the others that have looked at this...
If you don't understand something, feel free to ask. Just remember: setup the basic interfaces, create the vif, create any possible vlans, assign your IP's to the necessary "logical" interfaces, setup your default route, done.
I personally leave the ports at the default of “flowcontrol full” as the switch ports should only be configured to receive pause frames, they should not be allowed to send pause fames to the controller ports, hence leaving the default of “flowcontrol full” ensures that the controller can send frames, however it will never receive as the switch ports are not configured to send.
Obviously, if for some reason you cannot prevent the switch from sending then my config is of no use.
Yes, we have some Cisco switches that are configured to "Receive". That seemed to work fine from the switch's point of view and we also configured an ESX server to "Send" and that was OK as well, just not the FAS.
Under normal circumstances I would be happy to leave it on Full and Jumbo Frames enabled but this is for a client who specifically asked for best practises so we feel obliged to enable the optimal Flow Control settings.