Subscribe

vif Magic Assistance Needed

Bear with me on setting the stage on this.  My collegues and I have been going around in circles on how to address a problem and if it can be non-disruptively.  Here we go.............

I have a customer with a FAS3140A with a freshly installed quad port GbE NIC in each head.  It is currently set up so that ports e0a on each head serves the CIFS traffic while the e0b ports serve up thier iSCSI traffic.  Both the respective e0a and e0b ports back eachother up, i.e. FASA e0a <-> FASB e0a.  What we want to do is create a "cifs vif" consisting of ports e0a, e4a and e4b and a second one called, you guessed it, "iscsi vif" using ports e0b, e4d and e4d.  What I am seeing is that you cannot add a port to a vif that is active.  The same dead end we keep coming up with is to bring one of the FAS' ports down, have it fail over to the partner, disable the port, create the vif.............but now I cannot conceptualize how to fail back over.  Since the partner will want to fail back over to a port, not a vif.

What we are stuck with is bringing down the CIFS and iSCSI services, disable the port, create the vif, re-establish the failover relationship and reinstate services.  Is this the only option?

Cheers,

Steve

Re: vif Magic Assistance Needed

Steve,

I've recently done something similar to move my filers from 1GigE to 10GigE.  We started with 2 quad-port 1GigE cards and had previously existing vifs (vif_nfs, vif_cifs) of e3a/e4a and e3c/34c.  Added to the mix, we were also using e3b as a single interface, non-vif.  The same configuration was on both nodes of a 3140 HA pair.  We swapped the quad-port cards for dual-port 10GigE UTAs.  Briefly, we did the following:

1) Takeover controller A from controller B.
2) Swap the 1GigE cards for 10GigE cards on the controller A.
3) Bring controller A back online until it is "waiting for giveback".
4) Map a drive to \\controller-a\c$ and edit the rc file (at this point, the \\controller-a\c$ is being served by controller B since it has taken over controller A).
5) Update the rc file to create the new 10GigE vifs, vlans and ifconfigs, removing or commenting out your previous vif/ifconfig/vlan/etc statements e.g.:

  ifgrp create lacp vif-ipstore -b ip e3a e4a
  ifgrp create lacp vif-other -b ip e3b e4b
  vlan create vif-ipstore 45
  vlan create vif-other 3 25
  ifconfig vif-ipstore-45 10.1.45.1 netmask 255.255.255.0 broadcast 10.1.45.255 mtusize 1500 -wins partner vif_nfs
  ifconfig vif-other-3 10.1.3.1 netmask 255.255.255.0 broadcast 10.1.3.255 mtusize 1500 -wins partner e3b
  ifconfig vif-other-25 10.1.25.1 netmask 255.255.255.0 broadcast 10.1.25.255 partner vif_cifs

(at this point, the partner statements in controller A's /etc/rc file will correspond to the names of the vifs/interfaces on controller B)

6) Save the /etc/rc file.  Edit /etc/hosts file and adjust hostnames if necessary (assuming you have hostnames which correspond to your interfaces).  save /etc/hosts.
7) disconnect from \\controller-a\c$
8) Map a drive to \\controller-b\c$ and edit its /etc/rc file.
9) YOU ONLY CHANGE THE PARTNER STATEMENT ON EACH IFCONFIG TO MATCH THE NEW VIF NAME ON A, e.g.,

ifconfig vif_nfs 10.1.45.2 netmask 255.255.255.0 broadcast 10.1.45.255 mtusize 1500 -wins partner vif_nfs
CHANGES TO
ifconfig vif_nfs 10.1.45.2 netmask 255.255.255.0 broadcast 10.1.45.255 mtusize 1500 -wins partner vif-ipstore-45

(the partner statement matches the NEW name of controller A's vifs)

10) Be certain to adjust all partner statements in controller B's /etc/rc file.
11) Save the /etc/rc file and disconnect \\controller-b\c$
12) Perform giveback on controller-b for controller-a.  New VIFs/vlans are created on controller A during the startup sequence.
13) After giveback happens successfully source controller b's /etc/rc file via "source /etc/rc" on controller B.  This will update the in-memory configuration on controller b effectively adjusting the partner statements.  This does not drop or reset network interfaces. It will complain about the interfaces being up, but you can safely ignore that. You can verify this by running an "ifconfig -a" before and after the "source /etc/rc" file.  In the example above, the "before" would have a "partner vif_nfs" and the after would have a "partner vif-ipstore-45". IF YOU DO NOT PERFORM THIS STEP WHEN YOU TAKEOVER B FROM A, THE IN-MEMORY PARTNER STATEMENT ON B WILL NOT MATCH CONTROLLER A'S VIFS AND YOUR TAKEOVER WILL FAIL.
14) takeover controller b from controller a.  At this point, the controller-b vif_nfs will come up on the controller a vif-ipstore-45.

15) Perform all of the above steps 2-14 for controller B.  After the giveback for controller B, be certain to run "source /etc/rc" on controller A so it updates the in memory partner statements for the if config.  If you don't perform that step and later (much later) if A is takenover by B (e.g., in the event of some failure on A or perhaps an NDU for patching), the takeover will fail because the in-memory partner statements on A won't match the interface names on B.

(I haven't shown you, but when you takeover B from A, in the above example, the e3b on B is configured to partner with a VIF on A.  It certainly is possible to takeover a single interface (e.g., e3b) by a vif (e.g., vif-other-3).  I haven't tried the other way around but it should work as well.  As long as the underlying network configuration is the same, it will work.  All the partner statement does is effectively bring up B's ip address on the named interface on A, piggy-backing on the configuration of that named interface on A)

I have performed this a few weeks ago NON DISRUPTIVELY during a maintenance window on a 3140 pair running 8.0.1 and will be performing it again in two days on a 3170 pair running 8.0.1 (that's why my example used ifgrp rather than vif commands).

WARNING: Check and recheck the /etc/rc files and /etc/hosts.  A wrong command, an out-of-place period, a bad dash and your non-disruptive reconfig will become VERY disruptive.  This can be done NON DISRUPTIVELY if you plan meticulously for it and do all of your homework upfront. I checked, rechecked, triple checked and checked some more all of my config statements prior to the maintenance window.  I did my best to copy and paste my ifgrp/vlan/ifconfig statements rather than type them during the maintenance.  I did not want to chance  a typo and not catching it.

Obviously it would be good to practice this in a lab or test environment first.

Good luck.

Jason

Re: vif Magic Assistance Needed

Jason,

Wow!  This looks to be right down the alley of what we will need to be doing.  Certainly there will be some considerable preplanning and attention to detail that will need to be done on my part.  I will let you know how it goes and give credit where credit is due.

Regards,

Steve

Re: vif Magic Assistance Needed

Jason,

Quick syntax check.  On this section:

  ifgrp create lacp vif-ipstore -b ip e3a e4a
  ifgrp create lacp vif-other -b ip e3b e4b

it should read "ifgrp create lacp vif-ipstore -b port e3a e4a"  since we are specifying ports not IP addresses, correct.

We have a window of opportunity next week when we have to bring the system's down and disconnect all services.   I am still going to try the non-disruptive method for practice

Regards,

Steve

Re: vif Magic Assistance Needed

Steve,

My original syntax was correct for to create the VIF (aka interface group in 8.0.1):

ifgrp create lacp vif-ipstore -b ip e3a e4a
ifgrp create lacp vif-other -b ip e3b e4b

I have taken those lines directly from my /etc/rc file.  No "port" keyword necessary.  The "-b ip" only defines the load balancing across that interface group (ip hash based in thise case as opposed to mac hash based or round-robin).  It doesn't indicate that the following items are or should be IP addresses.

Hope this helps.

Thanks!

Jason

Re: vif Magic Assistance Needed

Thanks again.  I didn't see in NetApp's documentation that the -b flag indicated the balancing method.

Steve

Re: vif Magic Assistance Needed

Steve,

Here's a quote from the ifgrp manpage from 8.0.1:

=============================

create
Creates a new instance of a virtual interface. If no mode is specified, the virtual interface is created in multi mode. If a list of interfaces is provided, the interfaces are configured and added to the virtual interface trunk. Load balancing is specified with the -b option.

- rr refers to Round-robin Load balancing.


- ip refers to IP-based load
balancing. The IP based load balancing is used as default for multi-mode ifgrps if none is specified by user.

- mac refers to MAC-based load balancing.
============================

Thanks!

Jason