Re: Configuring additional interfaces into a VIF

newseepie · ‎2013-01-18

Hi,

We have a HA pair of FAS3140 configured with a VIF consiting of 4 interfaces (e0b, e4b, e4c, e4d - slot 4 being a Quad Port Gigabit Ethernet PCIe Controller)

I've been reviewing the entire NetApp setup when I came across this and it struck me as very strange - I always (wrongly, as it turns out) assumed the the Gigabit ethernet card was what made up the VIF. But as it turns out that we are using the built in NIC & 3 of the 4 ports on the add in NIC (slot 4).

This has left us with interface e4a not doing anything (it's still configured down)

My question is this - can this interface be connected to the network, configured and brought online to be added to the VIF for additional capacity relatively easily?

I've been reading up about it, but this is the first time I am getting involved on the network side of things so trying to digest all the information that is out there is proving to be rather challanging.

Thanks

Werner

AGUMADAVALLI · ‎2013-01-18

paste the output of ifconfig -a, rdfile /etc/rc and ifgrp status. So we will have clear picture.

thank you,

AK G

newseepie · ‎2013-01-21

Thanks for the response:

Posting what was asked for: (i've removed some

IP addresses)

ifconfig -a
e0a: flags=0x6d48867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500 PRIVATE
inet 198.15.1.96 netmask-or-prefix 0xffffff00 broadcast 198.15.1.255 noddns
ether 00:a0:98:27:de:14 (auto-100tx-fd-up) flowcontrol full
e0b: flags=0x8ad08867<BROADCAST,RUNNING,MULTICAST,TCPCKSUM,VLAN> mtu 1500
ether 02:a0:98:27:de:14 (auto-1000t-fd-up) flowcontrol full
trunked mvif0
e4a: flags=0x2508866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 00:15:17:ef:76:c8 (auto-unknown-cfg_down) flowcontrol full
e4b: flags=0x8ad08867<BROADCAST,RUNNING,MULTICAST,TCPCKSUM,VLAN> mtu 1500
ether 02:a0:98:27:de:14 (auto-1000t-fd-up) flowcontrol full
trunked mvif0
e4c: flags=0x8ad08867<BROADCAST,RUNNING,MULTICAST,TCPCKSUM,VLAN> mtu 1500
ether 02:a0:98:27:de:14 (auto-1000t-fd-up) flowcontrol full
trunked mvif0
e4d: flags=0x8ad08867<BROADCAST,RUNNING,MULTICAST,TCPCKSUM,VLAN> mtu 1500
ether 02:a0:98:27:de:14 (auto-1000t-fd-up) flowcontrol full
trunked mvif0
e0M: flags=0x2948867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
inet 172.31.232.4 netmask-or-prefix 0xffffff00 broadcast 172.31.232.255
partner e0M (not in use)
ether 00:a0:98:27:de:16 (auto-100tx-fd-up) flowcontrol full
lo: flags=0x1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
inet 127.0.0.1 netmask-or-prefix 0xff000000 broadcast 127.0.0.1
ether 00:00:00:00:00:00 (VIA Provider)
mvif0: flags=0xa2d08863<BROADCAST,RUNNING,MULTICAST,TCPCKSUM,VLAN> mtu 1500
ether 02:a0:98:27:de:14 (Enabled virtual interface)
mvif0-444: flags=0x6948863<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500
inet 172.31.232.5 netmask-or-prefix 0xffffff00 broadcast 172.31.232.255
inet x.x.x.x netmask-or-prefix 0xffffff00 broadcast 172.31.232.255
inet x.x.x.x netmask-or-prefix 0xffffff00 broadcast 172.31.232.255
partner mvif0-444 (not in use)
ether 02:a0:98:27:de:14 (Enabled virtual interface)
nfo enabled
mvif0-445: flags=0x6948863<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500
inet 172.31.233.4 netmask-or-prefix 0xffffff00 broadcast 172.31.233.255
partner mvif0-445 (not in use)
ether 02:a0:98:27:de:14 (Enabled virtual interface)
mvif0-446: flags=0x6948863<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500
inet 172.31.234.4 netmask-or-prefix 0xffffff00 broadcast 172.31.234.255
partner mvif0-446 (not in use)
ether 02:a0:98:27:de:14 (Enabled virtual interface)
mvif0-447: flags=0x6948863<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500
inet 172.31.235.4 netmask-or-prefix 0xffffff00 broadcast 172.31.235.255
partner mvif0-447 (not in use)
ether 02:a0:98:27:de:14 (Enabled virtual interface)

/etc/rdfile

hostname dcnas01
vif create lacp mvif0 -b ip e0b e4b e4c e4d
vlan create mvif0 444 445 446 447
ifconfig mvif0-444 `hostname`-cifs netmask 255.255.255.0 -wins partner mvif0-444 nfo
route add default x.x.x.x 1 # Placing here ensures DG is on mvif0-444
ifconfig mvif0-445 `hostname`-av netmask 255.255.255.0 -wins partner mvif0-445
ifconfig mvif0-446 `hostname`-snap netmask 255.255.255.0 -wins partner mvif0-446
ifconfig mvif0-447 `hostname`-nfs netmask 255.255.255.0 -wins partner mvif0-447
ifconfig e0a -wins no_ddns # Disable WINS, DNS on ACP NIC
ifconfig e0M `hostname`-e0M netmask 255.255.255.0 partner e0M # This is the 100Mb mgmt interface used by SCOM, DFM i.e. dcnas01
ifconfig mvif0-444 alias x.x.x.x netmask 255.255.255.0
ifconfig mvif0-444 alias x.x.x.x netmask 255.255.255.0
route add net x.x.x.x/24 172.31.234.3 1 # Force snapmirror traffic over snap VIFs
route add host x.x.x.x 172.31.232.4 0 # Force DFM traffic to talk over e0M
route add host x.x.x.x 172.31.232.4 0 # ASUP Mail Gateway
route add host x.x.x.x 172.31.233.4 0 # Force AV traffic over AV VIF
route add host x.x.x.x 172.31.233.4 0 # Force AV traffic over AV VIF
routed on
options dns.domainname abc.abc.com
options dns.enable on
options nis.enable off
savecore
priv set diag
setflag smb_vscan_delay_close_resp 1
--------------------------
We are running OnTap 7.3.6 - so instead of ifgrp, the command is 'vif status'

vif status

default: transmit 'IP Load balancing', VIF Type 'multi_mode', fail 'log'

mvif0: 4 links, transmit 'IP Load balancing', VIF Type 'lacp' fail 'default'

VIF Status Up Addr_set

up:

e4b: state up, since 07Jun2012 23:06:33 (227+12:04:57)

mediatype: auto-1000t-fd-up

flags: enabled

active aggr, aggr port: e0b

input packets 78519800731, input bytes 18829470990123

input lacp packets 708256, output lacp packets 655228

output packets 35442257716, output bytes 28314866613817

up indications 3, broken indications 1

drops (if) 0, drops (link) 0

indication: up at 07Jun2012 23:06:33

consecutive 0, transitions 4

e4c: state up, since 07Jun2012 23:06:32 (227+12:04:58)

mediatype: auto-1000t-fd-up

flags: enabled

active aggr, aggr port: e0b

input packets 64972455447, input bytes 32931226412620

input lacp packets 708301, output lacp packets 655225

output packets 55403116031, output bytes 44959698565653

up indications 2, broken indications 0

drops (if) 0, drops (link) 0

indication: up at 07Jun2012 23:06:32

consecutive 0, transitions 2

e4d: state up, since 07Jun2012 23:06:32 (227+12:04:58)

mediatype: auto-1000t-fd-up

flags: enabled

active aggr, aggr port: e0b

input packets 78114188849, input bytes 19886216266101

input lacp packets 708177, output lacp packets 655226

output packets 120974124979, output bytes 95015729900386

up indications 2, broken indications 0

drops (if) 0, drops (link) 0

indication: up at 07Jun2012 23:06:32

consecutive 0, transitions 2

e0b: state up, since 07Jun2012 23:06:30 (227+12:05:00)

mediatype: auto-1000t-fd-up

flags: enabled

active aggr, aggr port: e0b

input packets 45787764701, input bytes 12578313299938

input lacp packets 708285, output lacp packets 655225

output packets 90200167276, output bytes 66721850583072

up indications 2, broken indications 0

drops (if) 0, drops (link) 0

indication: up at 07Jun2012 23:06:30

consecutive 0, transitions 2

Thanks,

Werner

ALEXJILITSKY · ‎2013-01-18

Hi,

If you wish to leave the IFGRP to use the existing 4 interfaces, and add e4a as a fifth interface, you can simply do so with the following command:

ifgrp add <ifgrp_name> e4a

However, if you also wish to remove the built-in NIC from the IFGRP, you will have to shut the IFGRP down following these steps:

1. Shut down the interface group:

ifconfig <ifgrp_name> down

2. Add the fourth interface:

ifgrp add <ifgrp_name> e4a

3. Remove the built-in NIC:

ifgrp delete <ifgrp_name> e0b

4. Bring the IFGRP back up:

ifconfig <ifgrp_name> up

If you're at it, you may also want to make sure that the interface group is configure as multi-mode, as I understand you are looking for additional capacity and/or performance.

Please also note that you might want to configure different interfaces for different uses such as user access, management access, snapmirror etc. Adding more interfaces to the same IFGRP isn't always the best option.

newseepie · ‎2013-01-21

Thanks Alex, found that useful.

We are still running OnTap 7.3.6 (will get moving the upgrade to 8 soon) so the commands still reference VIF (instead of IFGRP) - but the syntax appears to still be the same.

Yes, I am looking at additional capacity for the controllers.

The group is set to multi-mode - I wouldn't have known to check.

We'll have a discussion around splitting the interfaces for different uses as you pointed out.