How to move IP addresses nondisruptive to 10GbE interface group?

cgeck0000 · ‎2013-09-17

Looking for some guidence. No one seems to understand what I am trying to do including NetApp technical support and our SE.

Current environment

FAS3240 HA

ONTAP 7-Mode version 8.1.2

2 interface groups on each node

1 interface group includes the 1GbE nics with 2 IP addresses asigned

1 interface group includes the 10GbE nics with 2 IP addresses assigned

We have NFS datastores being presented to VMware vSphere 4.1 and up hosts using all 4 IP addresses

What we would like to do is move the 2 IP addresses assigned to the 1GbE interface group to the 10GbE interface group as aliasses. We want to do this nondisruptly so that the NFS datastores do not go offline.

Can anyone share some insight into how we can accomplish this if possible?

DOUGLASSIGGINS · ‎2013-09-17

Copy and paste what you would do to remove the configuration of the 1G interfaces as well as what you would do to alias the 10G interfaces preferably into the serial or SP/RLM console. This should be nearly instantaneous, and I don't presume any issues during the transition. Although I myself have only done this to regular NFS clients, and not specifically any large VMWare hosts. I would presume the VMWare datastores allow for a pretty large disruption to connectivity and 1-2 seconds shouldn't be noticed.

Please also remember to edit your /etc/rc on both heads if needed to make the changes permanent.

Test the changes in a simulator/VM just to be sure you have the correct commands.

aborzenkov · ‎2013-09-17

Takeover filer; edit /etc/rc for new configuration; giveback.

DOUGLASSIGGINS · ‎2013-09-17

aborzenkov,

Is there any particular reason you suggested a disruptive takeover? I have many times in the past edited the network configuration interactively via serial console or SP with minimal disruption -- certainly less disruption than a takeover.

scottgelb · ‎2013-09-17

I would also edit rc (and hosts if necessary for hostname substitution), confirm the partner interfaces are configured and available, and then takeover and giveback on the node. With nfs it will be an ndu event and is a supported method and also tests rc/hosts syntax. Editing interactively without a takeover/giveback does work but I would not do it...agreed it is more efficient to do without takeover/giveback but I would be concerned about not being able to paste the 2 ifconfig statements fast enough to remove from the 1GB ifgrp and then alias the 10GB ifgrp..or in the event the console locked up which happens sometimes and can seem like an eternity even if only seconds and in more rare cases a minute or more..

aborzenkov · ‎2013-09-17

Moving address to different physical port almost certainly requires adjustment to network infrastructure. I cannot predict how long it may take.

Takeover should be non-disruptive for ESX access. If it is, your HA pair is no more HA in the first place and no NDU would be possible.

jbogardus3 · ‎2013-09-19

I did the same process last spring on a few different systems. 3170 and 6280s.

ONTAP 7-Mode version 8.1.1P1

Both doing 1 Gbps to 10 Gbps migration to Cisco Nexus connections, and 10 Gbps to 10 Gbps when the network team upgraded Nexus switches I was connecting to

These NetApps were hosting VMware NFS and iSCSI datastores.

In my situation I initially moved IPs from a 10 Gbps interface group to a temporary standalone interface e0f to reconfigure the 10 Gbps connections, then below is the example for moving my VLAN 8 (NFS VLAN) IPs back to the reconfigured 10 Gbps

When moving alias IPs you simply need to remove the alias from the old interface and add to the new interface group (and vlan if used) you have prepared

ifconfig e0f-8 -alias 172.31.8.182

ifconfig vif-nexus-8 172.31.8.182 netmask 255.255.255.0 partner vif-nexus-8 mtusize 9000 trusted -wins up

When moving primary IPs you need to shutdown the interface and delete the vlan, then you can reassign the IP to the new interface group without ONTAP complaining that it is already assigned

ifconfig e0f-8 down

vlan delete e0f 8

ifconfig vif-nexus-8 172.31.8.181 netmask 255.255.255.0 partner vif-nexus-8 mtusize 1500 trusted -wins up

Each of these sets of interface reassignment commands can be pasted into your console session to complete the IP reassignment in one second. I can't remember if the ESX hosts logged anything about the one second disconnect, but it's similar but much shorter than the disconnects that happen when doing a failover/failback operation. No impact on VMs.

Update the /etc/rc with the same interface group and IP config info

No failover/failback required as long as you are confident in the commands used to reconfigure and that the contents of the /etc/rc is the same and doesn't need to be tested by the reboot.

cgeck0000 · ‎2013-09-23

I am looking at going the route of a takeover\giveback to reconfigure these settings. We've done numerous maintenance windows over the years with a takeover\giveback for both nodes and the only hiccups seen are on a couple of iSCSI connected SQL clusters, NFS virtual machines keep humming right along.rfrfr

So I should do a takeover of one node first and then modify its /etc/rc file? I've always modified the /etc/rc file first and then initiated a takeover. Is that not the correct way? What is the negative of modifying the /etc/rc file first and then taking over the node?

This is the current running /etc/rc file:

hostname ****************

ifgrp create lacp ifg03 -b ip e0a e0b

ifgrp create lacp ifg04 -b ip e1a e1b

vlan create ifg03 61

vlan create ifg04 61

ifconfig e0M `hostname`-e0M flowcontrol full netmask 255.255.255.0 partner e0M

ifconfig ifg03-61 `hostname`-ifg03-61 netmask 255.255.255.0 partner ifg01-61 mtusize 1500 trusted -wins

ifconfig ifg03-61 alias 10.16.1.65 netmask 255.255.255.0 broadcast 10.16.1.255

ifconfig ifg04-61 `hostname`-ifg04-61 netmask 255.255.255.0 partner ifg02-61 mtusize 9000 trusted -wins up

ifconfig ifg04-61 alias 10.16.1.67 netmask 255.255.255.0 broadcast 10.16.1.255

route add default 10.16.1.10 1

routed on

options dns.domainname ******************

options dns.enable on

options nis.enable off

savecore

Modifying it would look like this:

hostname *************

ifgrp create lacp ifg03 -b ip e0a e0b

ifgrp create lacp ifg04 -b ip e1a e1b

vlan create ifg03 61

vlan create ifg04 61

ifconfig e0M `hostname`-e0M flowcontrol full netmask 255.255.255.0 partner e0M

ifconfig ifg03-61 `hostname`-ifg03-61 netmask 255.255.255.0 partner ifg01-61 mtusize 1500 trusted -wins -----> Would change the /etc/hosts file to reflect 10.16.1.70

ifconfig ifg03-61 alias 10.16.1.71 netmask 255.255.255.0 broadcast 10.16.1.255

ifconfig ifg04-61 `hostname`-ifg04-61 netmask 255.255.255.0 partner ifg02-61 mtusize 9000 trusted -wins up

ifconfig ifg04-61 alias 10.16.1.67 netmask 255.255.255.0 broadcast 10.16.1.255

ifconfig ifg04-61 alias 10.16.1.64 netmask 255.255.255.0 broadcast 10.16.1.255

ifconfig ifg04-61 alias 10.16.1.65 netmask 255.255.255.0 broadcast 10.16.1.255

route add default 10.16.1.10 1

routed on

options dns.domainname **************

options dns.enable on

options nis.enable off

savecore

scottgelb · ‎2013-09-23

I modify before…the partner already has takeover settings… new rc/hosts take effect on the giveback so shouldn’t matter at what time modified before or after takeover.

aborzenkov · ‎2013-09-23

A matter of habit I guess. But I like to retain possibility to quickly perform giveback if something goes wrong during takeover.

Otherwise in simple cases it probably does not matter and in complex cases editing /etc/rc alone may not be enough and you need temporary adjustments on partner as well.

DAVE_WITHERS · ‎2013-10-09

What I have done is run my commands in the CLI to change the network, verify it actually worked, then write to the rc file and initiate a takeover to verify the settings stick. Then hit the second node after.

But yeah, change RC file before taking over.