How do i enable an ifgrp in clustered mode

[Singhz]

Hi 

Can someone help me with the command to enable an ifgrp in cluster mode (cdot). I currently have an ifgrp which entered a disabled state and if i run the command 'ifgrp show' i get 'active ports = none' on the ifgrp with the fault. All other ifgrps have 'activeports = full' and list their associated ports. There doesn't seem to be an option to enable the ifgrp from system manager. 

Thanks in advance

[ekashpureff]

Singhz -

Add physical ports to the ifgrp ?

I hope this response has been helpful to you.

At your service,

Eugene E. Kashpureff, Sr.
Independent NetApp Consultant http://www.linkedin.com/in/eugenekashpureff
Senior NetApp Instructor, FastLane US http://www.fastlaneus.com/
(P.S. I appreciate 'kudos' on any helpful posts.)

[Singhz]

Hi

Thanks for your response. I have 2 ports, e0c and e0d as part of the ifgrp configured as LACP, which was working for quite some time until last week the ifgrp had become disabled. e0c and e0d show in the up status. I can't see anyway to enable the ifgrp..

[Singhz]

if i run a 'network port ifgrp show' i get the following output....see how the active ports show 'none' on the first node

clstn1::*> network port ifgrp show
              Port       Distribution                             Active
Node     IfGrp      Function     MAC Address       Ports   Ports
-------- ---------- ------------ ----------------- ------- -------------------
clstn1-01
         a0a      ip           *****************                none    e0c, e0d
         a0m      port         *****************                 full    e0f
clstn1-02
         a0a      ip           *****************                   full    e0c, e0d
         a0m      port         *****************                 full    e0e
4 entries were displayed.

[ekashpureff]

It looks like those interfaces are both down.

Check physical connectivity and configuration on the switch ?

I hope this response has been helpful to you.

At your service,

Eugene E. Kashpureff, Sr.
Independent NetApp Consultant http://www.linkedin.com/in/eugenekashpureff
Senior NetApp Instructor, FastLane US http://www.fastlaneus.com/
(P.S. I appreciate 'kudos' on any helpful posts.)

[Singhz]

Both ports on the netapp and their corresponding switch port connection show as status up. I've just tried taking one port out of the ifgrp and creating a new ifgrp in single mode and i get connectivity. I'm now thinking it's a problem with LACP on our switches which our networking guys said everything was fine when they checked it.

Any other suggestions?

[Naveenpusuluru]

Hi @Singhz

If you are using LACP ifgrp on NetApp you must configure LACP configuration on switch side. Please ask you Network team to check the configuration from their side.

[Singhz]

LACP config on the switches all look fine. By taking 1 port out of the ifgrp and creating a single mode ifgrp with one port, the switch stats show this port as being detached, which i'm told is what you would see if there is misconfiguration. So switch is configured correctly for LACP but the Netapp ports / ifgrp is misbehaving for some weird reason.

[Naveenpusuluru]

Hi @Singhz

Please check broadcast-domain and ipspace details.

[asulliva]

Probably preaching to the choir, but be sure to validate your settings and your port configuration(s):

net port ifgrp show -node clstn1-01 -ifgrp a0a -instance
net port show -node clstn1-01 -port e0c,e0d -instance

[Singhz]

Sorry for the late reply and thanks for all your suggestions guys. I'm still no closer to fixing this. LACP switch config all looks correct. We have even broken LACP on the switch and created an ifgrp in multimode (non LACP) on the controller. When set to non LACP, controller ports show link UP, and ifgrp ENABLED, but not able to communicate across the network, cannot ping. My switches can detect a MAC address, but that's about it.

Will see what Neatpp have to say, unless someone has a lightbulb moment, so please contribute..

Cheers

[Singhz]

Hi

Port configurations all look good, both ports showing status up. We have now gone back to basics and destroyed LACP on the switches and on the netapp and ar estill experiencing the same problem

If i try to ping an IP on the network from a lif or port on the ifgrp, pings are dropped. I'm not even able to ping the gateway address even though static routes exits for each subnet.

[nitish]

Can you run "netstat -rn" from the node

system node run -node <node-name>

priv set diag

netstat -rn

ifconfig 

Do you see the Gateway MAC address for ports a0a, e0c, e0d

If you don't see the gateway IP address and it's MAC address, it's a configuration issue with LACP on Network switches.

Please share the output. 

[Singhz]

Hi Nitish

I don't see the ip address and MAC of the gateway...

Internet:
Destination Gateway Flags Refs Use Interface PMTU rt_ifa
default 172.24.31.1 UGS 1 110 a246m-1631 1500 172.24.31.31
172.24.31/24 link#12 UC 1 0 a246m-1631 1500 172.24.31.31
172.24.31.255 ff:ff:ff:ff:ff:ff UHL 1 12 a246m-1631 1500 172.24.31.31

Compared to the below which is the output of a different interface to which i do see the ip and gateway. This is a working port

Internet:
Destination Gateway Flags Refs Use Interface PMTU rt_ifa
default 172.24.31.1 UGS 1 67887698 a430m-1631 1500 172.24.31.13
172.24.31/24 link#16 UC 1 0 a430m-1631 1500 172.24.31.13
172.24.31.1 0:10:db:ff:fd:2 UHL 2 0 a430m-1631 1500 172.24.31.13
172.24.31.16 0:a0:98:9a:26:c3 UHL 1 0 a430m-1631 1500 172.24.31.13

What does this mean? and where is the root of the problem?

[PatrickGruler]

Did you try to delete the ifgrp and recreate it from scratch? Try other ports on the switch - maybe it is a physical problem on the switch.

[tmac]

the following data would be helpful:

From the switch: show the configuration being used (like cisco: show running-config -> list the interfaces and the port-channels in question)

From the netapp:

ifgroup show -instance

net port show -node xx -port e0c,e0d -instance (where xx is the nodename)

system node run -node * options cdpd.enable on (turn on device discovery)

network device-discovery show -port e0c|e0d -instance

[a-kupi]

Was this ever resolved? I am having the same issue in my environment.

Thanks,

[Singhz]

Apologies for the late reply.

I raised a case with Netapp and was asked to collect some data running the following commands. The moment i ran step 13 below, which corresponds to the ports involved in the LACP itslef, my ifgrp was back online. We were running 8.3p4 at the time and Netapp put this down to a firmware issue with the CNA Qlogic cards on the controller. After upgrading the firmware, never experienced the problem again.

What version of firmware are you running?

Hope this is of some use.

Trigger ASUP

1. *> node run -node <nodename>
2. *> priv set diag
3. *> rastrace dump -m 46
4. *> mbstat
5. *> rtag -t mbuf
6. *> ifstat -a
7. *> ifinfo -a
8. *> cna dump -d

Wait 1 minute

9. *> mbstat
10. *> rtag -t mbuf
11. *> ifstat -a
12. *> ifinfo -a
13. *> cna dump 0c <<<< that is the corresponding fiber port #

Trigger ASUP

Upload command outputs

Upload RAStraces from

     /mroot/etc/log/rastrace  ->  QLA_*.dmp (Rastrace dump)

Upload CNA firmware dump from

     /mroot/etc/log  ->  qla_<port>_*.dmp (FW dump)

[a-kupi]

I had a similar issue with implementing NFS for VMWare on AFF300.

The root cause was traced to an incorrect vpc configuration on the switch. Due to a misslabeling of the switch ports, we were configuring vpc for ports belonging to separate NetApp nodes. This caused one physical port/node on NetApp be down.

networking group ran the following command:

show vpc consistency-parameters vpc # (ie 19), and realized that lag-id was not the same on (in our case one port per swtich).

on the netapp side, we ran the following command (two nodes in our environment): 

node run -node "nodename" cdpd show-neighbors 

node run -node "nodename" cdpd show-neighbors 

This provided us information showing where the 10 GB NetApp NFS designated ports were connecting to the Cisco Nexus switch. Once the right ports were configured for vpc, ifgrp was up an health, and both ports were healthy as well.

Hope this helps.