FAS2552, Nexus 3000, ESXi 5.5 VPC LACP config

[ThePoster]

Hey guys,

Having issues with communication between my devices.

I am trying to get a ping from either 3k2 or 3k3 in below image to the links i have created on my fas2552, any help would be greatly appreciated.

Both 3ks are connected via VPC peerlink (this is the green lines in image below)

   interface port-channel10

     switchport
     switchport mode trunk
     switchport trunk allowed vlan 1,400
     spanning-tree port type network
     vpc peer-link

The 3ks have vPC to individual controllers

interface port-channel101 (this is the blue lines in image below)
  switchport
  switchport access vlan 400
  no negotiate auto
  vpc 101

interface port-channel102 (this is the purple lines image below)
  switchport
  switchport access vlan 400
  no negotiate auto
  vpc 102

vPC status shows active
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
101    Po101       up     success     success                    400
102    Po102       up     success     success                    400

HSRP gateway

Interface   Grp Prio P State    Active addr      Standby addr     Group addr
Eth1/44     2   110  P Active   local            172.16.250.3     172.16.250.1    (conf)

the FAS has 2 ifgrps on vlan 400 both active

            dmm1         up/up    172.16.250.211/24  GMK200-01     a0a-400 true (this is the blue lines in image below)
            dmm2         up/up    172.16.250.221/24  GMK200-02     a0a-400 true (this is the purple lines image below)

[DREW_RUSSELL]

Can you clarify whether or not you are running 7-Mode or clustered Data ONTAP on the NetApp Controllers? It should not affect the configuration but will help in terms of providing releveant commands if needed.

As for your vPC Configuration, po101 & 102 should be configured as switchport mode trunks and should also be configured as a spanning-tree port type edge trunk. For example:

interface port-channel101 
switchport

switchport mode trunk
switchport mode trunk allowed vlan 400

spanning-tree port type edge turnk
vpc 101

!

interface port-channel1012
switchport
switchport mode trunk
switchport mode trunk allowed vlan 400
spanning-tree port type edge turnk
vpc 102

Since the Nexus 3K has 1 Gb ports I would also keep auto negotiate enabled espeically if you are not connected to e0a and e0b on the FAS2554 side.

[ThePoster]

Hi Drew,

I have configured ports as per your advice and still no luck

interface port-channel101
  switchport
  switchport mode trunk
  switchport trunk allowed vlan 1,400
  spanning-tree port type edge trunk
  no negotiate auto
  vpc 101

interface port-channel102
  switchport
  switchport mode trunk
  switchport trunk allowed vlan 1,400
  spanning-tree port type edge trunk
  no negotiate auto
  vpc 102

i am unable to remove the 'no negotiate auto' error - 'ERROR: port-channel101: Configuration does not match the port capability.'

could this be because they are 10Gb links?

we are running clustered data onTap

[DREW_RUSSELL]

Can you try running negotiate auto on the individual ports and see if that works?

Also, can you clarify what exactly you're trying to ping (i.e from what to what)?

On the Nexus 3K side can you run a show interface brief ?

On the FAS2554 can you run the following:

• network interface show
• network port show
• network port show -node GMK200-01 -port a0a-400  (correct the syntax for your setup if i messed that up)
• network port show -node GMK200-02-port a0a-400  (correct the syntax for your setup if i messed that up)
• network traceroute -node GMK200-01  -destination <Remote InetAddress>

[ThePoster]

Hey Drew,

I got the ping working, setting to trunk port and spanning tree edge was the fix.

After executing those settings, i tried pinging from the 3k and it failed.. then i reposted, and it came to me that i should try pinging from one of the edge devices as there is no vlan interface on the 3k.

did the ping from UCS (vmware host) to the netapp and it worked!!!!

thanks for your help, i can begin the iSCSI initialization now

[DREW_RUSSELL]

Awesome!! Glad you're up and running.

[ThePoster]

So now that i am able to ping the devices i am having issues finding the storage device through iSCSI.

I have setup igroups

GMK200::> igroup show
Vserver   Igroup       Protocol OS Type  Initiators
--------- ------------ -------- -------- ------------------------------------
gmk200-iscsi
          GMK-VM-02    iscsi    vmware   iqn.1998-01.com.vmware:gmk-vm-02-
                                         68baaf70

but when i scan on my software adapter on vmware it doesnt find anything. any ideas, or should i be posting on a vmware forum??

[DREW_RUSSELL]

We can certainly help here! However, I would create a new post just to make sure your issue does not get burried here.

To stat with though I would make sure that your vServer is configured for iSCSI. You can do a vserver show and then look for the Allowed Protocols section. Also you'll need to make sure that you have at least one iSCSI LIF configured on the SVM.

Can you also confirm which NIC you are using in your UCS Blades?

[ThePoster]

We are using Cisco VIC ethernet NIC , 10GB on the UCS220

I have created a LAG on the esxi host, which links 2 of the 10gb ports.

This sits on a distributed switch but when i try and bind the storage adapter to a vmkernel it doesnt find the one i have created and attached to the dSwitch.

It appears only vSwitch kernels are shown

Below is some more infor from the netapp side, please let me know if i can supply more.

thx

GMK200::> vserver show
                    Admin     Root                  Name    Name
Vserver     Type    State     Volume     Aggregate  Service Mapping
----------- ------- --------- ---------- ---------- ------- -------
GMK200      admin   -         -          -          -       -
GMK200-01   node    -         -          -          -       -
GMK200-02   node    -         -          -          -       -
gmk200-iscsi
            data    running   gmk200iscs aggr_      file,   file
                              i_root     GMK200_1_  nis
                                         Flash

GMK200::> igroup show

Vserver   Igroup       Protocol OS Type  Initiators
--------- ------------ -------- -------- ------------------------------------
gmk200-iscsi
          GMK-VM-02    iscsi    vmware   iqn.1998-01.com.vmware:gmk-vm-02-
                                         68baaf70

GMK200::> network interface show

            Logical    Status     Network            Current       Current Is
Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
----------- ---------- ---------- ------------------ ------------- ------- ----
GMK200
            cluster_mgmt up/up    192.168.115.9/24   GMK200-02     e0M     true
GMK200-01
            clus1        up/up    169.254.125.254/16 GMK200-01     e0e     true
            clus2        up/up    169.254.76.228/16  GMK200-01     e0f     true
            mgmt1        up/up    192.168.115.10/24  GMK200-01     e0M     true
GMK200-02
            clus1        up/up    169.254.3.63/16    GMK200-02     e0e     true
            clus2        up/up    169.254.245.78/16  GMK200-02     e0f     true
            mgmt1        up/up    192.168.115.6/24   GMK200-02     e0M     true
gmk200-iscsi
            dmm1         up/up    172.16.250.211/24  GMK200-01     a0a-400 true
            dmm2         up/up    172.16.250.221/24  GMK200-02     a0a-400 true

[ThePoster]

please disregard my stupidity. i have found out why i was not able to find the LUN... during earlier testing i had CHAP auth enabled. when i cancelled it on the netapp side during our network issues i forgot to remove from vmware. i removed and found the LUN!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Thanks again, it all makes sense now \