FAS2750/AFF-A220 - Are e0a/e0b different to other ethernet ports?

[Stefan_R]

Hi,

in my lab i'm using a single controller FAS2750 and i'm trying to use the e0a and e0b to connect them to a ethernet switch, but i don't get a link. When using the same cable and make a bridge from e0a to e0b i'm getting a link, but not when connecting to the switch. Same on the switch, when i connecting both ports i'm getting also a link.

Switch and NetApp are showing all SFP informations of the connected SFP.

I have already tryed different DAC cables and SFP modules with optical cables. The switch is a Mellanox MSN2010.

Do i missed something?

Kind regards

Stefan

[Stefan_R]

I made some additional tests.

When removing the cable, to the switch with no link, from e0a and putting it to e0f, i got a link. When putting a cable from e0a to e0f i got a link. I tested seceral DAC cables and SFP's and it is every time the same. I switched off the switch and after a clean power on it is still the same.

So the only thing what i havn't tested is a different switch...

I'm totally confused...

[darr613]

Hello Stefan,

Ports e0a and e0b on the FAS27xx are cluster ports, so communicate internally a 2 nodes' cluster.
If I'm not mistaken, those are 40GB ports, they are designed to work as a link aggregate,
and usually work in a different IPSPACE.

All the other Ethernet ports (e0c to e0g) are data ports with 10GB of speed.

[Stefan_R]

Thank you for the answer, yes e0a/e0b are the cluster ports, but when installed as a single node system this ports can be configured like the other ports.

And yes cluster ports are in a different IPSPACE, but even if so, i should get a physical link. In a single node cluster this ports are in the default IPSPACE.

In a 27x0 this ports are 10G SFP+, if it will be 40GB ist must be a QSFP port, with this i can't change the cable from e0a to e0f.

Now i used a different DLink 10G switch on port e0b:

slot 0: 10 Gigabit Ethernet Controller IX5-SFP+

e0a MAC Address:    d0:39:ea:12:ba:60 (auto-unknown-fd-down)

    SFP Vendor:         CISCO-MOLEX INC

    SFP Part Number:    74752-9025      

    SFP Serial Number:  MOC12251447     

e0b MAC Address:    d0:39:ea:12:ba:61 (auto-10g_twinax-fd-up)

    SFP Vendor:         CISCO-MOLEX INC

    SFP Part Number:    74752-9025      

    SFP Serial Number:  MOC13353263     

Device Type:        X550EM(0x15AC) (BSD99)

Firmware Version:   2.10-7.1 etid 800009be phy 20150807

I'm getting a link with that switch, so it must be a problem with the Mellanox MSN2010 switch, very strange...

[andris]

OS and version on the 2010?

Try setting the speed manually to 10GbE on the switch ports. It might be an autonegotiation issue.

[Stefan_R]

Onyx - 3.8.2306

I tryed both, auto and fixed speed...

[darr613]

It seems the issue is that the switch is not Netapp Ontap OS compatible... it was designed

 

for Element OS

[darr613]

 The 2750 was designed for Ontap and it's not compatible with the Mellanox MSN2010 switch

[darr613]

The Mellanox MSN2010 switch is compatible with the above hardware Element OS and the below hardware

 

[darr613]

You can check for yourself in the ITM for compatibility for your switch
https://imt.netapp.com/matrix/#search

[Stefan_R]

You will tell me, that when i use a NetApp with ONTAP and CIFS or NFS, that the used Switch at which the NetApp is connected should be in the IMT? Sorry, but this is not correct.

But yes when using Element OS in a HCI environment, or when using a Switch in a MetroCluster then it must be supportet, but this is a complete other issue...

ONTAP is BSD based, i found an link issue in the X550EM driver in FreeBSD, perhaps we have an ONTAP driver issue here.

I hope someone has a magic hint for a setting that the e0a/e0b ports can work with a 2010 switch. On the NetApp or on the switch...

If not i must live with that, because the controller has no support, then i use it in a single node environment which also isn't supportet (last single node system was the 26x0). And i have no support for the Mellanox switch, so i can't get a FW update for that. It is only a lab environment...

[andris]

For front-end data networking use, we have a flexible policy. Using 3^rd party optics/cables is permitted, as long as they are similar in technology and compliant with applicable Ethernet standards. Generally, you would avoid active copper and active optical technologies, unless we specifically have our own solutions for them (per HWU). If using another vendor’s cables/optics, best to stick with ones that are approved by the device/switch vendor.

So, “Try it – it might work.  YMMV”. But since you do not have support entitlement, your options might be limited.

[Stefan_R]

I have now updated a other SN2010 to the latest FW but no change...

[Stefan_R]

I got it, setting the port to 10G isn't enough, the command needs to be with "no-autoneg".

I used on the interface:

speed 10G

But correct is:

speed 10G no-autoneg

Confusing because i was thinking that "auto" will be:

speed auto

and when setting a fixed speed that "auto" is disabled. So setting a fixed speed don't disable auto negotiation... A little bit confusing to me, but it is working now...