Migrate from dual chassis HA to single chassis HA - Interconnect down

[timmwestedt]

Deal all,

we just moved our storage services from FAS3240 dual Chassis HA (7-Mode) to FAS8200 dual Chassis HA (cDot). Now we want to migrate the "old" FAS3240 7-Mode back to a single chassis HA. In the past the dual chassis FAS3240 used "external" onboard 10GB SFPs for cluster interconnect (Port c0a and c0b). Now we moved the second controller into the chassis of the first controller and disconnect the "external" cluster interconnect cables, because the cluster pair should use the internal Infiniband interconnect on the backplane. 

Unfortunaltely the cluster still wants to use the ports c0a and c0b for cluster interconnect. Due to the reasons that we unplug the "external" cables, Ontap told us:

Link 0: down
IC RDMA connection : down

If we enable cf the messages appaers:

Filer-B is up, takeover disabled because of reason (interconnect error)
Filer-A has disabled takeover by Filer-B (interconnect error)
RDMA Interconnect is down (Link 0 down).

How to tell Ontap to use internal Infiniband IC's insetad of "externel" cluster ports c0a and c0b?

7-Mode version 8.1.3

Many thanks for your help!!!

[ekashpureff]

Tim -

In a single chasis install we use the backplane for the HA interconnect, but not for the cluster network.

You still need to use the 10G ports for the cluster networking.

I hope this response has been helpful to you.

At your service,

Eugene E. Kashpureff, Sr.
Independent NetApp Consultant http://www.linkedin.com/in/eugenekashpureff
Senior NetApp Instructor, Fast Lane US http://www.fastlaneus.com/
(P.S. I appreciate 'kudos' on any helpful posts.)

[timmwestedt]

Hello Eugene, 

thanks for your reply. 

Are you sure this is required even in 7-Mode. I though it's only required in cDot-Mode.

Does the 7-Mode cluster communicate by cf hw_assist via e0m and internal IC only?

Best Regards

Timm 

[aborzenkov]

---

@timmwestedt wrote:

How to tell Ontap to use internal Infiniband IC's insetad of "externel" cluster ports c0a and c0b?

---

You need to set psm-cc-config? LOADER variable (setenv psm-cc-config? true). Question mark is part of variable name.

[sgrant]

Hi Timm, I think the variable you are looking for is the psm-cc-config? Since this needed to be unset when converting from a single chassis HA to a dual chassis HA, you'll need to set it back to true.

At the LOADER prompt check the status, enter printenv psm-cc-config?

If the value is empty, enter set psm-cc-config? true

Also, just check that in Maintenance Mode the ha-config show output has both:

Chassis HA configuration: ha

Controller HA configuration: ha

Hope this helps.

Cheers,

Grant.

[andris]

+1 on the bootarg.

This bootarg is documented in the "Adding a second chassis to an existing HA pair" doc.  You would just be configuring things in reverse/opposite, if going the other way.

https://library.netapp.com/ecm/ecm_get_file/ECMM1280401

[timmwestedt · Accepted solution]

Hello,

thanks for your help. The psm-cc-config setting were already set, because it was an HA pair before as well (just a dual chassis cluster pair).

It's fixed now by rebooting the first node again... finally after the second reboot the controller started up the RDMA Link 0 interconnect and cf is running well. No more interconnect errors. Failover is working as well.

Thanks all again!