ONTAP Discussions

Interconnect issues - FAS3270


Hello All,

Trying to help a customer out convert a pair of controllers that were in a metro cluster into a HA pair, FAS3270 with IOXM.

We are using new disks so wiped out all disk ownership and installed the new disks, completed a option 4 and installed ONTAP on both, then 1 of the 2 nodes died and would not boot so replaced it with a new node, completed the node replacement process and got the node back up, enabled CF from the first time and it wouldnt syncronise, did a CF monitor and both links showed as down, this is where it gets odd.


Currently running 1 cable between c0a, looking for a 2nd cable now, we do see a green light.


1st node -

dersan03*> ic status
        Link 0: down
        Link 1: down
        IC RDMA connection : down

dersan03*> ic config show
Interconnect Type: iWARP
  Local System ID: 1575077239
Partner System ID: 1574064502
        inet netmask 0xffffff00 broadcast
        ether 00:a0:98:1d:83:c4 (auto-10g_twinax-fd-up) flowcontrol full
        inet netmask 0xffffff00 broadcast
        ether 00:a0:98:1d:83:c5 (auto-unknown-enabling) flowcontrol full


Node2 -

dersan04*> ic status
        Link 0: down
        Link 1: down
        cfo_rv connection state :                       NOT CONNECTED
        cfo_rv nic used :                               0
        cfo_rv connection role :                        source_rv
        cfo_rv1 connection state :                      NOT CONNECTED
        cfo_rv1 nic used :                              1
        cfo_rv1 connection role :                       none
        cfo_rv2 connection state :                      NOT CONNECTED
        cfo_rv2 nic used :                              1
        cfo_rv2 connection role :                       not applicable

dersan04*> ic config show

Error: Command not supported on this platform.


I get the feeling there is a misconfig on node02 but ha-config for both chassis and controller is set to ha, psm-cc-config is unset and the partner system ID's are correct.


What am I missing, crying in pain over here, both nodes running 8.2.4P6, confirmed part numbers between the 2 controllers are the same but the revision number is different.


Thanks in advance.




Re: Interconnect issues - FAS3270


For kicks...


when you re-initialized the system, at the LOADER prompt, did you "set-defaults"?

Maybe there is a ENV VAR set that is messing things up?


From MAINT mode, you did verify that "ha-config show" on both nodes looks correct?

For controller and chassis (ha) ?

Re: Interconnect issues - FAS3270


I didnt, that is a good thing to try though,  once I have done that do I have to reset the partner ID?


Yes, ha looks set correct to me

*> ha-config show
Chassis HA configuration: ha
Controller HA configuration: ha

Attached are the printenv from each node, I dont see anything suspecious.


To me it looks like node2 is trying to use the internal chassis HA connections opposed to c0a/c0b but I dont know how to get it to change

Re: Interconnect issues - FAS3270


If possible, start over.

Boot both into maintenance mode.

Make sure both are ha for the the chassis and controller.

If not, use the ha-config command to set both sides to ha.

Then at the loader prompt on both sides:


Follow the directions in this link: https://kb.netapp.com/app/answers/answer_view/a_id/1030427


Hopefully that helps



View solution in original post

Re: Interconnect issues - FAS3270


Thanks for this, yes I will give it a go tomorrow and let you know how I get on.



Re: Interconnect issues - FAS3270


Turns out there was an FCVI card in slot 1 which was taking precident over the onboard cards, removed the card and all started working.


Cheers for your help

Earn Rewards for Your Review!
GPI Review Banner
All Community Forums