ONTAP Discussions

Interconnect issues - FAS3270

John_Blackwell
6,048 Views

Hello All,

Trying to help a customer out convert a pair of controllers that were in a metro cluster into a HA pair, FAS3270 with IOXM.

We are using new disks so wiped out all disk ownership and installed the new disks, completed a option 4 and installed ONTAP on both, then 1 of the 2 nodes died and would not boot so replaced it with a new node, completed the node replacement process and got the node back up, enabled CF from the first time and it wouldnt syncronise, did a CF monitor and both links showed as down, this is where it gets odd.

 

Currently running 1 cable between c0a, looking for a 2nd cable now, we do see a green light.

 

1st node -

dersan03*> ic status
        Link 0: down
        Link 1: down
        IC RDMA connection : down

dersan03*> ic config show
Interconnect Type: iWARP
  Local System ID: 1575077239
Partner System ID: 1574064502
c0a: flags=0x1f4e867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 9000 PRIVATE
        inet 192.168.1.174 netmask 0xffffff00 broadcast 192.168.1.255
        ether 00:a0:98:1d:83:c4 (auto-10g_twinax-fd-up) flowcontrol full
c0b: flags=0x174e867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 9000 PRIVATE
        inet 192.168.2.246 netmask 0xffffff00 broadcast 192.168.2.255
        ether 00:a0:98:1d:83:c5 (auto-unknown-enabling) flowcontrol full

 

Node2 -

dersan04*> ic status
        Link 0: down
        Link 1: down
        cfo_rv connection state :                       NOT CONNECTED
        cfo_rv nic used :                               0
        cfo_rv connection role :                        source_rv
        cfo_rv1 connection state :                      NOT CONNECTED
        cfo_rv1 nic used :                              1
        cfo_rv1 connection role :                       none
        cfo_rv2 connection state :                      NOT CONNECTED
        cfo_rv2 nic used :                              1
        cfo_rv2 connection role :                       not applicable


dersan04*> ic config show

Error: Command not supported on this platform.

 

I get the feeling there is a misconfig on node02 but ha-config for both chassis and controller is set to ha, psm-cc-config is unset and the partner system ID's are correct.

 

What am I missing, crying in pain over here, both nodes running 8.2.4P6, confirmed part numbers between the 2 controllers are the same but the revision number is different.

 

Thanks in advance.

 

John

1 ACCEPTED SOLUTION

TMACMD
5,988 Views

If possible, start over.

Boot both into maintenance mode.

Make sure both are ha for the the chassis and controller.

If not, use the ha-config command to set both sides to ha.

Then at the loader prompt on both sides:

set-defaults

Follow the directions in this link: https://kb.netapp.com/app/answers/answer_view/a_id/1030427

 

Hopefully that helps

 

 

View solution in original post

5 REPLIES 5

TMACMD
6,005 Views

For kicks...

 

when you re-initialized the system, at the LOADER prompt, did you "set-defaults"?

Maybe there is a ENV VAR set that is messing things up?

 

From MAINT mode, you did verify that "ha-config show" on both nodes looks correct?

For controller and chassis (ha) ?

John_Blackwell
5,992 Views

I didnt, that is a good thing to try though,  once I have done that do I have to reset the partner ID?

 

Yes, ha looks set correct to me

*> ha-config show
Chassis HA configuration: ha
Controller HA configuration: ha


Attached are the printenv from each node, I dont see anything suspecious.

 

To me it looks like node2 is trying to use the internal chassis HA connections opposed to c0a/c0b but I dont know how to get it to change

TMACMD
5,989 Views

If possible, start over.

Boot both into maintenance mode.

Make sure both are ha for the the chassis and controller.

If not, use the ha-config command to set both sides to ha.

Then at the loader prompt on both sides:

set-defaults

Follow the directions in this link: https://kb.netapp.com/app/answers/answer_view/a_id/1030427

 

Hopefully that helps

 

 

John_Blackwell
5,982 Views

Thanks for this, yes I will give it a go tomorrow and let you know how I get on.

 

John

John_Blackwell
5,696 Views

Turns out there was an FCVI card in slot 1 which was taking precident over the onboard cards, removed the card and all started working.

 

Cheers for your help

Public