ONTAP 9.4 2750 controllers :: New Switched cluster setup failing.

[Rehan88]

Hi,

I have 3 nodes 2750 each with 2 controller. I have 2 netapp switch.

e0a from each controller is connected to Switch A 6 connections and e0b from each controller to Switch B. All connections are solid and on every controller e0a and e0b are populated with internal IP's.

on my first Node 1 controller 1 i created a cluster and Node 1 controller 2 joined without any issues.

Node 2  and 3 controllers  1 and 2  ran cluster setup and join and provided  { e0a IP  Node 1 controller 1 }  and they all failed and this is happening on all 4 controllers. I also tried to do it from GUI Guided setup and failed.

Error stated that node are not rechable, not able to ping.

From Node 2 and 3 from every controller i am able to ping clus1 and clus2 default interconnect lifs cluster  e0a and e0b and e0a and e0b.

Need some help what am i missing ??

Any help will be appreciated.  

[SpindleNinja]

That sounds like it is cabled correctly.   

Are the CN1610s configured correctly including the ISL?   

All link lights showing good? 

And are you using all compatiable Cluster cables or SFPs?  (see hwu.netapp.com for a full list) 

[Rehan88]

its all netapp hw

as I satted i see all the e0a and e0b's are up and have IP.

I am not sure where its goofed up.

Node 1 controller 1 and 2 are clusterred

Node 2 controller 1 and 2  successfull ping to Node 1 controller 1 and 2  clus1 and clus 2 e0a, e0b,

Node 3 controller 1 and 2 successfull ping to Node 1 controller 1 and 2  clus1 and clus 2 e0a, e0b,

I am not able to ping from Node 1 controller 1 and 2 to. It asked me to select the node  

Node 2 controller 1 and 2   e0a, e0b,

Node 3 controller 1 and 2  e0a, e0b,

How can i break or remove or un cluster Node 1 controller 1 and 2 and start fresh or anu other hint.

Thanks

[SpindleNinja]

Check this setting also "network options switchless-cluster show"

It should be set to false, if not modity it and try again.   

As far as starting over...   If there's no data on them you could just option 4 in the boot menu and start over.     

Create the cluster using cluster setup wizard. 

 https://docs.netapp.com/ontap-9/index.jsp?topic=%2Fcom.netapp.doc.dot-cm-cmpr-950%2Fcluster__setup.html&cp=3_1_10_7

After you create the cluster on 01,  join 02 using 01's IP address.     Join 03 using an 169.x address off 01, do the same for 04, and then 05&06.  

Just for clarifaction of terms;  The terms node and controller tend to be used interchangeable i've noticed.   

You have 3 HA pairs and 6 controllers/nodes.   When the cluster is fully setup you will have nodes  01 & 02  then add 03 & 04 and lastly 05 & 06.   

[Rehan88]

Thanks for you help,

switchless is set to false.

I agree and understand exactly what you have explained.

There is no data so i think i have to restart from fresh. so now

I am new to FAS is there a way to reboot and how do i break to a boot menu ?

controller 1 reboot use option 4

controller 2 reboot use option 4 cluster setup join 

controller 3 cluster setup join    ( do i have to do option 4 on all the rest of the controllers)

controller 4 cluster setup join 

controller 5 cluster setup join 

controller 6 cluster setup join 

[SpindleNinja]

reboot will reboot it.   

halt will reboot and park it at LOADER> 

You will see two "Press control-c"  during boot.  You will want to press ctrl-c on the second one for the boot menu.   

If you haven't joined the nodes, I don't think you need to run opt 4.  

It's really odd that they can't see each other.   Do you know what version of the RFC the CN1610s are running?  

https://mysupport.netapp.com/NOW/download/software/sanswitch/fcp/NetApp/cn1610cm/download.shtml

[Rehan88]

not sure of the version of the RFC the CN1610s 

I will start the process on controller 1 and 2 option 4

on the rest of the controllers I will not

Thanks for the help

[Rehan88]

Performed

Controller 1 reboot used Option 4 create cluster successfull

Controller 2 reboot used Option 4 Join cluster successfull

Controller 3 reboot used Option 4  Join cluster failed used controller clus1  e0a

Controller 4 reboot used Option 4 Join cluster failed used controller clus 1 e0a 

Controller 5 reboot used Option 4 Join cluster failed used controller clus 1 e0a 

Controller 6 reboot used Option 4 Join cluster failed used controller clus 1 e0a 

Question :: We have Switch A and B :: From all controllers e0a goes to Switch A and e0b goes to Switch B

I see in the error logs that controller 3 e0a IP failed to ping Controller 1 e0b. I think it failed because both switchs are not interconnected.  Do we need to interconnect the switch ?  

From Controller 3, 4 ,5 and 6 ping successfull to Controller clus 1 e0a and clus 2 e0b

Totally lost here.  

[SpindleNinja]

Yes,  there is a pre-configured ISL between the two CN1610s using the last 4 ports on the switch.   You should have 4 short twinax cables to connect them.   (Ports 13,14,15,16) 

Cabling example:  

https://library.netapp.com/ecmdocs/ECMP1197116/html/GUID-3F4B53E3-C5D4-40D8-ABF3-4FD8AFD6D10D.html

and ISL: 

https://library.netapp.com/ecmdocs/ECMP1197116/html/GUID-29E2812B-1357-4FB8-A117-EDB72DB123CF.html

[Rehan88]

I think we got it. Will make those connections now and will update.

Thanks for the help  

[Rehan88]

All good here. as sson ISL connections are made took me 10 min to finish up the cluster setup.

Appreciate your help.

Thanks  

[SpindleNinja]

No problem!