Network set up error - Cluster join request rejected

[VMUNOZ_NTT]

Hi all,

Next week I'm attending a Clustered course, and instead of setup an environment with simulators, I'm installing an environment using an old FAS3070 with an old DS14MK4 and one additional Quad Port NIC per node, for laboratory purposes only.

I created a dedicated VLAN in my switches for cluster interconnect purposes and I configured it with MTU 9000 in the switch ports related to controller ports that I'm using with cluster role, basically e0b & e1b of each controller.

The cluster setup process in the first node goes fine, but joining the second node to the cluster fails: The exact message that I get is:

Network set up .....Error: Cluster join request rejected: Cluster RPC connectivity test failed. Verify that the port MTU values

on all controllers and network switches on the cluster network are set to the same MTU value

However, I have ping visibility of cluster IPs between the nodes, node01 sees node02 and vice versa. I reviewed MTU on switch ports and all are set to 9000.

Any ideas?

[VMUNOZ_NTT]

Update: Reconfigured MTU on switch ports from 9000 to 9014, but without success results. node 2 still can't join the cluster. Ping connectivity between nodes exists but second node can't pass Network set up tests to join the cluster, same error message.

[mrinal]

Hi,

Could you the output of "network port show -role cluster" and "network interface show -role cluster"? My theory is that roles for the ports might have reset to system-default values.

[VMUNOZ_NTT]

Hi Mrinal,

cluster::> net port show -role clust

    (network port show)

                                        Auto-Negot  Duplex     Speed (Mbps)

  Node   Port   Role         Link   MTU Admin/Oper  Admin/Oper Admin/Oper  

  ------ ------ ------------ ---- ----- ----------- ---------- ------------

  cluster-01

         e0b    cluster      up    9000  true/true  full/full   auto/1000

         e1b    cluster      up    9000  true/true  full/full   auto/1000

  2 entries were displayed.

  cluster::> net int show -role cluster

    (network interface show)

              Logical    Status     Network            Current       Current Is

  Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home

  ----------- ---------- ---------- ------------------ ------------- ------- ----

  cluster-01

              clus1        up/up    169.254.169.69/16  cluster-01    e0b     true

              clus2        up/up    169.254.154.91/16  cluster-01    e1b     true

  2 entries were displayed.

And on the second node:

  ::> net port show -role cluster

    (network port show)

                                        Auto-Negot  Duplex     Speed (Mbps)

Node   Port   Role         Link   MTU Admin/Oper  Admin/Oper Admin/Oper

------ ------ ------------ ---- ----- ----------- ---------- ------------

localhost

       e0b    cluster      up    9000  true/true  full/full   auto/1000

       e1b    cluster      up    9000  true/true  full/full   auto/1000

2 entries were displayed.

::> net int show -role cluster

  (network interface show)

            Logical    Status     Network            Current       Current Is

Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home

----------- ---------- ---------- ------------------ ------------- ------- ----

localhost

            clus1        up/-     169.254.154.247/16 localhost     e0b     true

            clus2        up/-     169.254.85.145/16  localhost     e1b     true

2 entries were displayed.

I think all seems ok. I'm using 1Gb ports, but I think don't should be the problem. Maybe could I try setting MTU to 1500? in switches and controllers ports.

Thanks,

[mrinal]

Hi,

You should be using MTU setting of 9000 for the cluster ports. Couple of things...

1. The "Oper Status" of the cluster LIFs is shown as "-". Could you check the lights on the cluster interconnect switch to make sure that there are valid links?

2. Could you send the console logs from the "cluster join" command on the second node?

[VMUNOZ_NTT]

Hi,

Finally I fixed the problem. The troubleshooting steps was:

- Set MTU to 1500 in switch ports and controllers cluster ports to isolate if was a VLAN problem, or only just MTU as the error message showed. The results was successful, the second node joined the cluster without problems.

- Set MTU to 9198 on Cisco switch ports. No health alerts reported by the cluster.

- Set MTU to 9000 on controllers cluster ports. No health alerts reported. All is working!!!

Thanks anyway for your help Mrinal