Intercluster Connections For Peer

[pcmssysadmin]

Hi I currently have 2 clusters in different sites on networks that we fully control and I have created the peer connections successfully using a vlan718. We have now got a third site which is managed by the customer and to which we have very little access we have a vlan719 that has been set up and we have been given IP address for both of our filers.

If I create the intercluster lifs with the new vlan tag and IP's on the existing filers the current peer connection can only connect on half the lifs so goes from available to partial and the new filer will send the peer offer to the existing clusters but if you try to send a peer create back it times out with

TA ONTAP API failed: An Introductory RPC to the peer address "<IP ADDRESS>" failed to connect RPC:Remote system error - operation timed out verify that the peer address is correct and try again

Is it possible to have multiple sets of intercluster lifs on the cluster each on different vlans/networks I am guessing if it is then some sort of routing needs to be set up but I have not yet found a way to get this to work. All of the clusters are running 8.3.

[scottgelb]

What is your routing output for the admin (cluster) SVM?  "net route show"?  All intercluster lifs must see each other in the mesh to work.. vlan718 and vlan719 are routable to each other for the connection with a network route on the cluster?

[pcmssysadmin]

The network routing for the svm is just how it comes by default as the networks are added.

PGUKCSTGCL01
                    0.0.0.0/0       10.210.11.1     10
                    0.0.0.0/0       10.255.3.33     20
                    0.0.0.0/0       10.255.3.65     20

[scottgelb]

There are routes on both clusters to each other?

[pcmssysadmin]

Unfortunately the new third filer is currently powered down until it is put into a proper rack next week.

Our onsite and offsite filers both have the same routing config (apart for the management network being different).

PGUKCSTGCL01
                    0.0.0.0/0       10.210.11.1     10
                    0.0.0.0/0       10.255.3.33     20
                    0.0.0.0/0       10.255.3.65     20

PGUKLRBSTGCL01
                    0.0.0.0/0       10.212.11.1     10
                    0.0.0.0/0       10.255.3.33     20
                    0.0.0.0/0       10.255.3.65     20

The new filer only has the management network route and '0.0.0.0/0       10.255.3.65     20' as it currently knows nothing about the 10.255.3.33 network

[Hlynur]

Have you stumbled upon a solution to this problem? I'm having a very similar issue.

I can ping and traceroute to the cluster I'm trying to peer with, but when I try to establish a peer connection I get a "no route to host" error.

network ping 10.x.x.11
10.x.x.11 is alive

cluster peer create -peer-addrs 10.x.x.11,10.x.x.12

Notice: Choose a passphrase of 8 or more characters. To ensure the authenticity of the peering relationship, use a phrase or sequence of characters that would be hard to guess.

Enter the passphrase:
Confirm the passphrase:

Error: command failed: An introductory RPC to the peer address "10.x.x.11" failed to connect: RPC: Remote system error - No route to host. Verify that the peer address is correct and try again.

[PaulN]

Old thread, but since I just had a similar issue and had found this thread looking for a solution I figured I'd add in what resolved my issue.  We had a firewall in between the 2 clusters and it didn't have the proper ports open.  I had to make sure 443, 10000, 11104, and 11105 were all open between the 2 clusters.

Firewall requirements

Firewalls and the intercluster firewall policy must allow the following:

• ICMP service
• TCP to the IP addresses of all of the intercluster LIFs over all of the following ports: 10000, 11104, and 11105
• HTTPS

  Although HTTPS is not required when you set up cluster peering, HTTPS is required later if you use OnCommand System Manager to configure data protection. However, if you use the command-line interface to configure data protection, HTTPS is not required to configure cluster peering or data protection.

[mujahed]

Hi Paul,

I also have the similar issue, when you say you have firewall issue between two clusters. does it mean that firewall issue between cluster management LIFs or between the ICLs of each clusters. 

---

@PaulN wrote:

Old thread, but since I just had a similar issue and had found this thread looking for a solution I figured I'd add in what resolved my issue.  We had a firewall in between the 2 clusters and it didn't have the proper ports open.  I had to make sure 443, 10000, 11104, and 11105 were all open between the 2 clusters.

Firewall requirements

Firewalls and the intercluster firewall policy must allow the following:

• ICMP service
• TCP to the IP addresses of all of the intercluster LIFs over all of the following ports: 10000, 11104, and 11105
• HTTPS

  Although HTTPS is not required when you set up cluster peering, HTTPS is required later if you use OnCommand System Manager to configure data protection. However, if you use the command-line interface to configure data protection, HTTPS is not required to configure cluster peering or data protection.

---

[Emile-Bodin]

Hi mujahed,

The firewall ports between the ICL need to be opened.

You can check this Kb for reference: https://kb.netapp.com/support/index?page=content&id=3014253

Regards,

Emile