Metrocluster IP setup

Gmox — Fri, 28 Mar 2025 11:31:22 GMT

Hello,

To have better knowledge/understanding about MCCIP, i try to configure one in my lab.

I use AFF A220 HA pair and a Lenovo DM5000H HA pair + BES53248 switchs => It is only for test.

All node/ chassis put in mccip
intercluster lif are ok
cluster peering ok

A220_MCC1::> net int show
(network interface show)
Logical Status Network Current Current Is
Vserver Interface Admin/Oper Address/Mask Node Port Home
----------- ---------- ---------- ------------------ ------------- ------- ----
A220_MCC1
A200_MCC1-01_icl01
up/up 192.168.0.101/16 A220_MCC1-01 e0c true
A200_MCC1-01_icl02
up/up 192.168.0.102/16 A220_MCC1-01 e0d true
A200_MCC1-02_icl01
up/up 192.168.0.103/16 A220_MCC1-02 e0c true
A200_MCC1-02_icl02
up/up 192.168.0.104/16 A220_MCC1-02 e0d true
A220_MCC1-01_mgmt1
up/up 10.72.12.150/21 A220_MCC1-01 e0M true
A220_MCC1-02_mgmt1
up/up 10.72.12.151/21 A220_MCC1-02 e0M true
cluster_mgmt up/up 10.72.12.170/21 A220_MCC1-01 e0M true
Cluster
A220_MCC1-01_clus1
up/up 169.254.241.189/16 A220_MCC1-01 e0a true
A220_MCC1-01_clus2
up/up 169.254.116.19/16 A220_MCC1-01 e0b true
A220_MCC1-02_clus1
up/up 169.254.33.121/16 A220_MCC1-02 e0a true
A220_MCC1-02_clus2
up/up 169.254.129.164/16 A220_MCC1-02 e0b true
11 entries were displayed.

A220_MCC1::> net port show
(network port show)

Node: A220_MCC1-01
Speed(Mbps) Health
Port IPspace Broadcast Domain Link MTU Admin/Oper Status
--------- ------------ ---------------- ---- ---- ----------- --------
e0M Default Default up 1500 auto/1000 healthy
e0a Cluster Cluster up 9000 auto/10000 healthy
e0b Cluster Cluster up 9000 auto/10000 healthy
e0c Default Default up 1500 auto/10000 healthy
e0d Default Default up 1500 auto/10000 healthy
e0e Default Default down 1500 auto/- -
e0f Default Default down 1500 auto/- -

Node: A220_MCC1-02
Speed(Mbps) Health
Port IPspace Broadcast Domain Link MTU Admin/Oper Status
--------- ------------ ---------------- ---- ---- ----------- --------
e0M Default Default up 1500 auto/1000 healthy
e0a Cluster Cluster up 9000 auto/10000 healthy
e0b Cluster Cluster up 9000 auto/10000 healthy
e0c Default Default up 1500 auto/10000 healthy
e0d Default Default up 1500 auto/10000 healthy
e0e Default Default down 1500 auto/- -
e0f Default Default down 1500 auto/- -
14 entries were displayed.

A220_MCC1::> cluster peer show
Peer Cluster Name Cluster Serial Number Availability Authentication
------------------------- --------------------- -------------- --------------
DM5000H_MCC2 1-80-000011 Available ok

DM5000H_MCC2::> net int show
(network interface show)
Logical Status Network Current Current Is
Vserver Interface Admin/Oper Address/Mask Node Port Home
----------- ---------- ---------- ------------------ ------------- ------- ----
Cluster
DM5000H_MCC2-01_clus1
up/up 169.254.161.255/16 DM5000H_MCC2-01
e0a true
DM5000H_MCC2-01_clus2
up/up 169.254.44.35/16 DM5000H_MCC2-01
e0b true
DM5000H_MCC2-02_clus1
up/up 169.254.248.44/16 DM5000H_MCC2-02
e0a true
DM5000H_MCC2-02_clus2
up/up 169.254.132.80/16 DM5000H_MCC2-02
e0b true
DM5000H_MCC2
DM5000H_MCC2-01_icl01
up/up 192.168.0.201/16 DM5000H_MCC2-01
e0c true
DM5000H_MCC2-01_icl02
up/up 192.168.0.202/16 DM5000H_MCC2-01
e0d true
DM5000H_MCC2-01_mgmt_auto
up/up 10.72.12.166/21 DM5000H_MCC2-01
e0M true
DM5000H_MCC2-02_icl01
up/up 192.168.0.203/16 DM5000H_MCC2-02
e0c true
DM5000H_MCC2-02_icl02
up/up 192.168.0.204/16 DM5000H_MCC2-02
e0d true
DM5000H_MCC2-02_mgmt1
up/up 10.72.12.167/21 DM5000H_MCC2-02
e0M true
cluster_mgmt up/up 10.72.12.178/21 DM5000H_MCC2-01
e0M true
11 entries were displayed.

DM5000H_MCC2::> net port show
(network port show)

Node: DM5000H_MCC2-01
Speed(Mbps) Health
Port IPspace Broadcast Domain Link MTU Admin/Oper Status
--------- ------------ ---------------- ---- ---- ----------- --------
e0M Default Default up 1500 auto/1000 healthy
e0a Cluster Cluster up 9000 auto/10000 healthy
e0b Cluster Cluster up 9000 auto/10000 healthy
e0c Default Default up 1500 auto/10000 healthy
e0d Default Default up 1500 auto/10000 healthy
e0e Default Default down 1500 auto/- -
e0f Default Default down 1500 auto/- -

Node: DM5000H_MCC2-02
Speed(Mbps) Health
Port IPspace Broadcast Domain Link MTU Admin/Oper Status
--------- ------------ ---------------- ---- ---- ----------- --------
e0M Default Default up 1500 auto/1000 healthy
e0a Cluster Cluster up 9000 auto/10000 healthy
e0b Cluster Cluster up 9000 auto/10000 healthy
e0c Default Default up 1500 auto/10000 healthy
e0d Default Default up 1500 auto/10000 healthy
e0e Default Default down 1500 auto/- -
e0f Default Default down 1500 auto/- -
14 entries were displayed.

DM5000H_MCC2::> cluster peer show
Peer Cluster Name Cluster Serial Number Availability Authentication
------------------------- --------------------- -------------- --------------
A220_MCC1 1-80-000011 Available ok

DR-group created:

A220_MCC1::> dr-group show
(metrocluster configuration-settings dr-group show)
DR Group ID Cluster Node DR Partner Node
----------- -------------------------- ------------------ ------------------
1 A220_MCC1
A220_MCC1-02 DM5000H_MCC2-02
A220_MCC1-01 DM5000H_MCC2-01
DM5000H_MCC2
DM5000H_MCC2-02 A220_MCC1-02
DM5000H_MCC2-01 A220_MCC1-01
4 entries were displayed.

But when i try to create the interconnect, i've got issues:

A220_MCC1::> metrocluster configuration-settings show
Cluster Node Configuration Settings Status
-------------------------- ------------------ ---------------------------------
A220_MCC1
A220_MCC1-01 connection error
Error: Did not receive a "ping" response using the network
addresses "10.1.1.1" and "10.1.1.3" on node "A220_MCC1-01" in
cluster "A220_MCC1".
A220_MCC1-02 connection error
Error: Did not receive a "ping" response using the network
addresses "10.1.1.2" and "10.1.1.4" on node "A220_MCC1-02" in
cluster "A220_MCC1".
DM5000H_MCC2
DM5000H_MCC2-01 connection error
Error: Did not receive a "ping" response using the network
addresses "10.1.1.3" and "10.1.1.4" on node "DM5000H_MCC2-01" in
cluster "DM5000H_MCC2".
DM5000H_MCC2-02 connection error
Error: Did not receive a "ping" response using the network
addresses "10.1.1.4" and "10.1.1.3" on node "DM5000H_MCC2-02" in
cluster "DM5000H_MCC2".
4 entries were displayed.

A220_MCC1::> metrocluster configuration-settings interface show
DR Config
Group Cluster Node Network Address Netmask Gateway State
----- ------- ------- --------------- --------------- --------------- ---------
1 A220_MCC1
A220_MCC1-02
Home Port: e0a-10
10.1.1.2 255.255.255.0 - completed
Home Port: e0b-20
10.1.2.2 255.255.255.0 - completed
A220_MCC1-01
Home Port: e0a-10
10.1.1.1 255.255.255.0 - completed
Home Port: e0b-20
10.1.2.1 255.255.255.0 - completed
DM5000H_MCC2
DM5000H_MCC2-02
Home Port: e0a-10
10.1.1.4 255.255.255.0 - completed
Home Port: e0b-20
10.1.2.4 255.255.255.0 - completed
DM5000H_MCC2-01
Home Port: e0a-10
10.1.1.3 255.255.255.0 - completed
Home Port: e0b-20
10.1.2.3 255.255.255.0 - completed
8 entries were displayed.

A220_MCC1::> sto fail show
(storage failover show)
Takeover
Node Partner Possible State Description
-------------- -------------- -------- -------------------------------------
A220_MCC1-01 A220_MCC1-02 false Waiting for A220_MCC1-02, Takeover
is not possible: Storage failover
interconnect error, NVRAM log not
synchronized, Disk inventory not
exchanged
A220_MCC1-02 A220_MCC1-01 false Waiting for A220_MCC1-01, Takeover
is not possible: Storage failover
interconnect error, NVRAM log not
synchronized, Disk inventory not
exchanged
2 entries were displayed.

I guess i'm missing something but don't know what.

If anybody have a clue or hint here it will be really helpfull.

edit:

A220_MCC1::> metrocluster interconnect adapter show
Adapter Link
Node Adapter Name Type Status IP Address Port Number
------------ --------------- ------- ------ ----------- -----------
A220_MCC1-01 e0a-10 iWARP UP 10.1.1.1 e0a-10
A220_MCC1-01 e0b-20 iWARP UP 10.1.2.1 e0b-20
A220_MCC1-02 e0a-10 iWARP UP 10.1.1.2 e0a-10
A220_MCC1-02 e0b-20 iWARP UP 10.1.2.2 e0b-20
4 entries were displayed.

DM5000H_MCC2::> metrocluster interconnect adapter show
Adapter Link
Node Adapter Name Type Status IP Address Port Number
------------ --------------- ------- ------ ----------- -----------
DM5000H_MCC2-01
e0a-10 iWARP UP 10.1.1.3 e0a-10
DM5000H_MCC2-01
e0b-20 iWARP UP 10.1.2.3 e0b-20
DM5000H_MCC2-02
e0a-10 iWARP UP 10.1.1.4 e0a-10
DM5000H_MCC2-02
e0b-20 iWARP UP 10.1.2.4 e0b-20
4 entries were displayed.

Re: Metrocluster IP setup

Sanaman — Tue, 01 Apr 2025 00:38:37 GMT

Did you check your cluster switch ports ( BES53248 switches) configuration? Not seeing any outputs from them.

Re: Metrocluster IP setup

chenguanghui — Tue, 01 Apr 2025 08:48:59 GMT

hello:

The MetroCluster IP HA within the cluster has an unconnected ISL. Check and correct the cabling and/or configuration of the MetroCluster IP interconnect ports. The following article is hoped to be of assistance to you.

https://kb.netapp.com/on-prem/ontap/mc/MC-KBs/MetroCluster_IP_prompt_Storage_failover_interconnect_error_NVRAM_log_not_synchronized_Disk_inventory_not_exchanged

Best regards.

Re: Metrocluster IP setup

Gmox — Tue, 01 Apr 2025 15:12:08 GMT

Hello,

Than you both for your reply.

I agree, pretty sure it is related to switch config/cabling.

As i use only 2 BES53248 i try to modify the config after applying the RCF file.

| * NetApp Reference Configuration File (RCF)
| *
| * Switch : BES-53248 (direct storage, L2 Networks, direct ISL)
| * Filename : BES-53248_v1.33_Switch-A1.txt
| * Platforms : FAS2750, AFF-A220
| * Date : Generator: v1.8a 2025-01-22_001, file creation: 2025-03-31, 15:
> 26:51
| *
| * Port Usage:
| * Ports 01 - 02: Intra-Cluster Ports
| * Ports 03 - 04: Unused ports
| * Ports 05 - 06: MetroCluster Ports, VLAN 10
| * Ports 07 - 08: Unused ports
| * Ports 09 - 10: Cluster/MetroCluster Ports, VLAN 1,10
| * Ports 11 - 12: Unused ports
| * Ports 13 - 16: MetroCluster ISL, VLAN 10, LAG 1/10, 10G / 25G
| * Ports 17 - 48: Unused Ports, 10G / 25G
| * Ports 49 - 54: Unused Ports, 100G
| * Ports 55 - 56: Intra-Cluster ISL Ports, LAG 1/1, 100G
| *****************************************************************************
> *
End of banner message from server
******************************************************************************
* NetApp Reference Configuration File (RCF)
*
* Switch : BES-53248 (direct storage, L2 Networks, direct ISL)
* Filename : BES-53248_v1.33_Switch-A1.txt
* Platforms : FAS2750, AFF-A220
* Date : Generator: v1.8a 2025-01-22_001, file creation: 2025-03-31, 15:26:51
*
* Port Usage:
* Ports 01 - 02: Intra-Cluster Ports
* Ports 03 - 04: Unused ports
* Ports 05 - 06: MetroCluster Ports, VLAN 10
* Ports 07 - 08: Unused ports
* Ports 09 - 10: Cluster/MetroCluster Ports, VLAN 1,10
* Ports 11 - 12: Unused ports
* Ports 13 - 16: MetroCluster ISL, VLAN 10, LAG 1/10, 10G / 25G
* Ports 17 - 48: Unused Ports, 10G / 25G
* Ports 49 - 54: Unused Ports, 100G
* Ports 55 - 56: Intra-Cluster ISL Ports, LAG 1/1, 100G
******************************************************************************

Cluster port for ha pair and MCCIP interconnect use port e0a and e0b /e0a-10 e0b-20

On which port of the switch do i need to connect them?

Re: Metrocluster IP setup

Gmox — Mon, 07 Apr 2025 06:37:42 GMT

Just for knowledge, it was caused by a configuration error in the IP switch.

MCCIP use VLAN e0a-10 and e0b-10.

I modified the conf in the switch and all is good.

topic Metrocluster IP setup in ONTAP Discussions

Metrocluster IP setup

Re: Metrocluster IP setup

Re: Metrocluster IP setup

Re: Metrocluster IP setup

Re: Metrocluster IP setup