topic Re: CDOT e0M / Service Processor not working in ONTAP Discussions

CDOT e0M / Service Processor not working

axsys — Wed, 04 Jun 2025 23:53:18 GMT

Hey Guys,

We've got a new FAS2552 with CDOT 8.3 and I'm not able to get the e0M or the Service Processor to work.

As they weren't working at first, I've setup "cluster_mgmt" on the 10GB trunked lif with vlan 3 (which is the same network e0M / Service Processor are in) so I could at least setup the Filer.

Now as everything is working fine and I've gone through setting the filer up, I wanted to get the SP / e0M working and removed the vlan3 assosiation for "cluster_mgmt". Now I'm only able to work with the serial console but both the SP & e0M are still not reachable.

Switchport Setup
interface GigabitEthernet1/0/21
description filername1-mgmt
switchport access vlan 3
switchport mode access
no logging event link-status
no snmp trap link-status
end

interface GigabitEthernet1/0/22
description filername2-mgmt
switchport access vlan 3
switchport mode access
no logging event link-status
no snmp trap link-status
end

sh mac address-table | incl Gi1/0/21
   3    00a0.9882.4798    DYNAMIC     Gi1/0/21
   3    00a0.9882.4799    DYNAMIC     Gi1/0/21
sh mac address-table | incl Gi1/0/22
   3    00a0.9882.4630    DYNAMIC     Gi1/0/22
   3    00a0.9882.4631    DYNAMIC     Gi1/0/22

net port show -fields node,port,mac
(network port show)
node port mac
------------- ---- -----------------
filername1 a0a 02:a0:98:82:47:95
filername1 a0a-3 02:a0:98:82:47:95
filername1 a0a-310 02:a0:98:82:47:95
filername1 a0a-312 02:a0:98:82:47:95
filername1 a0a-315 02:a0:98:82:47:95
filername1 a0a-317 02:a0:98:82:47:95
filername1 a0a-351 02:a0:98:82:47:95
filername1 a0a-360 02:a0:98:82:47:95
filername1 a0a-399 02:a0:98:82:47:95
filername1 e0M 00:a0:98:82:47:97
filername1 e0a 00:a0:98:82:47:95
filername1 e0b 00:a0:98:82:47:96
filername1 e0c 00:a0:98:82:47:9b
filername1 e0d 00:a0:98:82:47:9c
filername1 e0e 00:a0:98:82:47:a1
filername1 e0f 00:a0:98:82:47:a2
filername2 a0a 02:a0:98:82:46:2d
filername2 a0a-3 02:a0:98:82:46:2d
filername2 a0a-310 02:a0:98:82:46:2d
filername2 a0a-312 02:a0:98:82:46:2d
filername2 a0a-315 02:a0:98:82:46:2d
filername2 a0a-317 02:a0:98:82:46:2d
filername2 a0a-351 02:a0:98:82:46:2d
filername2 a0a-360 02:a0:98:82:46:2d
filername2 a0a-399 02:a0:98:82:46:2d
filername2 e0M 00:a0:98:82:46:2f
filername2 e0a 00:a0:98:82:46:2d
filername2 e0b 00:a0:98:82:46:2e
filername2 e0c 00:a0:98:82:46:33
filername2 e0d 00:a0:98:82:46:34
filername2 e0e 00:a0:98:82:46:39
filername2 e0f 00:a0:98:82:46:3a

system service-processor show -instance

                               Node: filername1
                     Type of Device: SP
                             Status: online
              Is Network Configured: true
                  Public IP Address: 172.20.189.180
                        MAC Address: 00:a0:98:82:47:99
                   Firmware Version: 2.3
                        Part Number: unknown
                      Serial Number: unknown
                    Device Revision: Not Applicable
     Is Firmware Autoupdate Enabled: true

                               Node: filername2
                     Type of Device: SP
                             Status: online
              Is Network Configured: true
                  Public IP Address: 172.20.189.181
                        MAC Address: 00:a0:98:82:46:31
                   Firmware Version: 2.3
                        Part Number: unknown
                      Serial Number: unknown
                    Device Revision: Not Applicable
     Is Firmware Autoupdate Enabled: true

This should actually prove that the SP is really connected to Gi1/0/21 + 22. What I find weird is that the mac for both the e0M interfaces do not match with what I see on the Switch.
I suspect a problem there but don't know for sure? Anyway a ping to that subnet's gateway (which is infact my default gw) does not reply when going over the node's mgmt ports:

filercluster01::> route show
Vserver             Destination     Gateway         Metric
------------------- --------------- --------------- ------
filercluster01
                    0.0.0.0/0       172.20.189.1    20
                    172.18.0.0/24   172.20.160.46   20

network inte show
            Logical    Status     Network            Current       Current Is
Vserver     Interface Admin/Oper Address/Mask       Node          Port    Home
----------- ---------- ---------- ------------------ ------------- ------- ----
Cluster
            filerclustername1-01_clus1 up/up 169.254.60.145/16 filername1 e0e true
            filerclustername1-01_clus2 up/up 169.254.162.113/16 filername1 e0f true
            filerclustername1-02_clus1 up/up 169.254.191.207/16 filername2 e0e true
            filerclustername1-02_clus2 up/up 169.254.96.71/16 filername2 e0f true
filerclustername1
            filerclustername1-01_mgmt1 up/up 172.20.189.41/24 filername1 e0M true
            filerclustername1-02_mgmt1 up/up 172.20.189.42/24 filername2 e0M true
            cluster_mgmt up/up    172.20.175.40/24   filername1 a0a-360 true
            icl_1        up/up    172.20.160.41/29   filername1 e0a     true
vserveraxn_caru
            lif_vserveraxn_caru_sn310 up/up 172.20.164.81/24 filername1 a0a-310 true
            lif_vserveraxn_caru_sn312 up/up 172.20.165.81/24 filername1 a0a-312 true
            lif_vserveraxn_caru_sn351 up/up 172.20.169.81/24 filername1 a0a-351 true
vserveraxn_p2ru
            lif_vserveraxn_p2ru_sn310 up/up 172.20.164.82/24 filername1 a0a-310 true
            lif_vserveraxn_p2ru_sn315 up/up 172.20.166.82/24 filername1 a0a-315 true
            lif_vserveraxn_p2ru_sn317 up/up 172.20.167.82/24 filername1 a0a-317 true
vserveraxn_pru
            lif_vserveraxn_pru_sn310 up/up 172.20.164.80/24 filername1 a0a-310 true
            lif_vserveraxn_pru_sn312 up/up 172.20.165.80/24 filername1 a0a-312 true
            lif_vserveraxn_pru_sn351 up/up 172.20.169.80/24 filername1 a0a-351 true
vserverdist
            lif_vserverdist_sn310 up/up 172.20.164.70/24 filername1 a0a-310 true
            lif_vserverdist_sn312 up/up 172.20.165.70/24 filername1 a0a-312 true
            lif_vserverdist_sn315 up/up 172.20.166.70/24 filername1 a0a-315 true
            lif_vserverdist_sn317 up/up 172.20.167.70/24 filername1 a0a-317 true
            lif_vserverdist_sn351 up/up 172.20.169.70/24 filername1 a0a-351 true
vserveresx
            lif_vserveresx_sn399 up/up 192.168.0.80/24 filername1 a0a-399 true

net ping -node filername1 172.20.189.1
(network ping)
no answer from 172.20.189.1

net ping -node filername2 172.20.189.1
(network ping)
no answer from 172.20.189.1

So I went down to the system node, to check out the routing table there and try a ping from there, unfortunately with no luck:

filername1> route -s
Routing tables

Routing group: __default_grp_v4
Routing group-id: 0 (CMVFID 0 REFCNT 1)

Internet:
Destination      Gateway            Flags     Refs     Use Interface
localhost        localhost          UH          1        0 lo
localhost-10     localhost-20       UHS         1        0 losk
localhost-20     localhost-20       UH          1        0 losk

Routing group: __default_grp_v4 (vsid: 4294967295)
Routing group-id: 4096 (CMVFID 1 REFCNT 1)

Internet:
Destination      Gateway            Flags     Refs     Use Interface
localhost        localhost          UHS         1        0 lo
localhost-10     localhost-20       UHS         1        0 losk
localhost-20     localhost-20       UHS         1        0 losk

Routing group: 4294967295_172.20.189.0/24
Routing group-id: 4097 (CMVFID 1 REFCNT 1)

Internet:
Destination      Gateway            Flags     Refs     Use Interface
default          172.20.189.1       UGS         1   105942 e0M
172.20.189/24    link#7             UC          1        0 e0M
172.20.189.181   0:a0:98:82:46:31   UHL         1        0 e0M

If anyone has any ideas howelse I could check this? Please let me know!

Re: CDOT e0M / Service Processor not working

aborzenkov — Mon, 06 Jul 2015 18:12:51 GMT

Looks like you connected to the wrong (ACP) port which is marked with crossed wrench. You need to connect to non-crossed one.

Re: CDOT e0M / Service Processor not working

axsys — Tue, 07 Jul 2015 08:52:11 GMT

Hey aborzenkov,

Really? Why would I see a Full Connectivity in the ACP setup if it was cabled wrong?

filernamecluster1::> system node run -node filername1 sysconfig -a
        NetApp Release 8.3: Mon Mar 9 19:20:57 PDT 2015
        System ID: (filername1); partner ID: (filername2)
        System Serial Number: (filername1)
        System Rev: C1
        System Storage Configuration: Multi-Path HA
        System ACP Connectivity: Full Connectivity
        All-Flash Optimized: false
        Backplane Part Number: DS224
        Backplane Rev:
        Backplane Serial Number:
        slot 0: System Board 1.7 GHz (System Board XIX C1)
                Model Name:         FAS2552
                Part Number:        111-01324
                Revision:           C1
                Serial Number:
                BIOS version:       8.3.0
                Loader version:     4.3
                Processors:         4
                Processor ID:       0x106e4
                Microcode Version: 0x3
                Processor type:     Intel(R) Xeon(R) CPU           C3528 @ 1.73GHz
                Memory Size:        18432 MB
                Memory Attributes: Hoisting
                                    Normal ECC
                NVMEM Size:         1280 MB of Main Memory Used
                CMOS RAM Status:    OK
                Controller:         B
        Service Processor           Status: Online
                Firmware Version:   2.3
                Mgmt MAC Address:   00:A0:98:82:47:99
                Ethernet Link:      up, 100Mb, full duplex, auto-neg complete
                Using DHCP:         no
        IPv4 configuration:
                IP Address:         172.20.189.180
                Netmask:            255.255.255.0
                Gateway:            172.20.189.1
        IPv6 configuration:         Disabled
        slot 0: Internal 10/100/1000 Ethernet Switch
                Firmware Version:   Version4 SB_XX Switch0
                Wrench:             auto-100tx-fd-up
                Locked-wrench:      auto-1000t-fd-up
                Device Type:        88E6176
                        memory mapped I/O base 0x00000000dfd00000, size 0x80000
                        I/O base 0x0000000000002040, size 0x20
                        memory mapped I/O base 0x00000000dfe08000, size 0x4000
                        memory mapped I/O base 0x00000000dfd80000, size 0x80000
                        I/O base 0x0000000000002060, size 0x20
                        memory mapped I/O base 0x00000000dfe0c000, size 0x4000

filernamecluster1::> system node run -node filername2 sysconfig -a
        NetApp Release 8.3: Mon Mar 9 19:20:57 PDT 2015
        System ID: (filername2); partner ID: (filername1)
        System Serial Number: (filername2)
        System Rev: C1
        System Storage Configuration: Multi-Path HA
        System ACP Connectivity: Full Connectivity
        All-Flash Optimized: false
        Backplane Part Number: DS224
        Backplane Rev:
        Backplane Serial Number:
        slot 0: System Board 1.7 GHz (System Board XIX C1)
                Model Name:         FAS2552
                Part Number:        111-01324
                Revision:           C1
                Serial Number:
                BIOS version:       8.3.0
                Loader version:     4.3
                Processors:         4
                Processor ID:       0x106e4
                Microcode Version: 0x3
                Processor type:     Intel(R) Xeon(R) CPU           C3528 @ 1.73GHz
                Memory Size:        18432 MB
                Memory Attributes: Hoisting
                                    Normal ECC
                NVMEM Size:         1280 MB of Main Memory Used
                CMOS RAM Status:    OK
                Controller:         A
        Service Processor           Status: Online
                Firmware Version:   2.3
                Mgmt MAC Address:   00:A0:98:82:46:31
                Ethernet Link:      up, 100Mb, full duplex, auto-neg complete
                Using DHCP:         no
        IPv4 configuration:
                IP Address:         172.20.189.181
                Netmask:            255.255.255.0
                Gateway:            172.20.189.1
        IPv6 configuration:         Disabled
        slot 0: Internal 10/100/1000 Ethernet Switch
                Firmware Version:   Version4 SB_XX Switch0
                Wrench:             auto-100tx-fd-up
                Locked-wrench:      auto-1000t-fd-up
                Device Type:        88E6176
                        memory mapped I/O base 0x00000000dfd00000, size 0x80000
                        I/O base 0x0000000000002040, size 0x20
                        memory mapped I/O base 0x00000000dfe08000, size 0x4000
                        memory mapped I/O base 0x00000000dfd80000, size 0x80000
                        I/O base 0x0000000000002060, size 0x20
                        memory mapped I/O base 0x00000000dfe0c000, size 0x4000

Re: CDOT e0M / Service Processor not working

axsys — Tue, 07 Jul 2015 09:05:50 GMT

Looks like you're right. I just checked the rear view again online. I used a pic I found for the FAS2240 where the ACP is the above port. With FAS2552 it seems netapp switched the ACP port which is now the bottom port. Shame on me. I'll try and let you know if it's that!

Thanks!

axsys