topic Re: ACP module connection-error after ONTAP 9.1 upgrade in ONTAP Hardware

ACP module connection-error after ONTAP 9.1 upgrade

alessice — Wed, 04 Jun 2025 12:18:01 GMT

Hi,

I have upgrade my FAS2554 from 8.3 to 9.1. All works fine but about every 5 minutes in the Events Log I found this debug messages:

acp.command.connect: Could not establish connection with 192.168.2.145.

I read that starting from ONTAP 9 ACP move from out-of-band to in-band for communications, but I have no idea how to switch or solve for my case.

Here my configurations:

cluster2::> storage shelf acp module show
Node Module Name State
----------------- ----------------- ----------------------
cluster2-node01 1.0.B connection-error
1.0.A connection-error
cluster2-node02 1.0.B connection-error
1.0.A connection-error
4 entries were displayed.

cluster2::> storage shelf acp show
Node Channel Connectivity
------------------ --------------- ----------------------
cluster2-node01 out-of-band full-connectivity
cluster2-node02 out-of-band full-connectivity
2 entries were displayed.

cluster2::> storage shelf acp show -instance

Node: cluster2-node01
Channel: out-of-band
Enable Status: true
Port: e0P
IP Address: 192.168.2.55
Subnet: 192.168.0.1
Netmask: 255.255.252.0
Connection Status: full-connectivity

Node: cluster2-node02
Channel: out-of-band
Enable Status: true
Port: e0P
IP Address: 192.168.1.253
Subnet: 192.168.0.1
Netmask: 255.255.252.0
Connection Status: full-connectivity
2 entries were displayed.

cluster2::> storage shelf acp module show -instance

Node: cluster2-node01
Module Name: 1.0.B
Adapter Name: 0a
MAC Address: 00:a0:98:c6:ea:91
IOM Type: iom6e
Shelf Serial Number: SHJHU1702000451
IP Address: 192.168.2.145
Protocol Version: 2.1.01.22
Firmware Version: 01.22
Assigner ID: 537067641
State: connection-error
Last Contact: 514

Node: cluster2-node01
Module Name: 1.0.A
Adapter Name: 0a
MAC Address: 00:a0:98:c6:ed:91
IOM Type: iom6e
Shelf Serial Number: SHJHU1702000451
IP Address: 192.168.1.145
Protocol Version: 2.1.01.22
Firmware Version: 01.22
Assigner ID: 537067641
State: active
Last Contact: 36

Node: cluster2-node02
Module Name: 1.0.B
Adapter Name: 0b
MAC Address: 00:a0:98:c6:ea:91
IOM Type: iom6e
Shelf Serial Number: SHJHU1702000451
IP Address: 192.168.2.145
Protocol Version: 2.1.01.22
Firmware Version: 01.22
Assigner ID: 537067641
State: connection-error
Last Contact: 514

Node: cluster2-node02
Module Name: 1.0.A
Adapter Name: 0b
MAC Address: 00:a0:98:c6:ed:91
IOM Type: iom6e
Shelf Serial Number: SHJHU1702000451
IP Address: 192.168.1.145
Protocol Version: 2.1.01.22
Firmware Version: 01.22
Assigner ID: 537067641
State: active
Last Contact: 36
4 entries were displayed.

cluster2::> system service-processor show
IP Firmware
Node Type Status Configured Version IP Address
------------- ---- ----------- ------------ --------- -------------------------
cluster2-node01
SP online true 2.4.1P2 192.168.7.210
cluster2-node02
SP online true 2.4.1P2 192.168.7.211
2 entries were displayed.

I read the KB https://kb.netapp.com/app/answers/answer_view/a_id/1029778 but I have no understand if for my setup is sufficient an

storage shelf acp configure -is-enabled true -channel in-band

or can cause problems if firmware or similar are not at the right version.

Thanks

Re: ACP module connection-error after ONTAP 9.1 upgrade

SpindleNinja — Tue, 27 Aug 2019 14:31:30 GMT

you post the output of the following:

storage shelf acp module show -fields module-name,protocol-version,firmware-version,shelf-serial-number,iom-type,state

Re: ACP module connection-error after ONTAP 9.1 upgrade

SpindleNinja — Tue, 27 Aug 2019 14:32:37 GMT

you post the output of the following:

storage shelf acp module show -fields module-name,protocol-version,firmware-version,shelf-serial-number,iom-type,state

Re: ACP module connection-error after ONTAP 9.1 upgrade

alessice — Tue, 27 Aug 2019 14:38:44 GMT

Thanks, here te output:

cluster2::> storage shelf acp module show -fields module-name,protocol-version,firmware-version,shelf-serial-number,iom-type,state
node            module-name protocol-version firmware-version shelf-serial-number iom-type state            
--------------- ----------- ---------------- ---------------- ------------------- -------- ---------------- 
cluster2-node01 1.0.B       2.1.01.22        01.22            SHJHU1702000451     iom6e    connection-error 
cluster2-node01 1.0.A       2.1.01.22        01.22            SHJHU1702000451     iom6e    connection-error 
cluster2-node02 1.0.B       2.1.01.22        01.22            SHJHU1702000451     iom6e    connection-error 
cluster2-node02 1.0.A       2.1.01.22        01.22            SHJHU1702000451     iom6e    connection-error 
4 entries were displayed.

Re: ACP module connection-error after ONTAP 9.1 upgrade

SpindleNinja — Tue, 27 Aug 2019 16:11:46 GMT

based on the firmware levels, doesn't look like you can move to inband.

Just to confirm, Is this just internal disks or is there a shelf attatched?

have you opened a support case with netapp yet?

Re: ACP module connection-error after ONTAP 9.1 upgrade

alessice — Tue, 27 Aug 2019 16:15:18 GMT

This is a single FAS2554 in HA with 24 internal disk, no external shelf.

From ActiveIQ all firmware are marked as update. For now I haven't open a case.

Re: ACP module connection-error after ONTAP 9.1 upgrade

SpindleNinja — Tue, 27 Aug 2019 16:59:00 GMT

have you tried a manual update of the lastest firmwares?

https://mysupport.netapp.com/NOW/cgi-bin/shelffwdnld.cgi/download/tools/diskshelf/bin/all_shelf_fw

Re: ACP module connection-error after ONTAP 9.1 upgrade

GidonMarcus — Thu, 29 Aug 2019 16:54:18 GMT

Hi All

i can't find a reference for it now. but i had similar issue with two FAS2240 also going from 8.3.2 to 9.1 where ACP gone to no-connection. the solution was to turn ACP off and on again.

i think there was a bug open for it, but i can't find it...

Gidi

Re: ACP module connection-error after ONTAP 9.1 upgrade

AlexDawson — Fri, 30 Aug 2019 04:08:04 GMT

BURT ID 1121636

After the Data ONTAP software upgrade from 8.3.x to 9.1.Px, ONTAP might report
 that the Alternate Control Path (ACP) processor on IOM6/IOM6E is no longer
 active, and it is reported as a "connection-error."
 You can verify this issue by using the "storage shelf acp module show" command
 to display the state of all ACP processors on IOM modules.
 cluster1::> storage shelf acp module show
 Node               Module Name        State
 -----------------  -----------------  ----------------------
 cluster1-01        1.10.B             active
                     1.11.B             active
                     1.11.A             active
                     1.10.A             active
                     1.0.A              connection-error
                     1.0.B              active
 cluster1-02        1.10.B             active
                     1.11.B             active
                     1.11.A             active
                     1.10.A             active
                     1.0.A              connection-error
                     1.0.B              active

 1. Disable ACP
 cluster1::> storage shelf acp configure -is-enabled false
 2. Wait for 5 minutes to allow the ACP to reset completely.
 3. Enable out-of-band ACP again and use the default IP address. ACP provides the
 IP address.
 cluster1::> storage shelf acp configure -is-enabled true -channel out-of-band

Re: ACP module connection-error after ONTAP 9.1 upgrade

alessice — Fri, 30 Aug 2019 08:26:52 GMT

Thanks @AlexDawson

the bug report exactly my issue, I will try to disable/enable ACP and let you know if it works and solve my issue.

Re: ACP module connection-error after ONTAP 9.1 upgrade

AlexDawson — Fri, 30 Aug 2019 13:08:02 GMT

Sounds good!

Re: ACP module connection-error after ONTAP 9.1 upgrade

alessice — Fri, 30 Aug 2019 20:04:41 GMT

I'm happy, now ACP works fine:

cluster2::> storage shelf acp module show          
Node               Module Name        State
-----------------  -----------------  ----------------------
cluster2-node01    1.0.B              active
                   1.0.A              active
cluster2-node02    1.0.B              active
                   1.0.A              active
4 entries were displayed.