FAS and V-Series Storage Systems Discussions

ACP module connection-error after ONTAP 9.1 upgrade

alessice

Hi,

 

I have upgrade my FAS2554 from 8.3 to 9.1. All works fine but about every 5 minutes in the Events Log I found this debug messages:


acp.command.connect: Could not establish connection with 192.168.2.145.

acp.command.connect: Could not establish connection with 192.168.2.145.

 

I read that starting from ONTAP 9 ACP move from out-of-band to in-band for communications, but I have no idea how to switch or solve for my case.

 

Here my configurations:

cluster2::> storage shelf acp module show
Node Module Name State
----------------- ----------------- ----------------------
cluster2-node01 1.0.B connection-error
1.0.A connection-error
cluster2-node02 1.0.B connection-error
1.0.A connection-error
4 entries were displayed.

cluster2::> storage shelf acp show
Node Channel Connectivity
------------------ --------------- ----------------------
cluster2-node01 out-of-band full-connectivity
cluster2-node02 out-of-band full-connectivity
2 entries were displayed.

cluster2::> storage shelf acp show -instance

Node: cluster2-node01
Channel: out-of-band
Enable Status: true
Port: e0P
IP Address: 192.168.2.55
Subnet: 192.168.0.1
Netmask: 255.255.252.0
Connection Status: full-connectivity

Node: cluster2-node02
Channel: out-of-band
Enable Status: true
Port: e0P
IP Address: 192.168.1.253
Subnet: 192.168.0.1
Netmask: 255.255.252.0
Connection Status: full-connectivity
2 entries were displayed.

cluster2::> storage shelf acp module show -instance

Node: cluster2-node01
Module Name: 1.0.B
Adapter Name: 0a
MAC Address: 00:a0:98:c6:ea:91
IOM Type: iom6e
Shelf Serial Number: SHJHU1702000451
IP Address: 192.168.2.145
Protocol Version: 2.1.01.22
Firmware Version: 01.22
Assigner ID: 537067641
State: connection-error
Last Contact: 514

Node: cluster2-node01
Module Name: 1.0.A
Adapter Name: 0a
MAC Address: 00:a0:98:c6:ed:91
IOM Type: iom6e
Shelf Serial Number: SHJHU1702000451
IP Address: 192.168.1.145
Protocol Version: 2.1.01.22
Firmware Version: 01.22
Assigner ID: 537067641
State: active
Last Contact: 36

Node: cluster2-node02
Module Name: 1.0.B
Adapter Name: 0b
MAC Address: 00:a0:98:c6:ea:91
IOM Type: iom6e
Shelf Serial Number: SHJHU1702000451
IP Address: 192.168.2.145
Protocol Version: 2.1.01.22
Firmware Version: 01.22
Assigner ID: 537067641
State: connection-error
Last Contact: 514

Node: cluster2-node02
Module Name: 1.0.A
Adapter Name: 0b
MAC Address: 00:a0:98:c6:ed:91
IOM Type: iom6e
Shelf Serial Number: SHJHU1702000451
IP Address: 192.168.1.145
Protocol Version: 2.1.01.22
Firmware Version: 01.22
Assigner ID: 537067641
State: active
Last Contact: 36
4 entries were displayed.

cluster2::> system service-processor show
IP Firmware
Node Type Status Configured Version IP Address
------------- ---- ----------- ------------ --------- -------------------------
cluster2-node01
SP online true 2.4.1P2 192.168.7.210
cluster2-node02
SP online true 2.4.1P2 192.168.7.211
2 entries were displayed.

I read the KB https://kb.netapp.com/app/answers/answer_view/a_id/1029778 but I have no understand if for my setup is sufficient an

 

storage shelf acp configure -is-enabled true -channel in-band

or can cause problems if firmware or similar are not at the right version.

 

Thanks

 

 

11 REPLIES 11

Re: ACP module connection-error after ONTAP 9.1 upgrade

SpindleNinja

you post the output of the following: 

 

storage shelf acp module show -fields module-name,protocol-version,firmware-version,shelf-serial-number,iom-type,state  

Re: ACP module connection-error after ONTAP 9.1 upgrade

SpindleNinja

you post the output of the following: 

 

storage shelf acp module show -fields module-name,protocol-version,firmware-version,shelf-serial-number,iom-type,state

Re: ACP module connection-error after ONTAP 9.1 upgrade

alessice

Thanks, here te output:

 

cluster2::> storage shelf acp module show -fields module-name,protocol-version,firmware-version,shelf-serial-number,iom-type,state
node            module-name protocol-version firmware-version shelf-serial-number iom-type state            
--------------- ----------- ---------------- ---------------- ------------------- -------- ---------------- 
cluster2-node01 1.0.B       2.1.01.22        01.22            SHJHU1702000451     iom6e    connection-error 
cluster2-node01 1.0.A       2.1.01.22        01.22            SHJHU1702000451     iom6e    connection-error 
cluster2-node02 1.0.B       2.1.01.22        01.22            SHJHU1702000451     iom6e    connection-error 
cluster2-node02 1.0.A       2.1.01.22        01.22            SHJHU1702000451     iom6e    connection-error 
4 entries were displayed.

Re: ACP module connection-error after ONTAP 9.1 upgrade

SpindleNinja

based on the firmware levels,  doesn't look like you can move to inband.     

 

Just to confirm, Is this just internal disks or is there a shelf attatched? 

 

have you opened a support case with netapp yet? 

Re: ACP module connection-error after ONTAP 9.1 upgrade

alessice

This is a single FAS2554 in HA with 24 internal disk, no external shelf.

 

From ActiveIQ all firmware are marked as update. For now I haven't open a case.

Re: ACP module connection-error after ONTAP 9.1 upgrade

SpindleNinja

have you tried a manual update of the lastest firmwares?  

 

https://mysupport.netapp.com/NOW/cgi-bin/shelffwdnld.cgi/download/tools/diskshelf/bin/all_shelf_fw 

 

Re: ACP module connection-error after ONTAP 9.1 upgrade

GidonMarcus

Hi All

 

i can't find a reference for it now. but i had similar issue with two  FAS2240 also going from 8.3.2 to 9.1 where ACP gone to no-connection. the solution was to turn ACP off and on again.

 

i think there was a bug open for it, but i can't find it...

 

Gidi

Gidi Marcus (Linkedin) - Storage and Microsoft technologies consultant - Hydro IT LTD - UK

Re: ACP module connection-error after ONTAP 9.1 upgrade

AlexDawson

BURT ID 1121636

After the Data ONTAP software upgrade from 8.3.x to 9.1.Px, ONTAP might report
 that the Alternate Control Path (ACP) processor on IOM6/IOM6E is no longer
 active, and it is reported as a "connection-error."
 You can verify this issue by using the "storage shelf acp module show" command
 to display the state of all ACP processors on IOM modules.
 cluster1::> storage shelf acp module show
 Node               Module Name        State
 -----------------  -----------------  ----------------------
 cluster1-01        1.10.B             active
                     1.11.B             active
                     1.11.A             active
                     1.10.A             active
                     1.0.A              connection-error
                     1.0.B              active
 cluster1-02        1.10.B             active
                     1.11.B             active
                     1.11.A             active
                     1.10.A             active
                     1.0.A              connection-error
                     1.0.B              active
 1. Disable ACP
 cluster1::> storage shelf acp configure -is-enabled false
 2. Wait for 5 minutes to allow the ACP to reset completely.
 3. Enable out-of-band ACP again and use the default IP address. ACP provides the
 IP address.
 cluster1::> storage shelf acp configure -is-enabled true -channel out-of-band

 

View solution in original post

Re: ACP module connection-error after ONTAP 9.1 upgrade

alessice

Thanks @AlexDawson 

 

the bug report exactly my issue, I will try to disable/enable ACP and let you know if it works and solve my issue.

Re: ACP module connection-error after ONTAP 9.1 upgrade

AlexDawson

Sounds good!

Re: ACP module connection-error after ONTAP 9.1 upgrade

alessice

I'm happy, now ACP works fine:

 

cluster2::> storage shelf acp module show          
Node               Module Name        State
-----------------  -----------------  ----------------------
cluster2-node01    1.0.B              active
                   1.0.A              active
cluster2-node02    1.0.B              active
                   1.0.A              active
4 entries were displayed.
2021 NetApp Partner Experience Survey
PES Banner
All Community Forums
Public