Both Controller are in Lockdown state

ShahbazRiaz · ‎2023-08-29

Hello,

Few days ago CTL-A was fail and it will go in lockdown state, i tried below command but controller is still lockdown state.

-> lemClearLockdown
08/29/23-07:59:30 (tShell0): ASSERT: Assertion failõ
-=<###>=- reading fw/active/System

8276946 bytes read

Adding 18180 symbols for standalone.
wrproxy:Instantiating /bd0 as rawFs, device = 0x10001

RTC Error: Real-time clock device is not working
/bd16/fw/staged/: errno = 0x380003
Unexpected PCI bus width 0(4) for BDF[1,8,0]
Unexpected PCI bus speed 1(3) for BDF[1,8,0]
TRIGGERING LOCKDOWN.
Unexpected PCI bus width 0(4) for BDF[1,9,0]
Unexpected PCI bus speed 1(3) for BDF[1,9,0]
TRIGGERING LOCKDOWN.

PCI DEVICE 0 NOT FOUND: Host-side Fibre Device=0 Function=0 ClassCode=0xc0400 ParentBridge=PLX PCI-E Bridge to Host Card Port 0
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: Host-side Fibre Device=0 Function=0 ClassCode=0xc0400 ParentBridge=PLX PCI-E Bridge to Host Card Port 0

PCI DEVICE 0 NOT FOUND: Host-side Fibre Device=0 Function=1 ClassCode=0xc0400 ParentBridge=PLX PCI-E Bridge to Host Card Port 0
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: Host-side Fibre Device=0 Function=1 ClassCode=0xc0400 ParentBridge=PLX PCI-E Bridge to Host Card Port 0

PCI DEVICE 2 NOT FOUND: iSCSI Ethernet Device=0 Function=2 ClassCode=0x20000 ParentBridge=PLX PCI-E Bridge to Host Card Port 0
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: iSCSI Ethernet Device=0 Function=2 ClassCode=0x20000 ParentBridge=PLX PCI-E Bridge to Host Card Port 0

PCI DEVICE 2 NOT FOUND: iSCSI Ethernet Device=0 Function=3 ClassCode=0x20000 ParentBridge=PLX PCI-E Bridge to Host Card Port 0
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: iSCSI Ethernet Device=0 Function=3 ClassCode=0x20000 ParentBridge=PLX PCI-E Bridge to Host Card Port 0

PCI DEVICE 0 NOT FOUND: Host-side iSCSI Device=0 Function=4 ClassCode=0x28000 ParentBridge=PLX PCI-E Bridge to Host Card Port 0
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: Host-side iSCSI Device=0 Function=4 ClassCode=0x28000 ParentBridge=PLX PCI-E Bridge to Host Card Port 0

PCI DEVICE 0 NOT FOUND: Host-side iSCSI Device=0 Function=5 ClassCode=0x28000 ParentBridge=PLX PCI-E Bridge to Host Card Port 0
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: Host-side iSCSI Device=0 Function=5 ClassCode=0x28000 ParentBridge=PLX PCI-E Bridge to Host Card Port 0

PCI DEVICE 0 NOT FOUND: Host-side Fibre Device=0 Function=0 ClassCode=0xc0400 ParentBridge=PLX PCI-E Bridge to Host Card Port 1
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: Host-side Fibre Device=0 Function=0 ClassCode=0xc0400 ParentBridge=PLX PCI-E Bridge to Host Card Port 1

PCI DEVICE 0 NOT FOUND: Host-side Fibre Device=0 Function=1 ClassCode=0xc0400 ParentBridge=PLX PCI-E Bridge to Host Card Port 1
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: Host-side Fibre Device=0 Function=1 ClassCode=0xc0400 ParentBridge=PLX PCI-E Bridge to Host Card Port 1

PCI DEVICE 2 NOT FOUND: iSCSI Ethernet Device=0 Function=2 ClassCode=0x20000 ParentBridge=PLX PCI-E Bridge to Host Card Port 1
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: iSCSI Ethernet Device=0 Function=2 ClassCode=0x20000 ParentBridge=PLX PCI-E Bridge to Host Card Port 1

PCI DEVICE 2 NOT FOUND: iSCSI Ethernet Device=0 Function=3 ClassCode=0x20000 ParentBridge=PLX PCI-E Bridge to Host Card Port 1
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: iSCSI Ethernet Device=0 Function=3 ClassCode=0x20000 ParentBridge=PLX PCI-E Bridge to Host Card Port 1

PCI DEVICE 0 NOT FOUND: Host-side iSCSI Device=0 Function=4 ClassCode=0x28000 ParentBridge=PLX PCI-E Bridge to Host Card Port 1
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: Host-side iSCSI Device=0 Function=4 ClassCode=0x28000 ParentBridge=PLX PCI-E Bridge to Host Card Port 1

PCI DEVICE 0 NOT FOUND: Host-side iSCSI Device=0 Function=5 ClassCode=0x28000 ParentBridge=PLX PCI-E Bridge to Host Card Port 1
TRIGGERING LOCKDOWN.

PCI DEVICE NOT ACCESSIBLE: Host-side iSCSI Device=0 Function=5 ClassCode=0x28000 ParentBridge=PLX PCI-E Bridge to Host Card Port 1
usrSecurity: error adding default user '(null)' to the system (errno = 0x16)!
Instantiating /ram as rawFs, device = 0x50001
Formatting /ram for DOSFS
Instantiating /ram as rawFs, device = 0x50001
Formatting...Retrieved old volume params with %38 confidence:
Volume Parameters: FAT type: FAT32, sectors per cluster 0
0 FAT copies, 0 clusters, 0 sectors per FAT
Sectors reserved 0, hidden 0, FAT sectors 0
Root dir entries 0, sysId (null) , serial number 1d20000
Label:" " ...
Disk with 1024 sectors of 512 bytes will be formatted with:
Volume Parameters: FAT type: FAT12, sectors per cluster 1
2 FAT copies, 1010 clusters, 3 sectors per FAT
Sectors reserved 1, hidden 0, FAT sectors 6
Root dir entries 112, sysId VXDOS12 , serial number 1d20000
Label:" " ...
OK.
08/29/23-07:59:54 (tNetCfgInit): NOTE: eth0: LinkDown event
08/29/23-07:59:54 (tNetCfgInit): NOTE: eth1: LinkDown event
08/29/23-07:59:54 (tNetCfgInit): NOTE: Network Ready

Controller has been locked down due to Hardware errors:

================= EXCEPTION LOG =================
Entry count: 94
Wrap-arounds: 1
First entry time:
Current Controller date/time: AUG-29-2023 01:28:52 AM
Current Local (User) date/time: AUG-29-2023 07:59:54 AM

---- Log Entry #46 (Core 0) AUG-22-2023 01:56:59 AM ----

WARNING: Restart by watchdog time out on EOS domain

---- Log Entry #47 (Core 0) AUG-22-2023 01:58:29 AM ----

Root Complex TLP header[0] 30008000
Root Complex TLP header[1] 04000030
Root Complex TLP header[2] 00000000
Root Complex TLP header[3] 00000000
WARNING: More than 8 RC TLP headers
ERROR: PCI CFG ACCESS FAILED FOR BDF[0,1,0], 0x10b5:0x8732
ERROR: PCI CFG ACCESS FAILED FOR BDF[1,1,0], 0x10b5:0x8732
ERROR: PCI CFG ACCESS FAILED FOR BDF[2,0,0], 0x1000:0x0097
ERROR: PCI CFG ACCESS FAILED FOR BDF[2,0,0], 0x1000:0x0097
ERROR: PCI CFG ACCESS FAILED FOR BDF[1,8,0], 0x10b5:0x8732
ERROR: PCI CFG ACCESS FAILED FOR BDF[1,9,0], 0x10b5:0x8732
ERROR: PCI CFG ACCESS FAILED FOR BDF[1,10,0], 0x10b5:0x8732

---- Log Entry #48 (Core 0) AUG-22-2023 01:58:53 AM ----
Unexpected PCI bus width 0(4) for BDF[1,8,0]
Unexpected PCI bus speed 1(3) for BDF[1,8,0]
TRIGGERING LOCKDOWN.
Unexpected PCI bus width 0(4) for BDF[1,9,0]
Unexpected PCI bus speed 1(3) for BDF[1,9,0]
TRIGGERING LOCKDOWN.