ONTAP Hardware

FAS3220 Dual Chassis HA interconnect error

unitech89e9
33,861 Views

Hi everybody

I start to create a FAS3220 Dual Chassis HA enviroment,

but we don't have optional addition sas adapter,

so I decide to connect the controller to disk shelves by single path,

the topology just like that

but the cluster failover is not working,

when I use the cf status on each node,

the system show:

partner may be down, takeover disabled because of reason (partner mailbox disks not accessible or invalid)

FAS3220 has disabled takeover by partner (interconnect error)

RDMA Interconnect is down (Link 0 down).

When I use the cf monitor on each node,

the system show:

current time: 15Aug2014 04:19:51

UP 02:00:49, partner 'unknown', cluster monitor enabled

RDMA Interconnect is down (Link 0 down), takeover capability off-line (partner mailbox disks not accessible or invalid)

takeover by partner off-line (interconnect error)

partner may be down, last partner update UNKNOWN (15Aug2014 04:18:37)

When I use the cf hw_assist status

the system show

Local Node(FAS3220) Status:

        Inactive: FAS3220not monitoring alerts from partner(partner)

                Interconnect is down between nodes.

Partner Node(partner) Status:

        Can not read partner state

        Interconnect is down between nodes

so I open the Oncommand System Manager to see the network interface

I see the the c0a and c0b is show the status like this

but I don't know where is my mistake?

what will make the interconnect port(c0a and c0b) be down?

Please help me!

Thanks very much

1 ACCEPTED SOLUTION

aborzenkov
33,730 Views

Please boot into maintenance mode and use “ha-config show” to check current HA settings. Looks like your chassis is set to HA, which means it will disable external ports and try internal backplane connection.

View solution in original post

11 REPLIES 11

DOMINIC_WYSS
33,603 Views

it looks to me that you accidentally miscabled the interconnects. don't cross them out! c0a goes to c0a on the other head, not to c0b!

you get these link states (one interconnect up, the other down) when you crossed them out.

btw: if I remember correctly, it's now allowed to do ONE transition between IOM3 and IOM6 per stack (it was only allowed on FAS2000 series until recently).

so you can cable both shelf types into one stack and have it multipathed.

unitech89e9
33,603 Views

Hi Dominic:

I cable the c0a on the other c0a,

and c0b to the other c0b,

is that wrong?

DOMINIC_WYSS
33,603 Views

I cable the c0a on the other c0a,

and c0b to the other c0b,

is that wrong?

no, that is correct. and also your schema is correct. but double check if you have not accidentally crossed the ports.

and check if you have the right cables! there are two types of these copper cables with the same SFP+ interface. I think the Ciscos do not work on the interconnect ports.

what's the output of these ports in sysconfig -a?

shane_bradley
33,602 Views

Have you checked the disks are all visible from both nodes? could indicate that they cant see partner disks.

unitech89e9
33,602 Views

Hi Shane:

I can sure that, every node can see all the disk!

unitech89e9
33,602 Views

N6220-SAS> sysconfig -a

Data ONTAP Release 8.1.4P4 7-Mode: Sat Jun 28 01:52:51 PDT 2014

System ID: 2018635377 (N6220-SAS); partner ID: 2018730260 ()

System Serial Number: 2858C2513F009011 (N6220-SAS)

System Rev: B0

System Storage Configuration: Single-Path HA

System ACP Connectivity: NA

slot 0: System Board 2.3 GHz (System Board XVI B0)

                Model Name:         N6220

                Machine Type:       IBM-2858-C25

                Part Number:        111-01333

                Revision:           B0

                Serial Number:      9448935534

                BIOS version:       5.2.1

                Loader version:     3.4

                Processors:         4

                Processor ID:       0x1067a

                Microcode Version:  0xa0b

                Processor type:     Intel(R) Xeon(R) CPU           L5410  @ 2.33GHz

                Memory Size:        12288 MB

                Memory Attributes:  Bank Interleaving

                                    Hoisting

                                    Rank Interleaving

                                    Chipkill ECC

                NVMEM Size:         1600 MB of Main Memory Used

                CMOS RAM Status:    OK

Service Processor           Status: Online

  Firmware Version:   1.3.3

  Mgmt MAC Address:   00:A0:98:5A:FC:02

  Ethernet Link:      down

  Using DHCP:         yes

IPv4 configuration:

  IP Address:         unknown

  Netmask:            unknown

  Gateway:            unknown

IPv6 configuration:         Disabled

slot 0: Dual 10G Ethernet Controller T320E-SFP/KR

  Device Type:        CT-FE-3

  Version Number:     T3-SRAM1.1.0-BR1016-02-01-FW7.7.209-DR04

  Serial Number:     jb04050693     

  c0a MAC Address:    00:a0:98:5a:fb:fe (auto-unknown-cfg_down)

  c0b MAC Address:    00:a0:98:5a:fb:ff (auto-unknown-enabling)

                        memory mapped I/O base 0xde801000, size 0x1000

                        memory mapped I/O base 0xde000000, size 0x800000

                        memory mapped I/O base 0xde800000, size 0x1000

slot 0: Internal BGE 10/100 Ethernet Controller

  e0M MAC Address:    00:a0:98:5a:fc:00 (auto-unknown-down)

  e0P MAC Address:    00:a0:98:5a:fc:01 (auto-100tx-fd-up)

                Device Type:        BCM5721

                        memory mapped I/O base 0xdf800000, size 0x10000

                        memory mapped I/O base 0xdf900000, size 0x10000

slot 0: Dual 10/100/1000 Ethernet Controller G20

  e0a MAC Address:    00:a0:98:5a:fb:fc (auto-1000t-fd-up)

  e0b MAC Address:    00:a0:98:5a:fb:fd (auto-unknown-down)

  Device Type:        Rev 6

                        memory mapped I/O base 0x00000000df720000, size 0x20000

                        memory mapped I/O base 0x00000000df700000, size 0x20000

                        I/O base 0x0000000000003000, size 0x20

                        memory mapped I/O base 0x00000000df760000, size 0x20000

                        memory mapped I/O base 0x00000000df740000, size 0x20000

                        I/O base 0x0000000000003020, size 0x20

slot 0: SAS Host Adapter 0a (PMC-Sierra PM8001 rev. C, SAS, <UP>)

  Firmware rev:       01.11.07.00

  Base WWN:           5:00a098:00364e4:08

  Phy State:          [0] Enabled, 6.0 Gb/s

                      [1] Enabled, 6.0 Gb/s

                      [2] Enabled, 6.0 Gb/s

                      [3] Enabled, 6.0 Gb/s

  QSFP Vendor:        Molex Inc.     

  QSFP Part Number:   112-00177+A0   

  QSFP Type:          Passive Copper 2m ID:00

  QSFP Serial Number: 408720176

                02.0 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J4DDSP)

                02.1 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J4GWPP)

                02.2 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J4PE1P)

                02.3 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J4LDNP)

                02.4 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J4K6KP)

                02.5 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J4689P)

                02.6 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J4GWRP)

                02.7 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J467YP)

                02.8 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J466VP)

                02.9 : NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J5053P)

                02.10: NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J4WVNP)

                02.11: NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J5044P)

                02.12: NETAPP   X306_HMRKP02TSSA NA00 1695.4GB 512B/sect (P6J504GP)

                03.0 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1277461)

                03.1 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1272876)

                03.2 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1274858)

                03.3 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1274741)

                03.4 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1274793)

                03.5 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1273338)

                03.6 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1271576)

                03.7 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1272160)

                03.8 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1273637)

                03.9 : NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1274446)

                03.10: NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1272310)

                03.11: NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1277264)

                03.12: NETAPP   X306_WKOJN02TSSA NA01 1695.4GB 512B/sect (WD-WCC1P1275875)

  Shelf   2: IOM6  Firmware rev. IOM6 A: 0162 IOM6 B: 0162

  Shelf   3: IOM6  Firmware rev. IOM6 A: 0162 IOM6 B: 0162

slot 0: SAS Host Adapter 0b (PMC-Sierra PM8001 rev. C, SAS, <UP>)

  Firmware rev:       01.11.07.00

  Base WWN:           5:00a098:00364e4:0c

  Phy State:          [4] Enabled, 6.0 Gb/s

                      [5] Enabled, 6.0 Gb/s

                      [6] Enabled, 6.0 Gb/s

                      [7] Enabled, 6.0 Gb/s

  QSFP Vendor:        Molex Inc.     

  QSFP Part Number:   112-00177+A0   

  QSFP Type:          Passive Copper 2m ID:00

  QSFP Serial Number: 408720154

                00.0 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGMJHWF)

                00.1 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGNZ2NF)

                00.2 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGG3YAM)

                00.3 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGNZ67F)

                00.4 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGJ7V4R)

                00.5 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGNYTGF)

                00.6 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGNZ2JF)

                00.7 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGMRH8M)

                00.8 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGMU9NM)

                00.9 : NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGMW1PM)

                00.10: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGMT8XM)

                00.11: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGMSGYM)

                00.12: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGKNH0M)

                00.13: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGK33SR)

                00.14: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGMRPDM)

                00.15: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGL17AM)

                00.16: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGEX0XR)

                00.17: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGMSYMM)

                00.18: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGEJHVM)

                00.19: NETAPP   X422_HCOBE600A10 NA00 560.0GB 520B/sect (KWGMJ4UF)

                01.0 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44O0A0OCFUZ6)

                01.1 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44O0A0NJFUZ6)

                01.2 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44O0A0MRFUZ6)

                01.3 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44N0A058FUZ6)

                01.4 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44O0A0MBFUZ6)

                01.5 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44N0A055FUZ6)

                01.6 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44O0A0LMFUZ6)

                01.7 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44O0A0L4FUZ6)

                01.8 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44O0A0KOFUZ6)

                01.9 : NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44O0A0N7FUZ6)

                01.10: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44N0A05BFUZ6)

                01.11: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44N0A06YFUZ6)

                01.12: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44M0A05RFUZ6)

                01.13: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44M0A0ELFUZ6)

                01.14: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44N0A0AXFUZ6)

                01.15: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44N0A05JFUZ6)

                01.16: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44M0A0EBFUZ6)

                01.17: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44N0A04ZFUZ6)

                01.18: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44N0A09JFUZ6)

                01.19: NETAPP   X422_TAL13600A10 NA01 560.0GB 520B/sect (44M0A0E3FUZ6)

  Shelf   0: IOM6  Firmware rev. IOM6 A: 0162 IOM6 B: 0162

  Shelf   1: IOM6  Firmware rev. IOM6 A: 0162 IOM6 B: 0162

                        memory mapped I/O base 0xdf530000, size 0x10000

                        memory mapped I/O base 0xdf520000, size 0x10000

                        memory mapped I/O base 0x00000000df510000, size 0x10000

                        memory mapped I/O base 0x00000000df500000, size 0x10000

slot 0: FC Host Adapter 0c (QLogic 2432 rev. 2, N-port, <UP>)

  Firmware rev:      5.4.100

  Host Port Id:      0x10300

  FC Node Name:      5:00a:098003:64e400

  FC Port Name:      5:00a:098003:64e400

  SFP Vendor:        FINISAR CORP.

  SFP Part Number:   FTLF8524P2BNV

  SFP Serial Number: PQR1V3L

  SFP Capabilities:  1, 2 or 4 Gbit

  Link Data Rate:    4 Gbit

  Switch Port:       IBM_2498_B24:3

                        I/O base 0x0000000000002000, size 0x100

                        memory mapped I/O base 0xdf600000, size 0x4000

slot 0: FC Host Adapter 0d (QLogic 2432 rev. 2, N-port, <UP>)

  Firmware rev:      5.4.100

  Host Port Id:      0x10300

  FC Node Name:      5:00a:098103:64e400

  FC Port Name:      5:00a:098103:64e400

  SFP Vendor:        FINISAR CORP.

  SFP Part Number:   FTLF8524P2BNV

  SFP Serial Number: PQS0QLZ

  SFP Capabilities:  1, 2 or 4 Gbit

  Link Data Rate:    4 Gbit

  Switch Port:       IBM_2498_B24:3

                        I/O base 0x0000000000002400, size 0x100

                        memory mapped I/O base 0xdf604000, size 0x4000

slot 0: Intel ICH USB EHCI Adapter u0a (0xdfa01400)

  boot0 Micron Technology 0x655, class 0/0, rev 2.00/11.10, addr 2 1936MB 512B/sect (2DF0022700016909)

                        memory-mapped I/O base 0xdfa01400, size 0x400

slot 1: FC Host Adapter 1a (QLogic 2532 rev. 2, L-port, <OFFLINE (hard)>)

  Board name:        QLE2564

  Serial Number:     RFD1404U23108

  Firmware rev:      5.4.130

  Host Loop Id:      0

  FC Node Name:      2:000:0024ff:731854

  FC Port Name:      2:100:0024ff:731854

  SFP Vendor:        AVAGO

  SFP Part Number:   AFBR-57D9AMZ-QL

  SFP Serial Number: AD13393043V

  SFP Capabilities:  2, 4 or 8 Gbit

  Link Data Rate:    N/A

                        I/O base 0x00000000ffffff00, size 0x100

                        memory mapped I/O base 0xdee00000, size 0x4000

                        memory mapped I/O base 0xdec00000, size 0x100000

slot 1: FC Host Adapter 1b (QLogic 2532 rev. 2, L-port, <OFFLINE (hard)>)

  Board name:        QLE2564

  Serial Number:     RFD1404U23108

  Firmware rev:      5.4.130

  Host Loop Id:      0

  FC Node Name:      2:000:0024ff:731855

  FC Port Name:      2:100:0024ff:731855

  SFP Vendor:        AVAGO

  SFP Part Number:   AFBR-57D9AMZ-QL

  SFP Serial Number: AD13393043X

  SFP Capabilities:  2, 4 or 8 Gbit

  Link Data Rate:    N/A

                        I/O base 0x00000000ffffff00, size 0x100

                        memory mapped I/O base 0xdee04000, size 0x4000

                        memory mapped I/O base 0xded00000, size 0x100000

slot 1: FC Host Adapter 1c (QLogic 2532 rev. 2, L-port, <OFFLINE (hard)>)

  Board name:        QLE2564

  Serial Number:     RFD1404U23108

  Firmware rev:      5.4.130

  Host Loop Id:      0

  FC Node Name:      2:000:0024ff:731856

  FC Port Name:      2:100:0024ff:731856

  SFP Vendor:        AVAGO

  SFP Part Number:   AFBR-57D9AMZ-QL

  SFP Serial Number: AD13393043Y

  SFP Capabilities:  2, 4 or 8 Gbit

  Link Data Rate:    N/A

                        I/O base 0x00000000ffffff00, size 0x100

                        memory mapped I/O base 0xdf100000, size 0x4000

                        memory mapped I/O base 0xdef00000, size 0x100000

slot 1: FC Host Adapter 1d (QLogic 2532 rev. 2, L-port, <OFFLINE (hard)>)

  Board name:        QLE2564

  Serial Number:     RFD1404U23108

  Firmware rev:      5.4.130

  Host Loop Id:      0

  FC Node Name:      2:000:0024ff:731857

  FC Port Name:      2:100:0024ff:731857

  SFP Vendor:        AVAGO

  SFP Part Number:   AFBR-57D9AMZ-QL

  SFP Serial Number: AD13393043U

  SFP Capabilities:  2, 4 or 8 Gbit

  Link Data Rate:    N/A

                        I/O base 0x00000000ffffff00, size 0x100

                        memory mapped I/O base 0xdf104000, size 0x4000

                        memory mapped I/O base 0xdf000000, size 0x100000

slot 2: FCVI Host Adapter 2a (QLogic 2532(2562) rev. 2, unknown, <DOWN>)

  Physical link: DOWN

  FC Node Name: 21:00:00:24:ff:58:4f:58

  Firmware rev: 5.3.140 Serial No: BFD1408T29623

  Host Port Id: 0xffffffff

  Cacheline size: 8 FC Packet size: 2048

  SFP Vendor: FINISAR CORP.

  SFP Part Number: FTLF8528P3BCV-QL

  SFP Serial Number: PR11JG5

  SFP Capabilities:  2, 4 or 8 Gbit

                        I/O base 0x00000000ffffff00, size 0x100

                        memory mapped I/O base 0xdf200000, size 0x4000

slot 2: FCVI Host Adapter 2b (QLogic 2532(2562) rev. 2, unknown, <DOWN>)

  Physical link: DOWN

  FC Node Name: 21:00:00:24:ff:58:4f:59

  Firmware rev: 5.3.140 Serial No: BFD1408T29623

  Host Port Id: 0xffffffff

  Cacheline size: 8 FC Packet size: 2048

  SFP Vendor: FINISAR CORP.

  SFP Part Number: FTLF8528P3BCV-QL

  SFP Serial Number: PR11JFW

  SFP Capabilities:  2, 4 or 8 Gbit

                        I/O base 0x00000000ffffff00, size 0x100

                        memory mapped I/O base 0xdf204000, size 0x4000

unitech89e9
33,603 Views

rdfile /etc/log/messages
Sun Aug 17 00:19:34 GMT [N6220-SAS:callhome.management.log:info]: Call home for MANAGEMENT_LOG 

Sun Aug 17 00:19:34 GMT [N6220-SAS:callhome.weekly:info]: Call home for WEEKLY_LOG 

Sun Aug 17 00:19:35 GMT [N6220-SAS:kern.log.rotate:notice]: System N6220-SAS (ID 2018635377) is running 8.1.4P4 

Sun Aug 17 00:56:11 GMT [N6220-SAS:asup.post.drop:error]: AutoSupport message (HA Group Notification from N6220-SAS (PERFORMANCE DATA) INFO) for host (0) was not posted to IBM. The system will drop the message. 

Sun Aug 17 01:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   1:00am up  1 day, 22:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 01:00:03 GMT [N6220-SAS:raid.rg.scrub.start:notice]: /aggr0/plex0/rg2: starting scrub 

Sun Aug 17 01:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 01:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 2801 minutes 

Sun Aug 17 01:00:04 GMT [N6220-SAS:raid.rg.scrub.start:notice]: /aggr0/plex0/rg1: starting scrub 

Sun Aug 17 01:00:05 GMT [N6220-SAS:raid.rg.scrub.start:notice]: /aggr0/plex0/rg0: starting scrub 

Sun Aug 17 01:56:16 GMT [N6220-SAS:asup.post.drop:error]: AutoSupport message (HA Group Notification from N6220-SAS (MANAGEMENT_LOG) INFO) for host (0) was not posted to IBM. The system will drop the message. 

Sun Aug 17 02:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   2:00am up  1 day, 23:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 02:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 02:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 2861 minutes 

Sun Aug 17 02:56:26 GMT [N6220-SAS:asup.post.drop:error]: AutoSupport message (HA Group Notification from N6220-SAS (WEEKLY_LOG) INFO) for host (0) was not posted to IBM. The system will drop the message. 

Sun Aug 17 03:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   3:00am up  2 days, 40 mins, 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 03:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 03:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 2921 minutes 

Sun Aug 17 04:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   4:00am up  2 days,  1:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 04:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 04:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 2981 minutes 

Sun Aug 17 04:35:21 GMT [N6220-SAS:raid.rg.scrub.done:notice]: /aggr0/plex0/rg2: scrub completed in 3:35:17.00 

Sun Aug 17 04:35:21 GMT [N6220-SAS:raid.rg.scrub.summary.pi:notice]: Scrub found 0 parity inconsistencies in /aggr0/plex0/rg2. 

Sun Aug 17 04:35:21 GMT [N6220-SAS:raid.rg.scrub.summary.cksum:notice]: Scrub found 0 checksum errors in /aggr0/plex0/rg2. 

Sun Aug 17 04:35:21 GMT [N6220-SAS:raid.rg.scrub.summary.media:notice]: Scrub found 0 media errors in /aggr0/plex0/rg2. 

Sun Aug 17 04:35:21 GMT [N6220-SAS:raid.rg.scrub.summary.lw:notice]: Scrub found 0 RAID write signature inconsistencies in /aggr0/plex0/rg2. 

Sun Aug 17 05:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   5:00am up  2 days,  2:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 05:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 05:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3041 minutes 

Sun Aug 17 05:39:58 GMT [N6220-SAS:raid.rg.scrub.done:notice]: /aggr0/plex0/rg1: scrub completed in 4:39:53.00 

Sun Aug 17 05:39:58 GMT [N6220-SAS:raid.rg.scrub.summary.pi:notice]: Scrub found 0 parity inconsistencies in /aggr0/plex0/rg1. 

Sun Aug 17 05:39:58 GMT [N6220-SAS:raid.rg.scrub.summary.cksum:notice]: Scrub found 0 checksum errors in /aggr0/plex0/rg1. 

Sun Aug 17 05:39:58 GMT [N6220-SAS:raid.rg.scrub.summary.media:notice]: Scrub found 0 media errors in /aggr0/plex0/rg1. 

Sun Aug 17 05:39:58 GMT [N6220-SAS:raid.rg.scrub.summary.lw:notice]: Scrub found 0 RAID write signature inconsistencies in /aggr0/plex0/rg1. 

Sun Aug 17 06:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   6:00am up  2 days,  3:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 06:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 06:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3101 minutes 

Sun Aug 17 06:44:41 GMT [N6220-SAS:raid.rg.scrub.done:notice]: /aggr0/plex0/rg0: scrub completed in 5:44:36.00 

Sun Aug 17 06:44:41 GMT [N6220-SAS:raid.rg.scrub.summary.pi:notice]: Scrub found 0 parity inconsistencies in /aggr0/plex0/rg0. 

Sun Aug 17 06:44:41 GMT [N6220-SAS:raid.rg.scrub.summary.cksum:notice]: Scrub found 0 checksum errors in /aggr0/plex0/rg0. 

Sun Aug 17 06:44:41 GMT [N6220-SAS:raid.rg.scrub.summary.media:notice]: Scrub found 0 media errors in /aggr0/plex0/rg0. 

Sun Aug 17 06:44:41 GMT [N6220-SAS:raid.rg.scrub.summary.lw:notice]: Scrub found 0 RAID write signature inconsistencies in /aggr0/plex0/rg0. 

Sun Aug 17 07:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   7:00am up  2 days,  4:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 07:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 07:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3161 minutes 

Sun Aug 17 08:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   8:00am up  2 days,  5:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 08:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 08:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3221 minutes 

Sun Aug 17 09:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   9:00am up  2 days,  6:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 09:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 09:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3281 minutes 

Sun Aug 17 10:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:  10:00am up  2 days,  7:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 10:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 10:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3341 minutes 

Sun Aug 17 11:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:  11:00am up  2 days,  8:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 11:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 11:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3401 minutes 

Sun Aug 17 12:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:  12:00pm up  2 days,  9:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 12:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 12:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3461 minutes 

Sun Aug 17 13:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   1:00pm up  2 days, 10:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 13:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 13:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3521 minutes 

Sun Aug 17 14:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   2:00pm up  2 days, 11:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 14:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 14:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3581 minutes 

Sun Aug 17 15:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   3:00pm up  2 days, 12:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 15:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 15:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3641 minutes 

Sun Aug 17 16:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   4:00pm up  2 days, 13:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 16:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 16:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3701 minutes 

Sun Aug 17 17:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   5:00pm up  2 days, 14:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 17:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 17:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3761 minutes 

Sun Aug 17 18:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   6:00pm up  2 days, 15:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 18:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 18:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3821 minutes 

Sun Aug 17 19:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   7:00pm up  2 days, 16:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 19:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 19:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3881 minutes 

Sun Aug 17 20:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   8:00pm up  2 days, 17:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 20:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 20:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 3941 minutes 

Sun Aug 17 21:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:   9:00pm up  2 days, 18:40 0 NFS ops, 196 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 21:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled due to reason : partner mailbox disks not accessible or invalid. 

Sun Aug 17 21:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 4001 minutes 

Sun Aug 17 21:46:17 GMT [console_login_mgr:info]: root logged in from console
Sun Aug 17 21:46:25 GMT [N6220-SAS:cf.misc.operatorDisable:warning]: Failover monitor: operator initiated disabling of takeover 

Sun Aug 17 21:46:26 GMT [N6220-SAS:cf.fsm.takeoverOfPartnerDisabled:notice]: Failover monitor: takeover of partner disabled (Controller Failover takeover disabled). 

Sun Aug 17 21:46:26 GMT [N6220-SAS:cf.fsm.takeoverByPartnerDisabled:notice]: Failover monitor: takeover of N6220-SAS by partner disabled (Controller Failover takeover disabled). 

Sun Aug 17 21:58:16 GMT [rc:notice]: The system was down for 662 seconds
Sun Aug 17 21:48:53 GMT [N6220-SAS:fci.adapter.link.online:info]: Fibre Channel adapter 0c link online. 

Sun Aug 17 21:48:53 GMT [N6220-SAS:fci.adapter.link.online:info]: Fibre Channel adapter 0d link online. 

Sun Aug 17 21:48:54 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0c reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004110 WWPN: 5005076803044110, switch:port IBM_2498_B24:0. 

Sun Aug 17 21:48:54 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0c reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004111 WWPN: 5005076803044111, switch:port IBM_2498_B24:1. 

Sun Aug 17 21:48:54 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0d reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004111 WWPN: 5005076803084111, switch:port IBM_2498_B24:1. 

Sun Aug 17 21:48:54 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0d reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004110 WWPN: 5005076803084110, switch:port IBM_2498_B24:0. 

Sun Aug 17 21:48:59 GMT [N6220-SAS:fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 1a. 

Sun Aug 17 21:48:59 GMT [N6220-SAS:fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 1b. 

Sun Aug 17 21:48:59 GMT [N6220-SAS:fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 1d. 

Sun Aug 17 21:48:59 GMT [N6220-SAS:fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 1c. 

Sun Aug 17 21:49:00 GMT [N6220-SAS:netif.linkDown:info]: Ethernet Wrench Port: Link down, check cable. 

Sun Aug 17 21:49:05 GMT [N6220-SAS:diskown.isEnabled:info]: software ownership has been enabled for this system 

Sun Aug 17 21:49:05 GMT [N6220-SAS:dcs.framework.enabled:info]: The DCS framework is enabled on this node. 

Sun Aug 17 21:49:07 GMT [N6220-SAS:wafl.memory.status:info]: 7092MB of memory is currently available for the WAFL file system. 

Sun Aug 17 21:49:09 GMT [N6220-SAS:shelf.config.spha:info]: System is using single path HA attached storage only. 

Sun Aug 17 21:49:09 GMT [N6220-SAS:netif.linkUp:info]: Ethernet e0P: Link up. 

Sun Aug 17 21:49:10 GMT [N6220-SAS:ses.multipath.ReqError:CRITICAL]: SAS-Shelf24 detected without a multipath configuration. 

Sun Aug 17 21:49:11 GMT [N6220-SAS:mgr.boot.reason_ok:notice]: System rebooted after a reboot command. 

Sun Aug 17 21:49:11 GMT [N6220-SAS:callhome.reboot.reboot:info]: Call home for REBOOT (reboot command) 

Sun Aug 17 21:49:12 GMT [N6220-SAS:netif.linkDown:info]: Ethernet e0b: Link down, check cable. 

Sun Aug 17 21:49:12 GMT [N6220-SAS:netif.linkDown:info]: Ethernet e0a: Link down, check cable. 

Sun Aug 17 21:49:14 GMT [N6220-SAS:netif.linkUp:info]: Ethernet e0P: Link up. 

Sun Aug 17 21:49:14 GMT [N6220-SAS:netif.linkDown:info]: Ethernet e0M: Link down, check cable. 

Sun Aug 17 21:50:27 GMT [N6220-SAS:fci.link.break:error]: Link break detected on Fibre Channel adapter 0c. 

Sun Aug 17 21:50:58 GMT [N6220-SAS:fci.link.error:error]: Could not recover link on Fibre Channel adapter 0c after 30 seconds. Offlining the adapter. 

Sun Aug 17 21:51:41 GMT [N6220-SAS:fci.adapter.link.online:info]: Fibre Channel adapter 0c link online. 

Sun Aug 17 21:51:41 GMT [N6220-SAS:fci.adapter.online:info]: Fibre Channel adapter 0c is now online. 

Sun Aug 17 21:51:41 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0c reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004110 WWPN: 5005076803044110, switch:port IBM_2498_B24:0. 

Sun Aug 17 21:51:41 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0c reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004111 WWPN: 5005076803044111, switch:port IBM_2498_B24:1. 

Sun Aug 17 21:57:59 GMT [N6220-SAS:fci.adapter.link.online:info]: Fibre Channel adapter 0c link online. 

Sun Aug 17 21:57:59 GMT [N6220-SAS:fci.adapter.link.online:info]: Fibre Channel adapter 0d link online. 

Sun Aug 17 21:57:59 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0c reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004110 WWPN: 5005076803044110, switch:port IBM_2498_B24:0. 

Sun Aug 17 21:57:59 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0c reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004111 WWPN: 5005076803044111, switch:port IBM_2498_B24:1. 

Sun Aug 17 21:57:59 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0d reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004110 WWPN: 5005076803084110, switch:port IBM_2498_B24:0. 

Sun Aug 17 21:57:59 GMT [N6220-SAS:mlm.array.portEmpty.switch:info]: Fibre Channel host bus adapter 0d reports no LUNs presented by storage array IBM_2145_?, target port WWNN: 5005076803004111 WWPN: 5005076803084111, switch:port IBM_2498_B24:1. 

Sun Aug 17 21:58:05 GMT [N6220-SAS:fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 1a. 

Sun Aug 17 21:58:05 GMT [N6220-SAS:fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 1b. 

Sun Aug 17 21:58:05 GMT [N6220-SAS:fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 1c. 

Sun Aug 17 21:58:05 GMT [N6220-SAS:fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 1d. 

Sun Aug 17 21:58:05 GMT [N6220-SAS:netif.linkDown:info]: Ethernet Wrench Port: Link down, check cable. 

Sun Aug 17 21:58:11 GMT [N6220-SAS:diskown.isEnabled:info]: software ownership has been enabled for this system 

Sun Aug 17 21:58:11 GMT [N6220-SAS:dcs.framework.enabled:info]: The DCS framework is enabled on this node. 

Sun Aug 17 21:58:11 GMT [N6220-SAS:config.noPartnerDisks:CRITICAL]: No disks were detected for the partner; this node will be unable to takeover correctly 

Sun Aug 17 21:58:11 GMT [N6220-SAS:callhome.dsk.config:warning]: Call home for DISK CONFIGURATION ERROR 

Sun Aug 17 21:58:13 GMT [N6220-SAS:wafl.memory.status:info]: 7070MB of memory is currently available for the WAFL file system. 

Sun Aug 17 21:58:13 GMT [N6220-SAS:fmmb.instStat.change:info]: no mailbox instance on local side. 

Sun Aug 17 21:58:13 GMT [N6220-SAS:fmmb.instStat.change:info]: no mailbox instance on partner side. 

Sun Aug 17 21:58:13 GMT [N6220-SAS:cf.fm.noMBDisksOrIc:warning]: Could not find the local mailbox disks. Could not determine the firmware state of the partner through the HA interconnect. 

Sun Aug 17 21:58:31 GMT [N6220-SAS:tar.csum.match:info]: Stored checksum matches, not extracting /mroot_late.tgz. 

Sun Aug 17 21:58:31 GMT [N6220-SAS:tar.csum.match:info]: Stored checksum matches, not extracting /platform/pmroot_late.tgz. 

Sun Aug 17 21:58:32 GMT [N6220-SAS:sp.notConfigured:warning]: The system's Service Processor (SP) is not configured. Use the 'sp setup' command to configure it. 

Sun Aug 17 21:58:32 GMT [N6220-SAS:sp.network.link.down:warning]: Service Processor (SP) network port link down due to cable or network errors. 

Sun Aug 17 21:58:32 GMT [N6220-SAS:cf.hwassist.DefaultPrtnrPort:notice]: The system has chose the default hardware-assist partner port number 4444. If you want to use a different port number, change it using the command 'options cf.hw_assist.partner.port'. 

Sun Aug 17 21:58:32 GMT [N6220-SAS:cf.hwassist.DefaultEnabled:info]: Hardware-assisted takeover feature is enabled on this system. The partner can take over this node faster during certain failures. 

Sun Aug 17 21:58:32 GMT [N6220-SAS:cf.hwassist.Default.NodemgmtAddr:notice]: The system automatically chose 192.168.1.139 as the local hardware-assist IP address. 

Sun Aug 17 21:58:32 GMT [N6220-SAS:cf.hwassist.Default.NodemgmtAddr:notice]: The system automatically chose 192.168.1.139 as the local hardware-assist IP address. 

Sun Aug 17 21:58:36 GMT [N6220-SAS:nbt.nbns.socketError:error]: NBT: Cannot send broadcast message on NBNS socket. Error 0x32: Network is down. 

Sun Aug 17 21:58:49 GMT [N6220-SAS:nbt.nbns.registrationComplete:info]: NBT: All CIFS name registrations have completed for the local server. 

Sun Aug 17 21:59:00 GMT [N6220-SAS:monitor.globalStatus.critical:CRITICAL]: Controller failover partner unknown. Controller failover not possible. 

Sun Aug 17 21:59:10 GMT [console_login_mgr:info]: root logged in from console
Sun Aug 17 21:59:26 GMT [N6220-SAS:callhome.performance.snap:info]: Call home for PERFORMANCE SNAPSHOT 

Sun Aug 17 22:00:00 GMT [N6220-SAS:kern.uptime.filer:info]:  10:00pm up 2 mins, 0 NFS ops, 0 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops  

Sun Aug 17 22:00:04 GMT [N6220-SAS:sp.network.link.down:warning]: Service Processor (SP) network port link down due to cable or network errors. 

Sun Aug 17 22:00:04 GMT [N6220-SAS:cf.takeover.disabled:warning]: Controller Failover is licensed but takeover of partner is disabled. 

Sun Aug 17 22:00:04 GMT [N6220-SAS:cf.ic.hourlyBackplaneLinkDownTime:error]: Backplane Interconnect link has been down for 2 minutes 

Sun Aug 17 22:00:35 GMT [N6220-SAS:cf.misc.operatorEnable:notice]: Failover monitor: operator initiated enabling of failover. 

Sun Aug 17 22:00:35 GMT [N6220-SAS:cf.fsm.takeoverOfPartnerDisabled:notice]: Failover monitor: takeover of partner disabled (partner mailbox disks not accessible or invalid). 

Sun Aug 17 22:00:35 GMT [N6220-SAS:cf.fsm.takeoverByPartnerDisabled:notice]: Failover monitor: takeover of N6220-SAS by partner disabled (interconnect error). 

Sun Aug 17 22:00:39 GMT [N6220-SAS:cf.fsm.partnerNotResponding:notice]: Failover monitor: partner not responding 

Sun Aug 17 22:00:49 GMT [N6220-SAS:ha.takeoverImpDegraded:warning]: Takeover of the partner node is impossible due to lack of connectivity to the partner mailbox disks. 

Sun Aug 17 22:01:04 GMT [N6220-SAS:netif.linkUp:info]: Ethernet e0a: Link up. 

Sun Aug 17 22:03:11 GMT [N6220-SAS:wafl.quota.sec.change:notice]: security style for /vol/vol0/ changed from unix to mixed 

Sun Aug 17 22:04:00 GMT [N6220-SAS:wafl.quota.sec.change:notice]: security style for /vol/vol0/ changed from mixed to ntfs 

Sun Aug 17 22:06:20 GMT [N6220-SAS:netif.linkDown:info]: Ethernet e0a: Link down, check cable. 

Sun Aug 17 22:06:24 GMT [N6220-SAS:netif.linkUp:info]: Ethernet e0a: Link up. 

Sun Aug 17 22:08:51 GMT [telnet_0:info]: root logged in from host: 192.168.0.100
Sun Aug 17 22:09:30 GMT [N6220-SAS:netif.linkDown:info]: Ethernet e0a: Link down, check cable. 

Sun Aug 17 22:09:43 GMT [N6220-SAS:netif.linkUp:info]: Ethernet e0a: Link up. 

Sun Aug 17 22:09:50 GMT [telnet_0:info]: root logged in from host: 192.168.0.100
Sun Aug 17 22:10:48 GMT [N6220-SAS:callhome.partner.down:CRITICAL]: Call home for PARTNER DOWN, TAKEOVER IMPOSSIBLE 

Sun Aug 17 22:11:16 GMT [N6220-SAS:netif.linkDown:info]: Ethernet e0a: Link down, check cable. 

Sun Aug 17 22:25:08 GMT [N6220-SAS:netif.linkUp:info]: Ethernet e0a: Link up. 

Sun Aug 17 22:25:24 GMT [telnet_0:info]: root logged in from host: 192.168.0.100
N6220-SAS*>

DOMINIC_WYSS
33,602 Views

slot 0: Dual 10G Ethernet Controller T320E-SFP/KR

  Device Type:        CT-FE-3

  Version Number:     T3-SRAM1.1.0-BR1016-02-01-FW7.7.209-DR04

  Serial Number:     jb04050693     

  c0a MAC Address:    00:a0:98:5a:fb:fe (auto-unknown-cfg_down)

  c0b MAC Address:    00:a0:98:5a:fb:ff (auto-unknown-enabling)

I think you have a physical connection issue on the ports, they are not coming up.

which cables are you using?

aborzenkov
33,731 Views

Please boot into maintenance mode and use “ha-config show” to check current HA settings. Looks like your chassis is set to HA, which means it will disable external ports and try internal backplane connection.

DOMINIC_WYSS
15,788 Views

good idea about the ha-config!

but it has to be set to ha, even if it's a single-chassis.

https://library.netapp.com/ecmdocs/ECMM1249823/html/haconfig/GUID-F4091101-4D6B-4577-B337-CBA633143BAC.html

maybe one of the chassis is set to non-ha and does not respond to the interconnect requests?

shane_bradley
15,788 Views

ah of course !

didnt even think about that

Public