Waiting for SP ... Phoenix SecureCore Tiano(TM) Copyright 1985-2021 Phoenix Technologies Ltd. All Rights Reserved Build Date: 09/16/2019 ********************************************** * 9.8 * * ================================== * * PHOENIX SC-T 2009-2021 * ********************************************** CPU = 1 Processor(s) Detected, Cores per Processor = 6 Intel(R) Xeon(R) CPU E5-2620 @ 2.00GHz 24576 MB System RAM Installed 256 KB L2 Cache System BIOS shadowed Video BIOS shadowed USB Device: MICRON eUSB DISK Boot Loader version 5.7 Copyright (C) 2000-2003 Broadcom Corporation. Portions Copyright (C) 2002-2017 NetApp, Inc. All Rights Reserved. Starting AUTOBOOT press Ctrl-C to abort... Autoboot of PRIMARY image aborted by user. LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> ^C LOADER-B> sp status Firmware Version: 3.10 Ethernet Link: up, 100 Mb, full duplex, auto-neg complete Mgmt MAC Address: 00:A0:98:64:47:D0 IPv4 Settings Using DHCP: NO IP Address: 30.30.30.61 Netmask: 255.255.255.0 Gateway: 30.30.30.30 IPv6: Disabled LOADER-B> System halting... ................... Boot Loader version 4.3 Copyright (C) 2000-2003 Broadcom Corporation. Portions Copyright (C) 2002-2014 NetApp, Inc. All Rights Reserved. CPU Type: Intel(R) Xeon(R) CPU C3528 @ 1.73GHz LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> boot_ontap maint Loading X86_64/freebsd/image1/kernel:0x200000/1091691 0x30b000/10943424 0xf7b000/3783336 0x1316aa8/13717528 0x200200/1016 Entry at 0xffffffff8030b000 Loading X86_64/freebsd/image1/platform.ko:0x202c000/4424480 0x2464320/596232 Starting program at 0xffffffff8030b000 NetApp Data ONTAP 9.7P9 arc4random: no preloaded entropy cache ena_rss_init_default_deferred() [TID:100000]: No devclass ena arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache Copyright (C) 1992-2020 NetApp. All rights reserved. ******************************* * * * Press Ctrl-C for Boot Menu. * * * ******************************* cryptomod_fips: Executing Crypto FIPS Self Tests. cryptomod_fips: Crypto FIPS self-test: 'CPU COMPATIBILITY' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 ECB, AES-256 ECB' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CBC, AES-256 CBC' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 GCM, AES-256 GCM' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CCM' passed. cryptomod_fips: Crypto FIPS self-test: 'CTR_DRBG' passed. cryptomod_fips: Crypto FIPS self-test: 'KDF' passed. cryptomod_fips: Crypto FIPS self-test: 'SHA1, SHA256, SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'HMAC-SHA1, HMAC-SHA256, HMAC-SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'PBKDF2' passed. cryptomod_fips: Crypto FIPS self-test: 'Self-integrity' passed. Jun 14 08:59:13 Battery charge capacity: 3840 mA*hr. Power outage protection flash de-staging cycles: 115 Pensando Offload Driver, ver 1.0.1-E-31 Pensando Ethernet NIC Driver, ver: 1.0.1-E-39 ionic_rdma ver 1.0.1-E-39 : Pensando RoCE HCA driver pnso provider init started. pnso init failed in pnso_init() : 35 hwo: Node is using hardware provider : 1. cryptomod_fips: Executing Crypto FIPS Self Tests. cryptomod_fips: Crypto FIPS self-test: 'CPU COMPATIBILITY' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 ECB, AES-256 ECB' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CBC, AES-256 CBC' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 GCM, AES-256 GCM' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CCM' passed. cryptomod_fips: Crypto FIPS self-test: 'CTR_DRBG' passed. cryptomod_fips: Crypto FIPS self-test: 'KDF' passed. cryptomod_fips: Crypto FIPS self-test: 'SHA1, SHA256, SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'HMAC-SHA1, HMAC-SHA256, HMAC-SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'PBKDF2' passed. cryptomod_fips: Crypto FIPS self-test: 'Self-integrity' passed. Jun 14 09:00:01 [demonewcls-01:raid.autoPart.disabled:ALERT]: Disk auto-partitioning is disabled on this system: the system needs a minimum of 8 usable internal hard disks. Jun 14 09:00:01 [demonewcls-01:callhome.raid.adp.disabled:ALERT]: Disk auto-partitioning is disabled on this system: ADP DISABLED. 5 You have selected the maintenance boot option: the system has booted in maintenance mode allowing the following operations to be performed: ? acorn acpadmin aggr cdpd cna_flash coredump_save coredump_show disk disk_list disk_shelf diskcopy dumpblock environment fctest fru_led ha-config halt help ifconfig led_off led_on lldp nicadmin nv8 psmadmin raid_config sasadmin sasstat scsi sldiag storage stsb sysconfig systemshell ucadmin version vmdisk_object_store vmservices vol vol_db Type "help " for more details. In a High Availablity configuration, you MUST ensure that the partner node is (and remains) down, or that takeover is manually disabled on the partner node, because High Availability software is not started or fully enabled in Maintenance mode. FAILURE TO DO SO CAN RESULT IN YOUR FILESYSTEMS BEING DESTROYED NOTE: It is okay to use 'show/status' sub-commands such as 'disk show or aggr status' in Maintenance mode while the partner is up Continue with boot? yes yes Jun 14 09:00:37 [demonewcls-01:sas.link.error:error]: Could not recover link on SAS adapter 0a after 45 seconds. Offlining the adapter. Jun 14 09:00:58 [demonewcls-01:ses.status.psError:ALERT]: DS2246 (S/N SHFHU1516001252) shelf 1 on channel 0b power error for Power supply 2: critical status; AC Fail. This module is on the rear of the shelf at the bottom right. Jun 14 09:00:58 [demonewcls-01:callhome.shlf.power.intr:error]: Call home for SHELF POWER INTERRUPTED web services cannot be started in 'maintenance' mode. Mon Jun 14 09:01:02 UTC 2021 *> Jun 14 09:01:02 [demonewcls-01:kern.syslog.msg:notice]: root logged in from console Jun 14 09:01:02 [demonewcls-01:kern.syslog.msg:notice]: root logged in from SP *> *> *> *> disk show -a Local System ID: 536960613 DISK OWNER POOL SERIAL NUMBER HOME DR HOME ------------ ------------- ----- ------------- ------------- ------------- 0b.01.18 testcls-01(537062551) Pool0 S142NEAFA02448 testcls-02(537062842) 0b.01.4 testcls-01(537062551) Pool0 S142NEAFB00006 testcls-02(537062842) 0b.01.6 testcls-01(537062551) Pool0 S142NEAFA02455 testcls-02(537062842) 0b.01.17 testcls-01(537062551) Pool0 S142NEAFA02443 testcls-02(537062842) 0b.01.15 testcls-01(537062551) Pool0 S142NEAFA02432 testcls-02(537062842) 0b.01.20 testcls-01(537062551) Pool0 S3L1DDE1 testcls-01(537062551) 0b.01.11 testcls-01(537062551) Pool0 S142NEAFB00005 testcls-02(537062842) 0b.01.9 testcls-01(537062551) Pool0 S142NEAFB00061 testcls-02(537062842) 0b.01.10 testcls-01(537062551) Pool0 S142NEAFA05776 testcls-02(537062842) 0b.01.22 testcls-01(537062551) Pool0 S4033QM0 testcls-02(537062842) 0b.01.16 testcls-01(537062551) Pool0 W401PWSP testcls-02(537062842) 0b.01.12 testcls-02(537062842) Pool0 KZJ87D1H testcls-02(537062842) 0b.01.14 testcls-02(537062842) Pool0 L0H535EG testcls-02(537062842) 0b.01.3 testcls-01(537062551) Pool0 L0H1W19G testcls-01(537062551) 0b.01.19 testcls-01(537062551) Pool0 L0H95E8G testcls-02(537062842) 0b.01.1 testcls-01(537062551) Pool0 L0H787KG testcls-01(537062551) 0b.01.23 testcls-01(537062551) Pool0 X970A002F0BD testcls-02(537062842) 0b.01.0 testcls-02(537062842) Pool0 06K6EMWA testcls-02(537062842) 0b.01.13 testcls-01(537062551) Pool0 06K75WGA testcls-01(537062551) 0b.01.21 testcls-01(537062551) Pool0 06V4730Z testcls-02(537062842) *> disk reassign -s 537062842 -d 536960613 Partner node must not be in Takeover mode during disk reassignment from maintenance mode. Serious problems could result!! Do not proceed with reassignment if the partner is in takeover mode. Abort reassignment (y/n)? n After the node becomes operational, you must perform a takeover and giveback of the HA partner node to ensure disk reassignment is successful. Do you want to continue (y/n)? y Disk ownership will be updated on all disks previously belonging to Filer with sysid 537062842. Do you want to continue (y/n)? y Cannot do remote rescan. Use 'run local disk show' on the console of ?? for it to scan the newly assigned disks Jun 14 09:01:43 [demonewcls-01:diskown.RescanMessageFailed:error]: Could not send rescan message to ??. *> *> *> *> *> mailbox destroy local Destroying mailboxes forces a node to create new empty mailboxes, which clears any takeover state, removes all knowledge of out-of-date plexes of mirrored volumes, and will prevent management services from going online in 2-node cluster HA configurations. Are you sure you want to destroy the local mailboxes? yes Mailboxes destroyed. Warning: Two-node cluster configurations only: after reboot, verify that cluster HA is enabled using the (privilege: diag) "cluster ha show" command. *> *> *> *> ha-conofig show ha-conofig not found. Type '?' for a list of commands *> Jun 14 09:02:01 [demonewcls-01:ses.status.ACPWarn:error]: DS2246 (S/N SHFHU1516001252) shelf 1 on channel 0b ACP Processor warning for SAS shelf ACP processor 1: ACP status not available. Element in transition mode. This module is on the rear of the shelf at the top left, on shelf module A. ha-config show Chassis HA configuration: ha Controller HA configuration: ha *> *> *> *> *> halt Terminated . Uptime: 4m14s System halting... BIOS version: 9.8 Portions Copyright (c) 2011-2017 NetApp. All Rights Reserved Phoenix SecureCore Tiano(TM) Copyright 1985-2021 Phoenix Technologies Ltd. All Rights Reserved Build Date: 09/16/2019 ********************************************** * 9.8 * * ================================== * * PHOENIX SC-T 2009-2021 * ********************************************** CPU = 1 Processor(s) Detected, Cores per Processor = 6 Intel(R) Xeon(R) CPU E5-2620 @ 2.00GHz 24576 MB System RAM Installed 256 KB L2 Cache System BIOS shadowed Video BIOS shadowed USB Device: MICRON eUSB DISK Boot Loader version 5.7 Copyright (C) 2000-2003 Broadcom Corporation. Portions Copyright (C) 2002-2017 NetApp, Inc. All Rights Reserved. LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> boot_ontap ^C LOADER-B> show date Current date & time is: 06/14/2021 09:03:55 LOADER-B> printenv partner-sysid Variable Name Value -------------------- -------------------------------------------------- partner-sysid 0536960880 LOADER-B> setenv partner-sysid 537062551 LOADER-B> printenv partner-sysid Variable Name Value -------------------- -------------------------------------------------- partner-sysid 537062551 LOADER-B> saveenv LOADER-B> LOADER-B> LOADER-B> LOADER-B> LOADER-B> boot_ontap menu Loading X86_64/freebsd/image1/kernel:0x200000/1091691 0x30b000/10943424 0xf7b000/3783336 0x1316aa8/13717528 0x200200/1016 Entry at 0xffffffff8030b000 Loading X86_64/freebsd/image1/platform.ko:0x202c000/4424480 0x2464320/596232 Starting program at 0xffffffff8030b000 NetApp Data ONTAP 9.7P9 arc4random: no preloaded entropy cache ena_rss_init_default_deferred() [TID:100000]: No devclass ena arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache Copyright (C) 1992-2020 NetApp. All rights reserved. ******************************* * * * Press Ctrl-C for Boot Menu. * * * ******************************* cryptomod_fips: Executing Crypto FIPS Self Tests. cryptomod_fips: Crypto FIPS self-test: 'CPU COMPATIBILITY' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 ECB, AES-256 ECB' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CBC, AES-256 CBC' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 GCM, AES-256 GCM' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CCM' passed. cryptomod_fips: Crypto FIPS self-test: 'CTR_DRBG' passed. cryptomod_fips: Crypto FIPS self-test: 'KDF' passed. cryptomod_fips: Crypto FIPS self-test: 'SHA1, SHA256, SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'HMAC-SHA1, HMAC-SHA256, HMAC-SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'PBKDF2' passed. cryptomod_fips: Crypto FIPS self-test: 'Self-integrity' passed. Jun 14 09:06:54 Battery charge capacity: 3840 mA*hr. Power outage protection flash de-staging cycles: 115 Please choose one of the following: (1) Normal Boot. (2) Boot without /etc/rc. (3) Change password. (4) Clean configuration and initialize all disks. (5) Maintenance mode boot. (6) Update flash from backup config. (7) Install new software first. (8) Reboot node. (9) Configure Advanced Drive Partitioning. Selection (1-9)? 6 This will replace all flash-based configuration with the last backup to disks. Are you sure you want to continue?: yes Pensando Offload Driver, ver 1.0.1-E-31 Pensando Ethernet NIC Driver, ver: 1.0.1-E-39 ionic_rdma ver 1.0.1-E-39 : Pensando RoCE HCA driver pnso provider init started. pnso init failed in pnso_init() : 35 hwo: Node is using hardware provider : 1. cryptomod_fips: Executing Crypto FIPS Self Tests. cryptomod_fips: Crypto FIPS self-test: 'CPU COMPATIBILITY' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 ECB, AES-256 ECB' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CBC, AES-256 CBC' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 GCM, AES-256 GCM' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CCM' passed. cryptomod_fips: Crypto FIPS self-test: 'CTR_DRBG' passed. cryptomod_fips: Crypto FIPS self-test: 'KDF' passed. cryptomod_fips: Crypto FIPS self-test: 'SHA1, SHA256, SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'HMAC-SHA1, HMAC-SHA256, HMAC-SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'PBKDF2' passed. cryptomod_fips: Crypto FIPS self-test: 'Self-integrity' passed. Jun 14 09:08:54 [demonewcls-01:raid.autoPart.disabled:ALERT]: Disk auto-partitioning is disabled on this system: the system needs a minimum of 8 usable internal hard disks. Jun 14 09:08:54 [demonewcls-01:callhome.raid.adp.disabled:ALERT]: Disk auto-partitioning is disabled on this system: ADP DISABLED. There appears to be a consistent set of aggregates for a system with an NVRAM board other than the one in this system. NVRAM Jun 14 09:08:54 [demonewcls-01:cf.fm.kernelMismatch:error]: Failover monitor: possible kernel mismatch detected local 'NetApp//9.7P9', partner 'NetApp//9.7P7' contents will be discarded and the disks will be labeled with the serial Jun 14 09:08:54 [demonewcls-01:cf.fm.noMBDisksOrIc:error]: Could not find the local mailbox disks. Could not determine the firmware state of the partner through the HA interconnect. number of this system's NVRAM board. Jun 14 09:08:54 [demonewcls-01:raid.assim.tree.foreign:error]: raidtree_verify: Aggregate aggr0_testcls_02 is a foreign aggregate and is being taken offline. Use the 'storage aggregate online' command to bring it online. Jun 14 09:08:54 [demonewcls-01:raid.assim.tree.noRootVol:error]: No usable root volume was found! Jun 14 09:08:54 [demonewcls-01:raid.assim.label.noNativeVols:notice]: No native volumes or aggregates detected, assuming NVRAM card swap. 1 NVRAM system id(s) (537062842, 0, 0...) found among all volumes. NVRAM system id 537062842 selected. NVRAM card swap detected by RAID. boot_from_disk:last_booted_OS:9.7P9 Jun 14 09:08:54 [demonewcls-01:raid.assim.label.makeNativeVol:notice]: Aggregate aggr0_testcls_02 turned into a native aggregate. Jun 14 09:08:55 [demonewcls-01:cf.fm.notkoverClusterDisable:error]: Failover monitor: takeover disabled (restart) Jun 14 09:08:55 [demonewcls-01:wafl.transition.cp.completed:notice]: Transition CP with reason flush_b4_mounted, 00000000 for replaying=0,0 unmounting=0,0 total=2,1 volumes with a total of total=79 incoming=6 dirty buffers took 80ms with longest CP phases being CP_P3A_VOLINFO=28, CP_P2V_INO=27, CP_P2_FLUSH=20 on aggregate aggr0_testcls_02. Jun 14 09:08:55 [demonewcls-01:tar.csum.mismatch:notice]: Stored checksum 1733855820 does not match calculated checksum 464045747, extracting local://mnt/prestage/mroot.tgz. Jun 14 09:08:56 [demonewcls-01:cf.fsm.takeoverOfPartnerDisabled:error]: Failover monitor: takeover of testcls-01 disabled (Controller Failover takeover disabled). Jun 14 09:08:56 [demonewcls-01:wafl.transition.cp.completed:notice]: Transition CP with reason none, 00000000 for replaying=0,0 unmounting=0,0 total=2,1 volumes with a total of total=536 incoming=427 dirty buffers took 73ms with longest CP phases being CP_P2_FLUSH=32, CP_P3A_VOLINFO=22, CP_P1_CLEAN=5 on aggregate aggr0_testcls_02. Jun 14 09:08:58 [demonewcls-01:tar.csum.mismatch:notice]: Stored checksum 1198086147 does not match calculated checksum 1032807262, extracting local://mnt/prestage/pmroot.tgz. sysconfig: Card in slot 1 (9-1077-2532-9) is not supported. Jun 14 09:08:59 [demonewcls-01:cmds.sysconf.logErr:notice]: sysconfig: Card in slot 1 (9-1077-2532-9) is not supported. . Jun 14 09:08:59 [demonewcls-01:callhome.sys.config.crit:error]: Call home for SYSTEM CONFIGURATION ERROR Jun 14 09:08:59 [demonewcls-01:kern.syslog.msg:notice]: domain xing mode: off, domain xing interrupt: false Jun 14 09:08:59 [demonewcls-01:clam.invalid.config:error]: Local node (name=unknown, id=0) is in an invalid configuration for providing CLAM functionality. CLAM cannot determine the identity of the HA partner. Jun 14 09:09:00 [demonewcls-01:monitor.globalStatus.critical:EMERGENCY]: Controller failover of testcls-01 is not possible: Controller Failover takeover disabled. varfs_backup_restore: bootarg.abandon_varfs is set! Skipping /var backup. varfs_backup_restore: bootarg.abandon_varfs is set! Skipping /var backup. varfs_backup_restore: update checksum for varfs.tgz varfs_backup_restore: restore using /mroot/etc/varfs.tgz varfs_backup_restore: attempting to restore /var/kmip to the boot device varfs_backup_restore: failed to restore /var/kmip to the boot device varfs_backup_restore: attempting to restore env file to the boot device varfs_backup_restore: successfully restored env file to the boot device wrote key file "/tmp/rndc.key" varfs_backup_restore: bootarg.abandon_varfs is set! Skipping /var backup. varfs_backup_restore: bootarg.abandon_varfs is set! Skipping /var backup. varfs_backup_restore: timeout waiting for login varfs_backup_restore: Rebooting to load the new varfs Terminated . varfs_backup_restore: bootarg.abandon_varfs is set! Skipping /var backup. Uptime: 6m7s System rebooting... BIOS version: 9.8 Portions Copyright (c) 2011-2017 NetApp. All Rights Reserved Phoenix SecureCore Tiano(TM) Copyright 1985-2021 Phoenix Technologies Ltd. All Rights Reserved Build Date: 09/16/2019 ********************************************** * 9.8 * * ================================== * * PHOENIX SC-T 2009-2021 * ********************************************** CPU = 1 Processor(s) Detected, Cores per Processor = 6 Intel(R) Xeon(R) CPU E5-2620 @ 2.00GHz 24576 MB System RAM Installed 256 KB L2 Cache System BIOS shadowed Video BIOS shadowed USB Device: MICRON eUSB DISK Boot Loader version 5.7 Copyright (C) 2000-2003 Broadcom Corporation. Portions Copyright (C) 2002-2017 NetApp, Inc. All Rights Reserved. Starting AUTOBOOT press Ctrl-C to abort... Loading X86_64/freebsd/image1/kernel:0x200000/1091691 0x30b000/10943424 0xf7b000/3783336 0x1316aa8/13717528 0x200200/1016 Entry at 0xffffffff8030b000 Loading X86_64/freebsd/image1/platform.ko:0x202c000/4424480 0x2464320/596232 Starting program at 0xffffffff8030b000 NetApp Data ONTAP 9.7P9 arc4random: no preloaded entropy cache ena_rss_init_default_deferred() [TID:100000]: No devclass ena arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache Copyright (C) 1992-2020 NetApp. All rights reserved. Restoring env file from boot media... copy_env_file:scenario = head upgrade Successfully restored env file from boot media... Rebooting to load the restored env file... Uptime: 12s System rebooting... BIOS version: 9.8 Portions Copyright (c) 2011-2017 NetApp. All Rights Reserved Phoenix SecureCore Tiano(TM) Copyright 1985-2021 Phoenix Technologies Ltd. All Rights Reserved Build Date: 09/16/2019 ********************************************** * 9.8 * * ================================== * * PHOENIX SC-T 2009-2021 * ********************************************** CPU = 1 Processor(s) Detected, Cores per Processor = 6 Intel(R) Xeon(R) CPU E5-2620 @ 2.00GHz 24576 MB System RAM Installed 256 KB L2 Cache System BIOS shadowed Video BIOS shadowed USB Device: MICRON eUSB DISK Boot Loader version 5.7 Copyright (C) 2000-2003 Broadcom Corporation. Portions Copyright (C) 2002-2017 NetApp, Inc. All Rights Reserved. Starting AUTOBOOT press Ctrl-C to abort... Loading X86_64/freebsd/image1/kernel:0x200000/1091691 0x30b000/10943424 0xf7b000/3783336 0x1316aa8/13717528 0x200200/1016 Entry at 0xffffffff8030b000 Loading X86_64/freebsd/image1/platform.ko:0x202c000/4424480 0x2464320/596232 Starting program at 0xffffffff8030b000 NetApp Data ONTAP 9.7P9 arc4random: no preloaded entropy cache ena_rss_init_default_deferred() [TID:100000]: No devclass ena arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache arc4random: no preloaded entropy cache Copyright (C) 1992-2020 NetApp. All rights reserved. ******************************* * * * Press Ctrl-C for Boot Menu. * * * ******************************* cryptomod_fips: Executing Crypto FIPS Self Tests. cryptomod_fips: Crypto FIPS self-test: 'CPU COMPATIBILITY' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 ECB, AES-256 ECB' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CBC, AES-256 CBC' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 GCM, AES-256 GCM' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CCM' passed. cryptomod_fips: Crypto FIPS self-test: 'CTR_DRBG' passed. cryptomod_fips: Crypto FIPS self-test: 'KDF' passed. cryptomod_fips: Crypto FIPS self-test: 'SHA1, SHA256, SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'HMAC-SHA1, HMAC-SHA256, HMAC-SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'PBKDF2' passed. cryptomod_fips: Crypto FIPS self-test: 'Self-integrity' passed. WARNING: System ID mismatch. This usually occurs when replacing a boot device or NVRAM cards! Override system ID? {y|n} y Running the upgrade system bootarg.upgrade="true" bootarg.from.version="9.7P7" bootarg.to.version="9.7P9" Jun 14 09:14:40 Battery charge capacity: 3840 mA*hr. Power outage protection flash de-staging cycles: 115 Pensando Offload Driver, ver 1.0.1-E-31 Pensando Ethernet NIC Driver, ver: 1.0.1-E-39 ionic_rdma ver 1.0.1-E-39 : Pensando RoCE HCA driver pnso provider init started. pnso init failed in pnso_init() : 35 hwo: Node is using hardware provider : 1. cryptomod_fips: Executing Crypto FIPS Self Tests. cryptomod_fips: Crypto FIPS self-test: 'CPU COMPATIBILITY' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 ECB, AES-256 ECB' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CBC, AES-256 CBC' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 GCM, AES-256 GCM' passed. cryptomod_fips: Crypto FIPS self-test: 'AES-128 CCM' passed. cryptomod_fips: Crypto FIPS self-test: 'CTR_DRBG' passed. cryptomod_fips: Crypto FIPS self-test: 'KDF' passed. cryptomod_fips: Crypto FIPS self-test: 'SHA1, SHA256, SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'HMAC-SHA1, HMAC-SHA256, HMAC-SHA512' passed. cryptomod_fips: Crypto FIPS self-test: 'PBKDF2' passed. cryptomod_fips: Crypto FIPS self-test: 'Self-integrity' passed. Jun 14 09:16:18 [testcls-02:raid.autoPart.disabled:ALERT]: Disk auto-partitioning is disabled on this system: the system needs a minimum of 8 usable internal hard disks. Jun 14 09:16:18 [testcls-02:callhome.raid.adp.disabled:ALERT]: Disk auto-partitioning is disabled on this system: ADP DISABLED. Jun 14 09:16:18 [testcls-02:cf.fm.kernelMismatch:error]: Failover monitor: possible kernel mismatch detected local 'NetApp//9.7P9', partner 'NetApp//9.7P7' Jun 14 09:16:18 [testcls-02:cf.fm.notkoverClusterDisable:error]: Failover monitor: takeover disabled (restart) Jun 14 09:16:19 [testcls-02:wafl.transition.cp.completed:notice]: Transition CP with reason flush_b4_mounted, 00000000 for replaying=0,0 unmounting=0,0 total=2,1 volumes with a total of total=79 incoming=3 dirty buffers took 92ms with longest CP phases being CP_P3A_VOLINFO=32, CP_P2V_INO=30, CP_P2_FLUSH=23 on aggregate aggr0_testcls_02. Jun 14 09:16:19 [testcls-02:kern.syslog.msg:notice]: The system was down for 242 seconds Jun 14 09:16:19 [testcls-02:cf.fsm.takeoverOfPartnerDisabled:error]: Failover monitor: takeover of testcls-01 disabled (Controller Failover takeover disabled). sysconfig: Card in slot 1 (9-1077-2532-9) is not supported. Jun 14 09:16:19 [testcls-02:cmds.sysconf.logErr:notice]: sysconfig: Card in slot 1 (9-1077-2532-9) is not supported. . Jun 14 09:16:19 [testcls-02:callhome.sys.config.crit:error]: Call home for SYSTEM CONFIGURATION ERROR Jun 14 09:16:20 [testcls-02:kern.syslog.msg:notice]: domain xing mode: off, domain xing interrupt: false Jun 14 09:16:20 [testcls-02:clam.invalid.config:error]: Local node (name=unknown, id=0) is in an invalid configuration for providing CLAM functionality. CLAM cannot determine the identity of the HA partner. wrote key file "/tmp/rndc.key" Jun 14 09:17:00 [testcls-02:monitor.globalStatus.critical:EMERGENCY]: Controller failover of testcls-01 is not possible: Controller Failover takeover disabled. Mon Jun 14 09:22:06 UTC 2021 login: admin Password: ****************************************************** * This is a serial console session. Output from this * * session is mirrored on the SP console session. * ****************************************************** testcls::> testcls::> testcls::> testcls::> network port show show show-address-filter-info testcls::> network port show Node: testcls-02 Speed(Mbps) Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status --------- ------------ ---------------- ---- ---- ----------- -------- e0M Default - up 1500 auto/1000 - e0a Default - up 1500 auto/10000 - e0b Default - up 1500 auto/10000 - e0c Default - down 1500 auto/auto - e0d Default - down 1500 auto/auto - e0e Cluster - down 9000 auto/auto - e0f Cluster - down 9000 auto/auto - 7 entries were displayed. testcls::> network port modify -node testcls-0 testcls-01 testcls-02 testcls::> network port modify -node testcls-02 -port e0 e0M e0a e0b e0c e0d e0e e0f testcls::> network port modify -node testcls-02 -port e0a -mtu 9000 -ipspace Cluster testcls::> network port modify -node testcls-02 -port e0b -mtu 9000 -ipspace Cluster testcls::> network interface s service service-policy show testcls::> network interface show Logical Status Network Current Current Is Vserver Interface Admin/Oper Address/Mask Node Port Home ----------- ---------- ---------- ------------------ ------------- ------- ---- testcls testcls-02_mgmt1 up/up 30.30.30.118/24 testcls-02 e0M true testcls::> network port show Node: testcls-02 Speed(Mbps) Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status --------- ------------ ---------------- ---- ---- ----------- -------- e0M Default - up 1500 auto/1000 - e0a Cluster - up 9000 auto/10000 - e0b Cluster - up 9000 auto/10000 - e0c Default - down 1500 auto/auto - e0d Default - down 1500 auto/auto - e0e Cluster - down 9000 auto/auto - e0f Cluster - down 9000 auto/auto - 7 entries were displayed. testcls::> system reboot -node testcls-0 testcls-01 testcls-02 testcls::> system reboot -node testcls-02 - -inhibit-takeover -reason -dump -skip-lif-migration-before-reboot -ignore-quorum-warnings -ignore-strict-sync-warnings testcls::> system reboot -node testcls-02