ONTAP Discussions

Highlighted

Not all disks available after zero and initialisation

I need some help with a filer, it is running 9.4 and has 24x1.2TB drives. 

 

It was already deployed but has to be wiped and redeployed elsewhere. 

 

Ive console onto both controllers at the same time, rebooted each node, broken the boot process and selected option 4. 

 

The process completes and asks for the management interface to be configured, which I’ve done for both. Completed the cluster configuration. However when I try to create an aggregate I can only assign 7.5TB, there are no other spare drives. 

 

I cam see all 24 drives on the filer, can someone tell me what I’ve missed or how I can resolve this please?

 

system node run -node Filer01-01 -command sysconfig -r

Aggregate Filer01_01_root (online, raid_dp) (block checksums)

Plex /Filer01_01_root/plex0 (online, normal, active, pool0)

RAID group /Filer01_01_root/plex0/rg0 (normal, block checksums)

 

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

dparity 0a.00.1P2 0a 0 1 SA:B 0 SAS 10000 55176/113000448 55184/113016832

parity 0a.00.3P2 0a 0 3 SA:B 0 SAS 10000 55176/113000448 55184/113016832

data 0a.00.5P2 0a 0 5 SA:B 0 SAS 10000 55176/113000448 55184/113016832

data 0a.00.7P2 0a 0 7 SA:B 0 SAS 10000 55176/113000448 55184/113016832

data 0a.00.9P2 0a 0 9 SA:B 0 SAS 10000 55176/113000448 55184/113016832

data 0a.00.11P2 0a 0 11 SA:B 0 SAS 10000 55176/113000448 55184/113016832

data 0a.00.13P2 0a 0 13 SA:B 0 SAS 10000 55176/113000448 55184/113016832

data 0a.00.15P2 0a 0 15 SA:B 0 SAS 10000 55176/113000448 55184/113016832

data 0a.00.17P2 0a 0 17 SA:B 0 SAS 10000 55176/113000448 55184/113016832

data 0a.00.19P2 0a 0 19 SA:B 0 SAS 10000 55176/113000448 55184/113016832

 

 

Pool1 spare disks (empty)

 

Pool0 spare disks

 

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

Spare disks for block checksum

spare 0a.00.21P2 0a 0 21 SA:B 0 SAS 10000 55176/113000448 55184/113016832

spare 0a.00.23P2 0a 0 23 SA:B 0 SAS 10000 55176/113000448 55184/113016832

spare 0a.00.1P1 0a 0 1 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.3P1 0a 0 3 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.5P1 0a 0 5 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.7P1 0a 0 7 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.9P1 0a 0 9 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.11P1 0a 0 11 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.13P1 0a 0 13 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.15P1 0a 0 15 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.17P1 0a 0 17 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.19P1 0a 0 19 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592 (not zeroed)

spare 0a.00.21P1 0a 0 21 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592

spare 0a.00.23P1 0a 0 23 SA:B 0 SAS 10000 1087139/2226462208 1087147/2226478592

 

Partner disks

 

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

partner 0b.00.4P2 0b 0 4 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.2P2 0b 0 2 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.20P2 0b 0 20 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.10P2 0b 0 10 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.20P1 0b 0 20 SA:A 0 SAS 10000 0/0 1087147/2226478592

partner 0b.00.16P2 0b 0 16 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.0P2 0b 0 0 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.18P2 0b 0 18 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.14P2 0b 0 14 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.12P2 0b 0 12 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.8P2 0b 0 8 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.22P1 0b 0 22 SA:A 0 SAS 10000 0/0 1087147/2226478592

partner 0b.00.22P2 0b 0 22 SA:A 0 SAS 10000 0/0 55184/113016832

partner 0b.00.6P2 0b 0 6 SA:A 0 SAS 10000 0/0 55184/113016832

5 REPLIES 5
Highlighted

Re: Not all disks available after zero and initialisation

what's the output of: 

"storage disk show -partition-ownership "

 

and sysconfig without the -r  

 

system node run -node * -command sysconfig

Highlighted

Re: Not all disks available after zero and initialisation

storage disk show -partition-ownership
Disk     Partition Home              Owner             Home ID     Owner ID
-------- --------- ----------------- ----------------- ----------- -----------
 
Info: This cluster has partitioned disks. To get a complete list of spare disk capacity use "storage aggregate
      show-spare-disks".
1.0.0    Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
1.0.1    Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.2    Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
1.0.3    Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.4    Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
1.0.5    Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.6    Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
1.0.7    Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
 
Disk     Partition Home              Owner             Home ID     Owner ID
-------- --------- ----------------- ----------------- ----------- -----------
1.0.8    Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
1.0.9    Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.10   Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
1.0.11   Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.12   Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
1.0.13   Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.14   Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
1.0.15   Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.16   Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
 
Disk     Partition Home              Owner             Home ID     Owner ID
-------- --------- ----------------- ----------------- ----------- -----------
1.0.17   Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.18   Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02                        537127838   537127495
1.0.19   Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.20   Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02      Filer01-02      537127838   537127838
1.0.21   Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
1.0.22   Container Filer01-02      Filer01-02      537127838   537127838
         Root      Filer01-02      Filer01-02      537127838   537127838
         Data      Filer01-02      Filer01-02      537127838   537127838
1.0.23   Container Filer01-01      Filer01-01      537127495   537127495
         Root      Filer01-01      Filer01-01      537127495   537127495
         Data      Filer01-01      Filer01-01      537127495   537127495
24 entries were displayed.
 
 
system node run -node * -command sysconfig
2 entries were acted on.
 
Node: Filer01-01
        NetApp Release 9.4P1: Fri Jul 20 23:30:57 EDT 2018
        System ID: #### (Filer01-01); partner ID: #### (Filer01-02)
        System Serial Number: #### (Filer01-01)
        System Rev: B1
        System Storage Configuration: Multi-Path HA
        System ACP Connectivity: Inband Active
        All-Flash Optimized: false
        Backplane Part Number: DS22412
        Backplane Rev:
        Backplane Serial Number: #####
        slot 0: System Board
                Model Name:         FAS2650
                Processors:         6
                Processor type:     Intel(R) Xeon(R) CPU D-1528 @ 1.90GHz
                Memory Size:        32768 MB
                Memory Attributes:  Hoisting
                                    Rank Interleaving
                                    Channel Interleaving
                Controller:         A
        Service Processor           Status: Online
        slot 0: Internal 10/100/1000 Ethernet Switch
                Wrench:             auto-1000t-fd-up
                Device Type:        88E6176
        slot 0: Gigabit Ethernet I210
                e0M MAC Address:    00:a0:98:d5:1b:e8 (auto-1000t-fd-up)
        slot 0: 10 Gigabit Ethernet Controller IX5-SFP+
                e0a MAC Address:    00:a0:98:d5:1b:da (auto-10g_twinax-fd-up)
                e0b MAC Address:    00:a0:98:d5:1b:db (auto-10g_twinax-fd-up)
        slot 0: 1G/10G Ethernet Controller CNA EP 8324
                (Multiple Dual-port, QLogic CNA 8324(8362) rev. 2)
                e0c MAC Address:    00:a0:98:d5:1b:dc (auto-unknown-fd-down)
                e0d MAC Address:    00:a0:98:d5:1b:dd (auto-1000t-fd-up)
                e0e MAC Address:    02:a0:98:d5:1b:e8 (auto-1000t-fd-up)
                e0f MAC Address:    00:a0:98:d5:1b:df (auto-unknown-fd-down)
        slot 0: PLX DMA Engine 2 (PLX87d0)
                State:             Active
        slot 0: Interconnect HBA: Generic OFED Provider
                Port Name:          ic0a
                GID:                fe80:0000:0000:0000:0000:0000:0000:0104
        slot 0: NTB Interconnect (PLX87b0)
                Max HW Data Rate:  PCIe Gen 3 x 8
                HW Data Rate:      PCIe Gen 3 x 8
                SW Data Rate:      PCIe Gen 3 x 8
                Logical Link:      Up
                Port State:        Enabled
        slot 0: SAS Host Adapter 0a
                24 Disks:            27416.4GB
                1 shelf with IOM12E
        slot 0: SAS Host Adapter 0b
                24 Disks:            27416.4GB
                1 shelf with IOM12E
        slot 0: Fibre Channel Target Host Adapter 0c
        slot 0: Fibre Channel Target Host Adapter 0d
        slot 0: Fibre Channel Target Host Adapter 0e
        slot 0: Fibre Channel Target Host Adapter 0f
        slot 0: Intel BDW SATA Controller u0a (0x92311000)
                boot0   SV9MST6D120GLM41NP     FW1122 114473MB  512B/sect        (SPG1750076Z)
        slot 0: Intel USB EHCI Adapter u0b (0x92312000)
        slot 1: Flash Cache NVMe
                Serial Number:      #######
                Part Number:        119-00328
                Hardware Revision:  A0
                Firmware Version:   NA01
                Model Name:         X3310A
                Capacity:           512 GB
                Vendor Name:        Samsung
                State:              ok
 
Node: Filer01-02
        NetApp Release 9.4P1: Fri Jul 20 23:30:57 EDT 2018
        System ID: #### (Filer01-02); partner ID: #### (Filer01-01)
        System Serial Number: #### (Filer01-02)
        System Rev: B1
        System Storage Configuration: Multi-Path HA
        System ACP Connectivity: Inband Active
        All-Flash Optimized: false
        Backplane Part Number: DS22412
        Backplane Rev:
        Backplane Serial Number: ####
        slot 0: System Board
                Model Name:         FAS2650
                Processors:         6
                Processor type:     Intel(R) Xeon(R) CPU D-1528 @ 1.90GHz
                Memory Size:        32768 MB
                Memory Attributes:  Hoisting
                                    Rank Interleaving
                                    Channel Interleaving
                Controller:         B
        Service Processor           Status: Online
        slot 0: Internal 10/100/1000 Ethernet Switch
                Wrench:             auto-1000t-fd-up
                Device Type:        88E6176
        slot 0: Gigabit Ethernet I210
                e0M MAC Address:    00:a0:98:d5:36:7a (auto-1000t-fd-up)
        slot 0: 10 Gigabit Ethernet Controller IX5-SFP+
                e0a MAC Address:    00:a0:98:d5:36:6c (auto-10g_twinax-fd-up)
                e0b MAC Address:    00:a0:98:d5:36:6d (auto-10g_twinax-fd-up)
        slot 0: 1G/10G Ethernet Controller CNA EP 8324
                (Multiple Dual-port, QLogic CNA 8324(8362) rev. 2)
                e0c MAC Address:    00:a0:98:d5:36:6e (auto-1000t-fd-up)
                e0d MAC Address:    00:a0:98:d5:36:6f (auto-1000t-fd-up)
                e0e MAC Address:    02:a0:98:d5:36:7a (auto-1000t-fd-up)
                e0f MAC Address:    00:a0:98:d5:36:71 (auto-unknown-fd-down)
        slot 0: PLX DMA Engine 2 (PLX87d0)
                State:             Active
        slot 0: Interconnect HBA: Generic OFED Provider
                Port Name:          ic0a
                GID:                fe80:0000:0000:0000:0000:0000:0000:0105
        slot 0: NTB Interconnect (PLX87b0)
                Max HW Data Rate:  PCIe Gen 3 x 8
                HW Data Rate:      PCIe Gen 3 x 8
                SW Data Rate:      PCIe Gen 3 x 8
                Logical Link:      Up
                Port State:        Enabled
        slot 0: SAS Host Adapter 0a
                24 Disks:            27416.4GB
                1 shelf with IOM12E
        slot 0: SAS Host Adapter 0b
                24 Disks:            27416.4GB
                1 shelf with IOM12E
        slot 0: Fibre Channel Target Host Adapter 0c
        slot 0: Fibre Channel Target Host Adapter 0d
        slot 0: Fibre Channel Target Host Adapter 0e
        slot 0: Fibre Channel Target Host Adapter 0f
        slot 0: Intel BDW SATA Controller u0a (0x92311000)
                boot0   SV9MST6D120GLM41NP     FW1122 114473MB  512B/sect        (SPG1750079L)
        slot 0: Intel USB EHCI Adapter u0b (0x92312000)
        slot 1: Flash Cache NVMe
                Serial Number:      ####
                Part Number:        119-00328
                Hardware Revision:  A0
                Firmware Version:   NA01
                Model Name:         X3310A
                Capacity:           512 GB
                Vendor Name:        Samsung
                State:              ok

 

Highlighted

Re: Not all disks available after zero and initialisation

Looks like they're all there, but some (looks like 02's) are not owned.     (Also, you might want to zero your spares) 

 

How about these: 

storage disk show -container-type unassigned 

 

storage aggregate show-spare-disks

Highlighted

Re: Not all disks available after zero and initialisation

Do this which will completely wipe all data from the system

 

1. Boot both controllers to maintenance mode

2. Use the disk commands to work until all disks are unowned

3. Halt both controllers

4. set-defaults

5. saveenv

6. Boot both controllers to the boot menu with > boot_ontap menu

7. At the boot menu choose option 9 on both controllers and wait for prompts on both.

8. Choose 9a on one controller and wait for it to finish. 

9. Choose option 9a on the other controller and wait for it to finish. You should see 24 drives now listed as unpartitioned and unowned.

10 if you want to use advanced disk partitioning then choose option 9b on one controller or if you want to use whole disks (I would not with only 24) choose option 9bc. In either case wait for the controller to reboot and start zeroing. You will see a few ...... on the screen.

11. After the first controller starts the zeroing process choose the same option on the second controller.

12. When both are done you should have access to all disks.

 

Note that ONTAP 9.4 introduced fast zeroing. This means it only takes a few seconds to zero the disks in any case..

 

You will need to reaquire the license keys.

 

You should be able to now configure ONTAP

View solution in original post

Highlighted

Re: Not all disks available after zero and initialisation

Ok. After a wipe the controllers should each own 12 drives. 

 

You should see 12 P1 (data) 53GiB partitions for each Controller and 12 P2  (root) 1016GiB partitions for each Controller

 

Losing 2 for parity and one for spare leaves 9 data partitions for each Controller

 

Take away wafl overhead and you may be around 7-8TB PORT CONTROLLER.

 

 

Check out the KB!
Knowledge Base
All Community Forums