Solved: Re: Remove old restricted aggregate

SorinAndruseac · ‎2016-10-14

Hello,

It seems that I have an "old aggregate" that is restricted and I cant find a way to delete it. This is inconvenient because I can't use the disks. Moreover the aggregate is visible only from one node. Any ideea how I can delete this old aggregate?

netapp11::> system node run -node * sysconfig -r
2 entries were acted on.

Node: netapp11-01
Aggregate aggr0 (restricted, raid_dp) (block checksums)
Plex /aggr0/plex0 (online, normal, active, pool0)
RAID group /aggr0/plex0/rg0 (normal, block checksums)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity 0b.01.6 0b 1 6 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
parity 0a.00.6 0a 0 6 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
data 0b.01.4 0b 1 4 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400

Aggregate aggr0_netapp11_01_0 (online, raid_dp) (block checksums)
Plex /aggr0_netapp11_01_0/plex0 (online, normal, active, pool0)
RAID group /aggr0_netapp11_01_0/plex0/rg0 (normal, block checksums)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity 0a.00.0 0a 0 0 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
parity 0a.00.1 0a 0 1 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
data 0a.00.2 0a 0 2 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
Spare disks for block checksum
spare 0a.00.3 0a 0 3 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.4 0a 0 4 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.5 0a 0 5 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.7 0a 0 7 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.8 0a 0 8 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.9 0a 0 9 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.10 0a 0 10 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.11 0a 0 11 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.12 0a 0 12 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.13 0a 0 13 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.14 0a 0 14 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.15 0a 0 15 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.16 0a 0 16 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.17 0a 0 17 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.18 0a 0 18 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.19 0a 0 19 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.20 0a 0 20 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.21 0a 0 21 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.22 0a 0 22 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0a.00.23 0a 0 23 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.3 0b 1 3 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.5 0b 1 5 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.7 0b 1 7 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.8 0b 1 8 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.9 0b 1 9 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.10 0b 1 10 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.11 0b 1 11 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.12 0b 1 12 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.13 0b 1 13 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.14 0b 1 14 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.15 0b 1 15 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.16 0b 1 16 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.17 0b 1 17 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.18 0b 1 18 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.19 0b 1 19 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.20 0b 1 20 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.21 0b 1 21 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.22 0b 1 22 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400
spare 0b.01.23 0b 1 23 SA:B 0 FSAS 7200 5614621/11498743808 5625872/11521787400

Partner disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
partner 0b.01.2 0b 1 2 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.0 0b 1 0 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.1 0b 1 1 SA:B 0 FSAS 7200 0/0 5625872/11521787400

Node: netapp11-02
Aggregate aggr0_netapp11_02_0 (online, raid_dp) (block checksums)
Plex /aggr0_netapp11_02_0/plex0 (online, normal, active, pool0)
RAID group /aggr0_netapp11_02_0/plex0/rg0 (normal, block checksums)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity 0b.01.0 0b 1 0 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
parity 0b.01.1 0b 1 1 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400
data 0b.01.2 0b 1 2 SA:A 0 FSAS 7200 5614621/11498743808 5625872/11521787400

Pool1 spare disks (empty)

Pool0 spare disks (empty)

Partner disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
partner 0a.00.10 0a 0 10 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.11 0a 0 11 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.23 0a 0 23 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.21 0a 0 21 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.7 0a 0 7 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.18 0a 0 18 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.16 0a 0 16 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.20 0a 0 20 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.12 0a 0 12 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.8 0a 0 8 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.5 0a 0 5 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.9 0a 0 9 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.14 0a 0 14 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.17 0a 0 17 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.19 0a 0 19 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.22 0a 0 22 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.13 0a 0 13 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.15 0a 0 15 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.20 0b 1 20 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.22 0b 1 22 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.10 0b 1 10 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.4 0b 1 4 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.12 0b 1 12 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.8 0b 1 8 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.14 0b 1 14 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.16 0b 1 16 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.5 0b 1 5 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.18 0b 1 18 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.13 0b 1 13 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.6 0b 1 6 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.17 0b 1 17 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.15 0b 1 15 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.21 0b 1 21 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.11 0b 1 11 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.7 0b 1 7 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.9 0b 1 9 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.19 0b 1 19 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.3 0b 1 3 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0b.01.23 0b 1 23 SA:A 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.4 0a 0 4 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.6 0a 0 6 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.3 0a 0 3 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.1 0a 0 1 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.2 0a 0 2 SA:B 0 FSAS 7200 0/0 5625872/11521787400
partner 0a.00.0 0a 0 0 SA:B 0 FSAS 7200 0/0 5625872/11521787400

netapp11::> disk show
Usable Disk Container Container
Disk Size Shelf Bay Type Type Name Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
1.0.0 5.35TB 0 0 FSAS aggregate aggr0_netapp11_01_0
netapp11-01
1.0.1 5.35TB 0 1 FSAS aggregate aggr0_netapp11_01_0
netapp11-01
1.0.2 5.35TB 0 2 FSAS aggregate aggr0_netapp11_01_0
netapp11-01
1.0.3 5.35TB 0 3 FSAS spare Pool0 netapp11-01
1.0.4 5.35TB 0 4 FSAS spare Pool0 netapp11-01
1.0.5 5.35TB 0 5 FSAS spare Pool0 netapp11-01
1.0.6 5.35TB 0 6 FSAS aggregate aggr0 netapp11-01
1.0.7 5.35TB 0 7 FSAS spare Pool0 netapp11-01
1.0.8 5.35TB 0 8 FSAS spare Pool0 netapp11-01
1.0.9 5.35TB 0 9 FSAS spare Pool0 netapp11-01
1.0.10 5.35TB 0 10 FSAS spare Pool0 netapp11-01
1.0.11 5.35TB 0 11 FSAS spare Pool0 netapp11-01
1.0.12 5.35TB 0 12 FSAS spare Pool0 netapp11-01
1.0.13 5.35TB 0 13 FSAS spare Pool0 netapp11-01
1.0.14 5.35TB 0 14 FSAS spare Pool0 netapp11-01
1.0.15 5.35TB 0 15 FSAS spare Pool0 netapp11-01
1.0.16 5.35TB 0 16 FSAS spare Pool0 netapp11-01
1.0.17 5.35TB 0 17 FSAS spare Pool0 netapp11-01
1.0.18 5.35TB 0 18 FSAS spare Pool0 netapp11-01
1.0.19 5.35TB 0 19 FSAS spare Pool0 netapp11-01
1.0.20 5.35TB 0 20 FSAS spare Pool0 netapp11-01
1.0.21 5.35TB 0 21 FSAS spare Pool0 netapp11-01
1.0.22 5.35TB 0 22 FSAS spare Pool0 netapp11-01
1.0.23 5.35TB 0 23 FSAS spare Pool0 netapp11-01
1.1.0 5.35TB 1 0 FSAS aggregate aggr0_netapp11_02_0
netapp11-02
1.1.1 5.35TB 1 1 FSAS aggregate aggr0_netapp11_02_0
netapp11-02
1.1.2 5.35TB 1 2 FSAS aggregate aggr0_netapp11_02_0
netapp11-02
1.1.3 5.35TB 1 3 FSAS spare Pool0 netapp11-01
1.1.4 5.35TB 1 4 FSAS aggregate aggr0 netapp11-01
1.1.5 5.35TB 1 5 FSAS spare Pool0 netapp11-01
1.1.6 5.35TB 1 6 FSAS aggregate aggr0 netapp11-01
1.1.7 5.35TB 1 7 FSAS spare Pool0 netapp11-01
1.1.8 5.35TB 1 8 FSAS spare Pool0 netapp11-01
1.1.9 5.35TB 1 9 FSAS spare Pool0 netapp11-01
1.1.10 5.35TB 1 10 FSAS spare Pool0 netapp11-01
1.1.11 5.35TB 1 11 FSAS spare Pool0 netapp11-01
1.1.12 5.35TB 1 12 FSAS spare Pool0 netapp11-01
1.1.13 5.35TB 1 13 FSAS spare Pool0 netapp11-01
1.1.14 5.35TB 1 14 FSAS spare Pool0 netapp11-01
1.1.15 5.35TB 1 15 FSAS spare Pool0 netapp11-01
1.1.16 5.35TB 1 16 FSAS spare Pool0 netapp11-01
1.1.17 5.35TB 1 17 FSAS spare Pool0 netapp11-01
1.1.18 5.35TB 1 18 FSAS spare Pool0 netapp11-01
1.1.19 5.35TB 1 19 FSAS spare Pool0 netapp11-01
1.1.20 5.35TB 1 20 FSAS spare Pool0 netapp11-01
1.1.21 5.35TB 1 21 FSAS spare Pool0 netapp11-01
1.1.22 5.35TB 1 22 FSAS spare Pool0 netapp11-01
1.1.23 5.35TB 1 23 FSAS spare Pool0 netapp11-01
48 entries were displayed.

BIG THANKS in advance!!!

aborzenkov · ‎2016-10-14

So do it from node shell.

View solution in original post

aborzenkov · ‎2016-10-14

https://kb.netapp.com/support/s/article/how-to-remove-or-access-a-foreign-aggregate-an-aggregate-made-out-of-orphaned-disks-in-clustered-data-ontap?la...

SorinAndruseac · ‎2016-10-14

Tried that ... this is the output

netapp11::*> storage aggregate remove-stale-record -nodename netapp11-01 -aggregate aggr0

Error: command failed: Failed to delete aggregate "aggr0". Reason: aggregate 'aggr0' cannot be destroyed because it is restricted. You will need to take it offline before
destroying it.

netapp11::*> aggr offline aggr0

Error: command failed: Failed to check if aggr0 is a foreign(unknown) aggregate. Reason: entry doesn't exist.

Any other ideea would be great appreciated

aborzenkov · ‎2016-10-14

So do it from node shell.

SorinAndruseac · ‎2016-10-14

Well,

This is strage, I'm allmost sure that I've tried that before and also I tried it from maintenance mode.

However this time worked like a charm.

Thank you very much, it was really helpfull.

juliusdabre12 · ‎2017-02-21

I have same issue can you tell me how did you run those commands

tahsnks in advance

SorinAndruseac · ‎2017-02-21

Hello,

I guess I was doing:

system node run -node <nodename> -command <commands_from_above>

Good Luck!!!

Paul123456789 · ‎2019-01-04

elevated command needed: set -privilege advanced

Run the following command to identify the node with the aggregate flagged as foreign.
The relevant aggregate will have the label foreign.
::> system node run -node * -command sysconfig -r
Based on the above output, run the following clustershell commands, referencing the node that has the foreign aggregate.
::> set -privilege advanced
::*> storage aggregate remove-stale-record -nodename [node_name] -aggregate [foreign aggregate name]

chamfer · ‎2022-09-06

I would like to provide some clarification to this as the answer is not complete for those who come across this problem.

I noticed aggr0 on an array and I picked it up using NetApp Config Advisor when checking out the RAID Disk Visualization tool.

I found the NetApp KB that @Paul123456789 lists the commands from https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/How_to_remove_a_foreign_aggregate_(made_out_of_orphaned_disks)_in_ONTA... BUT....................... the aggregate IS NOT labelled as foreign when looking at the output of "...... sysconfig -r"

The aggregate is actually just listed as restricted and in my scenario I had the aggregate listed twice, once for each controller.

SO HOW DO I RESOLVE? Well here are the steps (I have 9.9.1P10 code)

1) set advanced

2) system node run -node <node name> -command aggr offline <aggregate name>

NOTE:

The system WILL NOT take the command system node run -node <node> -command storage aggregate offline -aggregate <aggregate name>
You MUST take the aggregate offline as the below command will fail of the aggregate is online and restricted

3) storage aggregate remove-stale-record -nodename <node name>

4) storage disk zerospares -owner <node name>

Steps 3 and 4 need to be performed for each node with the foreign aggregates.