2554 large aggregates with ADP

[cshenberger]

I have  several new 2554 clusters,   each of them is 2 node with 24 internal 6TB drives and a Shelf of 6TB drives for a total of 48 Drives.   

I would like to create 2 large aggregates on each of these clusters with About 90TB each.   When I try to create a new aggregate I can't select 23 disks because there are 4 pools of 12 disks each. 

A node has 12 5.35 TB disks, and 12 5.3 TB disks   

B node has 12 5.35 TB disks, and 12 5.3 TB disks   

I assume that the difference in disk size is due to ADP being enabled.   but how do I get all the disks so that I can create just 2 large aggregates?   

[aborzenkov]

You need to disable ADP and reinstall (reinitialize root aggregate/volume). Do you have access to this article? How to disable Advanced Drive Partitioning (ADP)

[JamesIlderton · Accepted solution]

You'll have to create the aggregate using the smaller 5.3TB disks then add the 5.35TB ones after.   You'll lose the 0.05TB on each (as if you were replacing a failed 1TB disk with a 2TB one) but it should let you expand the aggregate to make use of all the disks without having to lose the full diska to a dedicated root aggregate.

[asulliva]

As @JamesIlderton said, you'll have to create the aggregate in two steps.  You don't want to remove ADP, it is a huge space savings (without it, each controller will need 3 drives for a root aggr...six 6TB drives is a lot of space compared to a few 10s of GB from each onboard disk for ADP).

Step 1) create the aggreate using the disks which have the ADP data partitions.  These will be the 5.3TB sized disks.

Step 2) add a new RAID group with the non-ADP disks.

You will not lose the 50GB or so of capacity on the second set of disks so long as they are added as a separate RAID group.  I don't have anything that mimics your setup, but I believe this will work:

# create a new aggr with the ADP disks (5.3TB * 1024 = 5430GB)
aggregate create -aggregate node1_data -diskcount 12 -disksize 5430 -node node1

# add a new RAID group with the other disks
aggregate add-disks -aggreagte node1_data -diskcount 11 -raidgroup new

Make sure you leave a spare disk!

Use the "-simulate" parameter to ensure the behavior you desire is happening.  If you want to be absolutely sure of the disks, then use the name...e.g. "-disklist 0.1.*"...to capture only the onboard drives for the first RG.  If you get it wrong, it's not the end of the world (there's no data on it yet!), you'll just have to wait for the drives to rezero.

Hope that helps.

Andrew