Adding Disks to Aggrs with Strange (I think) Results

[murdock01]

I am working on adding a bunch of new disks to my current cluster. I have gotten to the point of all my disks are in the spare pool and ready to be added to my aggrs.
The general plan is to change the max raid group size from 8 to 14. The aggrs in question have two raid groups with 8 disks each.

aggr1_03              "/aggr1_03/plex0/rg0 (block)","/aggr1_03/plex0/rg1 (block)"
aggr1_04              "/aggr1_04/plex0/rg0 (block)","/aggr1_04/plex0/rg1 (block)"

and example this is aggr1_03 (edited so it would not be so long)

disk raid-group
----- ----------
2.0.2 rg0, 2.0.3 rg0, 2.0.4 rg0, 2.0.5 rg0, 2.1.1 rg0, 2.1.2 rg0, 2.1.3 rg0, 2.1.4 rg0
2.0.6 rg1, 2.0.7 rg1, 2.0.8 rg1, 2.0.9 rg1, 2.1.5 rg1, 2.1.6 rg1, 2.1.7 rg1, 2.1.8 rg1

So I go to simulate adding a bunch of disks to one of the aggregates and get this 

First Plex

  RAID Group rg1, 14 disks (block checksum, raid_dp)
                                                      Usable Physical
    Position   Disk                      Type           Size     Size
    ---------- ------------------------- ---------- -------- --------
    data       2.3.6                     FSAS         3.63TB   3.64TB
    data       2.3.7                     FSAS         3.63TB   3.64TB
    data       2.3.8                     FSAS         3.63TB   3.64TB
    data       2.3.9                     FSAS         3.63TB   3.64TB
    data       2.3.10                    FSAS         3.63TB   3.64TB
    data       2.3.11                    FSAS         3.63TB   3.64TB

  RAID Group rg2, 14 disks (block checksum, raid_dp)
                                                      Usable Physical
    Position   Disk                      Type           Size     Size
    ---------- ------------------------- ---------- -------- --------
    dparity    2.4.0                     FSAS              -        -
    parity     2.4.1                     FSAS              -        -
    data       2.4.2                     FSAS         3.63TB   3.64TB
    data       2.4.3                     FSAS         3.63TB   3.64TB
    data       2.4.4                     FSAS         3.63TB   3.64TB
    data       2.4.5                     FSAS         3.63TB   3.64TB
    data       2.4.6                     FSAS         3.63TB   3.64TB
    data       2.4.7                     FSAS         3.63TB   3.64TB
    data       2.6.0                     FSAS         3.63TB   3.64TB
    data       2.6.1                     FSAS         3.63TB   3.64TB
    data       2.6.2                     FSAS         3.63TB   3.64TB
    data       2.6.3                     FSAS         3.63TB   3.64TB
    data       2.6.4                     FSAS         3.63TB   3.64TB
    data       2.6.5                     FSAS         3.63TB   3.64TB

  RAID Group rg3, 6 disks (block checksum, raid_dp)
                                                      Usable Physical
    Position   Disk                      Type           Size     Size
    ---------- ------------------------- ---------- -------- --------
    dparity    2.6.6                     FSAS              -        -
    parity     2.6.7                     FSAS              -        -
    data       2.6.8                     FSAS         3.63TB   3.64TB
    data       2.6.9                     FSAS         3.63TB   3.64TB
    data       2.6.10                    FSAS         3.63TB   3.64TB
    data       2.6.11                    FSAS         3.63TB   3.64TB

As you can see it adds 6 disks to rg1, created another group called rg2 and filled with 14 disks, then created another group called rg3 and added 6. The result would be rg0,rg1,rg2,rg3 with 8,14,14,6 disks respectively.
It was my understanding  that ontap would add 6 disks to rg0, add 6 disks to rg1, and then create another group called rg2 and use up my last 14 spares; so I would end up with rg0,rg1,rg2 each have 14 disks.
Why did I get the results I did on the simulate?

[Ontapforrum]

Is the concerned aggr maxraidsize size modified already ?

::>aggr show -aggr <>  -fields maxraidsize

Also, have you tried adding disk by specifiying specific rgX and disk count. For exL

::> agg add-diks -aggr <> -raidgroup rgX -diskcount 6

[murdock01]

Yes, the max raid size for these 2 aggrs is currently 14. That was step 1, before I put the disks into the spare pool assigning owners.
Not yet, and I do know I can manually put disks where I think they should go. But I ran the simulate command basically to confirm what I was doing; with my expectations that ontap would give me the 3 groups of 14 disks each. That is why when I saw what ontap would do, I was puzzled. Then I started to wonder if ontap was seeing something I am not. 

[junwang]

Hi, murdock01,

My understanding is that your requirement is to implement an aggr with 3 raid groups and 14 disks each, right? If so, follow the steps below.Take aggr1_03 as an example.

Firstly, you need to modify the maxraid size in your aggr to 14,
::> storage aggregate modify -aggregate aggr1_03 -maxraidsize 14

Afterwards, it is necessary to manually specify the addition of 6 disks to the rg0 of the aggr1_03 aggr, so that there are 14 disks in this raidgroup.

::> storage aggregate add-disks -aggregate  aggr1_03 -diskcount 6 -raidgroup rg0  -simulate true

Then, add 6 disks to the rg1 of the aggr1_03 aggr, so that there are 14 disks in this raidgroup.

::> storage aggregate add-disks -aggregate  aggr1_03 -diskcount 6 -raidgroup rg1  -simulate true

Create a new RAID group and add 14 disks, so that there will be 3 RAID groups with 14 disks in each group.

::> storage aggregate add-disks -aggregate aggr1_03 -diskcount 14 -raidgroup new -simulate true