Aggregate extension

[AlexeyF]

Hello

On our disk array we have 2 aggregates of 6 RG(13+2) each. these are SATA disks.

We have just added a disk shelf with 24 SATA disks of the same size and would like to use them with maximum efficiency. 

How would you recommend to add these disks to the aggregates ?

The storage array is heavily utilized and we often have almost 100% disk utilisation on one of the aggregates.

So I 'm concerned about blocks reallocation. it can ruine all the performance for multiple days (weeks?).

If I add 12 disks to each aggregate and resize each RG from 15 to 17 disks, will it trigger the reallocation and parity recalculation?

Thanks for your advices

[JFM]

What type of workload does the system support? If mostly read, aggregate rebalances to my knowledge will take more time as this happens on write ops... unless you want to perform a manual operation which will be time and resources consuming... 

[TMACMD]

Parity will not recalc. When adding drives, they pre-zero'd so the parity does not change. The only time the reallocation runs is if an admin kicks it off (unless something changed)

Would need to know what size drives you are currently using and what platform. Would not want to hit max aggr size.

It may be beneficial to add disks to all the raid groups, but it may be more of an advantage to simple add a new raid group. 

[AlexeyF]

FAS8040 disks of 3,6TB 7200rpm

Aggregate : aggr1

Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 223.5TB 88%
Aggregate Metadata 0B 0%
Snapshot Reserve 0B 0%
Total Used 223.5TB 88%

Total Physical Used 211.3TB 83%

Aggregate : aggr2

Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 232.5TB 91%
Aggregate Metadata 86.74GB 0%
Snapshot Reserve 0B 0%
Total Used 232.6TB 91%

Total Physical Used 232.3TB 91%

If I add a raid group of 15 disks to the aggr2 I will have only 9 disks for aggr1. Will I be able to use them?

If I extend the RG size from 15 to 17 I will be able to use all 24 disks.

[AlexeyF]

Smth is wrong with my maths 🙂

Current aggregate size is 254.9TB

When I run a command to simulate 12x3,6TB disk addition I get an error:

storage aggregate add-disks -aggregate aggr2 -simulate true -diskcount 12 -disktype FSAS -raidgroup all

Error: command failed: Addition of disks would fail for aggregate "aggr2" on node "NODE002". Reason: Cannot add specified disks to the aggregate
because the new aggregate size would exceed the system limit. Limit: 324.00 TB.

[TMACMD]

The FAS8040 is limited to 324TiB.

With a RaidGroup size of 17 and 6 RaidGroups, you can have at MOST 101 disks.

aggr modify -aggregate aggr2 -maxraid size 17

aggr add-disks -diskcount 11 -g all -disktype FSAS -simulate true

That should work. Your original try was exceeding the limit by one disk.

However, you may be better served by just adding another raidgroup.

The 324TiB limit does not count parity drives.

So you may be able to instead try:

aggr add-disks -aggregate aggr2 -diskcount 12 -disktype FSAS -simulate true

That will result in a full open raid stripe that can use chunks from the other 6 raid groups. I personally would do this option.

[AlexeyF]

---

@TMACMD wrote:

That will result in a full open raid stripe that can use chunks from the other 6 raid groups. I personally would do this option.

---

Sorry, did not get it, could you please explain? what do you mean by "open raid stripe"? striping chunks across multiple raid groups? 

I've read that after adding empty disks to an aggregate WAFL will write blocks on them first of all until they get to the same usage level as others. I guess this may compromise write performance heavily if all the writes will be done on 12 disks only. If this is true do you bet that adding a new raid group will result in a smaller performance degradation that 2 disks in each RG?

Thanks.  

[TMACMD · Accepted solution]

If your aggregate is full and you are adding disks, the best practice is to add a whole RAID group.

If you add a handful of disks to each raid group (2) you are NOT helping yourself. ONTAP needs to work harder to find appropriate stipes to place data.

By creating a RAID group, it has a place to lay a large stripe and possibly use the raid groups if needed.

I would not add disks to the other raid groups in your case.

You are using 88-91% of capacity if I read your post right. 

Create the raid group and add the drives there. Do not shoot yourself in the foot

Others can chime in. I suspect they would agree with me,

[AlexeyF]

Thank you

I think I will choose this way, it looks more natural for me eventhough the smaller raid group will create a little bit of imbalance in performance.

I don't have experience with WAFL, rather with traditional disk arrays from HDS and EMC, so the concept of a raid group extension looked weired to me. I still have difficulty to imagine how the writes would be done to a Raid Group with 2 empty disks. 

As you say WAFL will search for the empty space on the remaining 13 disks to write the whole stripe on 15 disks, but they will end at some stage. So when they end, what will happen? 70-80% of disk space on new 2 disks will not be used ?

btw, what does these different stripe size mean in the statit output ?

is it reated to the RG size ?