topic Re: Raid group size recommendation in ONTAP Hardware

Raid group size recommendation

janjacobbakker — Thu, 05 Jun 2025 07:34:18 GMT

Hi Folks,

I'm trying to design a storage solution.

A FAS3020 with 3 shelves full with 42x 300GB FC disks.

Default the Raid Group size is 16 (14 + 2 spare) de max raid group size is 28.

Does anyone have some best practice information? The NOW site hasn't much info on that.

Kind regards

Re: Raid group size recommendation

kusek — Fri, 11 Jul 2008 13:30:10 GMT

Hello Jan-Jacob,

That's an excellent question, hopefully I can shed some light on it here.

Being that you're using a 3020 and 300GB FC disks, in order to meet the maximum capacity of a single aggregate, your ideal RG size would be 15

With these particular disks (300GB FC disks) you can have a maximum of 59 disks allocated to the aggregate, leaving you with room to grow into the final drives to fill out this aggregate. While a RG size of 16 would also work, you'll get the best performance and space utilization by establishing an RG of 15.

I hope this was able to address your question and help you complete your storage design.

Thanks,

Christopher

Re: Raid group size recommendation

janjacobbakker — Sat, 12 Jul 2008 07:11:22 GMT

Christopher

Thanks for you quick response! I'm now building the 42disk aggregate with a 15 disk RG's

Regards,

Jan-Jacob

Re: Raid group size recommendation

mbeijnen1 — Fri, 18 Jul 2008 15:03:53 GMT

Dear Christopher,

Just curious to know how you came up with your answer... based on the 42 disks that are availabe to Jan-Jacob, wouldn't it be more advisable to choose RG size of 14 disks to have them even more equally divided?

Suppose 1 disk is used as hotspare, your offered solution (RG size of 15 disks) yields:

2x (13 spindels + 2 parity disks) +

1x (9 spindels + 2 parity disks) +

1 hotspare

A 14 disks RG size yields:

2x (12 spindels + 2 parity disks) +

1x (11 spindels + 2 parity disks) +

1 hotspare

Since performance is limited by slowest set, the equally devided solution would give a better performance.

Can you please provide the reasoning behind your answer, hopefully increasing my insight in the NetApp setup 🙂

Re: Raid group size recommendation

kusek — Fri, 18 Jul 2008 15:40:38 GMT

Great question Mark, I'll try to do your question justice!

I was looking at this from two aspects: Performance, and long-term capacity.

While the system does indeed have 42 disks today, tomorrow it may have a need for additional capacity.

So, by choosing a 15disk raid-group, I'm assuring myself not only maximum efficient RG design, I'm also committing to the maximum amount of space.

Also, with his third disk-set sitting at 9 disks (7), it is usually seen that your smallest RG need be atleast half the size of the RG size itsel, by having 7(9) disks there, we meet that criteria. (Although, being that Jan-Jacob did not require being capacity bound, that 3rd plex need not be added until necessary)

Coming back to other possible options, based upon todays available disk.

If I'm guaranteed a maximum amount of disks (42) in my system and never expect to grow it, then a 14 disk RG could indeed work.

But as storage is always growing, tomorrow comes around and I add another 2 shelves to my system, and when I try to grow it based upon a 14 disk RG I would end up with 56 disks allocated to the aggregate.

A 56 disk aggr isn't bad, it fulfills the criteria of performance by giving me appropriate spindles and also provides me a sizable amount of space - but my maximum capacity (bound by the RG) is stuck at 1TB less than the maximum availble to me in a 59 disk aggr (fulfilled by the 15 disk RG). With that in mind (I considered it) I opt'd to go for the best practice for best performance while also being able to fulfill maximum capacity in the long-run.

My sources for this were a calculator and capacity documents.

Hopefully this helped bring some insight into the operation and my decisions around it.

Thanks Mark,

Christopher

Re: Raid group size recommendation

mbeijnen1 — Mon, 21 Jul 2008 09:28:02 GMT

Dear Christopher,

Thank you very much for your reply, explaining the reasoning behind your recommendation.

Based on the reply I start running into more fundamental questions of sizing myself - probably a reference to the capacity documents mentioned in your post could help me with that.

When trying to calculate the maximum aggregate size I am still having a hard time deriving the 59 disks maximum that you mentioned. Once I get that I can understand your recommended RGsize of 15 disks since that would optimize capacity/performance for 59 disks in total, effectively having 4 raidgroups of (almost) equal size.

Some of the questions I find myself asking now:

- the 16 TB limit for an aggregate seems to include all disks (raw capacity), including parity disks?

- when calculating optimized sizes what GB's should be used, 1000/1024 based?

We are currently in the process of expanding a NetApp storage system and are looking into optimized setup, both now and in the future - yes, I tend to agree that we will in the end grow to the maximum size as well 🙂

Would it be possible to provide some pointers in sizing/capacity documentation to get us started? Also, a brief explanation on how you derived the 59 disks maximum for a 300 GB disk size would be much appreciated.

Kind regards,

Mark.

Re: Raid group size recommendation

kusek — Mon, 25 Aug 2008 17:00:34 GMT

As promised - Some details on disk max sizes and performance considerations!

Maximum data drives per 16-TB aggregate

With the aggregate size calculation changes present in Data ONTAP 7.3, you can include more data drives in an aggregate without exceeding the aggregate size limit.

The following table shows the maximum number of data drives that can be included in a 16-TB aggregate for Data ONTAP 7.3 and for previous releases.

Drive size	Drive type	Data ONTAP 7.2 and earlier	Data ONTAP 7.3
36 GB	FC	427	493
72 GB	FC	212	246
144 GB	FC/SAS	106	123
300 GB	FC/SAS	51	61
250 GB	SATA	68	79
320 GB	SATA	53	61
500 GB	SATA	33	39
750 GB	SATA	22	26
1 TB	SATA	15	19

Hopefully this helps with its reference to documents in the notes.

Also, if there are ever any questions - it never hurts to ask and validate your concerns against best practices and what is being actively used.

Take care,

Christopher

Re: Raid group size recommendation

chriskranz — Fri, 29 Aug 2008 16:10:53 GMT

This table is pretty confusing as it doesn't show the parity disks at all.

The SATA numbers also don't add up, and I've never understood fully the SATA aggregate limits...

68x 250 = 17000

53x 320 = 16960

33x 500 = 16500

22x 750 = 16500

15x 1000 = 15000

Obviously all except the TB disks are well over the aggregate limit.

With FC disks are worked out on RAW including the parity disks, but SATA it doesn't seem so...

Back to the topic a little more. Are there any stats to show the performance improvements by having similar RAID group sizes across an entire aggregate? It'd be useful to backup the reasoning for changing the RAID group defaults.

Re: Raid group size recommendation

kevin_graham — Fri, 29 Aug 2008 20:18:08 GMT

This table is pretty confusing as it doesn't show the parity disks at all.

That's because with 7.3, parity disks are no longer relevant to the max aggr size. That table is the maximum number of data disks per aggr.

I don't recall seeing anything published, but the key would just be ensure uniform performance across the entire aggr.

Re: Raid group size recommendation

kusek — Sat, 30 Aug 2008 02:10:01 GMT

Kevin, you hit it right on the head - regarding 7.3

During the original post I had promised to try to provide as much citeable evidence as possible as to why certain RG's would be more ideal than another (outside of the default best practice) in order to achieve maximum space while providing for maximum performance. Every bit of public collateral I can find (such as that 7.3 based table) I wanted to ensure was available and in a consumable format.

As for the performance of similar raid group sizes also, best practice dictates that RG's should have at a minimum half as many disks as the RG size (So for a 14 disk RG you're looking at 7 disks minimum preferable) in order to avoid running into any kind of hot-disk scenarios and an immediate need to reallocate

Ofcourse as always, I prefer citable evidence as to the impacts, so until that point I'll continue setting preference for my min-half..Full philosophy - until I hear otherwise. 🙂

Thanks for your input and feedback on this Kevin!

Christopher

Re: Raid group size recommendation

chriskranz — Sat, 30 Aug 2008 06:37:50 GMT

kevin.graham wrote:
That's because with 7.3, parity disks are no longer relevant to the max aggr size. That table is the maximum number of data disks per aggr.

That's fair enough, but because the thread is about RAID group sizes, the table is still a little confusing as it lays it out for standard RAID group sizes. As you say, the parity disks get dropped for 7.3 so it is much more relevant then (however I still don't understand the SATA calculations!).

It makes sense though to have a new table that defines the optimal RAID group sizes for each disk type as obviously this is variable. Something along the lines of the following (this was just a quick calculation from me, so could be improved)...

Drive size	Drive type	Data ONTAP 7.2 and earlier	Data ONTAP 7.3
		Spindle Count	Number of Parity Disks	Optimal RAID Group Size	Resultant RAID Groups	Spindle Count	Number of Parity Disks	Optimal RAID Group Size	Resultant RAID Groups
36 GB	FC	427	66	15	32.87	493	66	17	32.88
72 GB	FC	212	28	16	15.00	246	42	14	20.57
144 GB	FC/SAS	106	16	16	7.63	123	18	16	8.81
300 GB	FC/SAS	51	8	15	3.93	61	10	15	4.73
250 GB	SATA	68	10	16	4.88	79	12	16	5.69
320 GB	SATA	53	10	13	4.85	61	10	15	4.73
500 GB	SATA	33	6	13	3.00	39	6	15	3.00
750 GB	SATA	22	4	13	2.00	26	4	15	2.00
1 TB	SATA	15	2	17	1.00	19	4	12	1.92

Re: Raid group size recommendation

chriskranz — Sat, 30 Aug 2008 06:39:23 GMT

Sorry, the table gets a bit messed up as a post, I'll attach the excel spreadsheet instead...

Re: Raid group size recommendation

kusek — Sat, 30 Aug 2008 16:53:57 GMT

I like the consolidated table Chris,

The only modifications I would look at are changing the 320gb SATA optimal RG in a Pre-7.3 to 15 disks and the Optimal RG in Pre-73 for 144gb disks to 15 as well

Those would allow for the maximum capacity, and along the same token spread-iops as well.

Curious how you got some of the numbers for your tables (Like the spindle count column) because it doesn't entirely match up with what I'm working with - Which slightly skews the results. Also you need to account for different maximum capacity sizes of 15k disks over 10k disks. (which at the smaller disk sizes makes it less of an apples to apples comparison) Otherwise the table itself seems pretty nice!

Thanks for your attention on this Chris!

Christopher

Re: Raid group size recommendation

chriskranz — Sat, 30 Aug 2008 20:04:41 GMT

Yeah sorry the table isn't perfect, I just took the one posted before and quickly did some maths. I reckon you could build some fancy excel table to do it a lot better. And you're right, I definitely didn't take into account any limitations, just took the table from before.

It would be nice to see something official like this from NetApp, just a quick cheat sheet.

Re: Raid group size recommendation

kusek — Sat, 30 Aug 2008 20:42:48 GMT

Chris,

I personally am trying to make sure the collateral like that link and table I referenced before is able to be found easier.

I know hte data is out there, it's just a matter of pulling away the tangles and the brush to find it.

I'll keep you and everyone informed!

Christopher

Re: Raid group size recommendation

cebulrdcis — Fri, 26 Sep 2008 03:23:47 GMT

Hi Guyz,

need your expert recommendations on what raid group size do you recommend on a 1TB SATA Shelve? I have 3 x 1TB NetApp shelf to be used as my snapvault target ( secondary storage).

Also, I have some 1TB shelves on my primary storage for CIFS Shares.. and I dont know if I used the best practice for raid group size.I used raidsize=14 to maximize the capacity limit..

hoping im correct..

Re: Raid group size recommendation

scottgelb — Fri, 26 Sep 2008 06:07:43 GMT

Depending on if the ONTAP version is 7.2.x or 7.3.x, here is what I have been doing for 1TB drives...based on the maximum drives supported per aggregate for each ontap version.

7.2.x 19 total, disks supported (15 data max per aggr) [ (RG0: 8D+2P) + (RG1: 7D+2P) ]

7.3.x 23 total disks supported (19 data max per aggr) [ (RG0: 10D+2P) + (RG1: 9D+2P) ]'

Since we need two raid groups to max the aggregate size (4 parity disks needed to get to max size), it's best to keep the raid groups as even as possible.. for example, with 7.3 using the default raid group size, you could go with rg0: 14D+2P, then rg1: 5D+2P - but that wouldn't be as good as 10D+2P and 9D+2P over two raid groups...

The trick with ONTAP 7.2 to do this is... create a 10 drive aggregate.. change the raidsize option to 10, then add 9 more drives to max it out..changing to 10 will force a new raid group. For example:

aggr create aggrname 10
aggr options aggrname raidsize 10
aggr add aggrname 9

Similarly, you can do the same with ONTAP 7.3 by creating with 12 drives, then change the raidsize option to 12, then add 11 more drives to max it out.. Additionally, 7.3 added a new option to create a new raid group with "-g new"

aggr create aggrname 12
aggr options aggrname raidsize 12
aggr add aggrname 11

aggr create aggrname 12
aggr add -g new aggrname 11 #-g forces a new raid group with the next 11 drives... prior to 7.3 -g is supported but only for existing raid groups..explicity use "new" as the raid group name and it will create a new raid group

Re: Raid group size recommendation

cebulrdcis — Fri, 26 Sep 2008 06:19:15 GMT

thanks scott..

I try this one in our new 3 x 1TB shelf for our Secondary storage (FAS3020)

once again thank you..

Re: Raid group size recommendation

rkaramchedu1 — Wed, 15 Oct 2008 11:21:03 GMT

The more I read this thread, the more confusing it becomes.

Let me stick to PRE-7.3 for a second.

The table I saw above says optimal RAID group size for a 1TB SATA Drive is 17. First off, for SATA, the max RAID group size allowed is 16. So 17 is not even a workable option.
Aggregate Size. From my readings, work and understanding, the total size of an aggregate was computed by adding together the raw size of all disks in the aggregate, including parity disks, and regardless of the amount of disk space available to be used in each data disk. What I am now confused about now is the use of the term "raw size". I believe it is the "physical capacity" might be the correct term. More specifically, it is the output from the "sysconfig -r" command from under the "Phys (MB/blks)" column.

For e.g. for a 300GB FC Disk Drive, the Physical Size is 274845 MB = 268.40 GB. So for an aggregate limit of 16TB, the total # of drives = 16 * 1024 / 268.40 = 61 drives (parity and data together).

Am I correct ?

Re: Raid group size recommendation

amiller_1 — Thu, 11 Dec 2008 22:13:27 GMT

Hmm....the table makes complete sense but I must admit I'm a bit confused. I'm looking at FilerView on a 3050 right now where I setup a 39 disk aggergate of 500 GB drives on 7.2.2 with total usable capacity of 12 TB.

If I understand the discussion correctly, putting (39) 500 GB disks is what I can now do under 7.3....but I did it under 7.2.2.

I'd originally thought that 7.3. adjusted aggregate maximum sizing so the maximum was calculated against usable space rather raw space....meaning I could put more like ~45 500 GB disks into a single aggregate (say 3 (14) 500 GB RAID groups in a single aggregate).

Thoughts? Thanks.