Raid group size recommendation

[janjacobbakker]

Hi Folks,

I'm trying to design a storage solution.

A FAS3020 with 3 shelves full with 42x 300GB FC disks.

Default the Raid Group size is 16 (14 + 2 spare) de max raid group size is 28.

Does anyone have some best practice information? The NOW site hasn't much info on that.

Kind regards

[kusek]

Hello Jan-Jacob,

That's an excellent question, hopefully I can shed some light on it here.

Being that you're using a 3020 and 300GB FC disks, in order to meet the maximum capacity of a single aggregate, your ideal RG size would be 15

With these particular disks (300GB FC disks) you can have a maximum of 59 disks allocated to the aggregate, leaving you with room to grow into the final drives to fill out this aggregate. While a RG size of 16 would also work, you'll get the best performance and space utilization by establishing an RG of 15.

I hope this was able to address your question and help you complete your storage design.

Thanks,

Christopher

[janjacobbakker]

Christopher

Thanks for you quick response! I'm now building the 42disk aggregate with a 15 disk RG's

Regards,

Jan-Jacob

[mbeijnen1]

Dear Christopher,

Just curious to know how you came up with your answer... based on the 42 disks that are availabe to Jan-Jacob, wouldn't it be more advisable to choose RG size of 14 disks to have them even more equally divided?

Suppose 1 disk is used as hotspare, your offered solution (RG size of 15 disks) yields:

2x (13 spindels + 2 parity disks) +

1x (9 spindels + 2 parity disks) +

1 hotspare

A 14 disks RG size yields:

2x (12 spindels + 2 parity disks) +

1x (11 spindels + 2 parity disks) +

1 hotspare

Since performance is limited by slowest set, the equally devided solution would give a better performance.

Can you please provide the reasoning behind your answer, hopefully increasing my insight in the NetApp setup :-)

[kusek]

Great question Mark, I'll try to do your question justice!

I was looking at this from two aspects: Performance, and long-term capacity.

While the system does indeed have 42 disks today, tomorrow it may have a need for additional capacity.

So, by choosing a 15disk raid-group, I'm assuring myself not only maximum efficient RG design, I'm also committing to the maximum amount of space.

Also, with his third disk-set sitting at 9 disks (7), it is usually seen that your smallest RG need be atleast half the size of the RG size itsel, by having 7(9) disks there, we meet that criteria. (Although, being that Jan-Jacob did not require being capacity bound, that 3rd plex need not be added until necessary)

Coming back to other possible options, based upon todays available disk.

If I'm guaranteed a maximum amount of disks (42) in my system and never expect to grow it, then a 14 disk RG could indeed work.

But as storage is always growing, tomorrow comes around and I add another 2 shelves to my system, and when I try to grow it based upon a 14 disk RG I would end up with 56 disks allocated to the aggregate.

A 56 disk aggr isn't bad, it fulfills the criteria of performance by giving me appropriate spindles and also provides me a sizable amount of space - but my maximum capacity (bound by the RG) is stuck at 1TB less than the maximum availble to me in a 59 disk aggr (fulfilled by the 15 disk RG). With that in mind (I considered it) I opt'd to go for the best practice for best performance while also being able to fulfill maximum capacity in the long-run.

My sources for this were a calculator and capacity documents.

Hopefully this helped bring some insight into the operation and my decisions around it.

Thanks Mark,

Christopher

[mbeijnen1]

Dear Christopher,

Thank you very much for your reply, explaining the reasoning behind your recommendation.

Based on the reply I start running into more fundamental questions of sizing myself - probably a reference to the capacity documents mentioned in your post could help me with that.

When trying to calculate the maximum aggregate size I am still having a hard time deriving the 59 disks maximum that you mentioned. Once I get that I can understand your recommended RGsize of 15 disks since that would optimize capacity/performance for 59 disks in total, effectively having 4 raidgroups of (almost) equal size.

Some of the questions I find myself asking now:

- the 16 TB limit for an aggregate seems to include all disks (raw capacity), including parity disks?

- when calculating optimized sizes what GB's should be used, 1000/1024 based?

We are currently in the process of expanding a NetApp storage system and are looking into optimized setup, both now and in the future - yes, I tend to agree that we will in the end grow to the maximum size as well :-)

Would it be possible to provide some pointers in sizing/capacity documentation to get us started? Also, a brief explanation on how you derived the 59 disks maximum for a 300 GB disk size would be much appreciated.

Kind regards,

Mark.

[kusek]

As promised - Some details on disk max sizes and performance considerations!

Maximum data drives per 16-TB aggregate

With the aggregate size calculation changes present in Data ONTAP 7.3, you can include more data drives in an aggregate without exceeding the aggregate size limit.

The following table shows the maximum number of data drives that can be included in a 16-TB aggregate for Data ONTAP 7.3 and for previous releases.

Drive size	Drive type	Data ONTAP 7.2 and earlier	Data ONTAP 7.3
36 GB	FC	427	493
72 GB	FC	212	246
144 GB	FC/SAS	106	123
300 GB	FC/SAS	51	61
250 GB	SATA	68	79
320 GB	SATA	53	61
500 GB	SATA	33	39
750 GB	SATA	22	26
1 TB	SATA	15	19

Hopefully this helps with its reference to documents in the notes.

Also, if there are ever any questions - it never hurts to ask and validate your concerns against best practices and what is being actively used.

Take care,

Christopher

[chriskranz]

This table is pretty confusing as it doesn't show the parity disks at all.

The SATA numbers also don't add up, and I've never understood fully the SATA aggregate limits...

68x 250 = 17000

53x 320 = 16960

33x 500 = 16500

22x 750 = 16500

15x 1000 = 15000

Obviously all except the TB disks are well over the aggregate limit.

With FC disks are worked out on RAW including the parity disks, but SATA it doesn't seem so...

---

Back to the topic a little more. Are there any stats to show the performance improvements by having similar RAID group sizes across an entire aggregate? It'd be useful to backup the reasoning for changing the RAID group defaults.

[kevin_graham]

This table is pretty confusing as it doesn't show the parity disks at all. 

That's because with 7.3, parity disks are no longer relevant to the max aggr size. That table is the maximum number of data disks per aggr.

Back to the topic a little more. Are there any stats to show the performance improvements by having similar RAID group sizes across an entire aggregate? It'd be useful to backup the reasoning for changing the RAID group defaults.

I don't recall seeing anything published, but the key would just be ensure uniform performance across the entire aggr.

[kusek]

Kevin, you hit it right on the head - regarding 7.3

During the original post I had promised to try to provide as much citeable evidence as possible as to why certain RG's would be more ideal than another (outside of the default best practice) in order to achieve maximum space while providing for maximum performance. Every bit of public collateral I can find (such as that 7.3 based table) I wanted to ensure was available and in a consumable format.

As for the performance of similar raid group sizes also, best practice dictates that RG's should have at a minimum half as many disks as the RG size (So for a 14 disk RG you're looking at 7 disks minimum preferable) in order to avoid running into any kind of hot-disk scenarios and an immediate need to reallocate

Ofcourse as always, I prefer citable evidence as to the impacts, so until that point I'll continue setting preference for my min-half..Full philosophy - until I hear otherwise.

Thanks for your input and feedback on this Kevin!

Christopher