ONTAP Discussions

Any best practices for adding disks to an aggreate

michaeldparker
7,030 Views

Hi All,

I just upgraded my FAS3240 to 8.1 to take advantage of the larger maximum aggregate size that 8.1 provides.

On each controller, I currently have 1 - SATA aggr composed of 34 – 2TB disks.  The current aggr is 2 raid groups of 17 disks (15 data/2 Parity).  On each controller, I have 25 -2TB spare drives which I would like to add to the existing aggregates. 

If I added the 25 drives to the existing aggr at the existing raid group size, I’d be adding 1 full raid group of 17 and a partial of 8.  This would then be a total of 4 raid groups (3 full and 1 partial).  What I would rather do is modify my existing raid group size to the max allowable of 20 so that when I add the 25 drives, 3 drives would go to each existing raid group (6 drives) and the remaining 19 would go into a new raid group.  The end result would be 3 raid groups(2 full and 1 partial of 19).  As I said, the latter solution is the one I’d like to go with and I think I’ll be just under the max aggr size.

Which solution would be better or does someone have a better solution?   I don’t want to potentially do something wrong and waste space I could use or worse off make performance worse.  Honestly, I don’t need the new drives added to the exiting aggr for space, but for IOPS so the last thing want is to make it worse.

Thanks in advance for the help.

Michael

1 ACCEPTED SOLUTION

scottgelb
7,025 Views

That sounds like a good plan and yields well for usable. Going all 20 and a single 19 makes sense.

Sent from my iPhone 4S

View solution in original post

17 REPLIES 17

scottgelb
6,968 Views

Usually we prefer to keep all RAID group sizes even and the same number of disks per raid group for optimal performance.  When you add disks there can be a performance impact depending on the number of spindles added until those drives fill to the watermark of the other existing disks.  You can run reallocate to alleviate this but if running deduplication that isn't an option.  Ideally I would go with what makes the RGs all even sized and equal in size if possible up to the 90TB aggr limit on the 3240 with 8.1 (volume max is still 50TB though).

michaeldparker
6,969 Views

Hi Scott,

Thanks for the reply.  We are running deduplication, so it would seem I cannot reallocate based on your response.  I am not sure I understand what you mean by even sized RGs.  Setting my RG size to 20 would make them the most equal in size I would think as I'd end up with 2 x 20 and 1 x 19.  To accomplish that though, I would have to add 3 drives each to two my my original 17 RGs.  Thoughts?

Thanks 

scottgelb
7,026 Views

That sounds like a good plan and yields well for usable. Going all 20 and a single 19 makes sense.

Sent from my iPhone 4S

michaeldparker
6,969 Views

Excellent. Thanks for the input. I think I’ll run with that then.

ERIC_TSYS
6,969 Views

Dont forget you need spare disks. I didnt see any mention of them. I assume you got it covered. you dont need too many of the either so maybe

you could take 1 and fill all your RG with 20?

Eric

michaeldparker
6,970 Views

Yes, thanks for the suggestion. I had planned on leaving 1 spare per controller, so I think that should do. I am still just afraid that by growing my RG to 20 and throwing 3 disks in I will encounter some hot spindles.

Thanks

scottgelb
6,970 Views

How much write percent is the aggregate? New data disks will be used for write regardless of raid group but number of disks can affect performance after adding.

michaeldparker
6,970 Views

Probably at least 90% write. These aggr’s are used for our snapvault location.

scottgelb
6,971 Views

I would run a perfstat or statit to see disk utilization during a busy sample time. Then you can extrapolate disk utilization

michaeldparker
6,129 Views

I’m pretty confident our util is very high at times. Beyond that, I’m confused in how that will help me to ensure I don’t get hotspots as I add drives.

Thanks

scottgelb
6,129 Views

You could take the average percent. If 60 percent for example. Divide by total drives. Then divide buy new drive count to estimate utilization proportion after you add. Until the drives even out.

Sent from my iPhone 4S

michaeldparker
6,129 Views

Ok.. gotcha. Thanks for the information. I’ll try that then.

pmeidell
6,126 Views

Folks, I suggest you review Jay White's Storage Subsystem Configuration Guide.

https://fieldportal.netapp.com/viewcontent.asp?qv=1&docid=24292

It makes no mention of any preference for RAID groups consisting of an even number of disks as opposed to odd number of disks. There is no bias either way, both in terms of resilience and performance.

The paper does state that it is preferred to maintain a balanced number of disks in all RAID groups of an aggregate. This is consistent with best practices, and Scott is correct on this point.

After you add drives to an existing aggregate it is strongly recommended that you reallocate the data, regardless of whether or not dedupe is deployed. The recommended approach is to run the command

reallocate start -f -p /vol/volname

on ALL volumes in the aggregate. Do not leave any volume out.

scottgelb
6,126 Views

Jay's papers are great.. he does have some internal versions as well that maybe he can post on.  From some support cases over the years and some battle scars after perfstat analysis it has been recommended in several cases to keep all raid groups the same size for best performance.

Are the BURTs updated for reallocate when running deduplication? The last I saw was to not run reallocate on any volumes where dedup is running.  The 8.1 release notes made a note it was ok but I heard that wasn't the case.

pmeidell
6,126 Views

I haven't reviewed all BURTs on this matter, but I can say the following:

It is best to reallocate after growing an aggregate. If you are using dedupe, your snapshot usage will balloon unless you include the -p switch in the reallocate command. The -p switch is a relatively recent addition. I don't remember which version of Data ONTAP was the first to include it. The recommendations to avoid reallocate for deduped volumes probably have their genesis prior to the availability of -p. If memory serves, the recommendations also suggested deleting all snapshots if possible before running reallocate, thus avoiding the exploding snapshot reserve usage. But using -p does come with its caveats. Read the documentation for the details. It has something to do with increased latency when reading data from snapshots. Look in the System Administration Guide for the release you're using for the details.

scottgelb
4,787 Views

-p was 7.3.3 a while ago from what I remember.  There were notes to not run any reallocate at all on deduplicated volumes (-p or any way) but can't see the internal burts on that.  I did see in the 8.1 release notes https://library.netapp.com/ecm/ecm_get_file/ECMP1120690 that support for reallocate (or the bug fixes?) is now supported "Starting in Data ONTAP 8.1, you can now enable reallocation scans, read reallocation, and extents on deduplicated volumes."

It says to only use -p for dedup otherwise there is no reallocate of dedup blocks which is good clarification.  But I would be hesitant to run reallocate until 8.1 with dedup volumes unless support can confirm it is ok (or if the internal burt info can be shared).

To rearrange the blocks that are shared between files by deduplication, use the -p

option when you run the reallocate start command. Reallocation scans

skip deduplicated data if you do not specify the -p option.

aborzenkov
4,788 Views

One more note - to use -p aggregate must have been created under 7.2 at least. For filers with long history that could be an issue (I did see it).

Public