Adding disks to existing aggr - reallocate questions

[strattonfinance]

Hi all,

Hoping the NetApp community can provide some assistance to this relatively novice NetApp administrator.

Firstly, to describe the environment / scenario:

- FAS2050 running DOT 7.3.5

- Head #1 (which this question relates to) has all 20 "internal" disks assigned to it

- Currently, 16 of these 20 disks are in a single aggregate, all in a single RAID-DP raid group. This aggregate is c. 90% full at present.

- This aggregate contains a bunch of flexvols used exclusively to store VMDK files for our VMware environment (provided over NFS), and has both snapshots and deduplication enabled on all volumes.

- Out of the remaining 4 disks, 2 are hotspares and 2 are not being used at all (freed up after some recent changes involving head #2)

- We want to add the 2 "unused" disks to the existing raid group / aggregate, increasing the raid group size to 18 in the process.

I understand that we will need to run a "reallocate" after adding the 2 extra disks to the existing raid group / aggregate or else we will end up with "hot spots" on these new disks.

From some research it appears that we need ensure DOT only does a physical redistribution of the blocks across all disk in the aggregate and does not try to otherwise rearrange them, otherwise we risk "un-deduping" our deduplicated data and/or greatly increasing the space consumed by our snapshots. Both of these would result in big problems for us as we'd rapidly run out of space in the aggregate - we need to perform the physical reallocation across the new disks without increasing space usage.

Therefore, what I need help with is: Again from research I think what we need to do is run "reallocate start -f -p <volume_name>" against each volume in the aggregate, but I'm not sure. Can anyone confirm if this is correct, and provide the correct syntax if it is not?

Also, a bonus question: I'm assuming the reallocate will put a significant load on our filer and hence should be run outside of peak hours. Can anyone give some (broad) guidelines as to how long we can expect a FAS2050 to take to reallocate c. 1.6TB of data across the newly-expanded 18-disk RAID-DP group (144GB 15K drives if it matters), assuming no other load on the filter?

Thanks!

[radek_kubka]

Hi,

 I think what we need to do is run "reallocate start -f -p <volume_name>" against each volume in the aggregate, but I'm not sure. 

Yes, this is correct. More details could be found here: http://now.netapp.com/NOW/knowledge/docs/ontap/rel733/html/ontap/cmdref/man1/na_reallocate.1.htm

Can anyone give some (broad) guidelines as to how long we can expect a FAS2050 to take to reallocate c. 1.6TB of data across the newly-expanded 18-disk RAID-DP group (144GB 15K drives if it matters), assuming no other load on the filter?

I have no idea to be perfectly honest. FAS2050 is a rather limited box in the terms of its CPU & memory. On the other hand reallocate will not shift the entire 1.6TB from one place to another, only a portion of it required for proper re-balancing.

As a plan B, you can quiesce reallocate job, should it still be unfinished when the maintenance window ends.

Regards,

Radek

[strattonfinance]

Hi Radek,

> > I think what we need to do is run "reallocate start -f -p <volume_name>" against each volume in the aggregate, but I'm not sure.

> Yes, this is correct. More details could be found here: http://now.netapp.com/NOW/knowledge/docs/ontap/rel733/html/ontap/cmdref/man1/na_reallocate.1.htm

Thanks. I did read that document but it didn't make things completely clear for our specific situation.

Are you confident that using the above command won't "un-deduplicate" our de-duped data / cause snapshot space to balloon, as per my concerns?

> > Can anyone give some (broad) guidelines as to how long we can expect a FAS2050 to take to reallocate c. 1.6TB of data across the newly-expanded 18-disk RAID-DP group (144GB 15K drives if it matters), assuming no other load on the filter?

>I have no idea to be perfectly honest. FAS2050 is a rather limited box in the terms of its CPU & memory. On the other hand reallocate will not shift the entire 1.6TB from one place to another, only a portion of it required for proper re-balancing.

> As a plan B, you can quiesce reallocate job, should it still be unfinished when the maintenance window ends.

OK - thanks for the info anyway, and the idea re quiescing the reallocate job.

[radek_kubka]

Hi,

Physical reallocation will not cause snapshots to grow, that's fine.

I missed the bit about de-duplication though, and as per Scott's post, it makes running reallocation not recommended.

Regards,

Radek

[strattonfinance]

> Physical reallocation will not cause snapshots to grow, that's fine.

OK, thanks for confirming that. So we just need to find out about de-dupe.

[scottgelb]

Are you running Asis? With dedup reallocate can be a challenge...not recommended. On 8.1 the release notes list fixes with this but I heard there are still burts about it.

If no asis reallocate make sense if disk I/o will be an issue after a small disk addition.

[strattonfinance]

Hi Scott,

> Are you running Asis? With dedup reallocate can be a challenge...not recommended. On 8.1 the release notes list fixes with this but I heard there are still burts about it.

Yes, we're running dedupe: "This aggregate contains a bunch of flexvols used exclusively to store VMDK files for our VMware environment (provided over NFS), and has both snapshots and deduplication enabled on all volumes.".

You say not recommended - as in there is no "safe" way to do it? Or we just have to be careful to run the correct command (as per my original question)?

> If no asis reallocate make sense if disk I/o will be an issue after a small disk addition. 

I thought that after adding two disks to a very full aggregate a reallocate of some sort was pretty much mandatory, dedupe or not?

Thanks.

[scottgelb]

This is a tough one. Reallocate likely would help with I/o but we shouldn't run it with Asis. I would run this by support and post back. With 8.1 the release notes say ok to do buy that is rc code and I heard the notes may not be correct since there are reallocate/asis interaction Issues

[David_moore]

I don't believe the FAS2050 will run 8.1

[scottgelb]

Correct... 7.3.x is the highest.. it can't run 8.0 or higher code. 

[strattonfinance]

> This is a tough one.  Reallocate likely would help with I/o but we shouldn't run it with Asis. I would run this by support and post back.

Thanks for the additional response. I've just opened a case with support and will post back what they say.

Also, do you happen to remember where it says you shouldn't run reallocate on a de-duplicated volume? I know I've read it to, but I can't find the reference now.

[scottgelb]

Bug reports. And the tr also if I remember correctly.