Thanks for the help and insight.
Just to be sure, these commands are non-destructive correct?
I want to make sure I understand each step of this process, so I hope you don't mind I've added questions/comments on the commands you've given.
"->storage disk option modify -node * -autoassign off"
I understand this as simply turning off disk auto assign. As I understand it, the disks won't automatically get assigned to a particular node.
-> storage disk removeowner -disk 1.1.x
This command is simply removing the current disk owner of the drives in the new shelf that was installed.
-> storage disk assign -owner node2
This will then assign the disks that were removed from the above command to the second node in the cluster. Will not specifying the specific disks just do all available disks to that node, including available disks not on shelf01? - You can disregard this question - I now understand the specific disks or disk list needs to be specified.
2. move all volumes off aggr2 if they're any left.
Simple enough, but to be sure this is moving all the volumes on the original aggregates on NODE2 to the new one I just created on NODE2, correct? There are two SVMs on this device as well, will that have any impact? Finally there are 4 total aggregates, two of which have the ability to move the volume, but the other two I believe are root aggregates and indicate there are no volumes to move when attempting to move anything. I've included a screen shot for further explanation.
3. delete aggr2 (the partitions will now show as spares)
Self-explanatory, deleting the aggregates listed in the screen shot above, but does this include the aggr0 and aggr0_Hart2520CL_02_0?
4. You can re-assign the data partitions to controller 1 by the following:
-> set advanced
Entering advanced mode in CLI
-> disk removeowner -disk x.x.x -data true
Am I correct in assuming I need to specify x.x.x? Where x.x.x would be the disks assigned to NODE2? So as I understand it, this would only be the (6) 2.42TB disks currently assigned to NODE2? For example -disk 1.0.0, 1.0.2, 1.0.4, 1.0.6, 1.0.8, 1.0.10, what does the -data true switch do?
-> disk assign -disk x.x.x -data true -owner NODE1
This is then assigning the disk to NODE1, same as above I assume I need to specify disk x.x.x, also curious what the -data true switch does here.
-> disk show -fields data-owner, root-owner (to verify that everything is moved over correctly)
Self-explanatory as this just verifies the work performed above.
5. zero all the spare
-> storage disk zerospares
This will then zero the spares that are now available after the above commands. Allowing me to assign the disks pulled from NODE2 to the aggregate on NODE1?
Re-enable disk autoassign
->storage disk option modify -node * -autoassign on
Self-explanatory - this is just re-enabling autoassign for all disks across all nodes.
Finally, as I understand it, once this is complete I will then have NODE1 with 12 disks with an aggregate and NODE2 with 12 disks and an aggregate.
Thank you very much for all your help and guidance, I have some experience in storage systems, but NetApp is pretty new to me and I start training soon, so I appreciate all the help you've given.
Just to be sure, these commands are non-destructive correct?
- Honestly, some can be if you're not careful others have "idiot proofing" build in. Just be careful that what you're running commands against doesn't have any live data on it.
I want to make sure I understand each step of this process, so I hope you don't mind I've added questions/comments on the commands you've given.
"->storage disk option modify -node * -autoassign off"
I understand this as simply turning off disk auto assign. As I understand it, the disks won't automatically get assigned to a particular node.
-Nope. this will stop the node from automatically re-assigning the disks/partitions that you are doing in the next steps.
-> storage disk removeowner -disk 1.1.x
This command is simply removing the current disk owner of the drives in the new shelf that was installed.
-Correct, these will be the ones on your external shelf
-> storage disk assign -owner node2
This will then assign the disks that were removed from the above command to the second node in the cluster. Will not specifying the specific disks just do all available disks to that node, including available disks not on shelf01? - You can disregard this question - I now understand the specific disks or disk list needs to be specified.
- I like to specify what disks i'm assigning to what so add a -disk x.x.x flag in there.
2. move all volumes off aggr2 if they're any left.
Simple enough, but to be sure this is moving all the volumes on the original aggregates on NODE2 to the new one I just created on NODE2, correct? There are two SVMs on this device as well, will that have any impact? Finally there are 4 total aggregates, two of which have the ability to move the volume, but the other two I believe are root aggregates and indicate there are no volumes to move when attempting to move anything. I've included a screen shot for further explanation.
- ignore the root aggrs, (the two smaller ones) you can't do anything with them. Just move everything off the aggr you're going to be deleting.
-You will need to move the SVM's root volumes off the aggr as well, SVMs will continue to serve out data.
-Also to note, I would rename your root aggrs to show that they are root. I typically do N1_aggr0_root or root_aggr0_N1 or something like that for node 1.
3. delete aggr2 (the partitions will now show as spares)
Self-explanatory, deleting the aggregates listed in the screen shot above, but does this include the aggr0 and aggr0_Hart2520CL_02_0?
- nope, it'll just delete the data aggr. I think even if you click delete on the root aggr it will say "hey dummy you can't do that"
4. You can re-assign the data partitions to controller 1 by the following:
-> set advanced
Entering advanced mode in CLI
-> disk removeowner -disk x.x.x -data true
Am I correct in assuming I need to specify x.x.x? Where x.x.x would be the disks assigned to NODE2? So as I understand it, this would only be the (6) 2.42TB disks currently assigned to NODE2? - correct For example -disk 1.0.0, 1.0.2, 1.0.4, 1.0.6, 1.0.8, 1.0.10, what does the -data true switch do?
- You will need to specify the disk, I typically do one at a time, just incase I fat finger something.
- With ADP each disk as a "root" partition and a "data" partition (AFF systems have a root-data1-data2 config)
the -data true will specify that you want to remove the owner of just the data part. The root partition and the physical disk container will remain owned to what they are currently owned too.
-> disk assign -disk x.x.x -data true -owner NODE1
This is then assigning the disk to NODE1, same as above I assume I need to specify disk x.x.x, also curious what the -data true switch does here.
- Same as above, you are assigning just the data part to node1. node 2 will still own the small root.
-> disk show -fields data-owner, root-owner (to verify that everything is moved over correctly)
Self-explanatory as this just verifies the work performed above.
5. zero all the spare
-> storage disk zerospares
This will then zero the spares that are now available after the above commands. Allowing me to assign the disks pulled from NODE2 to the aggregate on NODE1?
- This just makes them ready to add to node1's data aggr. If you don't zero them, they will be zeroed during the aggr grow, and will just take longer to actually grow the aggr.
Re-enable disk autoassign
->storage disk option modify -node * -autoassign on
Self-explanatory - this is just re-enabling autoassign for all disks across all nodes.
Finally, as I understand it, once this is complete I will then have NODE1 with 12 disks with an aggregate and NODE2 with 12 disks and an aggregate.
I didn't see any of the steps above to actually create the aggrs, so you will need to create 2 aggrs with all the newly assigned spares.
If you do "storage aggregate show-spare-disks" and "storage disk show -container-type spare" you should see a long of spares.
There should be 24 1.1.x that are assigned to node2. And then the spare partitions on node 1. Can you post the output of the above commands so we can verify before we create the aggr on node 2. and grow the aggr on node 1?
Thank you very much for all your help and guidance, I have some experience in storage systems, but NetApp is pretty new to me and I start training soon, so I appreciate all the help you've given. - No problem.
As I am waiting for this final volume to move I was reviewing our posts and I think the last step has me a little confused. I've included a screenshot of the current setup of disks as it is right now. I noticed that the 12-2.43TB disks are listed as "Shared" and the 12-3.64TB disks are listed as "Aggregate" once these steps are complete will the 12-2.43TB disks be Shared or Aggregate?
The sata_data_2 is the last aggregate that lives on NODE2 (other than root aggregate) and the last volume is in progress of moving to the Shelf01Aggr that I created using the 12-3.64TB disk on NODE2. Once that volume is done moving there will no longer be any data on the sata_data_2 aggregate and it will be deleted. I guess where I am getting held up is when I move those disks over to NODE1, will they still be shared?
"There should be 24 1.1.x that are assigned to node2. And then the spare partitions on node 1. Can you post the output of the above commands so we can verify before we create the aggr on node 2. and grow the aggr on node 1? "
That part is a little confusing for me as there are only 12 total disks in the new shelf and they've already been used to create an aggregate on NODE2.
