Subscribe

Ontap 9.x root-data-data partitioning discussion

[ Edited ]

Hi,

 

I've been testing some root-data-data partitioning setups with Ontap 9.1 and wanted to share my results with a few comments and also to get some feedback from the community.

 

 

First test scenario - Full system initialization with 1 disk shelf (ID: 0)

 

The system splits the ownership of the drives and partitions evenly amongst the 2 nodes in the HA pair.

 

Between disks 0 - 11, partitions 1 and 3 are assigned to node 1 and partition 2 is assigned to node 2

Between disks 12 - 23, partitions 2 and 3 are assigned to node 2 and partition 1 is assigned to node 1.

 

Creating a new aggregate on Node 1 with Raid Group size 23 will give me:

 

  • RG 0 - 21 x Data and 2 x Parity
  • 1 x data spare

 

One root partition is roughly 55GB on a 3.8TB SSD

 

Benefits of this setup:

 

  • root and data aggregates spread their load amongst both nodes

 

Cons:

 

  • Single disk failure affects both node data aggregates.

 

It could possibly be better to re-assign partition ownership so that disk ID 0 - 11 are owned by node 1 and 12 - 23 are owned by node 2 ?

 

netapp-adp2-single-shelf

 

 

Second test scenario - Full system initialization with 2 disk shelves (ID: 0 and ID:10) - Example 1

 

The system splits the ownership of the drives between both shelves with the following assignments:

 

Node 1 owns all disks and partitions (0 - 23) in shelf 1

Node 2 owns all disks and partitions (0 - 23) in shelf 2

 

Creating a new aggregate on Node 1 with Raid Group size 23 will give me:

 

  • RG 0 - 21 x Data and 2 x Parity
  • RG 1 - 21 x Data and 2 x Parity
  • 2 x data spares

 One root partition is roughly 22GB on a 3.8TB SSD

 

The maximum amount of partitioned disks you can have in a system is 48, so with the 2 shelves, we are at maximum capacity for partitioned disks. For the next shelf, we will need to utilize the full disk size in new aggregates.

 

Benefits of this setup I see:

 

  • in the case of 1 disk failure or a shelf failure, only 1 node/aggregate would be affected.

Cons of this setup:

 

  • A single node root and data aggregate workload is pinned to 1 shelf

 

It's possible to reassign disks so that 1 partition is owned by the partner node which will allow you to split the aggregate workload between shelves, however in the case of a disk or shelf failure both aggregates would be affected.

 

netapp-adp2-2-shelves

 

Third test scenario - Full system initialization with 2 disk shelves (ID: 0 and ID:10) - Example 2

 

In this example, I re-initialized the system with only 1 disk shelf connected.

 

Disk auto assignment was as follows:

 

  • between shelf 1 disks 0 - 11, partition 1 and 3 are assigned to Node 1 and partition 2 is assigned to Node 2
  • between shelf 1 disks 12 - 23, partition 2 and 3 are assigned to Node 2 and partition 1 is assigned to Node 1

 

I then completed the cluster setup wizard and connected the 2nd disk shelf.

 

The system split the disk ownership up for shelf 2 in the following way:

 

  • Disks 0 - 11 owned by node 1
  • Disks 12 - 23 owned by node 2

Next, I proceeded to add disks 0 - 11 to the node 1 root aggregate and disks 12 - 23 to the node 2 root aggregate. This partitioned the disks and assigned ownership of the partitions the same as shelf 1.

 

Because the system was initialized with only 1 shelf connected, it created the root partition size as 55GB as opposed to 22GB in my second test scenario above. What this means is that a 55GB root partition is used across the whole 2 shelves as opposed to 22GB. How much space do you actually save when using 3.8TB SSD's:

 

55GB x 42 (Data disks) =  2,310GB

22GB x 42 (Data disks) = 924GB

 

Difference = 1, 386GB or 40%

 

Benefits of this setup:

 

  • Load distribution amongst shelf 1 and 2

 

Cons:

 

  • Larger root partition
  • single disk or shelf failure affects both aggregates

netapp-adp2-2-shelves-2 

 

 

Fourth test scenario - Full system initialization with 2 disk shelves (ID: 0 and ID:10) - Example 3

 

Following on from my thrid test scenario, I re-assign the partitions so that partitions on disk id: 0 - 11 are owned by node 1 and 12 - 23 are owned by node 2

 

Benefits of this setup:

 

  • 1 disk failure only affects 1 node root and data aggregate
  • equal load distribution amongst the shelves.

 

Cons:

 

  • Larger root partition
  • 1 shelf failure will affect both nodes

 netapp-adp2-2-shelves-3

 

Interested to hear feedback on the above setups, which ones do you prefer and why ? 

 

Also feel free to add additional comments or setups that are not listed above.

 

Re: Ontap 9.x root-data-data partitioning discussion

If I'm going wiht ADP.. 

 

First scenario : Will be my choice, if i have plan to add more disk/shelf in future. So one/two Aggr with wired disk size.. remaining all same size. 

Third scenario : WIll be my choice, if i'm sure, im not planning to add more disk/shelf to that aggr in next 2 ~ 3 years. 

 

Shelf Failure : Yes its an important factor, in my 18+ yeard of experience.. never heard/experience a complete shelf failure on both module-path (im sure it happened to some)

 

 

Couple other points..

If SSD, dont add more than 2 shelf in a loop.

With Raid-TEC aggr reconstructing time is significantly less.

 

 

Re: Ontap 9.x root-data-data partitioning discussion

Great Blog entry from @davidrnexon to start things off .. @robinpeter  talk to us about "If SSD, dont add more than 2 shelf in a loop " - so if there is a fantasy huge budget Smiley Wink and a AFF A700 tons of SSD shelves, can we configure 20 loops or more, and still have plenty of ports for FC and 10/40 GB nics ? thanks

- if i get my hands on a big AFF i want to use all the tools for the drives ..  RAID-TEC, ADP, and Root-Data-Data

Re: Ontap 9.x root-data-data partitioning discussion

@robinpeter we actually had a customer that had a chassis failure in their 2000 series last week. Took the whole storage down affecting 500+ staff. 5-6 hour turn around for parts and engineer to replace the chassis. I never heard of this before but unfortunately these really bad situations do happen Smiley Sad

 

@xiawiz I doubt you will fit 20 loops in any system, also you wouldn't run Raid-tec with SSD it's more for the larger SATA drives, in which case you would be looking at the 8200 series.

 

 

Re: Ontap 9.x root-data-data partitioning discussion

Thank you so much for explaining this.

 

We recently purchased an AFF-A200 and we noticed that in the aggregate\disk information are, the usable size was half of the physical size.  Absolutely no one has been able to explain this until now.

 

Great info.

Re: Ontap 9.x root-data-data partitioning discussion

Hi TimmyT, thats excellent. Glad you found these post useful.