Subscribe

New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

Hi all,

 

I recently added a DS2246 to our existing FAS3220 HA pair running 7-mode.  The new shelf contains 20 SAS and 4 SSD disks with the intent of adding some needed IOPS to our existing infrastructure which was originally built solely wth SATA (2 DS4246 full of SATA drives).

 

My question is how best to "engineer" the flash pool(s) so that the performance benefit is seen across the aggregates and volumes that already exist.

 

Our setup is rather simple at this point, we have an aggr0 on each controller with VMFS volumes contained in each.  My thought is to do something such as adding 2 SSD and 3 SAS to each of the aggregates and then putting them in a flash pool so that the benefits of our new SSD and SAS drives are equally spread across the entire system.

 

 

Does this make sense?  Is there a better way to do this?

 

Thanks!

Re: New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

Hi

 

adding (forcing actually) the SAS disks to SATA AGGR will not improve perf. as the AGGR will work anyway up to the speed of the slowest disk.

The flash pool is only a caching technology and SSD's are not actually participate in the AGGR. you maybe want to read this:

https://kb.netapp.com/support/index?page=content&id=3014300&locale=en_US&access=s.

 

if you already got the SAS disks now. i will suggest to create a new AGGR for these. use the existing SATA-flashpool AGGR for data that has good DEUP ratio and need fast read Perf. and move data that doesn’t or one that require fast write to the new SAS disks AGGR (for example the root vol, DB transaction log, unique OS or virtual appliances)

 

Good luck

Re: New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

[ Edited ]

Thanks for your reply.

 

I actually have looked at that very article you referenced.  I read some other information as well that mentioned as a best practice that NetApp recommends at least 5 disks in a flash pool (can be SSD and SAS according to what I've read) which is why I suggested the 2 SSD and 3 SAS per aggregate flash-pool.

 

So if I were to do as mentioned above and take the 2 SSD and 3 SAS, add them to the aggregate and then to the flash pool, the aggregate and it's contained volumes would not benefit from the 2 SSD and SAS drives?  I was hoping to be able to split the 4 SSD in the new shelf between the 2 root aggregates we have on each of the controllers in order to gain some needed IOPS for the FC Lun's contained therein.

 

I should reitirate that we have two existing aggr0 between the 2 controllers and I was hoping to be able to use these new SSD and SAS across both aggr for an even gain in IOPS.  From what I've read it would not be recommended to add only 2 SSD in a flash pool for an aggregate, which is  why I suggested the additional SAS drives in the flash pool.

Re: New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

You have a reference for the SAS can participate in the flash pool?

the main doc about it http://www.netapp.com/us/media/tr-4070.pdf not mentions it

Re: New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

https://community.netapp.com/fukiw75442/attachments/fukiw75442/data-ontap-discussions/17208/1/43347_TR-4070_NetApp_Flash_Pool_Design_and_Implementatio...

 

First chapter, second paragraph.

 

"A Flash Pool is built from a Data ONTAP aggregate in a two-step process, described in section 3,1, "7- Mode CLI." Essentially it is the addition of SSDs into an aggregate to provide a high-bandwidth, lowlatency location that is capable of caching random reads and random overwrites. The feature does not require a license and works with any NetApp SSDs and one type of HDD per Flash Pool. That is, SSD and SAS performance drives can be combined to make a Flash Pool, or SSD and SATA capacity drives can be combined to make a Flash Pool. You cannot combine SSD, SAS, and SATA into a single Flash Pool"

Re: New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

Well - yes, that's the idea on the combination on type of drive in practice.  That quoted text doesn't say anything about count of disks, and that's where you appear to be combining two true statements incorrectly.  The minimum recommendation of 5 disks in an aggregate is not combined SAS + SSD total, but 5 of the base type alone.  A five disk aggregate gets you 2 parity plus three data disks.  

 

Consider - with RAID-DP - 3 SAS disks has two parity disks and 1 data disk in a raid group.  So your aggregate size is limited to the useable size of the one SAS disk.  Your effective IO rate is also limited to the performance of one SAS disk.  That's why the minimum disk is 5 - to at least get you 3 data disks.  

 

SSD's are added to a base aggregate in raid groups.  They don't live in isolation or get just added on to the base disks in an aggregate.  You can (at least in cDot 8.2, not sure about 7-mode) create the SSD cache as RAID-4 even if the main aggregate is RAID-DP which gives you 2 cache data, 1 cache parity, and 1 SSD spare to add onto whatever aggregate you want.

 

With the available disks I concur with previous comments.  Create one aggregate of all the SAS (leaving at least 1 spare).  If you do that, be sure to specify the proper raid group size during the aggregate creation so that you don't end yup with a raid group of 16 disks and another one of 3 disks (19 total) for instance - that would just as quickly destroy performance.  Better to define a raid group size of 19 up front.

 

You're definitely not going to split the SSD's between both nodes with only 4 available, if that was your intent.

 

 

I hope this helps you.

 

Bob Greenwald

Lead Storage Engineer

Huron Legal | Huron Consulting Group

NCDA, NCIE - SAN Clustered, Data Protection

 

Re: New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

Thanks Bob, that helps some.

 

I can't find the article off-hand but I read somewhere else as well that the recommended flash-pool size is also a raid group of five.  Our current raid group is RAID-DP as you mentioned, which would mean that the flash pool would also be raid-dp, so I would think that would give us 2 cache disks, 2 parity and one spare if I were to use the 2 SSD and 3 SAS.  To answer your last question, yes, my intent was to split the 4 SSD between both controllers, but, with the addition of the 3 SAS drives in the pool.  So you're saying that won't work and/or I shouldn't do it though?

 

That essentially leaves me having one of the two current aggregates solely running SATA drives if I can't spread the SSD/SAS drives somewhat evenly between the two aggregates in a flash pool.

 

So I have the option of creating a 19 disk SAS Aggregate and then adding my 4 SSD in a flash pool to either that aggregate or one of the two SATA aggregates....but no way of splitting the SSD's between the two existing aggregates in a combination SSD/SAS flash pool?

Re: New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

Actually, once you create the Storage Pool with the SSDs, it will allow you to assign the pool to any aggregates in the HA pair in allocation units (25% per unit, so the pool can be split to as many as 4 different aggregates).

https://library.netapp.com/ecmdocs/ECMP12458210/html/GUID-E7778CC5-20E6-48CC-B5B4-8C548C1237FF.html

 

I've got several clusters using Flash Pool combined with SAS disks in this fashion.  What I would suggest would be to determine the actual performance requirements of your various data sets and use the pool's allocation units to accelerate the highest random write volume(s), combining them with either the existing SATA or the new SAS aggregate based on the capacity requirement for the same data (there are various tools from NetApp to do so, your SE can help you determine which best fits your environment).  If all of the data is VMware as you indicated, you may also be able to put a little more effort into it (or Pro Services time if necessary) to move your root volume for one of the nodes to the SAS aggregate so you can destroy a SATA aggregagate and add the disks to the other one to get the best performance and capacity from it, essentially making one of your nodes host the SATA and the other host the SAS.

Re: New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

Hi,

In page 12 of the tr I linked to earlier there's the recommended amount of ssd in flash pool.

At the same doc there also the phrase

Starting with the Data ONTAP 8.2 software release, the RAID policies for the SSD RAID group (or groups) are independent of the policies for the HDD RAID groups within a Flash Pool aggregate. For example, a SSD RAID group in a Flash Pool aggregate can be configured with RAID 4 and a group size of 8, and the HDD RAID groups in the same Flash Pool aggregate can use RAID-DP with a group size of 16.



So. Yes add your ssd to one controller, and the SAS to another. Note that the flashpool will take some CPU resources, so seat and think about the sizing you do from all the aspects.

Re: New DS2246 SAS/SSD mix - How best to add Flash pool to existing controllers/aggregates?

Yes - you're getting closer.

 

To be clear, there is no such thing as a "combination SAS/SSD flash pool".  Flash Pool is SSD disks only that are added to an already established aggregate of either SATA or SAS disks.  You cannot make a flash pool of 2 SSD + 3 SAS disks to get to 5 disks for the recommended minimum aggregate size.  You have to count disk type separately to establish a raid group, as in 5 disks of each type.  The absolute minimum is 3 disks of each type in RAID-DP.  

 

You start with an aggregate of SATA disks or an aggregate of SAS disks, which stand alone.  To that you add a separate raid group of SSD disks which can function as a cache mechanism.  The combination of the base aggregate plus the add on SSD's makes it a "flash pool" aggregate.   The key point is that SSDs are not combined with the other disks - each type is counted and stands alone in a raid group.  The raid groups, when combined in a single aggregate, make the aggregate a flash pool.

 

That's the high level process.  The standard NetApp Data ONTAP documentation will have the exact steps to follow for both 7-mode and cDot.  TR-4070 describes the separate raid groups or the base disk type and the SSD in section 3.2.

 

Because you have only 4 SSDs, the best option to maximize the capacity of the flash pool is to use RAID-4 (single parity) on the SSDs so you can have 2 data, 1 parity and 1 spare SSD.  And yes, you are limited to one flash pool aggregate - which ever one you choose.  

 

 

I hope this helps.

 

Bob Greenwald

Lead Storage Engineer

Huron Legal | Huron Consulting Group

NCDA, NCIE - SAN Clustered, Data Protection

 

Kudos and accepted solutions are always appreciated.