nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Physical Layout of Mixed SSD/HDD on 2720 with 9xDS212C

JONATHAN_MARIANU_PB

I am configuring a 2720 with 9 DS212C shelves.

It is populated with the 16TB HDD and six, SSDs.

I would like to configure this for maximum write IOPS and fault tolerance.

Please critique my plan.

SSDs

Physically I intend to place one SSD each on six of the shelves. In this way a single shelf failure or replacement will not take down the aggregate built on the SSDs.

Logically I intend to partition the SSD and create two aggregates; one assigned to each node.

HDDs

I intend to partition the HDDs and organize the RAID as follows.

Because I have 10 shelves(1Head+9DS212C), each with 12 bays I would create 11, 10-Drive RAID-DP groups with one member from each shelf. Because the HDDs are partitioned there would be 11 HDD RAID groups per HDD aggregate. The remaining four HDDs would be used for spares.

Because the drives are partitioned, P1 would be assigned to node 1 and P2 would be assigned to node 2.

The trade off of this configuration is that I will have more parity drives that if I had a larger RAID group.

However my assumption is that a smaller RAID group will provide improved WRITE IOPS .

In the end I will have four aggregates; two HDD based and two SSD based with one HDD and one SSD on each controller.

Your suggestions and comments are greatly appreciated. Thank you.

Find more posts tagged with

Accepted answers

SpindleNinja

"I ordered the six pack of 3.8TB SSDs (X364A-6-SK). Again I did ask the pre-sales engineer about this and the ideal SSD count for my intended layout but my questions were unanswered. So I researched, requested feedback, and then ordered what I thought was best." - Was this a NetApp SE? Or a Partner SE?

"I intend to configure the SSDs as RAID5 with a hot spare."

You can’t configure RAID5, there is RAID4, DP and TEC.
https://docs.netapp.com/us-en/ontap/disks-aggregates/default-raid-policies-aggregates-concept.html
(RAID4 is rarely really ever used anymore).

"I would appreciate *any* feedback on the merits or disadvantages of placing one SSD in each of the six disk shelves. The installer discouraged this but did not have any substantiating documentation."

- There’s not really much gain on a 27xx system, IMHO. There is a case however if you have your SSDs on one stack and HDD on the other, however on your system you can only do one stack. I’ve seen one system like this in the wild, It was a 2554 with a few shelves and an SSD in each one and I didn’t see any substantial gain from it. It does however, spread the IO out across the Shelves, but keep in mind the back in is 12GB. Which can handle A LOT of disk IO.

One other thing. I requested that the onsite NetApp installer use the SSD to install ONTAP because I understand with ADPv2 you can partition a drive as RDD but he didn’t seem to know what I was talking about and I wasn’t certain if the minimum drive count was 6 or 8 drives for the root aggr if it was SSD.

There is no ADPv2 (RDD) on FAS system with HDDs.
The min is 8 drives for this system. (typically the 2720 has either 12 SATA drives or 4SSD + 8 SATA drives internally).

Other than paying for professional services I feel like I’m not getting the support I need for this nor can I find a best practice document for this configuration.Support isn’t really there to give config recommendations, that is typically between the customer, PS and/or the SE (either NetApp or Partner). My idea for the config recommendation was really just to maximize capacity and make management easy.

You could though create 1 SSD AGGR, You’d have a RAID DP with 3 Data drives + 2 Parity Drives + 1 Hot Spare. (est 9172.440GiB) With StoragePool you’re able to better utilize a small amount of SSDs on a system - https://docs.netapp.com/us-en/ontap/disks-aggregates/flash-pool-ssd-partitioning-aggregates-concept.html

You’d end up with 4 separate partitions and can divide them across HDD aggrs to create the hybrid aggrs, You can add as needed, i.e. 3 partitions to one aggr There is also a built in tool to better understand just how much FlashPool (StoragePool) is best for each aggr so you can see just how much Flash you should add to each aggr. Otherwise, just do it evenly across the aggrs.
https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Systems/FAS_Systems/How_to_capture_Automated_Workload_Analyzer_(AWA_)_data_on_ONTAP_systems

All comments

paul_stejskal

What is the use case? Always set up systems with your account team on the comms because they are the ones to help consult and size, not Support.

aladd

Depending on the version of ONTAP, you have the availability to create hybrid aggregates as well, this would allow you to take advantage of the SSD's for speed and the HDD's for capacity in each aggregate.

https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/What_is_a_Hybrid_aggregate%3F

JONATHAN_MARIANU_PB

Paul, Thank you for the reply.

I provided the description and a visio of the disk layout to the account team and sales engineer. I asked if this was best practice and if they had alternative suggestions. They did not and said I should go over this with the installation team.

I sent the description and visio to the individual coordinating the installation. Their response was that this is outside the scope of that team and that I would need to pay for professional services to evaluate the layout.

I opened a support case but the support engineer told me, understandably that their scope is break/fix and not new installation or making configuration recommendations.

In all three interactions I requested documentation or links to best practices.

Nothing covered my use case though.

So the layout I described in my OP is based on my research of best practices as I interpret them and based on previous experiences I had at a large telco I worked for that has deployed about 500 Netapps for which I was responsible for the backup and recovery of.

I did not manage them directly but I was very familiar with the layout although nothing was hybrid; It was either all disk or all flash.

Installation occurred yesterday but the scope was limited to physical setup, and installation of ontap and assigning IPs to the management interface. It did not include configuration of the RAID, vservers, or exports. The installer was unable to comment on my proposed configuration. I asked him to circulate it internally but no one really was able to provide a useful comment.

The installer did state that he was unclear whether he could place one SSD in each of the six shelves and that past installations placed all SSDs in one shelf so we used that configuration instead.

So it wasn't the configuration I wanted but neither he or I were able to find a reference or blueprint that covered how to lay out disks in a mixed SSD/HDD environment.

If anyone can provide documentation or comments I would be grateful.

JONATHAN_MARIANU_PB

Aladd,

This is a mixed use environment that will serve two purposes; hosting of Virtual Machines (random high IOPS) and storage of rich media and binary data from streaming scientific equipment (sequential).

I chose the 16TB drives to maximize storing capacity for the sequential data and SSD for the random data. I may utilize a flash pool in the future but at this point I intend to keep them separate.

I'd greatly appreciate any comments related to the feasibility of the disk placement / layout, the size of the RAID groups, and my partitioning scheme of assigning one partition of each disk to both nodes (through the aggregate) so that each node has a physical connection to every disk.

My thought is that this will provide maximum IOPS.

SpindleNinja

As odd as it sounds. I'd ditch ADP for this as it's easier to manage. Downside is these are large drives.

Create 2 or 4 aggrs and use the SSDs as a storagepool.

Total useable comes out to approx 1.2PiB.

JONATHAN_MARIANU_PB

SpindleNinja,

Would you be more specific on your recommendation and rationale?

When you wrote "this" I assume you mean a hybrid aggregate.

Why would you recommend a hybrid aggregate for my use case of VM (that can fit within the SSD tier) and a large amount of streaming media and raw scientific data which is sequential in nature?

Given that I have VMs and media/scientific data content why would I place both data types on the same hybrid aggregate rather than keep the media/scientific on an HDD aggregate and the VMs on the SSD aggregate which seem more appropriate for their respective IO types?

All VM data can fit on my SSDs and the VMs will greatly benefit from SSD as it is highly random IO.

Rich (streaming) media does not necessarily benefit from an SSD cache (please correct me if I am mistaken).

In fact I would think it is deleterious to overall performance because the data path is first an IO from HDD to SSD and then SSD to the network client requesting the data.

The streaming media will not necessarily be played again or by another client so I see marginal benefit to being cached.

However that media data is consuming space in the SSD pool and that SSD pool space could have been used to hold VM data which will benefit from being on SSD.

Instead the VMs are getting paged to HDD because now the SSD performance tier will not have enough capacity to keep all VM data because it is competing with the streaming media.

I welcome and look forward to any corrections or misunderstandings I may have on this.

Also, did you have a comment or thought on the advantages and disadvantages of how I would like to physically lay out the drives. Do you have any document citations I can refer to?

SpindleNinja

"this" i was just talking about the overall deployment. not using ADP will just allow disk mgmt to be simpler.

Additional question - What size are these SSDs btw? I didn't see it listed.

What concerned me about your 2x SSD aggrs, 1 per. You have 6 total disks. Which... doesn't really work out for DP aggrs across 2 controllers.

JONATHAN_MARIANU_PB

I ordered the six pack of 3.8TB SSDs (X364A-6-SK).

Again I did ask the pre-sales engineer about this and the ideal SSD count for my intended layout but my questions were unanswered.

So I researched, requested feedback, and then ordered what I thought was best.

I intend to configure the SSDs as RAID5 with a hot spare.

In this way there are four data stripes and one parity stripe.

Any data size sent to that RAID group can be evenly divided over four stripes, optimizing IO. That only works when there are 2, 4, or 8 data stripes.

This will provide enough storing space to create shares that can contain all of the virtual machines.

I would appreciate *any* feedback on the merits or disadvantages of placing one SSD in each of the six disk shelves. The installer discouraged this but did not have any substantiating documentation.

In regards to ADP, my assumption is that overall performance is improved if I partition a disk and assign one partition to each controller creating two aggregates; rather than assigning a whole disk to one controller, creating one aggregate.

The theory is that this keeps the storage as busy as possible saturating the queue to the SSDs and the HDDs.

One other thing. I requested that the onsite NetApp installer use the SSD to install ONTAP because I understand with ADPv2 you can partition a drive as RDD but he didn't seem to know what I was talking about and I wasn't certain if the minimum drive count was 6 or 8 drives for the root aggr if it was SSD. So I let him go ahead and install it the way he was comfortable. But now I need to get this unit set up and deployed!

Other than paying for professional services I feel like I'm not getting the support I need for this nor can I find a best practice document for this configuration.

Comments, or document citations are welcomed!