If you're a regular reader of Tech OnTap, you know that there's a transition taking place in the storage industry from FC-AL to Serial-Attached SCSI (SAS) storage architectures and disk drives. In fact, Gartner Dataquest predicts that SAS will completely replace FC-AL by 2013.
This is happening because SAS offers better reliability and resiliency, greater bandwidth, and greatly improved connectivity. If you're not already familiar with SAS, you can read more about these advantages and get a good overview of SAS technology in a recent Tech OnTap® article.
NetApp is well into this technology transition. SAS-connected disk shelves now account for about 10% of the NetApp installed base and for more than 50% of the storage shipped with new NetApp® systems. The reliability of our DS4243 SAS disk shelf is the best in NetApp history, with no reports of shelf-induced outages as of January 2011. Recently, NetApp introduced the DS2246 disk shelf, which leverages the DS4243 architecture into a denser package. The DS2246 builds upon the success of the DS4243 by offering increased density and power savings with the same excellent reliability and resiliency.
This article discusses our SAS products, including best practices for selection and implementation.
NetApp SAS Technology
NetApp is committed to help you successfully navigate the transition to SAS with the right technology offerings and SAS expertise. Since 2007, we’ve delivered three generations of SAS products (starting with SAS support on the entry-level FAS2040 and FAS2050). The recently introduced FAS6200 series and FAS3200 series storage systems include onboard SAS ports to simplify the transition to SAS.
Our SAS disk shelves are designed to deliver exceptional resiliency and reliability with redundant components, cross-bar switches, alternate control path (ACP), and nondisruptive firmware upgrades. We recently gave density a boost with the new DS2246 disk shelf, which uses 2.5" small form factor drives. Compared to the existing DS4243 disk shelf, the DS2246 delivers double the storage density and higher performance density while requiring significantly less power. Engineering enhancements include a new I/O module that delivers 24Gb/sec bandwidth per SAS port.
The DS2246 disk shelf can hold up to 24 2.5" drives in a 2U enclosure with 6Gb/sec (24Gb/sec aggregate per SAS port) SAS connectivity.
Many of the specifics of the DS2246 are encoded in the product name:
DSxxxx: Disk shelf
DSx24x: 24 drives
DSxxx6: 6Gbps SAS (aggregate 24Gb/sec)
In other words, the DS2246 is a 2U enclosure that provides 24 SAS disk drives with 6Gb SAS connectivity. The DS2246 offers the same capacity as a DS4243 disk shelf filled with SAS drives, but with double the density.
In terms of performance density, our laboratory tests show that the DS2246 delivers up to 60% more IOPs per rack unit for OLTP workloads. It also offers 30-50% power savings over a similarly configured DS4243 owing to its use of 2.5" small form factor (SFF) disk drives. The DS2246 only requires two power supplies (PSUs) per shelf, whereas the DS4243 requires four. (Both shelves use the same PSU.)
SAS disk drives are currently available in two sizes for the DS2246:
- 450GB, 2.5", 10k rpm SAS drive
- 600GB, 2.5", 10k rpm SAS drive
The DS2246 offers full support for SAS 2.0 with 6Gb/sec bandwidth. It includes a new I/O module, the IOM6, which is SAS 2.0 compliant. NetApp storage systems connect to the IOM6 using a standard wide port: a set of four SAS lanes with an aggregate bandwidth of 24Gb/sec.
The DS4243 disk shelf—released in 2009—has a 4U form factor and holds up to 24 SAS disks with 3Gb/sec SAS connectivity—12Gb/sec per wide port. (Note that this is more bandwidth than an FC disk shelf using 4Gb or 8Gb FC.) The DS4243 is the disk shelf option for versatility, since this single model supports three disk types: SAS, SATA, and SSD. The DS14 FC disk shelf required you to purchase different models to support different disk types. The DS4243 also allows you to mix SAS and SATA populated shelves in the same SAS stack if you want to. Table 1 compares the DS2246 and DS4243 in terms of capacity and density in a 42U rack.
Comparison of the DS2246 and DS4243 in terms of density and power consumption.
The DS4243 disk shelf with 15k SAS disks provides the maximum hard disk drive performance. The DS2246 (using 10k rpm SAS) delivers slightly less performance than the DS4243, but offers greater capacity and performance density as well as significant power savings. Performance differences can be offset for many workloads by using NetApp Flash Cache. (See later section.)
NetApp SAS Adapters
NetApp began shipping the X2065A quad-port PCIe SAS adapter with the release of the DS4243. This adapter is 6Gb/sec capable and can therefore take advantage of the higher link speed when connected to the DS2246. It supports both the DS4243 and the DS2246.
To facilitate the transition to SAS, NetApp designed its latest storage systems, the FAS3200 series and the FAS6200 series, with onboard SAS ports. These onboard ports also support the 6Gb/sec link speeds and can be used with either the DS4243 or the DS2246. You can read more about the FAS3200 series in a companion article in this issue of Tech OnTap.
The onboard SAS port for the FAS2040 and the X2062 dual-port SAS HBA for the FAS2050 support both the DS4243 and DS2246, but continue to operate at a link speed of 3Gb/sec when connected to the DS2246.
NetApp Flash Cache (PAM II) is compatible with all NetApp disk and disk-shelf technology, including the DS2246 and the DS4243 as well as older DS14 models. The intelligent use of caching provides a way to decouple storage performance from the numbers and types of disks in the underlying disk array to substantially improve cost and at the same time decrease the administrative burden for performance tuning. Flash Cache provides an optional second-level cache, accepting blocks as they are ejected from the system buffer cache to create a large, low-latency block pool.
Flash Cache can cut storage costs by 50% or more while reducing footprint and power consumption by around 60%. It can cut your storage costs by reducing the number of spindles needed for a given level of performance by as much as 75% and by allowing you to replace high-performance disks with more economical options—for instance, SATA instead of SAS.
When you use Flash Cache in conjunction with NetApp deduplication or FlexClone® technologies, a significant cache amplification effect occurs, significantly increasing the number of cache hits and further reducing latency.
You can learn more about the use of Flash Cache in conjunction with NetApp technologies in a recent Tech OnTap article.
NetApp SAS Best Practices
Getting started with SAS technology is straightforward with NetApp systems, but attention to a few best practices can help you make sure of success. First of all, you need to decide which disk shelf, the DS2246 or the DS4243, is best for your needs. The selection criteria presented in Table 2 should help with this decision. Also refer back to Table 1 for density and power information.
Guidelines for choosing the correct SAS disk shelf.
Of course, if you need both shelves, you can connect the DS2246 and the DS4243 to the same storage system, subject to the guidelines in the following section.
Configuring SAS Stacks
NetApp uses the term "stack" to refer to a collection of correctly wired and interconnected SAS shelves and adapters. The following guidelines apply:
- Up to 10 shelves are supported per stack, except in the FAS2040 and FAS2050, which support up to 4 shelves per stack.
- The DS4243 and DS2246 cannot be mixed in the same stack. They can be mixed in the same system by connecting them to different SAS ports. If you connect them to different ports on the same adapter, the adapter will automatically use the correct link speed for each stack.
- When using the DS4243, SAS and SATA drives can be mixed in the same stack but not in the same shelf.
- You should not mix 15k and 10k RPM SAS drives in the same aggregate. While it is possible, this will very likely limit the performance you’ll see from the faster drives.
- NetApp extensively qualifies SAS cables for use with our SAS shelf family. SAS cables are a high-performance and critical component of the SAS architecture. Only official NetApp SAS cables are supported for use in SAS data path connections.
Configuring Shelf IDS
Shelf IDs can range from 00 to 99. We recommend assigning intervals of 10 to each stack attached to a storage system. For example, the first stack would have IDs 10 through 19 reserved, while stack two would have IDs 20 through 29 reserved, and so on. Those IDs are reserved even if the stacks are not fully populated. For example, if the first stack has four shelves, the shelf IDs would be 10, 11, 12, and 13. The second stack would start at shelf ID 20, even though IDs 14 through 19 are not currently utilized. This helps you to easily identify which shelves are assigned to each stack. Unused IDs can be used for future stack expansion. Both the DS4243 and DS2246 support the hot addition of capacity, and additional shelves can be added to a running storage system without disruption.
Alternate Control Path
ACP was introduced at the same time as the DS4243 for out-of-band management on SAS disk shelves. ACP gives you an alternate communications path to each of your disk shelves. It is completely separate from the SAS data path and provides options for nondisruptive recovery of shelf modules, including the ability to reset or power cycle an individual I/O module (IOM) or an entire domain (that is, all IOMs connected to the “A" shelf modules within a SAS stack). We designed in the ability to power cycle the entire shelf as well. For redundancy, each shelf contains two IOMs, and each IOM has ACP connectivity. ACP technology enhances the ability of Data ONTAP® to automatically reset a misbehaving component in order to return it to a fully operational mode without disruption.
ACP actively monitors the SAS data path for issues that can be corrected by a nondisruptive shelf module reset or power cycle. The proactive shelf recovery mechanism of Data ONTAP performs these actions automatically; this is enabled by default when using ACP.
Although NetApp highly recommends it, ACP is not required. Because ACP is completely separate from the data path, the data path continues to function when ACP is not connected or not operational. Many of the ACP functions described can be performed manually when ACP is not present.
Getting Started with SAS
With the DS14 approaching end of availability (see sidebar), it might be time for you to start looking seriously at SAS if you haven’t done so already. This article, along with the related Tech OnTap article from last year, should give you the information you need to understand the technology and make the right choices.