Jay Kidd, NetApp’s CTO and chief storage pundit, recently posted his blog with predictions for 2015. One of Jay’s 6 predictions in particular caught my eye: the evolving use of flash storage in the data center. As a storage technologist, I’m always excited by new technologies and trends. With something as pervasive as flash storage, it’s just a matter of time until it completely overtakes all other storage devices, right? Not so fast, says Jay:
The Future of All-Flash Arrays is Not All Flash.
“Flash is transformative to the future of Enterprise Storage. But the idea of an all-flash datacenter is utter nonsense, and at least 80% of data will continue to reside on disks. Cost matters, and the least expensive SSDs will likely be 10 times more expensive than the least expensive SATA disks through the end of the decade. Compression and deduplication apply to both disk and flash equally. Every storage architecture will incorporate flash to serve the ‘hot’ data. However, those that choose to only include flash, and have no integration with other hybrid flash/disk arrays, will be the hot rod in the garage of IT. Fun to tinker with, but not the reliable storage workhorse IT needs.”
To Jay’s point, flash and SSD’s are still far too expensive to use for stored data that does not require the utmost in IO performance. In fact, even at 2X the price, flash would still be too expensive to consider as a replacement for all HDD’s in the enterprise. We might someday see a time when solid-state storage reaches price parity with rotating disk storage, but in all likelihood it won’t be this decade and it won’t be flash. There are a group of technologies on the horizon, namely spin-torque, phase-change, and resistive RAM that are all threatening to supplant flash as the de facto enterprise solid-state storage, and these technologies are a threat to HDD as well in a more distant future.
So, rather than thinking about replacing all your HDDs with Flash in 2015, consider how a bit of well-placed flash can have a significant impact on speeding up data IO from HDDs, without breaking the IT bank. Hybrid storage arrays, as the name implies, combine traditional disk drives with flash, integrated within a single storage array and operating under the authority of a distinct storage controller.
NetApp offers two choices in hybrid storage. NetApp Flash Cache is a PCIe card that contains up to 2 TB in flash capacity per card. Flash Cache cards reside in storage controller slots, right next to the CPU and DRAM. NetApp Flash Pool, on the other hand, is a storage shelf consisting of SSDs that sits alongside shelves of traditional HDDs . In both cases, requests for stored data are immediately pulled into a flash read cache. There are no algorithms to determine what data should be cached. If you fetch it, it’s cached. Simple as that.
According to a recent report from Silverton Consulting, NetApp’s approach to flash read cache is superior to other approaches:
“Because Flash Cache and Flash Pool both act as extensions of the storage system’s memory or DRAM cache, they have an almost immediate impact on IO performance. For instance, the second a block is re-referenced; it can start benefiting from higher performance with Flash Cache or Flash Pool.
Unlike NetApp, other storage vendors’ automatic storage tiering functionality analyzes IO activity over time; then optimizes IO performance by moving data across two or three tiers of storage. Such IO analysis can easily take hours or even a day or more, delaying any performance benefit.”
This efficiency in NetApp’s caching scheme is apparent in real-world environments. Again, from the Silverton study:
“Silverton Consulting calculated the median % of flash capacity or flash efficiency for both NetApp Flash Cache controllers and Flash Pool aggregates, which come out to 0.7% and 2.0%, respectively. While similar install base information is not available for other vendors, the vendors themselves recommend the use of relatively more flash/SSD capacity for their hybrid storage systems. For enterprise-class storage, they recommend the use of 3% to 5% of system capacity in flash/SSD storage, which is a much larger percentage than what NetApp uses.
The logical conclusion is that NetApp hybrid arrays are more efficient in their use of flash capacity than other vendors’ storage and system solutions. Moreover, better flash efficiency should result in substantial cost savings when purchasing comparable storage capacities.”
All-flash data centers? Don’t expect to see them anytime soon. A-little-bit-of-flash data centers are much more likely, as vendors squeeze as much performance as possible from their good old economical disk drives.