This post was originally published on SANbytes Blog.
We have previously discussed how Flash memory can accelerate large amounts of data by only putting a small percentage of data on Flash memory. Let’s take a look at an OEM partner’s approach to increasing performance with Flash: Teradata Hybrid Storage. Teradata is a leading NetApp OEM Partner that takes an innovative approach to integrating NetApp E-Series into their data warehouse hardware platforms. As a dedicated business analytics system, a Teradata data warehouse knows exactly how often each data block is accessed and hence where it should be stored, so an even more optimized approach can be taken.
Let’s take a look under the hood: Teradata leveraged the basic E-Series feature set for mixing SSD’s and HDD’s within a single array to create the Hybrid Storage system. Here’s where the innovation comes in by providing fully automatic, intelligent data migration between SSD and HDD storage tiers at the database level – something the array cannot do by itself. . This secret sauce is called the Teradata Virtual Storage software:
Teradata Virtual Storage literally takes the data’s temperature by monitoring how often all data blocks are accessed and automatically migrating frequently accessed data to SSD and less frequently accessed data to HDD.
The way that data is used in data warehouse systems is surprisingly similar across all customers and industries. By monitoring 100s of systems Teradata has found that data access patterns follow the classic 80:20 rule. That means typically 20% of the data is hot since it is accessed 80% of the time. If this hot data is always stored in the SSD, there is a huge performance boost to be gained. So with a relatively small portion of the storage array in SSDs customers have seen up to a 60X improvement in the time to perform their analytic operations. Performance, performance and performance are the ultimate requirements for a data warehouse. That the investment in SSDs and Hybrid Storage can make such a huge difference has made it an instant hit with many of Teradata’s Fortune 1000 customers. They can now experience this added performance to make faster, better and smarter decisions about their business.
Yes, there are other ways to use SSD technology in database systems – namely their use as server side cache. Indeed, a cache oriented approach works well in transactional systems where lots of small segments of transaction data can be efficiently kept in the cache. Data warehousing and business analytics are a different story and require frequent use of large amounts of data which can be accessed with consistently low latency. Another consideration is the ability to scale the Flash technology at the array level beyond the limits of an individual server. For Teradata’s data warehouse use case, their specific hybrid approach to using Flash has proven to be the best fit.