Data is heavy and moving it can be a challenging task. When you keep your data centralized, remote teams and workloads need to send read and write requests great distances with high levels of latency to get access, slowing applications and reducing productivity. These performance hits are especially challenging for use cases where your teams are distributed across the globe but all access a common set of data. Many teams and workloads suffer with this but three industries stand out. Electronic design automation, media & entertainment and software development all have a few common qualities that make remote data access a challenge.
- Distributed geographic teams
- Accessing the same set of files
- Data is always synchronized to the last write
FlexCache makes light work of sharing data over distance.
NetApp ONTAP was an early pioneer of solutions to address this problem. In 2009 we created a solution for real-time caching for read-intensive workloads which would eventually become FlexCache. FlexCache creates a sparse copy or a “cache” of data located near where the work is being done. Local teams can use this cache for read purposes. For distributed teams doing read intensive work, this is a huge benefit because read requests access the local cache first, eliminating a lot of latency. You might think of this as reading at LAN speed rather than WAN speed or “reading like a local”.
Writes to the local cache would be sent back to the origin and committed there before confirmation of the write was sent back to the cache. The cache could not acknowledge the write to the client until it received confirmation from the origin. A single bit would be sent to all other caches letting them know that their copy of the data was invalid and that they would need to obtain a refreshed copy from the origin the next time a client accessed the data. This write and acknowledgement process over the WAN would add latency when writing files. The problem would increase with the volume of data being written.
FlexCache improvements in ONTAP 9.15.1
The latest release of ONTAP, version 9.15.1, introduces FlexCache write-back. With this enhancement, when a client writes to the local cache, the cache can commit the write to the local disk and provide an immediate acknowledge back to the client rather than sending the write back to the origin first. This means crossing the WAN is removed from the write flow (though it still happens out of band to update the origin). When this write-back to the local cache takes place, the origin gives lock delegation to the requesting cache for the file and immediately invalidates that file on all other caches. Any other cache must request a refreshed copy of the file when they receive a read request ensuring all data access is from the most up to date copy of data across all caches and the origin.
FlexCache write-back can reduce write latency by orders of magnitude.
Distributed teams working with files will benefit from an elimination of WAN latency when doing writes to their local copies. Imagine a team in Austin working on a 5GB chip design with the origin 3000 KM away in Seattle. The local copy of data stored on the FlexCache copy allows read/write of data with sub-millisecond write time. If the write had to go back to the origin first before it was committed and confirmation was sent back to the cache to send to the client, this could result in tens of milliseconds of latency slowing applications and reducing productivity of teams. When the write and write acknowledgement all happen on the same local network, the difference could be multiple factors faster than writes that need to be committed at a remote datacenter.
FlexCache allows you to avoid writing complex scripted copy jobs and heavy overhead management from the administrative teams to ensure the most recent copy is in use. With FlexCache, administration of the write-back is simple and wizard-driven through the GUI or scriptable in the CLI or API. That means enabling your teams to eliminate that WAN latency and work like a local can be accomplished with minimal overhead for you. And when you eliminate the risk of human error, you eliminate the potential headache of misconfiguration and split copies of data.
So let your hair down, kick your feet up, read and write your data from the cache and work like a local with FlexCache write-back, new with ONTAP 9.15.1. For additional details visit us at NetApp ONTAP.
