Google Cloud NetApp Volumes added support to create NetApp® FlexCache® volumes of NetApp ONTAP® based origin systems. FlexCache accelerates data access, reduces WAN latency, and lowers WAN bandwidth costs for read-intensive workloads, especially when clients repeatedly access the same data.
What is FlexCache?
FlexCache is a NetApp caching technology that creates a sparse, writable, persistent replica (a “cache volume”) of an ONTAP based origin volume. The cache volume can reside on the same or a different ONTAP cluster. Google Cloud NetApp Volumes is now capable of creating a cache volume, too.
FlexCache is designed to be efficient by acting as a sparse container, meaning it caches only the specific data blocks that are requested by clients (the “working dataset”), rather than the entire origin volume.
Key capabilities and benefits of FlexCache include:
- Accelerated data access and reduced latency. FlexCache brings data physically closer to the user, significantly accelerating access and reducing WAN latency, especially for read-intensive workloads.
- WAN cost and bandwidth reduction. By serving frequently accessed data from the local cache, it lowers WAN bandwidth consumption.
- Improved performance and load distribution. FlexCache distributes the read load across the origin and cache volumes, boosting overall performance.
- Data consistency and coherency. It maintains data integrity, so the cached data remains consistent, coherent, and current with the origin.
- Enhanced availability. If a network disconnects from the origin (disconnected mode), the cache can continue to serve data that’s already cached.
- Simplified management. Data management and disaster recovery strategies need to be implemented only at the origin volume.
- Global namespace. FlexCache enables you to create a true multiprotocol global file system namespace.
FlexCache use cases and topologies
FlexCache is particularly well suited for read-heavy workloads and is ideal for use cases such as:
- Electronic Design Automation (EDA)
- Media rendering
- AI, machine learning, and deep learning
- Software-build environments and common tool distribution
- Unstructured NAS data, like home directories
- Cloud bursting, acceleration, and caching
Extend or burst workloads to the cloud
The “burst to the cloud” use case involves extending an on-premises or separate data center workload into Google Cloud to handle peak demand or leverage cloud-specific compute resources.
FlexCache is an ideal solution for this use case because it allows the read-intensive part of the workload to run efficiently in the cloud against the same primary dataset.
Essentially, FlexCache allows you to access remote data at local speeds with full consistency, which is the core requirement for successfully bursting a workload to the cloud.
Scale out your workload
You can replicate an origin volume to multiple cache volumes (fan-out). Using this approach, you can add multiple caches in the same location to provide scale-out performance for large compute clusters.
A typical use case is scaling out the aggregated read performance of your origin volumes to keep large or multiple compute clusters utilized.
Working with the same data in geographically distributed teams
FlexCache can also be used to place caches in different locations, allowing geographically distributed teams to access a local cache instead of a remote origin volume. This approach enables better performance, while maintaining the source of truth on the origin volume.
A typical use case is a media or game design company that want to access the same digital assets from their studios all over the world.
Write handling and cache consistency
Write handling
FlexCache uses a write-around policy, meaning that all write operations are applied directly to the origin volume, completely bypassing the cache volume.
- When a client writes data to a file on a FlexCache volume, the write request is immediately forwarded to the origin volume, which processes the write and coordinates all necessary actions (locking, cache invalidations).
- The write operation never lands on the cache volume itself.
Cache consistency and coherence
FlexCache maintains data consistency, coherence, and currency by storing state in internal metadata files and using a cache invalidation mechanism:
- Centralized coordination. By executing all writes at the origin, the origin is responsible for coordinating simultaneous writes and enabling global data integrity.
- Cache invalidation. After the origin successfully executes a write to a file, it checks a central record for all cache volumes that have that file cached.
- The origin then sends an invalidation message to each relevant cache to revoke the data delegation for that file.
- The cache marks the file as invalid, and the next client read request for that file is forced to fetch the current data from the origin; thus, the client always accesses the latest version.
File locking
File locking in FlexCache is centrally coordinated by the origin volume, but for performance, much of the lock granting is distributed to the cache volumes.
- Distributed architecture. FlexCached manages locking by using a hidden remote lock entry metafile (RLEM) at both the origin and cache volumes.
- Read locks. All read locks are delegated and distributed to the caches. When a cache has the lock delegation for a file, it can grant read locks to clients without needing to contact the origin.
- Write locks:
- NLM (NFSv3). Write locks are centralized—still forwarded to and granted only at the origin.
- SMB and NFSv4.x. Write locks are delegated. The origin grants a write delegation to one cache, which can then grant local write locks to its clients. This delegation must be revoked by the origin if another cache or the origin itself needs a conflicting lock.
- Global file locking. With this optional feature, mandatory locks (like deny-read and exclusive byte-range locks) are honored globally, though this can affect performance by requiring caches to contact the origin on every read.
FlexCache enables new use cases
FlexCache brings crucial caching capabilities to Google Cloud NetApp Volumes, unlocking many new use cases that previously required cumbersome and time-consuming manual data copies. This caching approach greatly simplifies scenarios in which maintaining data consistency across multiple copies was a complex challenge. FlexCache offers a powerful and straightforward solution for such requirements.
FlexCache is available in allow-listed general availability. To learn more, check the documentation. For a detailed discussion of capabilities, use case fit, and sizing, contact our Google Cloud specialists.
