- Furthering storage scalability for the most demanding compute and data intensive applications
- Furthering overall Flexibility, TCO and Data Protection Features for VMware Cloud on AWS (VMC)
- Simplified user experience, higher scalability, and improved data protection for Amazon Elastic Kubernetes Service (EKS) with FSx for ONTAP Persistent...
It’s been little more than two years since Amazon Web Services (AWS) launched Amazon FSx for NetApp ONTAP in collaboration with us. Since then, thousands of customers have taken advantage of the ‘only unified, multi-protocol storage service’ available in AWS’ services portfolio. With the fully featured ONTAP storage implementation in the cloud, customers have been using FSx for ONTAP for a wide variety of workloads like user and application file shares, backup and disaster recovery, relational databases, VMware Cloud on AWS, SAP HANA, etc. Our customers love FSx for ONTAP for many reasons that include:
- Fully managed enterprise-grade storage service lowers administrative burden in managing storage and frees valuable resources to strategic initiatives that help drive business growth
- Multi-protocol capability via SMB/NFS/S3 and iSCSI reduces storage complexity by helping customers consolidate different applications on the same storage technology
- Fast and consistent performance for demanding workloads and flexibility to scale capacity and performance as per business needs
- Advanced data protection and data management features, including high availability within or across AWS Availability Zones (AZs), space-efficient snapshots, thin clones, data replication help improve data durability and availability while reducing cost and simplifying data management operations
- Data efficiency features such as tiering to lower cost storage, data deduplication, compression and compaction reduce overall total cost of ownership
- Robust security features, such as encryption at rest and in transit, to protect sensitive data along with ability to meet regulatory compliance requirements using NetApp SnapLock
While our joint teams have delivered tremendous innovation in the last two years, at AWS re:Invent 2023, we are very pleased to further the value we bring to our joint customers with a robust set of new capabilities.
Furthering storage scalability for the most demanding compute and data intensive applications
FSx for ONTAP Scale-out Architecture
In today’s fast paced business environments, our customers value speed – speed of processing vast amounts of data to get faster insights as well as reduce overall time and hence, cost of running the cloud infrastructure. Up until now, FSx for ONTAP file systems were powered by a single pair of storage servers, delivering up to 4 GBps of throughput and 160K IOPS. Most workloads are well-served by the performance of a single pair of storage servers. Depending on the workload, customers also had the ability to combine multiple discrete file systems together to drive much higher performance. An example of such configuration can be found in this blog.
While FSx for ONTAP has served the needs of many workloads so far, our customers with the most compute-intensive ONTAP workloads—such as large-scale electronic design automation, VFX production and film editing, seismic analysis, machine learning training and generative AI —needed much higher performance than a single pair of storage servers can deliver. In addition, given the large unstructured data sets that need to be analyzed, to simplify data management, our customers also need a single petabyte scale namespace to provide a single consolidated storage for distributed computing environments.
The new FSx for ONTAP scale-out architecture extends all the core benefits of FSx for ONTAP with vastly improved performance and capacity scalability. A single FSx for ONTAP storage system can now deliver up to 36GB per second of throughput and 1.2 million IOPS—a 9X improvement in performance from today’s 4GBps throughput and 160,00 IOPS limit. Up to six pairs of storage controllers, each providing up to 6GBps throughput and 200,000 IOPS at sub-millisecond file operation latencies, operate as one single storage cluster. Using NetApp’s FlexGroup technology, compute applications benefit from the higher aggregate performance and a single petabyte scale namespace accessible via SMB, NFS or parallel NFS (pNFS) protocols.
Key benefits of FSx for ONTAP scale-out storage:
- High scalability: Up to 20 PiB in a single cluster wide namespace with billions files on a cluster with 6 controller pairs.
- High performance: FlexGroup volumes can utilize the resources of the cluster to serve workloads that have high throughput and low latency. A single cluster with 6 controller pairs can serve up to 36GBps read, 6.6GBps write throughput and 1.2M IOPS at sub ms latency.
- Simplified management: A FlexGroup volume is a single namespace container that can be managed in a similar way as FlexVol volumes.
Our customers can easily lift and shift, extend or deploy new high-performance workloads on AWS, while leveraging AWS’s agility, scalability, and security–without having to modify their applications, tools, or processes. To get started, customers will need to specify the performance (throughput and IOPS) and capacity needed for the SSD storage layer. AWS will deploy the file system and notify you once the system is ready for use.
Amazon FSx for NetApp ONTAP scale-out capabilities are now available as part of Single AZ (SAZ) file systems in the following AWS regions: US East (Ohio, N. Virginia), US West (Oregon), Asia Pacific (Sydney), and Europe (Ireland) Regions. For more information, visit https://aws.amazon.com/blogs/aws/new-scale-out-file-systems-for-amazon-fsx-for-netapp-ontap
Some of our largest customers are eager to put their hands on the high-performance storage options and we can’t wait to hear how this innovation will help them drive their business outcomes - faster!
Manage and backup FSx for ONTAP FlexGroup volumes using the AWS Console and API
Our customers can now create, manage, and back up FlexGroup volumes using the AWS Management Console, FSx CLI, and AWS SDK, making it even easier to manage and protect large scale data stored in FlexGroups. Up until now, customers could not create Amazon FSx backups for FlexGroups and could only manage FlexGroups using ONTAP CLI or REST API.
Improving Service Deployment Flexibility by introducing FSx for ONTAP Multi-AZ Deployment in shared VPC Accounts
FSx for ONTAP now supports creating Multi-AZ file systems from Shared VPC accounts. VPC Sharing is an AWS feature that allows resource owners (“owner accounts") to share one or more VPC subnets with other accounts (“participant accounts”) in their AWS organization. This allows participants to manage their own application resources in the subnets shared with them. Before today, participants could only create single AZ file systems in the subnets shared with them. To take advantage of built-in cross-AZ availability for business-critical applications, now Multi-AZ file systems can be deployed in the shared VPCs. For more information, please visit https://aws.amazon.com/blogs/aws/introducing-shared-vpc-support-for-amazon-fsx-for-netapp-ontap/
Furthering overall Flexibility, TCO and Data Protection Features for VMware Cloud on AWS (VMC)
Disaggregated Storage Architecture with M7i Instances and FSx for ONTAP
Building on the advantages of disaggregated architecture to further the architectural flexibility, cost efficiency and performance for VMC, VMware today announced M7i instances to the VMC portfolio in addition to i3en and i4i instance types. The m7i.metal-24xl instance type is a compute optimized instance without any local vSAN storage, hence enabling a disaggregated compute and storage architecture leveraging FSx for ONTAP as the external NFS datastore. This joint solution is yet another testament to the strong partnership among VMware, NetApp and Amazon Web Services to bring joint innovation to our customers. NetApp's Ronen Schwartz and VMware's Narayan Bharadwaj discuss the partnership between NetApp and VMware in the video below.
With up to 48 physical cores, 348GiB of memory, the m7i instances when used with FSx for ONTAP will offer a great solution for use cases such as disaster recovery and backup infrastructure, compute intensive, AI and ML applications, etc. Our customers already benefit by utilizing FSx for ONTAP external datastores to improve total cost of ownership and leverage advance data protection, data management and efficiency features of ONTAP for their VMC infrastructure. However, with i3en and i4i instances, customers still needed to plan for utilizing vSAN storage for general purpose VMs while utilizing FSx for ONTAP for storage heavy and VMs running business critical applications. Expected to be availalbe in VMware's Q4FY24, the diskless m7i instances will simplify the architecture and storage management by utilizing just one external storage type for virtual machines, provide more flexibility and control for customers to scale compute and storage independently, and improve overall TCO. Below is a sneakpeak of how the m7i instances will work with FSx for ONTAP.
Extending Backup and Recovery to Enterprise Databases on VMC
With the launch of FSx for ONTAP, we introduced protection of MSSQL, Oracle, and SAP HANA databases using NetApp’s SnapCenter. Our data protection journey got a further boost at VMware Explore this year (Aug 2023), where we announced support for NetApp BlueXP™ backup and recovery for VMs to efficiently backup and restore virtual machines running on VMC with FSx for ONTAP NFS datastores. With a management experience that’s integrated within the VMware Cloud SDDC vCenter UI, our customers have been using space-efficient backup, recovery, and clone for VMs to protect their no performance impacting ‘datastore level’ snapshots of virtual machines residing on FSx for ONTAP NFS datastores. In addition to datastore level backups, our customers also want to protect their business and mission critical database applications via database consistent backups, however the options have been limited.
Up until now, customers needing database consistent backups for Microsoft SQL Server and Oracle could only take a snapshot at the virtual machine level using VMware snapshots. These snapshots taken at the virtual machine level capture a point in time copy of the state of virtual machine including the disk state. However, as described here, VMware recommends to avoid taking VM-level snapshots on production SQL Server workloads due to performance impact such snapshots could cause. The other alternative would be to use third party backup solutions which typically rely on resource intensive host-based backup techniques which could potentially impact performance and add incremental costs and complexity to managing backups.
We are now pleased to announce support for application consistent backups of Microsoft SQL Server and Oracle databases running on VMware Cloud– all within SnapCenter. Irrespective of the size of production databases, customers can take application consistent snapshots with no performance impact to business-critical applications. Based on the tiering policy, these snapshots can be tiered to FSx for ONTAP capacity pool for lowering overall storage costs. In addition, using SnapVault, customers could backup and retain these snapshots to another FSx for ONTAP file system running in a different AWS region. Overall, without the complexity and cost of managing a third-party backup solution or worrying about performance impact to production database workloads, customers can confidently protect their application databases with a fully integrated and supported solution from NetApp. With the introduction of m7i instances running with FSx for ONTAP, customers will also be able to use SnapCenter to simplify their backup architecture for the entire virtual machine infrastructure hosted on the NFS datastores.
Protection of virtualized MSSQL and Oracle databases running on VMware Cloud with FSx for ONTAP storage is available as a public for non production workloads at BlueXP backup and recovery for VMs (version 1.1 X5) SnapCenter 4.9 P1. For more information on how SnapCenter and BlueXP backup and recovery for VMware Cloud works to enable database consistent snapshots, please visit blog.
Other recent announcements for VMware Cloud on AWS solution:
- VPC Peering for VMware Cloud on AWS: To reduce the overall cost of VMware Cloud on AWS with FSx for ONTAP, earlier this month, VMware released VPC Peering network connectivity option to connect SDDC clusters to FSx for ONTAP Single-AZ configurations. Customers can now utilize the VPC Peering connectivity option that maintains performance and reduces overall costs incurred due to vTGW.
- At NetApp INSIGHT, we announced the public preview of NetApp BlueXP disaster recovery. This service offering provides low-cost disaster protection for VMware workloads from on-premises NetApp ONTAP® environments to VMware Cloud on AWS when used with Amazon FSx for NetApp ONTAP. We are looking forward to making the service generally available service soon. As an early adopter, our partner World Wide Technology, Inc. (WWT), has been evaluating the preview. Based on their testing at the WWT Advanced Technology Center (ATC), WWT has published a technical guide which can be accessed here. The introduction of m7i instances from VMware is very relevant to be used as a DR target infrastructure as we expect to improve cost efficiency and architectural simplicity.
If you have not put your hands on BlueXP disaster recovery service yet, access to the public preview can be requested here.
Simplified user experience, higher scalability, and improved data protection for Amazon Elastic Kubernetes Service (EKS) with FSx for ONTAP Persistent Storage
FSx for ONTAP for Kubernetes with NetApp’s Astra portfolio combines the benefits of ONTAP storage with advanced Kubernetes storage and data management. The integration offers seamless storage access, data protection, and simplified data management for stateful Kubernetes workloads. Our customers can easily protect, backup, and migrate Kubernetes workloads, and instantly create working application clones. Astra Control is bult on Astra Trident, which is a CSI-specification compliant dynamic storage orchestrator that provides storage access, provisioning, volume resizing, snapshot management, and cloning, enabling performant and scalable access to FSx for ONTAP. Astra Trident enables multiple drivers for customers to provision block storage using iSCSI or file storage via SMB or NFS shares. With applications running on Kubernetes typically architected for AZ level redundancy, our customers find the FSx for ONTAP Multi-AZ storage option very valuable as it provides multi-AZ storage capability without the overhead of architecting and managing high availability.
Simplifying Astra Trident User Experience for Amazon EKS
Until now, customers using the Astra Trident driver had to deploy Astra Trident separately from the EKS cluster. To provide an integrated experience for EKS customers wanting to leverage FSx for ONTAP as their persistent storage, we are now pleased to offer a preview of Astra Trident driver as an EKS add-on via the EKS marketplace. The solution will allow customers to streamline EKS deployments with FSx for ONTAP, thereby simplifying persistent storage deployment for EKS clusters right from the EKS console. We expect to make the add-on driver deployment method generally available before end of the calendar year.
For more details on how the new Astra Trident EKS add-on helps simplifying provisioning storage for EKS, please visit: https://bluexp.netapp.com/blog/aws-fsxn-blg-astra-trident-now-available-as-an-eks-add-on-with-fsx-for-ontap
Other Recent Astra Enhancements
Recently at KubeCon 2023, we announced new enhancements that improve overall support for data applications running on Kubernetes:
- Backup and restore of applications using qtrees for persistent volumes (PVs): With the ontap-nas-economy driver, Kubernetes deployments can scale up to 300 qtrees per FlexVol volume, allowing tens of thousands of PVs per FSx for ONTAP file system. Customers can now protect, using backup and restore, Kubernetes workloads using PVs that are backed by qtrees, which provide virtually unlimited scale.
- In-flight data encryption between containers and persistent volumes (PVs) based on Kerberos
- Ransomware protection by using object storage retention policies.
- Red Hat OpenShift Service on AWS (ROSA) support
For more information on recent enhancements to Astra for FSx for ONTAP, please visit: https://www.netapp.com/blog/astra-control-protection-for-ontap-qtrees-kubernetes/
After launching FSx for ONTAP in September 2021, this year’s launch is the most comprehensive innovation delivered to make FSx for ONTAP as the most versatile storage service for our customer’s applications running on AWS. If you are at re:Invent 2023, please visit us at booth 950 to experience a great set of demos and sessions that showcase all these new capabilities and a lot more. For a more comprehensive list of activities happening at the show, please visit https://www.netapp.com/aws/reinvent/