Businesses are looking to use advancements in AI and Analytics to generate new added-value benefits for their customers such as improved decision-making, enhanced customer experiences, and greater operational efficiency. To do this, they often use AWS cloud-based services such as Amazon Bedrock, Amazon SageMaker, and Amazon Athena to train new models, create data lakes and generate generative AI based search and analytics. However, these AWS services are designed to natively integrate with data sourced from Amazon Simple Storage Service (Amazon S3) and currently cannot directly access file data.
Today, all this has changed. Amazon FSx for NetApp ONTAP now supports S3 data access to NFS and SMB file systems, enabling their seamless integration with dozens of S3-based AWS services such as Amazon Bedrock, SageMaker, Athena, AWS Glue, and many more. Customers can now connect AWS services to all their data, be it stored in file, block, or object storage, on premises or in the cloud.
Let’s take a closer look at this game-changing capability and see how it works.
How S3 access for FSx for ONTAP helps customers get the most out of their data
The new Amazon S3 Access Points for FSx for ONTAP enables organizations to leverage their existing file data for S3-based AI/ML and analytics services, without complex migrations or data duplication. This provides business benefit across all industries. Here are just a few examples:
- In healthcare, providers can now securely analyze electronic medical records (EMR) files using Amazon Athena and AWS Glue, accelerating patient insights while maintaining compliance without refactoring their systems.
- Financial institutions can now leverage Amazon SageMaker and Amazon Bedrock to train fraud detection models on existing file datasets, improving risk management and customer trust without costly migrations.
- Electronic design automation (EDA) companies can use Amazon Bedrock for creating AI-driven design workflows based on on-premises file data, speeding up product development and reducing infrastructure costs.
- Enterprises in media, entertainment, and gaming can perform real-time log file analytics with Amazon OpenSearch, streamlining operations and incident response.
- Automotive manufacturers can gain real-time analytics and predictive maintenance by directly connecting telemetry files to Amazon QuickSight and AI services.
Let’s see how S3 Access Points work.
Attaching an S3 Access Point to FSx for ONTAP
Up until now, connecting NFS or SMB file data to Amazon S3-based services required data duplication and migration, format conversion or an application refactoring. These steps often introduce excessive delays, costs and risks for implementing AI/ML and analytics workflows in business-critical workloads.
Now, by attaching S3 Access Points to FSx for ONTAP’s NFS and SMB volumes, files stored in these volumes can be accessed by any AWS services as if they were in an S3 bucket. When attaching the Access Point, the user defines its unique id, specify the file access type (Unix or Windows) and adds a username for authorizing file access requests by the Acess Point. The creation and attachment is done using the AWS Management Console, CLI, or through NetApp Workload Factory.
Once the S3 Access Point is attached, it will appear in the AWS Management Console and will have a unique Access Point alias. This alias is used as the S3 bucket name provided to the AWS services to which you want to connect. For example, you can provide the alias to an Amazon Bedrock knowledge base, and it will then use the files in the FSx for ONTAP volume to provide contextual answers to queries.
You can attach multiple S3 Access Points to a single FSx for ONTAP volume, each with its own unique access level, allowing you to connect to as many AWS services a required, as can be seen in the following illustration:
Several unique advantages of S3 Access Points on FSx for ONTAP:
- When an S3 Access Point is attached to an FSx for ONTAP volume, each file residing in the volume is provided with a unique S3 object name that includes information of its location within the file directory to allow the AWS service to locate and access it. As explained, there is no need to copy the files to an S3 bucket or move the data out of FSx for ONTAP.
- The username provided when attaching the S3 Access Point defines the access permissions for the connected AWS service, based on the volume’s existing Unix or Active Directory (AD) permissions. This preserves existing access and governance policies.
- Since files are not duplicated or reformatted, rather the same files are accessed through different protocols (NFS, SMB, or S3), file management remains simple and seamless. If a file is modified or deleted, the change is immediately reflected for all access types.
- Because nothing has changed in the way data is stored and managed by FSx for ONTAP, users continue to enjoy all the unique storage efficiency capabilities it offers. Data deduplication and compression reduce storage costs by eliminating redundant data and compressing data to save space. Data tiering automatically moves infrequently accessed data to a lower cost storage tier to further optimize storage costs and performance.
- In a similar manner, all of FSx for ONTAP’s data management capabilities are kept. Snapshots can be created to store lightweight, point-in-time copies of data for recovering files in the event of data loss or file corruption. Windows users can independently restore previous versions of files without from snapshots. Clones, which are lightweight writable copies of the volume, can continue to be used for quick, cost efficient testing and development purposes. FSx for ONTAP also supports long term file locking for protection and compliance purposes.
Creating S3 Access Points for on-premises data
An additional advantage of this new capability is that it allows organizations already running on-premises ONTAP systems to seamlessly integrate their file data with Amazon S3-based services. This is done by easily mirroring the on-premises volumes to FSx for ONTAP using NetApp SnapMirror’s efficient, incremental data replication—as can be seen in the following illustration:
SnapMirror’s efficient, incremental data replication—as can be seen in the following illustration:
This allows the organization to leverage new AI/ML and analytics services on files that were generated by their on-premises databases, enterprise applications and IT systems without needing to change their infrastructure or operational workflows.
This is just the beginning
Amazon S3 Access Points for FSx for ONTAP offer customers across all industries substantial potential for growth and development. Allowing organizations to seamlessly integrate their existing file data with Amazon S3-based services while enjoying the advanced enterprise-grade capabilities FSx for ONTAP has to offer helps them turn their data into a catalyst for faster innovation, and enhanced resilience.
