If you are in the object storage or data lake space, you have probably heard about Amazon’s recent announcement of a new file system gateway for S3 object storage named Mountpoint for Amazon S3. Mountpoint provides high throughput file access to your S3 bucket of objects. Amazon is targeting the large data lake solutions used for research, analytics, and machine learning. In these cases, the data is ingested in object format, but many of the analysis tools rely on good old file access. Though there are several Linux file access solutions for object today, most have struggled to meet the demands of these workloads. Amazon is hoping to change that with Mountpoint by creating a solution that is tailor-made for these high-throughput, read-heavy applications. Unlike some of the other file gateways, Mountpoint is not POSIX compliant so it may not be suitable for all use cases. In the alpha release, the product is read-only with the plan to add support for sequential writes.
The question you probably have is, “Can Mountpoint be used with NetApp StorageGRID?” The answer is YES! StorageGRID supports the necessary APIs and the configuration of the application allows for non-Amazon endpoints.
As a test, I installed mountpoint-S3 on CentOS 8 Stream. It was a painless experience, following the instructions in the Github repository on installing the dependencies, cloning the repo, and compiling the client. Once the client is compiled, create a credentials file in /home/username/.aws/ just as you would with the AWS CLI tool.
Create the directory where your bucket would mount on and run the mountpoint command to attach the bucket. In this example the bucket name is td02, the directory I am using for my mount point is ~/mnt/, and the StorageGRID endpoint is sgdemo.netapp.com.
The object storage market is continuing to grow with its sights strongly fixed on the primary workloads dominated by filesystems. The need for file access protocols are not going quietly into the night. So, we are happy to see Amazon take on this challenge. Like many of you, I can’t wait to see how this product evolves and how we can help you integrate it into your workflows.