As businesses have to respond to changes with agility, IT teams are being pressured to modernize their infrastructures in a way that it can dynamically adapt to the new demands, conform to the security and governance requirements, and with a price that fits within tight budgets.
From time to time, I hear this question talking to customers during their data center modernization projects: does ONTAP support HTTP/FTP protocols natively? While this is a valid question, as the company probably already has something legacy to maintain and also because the person asking probably recognizes ONTAP as one of the best and most adopted unified storage platforms in the planet, there are some aspects of data center modernization and infrastructure consolidation that I would like to cover in this post.
People usually think about data center modernization in terms of “I need to tech refresh my platform”. This is a good way to think of it and NetApp recently announced a complete set of new hardware that aims to help customers with this modernization aspect. If you missed that announcement, you could take a look on this blog and check the latest advancements in the NetApp intelligent data infrastructure portfolio to modernize businesses' IT operations.
However, there are other disciplines that can’t be ignored during modernization and security is one of them. At-rest and in-flight encryption, mechanisms to protect against rogue administrators, using AI for real-time ransomware attack detection, and efficient data protection with a truly indelible logical air-gapped backup, are just a small set of security features that allow our customers to harden and modernize their security posture for building a data centric zero trust environment. Those are just some examples, but you can be protected from many other attack vectors using the full set of security features of NetApp ONTAP.
There are other topics that could be related to data center modernization like hybrid multicloud integration and flash adoption but for the sake of time, I will move on to infrastructure consolidation. In terms of data, storage infrastructure consolidation can present many dimensions, and protocol choice is one of the most important.
In the early 2000’s NetApp introduced the concept of unified storage, which allowed many enterprises to consolidate both NAS and SAN into a lean unified infrastructure. As things evolved, in 2020 NetApp announced the inclusion of the S3 protocol in ONTAP, which decisively granted it the title of the best unified storage in the industry. This gives users the freedom of choice to use the NAS, SAN or Object Storage protocol that best fit their workload requirements. The best thing about this: ONTAP provides the flexibility to use protocols separately or together with secure multitenancy.
With this long backdrop, let’s move to the unanswered question: does ONTAP support HTTP/FTP protocols natively? ONTAP definitely supports HTTP, as S3 fundamentally uses a specialized webserver that has some additional specifics, especially in terms of authentication, but allows for data transfer over HTTP/HTTPS. While S3 is high frequently used for ingesting data into the server (pre-signed URLs can be used for uploads instead, but that’s not common), the content can be simply consumed using standard HTTP requests. Historically, similar workflows were used for traditional HTTP servers, but instead of S3 you would probably use FTP or SCP. About FTP, first thing to think is: since everyone is considering modernization of the data center infrastructure, there is no reason to carry out unsecure and ancient protocols like FTP, while there are more modern options for file transfer/sharing out there.
FTP (File Transfer Protocol) is widely considered a poor choice for data transfer and sharing due to its numerous vulnerabilities. The protocol’s outdated security measures and inability to handle modern security requirements make it an unreliable and risky option for today’s data transfer and sharing needs. NetApp ONTAP S3 has a wealth of features that make it a better choice. Here are some of them:
- ONTAP S3 uses AWS-standard access and secret keys to sign programmatic authenticated requests to read/write in the buckets and to set values for specific bucket or object attributes. Beginning in ONTAP 9.14.1, there is a way to expire the credentials after a certain amount of time. This can bring more security to data transfer and sharing use cases.
- If there is a need to grant temporary access to this data without requiring the previous mentioned security credentials, there is support to create a pre-signed URL and allow someone to download an object from your S3 bucket without the need to make the bucket public. Pre-signed URLs can even be used to allow someone to upload an object to your S3 bucket without giving them access to the bucket itself. These URLs are time-limited, meaning they expire after a user-defined duration. This ensures that access is only granted for a short period, enhancing security.
- Over the wire encryption using TLS (HTTPS access) either using a system-generated certificate or optionally using signed certificates from a third-party certificate authority. HTTP can be enabled, but it is not recommended.
- Native data replication using defined synchronization schedules to meet specific recovery point objectives (RPOs) that can be used for Backup and Recovery OR Disaster recovery (DR) and failover. Backup and recovery is used where the objective is to restore from the destination bucket to the source bucket with no intention of failing over to the destination bucket (supported destinations are ONTAP S3, StorageGRID or Amazon S3). DR also can be implemented where the objective is to be able to serve data to client applications from the destination bucket in the event of a disaster (destination must be ONTAP S3).
- As of ONTAP 9.14.1, support for MetroCluster is included for both unmirrored and mirrored aggregates. This allows for streamlined business continuity operations for environments where zero RPO and near-zero RTO are required.
- ONTAP S3 avoids lock-in since S3 is the standard de facto object storage protocol that permits seamless data portability across disparate environments, on-prem or in the cloud.
- Federation with LDAP and Active Directory for a centralized database to manage S3 users and groups. This can be especially helpful when you want to provide simultaneous access to data via NAS (NFS or SMB) and S3. Why would you use S3 and NAS together? Let’s move to the next point.
- When the S3 protocol is enabled in multiprotocol NAS volumes, client applications can read and write data using S3, NFS, and SMB, which opens up a variety of additional use cases. One of the most common use cases is NAS clients writing data to a volume and S3 clients reading the same data and performing specialized tasks such as analytics, business intelligence, machine learning, and optical character recognition (OCR).
- A robust authorization framework for flexible control of access to the object storage system. Policy statements are built with a structure that specify permissions to allow (or deny) a user (or group) to perform a set of actions on some resources (a bucket, an object or both) when some optional conditions apply. ${aws:username} variable can be used to simplify policy management in order to build smaller statements instead of having to specify every S3 user individually, which is extremely helpful for environments with hundreds or thousands of users.
- S3 provides a native data protection mechanism that uses a versioning approach for managing multiple versions of an object within the same bucket. ONTAP S3 adopts standard Amazon S3 object versioning to provide an efficient way to recover from accidental deletions and overwrites. This can also be used for applications that require some level of data preservation for tracking changes at the object level.
- ONTAP 9.16.1 expanded object-level point-in-time recovery capability and added support for ONTAP S3 snapshots, which brings an excellent option for creating read-only, point-in-time images of your ONTAP S3 buckets. Those can be scheduled or created on-demand. Restores are straightforward. Each S3 snapshot is presented as an S3 bucket to S3 clients, then you just have to browse and restore the content from the snapshots into the original bucket. You can restore a single object, a set of objects or the entire bucket using standard S3 client applications.
- With S3 Object Lock, object versions can be further protected to avoid accidental or malicious attempts to delete or overwrite each individual object version before the user-defined period expires. Unlike other competitive products, ONTAP does not use any proprietary API for making this possible. Conversely, it just uses some APIs from the well-known Amazon S3 standard (no legal hold support as of ONTAP 9.16.1). Again, no lock-in!
- ONTAP S3 provides ways to balance capacity utilization and data aging. Intelligent bucket lifecycle policies can be used for automatic purging of objects that have already reached a point where they won’t be used anymore and don’t need to stay for archiving purposes. These policies can be set using ONTAP’s GUI, CLI, or REST APIs and even using S3 native API calls for providing the end users with the ability to self-govern this automation.
- As of ONTAP 9.16.1, ONTAP S3 supports Cross-Origin Resource Sharing (CORS), a feature that provides secure access to bucket objects using a web browser. So, yes, definitely we support HTTP/HTTPS! CORS unlocks the full potential of your web applications.
- Still talking about security, having audit is paramount important and ONTAP S3 provides native auditing capabilities to record auditable data and management events.
- A lot of commercial and open-source S3 clients are available for manipulating your object storage resources. In addition, you can programmatically manage your buckets and objects.
- In addition of data transfer and sharing use case, there are many more applications that can take advantage of ONTAP S3. Many customers use S3 buckets for their home grown apps or to integrate with some third-party validated solutions.
Hopefully you are more comfortable now and can start modernizing your protocols. If this is not the case yet, you can still follow this receipt that NetApp’s own Justin Parisi shared some time ago to use NetApp ONTAP as the storage repository for your native FTP services.
This article covered just the tip of the iceberg for replacing data transfer and sharing architectures that are based out of unsecure and old protocols. If you want to learn more about how ONTAP can contribute to your data infrastructure modernization, reach out to us.
