NetApp Private Storage for AWS - Part 1 - Overview

Originally published 7/7/14.

 

This series will cover the NetApp Private Storage (NPS) for Amazon Web Services (AWS) solution. The first part will cover a high level overview of the solution, the following part will explain NetApp's POC Lab, look more deeply into use cases, and finally go through the technical details.

 

Part 1 - Overview

Part 2 - NetApp's NPS for AWS PoC Lab

Part 3 - Use cases (post coming soon)

Part 4 - Technical Details (post coming soon)

 

What is AWS?

Amazon Web Services (AWS) is a collection of cloud computing services. While AWS consists of many services, the two most-known (and probably most used) services are Amazon EC2 and Amazon S3:

 

  • Amazon Elastic Compute Cloud (EC2) provides resizable compute capacity, in the form of virtual machines.
  • Amazon Simple Storage Service (S3) provides simple storage to store and retrieve any amount of data.

By combining these two base components with the many other components in AWS, it is possible to build very powerful applications and software-defined infrastructures. Scaling to more powerful setups is fairly simple in many cases. However, in a standard setup, all services in AWS are accessed through the internet, which does not guarantee any bandwidth and/or latency.

 

Storing and subsequently analyzing e.g., business critical data in AWS may also pose an issue. Reason is that this data may not be allowed to leave the own storage hardware, e.g., for compliance with regulations from various authorities (internal or external). While it could be served through a secure VPN connection into AWS, it would be very slow, and also not very cost effective. While traffic into AWS is free in most cases, outgoing traffic is priced by the GB.

 

Introducing NetApp Private Storage for AWS

To enable low-latency and high-bandwidth connections into AWS, Amazon provides so called Direct Connect co-locations. A co-location is a data center which is located very closely to the AWS region (i.e., Amazon's data centers), and provides direct links into AWS. All AWS Direct Connect co-location facilities are operated by Amazon's partners.

 

NetApp Private Storage (NPS) for AWS is a dedicated NetApp Storage solution/controller deployed in one of the AWS Direct Connect co-location facilities. You can think of it as a NetApp controller that has a direct network link to a AWS data center (well, obviously with some routers in between). This allows to directly serve data from the NetApp controller to the EC2 instances within AWS. By doing so, the solution combines the benefits of a public cloud with the benefits of proven enterprise storage. In this case, this allows you to:

 

  • Get the flexibility, power, and scalability that AWS offers (through their pay-as-you-go model)
  • Keep the data control, data management, compliance, mobility, and value that NetApp storage offers

 

How does it work?

It needs to be said, that there are several possibilities how NPS and AWS can be used together. The following figure depicts one of the most common setups:

 

arch_overview.png

 

Data residing on the on premises NetApp Storage can be mirrored to the NetApp Storage in the Direct Connect co-location via either NetApp SnapVault or SnapMirror. This data can be transferred through a secured VPN/MPLS- connection, e.g. through the internet or through a dedicated connection to the co-location data center (if available). NetApp Mirrored/Vaulted Volumes can then be made writeable by creating NetApp FlexClones (this is especially useful in test/dev scenarios). The EC2 instances in AWS can then be connected to the NetApp Storage via CIFS, NFS, and iSCSI. FCP or FCoE are not supported at this time. In all of these cases, the EC2 instances (i.e., the virtual machines) still reside on Amazon's disks, but any other data can be served from the NetApp controller.

 

Data exchange between the co-location (e.g., the NetApp Storage) and AWS occurs via AWS Direct Connect, which provides a consistent, low-latency, and high-bandwidth network connection. AWS Direct Connect provides up to 10 Gbps connections, and multiple connections can be provisioned for more capacity. Our latency tests made in the Northern Virginia AWS region with the Equinix co-location resulted in pings less than 1-2ms.

 

Advantages

We want to point out a few advantages/use cases of the NPS for AWS hybrid cloud solution:

 

  • Much more economical - this is especially true for large and enterprise workloads (e.g., Oracle, SAP)
  • The NetApp controller in the co-location is owned by you - However, multiple vendors exist that allow you to rent a NPS solution (some offer this even by the hour or via pay-as-you-go)
  • Allows to leverage the NetApp feature set (FlexClones, de-duplication, Caching, Replication through SnapMirror/SnapVault, Compression, etc.) - this enables simple data movement between on premise and AWS though well-known technology (SnapVault and SnapMirror)
  • Allows to re-use the on premise NetApp Management tools to manage the NetApp controllers in the co-location

 

Use Cases

While we'll look into the use cases with more detail in one of the next posts, here are some of the highlights:

 

  • NPS allows to develop in the hybrid cloud, then deploy in a private stack - this makes the solution very flexible and cost effective for test/dev scenarios with the ability to later migrate the setup to on premise
  • NPS allows to build cost-efficient and performant Disaster Recovery solution
  • A multi-tier backup/recovery setup can be built on top of NPS in AWS
  • Use EC2 for bursty, seasonal, or performance intensive workloads, without having to put all the data into AWS - this simplifies meeting internal and external regulations

 

In the next posts, we'll look into the technical aspects with more care, explain how NetApp's POC Lab works (yes, you can try this out for less than $10/day), and explain the use cases with more detail.

 

Stay tuned!