Tech ONTAP Articles

Realize the Full Potential of Cloud with the Data Fabric


March 2015

Jay Kidd
Chief Technology Officer, NetApp

The introduction of the IBM PC 34 years ago changed IT forever. It brought affordable computing into the hands of businesses of all sizes, and it enabled a deeper level of insight into business processes, increased efficiency, and encouraged innovation, while freeing individuals from the IT constraints of the time. Today, x86 technology represents in excess of 75% of the compute power in the enterprise.


An important lesson from the PC revolution was that the flow of new technology into the enterprise couldn’t be stopped. IT teams had to strike a balance between embracing technology to empower individuals and ensuring that operational discipline was maintained.


Like the PC before it, the cloud revolution is having a tremendous impact on enterprise IT. This article explores the opportunities and risks associated with cloud adoption, explains how a data fabric can help you take full advantage of a hybrid cloud without radically changing your management processes or retraining your IT staff, and explores what you should be doing now to become “data fabric-ready.”



Figure 1) The ability to share and manage data among clouds is key to hybrid cloud success.




Source: NetApp, 2015


Enterprise IT and the Cloud: Peril and Promise


Just five years ago, if someone had an idea for a new business application, that person had to beg for budget, purchase infrastructure and get it deployed, build a team, license software, and spend months in development and years in maintenance.


Now, a 22-year old with some knowledge of GitHub and a credit card can open up a hundred virtual machines on Amazon Web Services, download some open-source software, write a few thousand lines of Python code, load MongoDB, and create a massively scalable application—not only without permission from IT, but without IT even knowing about it. This may seem cool to some, but it’s also more than a little scary.


IT is once again faced with the challenge of embracing innovation, while maintaining the operational discipline necessary to protect the continuity of important business applications. If you ignore the cloud, shadow IT will be the result. That is, well-meaning people will develop new cloud applications—often without paying enough attention to security, data protection, or regulatory requirements. If you harness the cloud, you can transform the way your company does IT, decrease costs, and increase competiveness.



A Data Fabric Reduces the Cost of Failure, Encouraging More Innovation


Until now, it’s been hard to marshal the resources to support new application ideas. But what if you could enable innovators to get access to as much compute power as they need? And, what if you could give them an environment that includes a replica of the production data set with the same access methods and data management you use in your data center?


A data fabric accelerates development, lowers the cost of new applications, and lets you try more things. When you try more you succeed more, and when you fail you fail at lower cost.


Designing new applications in the cloud using a data fabric gives you the flexibility to choose where you run the finished application. If an application fails, you can spin everything down and you haven’t lost much. If the application takes off, you can easily move it on premises or to another cloud environment.

Data and the Cloud


For the most part, today’s clouds are isolated and consist of incompatible silos of data with different data storage models (see Figure 2). It’s difficult to move data from one cloud to another without developing an application to do so. Different clouds have different application services, and applications often have to be rewritten for each cloud. In addition, it’s important to recognize that cloud storage is fundamentally different than cloud compute. Performance characteristics are different, data sovereignty and security are important considerations, and data has gravity—moving data is not free or fast.



Figure 2) Today’s hybrid cloud consists of incompatible, isolated silos that impede the flow of data among clouds.




Source: NetApp, 2015



What’s needed is a way to manage, secure, protect, share, and move data among different clouds. Imagine a hybrid cloud where all of the data management capabilities within it are consistent, connected, and form a coherent, integrated, and compatible system—in essence a fabric woven through the cloud that joins various clouds (see Figure 3).



Figure 3) A data fabric provides efficient data transport, software-defined management, and a consistent data format, allowing data to move more easily among clouds.




Source: NetApp, 2015


NetApp Brings the Data Fabric to Life


NetApp® believes that a data fabric is key to making the hybrid cloud work for your business, today and in the future. A Data Fabric powered by NetApp delivers a degree of uniformity across disparate cloud resources, including:


  • A consistent data format so that you can share and use data across the fabric
  • Software-defined data management so that you can manage data the same way, no matter where it resides
  • Fast, efficient data transport so that you can easily move data where it needs to be


A Data Fabric powered by NetApp offers a common set of data services across the hybrid cloud:


  • Proven replication technology to move data among clouds
  • Block, file, and object storage access methods
  • Advanced data services, including snapshots, cloning, backup, disaster recovery, and more
  • Innovative data reduction and storage efficiency that enables you to store the maximum data using the smallest amount of storage space
  • Application integration that facilitates data protection and recovery
  • A common approach to managing data within and across clouds


With a unified set of services spanning multiple clouds, you no longer have to redesign your applications to run in different cloud environments. You just have to move them there.


A data fabric allows you to think about your overall hybrid cloud (your extended IT environment) as an integrated infrastructure. The decision about where to place an application is no longer limited by compatibility with a particular Platform-as-a-Service layer or technology stack—instead, it’s a matter of choosing the cloud that offers the right service level for the right price.


A Data Fabric Lets You Focus on Your Most Important Applications


Every company has lots of applications taking up space, power, and—perhaps most important—mindshare from skilled IT staff. There’s a substantial opportunity to move less-important applications off premises.


A data fabric enables you to move these applications to infrastructure you don’t own, managed by people you don’t have to hire. You can focus precious and scarce internal resources and mindshare on the applications you have to run yourself.

Data Fabric Options


NetApp offers a number of building blocks to help you create your Data Fabric. The ones you choose will depend on your requirements and goals:


  • Private cloud solutions create a foundation for the Data Fabric in your data center with NetApp FAS hardware or FlexPod integrated infrastructure running clustered Data ONTAP®. These platforms enable IT-as-a-Service within your environment and serve as an easy onramp to the hybrid cloud.
  • Hundreds of cloud service providers around the world have built their clouds with NetApp technology.
  • NetApp Private Storage lets you store your data near the cloud with high-speed, low-latency network connections to leading cloud providers. You retain full control of your data while taking advantage of cloud compute power as needed for analytics, DR, and other uses. NetApp Private Storage is available for Amazon Web Services (AWS), Microsoft Azure, and IBM SoftLayer.
  • Cloud ONTAP runs in a virtual machine in the cloud (currently available for AWS as an Amazon Machine Image or AMI), bringing all the data management and mobility features of NetApp Data ONTAP inside the cloud, so you can deploy a new instance of enterprise storage and commence working in minutes.
  • NetApp SteelStore Cloud-Integrated Storage has two modes of operation:
    • SteelStore appliances provide backup and archive to the cloud for your on-premises data with a local cache for rapid restore.
    • SteelStore runs as an AMI within AWS, allowing you to backup active data stored in the cloud to another AWS zone or to a different cloud.
  • NetApp StorageGrid® Webscale software-defined object storage enables secure, global management of unstructured data. The latest release, StorageGRID Webscale 10.1, supports Amazon S3 storage. Geo-distributed erasure coding reduces costs and increases security.
  • NetApp OnCommand® Insight lets you broker and monitor both on-premises and cloud storage with end-to-end visibility to enhance operations, increase service quality, and reduce costs.

The Cloud Will Change the Way You Run IT


If you embrace the cloud, the role of IT will change in significant ways. With the cloud, IT is no longer just about building infrastructure and running data centers; it’s about marshaling the tools and the applications to acquire, transform, apply, and protect the data on which your business depends.


As more data moves into the cloud, storage admins will increasingly take over the functions of cloud administration. We see this happening already for adopters of our SteelStore cloud-integrated appliances.


The hybrid cloud model—combining on-premises capabilities with resources and services available from various cloud providers—is poised to become the dominant model in enterprise IT, and a data fabric will enable your organization to take best advantage of this model.

Is Your Infrastructure Data Fabric-Ready?


A data fabric gives you more flexibility to choose where you run each workload. For most, this means running the workloads you absolutely must control on-premises, while migrating less-important applications to the cloud over time. As this happens, the way that you invest in IT will change. Your priorities will be focused on technologies that allow you to implement and fully utilize a data fabric, and on the performance and availability of on-premises infrastructure.


Deploying high-performance storage in the form of all-flash arrays and hybrid flash arrays in your data centers serves as a counterbalance to the cheap capacity and relatively slow speed of storage in the cloud. You can achieve several orders of magnitude better performance on-premises than you can in the cloud. All-flash storage allows you to accelerate applications up to 20x versus disks alone, and the ability to consolidate servers and licenses can provide significant ROI. (To learn more about the advantages of flash for your data center, see Flash Delivers Storage Performance for an Impatient World.)


Clustered Data ONTAP is one of the foundational technologies for a Data Fabric powered by NetApp. Clustered Data ONTAP offers:


  • Performance. The performance of both all flash and hybrid storage have been measured and proven using industry benchmarks.
  • Availability. User access to applications is protected with nondisruptive operations; flexible options are available for disaster recovery and continuous data availability.
  • Flexibility. Storage workloads can move dynamically to all-flash storage when performance is needed without disrupting applications that are currently running.
  • Multitenancy. Separate applications or clients can share the same storage; QoS ensures that storage resources are allocated by priority.
  • Software-defined management and automation. OnCommand tools included with clustered Data ONTAP provide storage orchestration that encapsulates best practices for more consistent results.
  • Monitoring. Optional monitoring software lets you keep tabs on all of your data across the hybrid cloud.


If you want to realize the full potential of your data in the hybrid cloud, upgrading to clustered Data ONTAP should be part of your roadmap.


The revolution in cloud-centric IT technology is just beginning. NetApp provides the technology to underpin your Data Fabric, enabling your business to move faster and giving you the freedom to pursue ideas that were not feasible before.



As NetApp’s CTO, Jay Kidd is responsible for the company’s overall technology direction, architecture, advanced development, as well as technology evangelism and standards. He has extensive cross-functional experience in storage, networking, software, and high-performance systems. Prior to becoming CTO, Jay was senior vice president of NetApp and was responsible for the engineering of NAS, SAN, data protection, and management products. Before joining the company, he was CTO and vice president of Product Management at Brocade.

Please Note:

All content posted on the NetApp Community is publicly searchable and viewable. Participation in the NetApp Community is voluntary.

In accordance with our Code of Conduct and Community Terms of Use, DO NOT post or attach the following:

  • Software files (compressed or uncompressed)
  • Files that require an End User License Agreement (EULA)
  • Confidential information
  • Personal data you do not want publicly available
  • Another’s personally identifiable information (PII)
  • Copyrighted materials without the permission of the copyright owner

Continued non-compliance may result in NetApp Community account restrictions or termination.

All Community Forums