Rapid instance creation with OpenStack on NetApp

Originally published on 9/14/14.


In this post, we'll show how running OpenStack on Clustered Data ONTAP enables you to rapidly create VM instances. This will not only enable fast instance creation, but also does it in a space-efficient manner by using file-based NetApp Snapshots.




NetApp offers an unified driver for OpenStack, that supports Clustered Data ONTAP, Data ONTAP (7-Mode), as well as E/EF-Series. With the OpenStack Icehouse release, the following protocols are supported:



Supported Protocols

Clustered Data ONTAP


Data ONTAP (7-mode)





NetApp's Unified Driver is included with the OpenStack Icehouse release. A complete summary on how to setup (or migrate) OpenStack on top of NetApp can be found in the OpenStack Deployment and Operations Guide. In this case, we'll focus on the NFS/pNFS solution running on Clustered Data ONTAP.


Traditional Data Flow


Let's assume a setup where we have Cinder and Glance running on the same/different host/s, both storing their files on some kind of NAS storage. When creating a new VM/Cinder Volume instance in OpenStack, the following steps happen (simplified):


  1. Nova tells Cinder to create a new Cinder Volume from a specific OS template
  2. Cinder requests the template from Glance
  3. Glance returns the template via HTTP over the network
  4. Cinder creates a Cinder Volume and writes the data back to the disk - through the network 

As we can see in the illustration, the data is transferred through the host running Glance, then through the host running Cinder, and finally back down to disk. This creates a lot of overhead on the CPU of the hosts and the network - given that all we want to do is copy a file from one location to another within (in most cases) the same storage system.




With Copy-Offloading


When using NFS or pNFS with Clustered Data ONTAP, we can leverage copy-offloading. In this case, Cinder/Glance just instruct the NetApp controller to directly copy the file from source to destination, without having to transfer any data through the network. The data will be copied directly within the storage controller itself:




This not only makes the instance/Cinder Volume create operation much faster, but also reduces CPU load on the Cinder/Glance hosts and reduces network traffic to basically no traffic at all.


Introducing: Local caching


Another feature that can be used in a NFS/pNFS setup within the NetApp's Unified Driver for Cinder is instance caching. This means, whenever we create a Cinder Volume out of an OS template, we cache it locally on the NetApp FlexVol that is going to host the Cinder Volume instance. Later, when we want to create the same OS instance again, Cinder just creates a space-efficient file clone. This clone will not take up any more space, as it will share the same blocks as the cached image - only the deltas will take up new blocks on the disk.




The cache also provides a time-to-live option, which invalidates old cache entries automatically after a specified period of time.

Given that the OS templates and OS instances share the NetApp FlexVol on the NetApp backend, we never need to use copy-offloading or caching. In this case, the NetApp driver just directly creates snapshot-based file clones when creating Cinder Volume instances!




Combining copy-offloading and instance caching for NFS/pNFS based Cinder Storage on NetApp provides a powerful setup that allows to create OS instances very quickly and in a space efficient manner. The full flowchart for how copy-offloading and instance caching works and how to set it up is documented in the OpenStack Deployment and Operations Guide.


I can't find the flowchart for the link to the OpenStack Deployment and Operations Guide that is referenced here in either the Newton or Ocata documentation.  Can you update the link?