Authors: Banu Sundhar, NetApp
Mayur Shetty, Red Hat
We are often faced with innovative technologies that challenge the existing paradigms. OpenShift Virtualization, built on the upstream KubeVirt project, is one such technology.
OpenShift Virtualization allows organizations to run traditional Virtual Machines (VMs) alongside containerized applications in a unified platform. This is particularly valuable for enterprises with legacy applications that need modernization without a complete overhaul. Using Kubernetes as a control plane provides significant benefits, such as scalability, self-healing, and management consistency. For organizations already invested in OpenShift, this integration simplifies operations, as teams can leverage existing tools and workflows.
OpenShift Virtualization is included with OpenShift at no additional licensing or subscription cost. On AWS, it runs on Red Hat OpenShift Service on AWS (ROSA), leveraging its orchestration capabilities to manage VMs alongside containerized applications. This integration allows teams to utilize the same management tools and practices for both VMs and containers, streamlining operations and enhancing efficiency.
The approach of running VMs on a container orchestrator addresses the growing need for flexibility in application deployment. Organizations can choose to run both VMs and containers based on specific workload requirements, without the need to manage separate infrastructure environments.
Amazon FSx for NetApp ONTAP (FSxN) provides fully managed shared storage in AWS Cloud with the popular data access and management capabilities of ONTAP. One key benefit of FSxN is that it is a true first party AWS offering just like EBS, meaning customers can retire their committed spend with AWS and get support directly from them as well. FSxN can be consumed by applications and VMs running on OpenShift using NetApp Trident as the storage orchestrator.
Solution overview
Below we will look into setting up FSxN as the default Storage Class for the ROSA cluster, and then create a VM that will leverage FSxN storage for its volumes. We will also look into connecting to the VM using the guest credentials and follow that by performing a live migration of the VM from the current node to a new node.
Prerequisites:
- AWS account
- A Red Hat account
- IAM user with appropriate permissions to create and access ROSA cluster
- AWS CLI
- ROSA CLI
- OpenShift command-line interface (oc)
- Helm 3 documentation
- A HCP ROSA cluster (with at least 2 bare-metal worker nodes)
- OpenShift Virtualization installed on ROSA Cluster
- Access to Red Hat OpenShift web console
This diagram shows the ROSA cluster deployed in multiple AZs. ROSA cluster’s master nodes, infrastructure nodes are in Red Hat’s VPC, while the worker nodes are in a VPC in the customer's account. Some of the worker nodes are metal instance type to support OpenShift Virtualization. FSxN file system is deployed within the same VPC. NetApp Trident provisioner is installed in the ROSA cluster, allowing all the subnets of this VPC to connect to the file system. OpenShift Virtualization is installed using the operator from the OperatorHub.
1. Initial Setup
Ensure default Storage Class is set to trident-csi.
The following yaml was used to create the storage class
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: trident-csi
provisioner: csi.trident.netapp.io
parameters:
backendType: "ontap-nas"
fsType: "ext4"
allowVolumeExpansion: True
reclaimPolicy: Retain
Note: Prior to creating the Storage class, the following yaml files were used to create a secret and the backend object using ontap-nas driver.
apiVersion: v1
kind: Secret
metadata:
name: backend-fsx-ontap-nas-secret
namespace: trident
type: Opaque
stringData:
username: vsadmin
password: <SVM password configured while creating FSxN>
apiVersion: trident.netapp.io/v1
kind: TridentBackendConfig
metadata:
name: backend-fsx-ontap-nas
namespace: trident
spec:
version: 1
backendName: fsx-ontap
storageDriverName: ontap-nas
managementLIF: <Management DNS name>
dataLIF: <NFS DNS name>
svm: <SVM NAME>
credentials:
name: backend-fsx-ontap-nas-secret
Ensure default VolumeSnapShotClasses is set as shown
The following yaml was used to create the VolumeSnapshotClass
kind: VolumeSnapshotClass
metadata:
name: fsx-snapclass
driver: csi.trident.netapp.io
deletionPolicy: Delete
If the defaults are not set, you can set it up either from the console or from command line
$ oc patch storageclass trident-csi -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "true"}}}'
$ oc patch VolumeSnapshotClasses fsx-snapclass -p '{"metadata": {"annotations": {"snapshot.storage.kubernetes.io/is-default-class": "true"}}}'
1. Create a VM from the template
Use the web console to create a VM from a template.
From the RedHat OpenShift Service on AWS console, create a VM.
There are templates available on the cluster that can be used to create the VM.
I am choosing fedora VM template from this list. Give the VM a name, and then click on Customize Virtual Machine. Select the Disks tab and click on Add disks. Change the name of the disk, preferably to something meaningful, ensure that trident-csi is selected for storage class. Click on Save. Click on Create VirtualMachine.
After a few minutes, the VM is in the running state
2. Review all the objects created for the VM
The storage disks.
The file systems of the VM will show the Partitions, File system type and the Mount points.
2 PVCs are created for the VM, one from the boot disk and one for the hot plug disk.
The PVC for the boot disk shows that the Access mode is ReadWriteMany, and the Storage Class is trident-csi.
Similarly, the PVC for the hot-plug disk shows that the Access mode is ReadWriteMany, and the Storage Class is trident-csi.
In the screenshot below we can see that the pod for the VM has a Status of Running.
Here we can see the two Volumes associated with the VM pod and the 2 PVCs associated with them.
Issue the following commands:
$ df (to display information about the disk space usage on a file system).
$ dd if=/dev/urandom of=random.dat bs=1M count=10240 (to create a file called random.dat in the home dir and fill it with random data).
The disk is filled with 11 GB of data.
Use vi to create a sample text file that we will use to test.
4. Restart the virtual machine
The VM comes back to the running state with the exact same filesystems, PVCs and files in the filesystems
5. Live migration
Live migration refers to the process of moving a running VM from one physical host to another host without disrupting normal operations or causing any downtime, or other adverse effects for the end user.
Live migration is considered a major step in Virtualization. It allows an entire VM to be moved with a running operating system (OS), Memory, storage, and network connectivity from their current node to the destination.
Below we will see how to perform a Live Migration of the VM from the current node to a new node.
Note the node on which the VM is running.
On the Overview page, you can see that the migration has succeeded, and the Status has changed to Succeeded.
After the completion of the Live Migration, the VM is now on a different node.
Open the web console and view the contents of the disks. It still has the same 2 files that we previously created before Live Migration.
The storage for the VM on the new node still shows the same disks.
Also, the PVCs are the same.
Volumes associated with the VM pod are also the same (2 PVCs) as before.
Conclusion:
Combining OpenShift Virtualization with FSx for NetApp ONTAP creates a powerful solution for organizations seeking to optimize their hybrid cloud environments. OpenShift Virtualization allows users to manage both VMs and containerized applications within a unified Kubernetes platform, enhancing operational efficiency and flexibility. Meanwhile, FSx for NetApp ONTAP provides scalable, high-performance storage that seamlessly integrates with OpenShift, ensuring that data is readily accessible and protected.
This combination empowers businesses to modernize their applications while efficiently managing legacy workloads. With the ability to run diverse workloads in a single environment, teams can streamline their operations, reduce complexity, and improve resource utilization. Together, OpenShift Virtualization and FSx for NetApp ONTAP deliver a robust infrastructure that meets the demands of today's dynamic enterprise landscape, enabling innovation and growth.
For more information about Trident, refer to the NetApp Trident documentation. See the Red Hat OpenShift Virtualization section in the NetApp Solutions documentation for additional information and videos. If you’d like to clean up the solution set up in this post, follow the instructions in the github repository.
