In part 1, we’ve seen how to consume vSphere datastores for Kubernetes persistent volume needs with ONTAP. Now, in this part, we will explore the option of consuming ONTAP directly from Kubernetes using network-based protocols such as iSCSI and NFS. From the vSphere administrator perspective, it is more like in-guest storage access.
The Container Storage Interface (CSI) that we will be using with ONTAP for various access mode requirements is Astra Trident. You need to deploy the CSI on the Kubernetes cluster. I prefer Trident Orchestrator-based deployment with helm chart, and when it needs to be deployed on multiple Kubernetes clusters, currently, I prefer to use ArgoCD ApplicationSet. Astra Control and BlueXP can also be used to deploy Astra Trident on specific use cases. For system requirements, please check here.
The worker nodes must have the required components to access the ONTAP volume with network interface card(s). The implementation details vary based on the OS used for the worker nodes. Please refer here for a sample deployment of iSCSI components using the daemon set.
Astra Trident provides the following drivers for ONTAP (including AWS FSx ONTAP). Drivers beginning with nas for the file-based protocol (NFS & SMB) access & with san for the block-based protocol (iSCSI) with dedicated ONTAP volumes. The economy driver allows sharing of the ONTAP volume across multiple persistent volumes (like how VMFS or NFS datastores are consumed in a vSphere environment). Driver ending with flexgroup creates flexgroup volumes, which can grow capacity in PB scale.
In Kubernetes, the Astra Trident driver details are stored in Custom Resource Definitions (CRD), Trident Backend Configuration (TBC) and Trident Backend (TBE). Trident backend configuration is specific for Kubernetes GitOps implementation and contains backend storage information. On successful validation, it will be bound to Trident Backend object with defaults applied.
Using ONTAP SVM credentials in TBC keeps the same backend UUID as the SVM fails over in MetroCluster Configuration or in SVM-DR. Trident backend configuration details for nas and san can be referenced on the documentation page with provided links.
Kubernetes admin can define the storage class with Trident CSI along with parameters to select the appropriate trident backend. Virtual Pools provide a layer of abstraction between Trident storage backends and Storage Classes. ONTAP NFS performance enhancement features like nconnect, session trunking, etc. can be utilized using the mount options. If you need to review sample configs, please refer here for various scenarios.
Kubernetes applications consume persistent storage using its claims. The Persistent Volume Claim typically maps to a Storage Class for dynamic provisioning. Trident CSI uses the information on Trident backend (based on storage class filters) to orchestrate the storage tasks required to present the persistent volume to a worker node.
Trident volume publication CRD shows the details of the node to which persistent volume is mounted.
To identify the persistent volume details on storage system, check backendUUID and internalName. Map backendUUID to trident backend and search for internalName on the storage system which will present the ONTAP volume/LUN/Qtree based on the driver used.
For additional details on volume operations (like importing existing data, topology aware deployment, etc.,) with Trident CSI, check here. If using topology with high availability options (like MetroCluster or SMBC or AWS FSx ONTAP with Multi-AZ), please be aware of pod topology spread constraint options.
Astra Control provides application centric data protection with low RPO and RTO using storage-based replication on ONTAP systems. It provides detailed metrics info using NetApp harvest tool and has integration to BlueXP Observability (Cloud Insights).
In summary, while deploying Kubernetes environment on vSphere with ONTAP, you can weigh in the options with VMware vSphere CSI/Astra Trident CSI. You can use combination of both based on application or operations requirements. VMware CSI supports multi-vendor storage systems (including protocols of FC, NVMe-oF, etc.). Astra Trident supports Kubernetes on many cloud infrastructures including bare metal, enables to consume ONTAP features like FlexCache to Kubernetes workloads & allows disaster recovery option with low RPO & RTO.