Introduction
In today's digital landscape, Kubernetes has become the de facto standard for container orchestration and application deployment. With its scalability and flexibility, it offers numerous benefits for managing and scaling applications. However, as organizations rely more heavily on Kubernetes for critical workloads, it becomes crucial to have a robust data protection strategy in place.
NetApp® Astra™ Control provides advanced data management capabilities that enhance the functionality and availability of Kubernetes applications. Astra Control simplifies the management, protection, and movement of containerized workloads across public clouds and on-premises environments. It also offers automation capabilities through its REST API and SDK, enabling programmatic access for seamless integration with existing workflows.
With several software releases in the first half of 2024, Astra Control is launching a new Kubernetes-native architecture, enabling numerous new data protection workflows while staying backward compatible with the existing API and SDK. By seamlessly integrating with Kubernetes APIs and resources, data protection can become an inherent part of the application lifecycle through an organization’s existing continuous integration and continuous deployment (CI/CD) and/or GitOps tools.
In this blog, we’ll show how to manage a Kubernetes cluster with this new architecture, and we’ll protect an application running on the cluster by running several kubectl commands. We’ll also dive into the structure and format of these commands, and then we’ll wrap everything up by providing some additional resources and areas to investigate to learn even more about the new Astra Control Kubernetes-native architecture.
Prerequisites
If you plan to follow this blog step by step, you need to have the following available:
- An Astra Control Center (ACC; self-managed) or Astra Control Service (ACS; NetApp managed service) instance
- An unmanaged Kubernetes cluster and its associated kubeconfig
- A workstation with kubectl configured to use the kubeconfig
- A workstation with Helm installed (or a different means of deploying a sample Kubernetes application)
Cluster management
The new Kubernetes custom resource (CR) driven architecture requires installing some components on the cluster: Kubernetes custom resource definitions (CRDs) and the Astra Connector Operator, which facilitates communication between the Kubernetes cluster and Astra Control. Fortunately, this is a straightforward process that is handled during cluster management. (An upcoming Astra Control release will provide the ability for a cluster already under management to move to the new architecture.)
To get started, head over to the Astra Control UI, select Clusters in the left column, and then click Add.
On the first page of the Add Cluster wizard, fill out the following fields, and then click Next:
- Leave the “Add manually using command line” checkbox selected. (This checkbox instructs Astra Control which architecture to use, so deselecting it would fall back to the older architecture.)
- Enter a descriptive and identifiable cluster name.
- Select the cloud instance this cluster will belong to, or optionally add a new cloud instance.
On the second page of the Add Cluster wizard, click the Generate button to automatically create an API token, or optionally paste an existing API token. (This token provides authorization from your Kubernetes cluster to Astra Control.) You can also manually create (or rotate) these tokens by clicking the user icon in the upper right and then selecting API Access.
Click the Copy icon of the CLI command in step 1 (Install Astra Connector Operator), head over to your terminal, paste in the command, and press Enter. You should see messages about custom resource definitions and other components being successfully installed.
$ kubectl apply -f https://github.com/NetApp/astra-connector-operator/releases/download/202403131314-main/astraconnector_operator.yaml
namespace/astra-connector created
customresourcedefinition.apiextensions.k8s.io/applications.astra.netapp.io configured
customresourcedefinition.apiextensions.k8s.io/appmirrorrelationships.astra.netapp.io configured
customresourcedefinition.apiextensions.k8s.io/appmirrorupdates.astra.netapp.io configured
customresourcedefinition.apiextensions.k8s.io/appvaults.astra.netapp.io configured
...
rolebinding.rbac.authorization.k8s.io/operator-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/operator-manager-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/operator-proxy-rolebinding unchanged
configmap/operator-manager-config created
service/operator-controller-manager-metrics-service created
deployment.apps/operator-controller-manager created
Repeat the same process for step 2 (Create the Astra API Secret). This step creates a Kubernetes secret that references the Astra API token generated (or pasted) at the top of the wizard.
$ kubectl create secret generic astra-api-token --from-literal=apiToken=LeBlVjDmdj6fh5bamLeyOGdt1pR4qCHtWt6s4HJI6ZM= -n astra-connector
secret/astra-api-token created
Repeat the same process for step 3 (Create the Docker Registry Secret). This step creates a Kubernetes secret to enable the pulling of necessary container images for the Astra Connector.
$ kubectl create secret docker-registry astra-connector-regcred --docker-username=7613e41e-9c17-479a-bd2e-c6de7f978f93 --docker-password=LeBlVjDmdj6fh5bamLeyOGdt1pR4qCHtWt6s4HJI6ZM= -n astra-connector --docker-server=cr.astra.netapp.io
secret/astra-connector-regcred created
Finally, repeat the same process for step 4 (Apply the Astra Connector CR). This step creates the Astra Connector custom resource, which is responsible for secure communication between the Kubernetes cluster and Astra Control.
$ kubectl apply -f - <<EOF
apiVersion: astra.netapp.io/v1
kind: AstraConnector
metadata:
name: astra-connector
namespace: astra-connector
spec:
astra:
accountId: 7613e41e-9c17-479a-bd2e-c6de7f978f93
tokenRef: astra-api-token
clusterId: c189f0ee-07b2-4c55-a65e-8fa156b081c4
skipTLSValidation: false
clusterName: dev-ol-astra-enterprise-4
imageRegistry:
name: cr.astra.netapp.io
secret: astra-connector-regcred
natsSyncClient:
cloudBridgeURL: https://10.193.126.216
autoSupport:
enrolled: true
EOF
astraconnector.astra.netapp.io/astra-connector created
In the UI, optionally copy the generated API token to a secure location (it cannot be shown again), and then click Exit. You should see your Kubernetes cluster in a pending state, waiting for the completion of the connector setup.
In your terminal, run the following command to view the status of the Astra Connector setup.
kubectl -n astra-connector get astraconnector
For the first several minutes, you should see various statuses and the Registered field as false.
$ kubectl -n astra-connector get astraconnector
NAME REGISTERED ASTRACLUSTERID ASTRACONNECTORID STATUS
astra-connector false Creating Service astra-connector/nats-cluster
After 3 to 5 minutes, the Registered field should switch to true.
$ kubectl -n astra-connector get astraconnector
NAME REGISTERED ASTRACLUSTERID ASTRACONNECTORID STATUS
astra-connector true 72b4a626-a45d-4540-8f3c-34d9a65d578c 95f1626c-3939-4869-a657-6997e6bef9ef Registered with Astra
Head back to the UI, where the cluster should be shown as Available.
A default bucket is now required for each managed cluster under the new architecture, because buckets are now used to store application (and snapshot) metadata in addition to backup data. By default, the cluster inherits the associated cloud’s default bucket, but this can be overridden with a cluster-specific bucket. Click the Edit button under the Default Bucket heading.
Select the object storage bucket of your choice and click Save. If you don’t have a bucket already defined, select Buckets in the left column and follow these instructions (for ACC or for ACS) to add a bucket.
The cluster is now ready to use, whether through existing means like the UI or API, or through Kubernetes custom resource driven actions.
App management
Now that our cluster has all the necessary software components installed, and it’s registered to our Astra Control instance, we’re ready to manage an application. We’ll use Helm to first deploy a sample app. However if your cluster already has a test application running on it, feel free to skip this step. In your workstation terminal, run the following command.
helm install wordpress -n wordpress --create-namespace bitnami/wordpress
Once it’s deployed, we’re ready to manage our application, which we’ll do with this kubectl apply command.
Note: It is not required to interact with the new Astra Control architecture in this manner; it’s just another option to the preexisting (and unchanging) UI and API.
kubectl apply -f - <<EOF
apiVersion: astra.netapp.io/v1
kind: Application
metadata:
name: wordpress
namespace: astra-connector
spec:
includedNamespaces:
- namespace: wordpress
EOF
Let’s dig into this custom resource a bit more by section.
apiVersion: astra.netapp.io/v1
kind: Application
If you’re already familiar with Kubernetes custom resources, this should be straightforward. The apiVersion field points to the specific version of API that the resource adheres to. In this case it’s an extension of Kubernetes API (as are all custom resource definitions). The kind specifies the type of resource under that API definition.
metadata:
name: wordpress
namespace: astra-connector
As with all Kubernetes resources, there’s a top-level metadata field, with the name of a custom resource and the namespace to which the custom resource is scoped.
Note: Custom resource definitions can be either cluster or namespaced-scoped. However all Astra Control CRDs are namespaced-scoped, and all should live within the astra-connector namespace.
spec:
includedNamespaces:
- namespace: wordpress
Finally, the spec contains the specifics of the application definition. This is a simple application with only a single namespace, but it’s possible for this specification to include any number of namespaces, label selectors, and cluster scoped resources.
Head back into the Astra Control UI and click Applications in the left pane. You should see the wordpress application in an available state.
Click on the wordpress app and investigate the application user interface. If you’re familiar with Astra Control, you should notice that the application looks like any other application from previous versions.
Now that our application is defined, let’s perform some application data management operations.
Snapshot creation
Our application has been successfully managed by creating a custom resource on the same cluster that the application lives on. We can now carry out any other application data management operation in the same manner, and we’ll start with a snapshot.
In your terminal, execute the following command.
kubectl -n astra-connector get appvaults
You should see the bucket that was set as the default bucket at the end of the cluster management step, with an age of when the application was defined in the previous step. You may wonder how or why this bucket (or application vault, appVault for short) was defined on the cluster.
Any time an application data management action (application definition, snapshot, backup, restore, etc.) is carried out on the cluster, the corresponding bucket is automatically defined as an appVault custom resource. Many application data management actions specifically reference a bucket/appVault (as we’ll see in a moment with our snapshot definition). However, those do not rely on the default bucket set at the cloud- and/or cluster-level (cluster taking precedence over cloud).
When our application was defined, a bucket was needed to store the application’s state, including application assets, backups, and snapshots. Therefore the bucket that was set as the default in the cluster management section was created by Astra Control as an appVault custom resource on our managed cluster. This appVault is then used to store necessary information about the application, enabling “self-contained” applications (or snapshots, or backups) that do not require an operational Astra Control instance to be used.
Since your bucket name is unique, run the following command to store the name of the appVault as an environment variable to be used in the next step.
appVault=$(kubectl -n astra-connector get appvaults | grep -v NAME | awk '{print $1}')
Finally, let’s create a snapshot by running the following command.
kubectl apply -f - <<EOF
apiVersion: astra.netapp.io/v1
kind: Snapshot
metadata:
name: wordpress-snapshot-1
namespace: astra-connector
spec:
applicationRef: wordpress
appVaultRef: $appVault
EOF
Let’s investigate the fields that are different from the application custom resource previously inspected.
apiVersion: astra.netapp.io/v1
kind: Snapshot
Although the apiVersion field is the same, the kind is unsurprisingly different. Rather than an “application” definition, this is a “snapshot” definition.
spec:
applicationRef: wordpress
appVaultRef: $appVault
The Spec field contains two references, one to the application we previously defined and one to the appVault that we just discussed. These references are the core of the instruction to Astra Control: take a snapshot of this application and store the application metadata in this appVault (or bucket).
Note: even though the application metadata is stored in an external bucket for a snapshot, the Kubernetes persistent volumes are still stored locally on the cluster through a CSI volume snapshot. This means that if the namespace or cluster is destroyed, so will the application snapshot data. This is the key difference between a snapshot and a backup, where the volume snapshot data is also stored on the referenced bucket.
Let’s make sure that our snapshot completed successfully, through both the CLI and the UI. First, run the following command.
$ kubectl -n astra-connector get snapshots
NAME STATE ERROR AGE
wordpress-snapshot-1 Completed 2m37s
Then head over to the UI, select our wordpress application, and view the Snapshots section of the Data Protection tab.
Both the CLI and the UI validate that we have successfully taken an application snapshot from our custom resource definition. Next, let’s make this snapshot more robust in case of a disaster.
Backup creation
As mentioned in the previous section, if our wordpress namespace or managed cluster is destroyed, we will lose our application’s persistent volumes. Let’s change that by applying the following custom resource to create a backup.
kubectl apply -f - <<EOF
apiVersion: astra.netapp.io/v1
kind: Backup
metadata:
name: wordpress-backup-1
namespace: astra-connector
spec:
applicationRef: wordpress
appVaultRef: $appVault
snapshotRef: wordpress-snapshot-1
EOF
Again, we see the same apiVersion field, but as we expected, a different kind (Backup). Let’s further inspect the Spec section.
spec:
applicationRef: wordpress
appVaultRef: $appVault
snapshotRef: wordpress-snapshot-1
We see the same application and appVault references as our snapshot custom resource, but we also see a new snapshot reference field. This is an optional entry for a backup. If it is not specified, a new snapshot will first be created. When the snapshot is complete, the CSI volume snapshot data is copied to the referenced appVault.
Let’s examine this in a bit more detail with the following command. (If you don’t have yq installed on your workstation, just omit the rest of the command starting with the pipe, and find the Conditions part of the Status section.)
$ kubectl -n astra-connector get backup wordpress-backup-1 -o yaml | yq '.status.conditions'
- lastTransitionTime: "2024-03-21T15:15:06Z"
message: Successfully reconciled
reason: Done
status: "True"
type: AppOwnerReferenceCreated
- lastTransitionTime: "2024-03-21T15:15:07Z"
message: Successfully reconciled
reason: Done
status: "True"
type: SourceSnapshotExists
- lastTransitionTime: "2024-03-21T15:15:07Z"
message: Successfully reconciled
reason: Done
status: "True"
type: SourceSnapshotCompleted
- lastTransitionTime: "2024-03-21T15:15:07Z"
message: Successfully reconciled
reason: Done
status: "True"
type: SnapshotAppArchiveCopied
- lastTransitionTime: "2024-03-21T15:15:07Z"
message: Successfully reconciled
reason: Done
status: "True"
type: PreBackupExecHooksRunCompleted
- lastTransitionTime: "2024-03-21T15:15:46Z"
message: Successfully reconciled
reason: Done
status: "True"
type: VolumeBackupsCompleted
- lastTransitionTime: "2024-03-21T15:15:46Z"
message: Successfully reconciled
reason: Done
status: "True"
type: PostBackupExecHooksRunCompleted
- lastTransitionTime: "2024-03-21T15:15:46Z"
message: Successfully reconciled
reason: Done
status: "True"
type: TemporarySnapshotCleanedUp
- lastTransitionTime: "2024-03-21T15:15:46Z"
message: Successfully reconciled
reason: Done
status: "True"
type: Completed
- lastTransitionTime: "2024-03-21T15:15:06Z"
message: Not yet reconciled
reason: Pending
status: Unknown
type: OnFailurePostBackupExecHooksRunCompleted
Depending on when you ran the above command, you may see more or fewer entries with a reason of Done. However, the Type field should be consistent with your output. Going through each entry in the list, you can see that the second and third entries are related to the underlying snapshot. The remaining entries are related to copying the volume snapshot to the bucket, various execution hooks, and cleaning up the temporary snapshot (if applicable; in this example it is not).
Note: This command is a good way to investigate the status of any long-running backup operations. It’s easy to check which step in the process the operation is currently at and make sure that everything is progressing as expected.
Finally, head back into the UI and verify that the backup has completed successfully.
Next steps
Although manually applying YAML via kubectl is great for a demonstration, a blog post, or other one-off actions, it’s not how most organizations prefer to perform application data management operations. Instead, these operations are typically automated through CI/CD platforms and GitOps tools, minimizing the likelihood of human error.
The new NetApp Astra Control architecture enables seamless integration between these tools and application data management. Deploying a new application through an automated pipeline? Simply add a final pipeline step that applies an applications.astra.netapp.io custom resource to the same cluster. Using a GitOps tool like Argo CD to update the application through a git push? Add a presync resource hook to the git repo that applies a backups.astra.netapp.io custom resource. These tools already have access to the application’s Kubernetes cluster, so applying one additional custom resource is a trivial step.
If you’re looking for more information about the available Astra Control custom resources, here are a handful of commands that you may find useful.
kubectl get crds | grep astra # get all Astra Control CRDs
for i in `kubectl get crds | grep astra | awk '{print $1}'`; do echo "=== $i ==="; kubectl -n astra-connector get $i; done # Get all Astra CRs grouped by CRD
kubectl describe crd <CRD-name> # Get information about the 'spec' fields that are available for <CRD-name>
Finally, if you’re currently an Astra Control SDK and Toolkit user, be sure to check out the newly added --v3 flag that enables CR-driven workflows.
$ actoolkit -h | tail -14
v3 group:
use CR-driven Kubernetes workflows rather than the Astra Control API
--v3 create a v3 CR directly on the Kubernetes cluster
(defaults to current context, but optionally specify a
different context, kubeconfig_file, or
context@kubeconfig_file mapping)
--dry-run {client,server}
client: output YAML to standard out; server: submit
request without persisting the resource
--insecure-skip-tls-verify
If specified, the server's certificate will not be
checked for validity (this will make your HTTPS
connections insecure)
$ actoolkit --v3 --dry-run=client restore wordpress-backup-1 wordpress-copy gke-dr-cluster
apiVersion: astra.netapp.io/v1
kind: BackupRestore
metadata:
name: backuprestore-84d2e86f-b50e-475e-9169-b4dc2105e6df
namespace: astra-connector
spec:
appArchivePath: wordpress_5b625745-b841-4260-bff6-77abdcee3515/backups/wordpress-backup-1_1650a644-bb18-4048-ba48-15e04d161b55
appVaultRef: ontap-s3-astra-bucket8
namespaceMapping:
- destination: wordpress-copy
source: wordpress
---
apiVersion: astra.netapp.io/v1
kind: Application
metadata:
name: wordpress-copy
namespace: astra-connector
spec:
includedNamespaces:
- labelSelector: {}
namespace: wordpress-copy
$ actoolkit --v3 restore wordpress-backup-1 wordpress-copy gke-dr-cluster
{"apiVersion": "astra.netapp.io/v1", "kind": "BackupRestore", "metadata": {"creationTimestamp": "2024-03-21T21:41:35Z", "generation": 1, "managedFields": [{"apiVersion": "astra.netapp.io/v1", "fieldsType": "FieldsV1", "fieldsV1": {"f:spec": {".": {}, "f:appArchivePath": {}, "f:appVaultRef": {}, "f:namespaceMapping": {}}}, "manager": "OpenAPI-Generator", "operation": "Update", "time": "2024-03-21T21:41:35Z"}], "name": "backuprestore-54b136e8-a87c-4ebb-adfc-1dc071baf064", "namespace": "astra-connector", "resourceVersion": "71352498", "uid": "99e20297-1efe-4fa7-8d83-7f6490156274"}, "spec": {"appArchivePath": "wordpress_5b625745-b841-4260-bff6-77abdcee3515/backups/wordpress-backup-1_1650a644-bb18-4048-ba48-15e04d161b55", "appVaultRef": "ontap-s3-astra-bucket8", "namespaceMapping": [{"destination": "wordpress-copy", "source": "wordpress"}]}}
{"apiVersion": "astra.netapp.io/v1", "kind": "Application", "metadata": {"creationTimestamp": "2024-03-21T21:41:35Z", "generation": 1, "managedFields": [{"apiVersion": "astra.netapp.io/v1", "fieldsType": "FieldsV1", "fieldsV1": {"f:spec": {".": {}, "f:includedNamespaces": {}}}, "manager": "OpenAPI-Generator", "operation": "Update", "time": "2024-03-21T21:41:35Z"}], "name": "wordpress-copy", "namespace": "astra-connector", "resourceVersion": "71352507", "uid": "5c3d7050-0b22-4d3a-8532-6e1e5a971852"}, "spec": {"includedNamespaces": [{"labelSelector": {}, "namespace": "wordpress-copy"}]}}
Conclusion
In summary, we managed a Kubernetes cluster with NetApp Astra Control by creating several custom resource definitions (CRDs) and an Astra Connector custom resource. We then managed a demo application by creating an “application” custom resource via kubectl, and then protected the application by creating snapshot and backup custom resources. We then wrapped everything up by providing some additional resources and steps that you can take to learn more about the new architecture.
Thank you for making it to the end of this blog, and we hope you found the content useful!