Tech ONTAP Blogs

Kubernetes-driven data management: The new era with NetApp Astra Control

MichaelHaigh
NetApp
741 Views

Introduction

 

In today's digital landscape, Kubernetes has become the de facto standard for container orchestration and application deployment. With its scalability and flexibility, it offers numerous benefits for managing and scaling applications. However, as organizations rely more heavily on Kubernetes for critical workloads, it becomes crucial to have a robust data protection strategy in place. 

 

NetApp® Astra™ Control provides advanced data management capabilities that enhance the functionality and availability of Kubernetes applications. Astra Control simplifies the management, protection, and movement of containerized workloads across public clouds and on-premises environments. It also offers automation capabilities through its REST API and SDK, enabling programmatic access for seamless integration with existing workflows. 

 

With several software releases in the first half of 2024, Astra Control is launching a new Kubernetes-native architecture, enabling numerous new data protection workflows while staying backward compatible with the existing API and SDK. By seamlessly integrating with Kubernetes APIs and resources, data protection can become an inherent part of the application lifecycle through an organization’s existing continuous integration and continuous deployment (CI/CD) and/or GitOps tools. 

 

In this blog, we’ll show how to manage a Kubernetes cluster with this new architecture, and we’ll protect an application running on the cluster by running several kubectl commands. We’ll also dive into the structure and format of these commands, and then we’ll wrap everything up by providing some additional resources and areas to investigate to learn even more about the new Astra Control Kubernetes-native architecture.

 

Prerequisites

 

If you plan to follow this blog step by step, you need to have the following available:

 

  • An Astra Control Center (ACC; self-managed) or Astra Control Service (ACS; NetApp managed service) instance 
  • An unmanaged Kubernetes cluster and its associated kubeconfig 
  • A workstation with kubectl configured to use the kubeconfig 
  • A workstation with Helm installed (or a different means of deploying a sample Kubernetes application)

 

Cluster management

 

The new Kubernetes custom resource (CR) driven architecture requires installing some components on the cluster: Kubernetes custom resource definitions (CRDs) and the Astra Connector Operator, which facilitates communication between the Kubernetes cluster and Astra Control. Fortunately, this is a straightforward process that is handled during cluster management. (An upcoming Astra Control release will provide the ability for a cluster already under management to move to the new architecture.) 

 

To get started, head over to the Astra Control UI, select Clusters in the left column, and then click Add. 

 

MichaelHaigh_0-1712341454445.png

 

On the first page of the Add Cluster wizard, fill out the following fields, and then click Next: 

 

  • Leave the “Add manually using command line” checkbox selected. (This checkbox instructs Astra Control which architecture to use, so deselecting it would fall back to the older architecture.) 
  • Enter a descriptive and identifiable cluster name. 
  • Select the cloud instance this cluster will belong to, or optionally add a new cloud instance. 

 

MichaelHaigh_1-1712341454446.png

 

On the second page of the Add Cluster wizard, click the Generate button to automatically create an API token, or optionally paste an existing API token. (This token provides authorization from your Kubernetes cluster to Astra Control.) You can also manually create (or rotate) these tokens by clicking the user icon in the upper right and then selecting API Access. 

 

MichaelHaigh_2-1712341454447.png

 

Click the Copy icon of the CLI command in step 1 (Install Astra Connector Operator), head over to your terminal, paste in the command, and press Enter. You should see messages about custom resource definitions and other components being successfully installed. 

 

MichaelHaigh_3-1712341454449.png

 

$ kubectl apply -f https://github.com/NetApp/astra-connector-operator/releases/download/202403131314-main/astraconnector_operator.yaml
namespace/astra-connector created 
customresourcedefinition.apiextensions.k8s.io/applications.astra.netapp.io configured
customresourcedefinition.apiextensions.k8s.io/appmirrorrelationships.astra.netapp.io configured
customresourcedefinition.apiextensions.k8s.io/appmirrorupdates.astra.netapp.io configured
customresourcedefinition.apiextensions.k8s.io/appvaults.astra.netapp.io configured
...
rolebinding.rbac.authorization.k8s.io/operator-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/operator-manager-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/operator-proxy-rolebinding unchanged
configmap/operator-manager-config created
service/operator-controller-manager-metrics-service created
deployment.apps/operator-controller-manager created

 

Repeat the same process for step 2 (Create the Astra API Secret). This step creates a Kubernetes secret that references the Astra API token generated (or pasted) at the top of the wizard. 

 

MichaelHaigh_0-1712341679038.png

 

$ kubectl create secret generic astra-api-token --from-literal=apiToken=LeBlVjDmdj6fh5bamLeyOGdt1pR4qCHtWt6s4HJI6ZM= -n astra-connector
secret/astra-api-token created

 

Repeat the same process for step 3 (Create the Docker Registry Secret). This step creates a Kubernetes secret to enable the pulling of necessary container images for the Astra Connector. 

 

MichaelHaigh_0-1712342256342.png

 

$ kubectl create secret docker-registry astra-connector-regcred --docker-username=7613e41e-9c17-479a-bd2e-c6de7f978f93 --docker-password=LeBlVjDmdj6fh5bamLeyOGdt1pR4qCHtWt6s4HJI6ZM= -n astra-connector --docker-server=cr.astra.netapp.io
secret/astra-connector-regcred created

 

Finally, repeat the same process for step 4 (Apply the Astra Connector CR). This step creates the Astra Connector custom resource, which is responsible for secure communication between the Kubernetes cluster and Astra Control.

 

MichaelHaigh_1-1712342320298.png

 

$ kubectl apply -f - <<EOF
apiVersion: astra.netapp.io/v1
kind: AstraConnector
metadata:
        name: astra-connector
        namespace: astra-connector
spec:
    astra:
        accountId: 7613e41e-9c17-479a-bd2e-c6de7f978f93
        tokenRef: astra-api-token
        clusterId: c189f0ee-07b2-4c55-a65e-8fa156b081c4
        skipTLSValidation: false
        clusterName: dev-ol-astra-enterprise-4
    imageRegistry:
        name: cr.astra.netapp.io
        secret: astra-connector-regcred
    natsSyncClient:
        cloudBridgeURL: https://10.193.126.216
    autoSupport:
        enrolled: true
EOF
astraconnector.astra.netapp.io/astra-connector created

 

In the UI, optionally copy the generated API token to a secure location (it cannot be shown again), and then click Exit. You should see your Kubernetes cluster in a pending state, waiting for the completion of the connector setup. 

 

MichaelHaigh_2-1712342419855.png

 

In your terminal, run the following command to view the status of the Astra Connector setup. 

 

kubectl -n astra-connector get astraconnector

 

For the first several minutes, you should see various statuses and the Registered field as false.

 

$ kubectl -n astra-connector get astraconnector 
NAME              REGISTERED   ASTRACLUSTERID   ASTRACONNECTORID   STATUS
astra-connector   false                                            Creating Service astra-connector/nats-cluster

 

After 3 to 5 minutes, the Registered field should switch to true. 

 

$ kubectl -n astra-connector get astraconnector
NAME              REGISTERED   ASTRACLUSTERID                         ASTRACONNECTORID                       STATUS
astra-connector   true         72b4a626-a45d-4540-8f3c-34d9a65d578c   95f1626c-3939-4869-a657-6997e6bef9ef   Registered with Astra

 

Head back to the UI, where the cluster should be shown as Available. 

 

MichaelHaigh_3-1712342495838.png

 

A default bucket is now required for each managed cluster under the new architecture, because buckets are now used to store application (and snapshot) metadata in addition to backup data. By default, the cluster inherits the associated cloud’s default bucket, but this can be overridden with a cluster-specific bucket. Click the Edit button under the Default Bucket heading. 

 

MichaelHaigh_4-1712342495840.png

 

Select the object storage bucket of your choice and click Save. If you don’t have a bucket already defined, select Buckets in the left column and follow these instructions (for ACC or for ACS) to add a bucket. 

 

MichaelHaigh_5-1712342495842.png

 

The cluster is now ready to use, whether through existing means like the UI or API, or through Kubernetes custom resource driven actions.

 

App management

 

Now that our cluster has all the necessary software components installed, and it’s registered to our Astra Control instance, we’re ready to manage an application. We’ll use Helm to first deploy a sample app. However if your cluster already has a test application running on it, feel free to skip this step. In your workstation terminal, run the following command. 

 

helm install wordpress -n wordpress --create-namespace bitnami/wordpress

 

Once it’s deployed, we’re ready to manage our application, which we’ll do with this kubectl apply command.

 

Note: It is not required to interact with the new Astra Control architecture in this manner; it’s just another option to the preexisting (and unchanging) UI and API.

 

kubectl apply -f - <<EOF 
apiVersion: astra.netapp.io/v1
kind: Application
metadata:
  name: wordpress
  namespace: astra-connector
spec:
  includedNamespaces:
  - namespace: wordpress
EOF

 

Let’s dig into this custom resource a bit more by section.

 

apiVersion: astra.netapp.io/v1
kind: Application

 

If you’re already familiar with Kubernetes custom resources, this should be straightforward. The apiVersion field points to the specific version of API that the resource adheres to. In this case it’s an extension of Kubernetes API (as are all custom resource definitions). The kind specifies the type of resource under that API definition.  

 

metadata:
  name: wordpress
  namespace: astra-connector

 

As with all Kubernetes resources, there’s a top-level metadata field, with the name of a custom resource and the namespace to which the custom resource is scoped. 

 

Note: Custom resource definitions can be either cluster or namespaced-scoped. However all Astra Control CRDs are namespaced-scoped, and all should live within the astra-connector namespace. 

 

spec:
  includedNamespaces:
  - namespace: wordpress

 

Finally, the spec contains the specifics of the application definition. This is a simple application with only a single namespace, but it’s possible for this specification to include any number of namespaces, label selectors, and cluster scoped resources.  

 

Head back into the Astra Control UI and click Applications in the left pane. You should see the wordpress application in an available state. 

 

MichaelHaigh_6-1712342750051.png

 

Click on the wordpress app and investigate the application user interface. If you’re familiar with Astra Control, you should notice that the application looks like any other application from previous versions.  

 

Now that our application is defined, let’s perform some application data management operations.

 

Snapshot creation

 

Our application has been successfully managed by creating a custom resource on the same cluster that the application lives on. We can now carry out any other application data management operation in the same manner, and we’ll start with a snapshot. 

 

In your terminal, execute the following command.

 

kubectl -n astra-connector get appvaults

 

You should see the bucket that was set as the default bucket at the end of the cluster management step, with an age of when the application was defined in the previous step. You may wonder how or why this bucket (or application vault, appVault for short) was defined on the cluster. 

 

Any time an application data management action (application definition, snapshot, backup, restore, etc.) is carried out on the cluster, the corresponding bucket is automatically defined as an appVault custom resource. Many application data management actions specifically reference a bucket/appVault (as we’ll see in a moment with our snapshot definition). However, those do not rely on the default bucket set at the cloud- and/or cluster-level (cluster taking precedence over cloud). 

 

When our application was defined, a bucket was needed to store the application’s state, including application assets, backups, and snapshots. Therefore the bucket that was set as the default in the cluster management section was created by Astra Control as an appVault custom resource on our managed cluster. This appVault is then used to store necessary information about the application, enabling “self-contained” applications (or snapshots, or backups) that do not require an operational Astra Control instance to be used. 

 

Since your bucket name is unique, run the following command to store the name of the appVault as an environment variable to be used in the next step. 

 

appVault=$(kubectl -n astra-connector get appvaults | grep -v NAME | awk '{print $1}')

 

Finally, let’s create a snapshot by running the following command.

 

kubectl apply -f - <<EOF
apiVersion: astra.netapp.io/v1
kind: Snapshot
metadata:
  name: wordpress-snapshot-1
  namespace: astra-connector
spec:
  applicationRef: wordpress
  appVaultRef: $appVault
EOF

 

Let’s investigate the fields that are different from the application custom resource previously inspected.

 

apiVersion: astra.netapp.io/v1
kind: Snapshot

 

Although the apiVersion field is the same, the kind is unsurprisingly different. Rather than an “application” definition, this is a “snapshot” definition.

 

spec:
  applicationRef: wordpress
  appVaultRef: $appVault

 

The Spec field contains two references, one to the application we previously defined and one to the appVault that we just discussed. These references are the core of the instruction to Astra Control: take a snapshot of this application and store the application metadata in this appVault (or bucket).

 

Note: even though the application metadata is stored in an external bucket for a snapshot, the Kubernetes persistent volumes are still stored locally on the cluster through a CSI volume snapshot. This means that if the namespace or cluster is destroyed, so will the application snapshot data. This is the key difference between a snapshot and a backup, where the volume snapshot data is also stored on the referenced bucket. 

 

Let’s make sure that our snapshot completed successfully, through both the CLI and the UI. First, run the following command.

 

$ kubectl -n astra-connector get snapshots
NAME                   STATE       ERROR   AGE
wordpress-snapshot-1   Completed           2m37s

 

Then head over to the UI, select our wordpress application, and view the Snapshots section of the Data Protection tab. 

 

MichaelHaigh_7-1712342987011.png

 

Both the CLI and the UI validate that we have successfully taken an application snapshot from our custom resource definition. Next, let’s make this snapshot more robust in case of a disaster.

 

Backup creation

 

As mentioned in the previous section, if our wordpress namespace or managed cluster is destroyed, we will lose our application’s persistent volumes. Let’s change that by applying the following custom resource to create a backup.

 

kubectl apply -f - <<EOF
apiVersion: astra.netapp.io/v1
kind: Backup
metadata:
  name: wordpress-backup-1
  namespace: astra-connector
spec:
  applicationRef: wordpress
  appVaultRef: $appVault
  snapshotRef: wordpress-snapshot-1
EOF

 

Again, we see the same apiVersion field, but as we expected, a different kind (Backup). Let’s further inspect the Spec section. 

 

spec:
  applicationRef: wordpress
  appVaultRef: $appVault
  snapshotRef: wordpress-snapshot-1

 

We see the same application and appVault references as our snapshot custom resource, but we also see a new snapshot reference field. This is an optional entry for a backup. If it is not specified, a new snapshot will first be created. When the snapshot is complete, the CSI volume snapshot data is copied to the referenced appVault.

 

Let’s examine this in a bit more detail with the following command. (If you don’t have yq installed on your workstation, just omit the rest of the command starting with the pipe, and find the Conditions part of the Status section.)

 

$ kubectl -n astra-connector get backup wordpress-backup-1 -o yaml | yq '.status.conditions'
- lastTransitionTime: "2024-03-21T15:15:06Z"
  message: Successfully reconciled
  reason: Done
  status: "True"
  type: AppOwnerReferenceCreated
- lastTransitionTime: "2024-03-21T15:15:07Z"
  message: Successfully reconciled
  reason: Done
  status: "True"
  type: SourceSnapshotExists
- lastTransitionTime: "2024-03-21T15:15:07Z"
  message: Successfully reconciled
  reason: Done
  status: "True"
  type: SourceSnapshotCompleted
- lastTransitionTime: "2024-03-21T15:15:07Z"
  message: Successfully reconciled
  reason: Done
  status: "True"
  type: SnapshotAppArchiveCopied
- lastTransitionTime: "2024-03-21T15:15:07Z"
  message: Successfully reconciled
  reason: Done
  status: "True"
  type: PreBackupExecHooksRunCompleted
- lastTransitionTime: "2024-03-21T15:15:46Z"
  message: Successfully reconciled
  reason: Done
  status: "True"
  type: VolumeBackupsCompleted
- lastTransitionTime: "2024-03-21T15:15:46Z"
  message: Successfully reconciled
  reason: Done
  status: "True"
  type: PostBackupExecHooksRunCompleted
- lastTransitionTime: "2024-03-21T15:15:46Z"
  message: Successfully reconciled
  reason: Done
  status: "True"
  type: TemporarySnapshotCleanedUp
- lastTransitionTime: "2024-03-21T15:15:46Z"
  message: Successfully reconciled
  reason: Done
  status: "True"
  type: Completed
- lastTransitionTime: "2024-03-21T15:15:06Z"
  message: Not yet reconciled
  reason: Pending
  status: Unknown
  type: OnFailurePostBackupExecHooksRunCompleted

 

Depending on when you ran the above command, you may see more or fewer entries with a reason of Done.  However, the Type field should be consistent with your output. Going through each entry in the list, you can see that the second and third entries are related to the underlying snapshot. The remaining entries are related to copying the volume snapshot to the bucket, various execution hooks, and cleaning up the temporary snapshot (if applicable; in this example it is not). 

 

Note: This command is a good way to investigate the status of any long-running backup operations. It’s easy to check which step in the process the operation is currently at and make sure that everything is progressing as expected. 

 

Finally, head back into the UI and verify that the backup has completed successfully. 

 

MichaelHaigh_8-1712343199387.png

 

Next steps

 

Although manually applying YAML via kubectl is great for a demonstration, a blog post, or other one-off actions, it’s not how most organizations prefer to perform application data management operations. Instead, these operations are typically automated through CI/CD platforms and GitOps tools, minimizing the likelihood of human error.

 

The new NetApp Astra Control architecture enables seamless integration between these tools and application data management. Deploying a new application through an automated pipeline? Simply add a final pipeline step that applies an applications.astra.netapp.io custom resource to the same cluster. Using a GitOps tool like Argo CD to update the application through a git push? Add a presync resource hook to the git repo that applies a backups.astra.netapp.io custom resource. These tools already have access to the application’s Kubernetes cluster, so applying one additional custom resource is a trivial step.

 

If you’re looking for more information about the available Astra Control custom resources, here are a handful of commands that you may find useful.

 

kubectl get crds | grep astra # get all Astra Control CRDs
for i in `kubectl get crds | grep astra | awk '{print $1}'`; do echo "=== $i ==="; kubectl -n astra-connector get $i; done # Get all Astra CRs grouped by CRD
kubectl describe crd <CRD-name> # Get information about the 'spec' fields that are available for <CRD-name>

 

Finally, if you’re currently an Astra Control SDK and Toolkit user, be sure to check out the newly added --v3 flag that enables CR-driven workflows.

 

$ actoolkit -h | tail -14
v3 group:
  use CR-driven Kubernetes workflows rather than the Astra Control API

  --v3                  create a v3 CR directly on the Kubernetes cluster
                        (defaults to current context, but optionally specify a
                        different context, kubeconfig_file, or
                        context@kubeconfig_file mapping)
  --dry-run {client,server}
                        client: output YAML to standard out; server: submit
                        request without persisting the resource
  --insecure-skip-tls-verify
                        If specified, the server's certificate will not be
                        checked for validity (this will make your HTTPS
                        connections insecure)

 

 

 

$ actoolkit --v3 --dry-run=client restore wordpress-backup-1 wordpress-copy gke-dr-cluster
apiVersion: astra.netapp.io/v1
kind: BackupRestore
metadata:
  name: backuprestore-84d2e86f-b50e-475e-9169-b4dc2105e6df
  namespace: astra-connector
spec:
  appArchivePath: wordpress_5b625745-b841-4260-bff6-77abdcee3515/backups/wordpress-backup-1_1650a644-bb18-4048-ba48-15e04d161b55
  appVaultRef: ontap-s3-astra-bucket8
  namespaceMapping:
  - destination: wordpress-copy
    source: wordpress
---
apiVersion: astra.netapp.io/v1
kind: Application
metadata:
  name: wordpress-copy
  namespace: astra-connector
spec:
  includedNamespaces:
  - labelSelector: {}
    namespace: wordpress-copy

 

 

 

$ actoolkit --v3 restore wordpress-backup-1 wordpress-copy gke-dr-cluster
{"apiVersion": "astra.netapp.io/v1", "kind": "BackupRestore", "metadata": {"creationTimestamp": "2024-03-21T21:41:35Z", "generation": 1, "managedFields": [{"apiVersion": "astra.netapp.io/v1", "fieldsType": "FieldsV1", "fieldsV1": {"f:spec": {".": {}, "f:appArchivePath": {}, "f:appVaultRef": {}, "f:namespaceMapping": {}}}, "manager": "OpenAPI-Generator", "operation": "Update", "time": "2024-03-21T21:41:35Z"}], "name": "backuprestore-54b136e8-a87c-4ebb-adfc-1dc071baf064", "namespace": "astra-connector", "resourceVersion": "71352498", "uid": "99e20297-1efe-4fa7-8d83-7f6490156274"}, "spec": {"appArchivePath": "wordpress_5b625745-b841-4260-bff6-77abdcee3515/backups/wordpress-backup-1_1650a644-bb18-4048-ba48-15e04d161b55", "appVaultRef": "ontap-s3-astra-bucket8", "namespaceMapping": [{"destination": "wordpress-copy", "source": "wordpress"}]}}
{"apiVersion": "astra.netapp.io/v1", "kind": "Application", "metadata": {"creationTimestamp": "2024-03-21T21:41:35Z", "generation": 1, "managedFields": [{"apiVersion": "astra.netapp.io/v1", "fieldsType": "FieldsV1", "fieldsV1": {"f:spec": {".": {}, "f:includedNamespaces": {}}}, "manager": "OpenAPI-Generator", "operation": "Update", "time": "2024-03-21T21:41:35Z"}], "name": "wordpress-copy", "namespace": "astra-connector", "resourceVersion": "71352507", "uid": "5c3d7050-0b22-4d3a-8532-6e1e5a971852"}, "spec": {"includedNamespaces": [{"labelSelector": {}, "namespace": "wordpress-copy"}]}}

 

 

Conclusion

 

In summary, we managed a Kubernetes cluster with NetApp Astra Control by creating several custom resource definitions (CRDs) and an Astra Connector custom resource. We then managed a demo application by creating an “application” custom resource via kubectl, and then protected the application by creating snapshot and backup custom resources. We then wrapped everything up by providing some additional resources and steps that you can take to learn more about the new architecture.

 

Thank you for making it to the end of this blog, and we hope you found the content useful!

Public