Tech ONTAP Blogs
Tech ONTAP Blogs
Trident 25.06 adds support for Kubernetes Volume Group Snapshots (tested with v1beta1) with ONTAP-SAN volumes. This feature enables point-in-time, crash-consistent snapshots of multiple Persistent Volumes (PVs) by leveraging NetApp ONTAP’s Consistency Group (CG) capabilities.
Trident has long supported Kubernetes Volume Snapshots, allowing admins to snapshot individual Kubernetes PVs with ONTAP storage. However, this doesn't work for scenarios where point-in-time consistency is required between multiple snapshots. With standard Kubernetes Volume Snapshots, neither Kubernetes nor Trident can guarantee point-in-time consistency.
That’s where Kubernetes Volume Group Snapshots come in. This feature allows you to take point-in-time consistent snapshots of a group of PVs.
To support this, Kubernetes introduces three new Custom Resources (CRs):
VolumeGroupSnapshot VolumeGroupSnapshotContent VolumeGroupSnapshotClass At the time of writing, Kubernetes 1.33 will support v1beta1 volume group snapshot CRDs. Trident will automatically detect if these CRDs are present to enable the relevant feature gates in our CSI sidecars. You will also need to enable the group snapshot feature gate in the "external-snapshotter" in Kubernetes.
Trident leverages ONTAP's Consistency Group (CG) feature to achieve point-in-time consistency for multiple PVs. Trident creates an ONTAP CG to take a group snapshot of a set of volumes. Once the group snapshot is complete, Trident removes the ONTAP CG. This innovation treats CGs in an ephemeral manner.
First, administrators will need to group volumes. This can be accomplished by applying a Kubernetes label consisting of their own custom key-value pair. A single Persistent Volume (PV) may be part of multiple groups, and those groups may be entirely disjoint or partially overlapping. For this example, we'll only map the PVC to a single group through the label: "group: myGroup". Labels may also be applied to existing PVCs that are already bound to a PV.
Example
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc1
labels:
group: myGroup
spec:
resources:
requests:
storage: 500Mi
storageClassName: sc1
Then, define a VolumeGroupSnapshotClass CR that specifies all fields necessary. Similar to the Kubernetes VolumeSnapshotClass, a VolumeGroupSnapshotClass CR may be reused for multiple volume group snapshot instances. This connects the group snapshot to the right CSI driver (like Trident). The "driver" field must reference Trident, or the group snapshot will not be created. Depending on your use cases, it may be wise to specify the proper deletion policy.
You do not have to create a unique VolumeGroupSnapshotClass CR for every group snapshot.
Example
apiVersion: groupsnapshot.storage.k8s.io/v1beta1 kind: VolumeGroupSnapshotClass metadata: name: vgsclass1 driver: csi.trident.netapp.io deletionPolicy: Delete
Next, create a VolumeGroupSnapshot CR.
Be sure to specify the aforementioned VolumeGroupSnapshotClass CR and appropriate label.
Example
apiVersion: groupsnapshot.storage.k8s.io/v1beta1
kind: VolumeGroupSnapshot
metadata:
name: vgs1
namespace: trident
spec:
volumeGroupSnapshotClassName: vgsclass1
source:
selector:
matchLabels:
group: myGroup
At this point, Trident receives a CSI request to create a group snapshot of the selected volumes.
This process creates several resources:
VolumeGroupSnapshot and VolumeGroupSnapshotContent custom resource (CR)VolumeSnapshot and VolumeSnapshotContent CR for each volume in the groupTridentGroupSnapshot CRTridentSnapshot CR for each individual snapshot in the group
In this concrete example, I've created a group snapshot for a group of two volumes.
Let's walk through each resource, its purpose, and provide an example.
Let's take a closer look at the following resources:
VolumeGroupSnapshot CRVolumeGroupSnapshotContent CRVolumeSnapshot CRsVolumeSnapshotContent CRs
This resource represents the request to create a group snapshot.
Example
$ kubectl get volumegroupsnapshot -n trident -o yaml
apiVersion: v1
items:
- apiVersion: groupsnapshot.storage.k8s.io/v1beta1
kind: VolumeGroupSnapshot
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"groupsnapshot.storage.k8s.io/v1beta1","kind":"VolumeGroupSnapshot","metadata":{"annotations":{},"name":"vgs1","namespace":"trident"},"spec":{"source":{"selector":{"matchLabels":{"group":"myGroup"}}},"volumeGroupSnapshotClassName":"csi-group-snap-class"}}
creationTimestamp: "2025-06-26T18:32:42Z"
finalizers:
- groupsnapshot.storage.kubernetes.io/volumegroupsnapshot-bound-protection
generation: 1
name: vgs1
namespace: trident
resourceVersion: "200373"
uid: 411d2111-89ea-4129-a454-015d1fd8ae85
spec:
source:
selector:
matchLabels:
group: myGroup
volumeGroupSnapshotClassName: csi-group-snap-class
status:
boundVolumeGroupSnapshotContentName: groupsnapcontent-411d2111-89ea-4129-a454-015d1fd8ae85
creationTime: "2025-06-26T18:01:21Z"
readyToUse: true
This resource represents the fulfillment of the VolumeGroupSnapshot request.
Example
$ kubectl get volumegroupsnapshotcontent -n trident -o yaml
apiVersion: v1
items:
- apiVersion: groupsnapshot.storage.k8s.io/v1beta1
kind: VolumeGroupSnapshotContent
metadata:
creationTimestamp: "2025-06-26T18:32:42Z"
finalizers:
- groupsnapshot.storage.kubernetes.io/volumegroupsnapshotcontent-bound-protection
generation: 1
name: groupsnapcontent-411d2111-89ea-4129-a454-015d1fd8ae85
resourceVersion: "200347"
uid: df01c3d7-8e91-4b0f-b992-eddc0a99e6ab
spec:
deletionPolicy: Delete
driver: csi.trident.netapp.io
source:
volumeHandles:
- pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500
- pvc-cf598c63-5fc6-42af-83b0-f490180d23aa
volumeGroupSnapshotClassName: csi-group-snap-class
volumeGroupSnapshotRef:
apiVersion: groupsnapshot.storage.k8s.io/v1beta1
kind: VolumeGroupSnapshot
name: vgs1
namespace: trident
resourceVersion: "200331"
uid: 411d2111-89ea-4129-a454-015d1fd8ae85
status:
creationTime: "2025-06-26T18:01:21Z"
readyToUse: true
volumeGroupSnapshotHandle: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
volumeSnapshotHandlePairList:
- snapshotHandle: pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
volumeHandle: pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500
- snapshotHandle: pvc-cf598c63-5fc6-42af-83b0-f490180d23aa/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
volumeHandle: pvc-cf598c63-5fc6-42af-83b0-f490180d23aa
These resources represent the request for a volume snapshot. For group snapshots, these will have a new "ownerReferences" entry that points to their parent VolumeGroupSnapshot as the owner.
Read more about owner references.
Example
$ kubectl get volumesnapshot -n trident -o yaml
apiVersion: v1
items:
- apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
creationTimestamp: "2025-06-26T18:32:47Z"
finalizers:
- snapshot.storage.kubernetes.io/volumesnapshot-in-group-protection
- snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection
generation: 1
name: snapshot-08f47b907f2f006f5d890a80afe11fbb977f63554498833f217435d95e435045
namespace: trident
ownerReferences:
- apiVersion: groupsnapshot.storage.k8s.io/v1beta1
kind: VolumeGroupSnapshot
name: vgs1
uid: 411d2111-89ea-4129-a454-015d1fd8ae85
resourceVersion: "200378"
uid: 2345f17a-5e8a-4ff6-a77d-47c2cc7b34d9
spec:
source:
persistentVolumeClaimName: pvc2-2
status:
boundVolumeSnapshotContentName: snapcontent-08f47b907f2f006f5d890a80afe11fbb977f63554498833f217435d95e435045
creationTime: "2025-06-26T18:01:21Z"
readyToUse: true
restoreSize: 100Mi
volumeGroupSnapshotName: vgs1
- apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
creationTimestamp: "2025-06-26T18:32:47Z"
finalizers:
- snapshot.storage.kubernetes.io/volumesnapshot-in-group-protection
- snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection
generation: 1
name: snapshot-258720deb7c66ef2f4198286be15515d5908186da7d1888c08af3800e3bfed1e
namespace: trident
ownerReferences:
- apiVersion: groupsnapshot.storage.k8s.io/v1beta1
kind: VolumeGroupSnapshot
name: vgs1
uid: 411d2111-89ea-4129-a454-015d1fd8ae85
resourceVersion: "200370"
uid: 0637880a-6134-428d-a8fc-67214c726e99
spec:
source:
persistentVolumeClaimName: pvc2
status:
boundVolumeSnapshotContentName: snapcontent-258720deb7c66ef2f4198286be15515d5908186da7d1888c08af3800e3bfed1e
creationTime: "2025-06-26T18:01:21Z"
readyToUse: true
restoreSize: 100Mi
volumeGroupSnapshotName: vgs1
kind: List
metadata:
resourceVersion: ""
These resources represent the result of creating the requested VolumeSnapshot.
Example
$ kubectl get volumesnapshotcontent -n trident -o yaml
apiVersion: v1
items:
- apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotContent
metadata:
annotations:
groupsnapshot.storage.k8s.io/volumeGroupSnapshotHandle: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
creationTimestamp: "2025-06-26T18:32:47Z"
finalizers:
- snapshot.storage.kubernetes.io/volumesnapshotcontent-bound-protection
generation: 2
name: snapcontent-08f47b907f2f006f5d890a80afe11fbb977f63554498833f217435d95e435045
resourceVersion: "200376"
uid: 8e161f10-40f9-478a-a2f1-04a2d70fa8a4
spec:
deletionPolicy: Delete
driver: csi.trident.netapp.io
source:
volumeHandle: pvc-cf598c63-5fc6-42af-83b0-f490180d23aa
sourceVolumeMode: Filesystem
volumeSnapshotRef:
kind: VolumeSnapshot
name: snapshot-08f47b907f2f006f5d890a80afe11fbb977f63554498833f217435d95e435045
namespace: trident
uid: 2345f17a-5e8a-4ff6-a77d-47c2cc7b34d9
status:
creationTime: 1750960881000000000
readyToUse: true
restoreSize: 104857600
snapshotHandle: pvc-cf598c63-5fc6-42af-83b0-f490180d23aa/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
volumeGroupSnapshotHandle: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
- apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotContent
metadata:
annotations:
groupsnapshot.storage.k8s.io/volumeGroupSnapshotHandle: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
creationTimestamp: "2025-06-26T18:32:47Z"
finalizers:
- snapshot.storage.kubernetes.io/volumesnapshotcontent-bound-protection
generation: 2
name: snapcontent-258720deb7c66ef2f4198286be15515d5908186da7d1888c08af3800e3bfed1e
resourceVersion: "200362"
uid: 5631b489-1bdf-4d47-a38d-153a08b944cf
spec:
deletionPolicy: Delete
driver: csi.trident.netapp.io
source:
volumeHandle: pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500
sourceVolumeMode: Filesystem
volumeSnapshotRef:
kind: VolumeSnapshot
name: snapshot-258720deb7c66ef2f4198286be15515d5908186da7d1888c08af3800e3bfed1e
namespace: trident
uid: 0637880a-6134-428d-a8fc-67214c726e99
status:
creationTime: 1750960881000000000
readyToUse: true
restoreSize: 104857600
snapshotHandle: pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
volumeGroupSnapshotHandle: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
kind: List
metadata:
resourceVersion: ""
Trident adds its own custom resources when creating a volume group snapshot. These Trident CRs are not meant to be manipulated by users directly but may be viewed via kubectl or tridentctl.
As part of a group snapshot, Trident creates the following CRs:
TridentGroupSnapshot CRTridentSnapshot CR
This stores useful metadata about the group snapshot for Trident. Creating a group snapshot in Kubernetes should result in a 1-1 relationship between the TridentGroupSnapshot and Kubernetes VolumeGroupSnapshotContent.
Example
$ kubectl get tridentgroupsnapshot -n trident -o yaml
apiVersion: v1
items:
- apiVersion: trident.netapp.io/v1
dateCreated: "2025-06-26T18:01:21.000Z"
kind: TridentGroupSnapshot
metadata:
creationTimestamp: "2025-06-26T18:32:47Z"
finalizers:
- trident.netapp.io
generation: 1
name: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
namespace: trident
resourceVersion: "200340"
uid: e905c9b1-73c4-48b2-92e6-ecedaaadbd39
snapshotIDs:
- pvc-cf598c63-5fc6-42af-83b0-f490180d23aa/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
- pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
spec:
internalName: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
name: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
version: "1"
volumeNames:
- pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500
- pvc-cf598c63-5fc6-42af-83b0-f490180d23aa
kind: List
metadata:
resourceVersion: ""
This stores useful metadata about the individual snapshot for Trident. Creating a group snapshot in Kubernetes should result in a 1-1 relationship between Trident Snapshots and Kubernetes Volume Snapshot Contents.
Example
$ kubectl get tridentsnapshot -n trident -o yaml
apiVersion: v1
items:
- apiVersion: trident.netapp.io/v1
dateCreated: "2025-06-26T18:01:21.000Z"
kind: TridentSnapshot
metadata:
creationTimestamp: "2025-06-26T18:32:47Z"
finalizers:
- trident.netapp.io
generation: 1
name: pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500-snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
namespace: trident
resourceVersion: "200342"
uid: 6e4e197e-2f13-4705-98f8-64df0a0fc475
size: 104857600
spec:
groupSnapshotName: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
importNotManaged: false
internalName: snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
name: snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
volumeInternalName: websterj_pvc_373e0043_ca12_42ab_bcd9_4360f0ddc500
volumeName: pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500
state: online
- apiVersion: trident.netapp.io/v1
dateCreated: "2025-06-26T18:01:21.000Z"
kind: TridentSnapshot
metadata:
creationTimestamp: "2025-06-26T18:32:47Z"
finalizers:
- trident.netapp.io
generation: 1
name: pvc-cf598c63-5fc6-42af-83b0-f490180d23aa-snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
namespace: trident
resourceVersion: "200341"
uid: 881bbe33-1553-485a-a0c7-f79142918bdd
size: 104857600
spec:
groupSnapshotName: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
importNotManaged: false
internalName: snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
name: snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
volumeInternalName: websterj_pvc_cf598c63_5fc6_42af_83b0_f490180d23aa
volumeName: pvc-cf598c63-5fc6-42af-83b0-f490180d23aa
state: online
kind: List
metadata:
resourceVersion: ""
Trident's CLI, tridentctl, may be used to view group snapshots via the new "groupsnapshot" subcommand.
Example
$ tridentctl get groupsnapshot -h -n trident
Get one or more group snapshots from Trident
Usage:
tridentctl get groupsnapshot [<group snapshot name>...] [flags]
Aliases:
groupsnapshot, gs, gsnap, groupsnapshots
Flags:
-h, --help help for groupsnapshot
--max-snapshots int Set the maximum number of snapshots (inclusive) to display when displaying grouped snapshots.
Setting this to 0 will show all snapshots in the group. (default 3)
Global Flags:
-d, --debug Set the log level to debug
-k, --kubeconfig string Kubernetes config path
--log-level string Log level (trace, debug, warn, info, error, fatal (default "info")
-n, --namespace string Namespace of Trident deployment
-o, --output string Output format. One of json|yaml|name|wide|ps (default)
-s, --server string Address/port of Trident REST interface (127.0.0.1 or [::1] only)
You may also specify an output format: "-o json|yaml|wide":
$ tridentctl get groupsnapshot -n trident
+----------------------------------------------------+
| NAME |
+----------------------------------------------------+
| groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85 |
+----------------------------------------------------+
$ tridentctl get groupsnapshot -n trident -o wide
+----------------------------------------------------+--------------------------+----------------------------------------------------------------------------------------+----------------+
| NAME | CREATED | SNAPSHOTS | SNAPSHOT COUNT |
+----------------------------------------------------+--------------------------+----------------------------------------------------------------------------------------+----------------+
| groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | 2025-06-26T18:01:21.000Z | pvc-cf598c63-5fc6-42af-83b0-f490180d23aa/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | 2 |
| | | pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | |
+----------------------------------------------------+--------------------------+----------------------------------------------------------------------------------------+----------------+
$ tridentctl get groupsnapshot -n trident -o json
{
"items": [
{
"version": "1",
"name": "groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85",
"internalName": "groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85",
"volumeNames": [
"pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500",
"pvc-cf598c63-5fc6-42af-83b0-f490180d23aa"
],
"snapshotIDs": [
"pvc-cf598c63-5fc6-42af-83b0-f490180d23aa/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85",
"pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85"
],
"dateCreated": "2025-06-26T18:01:21.000Z"
}
]
}
$ tridentctl get groupsnapshot -n trident -o yaml
items:
- dateCreated: "2025-06-26T18:01:21.000Z"
internalName: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
name: groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85
snapshotIDs:
- pvc-cf598c63-5fc6-42af-83b0-f490180d23aa/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
- pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85
version: "1"
volumeNames:
- pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500
- pvc-cf598c63-5fc6-42af-83b0-f490180d23aa
The wide format option will display 3 additional columns:
Because there may be a large number of snapshots in a single group snapshot, the number of snapshot handles is truncated to 3 but will display more should you supply the --max-snapshots flag. Specifying "0" will display all snapshots in the group.
$ tridentctl get groupsnapshot -o wide -n trident --max-snapshots=0 +----------------------------------------------------+--------------------------+----------------------------------------------------------------------------------------+----------------+ | NAME | CREATED | SNAPSHOTS | SNAPSHOT COUNT | +----------------------------------------------------+--------------------------+----------------------------------------------------------------------------------------+----------------+ | groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | 2025-06-26T18:01:21.000Z | pvc-cf598c63-5fc6-42af-83b0-f490180d23aa/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | 2 | | | | pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | | +----------------------------------------------------+--------------------------+----------------------------------------------------------------------------------------+----------------+ # If you specify a number less than the total count, but greater than 0, the list of constituent snapshot handles is truncated. $ tridentctl get groupsnapshot -o wide -n trident --max-snapshots=1 +----------------------------------------------------+--------------------------+----------------------------------------------------------------------------------------+----------------+ | NAME | CREATED | SNAPSHOTS | SNAPSHOT COUNT | +----------------------------------------------------+--------------------------+----------------------------------------------------------------------------------------+----------------+ | groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | 2025-06-26T18:01:21.000Z | pvc-cf598c63-5fc6-42af-83b0-f490180d23aa/snapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | 2 | | | | ... (+1 more) | | +----------------------------------------------------+--------------------------+----------------------------------------------------------------------------------------+----------------+
Additionally, Trident Snapshots taken as a group will have an extra field ("GroupSnapshot") identifying its Trident Group Snapshot parent:
$ tridentctl get snapshot -o wide -n trident +-----------------------------------------------+------------------------------------------+--------------------------+---------+-------+---------+----------------------------------------------------+ | NAME | VOLUME | CREATED | SIZE | STATE | MANAGED | GROUPSNAPSHOT | +-----------------------------------------------+------------------------------------------+--------------------------+---------+-------+---------+----------------------------------------------------+ | snapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | pvc-373e0043-ca12-42ab-bcd9-4360f0ddc500 | 2025-06-26T18:01:21.000Z | 100 MiB | | true | groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | | snapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | pvc-cf598c63-5fc6-42af-83b0-f490180d23aa | 2025-06-26T18:01:21.000Z | 100 MiB | | true | groupsnapshot-411d2111-89ea-4129-a454-015d1fd8ae85 | +-----------------------------------------------+------------------------------------------+--------------------------+---------+-------+---------+----------------------------------------------------+
tridentctl while their parent TridentGroupSnapshotCR exists.TridentGroupSnapshot with tridentctl is not recommended, as it may cause inconsistencies between Kubernetes and Trident.VolumeGroupSnapshot resource.VolumeGroupSnapshot also deletes all constituent snapshots, the TridentGroupSnapshot, and related TridentSnapshot resources.
In future releases, the Trident team plans to broaden support for volume group snapshots to the ONTAP-SAN-Economy, ONTAP-NAS, and ONTAP-SAN (NVMe) drivers. Try out our new volume group snapshot feature with ONTAP-SAN volumes and let us know what you think.
Thanks for reading!