Move iSCSI Volumes Live with Trident 26.06: A Hands-on Guide to Volume Move

jharrod

Move iSCSI Volumes Live with Trident 26.06: A Hands-on Guide to Volume Move

Overview

Kubernetes environments change constantly: aggregates fill, nodes need maintenance, and performance hot spots appear. With Trident 26.06, you can now move an iSCSI-backed volume non-disruptively to a different ONTAP node and aggregate within the same ONTAP cluster—while the application keeps running.

This post shows how to use Volume Move end-to-end and how to monitor it as it progresses.

What Volume Move is (and what it isn’t)

Volume Move is a Trident-orchestrated operation that relocates the ONTAP backing storage for a Kubernetes PV to a new aggregate within the same ONTAP cluster

Scope:

Supported: ontap-san driver, iSCSI volumes, using the ONTAP REST API
Not supported: NAS protocols or NVMe (for this feature)
Not a cluster-to-cluster migration: this is within one ONTAP cluster

Requirements checklist

Before you try it, confirm:

Trident version: 26.06+
Backend driver + protocol: ontap-san with iSCSI, using the ONTAP REST API
ONTAP cluster topology: at least two ONTAP nodes
Credentials / ONTAP version:
- ONTAP 9.19.1 or later if using a dedicated Trident role in ONTAP
- On earlier ONTAP releases, use cluster-admin credentials

What’s supported (operational notes)

A few practical points that help set expectations in production:

Access modes: ReadWriteOnce (RWO) and ReadWriteMany (RWX) volumes are supported.
Workloads can change during the move: volumes can be attached or detached while a move is in progress (pods being created, rescheduled, moved, or removed by Kubernetes). Trident reconciles attachments throughout; if a node attaches mid-move, it will be brought onto the correct path set automatically.
Moving duration varies by size: the Moving phase can range from minutes for small volumes to multiple hours for very large volumes.

How it works (why it’s non-disruptive)

Trident keeps I/O online by managing connectivity first:

Adds new iSCSI paths to the target location alongside the existing source paths
Lets ONTAP move/cut over the data in-cluster (this is typically the longest phase)
Gracefully removes stale paths post-cutover

From the host’s perspective, the multipath device retains access throughout. Trident will not start the move until all new paths are successfully added.

Hands-on: moving a volume

Step 1: Identify the PV and ONTAP source/target locations

In this step, you’ll identify:

The PV name to move (you’ll use this later in the tridentctl create volume-move command)
The volume’s current ONTAP aggregate (--source-pool)
The node/HA pair that aggregate belongs to (--source-node)
A target node + aggregate to move to and optionally on a different HA pair (--target-node, --target-pool)

1) Identify HA pairs in the cluster

If your goal is to move a volume to a different HA pair, start by confirming which nodes are paired:

> storage failover show
                              Takeover
                                 by
Node           Partner        Possible State Description
-------------- -------------- -------- --------------------------------------
node-1
               node-2         true     Connected to node-2   
node-2
               node-1         true     Connected to node-1
node-3
               node-4         true     Connected to node-4
node-4
               node-3         true     Connected to node-3   
4 entries were displayed.

From this output, you can see two HA pairs:

HA Pair A: node-1 ↔ node-2
HA Pair B: node-3 ↔ node-4

2) Find the volume’s current (source) aggregate

Next, determine where the ONTAP volume that backs your PV currently lives.

Run:

volume show -vserver <SVM> -volume <vol-name> -fields aggregate

Example:

> volume show -vserver vs0 -volume trident_pvc_60be4d54_ddb1_4e51_bdef_b0bd6c0b80cc -fields aggregate
vserver volume                                           aggregate
------- ------------------------------------------------ --------------------------
vs0     trident_pvc_60be4d54_ddb1_4e51_bdef_b0bd6c0b80cc node_3_aggr1

In this example, the source aggregate is node_3_aggr1.

3) Map that aggregate to the source node (and list candidate targets)

List available aggregates and see which node owns each one:

> storage aggregate show -fields node
aggregate                  node
-------------------------- --------------------
aggr0_node_1               node-1
aggr0_node_2               node-2
aggr0_node_3               node-3
aggr0_node_4               node-4
node_1_aggr1               node-1
node_2_aggr1               node-2
node_3_aggr1               node-3
node_4_aggr1               node-4
8 entries were displayed.

From this output, you can map the current aggregate to the current node:

node_3_aggr1 is owned bynode-3(source node)

4) Select the destination (target) on a different HA pair

Continuing the example:

The volume is currently on node_3_aggr1 (node node-3) → HA Pair B
To move it to the other HA pair (HA Pair A), choose a target on either:
- node node-1, or
- node node-2

Good target options:

Target aggregate: node_1_aggr1
Target node: node-1

or

Target aggregate: node_2_aggr1
Target node: node-2

5) Final mapping: the exact values you’ll pass to Trident

Using the example above, you now have both source and target values.

Source (current location):

--source-node node-3
--source-pool node_3_aggr1

Target (new location), choose one:

--target-node node-1 and --target-pool node_1_aggr1
--target-node node-2 and --target-pool node_2_aggr1

Step 2: Start the move

You can start a move either with tridentctl or by applying a TridentVolumeMove custom resource directly. Both approaches create the same move request in Trident.

Option A: Start the move using tridentctl

This command creates a TridentVolumeMove (tvm) CR under the hood (resource name = PV name):

tridentctl create volume-move \
  --volume      pvc-8a814d62-bd58-4253-b0d1-82f2885db671 \
  --source-node node-3 \
  --source-pool node_3_aggr1 \
  --target-node node-1 \
  --target-pool node_1_aggr1 \
  -n <Trident Namespace>

Notes:

tvm is a shorthand alias for volume-move
To automatically remove the resource after success, you can add --delete-after-success <DURATION>

Field meanings (quick reference):

sourceNode / targetNode: ONTAP node names (not UUIDs)
sourcePool / targetPool: ONTAP aggregate names
deleteAfterSuccess: optional, takes a duration (e.g. 30s, 10m)
- 0 deletes immediately upon success
- omit the option to keep the resource

Option B: Start the move by applying a TridentVolumeMove CR

Create a manifest (for example, volume-move.yaml).
The resource name must match the PV name and be created in Trident’s namespace:

apiVersion: trident.netapp.io/v1
kind: TridentVolumeMove
metadata:
  name: pvc-8a814d62-bd58-4253-b0d1-82f2885db671
  namespace: trident
spec:
  sourceNode: node-3
  sourcePool: node_3_aggr1
  targetNode: node-1
  targetPool: node_1_aggr1
  # Optional: delete the CR after success. Use a duration like 10m, 30s, etc.
  # Set to 0 to delete immediately, or omit the option to retain.
  # deleteAfterSuccess: 10m

Apply it:

kubectl apply -f volume-move.yaml

Monitor the move

Watch overall progress:

kubectl get tridentvolumemove -n trident

Inspect state + per-node progress:

kubectl describe tridentvolumemove <pv-name> -n trident

What to expect while it runs:

The Moving state duration can vary widely—from minutes or less for small volumes to hours for very large volumes.
Because the Moving phase can take time, Trident continues to handle attachment changes during the move. Pods can be created, rescheduled, or deleted, and Trident will add or remove the relevant host-side iSCSI paths as nodes attach to or detach from the volume.

Understanding the state machine (so you know what “normal” looks like)

You’ll see the move progress through these states:

Pending: request accepted, validated, info gathered (no storage changes yet)
ControllerStaging: Trident prepares new ONTAP LUN mappings (SLM) for the new location
NodeStaging: Trident adds target paths on each attached node (I/O continues across both path sets)
Moving: ONTAP copies and cuts over data to the target aggregate (Trident polls)
ControllerUnstaging: Trident removes old ONTAP LUN mappings
NodeUnstaging: stale source paths removed on hosts; only target paths remain
Succeeded: move completed successfully
Failed: move failed, see CR status.message for info

Attachment progress: .status.attachments

Each attached Kubernetes node has its own progress state:

Pending → Bridged → Migrated → Cleaned
Plus:
Detached (node detached mid-move; excluded while Trident evaluates readiness)
Failed (terminal node-level error) - see the message field for the failed attachment for info

.status.attachments tracks nodes that were attached when the move entered NodeStaging; new attachments after that may not be listed.

Cleanup and retries

If you set deleteAfterSuccess, Trident removes the TridentVolumeMove resource automatically. Otherwise, it can be deleted manually.

Important retry rule: before requesting another move for the same volume, ensure there is no existing TridentVolumeMove for that PV in Succeeded or Failed state.

Also by design:

Trident never auto-deletes Failed moves (it keeps them as an audit record)
To retry: fix the root cause, delete the Failed resource, then create a new move request

Where this fits operationally

If you’re a Kubernetes admin running Trident with ONTAP SAN (iSCSI), Volume Move gives you a practical new tool to rebalance storage placement, prepare for maintenance, and respond to capacity/performance shifts—without asking app teams for downtime.

See documentation for more information: Move a volume.