Philipp Lehmann @PhilippTheServer

Importing Pre-Existing Storage as a Static PersistentVolume

Making Kubernetes mount data it did not create, without letting it delete that data.

kubernetesstoragecephreliability

The problem

Dynamic provisioning is built on one assumption: a PersistentVolumeClaim asks for storage, a StorageClass’s provisioner creates a fresh, empty volume to satisfy it, and the two are bound. That assumption is wrong the moment the data you need to mount already exists somewhere — a Ceph RBD image left over from a host migration, a CephFS subvolume recovered after a disk failure, an NFS export that predates the cluster entirely. None of that is what a provisioner produces, and a plain PVC has no way to say “bind to this specific thing, don’t make me a new one.”

Write the obvious PVC and Kubernetes does exactly what it is designed to do — provision something new and ignore the existing data completely:

# Broken. This binds to whatever the default StorageClass provisions,
# not to any data that already exists on disk.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: existing-data
spec:
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 5Gi

There is a second half to this problem, and it is easy to miss until it costs you something: even once you do get a PVC bound to the right pre-existing volume, the default reclaim policy for dynamically provisioned storage is Delete. Delete a PVC under that policy and the underlying volume is deleted with it. For data Kubernetes did not create, that is not a lifecycle Kubernetes should be trusted with by default — a kubectl delete pvc typed against the wrong namespace, or run by a script that assumed the claim was disposable, destroys something that existed before the cluster did and cannot be regenerated by re-running a manifest.

Working through it

Take the volume out of the provisioner’s reach

Dynamic provisioning is driven by storageClassName. If a PVC’s storage class matches one with a provisioner configured, that provisioner will try to satisfy the claim by creating a new volume — full stop. To mount pre-existing data, you need a class name that no provisioner is watching, or an explicitly empty one (storageClassName: ""), so the claim can only be satisfied by a PersistentVolume you create by hand. This is the first piece: nothing should be racing to fill the claim with something new.

Make Retain the thing that protects you, not the binding

persistentVolumeReclaimPolicy: Retain on the PersistentVolume is what actually answers the “mistaken delete” half of the problem, and it is worth being precise about what it does. Delete the PVC bound to a Retain volume, and the PV does not disappear — it moves to a Released state, with the underlying data completely untouched. It also does not silently become available for the next claim that asks for the same storage class: a Released PV keeps a claimRef pointing at the PVC that no longer exists, and won’t bind to anything else until that claimRef is removed by hand. Getting the data back into use is therefore a deliberate, visible administrative step — someone has to look at the volume, decide it is safe to reuse, and edit it to make that so. That friction is not an oversight; it is the entire point. An accidental delete costs you a Released PV and a five-minute recovery, not the data itself.

Bind the pair to each other explicitly

A shared storageClassName alone isn’t enough to guarantee this PVC binds this PV, particularly if more than one static PV happens to use the same class name — Kubernetes will bind the first PVC that fits to the first matching PV it finds, which is exactly the kind of race you do not want when “the wrong PV” means “the wrong pre-existing data.” Pin the relationship from both ends: the PVC names its target PV directly via spec.volumeName, and the PV names its expected claim back via spec.claimRef (namespace and name). With both sides pointing at each other, there is no ambiguity for the scheduler to resolve — and no other PVC, however coincidentally similar its request, can bind to this PV first.

What this looks like against real storage

In infrastructure that already runs Ceph, this pattern is exactly how you adopt an existing RBD image or CephFS subvolume rather than provisioning a new one: the PV’s spec.csi.volumeHandle points at the existing image or subvolume identifier instead of one a CSI driver generated, and everything else — the class name, Retain, the explicit claimRef — works identically. That example depends on infrastructure a reader of this article will not have, so the runnable version below uses a hostPath volume on a local kind cluster instead, standing in for “storage that already has data on it.” The mechanics are the same regardless of what sits behind the PV.

The solution

Everything below runs on a laptop with kind.

kind create cluster --name static-pv-demo

Simulate pre-existing data by writing it directly onto the node, outside of anything Kubernetes provisioned:

docker exec static-pv-demo-control-plane mkdir -p /mnt/existing-data
docker exec static-pv-demo-control-plane sh -c \
  'echo "this file predates the PVC" > /mnt/existing-data/marker.txt'

# pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: existing-data-pv
spec:
  capacity:
    storage: 5Gi
  accessModes: ["ReadWriteOnce"]
  persistentVolumeReclaimPolicy: Retain
  storageClassName: manual-import
  hostPath:
    path: /mnt/existing-data
  claimRef:
    namespace: default
    name: existing-data

# pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: existing-data
  namespace: default
spec:
  accessModes: ["ReadWriteOnce"]
  storageClassName: manual-import
  resources:
    requests:
      storage: 5Gi
  volumeName: existing-data-pv

# pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: reader
spec:
  containers:
    - name: reader
      image: busybox:1.36
      command: ["sleep", "3600"]
      volumeMounts:
        - name: data
          mountPath: /data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: existing-data

Apply everything and confirm the pre-existing data is visible without anything having been provisioned:

kubectl apply -f pv.yaml -f pvc.yaml -f pod.yaml
kubectl wait --for=condition=Ready pod/reader --timeout=60s
kubectl exec reader -- cat /data/marker.txt
# this file predates the PVC

Now prove the delete-protection. The reader pod still has the claim mounted, and Kubernetes will not delete a PVC while a pod is using it — it holds a kubernetes.io/pvc-protection finalizer that blocks removal until the volume is released, so running kubectl delete pvc against a claim still in use just hangs. Delete the pod first, then the claim, and look at what happens to the volume:

kubectl delete pod reader --now
kubectl delete pvc existing-data
kubectl get pv existing-data-pv
# NAME               ... STATUS     CLAIM                    STORAGECLASS
# existing-data-pv   ... Released   default/existing-data    manual-import

docker exec static-pv-demo-control-plane cat /mnt/existing-data/marker.txt
# this file predates the PVC

The PV is Released, not gone, and the marker file is still exactly where it was. To make the volume usable again, the stale claimRef has to be cleared explicitly — this patch is the deliberate step that stands between “the PVC was deleted” and “the data was deleted”:

kubectl patch pv existing-data-pv --type=json \
  -p '[{"op": "remove", "path": "/spec/claimRef"}]'

After that, re-applying pvc.yaml binds cleanly to the same volume, with the same data, exactly as before.

Conclusion

Reclaim policy, not the binding mechanics, is what protects you. volumeName and claimRef control which PVC binds which PV; only persistentVolumeReclaimPolicy: Retain controls what happens to the data when that claim is deleted. Get the policy wrong and the most carefully pinned binding still ends in data loss.

Explicit, mutual binding avoids storage-class races. A shared class name is not a guarantee of which PVC gets which PV when more than one candidate exists; pinning both ends removes the ambiguity rather than hoping the scheduler resolves it the way you intended.

This is the general pattern for adopting data Kubernetes did not create — a Ceph RBD image or CephFS subvolume via csi.volumeHandle, an NFS export, a disk recovered after a failure. The volume type changes; the shape of the solution does not.

Rebinding a Retain volume is meant to require a human. The manual claimRef patch is not friction to route around — it is the checkpoint where someone confirms the data is safe to reuse before Kubernetes is allowed to hand it to something new.