A crashed stateless pod comes back with a random name and a new IP, and nobody minds. A database node that returns with a different identity can break cluster consensus and force expensive re-sharding. A StatefulSet fixes that with stable network identities, ordered operations and automatic per-pod storage.
Identity and storage
Diagram: FIG 1 · Each StatefulSet pod keeps a stable DNS name and its own PersistentVolumeClaim, and pods start in order. A restarted pod comes back as the same member. See the animated version.
Each pod gets a predictable DNS name, such as qdrant-0.qdrant-headless.ai-platform.svc.cluster.local, plus its own PVC. Pods start in order, qdrant-0 then qdrant-1, and the name survives a restart.
Why Deployments fail databases
Diagram: FIG 2 · Identity, storage and shutdown order all matter to a clustered database. StatefulSets handle each, and Deployments do not. See the animated version.
- Identity instability. Random names and IPs after a restart force the cluster to re-sync large indexes.
- Storage contention. A Deployment shares one claim among its pods. A StatefulSet uses
volumeClaimTemplatesto create a dedicated volume per replica. - Uncontrolled termination. A Deployment can stop many pods at once. A StatefulSet stops them in strict reverse order,
qdrant-2, thenqdrant-1, thenqdrant-0, which helps the cluster keep quorum.
Diagram: FIG 3 · Pods start from zero upward and stop from the highest ordinal downward, which helps clustered databases keep quorum during restarts and updates. See the animated version.
The manifest
This runs Qdrant in clustered mode. Qdrant needs each new peer to know where to bootstrap from, so the first pod starts the cluster and the others join it.
apiVersion: v1 kind: Service metadata: name: qdrant-headless namespace: ai-platform spec: clusterIP: None selector: { app: qdrant } ports: - { name: http, port: 6333 } - { name: internal-p2p, port: 6335 } --- apiVersion: apps/v1 kind: StatefulSet metadata: name: qdrant namespace: ai-platform spec: serviceName: qdrant-headless replicas: 3 podManagementPolicy: OrderedReady selector: matchLabels: { app: qdrant } template: metadata: labels: { app: qdrant } spec: containers: - name: qdrant image: qdrant/qdrant:v1.9.0 env: - { name: QDRANT__CLUSTER__ENABLED, value: "true" } - { name: QDRANT__CLUSTER__P2P__PORT, value: "6335" } command: ["/bin/sh", "-c"] args: - | HOST="$(hostname).qdrant-headless" if [ "$(hostname)" = "qdrant-0" ]; then ./qdrant --uri "http://${HOST}:6335" else ./qdrant --bootstrap "http://qdrant-0.qdrant-headless:6335" --uri "http://${HOST}:6335" fi ports: - { name: http, containerPort: 6333 } - { name: p2p, containerPort: 6335 } resources: requests: { cpu: "1000m", memory: 2Gi } limits: { cpu: "2000m", memory: 4Gi } volumeMounts: - { name: qdrant-data, mountPath: /qdrant/storage } volumeClaimTemplates: - metadata: { name: qdrant-data } spec: accessModes: [ReadWriteOnce] resources: requests: { storage: 20Gi } Check the bootstrap flags against the Qdrant version you run, because clustering options have changed between releases.
How many replicas?
Raft-based clusters need a majority of nodes to agree. One and two nodes tolerate no failures, three tolerate one, and five tolerate two. Two replicas are fine for a lab, but for real fault tolerance run three.
Diagram: FIG 4 · A Raft cluster needs a majority of nodes to agree. Three nodes tolerate one failure, five tolerate two, and two nodes tolerate none, so use at least three replicas when availability matters. See the animated version.
Proper StatefulSet design protects your vector indexes through restarts and upgrades, and it gives the rest of the stack a database that behaves predictably.
Run it yourself
This is the exact example from the video. Run against a real K3d cluster. A small nginx image stands in for Qdrant so three pods start quickly.
{apiVersion: v1, kind: Service, metadata: {name: qdrant-headless}, spec: {clusterIP: None, selector: {app: qdrant}}} --- apiVersion: apps/v1 kind: StatefulSet metadata: {name: qdrant} spec: serviceName: qdrant-headless replicas: 3 selector: {matchLabels: {app: qdrant}} template: metadata: {labels: {app: qdrant}} spec: {containers: [{name: qdrant, image: "nginx:1.27-alpine"}]} volumeClaimTemplates: - metadata: {name: data} spec: {accessModes: [ReadWriteOnce], resources: {requests: {storage: 1Gi}}} Run it:
kubectl apply -f statefulset.yaml Output I got while recording the video:
qdrant-0 Running qdrant-1 Running qdrant-2 Running data-qdrant-0 Bound data-qdrant-1 Bound data-qdrant-2 Bound headless clusterIP: None Coming Up Next
Day 12: Auto-scaling AI inference with Horizontal Pod Autoscalers for spiky API traffic.
StatefulSet #VectorDatabase #Qdrant #Milvus #CloudNative #DataEngineering #SystemDesign
Originally published at https://sureshpallapothu.in/blog/ai-int-day-11-statefulsets-vector-db, where this post includes animated diagrams.

