Sources: CD's devops wiki
03-kubernetes/08-lab-webapp-mongodb.md(minikube, 4 YAML files: ConfigMap, Secret, Mongo, WebApp), cross-checked against kubernetes.io, the Flare/Security Affairs research on exposed MongoDB (2026-02-02), GitGuardian's State of Secrets Sprawl 2026 and MongoDB Atlas pricing on 2026-10-28. Lab names only; screens are drawings.
An intern at Contoso follows the classic tutorial: a Node.js web app and a MongoDB, four YAML files, minikube start, it works. Two shortcuts make it into the real cluster a month later:
- The MongoDB Service is
NodePort, "so I can connect Compass from my laptop". - The Secret's base64 values are committed to the repo with the manifests, because the tutorial showed them that way.
Ten days after the cluster gets a public node pool, the orders database contains one collection, README, with one document: "Your data is backed up. Send 0.005 BTC to restore it." Nobody exploited anything. A port was open and a password was public.
This is not a story I made up for effect. Flare's research published on 2026-02-02 found 3,100 MongoDB servers fully exposed to the internet, and 1,416 of them (45.6%) already wiped and replaced with that exact note, almost all demanding about $500 in Bitcoin to one wallet. GitGuardian counted 28.65 million new hard-coded secrets pushed to public GitHub in 2025, up 34% in a year, and 64% of the credentials they found valid in 2022 were still valid in January 2026.
TL;DR
- Problem: the two-tier app is where most people first put a password in a Secret and a database behind a Service. Both are easy to get subtly wrong: a Secret is base64, not encryption; a NodePort is a hole in every node; a flat network lets any pod reach the database.
- Who it hurts: customers (the data is gone), the company (ransom, disclosure, fines), the intern (blamed for a tutorial), security (an incident with no CVE to point at).
- Fix: database on
ClusterIPonly, a StatefulSet with a disk, Secrets created outside git (or synced from a vault), a NetworkPolicy that lets only the web app talk to the database, and a backup. Four files become five, and none of them contains a password. - Cost: on minikube, $0. In production, a 128 GiB disk is about $10 a month; MongoDB Atlas starts at $0 (M0) and $56.94 a month (M10) if you would rather not run it; Azure Key Vault secrets cost $0.03 per 10,000 operations. The ransom is $500 and you still do not get the data.
- Prove it:
base64 -dthe Secret (it is readable), scan the cluster from outside (27017 is closed),ncfrom a random pod (times out), from the web app (connects), delete the database pod (data still there).
Table of contents
- 1. The problem
- 2. Who gets hurt
- 3. Cost and value
- 4. The lab, rebuilt safely: five files
- 5. Secrets: what a Kubernetes Secret is and is not
- 6. Prove it
- Gotchas
- Checklist
- What changed recently
- Sources
1. The problem
The tutorial architecture, from my notes, is right:
Browser → webapp-service (NodePort) → WebApp pod → mongo-service (ClusterIP) → MongoDB pod Five objects: 2 Deployments, 2 Services, 1 ConfigMap, 1 Secret. What goes wrong is never the architecture. It is three defaults nobody questioned:
| Shortcut | Why it felt fine | What it actually does |
|---|---|---|
mongo-service as NodePort | "Only for my laptop" | Opens 27017 (or 30017) on every node's IP. With a public node pool, on the internet |
| Secret values in git | "They're base64, that's encoded" | base64 is a text format, not encryption. `echo bW9uZ291c2Vy \ |
| No NetworkPolicy | "The database is inside the cluster" | Every pod in every namespace can open 27017 to it. Including the compromised one |
2. Who gets hurt
| Who | What they lose |
|---|---|
| Customers | Orders, accounts, documents. The "backup" in the ransom note does not exist in most of these attacks |
| The company | Disclosure duties, fines under data-protection law, a ransom that buys nothing |
| The intern | A tutorial copied faithfully becomes "the person who leaked the password" |
| Security team | An incident with no CVE, no exploit and no malware: a port and a text file. Hard to explain to an auditor, hard to detect with an EDR |
| Everyone who reused that password | GitGuardian: most leaked credentials stay valid for years |
3. Cost and value
What the safe version costs (checked 2026-10-28):
| Item | Lab (minikube) | Small production |
|---|---|---|
| MongoDB on the cluster (StatefulSet + 128 GiB Standard SSD) | $0 | about $9.60 a month for the disk, node capacity you already pay for |
| MongoDB Atlas instead (managed, backups, TLS, auth by default) | M0 free, 512 MB | Flex $8 to $30 a month; M10 $56.94 a month (2 vCPU, 2 GB RAM) |
| Secrets in Azure Key Vault, synced by External Secrets Operator | $0 | $0.03 per 10,000 secret operations; the operator is open source (v2.14) |
| NetworkPolicy, ClusterIP, backups via CronJob (last week's post) | $0 | $0 plus the backup storage |
What the shortcut costs: $500 in Bitcoin that restores nothing, plus whatever the data was worth, plus the disclosure. Flare's numbers suggest one actor collected up to $842,000 this way from servers that paid.
Value: the fix adds one file (the NetworkPolicy), removes one line ({% raw %}type: NodePort on the database), and moves the Secret out of git. The architecture, images and app code do not change. This is the cheapest security work you will ever do.
4. The lab, rebuilt safely: five files
Namespace contoso-shop. Images pinned: mongo:8.0 and the tutorial's demo app nanajanashia/k8s-demo-app:v1.0 (it reads USER_NAME, USER_PWD, DB_URL).
1. mongo-config.yaml, the ConfigMap (non-secret):
apiVersion: v1 kind: ConfigMap metadata: name: mongo-config namespace: contoso-shop data: mongo-url: mongo-service # the Service name; DNS does the rest 2. The Secret: created, not committed. Nothing with a password goes in the repo:
kubectl create secret generic mongo-secret -n contoso-shop \ --from-literal=mongo-user=app_orders \ --from-literal=mongo-password="$(openssl rand -base64 24)" In the repo, instead of mongo-secret.yaml, an ExternalSecret that pulls the values from a vault (section 5). For the lab, the command above is enough.
3. mongo.yaml, the database: a StatefulSet, a headless Service, ClusterIP only:
apiVersion: v1 kind: Service metadata: name: mongo-service namespace: contoso-shop spec: clusterIP: None # headless; no NodePort, no LoadBalancer. Ever. selector: { app: mongo } ports: [{ port: 27017, targetPort: 27017 }] --- apiVersion: apps/v1 kind: StatefulSet metadata: name: mongo namespace: contoso-shop spec: serviceName: mongo-service replicas: 1 selector: matchLabels: { app: mongo } template: metadata: labels: { app: mongo } spec: containers: - name: mongo image: mongo:8.0 ports: [{ containerPort: 27017 }] env: - name: MONGO_INITDB_ROOT_USERNAME valueFrom: { secretKeyRef: { name: mongo-secret, key: mongo-user } } - name: MONGO_INITDB_ROOT_PASSWORD valueFrom: { secretKeyRef: { name: mongo-secret, key: mongo-password } } volumeMounts: - name: data mountPath: /data/db volumeClaimTemplates: - metadata: { name: data } spec: accessModes: [ReadWriteOnce] resources: { requests: { storage: 10Gi } } 4. webapp.yaml, the app: a Deployment and the only NodePort in the namespace:
apiVersion: apps/v1 kind: Deployment metadata: name: webapp namespace: contoso-shop spec: replicas: 2 selector: matchLabels: { app: webapp } template: metadata: labels: { app: webapp } spec: containers: - name: webapp image: nanajanashia/k8s-demo-app:v1.0 ports: [{ containerPort: 3000 }] env: - name: USER_NAME valueFrom: { secretKeyRef: { name: mongo-secret, key: mongo-user } } - name: USER_PWD valueFrom: { secretKeyRef: { name: mongo-secret, key: mongo-password } } - name: DB_URL valueFrom: { configMapKeyRef: { name: mongo-config, key: mongo-url } } --- apiVersion: v1 kind: Service metadata: name: webapp-service namespace: contoso-shop spec: type: NodePort selector: { app: webapp } ports: [{ port: 3000, targetPort: 3000, nodePort: 30100 }] 5. netpol.yaml, the lock (needs a CNI that enforces: minikube start --cni=calico):
apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: mongo-only-from-webapp namespace: contoso-shop spec: podSelector: matchLabels: { app: mongo } policyTypes: [Ingress] ingress: - from: - podSelector: matchLabels: { app: webapp } ports: [{ port: 27017 }] kubectl apply -f mongo-config.yaml -f mongo.yaml -f webapp.yaml -f netpol.yaml kubectl get all,pvc,netpol -n contoso-shop minikube service webapp-service -n contoso-shop # opens the app in the browser 5. Secrets: what a Kubernetes Secret is and is not
A Secret is: a separate object with its own RBAC (you can let developers read ConfigMaps but not Secrets), mounted as env or file, not baked into the image, not in .env on a node.
A Secret is not encrypted by default. In the API it is base64. Anyone with get secrets in the namespace, or a copy of the manifest, or a backup of etcd, has the password:
kubectl get secret mongo-secret -n contoso-shop -o jsonpath='{.data.mongo-password}' | base64 -d Three steps up, in the order a small team should take them:
- Keep Secrets out of git. Create them with
kubectl create secretfrom a CI variable, or a sealed/encrypted form (Sealed Secrets, SOPS). The repo holds the shape, never the value. - Encrypt at rest. Managed clusters (AKS, EKS, GKE) encrypt etcd with a provider key; AKS can use a Key Vault key you own. On kubeadm, configure an
EncryptionConfiguration. - Pull from a vault. The External Secrets Operator (open source, v2.14) reads Azure Key Vault, AWS Secrets Manager or HashiCorp Vault and writes the Kubernetes Secret for you, on a refresh interval. The repo contains this instead of the password:
apiVersion: external-secrets.io/v1 kind: ExternalSecret metadata: name: mongo-secret namespace: contoso-shop spec: refreshInterval: 1h secretStoreRef: { name: kv-contoso-lab, kind: ClusterSecretStore } target: { name: mongo-secret } data: - secretKey: mongo-user remoteRef: { key: mongo-user } - secretKey: mongo-password remoteRef: { key: mongo-password } Rotate the password in Key Vault; within an hour every pod that restarts gets the new one. Nothing in git changed.
6. Prove it
# 1. The Secret is readable to anyone with RBAC: know this, do not be surprised by it kubectl get secret mongo-secret -n contoso-shop -o jsonpath='{.data.mongo-user}' | base64 -d; echo # 2. From outside: nothing on 27017 / 30017 on any node nmap -p 27017,30017,30100 $(minikube ip) # 27017 closed, 30017 closed, 30100 open (the app) # 3. From a random pod: timeout. From the web app: connected kubectl run probe -n contoso-shop --rm -it --image=busybox:1.36 -- nc -zv -w 3 mongo-service 27017 kubectl exec -n contoso-shop deploy/webapp -- nc -zv -w 3 mongo-service 27017 # 4. The data outlives the pod kubectl exec -n contoso-shop mongo-0 -- mongosh -u "$U" -p "$P" --eval 'db.getSiblingDB("orders").t.insertOne({ok:1})' kubectl delete pod mongo-0 -n contoso-shop kubectl exec -n contoso-shop mongo-0 -- mongosh -u "$U" -p "$P" --eval 'db.getSiblingDB("orders").t.countDocuments()' # 1 # 5. The repo holds no secret git grep -nE 'bW9uZ2|password:' -- '*.yaml' # nothing Gotchas
-
NodePorton a database is never "just for me". Usekubectl port-forward svc/mongo-service 27017:27017for Compass. It lives as long as your terminal. -
stringDatain a manifest is still a secret in git. It is base64 only after apply; before that it is plain text in your history forever. -
kubectl create secretleaves the value in shell history. Use--from-fileor--from-env-filefrom a file you delete, or let the operator do it. - The demo app's
DB_URLexpects a host name, notmongodb://.mongo-serviceresolves because both are in the same namespace. - NetworkPolicy without Calico/Cilium does nothing. Test 3 above must time out, or you have no policy.
- Enabling auth on an existing MongoDB with data needs the user created first; the
MONGO_INITDB_*variables only run on an empty volume. - A StatefulSet with one replica is not HA; it is durable. For HA, a MongoDB replica set (3 pods) or Atlas.
- Backups. A volume is not a backup. A CronJob with
mongodumpto object storage, tested by restoring, is.
Checklist
- [ ] Database Service is
ClusterIP/headless;kubectl get svc -A | grep NodePortshows only web front ends - [ ] No Secret manifest with values in git; Secrets created by CI or synced by External Secrets
- [ ] etcd encryption at rest on (managed cluster default, or
EncryptionConfiguration) - [ ] NetworkPolicy: database accepts only the app, on one port; probe pod times out
- [ ] Database is a StatefulSet with a PVC; delete the pod, data remains
- [ ] Backup CronJob, restore tested once
- [ ]
git grepfor base64 patterns andpassword:returns nothing - [ ] RBAC: who has
get secretsin production namespaces, reviewed
What changed recently
Checked on 2026-10-28:
- My lab notes (minikube, 4 YAML files) still work unchanged; this post adds the NetworkPolicy, moves the Secret out of git and turns the database into a StatefulSet.
mongo:8.0replaces the tutorial'smongo:5.0; the Docker Hub tags now go up to 8.3 and 9.0. - Exposed MongoDB is a live campaign, not history: Flare / Security Affairs, 2026-02-02: 3,100 fully exposed servers, 1,416 already wiped, about $500 per ransom, one wallet in 98% of cases. The 2017 wave (27,000 servers in a weekend) never really ended.
- GitGuardian, State of Secrets Sprawl 2026: 28.65 million new secrets on public GitHub in 2025 (+34%), 64% of 2022's valid credentials still valid in January 2026, internal repos 6 times more likely to contain hard-coded secrets than public ones.
- External Secrets Operator is at 2.14 (2.6 was June 2026); the API group is
external-secrets.io/v1. - MongoDB Atlas: M0 free (512 MB), Flex $8 to $30, M10 $56.94 a month, prices re-checked by a third party on 2026-09-20.
Sources
- Kubernetes Secrets, Encrypting data at rest, Network Policies
- Hackers exploit unsecured MongoDB instances to wipe data and demand ransom (Security Affairs, 2026-02-02)
- The State of Secrets Sprawl 2026 (GitGuardian)
- External Secrets Operator, MongoDB Atlas pricing (figures via BudgetForge, 2026-09-20)
- Azure Key Vault pricing
- My lab notes: devops-knowledge-base-and-labs, 03-kubernetes/08-lab-webapp-mongodb.md
Thanks for reading! I'm Cuong, and on CD Security Lab I rebuild real Microsoft security cases in a safe lab and explain them simply. Follow along:
- ▶️ YouTube (daily Shorts + explainers): CD Security Lab
- 💼 LinkedIn: Cuong Do
- 📘 Facebook: CD Security Lab
- 📸 Instagram: @cdsecuritylab
- 📝 More posts like this: Dev.to
Got a question about your own tenant? Drop it in the comments.





