Blog
Thoughts on engineering, design, and building great products.
CustomResourceDefinition: Add Your Own Kind
Part XII shifts from using Kubernetes to extending it. The first article is CustomResourceDefinition — declare a new kind of object, and the API server immediately serves it like a native resource: kubectl get works, it validates against a schema, it stores in etcd. We build a Widget CRD with type and value-range constraints, create a valid custom resource, watch two invalid ones get rejected, then update status through a separate subresource.
Secrets, the Detour and Hardening
Part XI closes out at Secrets and the holes still left. We read etcd directly to confirm Secrets are encrypted at-rest since Article 5, then build a real detour: a ServiceAccount with no permission to read a Secret still extracts its value by creating a pod that mounts that Secret and reading the log. The article ends with a table of hardening steps for a self-built cluster — which are done in the series, which are still missing.
Seccomp, AppArmor and Capabilities
Article 54 made pods declare runAsNonRoot, drop ALL capabilities, seccomp RuntimeDefault — but that's only Kubernetes-level policy. This article goes to the kernel layer to see what they actually do: read /proc/self/status from two pods, one default and one hardened, comparing CapEff, Seccomp, NoNewPrivs, AppArmor. Then prove by hand that dropping a capability blocks a specific operation — chown is denied even when the container still runs as root.
Pod Security Standards and Admission
RBAC decides who can create a pod, not what that pod asks for. A pod running privileged or borrowing hostNetwork is an escape hatch onto the node. Pod Security Admission blocks it at creation: one label on a namespace, the API server measures the pod against three levels — privileged/baseline/restricted — and rejects violators. This article turns restricted on for a namespace, watches a plain pod get kicked out, writes a compliant pod that runs, then tries warn mode.
ServiceAccount and Bound Tokens
Articles 51–52 used ServiceAccount without dissecting it. This article gets into the mechanism: every namespace has a default SA, the kubelet auto-injects a short-lived token via a projected volume, and that token is bound to the exact pod and node. To prove it's truly bound, we grab the token in a running pod, call the API successfully, then delete the pod — the old token immediately becomes 401. Plus how to turn off auto-mount and read the JWT claims.
RBAC: Turning Identity Into Permission
Article 51 stopped where the API server knows who you are. RBAC answers the rest: what may you do. This article stands up a ServiceAccount that can only read pods in one namespace, verifies it with both kubectl auth can-i and a real token — it lists pods but reading secrets or creating pods returns 403. Then we see how a RoleBinding points at a built-in ClusterRole to scope permission to one namespace, and why the view ClusterRole deliberately can't read secrets.
Authentication and the Path Into the API Server
Every kubectl command is an HTTPS request to the API server, and before it touches data it must pass three stages: authentication, authorization, admission. This article opens Part XI with the first stage — the API server figuring out who you are. We examine the three ways our self-built cluster authenticates a request: client certificate (the one admin.kubeconfig uses), ServiceAccount token, and anonymous request — using kubectl auth whoami and real commands on the cluster.