Blog
Thoughts on engineering, design, and building great products.
Full Teardown and Wrap-up
The final article. The EC2 cluster has done its job, and leaving it running burns money every hour. This article tears down all the infrastructure in order — terminate the six instances (taking their disks with them), release the Elastic IP, remove the IAM role — with a real cost table for three choices: keep running, stop to save it, or delete entirely. Then a look back: from a single self-signed certificate to a fully operational HA Kubernetes cluster, built by hand.
Node Log Query and Fine-grained Kubelet Authorization
Article 65 viewed system logs by SSHing into each node to run journalctl. v1.36 lets you query those logs straight through the kubelet API, no SSH. And it comes with a security change: kubelet API access, previously lumped under nodes/proxy, is now split per endpoint — letting you grant exactly nodes/metrics to a monitoring agent without handing over logs or exec. The final article of Part XIV, both touching components we built in Part I.
The New Storage of v1.36
Part IX built PV/PVC, StorageClass, EBS CSI, snapshots. v1.36 adds three storage pieces that just went stable, and a from-scratch cluster can try two of them right away: mounting an OCI image's content as a volume, and changing the IOPS/throughput of an EBS volume in use without recreating it — watching the change apply straight to AWS. The third, VolumeGroupSnapshot, is a different lesson: a feature being GA in Kubernetes doesn't mean every CSI driver can do it.
In-place Pod Resize
Throughout the series, changing a container's resources meant recreating the pod. In-place pod resize breaks that: adjust a running pod's CPU/memory without a restart, via the resize subresource. This article resizes a real pod, then inspects cgroup v2 on the node changing in place with restartCount still 0 — the 'no disruption' counterpart to Article 40's vertical scaling — and two constraints: you can't change QoS, and why memory needs its own resizePolicy.
Admission Policy with CEL
Article 58 built an admission webhook — a separate HTTPS service with a cert and a server to keep alive. From v1.36, most of that need can be met without any server: ValidatingAdmissionPolicy and MutatingAdmissionPolicy write rules in CEL right inside the API server. This article opens Part XIV — features that just graduated in v1.36 — by blocking :latest images and auto-injecting a pod label, entirely with policy objects, not a line of server.
Leader Election, Addons, and Node Autoscaling
The cluster runs three control planes, but controller-manager and scheduler keep only one instance active at a time — if all three acted, they'd step on each other. This article looks at the mechanism that prevents that: leader election via a Lease, and proves real failover by taking the leader down and watching another controller win the lock. Then it closes Part XIII with addons and node autoscaling — adding and removing machines by load.
Metrics, Traces, and API Priority and Fairness
Logs are for discrete events; metrics are for continuous measurements. This article inspects the Prometheus-format /metrics endpoint the apiserver and kubelet expose, then digs into API Priority and Fairness — the apiserver's mechanism for splitting request bandwidth into priority levels so one misbehaving client can't starve the rest. We look at the built-in FlowSchema and PriorityLevelConfiguration, and each level's live state via the debug endpoint.