Kubernetes From Scratch
Build a complete Kubernetes cluster by hand — no kubeadm, no scripts — from the first certificate to a real HA cluster, then use that cluster as a lab to deep-dive every Kubernetes concept. Part one: PKI/TLS, etcd, the control plane, workers, pod networking, CoreDNS. Part two: Pods, workload controllers, scheduling, storage, advanced networking (Cilium eBPF), security, extending the API, operations. Each component is both explained from the inside and stood up/configured by hand. Tested for real on AWS EC2 with Kubernetes v1.36; manifests/scripts at github.com/nghiadaulau/kubernetes-from-scratch. Grounded in the official docs at kubernetes.io.
Why Build Kubernetes by Hand, and What We're Going to Build
Series opener: 'from scratch' means creating every certificate by hand, bootstrapping every binary, wiring pod networking yourself — no kubeadm, no scripts. Why this long road makes Kubernetes click faster, how we'll stand up an HA cluster, then use it as a lab to deep-dive every Kubernetes concept.
Kubernetes Architecture, Up Close: Loops, Watches and the API Server
Going deeper than the familiar control plane / node diagram: where the control loop runs, how list-watch lets components coordinate without knowing each other, why everything goes through the api-server, and which stages a kubectl apply command passes through before the container runs.
PKI and TLS: Why a Cluster Needs So Many Certificates
A Kubernetes cluster needs a dozen certificates and three separate CAs. This article explains the PKI/TLS model underpinning every connection in the cluster: who the CA signs for, how two-way mTLS works, and one easily-forgotten point — the CN and O fields in a certificate are the very identity and RBAC group the api-server trusts.
Stand Up Six EC2 Machines and Prepare the OS
The first hands-on article: stand up a dedicated VPC on AWS with six EC2 machines (1 load balancer, 3 controllers, 2 workers), assign fixed private IPs, then prepare the OS — hostname, /etc/hosts, kernel modules, sysctl, disable swap — and install the kubectl and cfssl tooling. All done step by step, run for real.
Sign Every Certificate by Hand with cfssl
The core of 'from scratch': use cfssl to create three CAs (Kubernetes, etcd, front-proxy) then sign all the certificates for each component — apiserver with a full SAN, a kubelet per worker with special CN/O, controller-manager, scheduler, kube-proxy, the etcd client, and the service-account key pair. Each one done individually, each field correct, then verify the trust chain.
Bundle Certs into kubeconfig and Configure Secret Encryption
The certificates are ready, but the Kubernetes binaries read them through a pre-bundled format: kubeconfig. This article generates kubeconfigs for admin, controller-manager, scheduler, kube-proxy and each kubelet — explaining the cluster/user/context model and why the control plane points at 127.0.0.1 while workers go through the load balancer. It ends by creating the Secret-at-rest encryption file for etcd.
etcd: Quorum, Raft, and Standing Up a Three-Node Cluster
The first control plane component, and the foundation for everything above it: etcd. This article explains what etcd stores, why it needs an odd number of nodes, and what quorum means via the Raft algorithm — then stands up a three-node etcd cluster on the controllers with TLS, and verifies the leader and cluster health.
kube-apiserver: The Cluster's Entry Point and the Request Pipeline
The component that sits right in front of etcd and is the cluster's single entry point. This article digs into the authn → authz → admission chain every request must pass through, then stands up kube-apiserver on all three controllers: connecting to etcd over TLS, enabling Secret encryption, and actually verifying a Secret is encrypted by reading the raw bytes in etcd.
controller-manager and scheduler: Control Loops and Leader Election
The two components that turn desired state into action: kube-controller-manager runs dozens of control loops, kube-scheduler picks the node for a pod. This article stands up both on three controllers, explains how the scheduler filters and scores nodes, then watches leader election work for real — three instances running, but only one doing the work.
HAProxy Consolidates Three API Servers, and Remote kubectl
Three api-servers are running, but clients need to know which one to call. This article stands up HAProxy on lb-0 to consolidate them into a single address in TCP passthrough mode — preserving end-to-end mTLS, then configures kubectl on your laptop to point at the Elastic IP and sets up the RBAC so the api-server can call down to the kubelet.
Container Runtime and CRI: Installing containerd on the Workers
The kubelet doesn't run containers itself — it delegates that to a container runtime through a standard interface called CRI. This article explains what CRI is, who does what in the kubelet → containerd → runc stack, then installs containerd v2.3 and runc on the two workers, sets the cgroup driver to match the kubelet, and verifies it all with crictl.
kubelet: Bringing the Workers Into the Cluster
The kubelet is the only process on a worker that talks to the api-server and gives orders to containerd. This article distributes the certificate for each node, writes a KubeletConfiguration pointing at the containerd socket, builds the systemd unit, then watches the two workers register into the cluster — and understands why they show up as NotReady rather than Ready.