Bundle Certs into kubeconfig and Configure Secret Encryption
At the end of Article 4 we have fifteen cert pairs on the workstation, but no component can use them yet. The reason: a binary like kube-controller-manager, when it starts, needs to know three things at once — what address to call the api-server at, which CA to trust, and which client cert to present. Bundling those three things into one file is the job of kubeconfig. This article generates a kubeconfig for each component, then creates the Secret encryption config before we move on to bootstrapping.
What a kubeconfig contains
A kubeconfig file has three blocks, and once you understand these three, reading any kubeconfig later is easy:
┌─ clusters ──── where the api-server is + which CA to trust it with
│ (server: https://...:6443, certificate-authority-data)
│
├─ users ─────── my identity: client cert + client key
│ (client-certificate-data, client-key-data)
│
└─ contexts ──── pairs a cluster with a user (+ namespace) into a
named "context", and indicates the context in use
Each component (controller-manager, scheduler, kubelet...) will have its own kubeconfig, where the users block contains exactly the cert carrying its identity. When it calls the api-server, mTLS happens: it presents the cert from the kubeconfig, the api-server reads the CN/O as identity, then RBAC authorizes. All bundled in one file.
We embed the cert directly into the file (--embed-certs=true) rather than have the file point at external cert paths. That way each kubeconfig is a self-contained file, copy it to a node and it works, without dragging separate cert files along.
Two api-server addresses: 127.0.0.1 and the load balancer
Before generating the files, we need to decide one thing: which address each component points at for the api-server. There are two groups, with different reasons:
controller-manager ┐
scheduler ├─► https://127.0.0.1:6443 (api-server RIGHT ON that machine)
admin (on the node) ┘
kube-proxy ┐
kubelet ┴─────────► https://10.0.1.10:6443 (via the load balancer)
controller-manager and scheduler run on the same machine as an api-server (all three sit on a controller). Letting them call 127.0.0.1 is the cleanest and doesn't depend on the load balancer — each controller handles its own part. kubelet and kube-proxy, meanwhile, sit on the workers, with no api-server locally, so they go through the load balancer at 10.0.1.10 to be spread evenly across the three api-servers and to tolerate one api-server dying.
(That load balancer we've only specified the address for, not built yet — HAProxy comes up in Article 9. Only then do the workers really need it; the control plane uses 127.0.0.1 so it can bootstrap before the load balancer exists.)
Step 1 — kubeconfigs for the control plane
kubectl config is the tool that generates kubeconfigs. Each file needs four commands: declare the cluster, declare the user, pair them into a context, select the context. To avoid repetition, wrap it in a function (run in the ~/k8s-scratch/pki directory where the certs are):
cd ~/k8s-scratch/pki
LOCAL=https://127.0.0.1:6443
mk() { # $1=file $2=user $3=cert-name $4=server
kubectl config set-cluster k8s-scratch \
--certificate-authority=ca.pem --embed-certs=true \
--server=$4 --kubeconfig=$1
kubectl config set-credentials $2 \
--client-certificate=$3.pem --client-key=$3-key.pem \
--embed-certs=true --kubeconfig=$1
kubectl config set-context default --cluster=k8s-scratch --user=$2 --kubeconfig=$1
kubectl config use-context default --kubeconfig=$1
}
mk admin.kubeconfig admin admin $LOCAL
mk kube-controller-manager.kubeconfig system:kube-controller-manager kube-controller-manager $LOCAL
mk kube-scheduler.kubeconfig system:kube-scheduler kube-scheduler $LOCAL
Note each user matches the exact CN of the corresponding cert created in Article 4: system:kube-controller-manager, system:kube-scheduler. The admin.kubeconfig here points at 127.0.0.1 to use on the controller while bootstrapping RBAC; configuring kubectl from the laptop (pointing at the load balancer's Elastic IP) we do in Article 9 once the cluster is ready to take remote commands.
Step 2 — kubeconfigs for the workers
The two worker components — kube-proxy and kubelet — point at the load balancer. The kubelet gets one file per node because each node carries a different system:node:<node> identity:
LB=https://10.0.1.10:6443
mk kube-proxy.kubeconfig system:kube-proxy kube-proxy $LB
mk worker-0.kubeconfig system:node:worker-0 worker-0 $LB
mk worker-1.kubeconfig system:node:worker-1 worker-1 $LB
Six kubeconfig files now complete. Look at the structure of one file (hiding the base64 cert data) to see the three blocks just described:
grep -E 'server:|name:|cluster:|user:|current-context:' admin.kubeconfig
- cluster:
server: https://127.0.0.1:6443
name: k8s-scratch
cluster: k8s-scratch
user: admin
name: default
current-context: default
- name: admin
clusters points at https://127.0.0.1:6443 with the CA embedded, contexts pairs the cluster k8s-scratch with the user admin, and current-context: default indicates the context in use. Exactly the three-block model.
Step 3 — Secret encryption in etcd
The rest of the article switches to a different topic but still in the "config preparation" phase: encrypting data at rest (encryption at rest).
By default, everything the api-server writes into etcd is plaintext — including Secrets. That means if someone reads etcd's data files, or gets hold of an etcd backup, they read directly every password, token, API key you stored in a Secret. To prevent this, the api-server supports encrypting the data field of certain resource types before writing them down to etcd, using a key we provide.
We declare that in an EncryptionConfiguration file. Generate a random 32-byte key then embed it in the file:
ENC_KEY=$(head -c 32 /dev/urandom | base64)
cat > encryption-config.yaml <<EOF
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aescbc:
keys:
- name: key1
secret: ${ENC_KEY}
- identity: {}
EOF
Two details worth understanding in this file:
resources: [secrets]— only encrypt Secrets. You can add other types, but Secrets are the most sensitive and the main target.- The order of
providersmatters a lot. The api-server uses the first provider to write (encrypt), and tries them top to bottom to read (decrypt). Hereaescbcis first so new Secrets are encrypted with AES-CBC;identity(no encryption) is last so the api-server can still read old Secrets that remain in plaintext. If you later want to rotate the key or turn off encryption, you change this order.
WRITE Secret: api-server ──aescbc(encrypt)──► etcd (use the first provider)
READ Secret: etcd ──► try aescbc, then identity ──► api-server
This file will be placed on the controllers and passed to the api-server via the --encryption-provider-config flag in Article 7. At that point we'll verify it for real: create a Secret then read the raw bytes in etcd to see it's been encrypted, no longer readable by eye.
🧹 Security note
Again there are no cloud resources to clean up, but the pki directory now has more sensitive files: six kubeconfigs (with client keys embedded inside) and encryption-config.yaml (containing the encryption key). Losing encryption-config.yaml means losing the ability to decrypt the encrypted Secrets in etcd — so besides keeping it private, in a real environment you must also back up this key carefully. Still don't commit them to Git.
Wrap-up
The preparation phase is fully done: infrastructure (Article 3), certificates (Article 4), and now kubeconfigs and the encryption config. Everything is still sitting on the workstation; no Kubernetes process is running yet. But from here each article switches on a real component.
Article 6 begins the bootstrap with the foundation of the control plane: etcd. We'll look at why etcd needs an odd number of nodes and how quorum works, then stand up a three-node etcd cluster on the controllers, using exactly the etcd and etcd-ca certs we created. Once etcd is running, we finally have a place for the api-server to write state.