ConfigMap and Secret

Part VI moves into configuration and policy, and the first brick is the most basic thing every application needs: configuration. Baking APP_MODE=production or a database password straight into the image is taboo: changing config means rebuilding the image, and the password sits exposed in a layer. Kubernetes separates configuration into two objects: ConfigMap for non-confidential data, Secret for sensitive data, then injects it into the pod at runtime. This article digs into both in parallel because they're nearly twins, differing only on "sensitive".

ConfigMap: ordinary configuration

The docs: "A ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume." Two limits to remember right away: "ConfigMap does not provide secrecy or encryption. If the data ... are confidential, use a Secret" and "The data stored in a ConfigMap cannot exceed 1 MiB."

apiVersion: v1
kind: ConfigMap
metadata: {name: appcfg}
data:
  APP_MODE: "production"
  MAX_CONN: "100"
  app.properties: |        # multi-line value -> good as a config file
    color=blue
    retries=3

Secret: sensitive configuration

The docs: "A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key ... Secrets are similar to ConfigMaps but are specifically intended to hold confidential data." Declared with stringData (Kubernetes auto-encodes to base64 stored under data):

apiVersion: v1
kind: Secret
metadata: {name: appsecret}
type: Opaque                # default type for user-defined data
stringData:
  DB_PASSWORD: "s3cr3t-pa$$w0rd"
  api.key: "AKIA-FAKE-1234567890"

type: Opaque is the generic type; there are also specialized types (kubernetes.io/tls for a TLS key pair, kubernetes.io/dockerconfigjson for image-pull credentials, kubernetes.io/service-account-token...). Look at data after creation:

kubectl get secret appsecret -o jsonpath='{.data}'
kubectl get secret appsecret -o jsonpath='{.data.DB_PASSWORD}' | base64 -d

{"DB_PASSWORD":"czNjcjN0LXBhJCR3MHJk","api.key":"QUtJQS1GQUtFLTEyMzQ1Njc4OTA="}

s3cr3t-pa$$w0rd

This is the point you must burn in: czNjcjN0... is only base64, anyone can base64 -d it back to s3cr3t-pa$$w0rd in a second. The docs say it plainly: "Kubernetes Secrets are, by default, stored unencrypted in the API server's underlying data store (etcd)." base64 is character encoding, not security encryption. A Secret beats a ConfigMap not because it hides better, but because it's treated as sensitive (not printed to logs/describe, can have at-rest encryption enabled, separate RBAC controls).

Four ways to inject into a pod

The docs list four consumption paths: in command/args, as environment variables, as files in a volume, or read through the API. One pod uses the first three for both ConfigMap and Secret:

spec:
  containers:
  - name: c
    image: busybox:1.36
    command: ["sh","-c","sleep 3600"]
    env:
    - name: MODE_FROM_CM
      valueFrom: {configMapKeyRef: {name: appcfg, key: APP_MODE}}     # 1 CM key
    - name: PW_FROM_SECRET
      valueFrom: {secretKeyRef: {name: appsecret, key: DB_PASSWORD}}  # 1 Secret key
    envFrom:
    - configMapRef: {name: appcfg}        # ALL CM keys become env
    volumeMounts:
    - {name: cfgvol, mountPath: /etc/cfg, readOnly: true}
    - {name: secvol, mountPath: /etc/sec, readOnly: true}
  volumes:
  - name: cfgvol
    configMap: {name: appcfg}
  - name: secvol
    secret: {secretName: appsecret}

Verify the env:

kubectl exec cfg-pod -- sh -c 'echo "$MODE_FROM_CM | $PW_FROM_SECRET | $APP_MODE | $MAX_CONN"'

production | s3cr3t-pa$$w0rd | production | 100

MODE_FROM_CM/PW_FROM_SECRET pull exactly one key (configMapKeyRef/secretKeyRef); APP_MODE and MAX_CONN come from envFrom — gathering every ConfigMap key into an env var of the same name. Verify the volume:

kubectl exec cfg-pod -- sh -c 'ls /etc/cfg; cat /etc/cfg/app.properties; ls /etc/sec; cat /etc/sec/api.key'

APP_MODE  MAX_CONN  app.properties
color=blue
retries=3

DB_PASSWORD  api.key
AKIA-FAKE-1234567890

Each key becomes a file in the mount directory: app.properties (the multi-line value) becomes a proper config file for the app to read. Secret behaves the same, api.key becomes a file (its contents already base64-decoded — the app reads the real value, not base64).

The key difference: files auto-update, env doesn't

This is the often-missed point that causes production bugs. What happens if you edit a ConfigMap after the pod is already running? The docs draw a sharp line: a volume's "projected keys are eventually updated" whereas "ConfigMaps consumed as environment variables are not updated automatically and require a pod restart." Let's test: change APP_MODE and app.properties then wait for kubelet to sync:

kubectl patch configmap appcfg --type=merge -p '{"data":{"APP_MODE":"staging","app.properties":"color=red\nretries=9\n"}}'
# ... wait ~75 seconds (kubelet sync period) ...
kubectl exec cfg-pod -- sh -c 'echo VOLUME: $(cat /etc/cfg/APP_MODE); cat /etc/cfg/app.properties'
kubectl exec cfg-pod -- sh -c 'echo ENV: $MODE_FROM_CM / $APP_MODE'

VOLUME: staging
color=red
retries=9

ENV: production / production

The contrast is right here: the file in the volume changed by itself to staging/color=red (kubelet periodically refreshes it), but the environment variable stays frozen at production, because env is loaded once at container startup and frozen there. The operational lesson: if the app needs hot config reload, inject via volume and have the app watch the file; if you inject via env, changing the ConfigMap is meaningless until the pod restarts (and a Deployment will not auto-restart just because a ConfigMap changed — which is why people often add a ConfigMap hash to the pod template annotation to force a rollout). (Note the latency: the docs say a volume update takes up to "kubelet sync period + cache propagation delay" — not instant.)

Secret in etcd: at-rest, and our cluster already enabled encryption

The docs warn Secrets are "stored unencrypted in etcd" by default, and advise the first thing to do is "Enable Encryption at Rest". Our cluster did exactly that back in Article 5 (EncryptionConfiguration with provider aescbc). Read etcd directly to see the difference from ConfigMap (plaintext) in Article 30:

sudo etcdctl --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/etcd/etcd-ca.pem --cert=/etc/etcd/etcd.pem --key=/etc/etcd/etcd-key.pem \
  get /registry/secrets/default/appsecret | head -c 60 | tr -c '[:print:]' '.'

/registry/secrets/default/appsecret.k8s:enc:aescbc:v1:key1:.....

The prefix k8s:enc:aescbc:v1:key1: tells us the value behind it is ciphertext, encrypted by the api-server with an AES-CBC key before being written to etcd. Compare with Article 30 where the Deployment showed apps/v1 plaintext: ConfigMap/Deployment are stored in the clear, Secret is encrypted (on this cluster). This is also another warning from the docs, worth remembering: "anyone who is authorized to create a Pod in a namespace can use that access to read any Secret in that namespace". At-rest encryption blocks someone reading the disk/etcd, but anyone who can create a pod in the namespace can still mount the Secret out; full protection also needs RBAC (Part XI).

🧹 Cleanup

kubectl delete pod cfg-pod --now
kubectl delete configmap appcfg
kubectl delete secret appsecret

Objects in the cluster, deleting cleans them up. The cluster returns to two CoreDNS pods. Manifests are at github.com/nghiadaulau/kubernetes-from-scratch, directory 31-configmap-secret.

Wrap-up

Separate configuration from the image with ConfigMap (ordinary data, ≤1 MiB, not secret) and Secret (sensitive). Both inject into a pod four ways, of which we used three: configMapKeyRef/secretKeyRef (one key), envFrom (every key becomes env), and volume (each key a file). The key difference: a volume auto-updates on edit (after ~kubelet sync period) while env freezes at startup, needing a restart, so pick a volume if you need hot reload. A Secret is not fundamentally more encrypted than a ConfigMap: data is only base64 (decoded in a second), it's merely treated more sensitively. Real encryption needs at-rest enabled: our cluster enabled aescbc back in Article 5, so the Secret in etcd shows k8s:enc:aescbc:... (ciphertext) instead of plaintext like a ConfigMap; even so, anyone who can create a pod can read the Secret, so you still need RBAC.

Article 32 digs deeper into resource management from the operational angle: how Kubernetes accounts for node resources, cgroups, and how the management mechanisms touched in Article 22 (requests/limits/QoS) look now from the node and kubelet side.