CustomResourceDefinition: Add Your Own Kind

K
Kai··5 min read

The first eleven parts used Kubernetes's built-in resources — Pod, Service, Deployment. Part XII goes a different way: extending Kubernetes itself. The natural starting point is CustomResourceDefinition (CRD), a way to add a new kind of object to the API server without modifying or recompiling it. Once a CRD is declared, the API server serves the new kind just like a native one: kubectl get works, it validates, it versions, it stores in etcd.

What a CRD contains

A CRD declares metadata about the new kind: which API group it belongs to, which versions it has, whether its scope is namespaced or cluster, what it's named, and a schema to validate against. Build a Widget kind:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: widgets.kkloud.io          # required: <plural>.<group>
spec:
  group: kkloud.io
  scope: Namespaced
  names: {plural: widgets, singular: widget, kind: Widget, shortNames: [wg]}
  versions:
  - name: v1
    served: true                   # this version is served
    storage: true                  # exactly one version is the storage version
    subresources: {status: {}}     # split /status into its own subresource
    additionalPrinterColumns:
    - {name: Size,  type: integer, jsonPath: .spec.size}
    - {name: Color, type: string,  jsonPath: .spec.color}
    - {name: Phase, type: string,  jsonPath: .status.phase}
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            required: [size, color]
            properties:
              size:  {type: integer, minimum: 1, maximum: 5}
              color: {type: string, enum: [red, green, blue]}
          status:
            type: object
            properties:
              phase: {type: string}

Apply it, and the API server registers the new kind immediately, with kubectl seeing it in the resource list:

kubectl apply -f widget-crd.yaml
kubectl api-resources --api-group=kkloud.io
NAME      SHORTNAMES   APIVERSION     NAMESPACED   KIND
widgets   wg           kkloud.io/v1   true         Widget

widgets is now a real resource, with short name wg, namespaced, kind Widget. No controller installed, no API server restart.

Served dynamically: no apiserver restart

The phrase "no API server restart" is worth pausing on, because it's the core distinction of a CRD. Inside the API server is a controller that watches the CRD objects themselves; when a CRD appears, it dynamically stands up the REST handler for the new group/version (/apis/kkloud.io/v1/...) and refreshes the discovery table + OpenAPI schema — all within the running process. Verify it by comparing the apiserver's start time before and after creating the CRD:

ssh controller-0 'sudo systemctl show kube-apiserver -p ActiveEnterTimestamp --value'
kubectl get --raw /apis/kkloud.io   # before creating the CRD
# ...apply CRD...
kubectl get --raw /apis/kkloud.io/v1
ssh controller-0 'sudo systemctl show kube-apiserver -p ActiveEnterTimestamp --value'
Sat 2026-05-23 17:22:23 UTC                                  # start time (before)
Error from server (NotFound): the server could not find...   # /apis/kkloud.io doesn't exist yet
{"kind":"APIResourceList", ... "name":"widgets","namespaced":true,"kind":"Widget"}   # after: served immediately
Sat 2026-05-23 17:22:23 UTC                                  # start time (after) — IDENTICAL

The /apis/kkloud.io/v1 endpoint goes from NotFound to serving a full APIResourceList, while the apiserver's ActiveEnterTimestamp is unchanged — proving it never restarted, it just plugged in the handler while running. This is why a CRD is light and safe to add: unlike patching apiserver code or aggregation (Article 60, which bolts on a whole separate server), a CRD is just data that the apiserver wires up a serving path for on its own. Validation against openAPIV3Schema also runs right inside the apiserver, at the admission stage (Article 51) — the next section shows it blocking bad data.

Schema validates the custom resource

The openAPIV3Schema part isn't only descriptive — the API server uses it to validate every custom resource. Create a valid Widget:

kubectl -n crd-demo apply -f - <<'EOF'
apiVersion: kkloud.io/v1
kind: Widget
metadata: {name: w1}
spec: {size: 3, color: blue}
EOF
widget.kkloud.io/w1 created

Then try two that violate the schema — size over maximum, and color outside the enum:

# size: 9 (> maximum 5)
# color: purple (outside enum red/green/blue)
The Widget "bad1" is invalid: spec.size: Invalid value: 9: spec.size in body should be less than or equal to 5
The Widget "bad2" is invalid: spec.color: Unsupported value: "purple": supported values: "red", "green", "blue"

The API server rejects them immediately, with a message pointing out exactly which field is wrong and which rule was violated. Validation happens right at the API server, no external code needed — required, minimum/maximum, enum, and data types are all handled by the schema.

Printer columns and the status subresource

additionalPrinterColumns makes kubectl get show exactly the fields you want, instead of just name and age:

kubectl -n crd-demo get wg
NAME   SIZE   COLOR   PHASE
w1     3      blue    

PHASE is empty because no one has set status yet. Declaring subresources: {status: {}} splits status into its own /status endpoint — updated through a different path than spec:

kubectl -n crd-demo patch widget w1 --subresource=status --type=merge -p '{"status":{"phase":"Ready"}}'
kubectl -n crd-demo get wg w1
NAME   SIZE   COLOR   PHASE
w1     3      blue    Ready

Splitting spec and status isn't cosmetic: it allows separate RBAC permissions (a controller can write status, a user only writes spec) — matching the model where spec is the user's desire and status is the reality reported by the controller, which we'll use in Article 59. Custom resources are stored in etcd under their group, alongside native resources:

sudo etcdctl ... get /registry/kkloud.io/widgets/crd-demo/w1 --keys-only
/registry/kkloud.io/widgets/crd-demo/w1

🧹 Cleanup

kubectl delete crd widgets.kkloud.io      # deleting the CRD takes every Widget with it
kubectl delete namespace crd-demo

Deleting the CRD removes the API endpoint and every custom resource of that kind from etcd. Manifests are at github.com/nghiadaulau/kubernetes-from-scratch, directory 57-crd.

Wrap-up

CustomResourceDefinition adds a new kind of object to the API server without modifying or restarting it: declare group, versions (served/storage), scope, names, and openAPIV3Schema. Once applied, kubectl serves that kind like a native resource — we created a valid Widget, and two schema-violating ones (size over max, color outside enum) were rejected immediately by the API server, because validation runs at the API server against the schema. additionalPrinterColumns customizes the kubectl get columns; subresources: {status: {}} splits status into its own endpoint to separate spec/status permissions — the foundation for the controller model in Article 59. Custom resources are stored in etcd under /registry/<group> alongside native resources.

A CRD is only a data structure — creating a Widget makes nothing happen, because no one reacts to it yet. Article 58 adds a link into the write path: the admission webhook, an external service the API server calls to accept, reject, or modify an object before storing it.