Kubernetes Custom Resources Explained (CRDs, Controllers, & Operators)

kubernetes-custom-resources.md

Kubernetes Custom Resources Explained (CRDs, Controllers, & Operators)

Associated Thunder episode: Kubernetes Custom Resources Explained (CRDs, Controllers, & Operators)

---

Kubernetes Resource Definition

A Kubernetes resource definition is a schema definition of a resource that is stored in etcd. Nothing is running.

A custom resource definition (CRD) is a schema definition that is not installed as part of the Kubernetes cluster. Again, nothing is running.

Two types of CRDs:

• Vendor/product CRDs — ex: Prometheus, Dapr, OTel...
• Company-specific CRDs — completely unique to a business

A custom resource (CR) is a configuration that complies with the associated CRD.

---

Examples of K8s Resources (not custom)

ConfigMap

• Resource definition is part of the K8s installation SCHEMA
• The resource itself is an entry in etcd — DATA that adheres to the resource SCHEMA
• (Nothing being created in the cluster for ConfigMap)

Pod

• Pod resource definition installed as part of K8s SCHEMA
• The resource itself is an entry in etcd — DATA that adheres to the Pod resource schema
• Controller is watching and creates a running Pod when a resource is created

---

Custom Resource (company-specific)

• Install CRD SCHEMA
• The CR itself is an entry in etcd — DATA that adheres to the resource schema

If the CR requires it...

• The CR-associated controller(s) gets called and does its thing, which usually results in physical stuff — i.e. a running Pod

---

Controller

A controller is a piece of software that takes in a resource from kube-api (custom or otherwise) and does its thing (whatever that is) and returns status to kube-api. It runs as a reconciliation loop.

---

Operator

An operator is a CRD + associated controller + any other dependencies (i.e. RBAC: Role Based Access Control) and bundles it into one unit.

Usually associated with lifecycle management of an application (create, read, update, delete).

---

Admission Controller

An admission controller is a type of controller: a piece of software that runs based on kube-api triggers.

As part of the admission process (before storing the resource in etcd), two things can happen:

• VALIDATION — Is resource or CR allowed? Yes or No
• MUTATION — Can edit resource or CR before stored in etcd

Sometimes called "webhooks."

Validation admission controllers can add more complex schema validation rules, or permission rules, etc.

Mutation admission controllers might be used to add labels, for example, or sidecars.