Templates and ClusterClass
This page covers the clusterctl templates CAPTF ships, how to generate a
cluster from them, and the noop ClusterClass: what it creates, its
topology variables, and how to patch a module variable onto it. It is for
anyone creating clusters with CAPTF, especially with ClusterClass.
Before you begin
- The provider is installed and registered with
clusterctl(Installation). - A
TerraformClusterIdentityis applied for the namespace you are generating into (Identities and credentials). - Your cluster and machine module images are built, linted and pushed (tfcapi-lint, image contract).
- You know the values for the
clusterctlvariables you need (clusterctl variables).
The shipped flavors
CAPTF ships three flavors as templates/cluster-template*.yaml files;
once the provider is registered, clusterctl generate cluster --infrastructure terraform renders one of them from the registered
repository (Installation):
| Flavor | Template file | What it creates |
|---|---|---|
Default (no --flavor) | cluster-template.yaml | A Cluster, TerraformCluster, KubeadmControlPlane, two TerraformMachineTemplates (control plane and workers), a MachineDeployment with its KubeadmConfigTemplate, and a MachineHealthCheck for each of the control plane and the workers |
clusterclass | cluster-template-clusterclass.yaml | A Cluster with spec.topology.classRef.name: noop, referencing the noop ClusterClass |
libvirt | cluster-template-libvirt.yaml | The same shape as the default flavor, sized for the development host’s libvirt modules: kube-vip on the control plane, and nodes that install containerd and Kubernetes at boot from the version in the machine’s cloud-init metadata rather than a version clusterctl baked in |
Every flavor’s KubeadmControlPlane sets spec.remediation.maxRetry: 3
and retryPeriodSeconds: 300, so a module that fails deterministically
stops being retried instead of churning cloud resources, and its
localAPIEndpoint.bindPort (init and join) equals
Cluster.spec.clusterNetwork.apiServerPort (6443); kubeadm never reads
the Cluster field, so keep the two equal if you change one.
The libvirt flavor is for the project’s development host; see
the libvirt host guide and
the libvirt module.
Its identity template is templates/identity-libvirt.yaml, applied instead
of the default templates/identity.yaml.
Generate a cluster
Set the identity, image and sizing variables, then generate and apply. Default flavor:
export TERRAFORM_IDENTITY_NAME=<identity-name> \
TERRAFORM_CLUSTER_IMAGE=<cluster-module-image> \
TERRAFORM_MACHINE_IMAGE=<machine-module-image>
clusterctl generate cluster <cluster-name> --infrastructure terraform \
--target-namespace <namespace> \
--kubernetes-version <version> \
--control-plane-machine-count <count> --worker-machine-count <count> \
| kubectl apply -f -
ClusterClass flavor: enable the ClusterTopology feature gate when you
register the provider (Installation);
it is alpha and off by default. Apply the class to the namespace once, then
generate with --flavor clusterclass:
kubectl apply -n <namespace> -f templates/clusterclass-noop.yaml
clusterctl generate cluster <cluster-name> --infrastructure terraform \
--flavor clusterclass --target-namespace <namespace> \
--kubernetes-version <version> \
--control-plane-machine-count <count> --worker-machine-count <count> \
| kubectl apply -f -
clusterclass-noop.yaml carries no clusterctl variables of its own and no
namespace, so apply it once to every namespace that generates clusters from
it. Neither flavor defaults CLUSTER_NAME, KUBERNETES_VERSION,
CONTROL_PLANE_MACHINE_COUNT, WORKER_MACHINE_COUNT, the two image
variables or the identity name: a missing one fails clusterctl generate
instead of deploying something unintended. See
clusterctl variables for the full
list, including the libvirt flavor’s defaults.
The noop ClusterClass
templates/clusterclass-noop.yaml defines ClusterClass noop and the
templates it references:
TerraformClusterTemplate/noop, the infrastructure template.KubeadmControlPlaneTemplate/noop-control-planeandTerraformMachineTemplate/noop-control-plane, the control plane and its infrastructure.TerraformMachineTemplate/noop-workerandKubeadmConfigTemplate/noop-worker, the infrastructure and bootstrap for thedefault-workermachine deployment class.
Each of the three Terraform templates carries the placeholder image
example.invalid/captf/set-by-clusterclass:unset: a Cluster that fails to
patch a real image gets a failed image pull instead of running an
unintended module. The class also carries default MachineHealthCheck
timeouts for the control plane and the workers; the clusterclass flavor
overrides them through Cluster.spec.topology with the same variables as
cluster-template.yaml (see
clusterctl variables).
The class declares three required string topology variables —
identityName, clusterImage and machineImage — described in
the ClusterClass topology variables table.
Three patches apply them to the templates above:
identitysetsspec.template.spec.identityRefonTerraformClusterTemplate/nooptoidentityName, and also setsspec.template.spec.defaults.identityRefto the same value, so machines without their ownidentityRefinherit it.clusterImagereplacesspec.template.spec.source.imageonTerraformClusterTemplate/noopwithclusterImage.machineImagereplacesspec.template.spec.source.imageon bothTerraformMachineTemplate/noop-control-planeandTerraformMachineTemplate/noop-workerwithmachineImage, matched bymatchResources.controlPlaneandmatchResources.machineDeploymentClass.names: [default-worker].
cluster-template-clusterclass.yaml sets the three variables from
TERRAFORM_IDENTITY_NAME, TERRAFORM_CLUSTER_IMAGE and
TERRAFORM_MACHINE_IMAGE.
Patch a module variable through ClusterClass
A ClusterClass patch can also turn a topology variable into a module
variable (spec.template.spec.variables;
module variables). Declare the topology variable, then add a
patch whose jsonPatches write /spec/template/spec/variables (or one key
under it, if the template already sets others):
spec:
variables:
- name: workerInstanceType
required: true
schema:
openAPIV3Schema:
type: string
patches:
- name: workerInstanceType
definitions:
- selector:
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachineTemplate
matchResources:
machineDeploymentClass:
names: [default-worker]
jsonPatches:
- op: add
path: /spec/template/spec/variables
valueFrom:
template: |
instance_type: {{ .workerInstanceType }}
The same patch on TerraformClusterTemplate reaches the TerraformCluster
instead, whose spec.variables is mutable: changing the variable
re-applies the cluster module, and a destructive plan still waits for
approval (plan approval). A ConfigMap or Secret
named in variablesFrom is not part of the ClusterClass: create it,
labeled captf.io/variables=true, in each namespace that uses the class
(module variables).
Template immutability and rolling out a change
A *Template kind’s spec.template.spec cannot be edited in place once
created; only its metadata can change (see
mutability per kind). ClusterClass topology
reconciliation is exempt from that rule for its own dry-runs, which is how
patching works at all, but a direct edit to a *Template object is
rejected.
Two situations follow from this:
- A field the class exposes as a topology variable. Changing the
variable’s value on a Cluster’s
spec.topology.variablesis enough: since the resultingTerraformMachineTemplatewould differ from the one already referenced, and templates are immutable, the topology controller creates a newTerraformMachineTemplatewith the patched spec and rolls the affectedMachineDeploymentor control plane onto it. ChangingmachineImageon aclusterclass-flavor Cluster works this way. - A field the class does not expose as a variable, such as a
KubeadmControlPlaneTemplatefield or a new patch. Create a new template object under a new name with the desiredspec.template.spec, then update theClusterClass’s reference to it —spec.infrastructure.templateRef,spec.controlPlane.templateRef,spec.controlPlane.machineInfrastructure.templateRef, or the matchingworkers.machineDeployments[].infrastructure.templateRef/bootstrap.templateRef— and apply the class. Topology reconciliation then rolls every Cluster on the class onto the new template, subject to each machine deployment’s or control plane’s own rollout strategy (thenoopclass’s control plane setsmaxSurge: 0; itsdefault-workermachine deployment class leaves the rollout strategy unset, so the defaultmaxSurge: 1applies).
A TerraformCluster itself is mostly mutable — only
spec.controlPlaneEndpoint is immutable once it has a host — so a
clusterImage or identityName change re-applies the cluster module in
place rather than creating a new object.