Introduction

Cluster API Provider Terraform (CAPTF) is a Cluster API infrastructure provider. It provisions and manages cluster infrastructure by running Terraform or OpenTofu modules as Kubernetes Jobs, instead of implementing cloud-specific logic in Go. This page introduces CAPTF’s architecture, then points you at the part of the book you need.

Why CAPTF exists

Cluster API needs one infrastructure provider per cloud, and most teams already have Terraform or OpenTofu modules that provision that cloud’s infrastructure. CAPTF turns a module written to its contract into a Cluster API infrastructure provider directly: you package the module as an OCI image, and CAPTF runs it, reads its outputs back from state, and reconciles Cluster, Machine and MachinePool objects against them. There is no Go controller to write for a new cloud, and no second copy of infrastructure logic to keep in sync with the module that already exists.

How it fits together

flowchart LR
    subgraph capi["CAPI objects"]
        Cluster["Cluster, Machine, MachinePool"]
    end
    subgraph tf["Terraform* objects"]
        TFObj["TerraformCluster, TerraformMachine(Pool)"]
    end
    Manager["manager"]
    Job["runner Job"]
    Module["module image"]
    Cloud["cloud APIs"]
    State[("state Secret")]

    Cluster --> TFObj
    TFObj --> Manager
    Manager -- creates --> Job
    Job --> Module
    Module --> Cloud
    Module --> State
    State --> Manager

Cluster API’s core objects (Cluster, Machine, MachinePool) reference CAPTF’s own objects (TerraformCluster, TerraformMachine, TerraformMachinePool) as their infrastructure. The CAPTF manager reconciles those objects, renders their inputs, and runs a Kubernetes Job for each operation. The Job’s runner executes a Terraform or OpenTofu module image against those inputs, calling the cloud’s own APIs, and its state — including the outputs the manager reads back — lives in a Kubernetes Secret. See Architecture for the components behind this diagram and how one apply flows through them.

Where to go next

Project status

Every kind — TerraformCluster, TerraformClusterTemplate, TerraformMachine, TerraformMachineTemplate, TerraformMachinePool, TerraformMachinePoolTemplate and TerraformClusterIdentity — is at API version v1alpha1. The module contract they implement is also v1alpha1 and provisional: it is frozen for implementation, but may still change before the first real module has provisioned a cluster with it (see the contract’s changelog).

CAPTF has no end-to-end tests yet. Its test suite is unit tests that mock the Kubernetes API and Job execution; nothing in it creates a real cluster. See Testing for how the test suite is organized.

Quick Start

This tutorial takes CAPTF from an empty management cluster to a TerraformCluster and a control-plane TerraformMachine that both report Ready, using the no-op modules that ship with CAPTF: they create no real infrastructure, so the tutorial needs no cloud account and no credentials. This flow has not been run end to end against a live management cluster.

Because the no-op modules provision nothing real, nothing ever boots a kubelet: no Node ever joins, so KubeadmControlPlane never initializes and the Machine never reaches its Running phase. What you watch come up in this tutorial is CAPTF’s own objects finishing their applies, not a usable Kubernetes cluster. See Your First Module for writing a module that does create something, and the module contract for turning one into a real cloud provider.

Before you begin

• A Kubernetes cluster to use as the management cluster, and kubectl pointed at it.
• clusterctl, make, Go and podman or docker: CAPTF has no release yet, so this tutorial builds the provider’s and the no-op modules’ images from a clone of this repository instead of fetching them.
• A container registry you can push to, and that the management cluster can pull from.

Run every command below from the root of that clone: the make targets and the templates/... paths are relative to it.

1. Install the provider

CAPTF has not published a release, so clusterctl cannot fetch its manifest from a URL yet; build one into a local repository instead. Build and push the manager image, then render the manifest against it:

export IMG=registry.example.com/you/cluster-api-provider-terraform:v0.1.0
make docker-build docker-push IMG="${IMG}"
make manifests-release RELEASE_DIR="${HOME}/local-repository/infrastructure-terraform/v0.1.0" \
  RELEASE_IMG="${IMG}" VERSION=v0.1.0

Point a clusterctl config at that directory; the config entry’s name is terraform, CAPTF’s registered provider name:

# clusterctl.yaml
providers:
- name: terraform
  type: InfrastructureProvider
  url: file:///home/<you>/local-repository/infrastructure-terraform/v0.1.0/infrastructure-components.yaml

<you> is your home directory’s user name, so the url is the absolute path of the directory manifests-release just wrote.

clusterctl init --config clusterctl.yaml --infrastructure terraform:v0.1.0

This also installs Cluster API’s core, bootstrap and control-plane providers, and cert-manager itself if a compatible version is not already present, since CAPTF’s webhooks need it. See Installation for what this creates and how to confirm it, and Installing from a local repository for the general form of the local-repository steps above.

2. Apply an identity

Cloud credentials come from a cluster-scoped TerraformClusterIdentity. An admin applies one per set of credentials, naming the namespaces allowed to use it:

export TERRAFORM_IDENTITY_NAME=aws-prod NAMESPACE=team-a
clusterctl generate yaml --from templates/identity.yaml | kubectl apply -f -

The no-op modules read no credentials, so the generated Secret’s placeholder keys can stay as they are; a module that calls a real cloud provider reads its credentials from the same Secret. kubectl get terraformclusteridentity aws-prod shows Ready=True once the Secret exists and whoever applied the identity was allowed to get it — the admission webhook checks. See Identities and Credentials for creating, rotating and revoking credentials, and how they reach a Job.

3. Build the no-op module images

Build the cluster- and machine-role no-op images and push them where the management cluster can pull them:

make noop-images NOOP="cluster machine"
export NOOP_REGISTRY=registry.example.com/you
make noop-images-push NOOP="cluster machine" NOOP_REGISTRY="${NOOP_REGISTRY}" VERSION=v0.1.0

This pushes ${NOOP_REGISTRY}/noop-cluster and ${NOOP_REGISTRY}/noop-machine, each tagged v0.1.0-terraform and v0.1.0-opentofu; either base satisfies the image contract. This tutorial uses the terraform tag.

4. Generate and apply a cluster

Cluster objects and everything they own live in a namespace clusterctl does not create:

kubectl create namespace team-a

Generate the default flavor and apply it. This tutorial asks for one control-plane machine and no workers, since a worker never gets bootstrap data until a real control plane initializes, which the no-op modules never do:

export TERRAFORM_CLUSTER_IMAGE="${NOOP_REGISTRY}/noop-cluster:v0.1.0-terraform"
export TERRAFORM_MACHINE_IMAGE="${NOOP_REGISTRY}/noop-machine:v0.1.0-terraform"
export TERRAFORM_IDENTITY_NAME=aws-prod

clusterctl generate cluster my-cluster --from templates/cluster-template.yaml \
  --target-namespace team-a \
  --kubernetes-version v1.36.3 \
  --control-plane-machine-count 1 --worker-machine-count 0 \
  | kubectl apply -f -

--from renders the template file directly, so this command needs no provider registration and works the same before or after a release exists. A ClusterClass-based flavor is also available; see Templates and ClusterClass, which also covers every variable this template accepts, and clusterctl variables for their defaults and built-in safeguards.

5. Watch it come up

kubectl get clusters,machines,machinepools,terraformclusters,terraformmachines,terraformmachinepools -n team-a

machinepools returns nothing: the default flavor creates none. See Add an autoscaled pool to a generated cluster to add one to this cluster.

Each Terraform* kind reports a Ready condition, the only one Cluster API reads (it is mirrored into the owning Cluster’s or Machine’s InfrastructureReady). Ready is Unknown while an object waits on dependencies or on its apply to finish, and True once the module’s apply succeeds:

kubectl get terraformcluster -n team-a my-cluster \
  -o jsonpath='{.status.conditions[?(@.type=="Ready")]}'

Expect TerraformCluster my-cluster and the control-plane TerraformMachine to both reach Ready=True, and their PHASE columns to reach Provisioned on Cluster my-cluster and its control-plane Machine too, since the no-op modules do return a control-plane endpoint and a provider ID. What you will not see, because nothing real ever boots: the Machine reaching phase Running (no Node ever registers), and KubeadmControlPlane reporting itself initialized. That gap is expected here and is exactly what a module that creates real infrastructure closes. Watch within about 30 minutes: past that, the control-plane MachineHealthCheck’s node-startup timeout fires because no Node ever registers, and KubeadmControlPlane starts remediating the Machine (see Remediation). For what each condition type and reason means, see Conditions; for the reconcile flow behind these states, see The Reconcile Lifecycle.

6. Clean up

Delete the Cluster first, and wait for it to be gone, rather than deleting the namespace outright: once a namespace starts terminating, the API server refuses to create the destroy Jobs each Terraform* object still needs to run before its own finalizer clears.

kubectl delete cluster my-cluster -n team-a
kubectl wait --for=delete cluster/my-cluster -n team-a --timeout=10m

That wait only returns once every descendant — KubeadmControlPlane, the MachineDeployment, and both TerraformCluster and the control-plane TerraformMachine — is gone too, the latter two only after their own destroy Job finished.

An identity cannot be deleted while a TerraformCluster, TerraformMachine or TerraformMachinePool still references it, so delete it only once the wait above returns, then the now-empty namespace:

clusterctl generate yaml --from templates/identity.yaml | kubectl delete -f -
kubectl delete namespace team-a

To remove the provider itself as well:

clusterctl delete --config clusterctl.yaml --infrastructure terraform

See also

• Your First Module to write a module that provisions something real.
• Installation for what clusterctl init installs and how to verify it.
• Security Model for the trust boundary a Terraform* object’s Job operates inside.
• The Kinds for how CAPTF’s objects relate to Cluster API’s.

Your First Module

This tutorial writes a small machine-role module from scratch, lints it, packages it as the OCI image CAPTF runs, lints that image, and references it from a TerraformMachineTemplate. It ends with a working, lint-clean module image; it does not provision real infrastructure, since the module in this tutorial creates no cloud resources. For the full set of rules a module must follow, see the module contract and the image contract.

Before you begin

• tfcapi-lint, installed as in tfcapi-lint.
• podman or docker, to build the image.
• A directory to work in. This tutorial calls it machine/.

Terraform or OpenTofu itself is not required on your machine: the reference Containerfile below downloads it inside the build, and tfcapi-lint module parses the module’s files directly.

1. Write the module

A machine-role module implements one Terraform/OpenTofu module that CAPTF calls once per TerraformMachine. It receives the common contract inputs plus the machine role’s inputs, and must return the machine role’s outputs plus the common health output. This tutorial’s module returns fixed values instead of calling a cloud provider, which keeps every input and output visible and needs no credentials or provider plugins to build or lint. Swap the fixed values for calls to your cloud’s Terraform/OpenTofu provider once you understand the shape.

Create four files in machine/.

variables.tf

The contract inputs. captf_contract, captf_cluster, captf_object and captf_tags apply to every role; captf_cluster_outputs applies to the machine and machinepool roles only (the cluster role produces it, and so does not receive it); the rest are specific to the machine role.

# Contract inputs of the machine role, v1alpha1 (https://captf.io/docs/module-author/contract/v1alpha1/common.html
# and machine.html).

variable "captf_contract" {
  type = string
}

variable "captf_cluster" {
  type = object({
    name      = string
    namespace = string
  })
}

variable "captf_object" {
  type = object({
    kind      = string
    name      = string
    namespace = string
  })
}

# The cluster module's exports. The controller always sets it; the default
# follows the contract skeleton (machine.md).
variable "captf_cluster_outputs" {
  type    = any
  default = null
}

variable "captf_tags" {
  type = map(string)
}

variable "machine_name" {
  type = string
}

# Base64 of the bootstrap Secret's value.
variable "bootstrap_data" {
  type      = string
  sensitive = true
}

variable "bootstrap_format" {
  type = string
}

variable "failure_domain" {
  type    = string
  default = null
}

variable "kubernetes_version" {
  type    = string
  default = null
}

variable "control_plane" {
  type = bool
}

main.tf

The module’s own logic. This tutorial stands in a terraform_data resource for a real instance, so the module has something to hold its inputs; a real module replaces this with the resources that create an instance.

# No-op machine module: implements the v1alpha1 machine role with no cloud.

# The stand-in for an instance. user_data decodes bootstrap_data the way a
# module feeding a plain user-data argument would, which proves the
# controller's base64 encoding round-trips (the value stays sensitive).
resource "terraform_data" "instance" {
  input = {
    cluster            = var.captf_cluster
    object             = var.captf_object
    machine_name       = var.machine_name
    tags               = var.captf_tags
    backend_id         = try(var.captf_cluster_outputs.backend_id, null)
    user_data          = base64decode(var.bootstrap_data)
    bootstrap_format   = var.bootstrap_format
    failure_domain     = var.failure_domain
    kubernetes_version = var.kubernetes_version
    control_plane      = var.control_plane
  }
}

outputs.tf

The contract outputs. provider_id, addresses and failure_domain are required; interruptible must be declared even when it is always false; health is the common output every role returns.

# Contract outputs of the machine role, v1alpha1.

# Stable per Machine. With no Node behind it, the e2e suite never expects a
# nodeRef; a real module emits the CCM's or kubelet's format (machine.md).
output "provider_id" {
  value = "noop:///${var.captf_object.namespace}/${var.machine_name}"
}

output "addresses" {
  value = [{ type = "InternalIP", address = "10.0.0.1" }]
}

output "failure_domain" {
  value = var.failure_domain
}

output "interruptible" {
  value = false
}

output "health" {
  value = { state = "running", healthy = true, message = null, reasons = [] }
}

versions.tf

terraform {
  # terraform_data needs Terraform 1.4; the variants' plantimestamp() needs
  # 1.5. Every OpenTofu release (1.6+) has both.
  required_version = ">= 1.5"
}

This module declares no required_providers: terraform_data ships with Terraform/OpenTofu itself, so there is no provider to install or mirror. A module that calls a cloud provider adds a required_providers block here as usual.

2. Lint the module

tfcapi-lint module ./machine --role machine --strict

A clean module prints an empty finding list and an all-zero summary. --strict fails the command on a warning as well as an error, so a lint-clean module here stays lint-clean once you add real inputs, outputs and user variables. See tfcapi-lint for the full command reference and tfcapi-lint CLI for every check ID.

3. Package it as an image

CAPTF runs a module as one OCI image that bundles the module’s files and the Terraform or OpenTofu binary; there is no separate module source and no separate runtime image. Save the reference Terraform-based Containerfile into machine/:

# Reference source image, Terraform base (docs/book/src/module-author/image-contract.md "Reference:
# Terraform base"). Run from your module's root directory:
#
#   podman build -f Containerfile.terraform --build-arg ROLE=cluster \
#     --build-arg IMAGE_SOURCE=https://github.com/<org>/<repo> \
#     --build-arg IMAGE_REVISION="$(git rev-parse HEAD)" \
#     --build-arg IMAGE_VERSION=<tag> -t <registry>/<repo>:<tag> .
#
# The image tag is the module version. Lint first:
#   tfcapi-lint module . --role cluster --strict

ARG RUNTIME_VERSION=1.16.4
# The org.opencontainers.image.* labels below (image-contract.md "OCI
# labels"): leave these unset for a local/test build, or pass them from
# your CI pipeline (source repo URL, commit SHA, the image tag).
ARG IMAGE_SOURCE=""
ARG IMAGE_REVISION=""
ARG IMAGE_VERSION=""

# Optional but recommended: hermetic provider mirror for the platforms you
# publish. Needs registry egress at build time; drop this stage (and the
# COPY --from=mirror below) for a non-hermetic image.
FROM docker.io/hashicorp/terraform:${RUNTIME_VERSION} AS mirror
WORKDIR /src
COPY . /src
# get: `providers mirror` refuses a module whose nested local modules are not
# installed; get installs them (no providers), in this stage only.
# mkdir: `providers mirror` does not create the target when the module
# requires no providers, and the COPY --from=mirror below needs it.
RUN terraform get \
 && mkdir -p /captf/providers \
 && terraform providers mirror -platform=linux_amd64 -platform=linux_arm64 /captf/providers

FROM docker.io/hashicorp/terraform:${RUNTIME_VERSION}
ARG ROLE=cluster
ARG RUNTIME_VERSION
ARG IMAGE_SOURCE
ARG IMAGE_REVISION
ARG IMAGE_VERSION
COPY --from=mirror /captf/providers /captf/providers
COPY . /captf/module
RUN ln -s /bin/terraform /captf/runtime \
 && adduser -D -u 65532 captf \
 && chown -R 65532:65532 /captf
USER 65532
LABEL io.captf.contract="v1alpha1" \
      io.captf.role="${ROLE}" \
      io.captf.runtime="terraform" \
      io.captf.runtime.version="${RUNTIME_VERSION}" \
      org.opencontainers.image.source="${IMAGE_SOURCE}" \
      org.opencontainers.image.revision="${IMAGE_REVISION}" \
      org.opencontainers.image.version="${IMAGE_VERSION}"

An OpenTofu-based equivalent is also available (Containerfile.opentofu); either runtime satisfies the contract. Build from inside machine/:

export IMAGE=registry.example.com/acme/machine:v1.0.0
podman build -f Containerfile.terraform --build-arg ROLE=machine -t "$IMAGE" .

IMAGE is the registry, repository and tag you push to; ROLE=machine tags the image with the role it implements. See the image contract for the fixed paths, OCI labels and execution environment every image must satisfy.

4. Lint the image

tfcapi-lint image reads a pushed registry reference or a local OCI layout. Without a registry to push to yet, save the image podman just built to a local OCI directory and lint that:

podman save --format oci-dir -o /tmp/first-module-image "$IMAGE"
tfcapi-lint image --role machine "oci:/tmp/first-module-image"

Once you push $IMAGE to a registry, lint the pushed reference the same way: tfcapi-lint image --role machine "$IMAGE".

5. Reference it from a TerraformMachineTemplate

A TerraformMachineTemplate is what a MachineDeployment, a KubeadmControlPlane or a MachineSet points at to create TerraformMachines; its spec.template.spec.source.image names the image you just built:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachineTemplate
metadata:
  name: first-module
  namespace: default
spec:
  template:
    spec:
      source:
        image: registry.example.com/acme/machine:v1.0.0

Apply it:

kubectl apply -f first-module-template.yaml

kubectl get terraformmachinetemplate first-module -n default confirms the object exists; there is nothing to provision yet, since nothing references this template as a MachineDeployment, MachineSet or control plane’s infrastructureRef does. A TerraformMachine created from it needs an identity to run under: either its own spec.identityRef, or one inherited from its TerraformCluster’s spec.defaults.identityRef, as set up in the quick start.

Next steps

• Walk through the quick start to wire a machine template like this one into a full cluster.
• Read the module contract for every rule a production module must follow, and the cluster role and machine pool role for the other two roles a full deployment needs.
• Read Runtime Environment for what a module sees when the runner actually executes it.

Architecture

This page describes CAPTF’s components, what runs where, and how one apply flows through them. It is for anyone who needs to understand how CAPTF works before reading a more specific page.

Components

The manager Deployment is CAPTF’s one Deployment and one container image. It runs the controllers that reconcile every kind except TerraformClusterTemplate and TerraformMachinePoolTemplate, which have no reconciler of their own, and it serves the validating webhooks for all seven kinds from the same process, on a separate port. It watches every namespace by default: clusterctl init never sets a namespace restriction. See Configuration for how to scope or tune it.

The validating webhooks reject an invalid or disallowed change to a Terraform* object before it is persisted: an immutable field, a malformed image reference, a spec.variables name that collides with a contract input, or (for TerraformClusterIdentity) a Secret the requester cannot read. There are no mutating webhooks — CAPTF resolves every default at reconcile time instead of writing it back onto the object. See Security Model for the trust boundary a webhook decision sits inside, and RBAC for who can change what.

A runner Job is a batch/v1 Job the manager creates for one operation (apply, destroy, drift, refresh, restore or plan) on one Terraform* object, in that object’s own namespace. Kubernetes never retries a Job pod; the manager owns retries itself, one attempt per Job name. See The Reconcile Lifecycle for how the manager decides which operation runs next.

The module image is the OCI image a Terraform* object’s spec.source.image names. It bundles a Terraform or OpenTofu module written to CAPTF’s contract for one role (cluster, machine or machinepool) together with the runtime binary that runs it, at fixed paths, and optionally a provider filesystem mirror. It never bundles the runner. See Image Contract.

The runner binary is copied into each Job’s pod by an init container, rather than shipped in the module image. The manager’s own image bundles both /manager and a static /runner binary; the init container runs /runner copy /captf/bin/runner into an emptyDir the module container also mounts, so by default the runner every Job executes is exactly the build of the manager that created the Job; an operator can point it at a different image instead (see Configuration). The module container then runs that copy as its command, against the module and runtime the image provides. The init container’s resources are fixed and not configurable, since it only copies one static binary and never varies with the module.

The Kubernetes state backend is the Terraform/OpenTofu kubernetes backend, which the manager configures for every generated root module. State lives in Secrets in the Terraform* object’s own namespace, chunked when it is large, and labeled by the object’s immutable identity — namespace, kind and name, never its UID — so state survives a clusterctl move and a changed identity is never mistaken for existing state. See Terraform State.

What runs where

The data flow of one apply

The manager reconciles a TerraformCluster, TerraformMachine or TerraformMachinePool through a shared core (externally-managed check, owner lookup, finalizer, pause) and decides that an apply is the next operation. It renders that object’s inputs into a generated root module — main.tf.json, which configures the kubernetes backend and calls the module image’s role module with exactly the contract inputs, and terraform.tfvars.json, the input values — and writes them to a durable inputs Secret and a per-run Secret, hashing the inputs as it does. See Job Inputs for exactly where each value comes from.

The manager then builds and creates the Job: the init container that copies the runner binary, and the module container that mounts the per-run Secret read-only, the identity’s mirrored credentials Secret, and empty scratch volumes. Once the pod starts, the runner initializes the backend, plans and applies against the module’s runtime, and — for a TerraformCluster apply — stops before a plan that deletes or replaces a resource unless that plan was already approved (see Plan Approval). The generated root re-exports the module’s contract outputs, so they land in state alongside everything else the runtime tracks.

The runner reports how the Job finished as its pod’s termination message. The manager reads that message, and — once the Job has finished — the state Secrets themselves, to set the object’s status and conditions and, after a successful apply, to adopt (re-label) the new state Secrets as its own.

What a module author provides versus what an operator provides

A module author writes and builds the OCI image: the role module (and any local modules it calls), pinned to the module contract and the image contract, optionally with a provider mirror baked in. They check it with tfcapi-lint before publishing it.

An operator installs the manager (Installation), creates the TerraformClusterIdentity credentials CAPTF’s objects reference (Identities and Credentials), and creates the Terraform* objects — directly or through templates and ClusterClass — that name a module author’s image. They tune Job resources, deadlines and security contexts per object (Tuning Jobs), and watch the conditions, events and alerts the manager and runner produce (Observability).

The Kinds

If you are learning how CAPTF’s objects relate to Cluster API’s, start here. CAPTF adds seven kinds to one API group and version, infrastructure.cluster.x-k8s.io/v1alpha1. Three of them are the objects that carry the desired state Cluster API’s core controllers drive as a cluster comes up; three are templates that stamp those out; the seventh holds the cloud credentials the others use. Every field, default and validation rule is in the API reference; this page explains how the seven fit together, and what stays fixed once an object exists.

The seven kinds

The three workload kinds

A TerraformCluster, TerraformMachine or TerraformMachinePool runs one Terraform or OpenTofu module role — cluster, machine or machinepool — as a Job. What goes into that Job and how the reconcile loop drives it are covered in Job inputs and the reconcile lifecycle; this section is about the object itself.

Each of the three is owned by its Cluster API counterpart, not by CAPTF: the counterpart’s own core controller sets the owner reference. A TerraformCluster or TerraformMachinePool can be an object you create and point the Cluster or MachinePool at directly, or one a ClusterClass topology expands from a TerraformClusterTemplate or TerraformMachinePoolTemplate; either way, the resulting object is owned the same way. A TerraformMachine is normally expanded from a TerraformMachineTemplate, one per Machine, by a MachineDeployment, a MachineSet or a control-plane provider, whether or not a ClusterClass is involved; a Machine can also point at a directly created TerraformMachine, the same way a Cluster or MachinePool can point at a directly created TerraformCluster or TerraformMachinePool.

• TerraformCluster is owned by a Cluster. spec.controlPlaneEndpoint is mutable until it has a host — set by you, or once by the controller from the module’s output — and immutable after: every Machine and kubeconfig of the cluster points at it. Everything else, including the module image, can change on a live object; a new image or a changed input re-applies the module against the existing state. spec.identityRef is required. spec.applyPolicy and the destructive-plan guard are covered in plan preview and approval.
• TerraformMachine is owned by a Machine. spec.source, spec.identityRef, spec.variables and spec.variablesFrom define the machine and are immutable after creation: change them through a MachineDeployment or control-plane rollout, not in place. spec.providerID can only move from empty to non-empty, by the controller. spec.jobs, spec.drift and spec.remediation are operational policy and stay mutable at any time, so a stuck machine’s deadline or drift interval can be changed without rolling it. A direct delete is refused outside a few exceptions; see the reconcile lifecycle for deletion and finalizers.
• TerraformMachinePool is owned by a MachinePool. Unlike a TerraformMachine, every field is mutable: a change to spec.source, spec.variables, spec.variablesFrom or any other field re-applies the module on the next reconcile. A pool has no spec.applyPolicy and no delete guard. With autoscaling enabled, it also writes the group’s observed replica count back to MachinePool.spec.replicas; see machine pools.

Templates

TerraformClusterTemplate, TerraformMachineTemplate and TerraformMachinePoolTemplate each hold spec.template, the metadata and spec a TerraformCluster, TerraformMachine or TerraformMachinePool is created with. A TerraformClusterTemplate and a TerraformMachinePoolTemplate are each expanded by a ClusterClass topology into one TerraformCluster or TerraformMachinePool, which the Cluster or MachinePool then references directly. A TerraformMachineTemplate is expanded, with or without a ClusterClass, by a MachineDeployment, a MachineSet, a KubeadmControlPlane or another control-plane provider, into one TerraformMachine per Machine. See ClusterClass for how templates and flavors fit together.

spec.template.spec is immutable once a template exists, like every Cluster API template: create a new template and point the owner at it instead of editing one in place. spec.template.metadata stays mutable. The one exception is a ClusterClass topology dry-run, which the webhook recognizes and exempts from the immutability check, so a topology patch that only touches spec.template.spec can still be validated without tripping it.

A TerraformMachineTemplate additionally reports status.capacity and status.nodeInfo, resolved from its image’s labels, for Cluster Autoscaler scale-from-zero. TerraformMachinePoolTemplate has no status: a pool has no scale-from-zero, so there is no capacity to resolve.

TerraformClusterIdentity

TerraformClusterIdentity is cluster-scoped and has no Cluster API counterpart: it exists only to hold spec.secretRef, the credentials Secret a TerraformCluster’s, TerraformMachine’s or TerraformMachinePool’s module run is allowed to use, and spec.allowedNamespaces, which namespaces may reference it. Every field is mutable, including secretRef; changing it re-checks that the requester may read the new Secret. Deleting an identity is refused while any object still uses it or still has its credentials mirrored into a namespace. See identities for creating, rotating and revoking credentials, and how they reach a Job.

What a cluster passes to its machines and pools

A TerraformCluster’s spec.defaults apply only to its TerraformMachines and TerraformMachinePools, never to the cluster itself, and are merged field by field: a field the machine or pool sets wins, an unset one comes from spec.defaults, and a field neither sets gets the built-in default. A machine or pool finds its Cluster by its own cluster.x-k8s.io/cluster-name label, then the TerraformCluster through the Cluster’s spec.infrastructureRef.

• identityRef: a machine or pool uses its own identityRef when it sets one, else the cluster’s spec.defaults.identityRef, else the cluster’s own spec.identityRef.
• jobs: merged field by field with spec.defaults.jobs, own over defaults; see Tuning Jobs for the full merge algorithm.
• drift: only intervalSeconds is inherited — the object’s own, else spec.defaults.drift.intervalSeconds, else the manager’s built-in default. A pool’s own drift.action is never inherited; unset, it defaults to Report regardless of the cluster’s defaults, which carry no action to inherit.

The pool exception: a TerraformMachine’s drift can be turned off with intervalSeconds: 0, and an inherited spec.defaults.drift of 0 turns it off the same way, but a TerraformMachinePool’s drift can never be turned off; see Drift and health for why.

See also

• API Reference for every field, default and validation rule.
• The reconcile lifecycle for what runs when, retries, deletion and finalizers.
• Job inputs for what each role’s Job receives.
• ClusterClass for templates in use.
• Machine pools for autoscaling and replica write-back.
• Identities for TerraformClusterIdentity in depth.

The Reconcile Lifecycle

TerraformCluster, TerraformMachine and TerraformMachinePool share one reconcile flow. This page walks through a single pass of it: what runs before any decision, how the next operation is chosen, how a failed operation backs off, how two objects of the same cluster stay off each other’s feet, and what each kind does once the shared flow is done. It is for anyone who wants to know why CAPTF started (or did not start) a Job, or why a delete is taking a while.

One pass, one status patch

Every reconcile writes the object’s status once, in a single patch at the end of the pass. Two things are written earlier, directly, because a crash between them and the deferred patch must not leave a gap:

• the clusterctl move block, set just before a Job is created;
• the preamble’s own writes, described below, which can stop the pass before there is anything else to decide.

A reconcile that reaches the end of the pass runs through, in order:

1. Externally managed. An object carrying CAPI’s externally-managed annotation is left alone entirely; nothing below runs.
2. Owner lookup. Without an owner reference of the expected kind yet, the object waits (DependenciesReady=Unknown) unless it is being deleted with no state and no running Job, in which case the finalizer is dropped at once: there is nothing to destroy. An owner reference whose target is gone does not stop deletion; a destroy still runs from the durable inputs. An owner gate such as the owning Cluster’s infrastructure reference not naming a TerraformCluster sets DependenciesReady and stops the pass, unless the object is being deleted, in which case deletion proceeds anyway. The lookup also requires the owner to reference this object back (its own infrastructureRef, and, when set, a matching ownerRef UID and cluster-name label); an owner that fails this check is treated the same as one that is gone: DependenciesReady=False/OwnerMismatch, no Job, and deletion still proceeds. See Owner references are checked against the owner for the full rule and why it exists.
3. Finalizer. Added if missing; the pass stops for this reconcile so the write is visible before anything else happens.
4. Pause. A paused object, or one whose Cluster is paused, runs only the paused branch described below.

See Conditions for every condition this flow sets and what each reason means, and TerraformClusterIdentity for the identity and credential-mirror step that follows the preamble.

The paused branch

A paused object still processes its finished Jobs (bookkeeping, below) and deletes a Job that can never start, but starts nothing new. Once no Job is active, the clusterctl move block is cleared. This is what clusterctl move waits for after it pauses an object; see the move runbook.

Bookkeeping

Unpaused, the reconcile reads the object’s Jobs and processes every one that finished since the last pass it was seen: it records the run’s steps, error and drift summary in status.lastRun, sets the ApplyJobSucceeded and DriftJobSucceeded conditions, pins the image digest of a newly succeeded apply, prunes history beyond the configured limits, and finds whether a Job is active now. A stale state lock left by a Job whose pod is gone is flagged for the next Job to force-unlock; a lock held by something else (a workstation, another tool) is reported instead. A finished Job’s per-run inputs Secret is deleted once, which is also the point at which its run lease (and, for a TerraformCluster, its cluster write lease) is given back.

What a finished Job’s pod reported is only durable once the status patch at the end of the pass has succeeded: the reconcile marks a Job “bookkept” only after that patch, so a failed patch leaves the Job to be read again next time instead of losing its result.

While a Job is active, state is not read and no new operation is decided: the reconcile deletes the Job if it looks stuck (its per-run Secret is missing and no pod ever started, checked only once the Job is at least a minute old) and otherwise waits, holding the clusterctl move block.

Choosing the next operation

Once no Job is active, the reconcile reads state, builds the current inputs (for a mutable kind, or an immutable one not yet provisioned), and looks for a requested state restore. It then picks the next operation in this order:

1. A restore named by the captf.io/restore-state annotation, unless the object is being deleted: it is what was asked for, and checking or applying against the state about to be replaced would be wasted or worse.
2. Being deleted: destroy, or drop the finalizer at once if there is no state to destroy.
3. No state yet: apply (first apply).
4. State exists but carries no inputs hash: apply.
5. A mutable kind whose current inputs hash differs from the one state carries: apply.
6. A mutable kind whose newest finished apply failed (and was not blocked, was not stopped by a plan mismatch, and was not a drift remediation): apply again, even if the current inputs already match state, because the failed run may have changed reality without recording it.
7. A mutable kind with drift found and drift action Remediate, while the object has not yet reached its remediation-failure cap since the last successful drift check: apply. Drift detection and remediation are Drift and health’s topic.

Whichever of those an apply reason names but the first (no state yet, with nothing to break), an object under applyPolicy Manual does not apply directly: it waits for a plan and its approval first, or backs off behind the destructive-plan guard if neither policy applies. Both are Plan approval’s topic; this page only fixes where that wait sits in the order above.

When none of those apply reasons matched, the reconcile falls through to its refresh, membership and drift schedule, checked in this order and stopping at the first one due:

1. A successful apply not yet followed by a refresh, for kinds that ask for one (a machine or machine pool, so its addresses and health reach status without waiting for the next tick).
2. A TerraformMachinePool whose membership is still converging: refresh every 30 seconds, plus jitter, fixed rather than backed off.
3. Otherwise, while health reads pending: refresh after a delay that starts at 30 seconds and doubles (30s, 1m, 2m, 4m) up to a 5-minute ceiling, plus jitter.
4. A machine pool’s membership refresh interval elapsed: refresh.
5. The health-check interval elapsed: refresh.
6. The drift-check interval elapsed: drift.
7. Otherwise, requeue at whichever of the above is soonest.

Every interval above adds a small deterministic jitter derived from the object’s UID (up to a tenth of the interval), so objects created together, such as a MachineDeployment’s machines or everything moved by one clusterctl move, do not all check in lockstep. Configuring these intervals is Drift and health’s and Configuring drift’s topic; machine pool membership convergence is Machine pools’s.

Retry backoff

A failed Job’s operation is not retried immediately: the delay after n consecutive failures of that operation is one minute, doubling each time, capped at ten minutes. Once n reaches the failed-Jobs history limit (three by default), the delay jumps straight to the ten-minute cap instead of continuing to double, so the sequence with the default limit is one minute, two minutes, then ten minutes and no shorter. A Job stopped from outside (its pod was interrupted, or an apply was blocked before a destructive plan, or an approved apply’s plan changed since approval, or the run never started because a lease was held) does not count toward this backoff at all: those wait on a lease, an approval, or the object’s own next reconcile instead. A failed state restore is a special case: it is never retried for the same backup serial, only for a new one or after the failed Job is deleted.

Run leases and the cluster operation gate

Before a Job is created, the reconcile takes the object’s own run lease, so two managers (or two reconciles racing after a crash) never start two Jobs for the same object at once. If another Job already holds it, the operation waits instead: ApplyJobSucceeded, DriftJobSucceeded or RestoreJobSucceeded (matching the operation) goes Unknown with a message naming the Job to wait for.

With the manager’s --cluster-operation-gate flag (default enabled; see Manager configuration), an apply, destroy or restore Job of an object that names a cluster additionally crosses a second gate meant to keep a cluster’s own apply or destroy from running at the same time as its machines’ and machine pools’:

• A TerraformCluster takes the cluster’s write lease, then checks whether any machine or machine pool of the cluster is currently applying or destroying. If so, it keeps both leases and waits for them to finish before it starts: new machine and machine pool operations wait behind it meanwhile, so a stream of machine creates cannot starve the cluster’s own operation.
• A TerraformMachine or TerraformMachinePool checks whether the cluster’s write lease is currently held. If it is, the machine or pool gives back the run lease it just took and waits for the cluster’s operation to finish first.

Deletion order

A TerraformCluster being deleted waits for every TerraformMachine and TerraformMachinePool carrying its cluster name in the namespace to be gone before its own destroy runs; machines and machine pools carry no such wait of their own. An owner reference whose target has already been removed does not block a destroy: deletion still runs from the durable inputs. Once a destroy succeeds, or deletion finds no state to destroy at all, the reconcile deletes the object’s state and durable inputs, releases its leases, drops it from its credential mirror’s owners, and removes the finalizer. See Secrets for what those Secrets are and RBAC for the namespace RBAC sweep that follows a finalizer’s removal.

The clusterctl move block

Clusterctl’s block-move annotation is set on the object just before a Job is created and stays set for as long as a Job is active; it is cleared, paused or not, once no Job is active. clusterctl move pauses every object first and then waits for the annotation to clear, which is why the paused branch above still runs bookkeeping and the stuck-Job check: a Job that can never start would otherwise hold the block indefinitely.

Job names and history

A Job’s name is deterministic: captf-<kindshort>-<name>-<operation>-a<attempt>-<hash>, where <kindshort> is c, m or mp, <attempt> is one more than the highest attempt number retained for that operation, and <hash> is six hex characters derived from the inputs hash, the operation, the attempt and, for refresh and drift, a tick that changes each time the operation last succeeded (so a retry finds the same Job instead of starting a second one); a drift-remediation apply and a restore carry a tick of their own too, so neither collides with an unrelated Job of the same inputs hash and attempt. A name that would exceed 57 characters replaces the object name with a hash instead. History beyond the configured successful- and failed-Jobs limits is pruned per operation, oldest first, while always keeping each operation’s newest success and, while no success is newer, its newest failure, since those are what retry backoff and drift remediation’s failure cap read. Tuning those limits, and the rest of a Job’s resources, deadlines and identity, is Job tuning’s topic.

Jobs per bring-up

Bringing up one TerraformCluster with three control-plane and three worker TerraformMachines runs eight Jobs when each machine’s apply outputs give a definite health reading: one cluster apply, one apply per machine (six), and one no-change re-apply once the cluster’s control_plane_initialized input flips (the cluster’s inputs hash changes, so the shared flow applies again even though nothing else about the cluster changed). A machine whose apply outputs read pending instead keeps the extra post-apply refresh described above; with all six machines reading pending that adds six more Jobs, back up to fourteen. Count Jobs over a bring-up with sum(captf_jobs_total) rather than listing Jobs: a finished Job is pruned once it falls outside the history limits above.

After the shared flow

Each kind does a little more once the shared flow has read state and decided:

• A TerraformCluster decides its control_plane_endpoint input while the shared flow builds inputs, and writes an apply’s endpoint output back to spec.controlPlaneEndpoint while state is read: both are part of the shared flow, not a separate step. Each happens at most once.
• A TerraformMachine keeps cluster.x-k8s.io/remediate-machine on its owning Machine in step with its instance health once the shared flow finishes, skipped for a paused or externally-managed machine. See Machine health and remediation.
• A TerraformMachinePool writes its observed replica count back to its owning MachinePool’s spec.replicas when autoscaling is enabled, skipped for a paused, externally-managed or deleting pool. See Machine pools.

One pass, visually

flowchart TD
    A["Preamble: externally managed?<br/>owner lookup, finalizer, pause"]
    A -->|paused| P["Paused branch:<br/>bookkeeping, delete a stuck Job,<br/>clear the move block"]
    A -->|not paused| B["Credentials: identity,<br/>credential mirror, runner RBAC"]
    B --> C["Bookkeeping: finished Jobs,<br/>digest pin, lock check"]
    C -->|a Job is active| D["Delete it if stuck,<br/>else wait for it"]
    C -->|no Job is active| E["Clear the move block"]
    E -->|deleting, destroy succeeded| F["Cleanup: remove the finalizer"]
    E -->|deleting, machines or pools remain| G["Wait: deletion blocked"]
    E -->|otherwise| H["Read state, build inputs,<br/>look for a requested restore"]
    H --> K["Decide the next operation"]
    K -->|start a Job| L["Take leases, start the Job"]
    K -->|drop the finalizer| F
    K -->|nothing to start| M["Requeue at the decided delay"]
    D --> N["Patch status once"]
    L --> N
    M --> N
    P --> N
    F --> N
    G --> N

See also

• Conditions — every condition this flow sets.
• Events — the events it emits.
• Drift and health — how drift checks and health work.
• The state backend — reading and adopting state, and backups.
• Plan approval — the Manual apply policy and the destructive-plan guard.

Job Inputs

This explains what goes into a CAPTF Job’s Terraform inputs — terraform.tfvars.json and the generated root main.tf.json — and where each value comes from on the Kubernetes side. For each input’s type and whether it is required, see the module contract; this page traces the plumbing that fills the contract in, for anyone administering CAPTF or writing a module against it.

A TerraformCluster runs the module’s cluster role, each TerraformMachine runs its machine role, and each TerraformMachinePool runs its machinepool role. All three roles are rendered the same way from the same kind of input structure, so this page covers all three side by side.

1. The pipeline

flowchart TD
    subgraph capi[CAPI objects]
        Cluster["Cluster<br/>spec.topology.version<br/>spec.clusterNetwork<br/>spec.controlPlaneEndpoint<br/>status.initialization.controlPlaneInitialized"]
        Machine["Machine<br/>spec.version<br/>spec.failureDomain<br/>spec.bootstrap.dataSecretName<br/>labels[cluster.x-k8s.io/control-plane]"]
        MachinePool["MachinePool<br/>spec.replicas<br/>spec.failureDomains<br/>spec.template.spec.version<br/>spec.template.metadata.labels<br/>autoscaler min/max-size annotations"]
        Bootstrap["Bootstrap Secret (KubeadmConfig/RKE2Config)<br/>keys: value, format"]
    end
    subgraph objs[Terraform* objects]
        TC["TerraformCluster<br/>spec + status"]
        TM["TerraformMachine<br/>spec + status"]
        TMP["TerraformMachinePool<br/>spec + status"]
    end
    Exports["TerraformCluster state<br/>outputs: exports, failure_domains"]

    Cluster --> ClusterInputs
    TC --> ClusterInputs
    ClusterInputs[Cluster inputs] --> RenderC[Render, cluster role]
    RenderC --> DurableC["Durable Secret<br/>captf-inputs-c-name"]
    RenderC --> RunC["Per-run Secret<br/>captf-run-job<br/>mounted at /captf/config"]
    RunC --> RunnerC[Runner] --> ModuleC[Module, cluster role]
    ModuleC --> Exports

    Machine --> MachineInputs
    Bootstrap --> MachineInputs
    Cluster --> MachineInputs
    TM --> MachineInputs
    Exports -->|only while not yet provisioned| MachineInputs
    MachineInputs[Machine inputs] --> RenderM[Render, machine role]
    RenderM --> DurableM["Durable Secret<br/>captf-inputs-m-name"]
    RenderM --> RunM["Per-run Secret<br/>captf-run-job<br/>mounted at /captf/config"]
    RunM --> RunnerM[Runner] --> ModuleM[Module, machine role]

    MachinePool --> PoolInputs
    Bootstrap --> PoolInputs
    Cluster --> PoolInputs
    TMP --> PoolInputs
    Exports -->|every reconcile| PoolInputs
    PoolInputs[Machine pool inputs] --> RenderP[Render, machinepool role]
    RenderP --> DurableP["Durable Secret<br/>captf-inputs-mp-name"]
    RenderP --> RunP["Per-run Secret<br/>captf-run-job<br/>mounted at /captf/config"]
    RunP --> RunnerP[Runner] --> ModuleP[Module, machinepool role]

    DurableC -.->|destroy always; drift/refresh when gated or current inputs unavailable| RenderC
    DurableM -.->|destroy and drift/refresh always, machine is immutable| RenderM
    DurableP -.->|destroy always; drift/refresh when gated or current inputs unavailable| RenderP

Why two Secrets exist:

• Durable inputs Secret (captf-inputs-<kindshort>-<name>, owned by the Terraform* object, moves with it across clusterctl move): the record of what was rendered when the controller last started an apply Job for this object (written at Job start, not gated on the Job’s success). Destroy always prefers it; a cluster or pool (both mutable) whose durable Secret is missing falls back to freshly built current inputs, but a machine (immutable) never does — with no durable Secret and no live spec to fall back to, destroy has nothing to run against until the Secret is restored. Drift and refresh follow the same asymmetry: a cluster or pool checks against its current inputs when they build cleanly, falling back to the durable Secret only when they don’t (a dependency gate, an owner gone); a machine, being immutable, always checks against the durable Secret.
• Per-run Secret (captf-run-<job>, owned by the Job, mounted at /captf/config, deleted when the Job finishes): a private copy the Job’s pod reads, so a rewrite of the durable Secret by a later reconcile can never change the files under a running Job.

2. Cluster role inputs

For the full type table, see contract/v1alpha1/common.md and cluster.md. The Kubernetes-side sources are:

A trimmed real example:

{
  "captf_contract": "v1alpha1",
  "captf_cluster": { "name": "prod", "namespace": "team-a" },
  "captf_object": { "kind": "TerraformCluster", "name": "prod", "namespace": "team-a" },
  "captf_tags": {
    "captf.io/cluster": "prod",
    "captf.io/kind": "TerraformCluster",
    "captf.io/managed-by": "captf",
    "captf.io/name": "prod",
    "captf.io/namespace": "team-a",
    "captf.io/template": ""
  },
  "control_plane_endpoint": { "host": "api.prod.example.com", "port": 6443 },
  "kubernetes_version": "v1.36.2",
  "control_plane_initialized": true,
  "cluster_network": {
    "pods": ["192.168.0.0/16"],
    "services": ["10.96.0.0/12"],
    "service_domain": "cluster.local",
    "api_server_port": 6443
  }
}

The matching root main.tf.json declares one Terraform variable per key above (nullable ones get "default": null), a module "role" call forwarding every variable by name to the module, and a sensitive = true re-export of every required cluster output (control_plane_endpoint, failure_domains, exports, health). The cluster role never declares captf_cluster_outputs: it has no such input at all, not even null.

3. Machine role inputs

For the full type table, see common.md and machine.md. The Kubernetes-side sources are:

A trimmed real example:

{
  "captf_contract": "v1alpha1",
  "captf_object": { "kind": "TerraformMachine", "name": "prod-md-0-abcde", "namespace": "team-a" },
  "captf_cluster_outputs": { "network_id": "net-1", "note": "a<b & c>d" },
  "machine_name": "prod-md-0-abcde-xyz12",
  "bootstrap_data": "I2Nsb3VkLWNvbmZpZwpydW5jbWQ6...",
  "bootstrap_format": "cloud-config",
  "failure_domain": null,
  "kubernetes_version": "v1.36.2",
  "control_plane": false
}

The note: "a<b & c>d" above is deliberate: the machine role’s renderer keeps a module-authored exports value’s exact bytes through the durable Secret instead of turning </& into their \u... escapes (section 5 has the one exception to this).

The root variables and re-exported outputs (provider_id, addresses, failure_domain, interruptible, health) follow the same pattern as the cluster role.

4. Machine pool role inputs

A TerraformMachinePool runs the module’s machinepool role. For the full type table and lifecycle, see contract/v1alpha1/machinepool.md (this page does not duplicate it). Its inputs follow the same captf_contract/captf_cluster/captf_object/captf_tags pattern as the cluster and machine roles, plus captf_cluster_outputs (the owning TerraformCluster’s exports, as for a machine), and these role-specific inputs:

Unlike a TerraformMachine, a TerraformMachinePool is mutable: its inputs are rebuilt, and its inputs hash recomputed, on every reconcile for the whole life of the object, not only until it is provisioned; see machinepool.md for exactly which changes re-apply it.

The pool role’s renderer defaults failure_domains, cluster_failure_domains and node_labels to []/{} when unset through a step that, as a side effect, HTML-escapes <, > and & inside every string value it touches — including a module-authored exports value carried into captf_cluster_outputs. The cluster and machine roles do not escape these characters (section 3’s example). A \uXXXX escape and its literal character decode to the same JSON string, so a module never sees a difference; only someone reading the raw Secret bytes does. Still, don’t assume the two roles’ encodings match byte for byte when inspecting one.

5. What a cluster hands to its machines and pools

The cluster module’s exports output is the only handoff from a TerraformCluster to the TerraformMachines and pools of its Cluster. It is:

1. Written by the cluster module as any Terraform value (an object is the convention).
2. Stored, like every output, in the cluster’s own Terraform state — nothing else from that state is ever read by the controller for this purpose.
3. Read by each machine’s or pool’s own reconcile whenever it builds its inputs, and rendered into captf_cluster_outputs (see section 4 for the pool role’s one encoding difference).

When it is read. A machine’s inputs (and so exports) are rebuilt on every reconcile while it is not yet provisioned — before its status has latched status.initialization.provisioned = true — whether or not that reconcile ends up starting a Job. Once provisioned, the reconciler never rebuilds that machine’s inputs again, so it never re-reads exports or the bootstrap Secret. Concretely:

• Before a machine is provisioned, every reconcile re-reads the cluster’s current exports and re-renders captf_cluster_outputs from it (along with the rest of the machine’s inputs). Only when an apply Job actually starts does the freshly rendered result get written to the durable Secret; a reconcile that only re-renders without starting a Job leaves the durable Secret untouched.
• Once provisioned, a machine keeps exactly the captf_cluster_outputs its last apply Job start wrote to its durable Secret, forever: a later change to the cluster’s exports reaches only new machines created after that change, never existing ones. This is also why captf_cluster_outputs changes are never a trigger for a machine re-apply: machines are immutable, so their inputs hash, though recorded at apply time like any kind’s, is never compared against a fresh render to decide on a re-apply, and the only way a machine applies again is a fresh first apply (for example, retrying one that was interrupted before it recorded an inputs hash).
• A pool re-reads exports on every reconcile, for its whole life, unlike a machine: a TerraformMachinePool is mutable and never latches captf_cluster_outputs, so a later change to the cluster’s exports reaches every existing pool at its next reconcile, and (unlike a machine) a changed captf_cluster_outputs is in the pool’s inputs hash and so re-applies it.
• Externally managed cluster (cluster.x-k8s.io/managed-by annotation on the TerraformCluster): a machine’s or pool’s captf_cluster_outputs is {} without reading any state, since there is no cluster module run and so no cluster state to read.
• A malformed exports value — one that cannot be canonicalized for hashing, for example it contains a fractional or exponent number — is an output violation on the cluster side. Every unprovisioned machine, and every pool, waiting on it stays DependenciesReady=Unknown with reason WaitingForClusterExports until the cluster module’s exports output is fixed; this is the one operator-visible way a cluster module can break its machines and pools purely through inputs.

A concrete example is the libvirt modules: the cluster module’s exports publishes

output "exports" {
  value = {
    network = local.network
    pool    = local.pool
    vip     = local.vip
    lease   = libvirt_volume.vip_lease.name
  }
}

and the machine module’s captf_cluster_outputs variable carries a validation block that rejects any value missing network, pool or lease — refusing to run against an externally managed cluster ({}) or a cluster module that doesn’t publish a lease, with a clear error instead of an “unsupported attribute” failure deep inside the module.

6. User variables

Besides the contract inputs, a TerraformCluster, TerraformMachine or TerraformMachinePool (and their templates) can pass the module its own variables: instance sizes, CIDRs, SSH public keys, anything the module declares. See Module Variables for how to set them, the merge order between spec.variables and spec.variablesFrom, and what the module sees.

7. What is (and isn’t) in the inputs hash

captf.io/inputs-hash covers exactly: the hash scheme, the contract version, the role, spec.source.image as written (tag or digest — never the resolved digest), the full rendered inputs of the role, and the merged user variables when there are any (their canonical JSON and which of them are sensitive; an object without variables keeps exactly the hash it had before variables existed). Nothing else can change the hash: object metadata (uid, labels, annotations, generation), the resolved image digest, Job metadata, drift ticks, and spec.source.imagePullPolicy are all absent from the hashed value and so cannot trigger anything through it.

What a change does, per kind:

• TerraformCluster (mutable). A hash change re-applies: the controller compares the freshly rendered inputs’ hash against the state’s recorded hash on every reconcile and starts a new apply Job when they differ. A cluster apply that would delete or replace a resource is blocked (ApplyJobSucceeded=False/DestructivePlanBlocked) until the captf.io/approve-destructive-plan annotation names that exact inputs hash, and with spec.applyPolicy: Manual every re-apply also waits for a plan approval; see Plan Approval.
• TerraformMachine (immutable). Nothing, in the sense that matters: a machine’s inputs hash is recorded at apply time like any kind’s, but never compared against a fresh render to decide on a re-apply. Its inputs are still rebuilt on every reconcile up to the first successful apply (section 5), but once that apply has run and the object is provisioned, nothing — not a spec field, not a bootstrap-data change, not an exports change — is ever rendered or checked against it again. Roll out a new Machine instead.
• TerraformMachinePool (mutable). A hash change re-applies, the same way as a TerraformCluster, but with no destructive-plan guard and no applyPolicy: an apply just runs. replicas is excluded from the hash while autoscaling is enabled, so the native autoscaler’s own scaling never triggers a re-apply on its own; see machinepool.md “replicas”.

8. Inspecting the inputs of a live object

The durable Secret holds exactly two data keys, main.tf.json and terraform.tfvars.json, plus three annotations recording the pinned execution context: captf.io/image, captf.io/image-digest, captf.io/identity. Its name follows the pattern captf-inputs-c-<cluster-name> for a TerraformCluster, captf-inputs-m-<machine-name> for a TerraformMachine, and captf-inputs-mp-<pool-name> for a TerraformMachinePool.

To print the tfvars without the sensitive bootstrap_data key:

kubectl -n <namespace> get secret captf-inputs-m-<name> \
  -o jsonpath='{.data.terraform\.tfvars\.json}' | base64 -d | jq 'del(.bootstrap_data)'

That still prints user variables that came from a Secret. To drop them too, delete every key the generated root declares sensitive:

s=$(kubectl -n <namespace> get secret captf-inputs-m-<name> -o json)
sensitive=$(jq -r '.data["main.tf.json"] | @base64d | fromjson | .variable // {}
  | to_entries | map(select(.value.sensitive == true) | .key) | @json' <<<"$s")
jq -r '.data["terraform.tfvars.json"] | @base64d | fromjson' <<<"$s" \
  | jq --argjson drop "$sensitive" 'del(.bootstrap_data) | delpaths([$drop[] | [.]])'

Warning: both the durable Secret (captf-inputs-<kindshort>-<name>) and the per-run Secret (captf-run-<job>) carry bootstrap data in cleartext by design — for control-plane machines that includes the cluster CA and service-account private keys. Treat both Secrets as sensitive: avoid kubectl get -o yaml or -o json on them, which print every key including bootstrap_data unredacted (kubectl describe secret is safe: it prints only key names and byte counts, never values).

9. Limits

The rendered root (main.tf.json plus terraform.tfvars.json combined) is capped at 1,000,000 bytes, comfortably under the ~1 MiB a Kubernetes object can hold, with headroom for the Secret’s own metadata (labels, annotations, owner references). Exceeding it sets ApplyJobSucceeded=False/InputsTooLarge and does not start a Job — retrying cannot help until the inputs themselves shrink (fewer or smaller bootstrap_data, exports or user-variable bytes). The size is reported on the captf_inputs_bytes gauge (per object, by kind/namespace/name) even when the render was too large to run; see metrics.md.

10. Not an input

• Cloud credentials. They come from the TerraformClusterIdentity’s mirrored Secret, injected into the Job as environment variables and as files under a read-only mount. They are never rendered into terraform.tfvars.json; a module reads them the way its provider expects (environment variables, a credentials file), not as a contract input. See Identities.
• Backend configuration. The kubernetes state backend is a partial stub in the generated main.tf.json; the runner completes it at init time with backend-config flags built from the object’s identity, never from tfvars. See runner-cli.md.
• The image reference and pull policy. spec.source.image selects which Job runs, and spec.source.imagePullPolicy how the kubelet pulls it; both are Kubernetes Pod fields, not Terraform inputs. Only the image’s resulting module code and its declared variables reach the tfvars.
• captf.io/inputs-hash and delete guards. Passed to the runner as command-line flags used to gate a destructive plan and, under applyPolicy: Manual, an approved plan hash; see runner-cli.md. Not part of the module’s own input surface. Nor is spec.applyPolicy: it is not hashed, so switching it re-applies nothing.
• Object metadata and controller-written spec fields. uid, labels, annotations, generation, and fields the controller writes back (spec.providerID, spec.providerIDList, an endpoint copied from the module) are never rendered or hashed, so clusterctl move, a label edit, or the controller’s own status-driven writes never trigger a re-apply.
• Module-private variables. A module may declare extra variables of its own only if they carry a default; the generated root never sets them.

See also

• contract/v1alpha1/README.md — the normative type and required-ness of every contract input.
• Module Variables — passing your own variables to a module.
• Machine Pools — autoscaling, replicas write-back and membership refresh.
• Terraform State — where the durable Secrets sit relative to the state backend.

Terraform State

CAPTF stores every object’s Terraform or OpenTofu state in Kubernetes Secrets, alongside the object, and reads it back for outputs, drift and health. This page explains where that state lives, how CAPTF backs it up, and what happens to it when an object is deleted. For an inventory of every Secret CAPTF reads or writes, including state, see Secrets.

The backend

Every TerraformCluster, TerraformMachine and TerraformMachinePool gets its own Terraform or OpenTofu run, backed by the kubernetes backend configured to write into the object’s own namespace, in the default workspace (the only one CAPTF ever uses). The backend’s secret_suffix is derived from the object’s namespace, kind and name, which survive a clusterctl move, and never from its UID, so state follows the object across a move instead of being orphaned.

Secret names and the suffix

The suffix is the first 16 hex characters of hex(sha256(<namespace>/<kind>/<name>)), a hyphen, and a short kind code: c for TerraformCluster, m for TerraformMachine, mp for TerraformMachinePool. It never ends in -<digits>, which the backend would otherwise try to parse as a chunk index.

The base state Secret is named tfstate-default-<suffix>. status.stateSecretSuffix records the suffix; it is informational only, since the controller derives it deterministically and never reads it back.

The state lock is a coordination.k8s.io/v1 Lease named lock-tfstate-default-<suffix> in the same namespace. See Locks below.

Chunking and size caps

A Kubernetes Secret holds at most 1 MiB, and a large Terraform state can exceed that once compressed. Terraform splits an oversized state across additional Secrets named tfstate-default-<suffix>-part-1, -part-2 and so on; OpenTofu does not chunk state at all and always writes a single Secret. CAPTF reads whichever shape is present: it lists every Secret carrying the backend’s own labels for the suffix, orders them by chunk index, and concatenates their payloads before decompressing.

CAPTF caps what it is willing to read: at most 32 chunks and 64 MiB of decompressed state. Real state compresses 10-20x, so these limits are far beyond any plausible cluster or machine state; a state that exceeds them, or whose chunk set is incomplete, duplicated or names an unexpected Secret, is reported corrupt or inconsistent rather than partially read. CAPTFStateNearSecretLimit warns before a state’s compressed size approaches the 1 MiB Secret limit.

Locks

Both runtimes hold the lock for the duration of a run and release it on a clean exit. A runner Job waits up to lockTimeoutSeconds (see Job tuning for the field and its default) for a held lock before failing. Before starting a Job, the controller checks the lock Lease and reads its holder from the backend’s own lock info: a lock whose holder is one of the object’s own runner pods, and that pod either no longer exists or has already exited (a finished Job keeps its pod object until it is pruned), is stale, and the controller has the next Job force-unlock it automatically. A lock whose holder is unknown, or is not one of the object’s own runner pods, is left alone and reported as StateReadable=False/StateLocked. See the stale state lock runbook to force-unlock one by hand.

What CAPTF reads from state

CAPTF parses only the fields it needs from the Terraform state v4 file: the serial, lineage, Terraform/OpenTofu version, the root module’s outputs, and a count of managed resources (a resource with count or for_each counts once). It also tracks the compressed size of the concatenated chunks and, from an annotation on the base Secret, the inputs hash of the last successful apply. Resource instance attributes are never parsed. Outputs may be sensitive and are never logged.

After a successful apply or restore, the controller sets an owner reference to the object on every chunk Secret (so state moves with a clusterctl move and is garbage collected with the object) and records the applied inputs hash on the base Secret, since Terraform’s own chunk writes carry only the backend’s labels.

State backups

The manager keeps versioned copies of an object’s state so a lost, corrupted or wrongly overwritten state Secret can be recovered. Whenever the controller reads a state serial it has not observed before for the object, it copies the state Secrets’ data verbatim into a backup set named captf-state-backup-<suffix>-<serial> (with the same -part-N chunking as the source), owned by the Terraform* object itself rather than by the state, so a backup survives the state Secret being deleted by hand and is garbage collected only when the object is. An unchanged serial is not backed up again.

--state-backups (default 5; see manager flags) sets how many backups per object the manager keeps; it prunes older ones in the same pass. --state-backups=0 takes no new backups but leaves existing ones in place and restorable. status.stateBackups lists the newest backups (see reference/api.md for its fields).

A state that cannot be parsed with the reader’s own limits is never backed up: encrypted, corrupt, inconsistent, or beyond the chunk and size caps above; nor is one the manager failed to copy, for example on a transient API error. Either way the manager logs why and counts it in captf_state_backups_total with result="skipped". A state that cannot be parsed is left that way and the reconcile continues; a failed copy instead retries on the next reconcile, since that serial is not recorded as observed. No backups are taken while an object is being deleted.

Restore

An annotation asks the controller to push a listed backup’s content back into the backend as a new state, through a restore Job that runs state push -force. See the state restore runbook for the procedure, what a restore does and does not undo, and how to verify one.

OpenTofu state encryption

OpenTofu’s client-side state encryption wraps the state file in an envelope with no version field of its own. CAPTF detects that envelope and reports the state as encrypted (StateReadable=False/StateEncrypted) rather than misreading it as corrupt: reading an encrypted state’s outputs is not supported.

State on deletion

Neither backend deletes its own state: a destroy only empties the managed resources it recorded, and the default workspace cannot be deleted. The controller removes the state Secrets and the lock Lease itself, once it is safe to do so: after a destroy Job succeeds, or immediately on deletion of an object that was never applied and so has no state. That same cleanup also deletes the durable inputs Secret, the object’s run lease and, for a TerraformCluster, its cluster write lease. State backups are not deleted by this cleanup; they are owned by the object and are garbage collected when Kubernetes removes it after its finalizer is gone. If the finalizer is removed by hand before the state is cleaned up, see the stuck destroy runbook.

See also

• Secrets for the full Secret inventory, including state and its backups.
• State restore runbook.
• Stale state lock runbook.
• Stuck destroy runbook.
• Unreadable state runbook.

Drift and Health

This page explains how CAPTF checks a provisioned TerraformCluster, TerraformMachine or TerraformMachinePool against reality: the drift schedule, what the module’s health output means, and how both feed the InfrastructureHealthy and Ready conditions and, for machines, Cluster API remediation. It is for anyone who wants to understand the behavior before changing it. To configure drift checking, see Drift; to configure machine remediation, see Machine Remediation; for the full reason tables, see Conditions.

What runs, and when

Two kinds of Job read an object’s infrastructure after it is provisioned:

• A refresh Job runs apply -refresh-only: it updates the state from reality and produces a fresh health reading, but plans nothing and finds no drift.
• A drift Job does the same refresh, then plans with -refresh=false: a plan with no changes means no drift, and any add, change or destroy in the plan is drift. A cluster or pool plans against its freshly rendered current inputs (falling back to the durable inputs Secret only when the current ones can’t be built); a machine, being immutable, always plans against its durable inputs. See Job inputs.

Both count as one health sample once the object is provisioned, and both set the DriftJobSucceeded condition to report whether the Job succeeded; only a drift Job sets DriftDetected. A failed refresh or drift Job is retried under the reconciler’s normal backoff; see Reconcile flow for how retries and requeues work in general.

A drift check runs on a schedule: spec.drift.intervalSeconds, inherited from the TerraformCluster’s spec.defaults.drift for a machine or pool that sets none, else the manager’s --drift-default-interval (30 minutes by default; see Manager flags). The first check after provisioning is due one interval after the last successful apply, not immediately; each object’s schedule is jittered deterministically by its UID so that objects created together don’t all check at once. TerraformCluster and TerraformMachine both let spec.drift.intervalSeconds: 0 turn drift checks off entirely, which normally also stops routine health sampling after provisioning, since no more refresh or drift Jobs run on a schedule (a reading of InstancePending still gets occasional extra refreshes on its own, below). A TerraformMachinePool’s own interval must be at least 1 second — the CRD rejects 0 outright — and even a cluster’s spec.defaults.drift.intervalSeconds: 0, which does disable a machine’s drift, leaves a pool’s on the manager’s default instead: a pool’s periodic drift Job is what feeds its refreshed instance count into a plan, so disabling it would stop that count from ever reaching a plan; see Machine pools.

Outside the drift schedule, a TerraformMachine or TerraformMachinePool also takes a refresh right after each successful apply, to get an immediate health reading (unless the apply’s own outputs already gave a definite one); a TerraformMachinePool refreshes again on its own membershipRefreshIntervalSeconds cadence to keep membership current regardless of drift (see Machine pools), and, with remediation.annotateMachine set, a machine refreshes on remediation.healthCheckIntervalSeconds independent of drift too (see Machine Remediation); none of these apply to a TerraformCluster. For a TerraformCluster or TerraformMachine, while a reading is InstancePending, the next refresh backs off on its own doubling schedule (30 seconds up to 5 minutes) instead of waiting for the next drift interval, so a newly launched instance is checked again quickly. A TerraformMachinePool’s pending reading instead refreshes every 30 seconds flat, the same fixed cadence its membership convergence uses, since the two converge together; see Machine pools.

Report or remediate

Drift found by a drift Job is either just recorded or automatically corrected, per the object’s drift action, Report or Remediate:

• Report (the default for every kind) only sets DriftDetected; it records the finding but applies nothing.
• Remediate additionally re-applies the object’s current inputs to remove the drift.

Only a TerraformCluster’s own spec.drift.action and a TerraformMachinePool’s own spec.drift.action (never inherited from the cluster’s spec.defaults.drift, which has no action field at all) can select Remediate. A TerraformMachine’s drift policy has no action field either, and is always Report: a machine’s instance is immutable infrastructure, replaced by a Cluster API rollout, not reconciled in place by a re-apply.

When a pool or cluster remediates, a finding sets DriftDetected True/DriftPending rather than True/DriftReported, and the reconciler starts an apply of the current inputs. While that apply runs, DriftDetected reads True/DriftRemediating; if the apply succeeds after the drift check that found the drift, DriftDetected clears to False/NoDrift at once, without waiting for the next drift check. If the apply instead fails, is blocked, or stops because its approved plan changed, DriftDetected falls back to True/DriftPending with a note of what happened, and the reconciler keeps retrying the remediation apply (under the normal backoff) until as many attempts have failed since the last successful drift check as the object’s failed-Job history keeps (see Job tuning); past that, the drift stays pending until the next successful check re-evaluates it.

A remediation apply is still an apply of that kind, with the same guard as any other:

• A TerraformCluster’s remediation apply goes through the same destructive-plan guard as any other cluster apply — a plan that would delete or replace a resource blocks (ApplyJobSucceeded=False/ DestructivePlanBlocked) until it is approved — or, under spec.applyPolicy: Manual, the plan-approval flow instead of the guard. See Plan preview and approval.
• A TerraformMachinePool’s remediation apply is not guarded: a pool has no applyPolicy and no destructive-plan guard, so it just runs.

DriftDetected

DriftDetected is negative polarity: True means drift was found, and it is never an input to Ready, so drift alone never makes an object un-Ready. Before the first drift check it reads Unknown/DriftNotChecked. Its full reason table, including the messages each reason carries, is in Conditions.

From module health to InfrastructureHealthy

Every module role (cluster, machine, machinepool) declares a health output with a state (pending, running, degraded, stopped, terminated or unknown) and a healthy boolean, plus optional message and reasons. Each refresh or drift Job’s outcome — and, for a machine or pool, the reading an apply itself provides — maps to InfrastructureHealthy:

message and reasons, when the module sets them, are joined into the condition’s message. The full reason table is in Conditions.

For a TerraformMachine specifically, a provider_id output that turns null after provisioning is read as terminated even though the module reported no such health state: the instance is gone, whatever the health output says, and spec.providerID is kept rather than cleared.

InfrastructureHealthy and Ready

InfrastructureHealthy is not itself mirrored to Cluster API, but once an object is provisioned it becomes, with Deleting (and for a pool also ApplyJobSucceeded), the entire set of conditions that Ready summarizes — down from the full set of dependency, credential, RBAC, apply, state and output conditions Ready watches beforehand. See Reconcile flow for how status.initialization.provisioned latches, and Conditions for exactly which conditions feed Ready, per kind and per phase.

Unhealthy samples

A TerraformMachine counts consecutive unhealthy samples in status.unhealthySamples: each successful refresh or drift Job after provisioning is one sample (an apply’s own reading counts too, when it already gives a definite reading and stands in for the post-apply refresh). An InfrastructureHealthy reading of Healthy resets the count to zero; InstanceUnhealthy, InstanceDegraded or InstanceStopped adds one; InstancePending, HealthUnknown or InstanceTerminated leaves it as it is.

This count is what drives Cluster API machine remediation: see Machine Remediation for the threshold, the terminated-instance shortcut, the cluster.x-k8s.io/remediate-machine annotation and its withdrawal.

See also

• Drift
• Machine Remediation
• Plan preview and approval
• Reconcile flow
• Conditions

Security Model

This page states CAPTF’s trust boundary: what creating a TerraformCluster, TerraformMachine or TerraformMachinePool grants, what the Job that runs it can read, and what CAPTF keeps out of status, events and logs. Read it before deciding who may create or update a Terraform* object, and before deciding whether two tenants can share a namespace. For the mechanics behind each control, see Identities and Credentials, RBAC and Secrets.

A Terraform* object is a Pod

Once its Cluster API owner references it, a TerraformCluster, TerraformMachine or TerraformMachinePool makes CAPTF run Jobs whose main container runs spec.source.image, with jobs.env, as the namespace’s captf-runner ServiceAccount (or an opted-in override named by jobs.serviceAccountName), with the resolved identity’s credentials injected as environment variables and mounted as read-only files (Identities and Credentials). An object no owner references runs nothing (see Owner references are checked). Running a Job is equivalent to granting whoever controls the image everything the Pod runs with: the runner’s access to Secrets in the namespace (below) and the identity’s cloud credentials. Module code, and so any image a Terraform* object names, runs with that access; a provider "kubernetes" {} block or a local-exec provisioner can use it directly.

Consequences:

• The source image is the trust boundary. Whoever may set spec.source.image on an owned Terraform* object in a namespace, or on the template it is cloned from, already has, in effect, the Secret access described below and the cloud credentials of every identity allowed in that namespace. TerraformCluster and TerraformMachinePool are mutable, so update on them is enough. Restrict create/update on terraform* kinds and their templates to principals who already hold that. The controller checks only that spec.source.image is a syntactically valid image reference; it does not police which registry or repository it names. An admission policy on spec.source.image by registry prefix is the recommended control for restricting which images a namespace may run.
• One identity and one workload cluster per tenant namespace. Every Terraform* object in a namespace, and every image any of them names, can read the Secrets described below, including the credential mirrors of every other identity in use in that namespace and the state and inputs of every other object there. A namespace is the boundary between tenants; sharing one namespace between two tenants’ clusters or identities gives each tenant everything described in this page for the other’s cluster too.
• Pod Security Admission applies to Jobs like any workload. A namespace that enforces it gets the defaults described in Pod security.

Owner references are checked against the owner

CAPTF treats a Cluster API object as the owner of a Terraform* object only when the owner references it back. For a TerraformMachine, the Machine named by its Machine-kind ownerReferences entry must pass three checks:

• The Machine’s spec.infrastructureRef names this TerraformMachine: matching apiGroup, kind: TerraformMachine and name. Cluster API sets it before it adds the owner reference, so every TerraformMachine Cluster API creates passes.
• When the ownerReferences entry carries a UID, it equals the Machine’s UID.
• When the TerraformMachine has a cluster.x-k8s.io/cluster-name label, it equals the Machine’s spec.clusterName.

TerraformMachinePool applies the same checks to its MachinePool, using spec.template.spec.infrastructureRef, and TerraformCluster to its Cluster, using spec.infrastructureRef.

An owner that fails any check is not an owner: DependenciesReady=False/OwnerMismatch names the failed check, no Job runs, and CAPTF writes nothing to that object or its Cluster: no remediation annotation on a Machine, no replica count on a MachinePool. Deleting the Terraform* object works as it does when its owner is gone: destroy runs from the durable inputs.

The TerraformMachine delete webhook uses the same checks. It refuses a direct delete only while an owner that passes them exists and is not being deleted, so an object whose ownerReferences name a Machine that does not own it can always be deleted.

create on a terraform* kind alone therefore runs nothing: a Job starts only once a Cluster API object that references the new object exists. Who may create those, and who may set spec.source.image on an owned object, is what the section above describes.

What the runner can read, and why

The runner ServiceAccount’s permissions come from a single, static ClusterRole bound namespace-by-namespace; see RBAC for the exact rules and how a custom ServiceAccount opts in. On Secrets, it holds get, list, create, update and delete, and none of that is scoped by name or label: the Terraform and OpenTofu Kubernetes state backend needs list to enumerate its state chunks and workspaces on every read and write, and list (like create) cannot be restricted to named Secrets at all. get, update and delete could in principle be scoped with resourceNames, but the backend names each state chunk itself, per apply and per workspace, so no fixed rule can list them in advance. The runner, and therefore any module image, can as a result read, replace or delete every Secret in its namespace, which includes:

• the state and inputs Secrets of every Terraform* object in the namespace, not only the one the running Job belongs to;
• the credential mirrors (captf-creds-*) of every identity in use in the namespace, not only the one the running Job was given (Secrets lists every Secret CAPTF reads or writes and its sensitivity);
• any CAPI core Secret in the namespace, such as a cluster’s kubeconfig, certificate authority (CA) or a Machine’s bootstrap data. Write access here means a hostile module can substitute a cluster’s CA or kubeconfig, not only read them.

This is why one identity and one workload cluster per namespace, above, is the only real isolation CAPTF offers between tenants sharing a management cluster. The manager itself can also read every Secret in the cluster, as any CAPI infrastructure provider that runs the Kubernetes state backend effectively can.

Image pinning by digest

The first time an apply Job of a Terraform* object succeeds, the controller records the image digest the kubelet actually ran (read from the pod’s container status, not from spec.source.image) as captf.io/image-digest on the object’s durable inputs Secret; a failed apply pins nothing. See Annotations, Labels and Finalizers for the annotation.

• TerraformMachine is immutable (the kinds): once pinned, every later drift check and destroy Job for that machine runs repo@sha256:…, never the tag in spec.source.image. A tag that moves after the successful apply can therefore never change the code that destroys an existing machine.
• TerraformCluster and TerraformMachinePool are mutable: each spec-driven apply re-resolves the tag and re-pins the digest it ran.

Pinning guards against an accident — a tag moved out from under a running cluster changing what a later destroy runs — not against a hostile image: the pinned digest is whichever image spec.source.image named when the apply succeeded, and that image’s own runtime computed the plan and ran the providers. Approving a blocked destructive plan or a Manual-policy plan preview is exactly as privileged as setting spec.source.image, since both need update on the object; see Plan Approval.

CAPTF’s own manager image is not signed, and nothing in CAPTF verifies a module image’s signature before running it: digest pinning fixes which image ran after the fact, it does not check who published it.

Pod security

The Job’s pod defaults satisfy the Pod Security baseline profile without any configuration: seccomp defaults to RuntimeDefault, and fsGroup defaults to 65532 so a non-root image user can read the credential files, mounted at mode 0440, through that supplementary group regardless of the image’s own user or group. The main container additionally defaults to allowPrivilegeEscalation: false, capabilities.drop: [ALL] and readOnlyRootFilesystem: true, and the runner’s own init container is fixed non-root, fully locked down, and never configurable. runAsNonRoot is not defaulted at the pod level, because an image built FROM hashicorp/terraform runs as root unless it sets USER; reaching the restricted profile needs an image that tolerates runAsNonRoot: true, set through jobs.podSecurityContext or jobs.securityContext.

Regardless of profile, the admission webhook rejects privileged: true, allowPrivilegeEscalation: true and any capabilities.add in jobs.securityContext, on every object and template that carries a jobs policy (spec.jobs, spec.defaults.jobs on a TerraformCluster, and the same field on a TerraformMachine, TerraformMachinePool and all three *Template kinds), because that container holds the resolved identity’s cloud credentials. Pod Security Admission remains the namespace-wide control for everything else a jobs policy does not set, such as host namespaces and volume types.

What CAPTF keeps out of status, events and logs

Terraform* object status is readable by anyone who can get the object — far more people, in general, than can read Secrets in the namespace — so it never carries raw process output. status.lastRun.error.summary is the runner’s own short description of a failure, at most 512 bytes, never the failing step’s stderr; the full output stays in the Job’s own logs, which need pods/log access to read. The events the runner emits on the object (RunStarted, StepStarted, …, RunFinished; see Observability) carry step names, exit codes, durations and resource-change counts, and, on failure, the same curated summary as status — never tfvars, plan output or resource values.

A module variable sourced from a Secret (Module Variables) is automatically declared sensitive = true in the generated root — one sourced from a ConfigMap or given inline never is — so Terraform redacts it from the Job’s own plan and apply output and the controller redacts it from its own trace-level logs. That redaction stops there: like every other input, the value is written in clear into the object’s durable and per-run inputs Secrets and into the Terraform state Secret, so anyone who can read Secrets in the namespace can read it in either place (Secrets, Job Inputs). Cloud credentials never go through this path at all: the resolved identity’s credentials are mounted into the Job and never rendered into a variable, so they never reach the inputs Secrets or the state.

Network exposure

CAPTF ships no NetworkPolicy by default; applying one is an opt-in step covered in Installation. Without it, nothing restricts which pods the manager or a Job can reach, or which pods can reach them, beyond whatever the cluster otherwise enforces.

The manager needs only inbound traffic to its webhook, metrics and health-probe ports, and outbound traffic to DNS, the API server and the registries it reads module image metadata from. A Job’s pod is the more sensitive workload: it holds the resolved identity’s cloud credentials and a ServiceAccount token that can write every Secret in its namespace (above), so its egress is worth restricting to DNS, the API server, and the specific provider and registry endpoints the module it runs needs — nothing reaches it inbound. A NetworkPolicy only has an effect on a CNI that enforces one.

See also

• Identities and Credentials
• RBAC
• Secrets
• The Kinds

Identities and Credentials

A TerraformClusterIdentity is a cluster-scoped object that names a Secret of cloud credentials and the namespaces allowed to use it. Every TerraformCluster, TerraformMachine and TerraformMachinePool needs one, directly or by inheriting its cluster’s, before it can run a Job. This page covers creating an identity, choosing which namespaces it allows, referencing it from each kind, rotating and revoking its credentials, and deleting it. For the trust an identity’s credentials carry once mounted into a Job, see Security model.

Before you begin

• Cluster-scoped access to create TerraformClusterIdentity objects, and namespace access to create the Secret it names.
• get access to that Secret: creating an identity, or pointing an existing one at a different Secret, checks it.
• A TerraformCluster, TerraformMachine or TerraformMachinePool to reference the identity from, or a namespace to create one in.

How credentials reach a Job

An identity’s credentials never reach a Job directly. Into each allowed namespace where an object uses the identity, the controller mirrors the source Secret as captf-creds-<identity> and keeps it in sync. Each Job that resolves to this identity gets that mirror both ways:

• envFrom, so every key becomes an environment variable — the convention most provider SDKs (AWS_*, ARM_*, GOOGLE_*, …) read. A key starting with TF_ or KUBE_ never reaches the module’s environment, except TF_IN_AUTOMATION, TF_INPUT and KUBE_NAMESPACE, which the Job sets itself and which a Secret key cannot override; see Runtime environment for why.
• A read-only file mount at /var/run/captf/credentials/<key>, one file per key, for file-based authentication (GOOGLE_APPLICATION_CREDENTIALS, a kubeconfig, a PEM key). The TF_ and KUBE_ filtering above applies only to the environment: a matching key still appears as a file.

The runner ServiceAccount that a Job runs as can itself read every Secret in its namespace; the identity mechanism controls which credentials a Job carries, not what its ServiceAccount could otherwise reach — see RBAC. The mirror Secret itself, its lifecycle and its sensitivity are cataloged in Secrets; its labels and annotations are listed in Annotations and labels, and the MirrorCreated/MirrorRemoved events in Events.

Create the credentials Secret

Create a plain Opaque Secret with one key per credential your module’s providers read:

apiVersion: v1
kind: Secret
metadata:
  name: <identity-name>
  namespace: <credentials-namespace>
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: <value>
  AWS_SECRET_ACCESS_KEY: <value>

• <identity-name> — reused below as the identity’s own name; the two do not have to match, but it keeps the pair easy to find.
• <credentials-namespace> — any namespace the identity’s creator can read Secrets in. It is commonly a provider or platform namespace, separate from the tenant namespaces that use the identity.

The keys are exactly what reaches every Job that resolves to this identity: name them for what your module’s providers expect, not for CAPTF.

Create the identity

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformClusterIdentity
metadata:
  name: <identity-name>
spec:
  secretRef:
    name: <identity-name>
    namespace: <credentials-namespace>
  allowedNamespaces:
    list:
      - <tenant-namespace>

allowedNamespaces decides which namespaces may use the identity:

An empty selector reads as “every namespace” rather than “no namespace”, so allowedNamespaces: {} is rejected outright rather than treated as one or the other. list accepts up to 100 namespace names; selector follows the usual label selector rules. Full field details, including size limits, are in the API reference.

Creating the identity, and any later change to spec.secretRef, runs a SubjectAccessReview for the requesting user to get the named Secret, and the request is rejected if they may not. This stops a role that may manage identities but not read Secrets from pointing one at a Secret it cannot read and having the controller mirror it somewhere that role can. Other updates, including allowedNamespaces, are not re-checked.

Reference it

An identity is used through identityRef, which names it by name:

• TerraformCluster: spec.identityRef is required and mutable — changing it re-resolves the cluster’s credentials on its next reconcile.
• TerraformCluster defaults for machines and pools: spec.defaults.identityRef, mutable, is the identity a machine or pool without its own identityRef uses; unset, they fall back further to the cluster’s own spec.identityRef. See Kinds for how spec.defaults inheritance works, and Templates and ClusterClass for a TerraformClusterTemplate that leaves identityRef for a ClusterClass patch.
• TerraformMachine: spec.identityRef is optional, and immutable once the object is created — including from unset to a value. Change it by replacing the machine (a MachineDeployment or control-plane rollout), not by editing it in place. A machine created with no identityRef keeps falling back to its cluster’s current default or spec.identityRef, so changing those still changes which credentials such a machine uses.
• TerraformMachinePool: spec.identityRef is optional and mutable, with the same fallback as a machine.

identityRef.name must resolve to an identity that exists and allows the object’s namespace; the Conditions reference has every reason IdentityAllowed and CredentialsMirrored can carry.

Confirm it worked

kubectl get terraformclusteridentity <identity-name>
kubectl describe terraformcluster <cluster-name> -n <tenant-namespace>

The identity’s own Ready condition is True/SecretFound once its Secret exists, False/SecretNotFound otherwise, and status.namespaces lists every namespace currently holding a mirror of it. On the object that references it, IdentityAllowed and CredentialsMirrored turn True once the namespace is allowed and the mirror is in place; until then no Job for that object starts.

Rotate credentials

• Edit the credentials Secret’s data in place, keeping its name and namespace. The controller cannot watch that Secret, so this is not picked up at once: the mirror in each allowed namespace is rewritten the next time an object that uses the identity reconciles for any reason, and at the latest within one --sync-period (10 minutes by default). A Job created after that reconcile gets the new values through its envFrom. A Job already running gets them too, but only in its file mount: the kubelet resyncs that volume from the mirror on its own schedule, even for a pod that started before the rotation; whether a long-running provider process re-reads a changed file is up to the provider. Only the environment is fixed for the life of the pod.

  A Job already running when you rotate keeps the old credentials in its environment for its whole run: the old credential must stay valid until every mirror has picked up the rotation and the longest activeDeadlineSeconds any in-flight Job could still run for has passed (default 3600 seconds; see Deadlines and lock waits), not just until you edited the Secret.

  Every mirror carries a captf.io/source-hash annotation of the source Secret’s data at the time it was last written; it changes whenever the mirror is rewritten, so watching it move off the value it held before the edit confirms that namespace’s mirror has picked up the rotation, without comparing credential values directly:

  kubectl get secret captf-creds-<identity-name> -n <tenant-namespace> \
    -o jsonpath='{.metadata.annotations.captf\.io/source-hash}'
  

  Repeat for every namespace status.namespaces lists, or diff the mirror’s data against the source Secret’s data directly if you want to confirm the values themselves rather than only that a rewrite happened. Only once every mirror has moved past its pre-rotation hash is the old credential safe to revoke.

• Point the identity at a different Secret, by editing spec.secretRef. This is a change to the identity object itself, so it is watched: every object that uses the identity reconciles at once and refreshes its mirror. It also re-runs the SubjectAccessReview, so the user making the change needs get access to the new Secret.

Revoke access

Editing spec.allowedNamespaces to drop a namespace is a change to the identity object, so every object of that namespace using the identity reconciles at once: the controller starts no new Job for it, leaves any infrastructure it already created alone, and deletes the mirror Secret in that namespace, whatever objects still reference it. This also blocks a destroy: deleting such an object waits, reporting that the destroy is waiting for the identity to allow the namespace again, until access is restored.

Delete an identity

Deletion is refused while:

• a TerraformCluster, TerraformMachine or TerraformMachinePool, in any namespace, currently resolves to this identity through its own identityRef or a cluster’s fallback — an object being deleted still counts, since its destroy still needs the credentials; or
• status.namespaces still lists a namespace holding a mirror of it.

The rejection names one object still using it, or every namespace still holding a mirror. Switching an object’s identityRef away from this identity does not by itself release its mirror in that namespace: the mirror keeps that object as an owner until the object is deleted, so status.namespaces keeps listing the namespace, and the identity cannot be deleted, until it is. If nothing resolves to the identity there anymore, deleting the mirror Secret directly also releases the namespace: it is only a copy, so this does not touch the source Secret, and the identity’s status catches up as soon as the deletion is seen. Deleting an identity never deletes the Secret it names: that Secret belongs to you, and stays behind for you to remove or reuse.

Move considerations

TerraformClusterIdentity moves with clusterctl move, but its credentials Secret deliberately does not: see the move runbook for why, and the procedure for copying it to the target cluster yourself.

See also

• Security model — the trust boundary a mounted identity’s credentials sit inside.
• Secrets — every Secret CAPTF reads or writes, including the credential mirror.
• RBAC — the runner ServiceAccount that Jobs run as.
• Kinds — how spec.defaults inheritance works.
• Conditions — every IdentityAllowed and CredentialsMirrored reason.
• Module Variables — a variablesFrom Secret is a different mechanism: it supplies module-specific values (sizes, CIDRs, passwords), never cloud credentials, and CAPTF only ever reads it, in place, rather than mirroring it like an identity’s Secret.

Module Variables

This page shows you how to pass your own variables to a TerraformCluster, TerraformMachine or TerraformMachinePool’s module: instance sizes, CIDRs, SSH keys, database passwords, anything besides the contract inputs CAPTF sets itself.

Before you begin

• The module declares each variable you set, with a default so it still plans when nobody sets the variable (contract/v1alpha1/common.md). A variable the module does not declare fails the apply.
• To pass a variable from a ConfigMap or Secret, you need permission to create and label one in the object’s namespace.

Set an inline variable

Add the variable to spec.variables (or spec.template.spec.variables on a *Template kind), a JSON object:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachineTemplate
metadata:
  name: workers-large
spec:
  template:
    spec:
      source:
        image: ghcr.io/example/aws-machine:v1.4.0
      variables:
        instance_type: m6i.xlarge
        root_volume_gib: 100
        subnet_ids: [subnet-0a1, subnet-0b2]

Each key becomes a named argument of the module: the generated root declares it without a type and passes it through, so the module’s own declared type converts the value (a ConfigMap string "100" becomes the number 100 for a number variable).

Set a variable from a ConfigMap or Secret

1. Create the ConfigMap or Secret in the object’s namespace, and label it so CAPTF is allowed to read it:

   kubectl create configmap aws-sizing -n <namespace> \
     --from-literal=instance_type=m6i.xlarge
   kubectl label configmap aws-sizing -n <namespace> captf.io/variables=true
   

   <namespace> is the object’s own namespace: variablesFrom never reaches across namespaces. This Secret is unrelated to the credentials Secret a TerraformClusterIdentity names: the two are separate mechanisms with separate labels and separate readers (see Identities and Credentials).

   The manager watches only labeled ConfigMaps and Secrets, and reads their data straight from the API server when it renders a Job; it never caches it. The label is an explicit opt-in, part of the trust boundary: an object cannot read an arbitrary Secret of its namespace just by naming it.

2. Reference it from spec.variablesFrom:

   spec:
     variablesFrom:
     - configMapRef:
         name: aws-sizing
     - secretRef:
         name: db-credentials
       format: JSON
     - configMapRef:
         name: team-overrides
       optional: true
   

   Every data key of a labeled source becomes a variable of that name. Set optional: true on a source that may not exist yet: a missing or unlabeled required source blocks the object at DependenciesReady=False/VariablesSourceNotFound and no Job starts; an optional one just contributes nothing. Every value, including a Secret’s, must be UTF-8; a ConfigMap’s binaryData is read the same as data.

3. Confirm: DependenciesReady turns True once every required source resolves, and the object’s next Job carries the variable. Inspect the rendered terraform.tfvars.json of a live object as described in Job inputs.

Merge order and formats

1. spec.variablesFrom, in list order: every source’s keys become variables, and a later source wins on a key it shares with an earlier one.
2. spec.variables (inline) wins over every source.

A source’s format decides how its values are read:

• String (the default): each value is passed as a string. Use it for scalars; the module’s declared type converts "3", "true" and "0.5".
• JSON: each value is parsed as JSON, for lists, maps and objects (["a","b"], {"team":"a"}). A value that is not valid JSON holds the object at DependenciesReady=False/VariablesInvalid, naming the source and the key, never the value.

Names and limits

A variable name is a Terraform identifier: ^[a-zA-Z_][a-zA-Z0-9_-]*$. Rejected, whatever the format:

• a name starting with captf_;
• a contract input of the object’s role — a machine cannot set machine_name, but a cluster can, since machine_name is not one of its inputs (contract reference);
• a module meta-argument: source, version, providers, count, for_each, depends_on, lifecycle, locals.

spec.variables holds at most 256 keys, and when set, at least one; the admission webhook rejects a bad inline key or too many of them at once, and also rejects a variablesFrom entry that names both a ConfigMap and a Secret, or neither. A bad key in a referenced source is caught only when the controller reads it, as VariablesInvalid naming the key. spec.variablesFrom holds at most 16 sources.

Variables count toward the rendered-inputs limit shared with every other input (1,000,000 bytes): past it the apply is refused with ApplyJobSucceeded=False/InputsTooLarge (Size limits). They are also part of the inputs hash that decides whether a mutable object’s next reconcile re-applies.

Sensitivity

A variable’s winning value (after the merge above) is declared sensitive = true in the generated root when it came from a Secret, so Terraform and OpenTofu redact it in plan and apply output. A value that ends up inline or from a ConfigMap is not sensitive, even if an earlier, losing source was a Secret, and even if the module’s own declaration marks its variable sensitive. Sensitive or not, every variable is still stored like any other input: in the object’s inputs Secrets and in the state (Secrets, Terraform state).

What a change does per kind

• TerraformCluster: the variables are part of the inputs hash. Editing spec.variables, spec.variablesFrom or the data of a referenced source re-applies the module, guarded like any other input change (Plan approval). The controller re-reads every referenced source on each reconcile, and a change to a labeled source it references wakes it.
• TerraformMachine: spec.variables and spec.variablesFrom are immutable. The controller reads the sources until the machine is provisioned; after that it never reads them again, and the machine’s destroy, refresh and drift runs use the variables already pinned in its durable inputs Secret. Editing a referenced ConfigMap or Secret therefore affects only machines created afterward: roll the MachineDeployment, or bump the template, to replace existing ones.
• TerraformMachinePool: spec.variables and spec.variablesFrom are mutable, and the sources are re-read on every reconcile for the pool’s whole life, the same as a TerraformCluster; a change to a labeled source it references wakes it immediately, the same way. The variables are part of the inputs hash, but the resulting re-apply is never guarded: a TerraformMachinePool has no applyPolicy (Machine pools).
• Templates: spec.template.spec of a *Template kind, so its variables and variablesFrom too, is immutable like the rest of the template (The Kinds); a ClusterClass topology patch on spec.template.spec.variables rolls out through a new template (Templates and ClusterClass).
• spec.defaults on a TerraformCluster does not cover variables: a machine or pool without its own identityRef, jobs or drift inherits the cluster’s, but variables are never inherited, so set them on the machine, pool or their templates directly.
• clusterctl move never carries a variablesFrom source: CAPTF puts no owner reference on a ConfigMap or Secret it reads. A TerraformCluster or TerraformMachinePool, or a machine not yet provisioned, waits at VariablesSourceNotFound on the target until you recreate the source there (clusterctl move).

Troubleshooting

See conditions.md for every DependenciesReady reason.

See also

• Identities and Credentials — the separate Secret mechanism for cloud credentials.
• common.md — how a module declares a user variable.
• Job inputs — the full render pipeline and the inputs hash.
• Templates and ClusterClass — patching a variable per Cluster.
• Plan approval — the destructive-plan guard a TerraformCluster variable change is subject to.
• reference/api.md — the full field list for variables and variablesFrom.

Templates and ClusterClass

This page covers the clusterctl templates CAPTF ships, how to generate a cluster from them, and the noop ClusterClass: what it creates, its topology variables, and how to patch a module variable onto it. It is for anyone creating clusters with CAPTF, especially with ClusterClass.

Before you begin

• The provider is installed and registered with clusterctl (Installation).
• A TerraformClusterIdentity is applied for the namespace you are generating into (Identities and credentials).
• Your cluster and machine module images are built, linted and pushed (tfcapi-lint, image contract).
• You know the values for the clusterctl variables you need (clusterctl variables).

The shipped flavors

CAPTF ships three flavors as templates/cluster-template*.yaml files; once the provider is registered, clusterctl generate cluster --infrastructure terraform renders one of them from the registered repository (Installation):

Every flavor’s KubeadmControlPlane sets spec.remediation.maxRetry: 3 and retryPeriodSeconds: 300, so a module that fails deterministically stops being retried instead of churning cloud resources, and its localAPIEndpoint.bindPort (init and join) equals Cluster.spec.clusterNetwork.apiServerPort (6443); kubeadm never reads the Cluster field, so keep the two equal if you change one.

The libvirt flavor is for the project’s development host; see the libvirt host guide and the libvirt module. Its identity template is templates/identity-libvirt.yaml, applied instead of the default templates/identity.yaml.

Generate a cluster

Set the identity, image and sizing variables, then generate and apply. Default flavor:

export TERRAFORM_IDENTITY_NAME=<identity-name> \
  TERRAFORM_CLUSTER_IMAGE=<cluster-module-image> \
  TERRAFORM_MACHINE_IMAGE=<machine-module-image>
clusterctl generate cluster <cluster-name> --infrastructure terraform \
  --target-namespace <namespace> \
  --kubernetes-version <version> \
  --control-plane-machine-count <count> --worker-machine-count <count> \
  | kubectl apply -f -

ClusterClass flavor: enable the ClusterTopology feature gate when you register the provider (Installation); it is alpha and off by default. Apply the class to the namespace once, then generate with --flavor clusterclass:

kubectl apply -n <namespace> -f templates/clusterclass-noop.yaml
clusterctl generate cluster <cluster-name> --infrastructure terraform \
  --flavor clusterclass --target-namespace <namespace> \
  --kubernetes-version <version> \
  --control-plane-machine-count <count> --worker-machine-count <count> \
  | kubectl apply -f -

clusterclass-noop.yaml carries no clusterctl variables of its own and no namespace, so apply it once to every namespace that generates clusters from it. Neither flavor defaults CLUSTER_NAME, KUBERNETES_VERSION, CONTROL_PLANE_MACHINE_COUNT, WORKER_MACHINE_COUNT, the two image variables or the identity name: a missing one fails clusterctl generate instead of deploying something unintended. See clusterctl variables for the full list, including the libvirt flavor’s defaults.

The noop ClusterClass

templates/clusterclass-noop.yaml defines ClusterClass noop and the templates it references:

• TerraformClusterTemplate/noop, the infrastructure template.
• KubeadmControlPlaneTemplate/noop-control-plane and TerraformMachineTemplate/noop-control-plane, the control plane and its infrastructure.
• TerraformMachineTemplate/noop-worker and KubeadmConfigTemplate/noop-worker, the infrastructure and bootstrap for the default-worker machine deployment class.

Each of the three Terraform templates carries the placeholder image example.invalid/captf/set-by-clusterclass:unset: a Cluster that fails to patch a real image gets a failed image pull instead of running an unintended module. The class also carries default MachineHealthCheck timeouts for the control plane and the workers; the clusterclass flavor overrides them through Cluster.spec.topology with the same variables as cluster-template.yaml (see clusterctl variables).

The class declares three required string topology variables — identityName, clusterImage and machineImage — described in the ClusterClass topology variables table. Three patches apply them to the templates above:

• identity sets spec.template.spec.identityRef on TerraformClusterTemplate/noop to identityName, and also sets spec.template.spec.defaults.identityRef to the same value, so machines without their own identityRef inherit it.
• clusterImage replaces spec.template.spec.source.image on TerraformClusterTemplate/noop with clusterImage.
• machineImage replaces spec.template.spec.source.image on both TerraformMachineTemplate/noop-control-plane and TerraformMachineTemplate/noop-worker with machineImage, matched by matchResources.controlPlane and matchResources.machineDeploymentClass.names: [default-worker].

cluster-template-clusterclass.yaml sets the three variables from TERRAFORM_IDENTITY_NAME, TERRAFORM_CLUSTER_IMAGE and TERRAFORM_MACHINE_IMAGE.

Patch a module variable through ClusterClass

A ClusterClass patch can also turn a topology variable into a module variable (spec.template.spec.variables; module variables). Declare the topology variable, then add a patch whose jsonPatches write /spec/template/spec/variables (or one key under it, if the template already sets others):

spec:
  variables:
  - name: workerInstanceType
    required: true
    schema:
      openAPIV3Schema:
        type: string
  patches:
  - name: workerInstanceType
    definitions:
    - selector:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
        kind: TerraformMachineTemplate
        matchResources:
          machineDeploymentClass:
            names: [default-worker]
      jsonPatches:
      - op: add
        path: /spec/template/spec/variables
        valueFrom:
          template: |
            instance_type: {{ .workerInstanceType }}

The same patch on TerraformClusterTemplate reaches the TerraformCluster instead, whose spec.variables is mutable: changing the variable re-applies the cluster module, and a destructive plan still waits for approval (plan approval). A ConfigMap or Secret named in variablesFrom is not part of the ClusterClass: create it, labeled captf.io/variables=true, in each namespace that uses the class (module variables).

Template immutability and rolling out a change

A *Template kind’s spec.template.spec cannot be edited in place once created; only its metadata can change (see mutability per kind). ClusterClass topology reconciliation is exempt from that rule for its own dry-runs, which is how patching works at all, but a direct edit to a *Template object is rejected.

Two situations follow from this:

• A field the class exposes as a topology variable. Changing the variable’s value on a Cluster’s spec.topology.variables is enough: since the resulting TerraformMachineTemplate would differ from the one already referenced, and templates are immutable, the topology controller creates a new TerraformMachineTemplate with the patched spec and rolls the affected MachineDeployment or control plane onto it. Changing machineImage on a clusterclass-flavor Cluster works this way.
• A field the class does not expose as a variable, such as a KubeadmControlPlaneTemplate field or a new patch. Create a new template object under a new name with the desired spec.template.spec, then update the ClusterClass’s reference to it — spec.infrastructure.templateRef, spec.controlPlane.templateRef, spec.controlPlane.machineInfrastructure.templateRef, or the matching workers.machineDeployments[].infrastructure.templateRef / bootstrap.templateRef — and apply the class. Topology reconciliation then rolls every Cluster on the class onto the new template, subject to each machine deployment’s or control plane’s own rollout strategy (the noop class’s control plane sets maxSurge: 0; its default-worker machine deployment class leaves the rollout strategy unset, so the default maxSurge: 1 applies).

A TerraformCluster itself is mostly mutable — only spec.controlPlaneEndpoint is immutable once it has a host — so a clusterImage or identityName change re-applies the cluster module in place rather than creating a new object.

See also

• Identities and credentials
• Module variables
• Plan approval
• The kinds
• clusterctl variables

Machine Pools

This page shows you how to create a MachinePool backed by a TerraformMachinePool: a group of nodes CAPTF provisions and scales as one native cloud scaling group (an autoscaling group, a scale set, an instance group, or similar), rather than as individual TerraformMachines. It covers fixed and autoscaled replica counts, the group’s reported members, and deletion. See The Kinds for how a TerraformMachinePool compares to a TerraformMachine, and the machinepool contract for everything the module role must implement.

Before you begin

• The provider installed, and a TerraformCluster provisioned or being provisioned (see Installation).
• A TerraformClusterIdentity allowed in your namespace (see Identities and Credentials).
• A machinepool-role module image: one that implements the machinepool contract, managing one scaling group and reporting its provider IDs, desired capacity and members.

Create a MachinePool

A MachinePool has a single infrastructure object for its whole group, not one per member: MachinePool.spec.template.spec.infrastructureRef names one TerraformMachinePool directly, by kind and name, the same way Cluster.spec.infrastructureRef names one TerraformCluster. There is no per-replica cloning, so you create the TerraformMachinePool yourself rather than pointing at a TerraformMachinePoolTemplate; a TerraformMachinePoolTemplate exists only for a MachinePool a ClusterClass topology manages, covered in Templates and ClusterClass.

The TerraformMachinePool must carry the cluster’s cluster.x-k8s.io/cluster-name label: CAPTF looks up the owning Cluster by that label, not by the MachinePool’s spec.clusterName. Cluster API’s MachinePool controller patches this label on from spec.clusterName once the TerraformMachinePool exists, but only on its own next reconcile; setting the label yourself avoids that wait.

For example:

apiVersion: cluster.x-k8s.io/v1beta2
kind: MachinePool
metadata:
  name: my-cluster-workers
  namespace: team-a
spec:
  clusterName: my-cluster
  replicas: 3
  template:
    spec:
      clusterName: my-cluster
      version: v1.31.4
      bootstrap:
        configRef:
          apiGroup: bootstrap.cluster.x-k8s.io
          kind: KubeadmConfig
          name: my-cluster-workers
      infrastructureRef:
        apiGroup: infrastructure.cluster.x-k8s.io
        kind: TerraformMachinePool
        name: my-cluster-workers
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta2
kind: KubeadmConfig
metadata:
  name: my-cluster-workers
  namespace: team-a
spec:
  joinConfiguration: {}
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachinePool
metadata:
  name: my-cluster-workers
  namespace: team-a
  labels:
    cluster.x-k8s.io/cluster-name: my-cluster
spec:
  source:
    image: ghcr.io/example/machinepool-module:v0.1.0
  identityRef:
    name: aws-prod

The KubeadmConfig is the bootstrap provider’s own object (one per pool, for the same reason: a pool has no per-member Machine to hold one each); its content is the bootstrap provider’s concern, not CAPTF’s. With spec.replicas unset, Cluster API defaults it to 1; set it to the fixed size you want, as above. Every field of TerraformMachinePoolSpec is mutable: changing spec.source, spec.identityRef, spec.variables or any other field re-applies the module on the next reconcile, and there is no spec.applyPolicy and no delete guard (see The Kinds). Pass module-specific configuration through spec.variables or spec.variablesFrom as for any other kind (see Module Variables); spec.jobs tunes the Job the same way it does for a TerraformMachine (see Tuning Jobs). A pool with none of its own inherits spec.identityRef, spec.jobs and spec.drift.intervalSeconds from the owning TerraformCluster’s spec.defaults.

node_labels (rendered from spec.template.metadata.labels, not the TerraformMachinePool’s own metadata) and the other inputs a module sees are listed in the machinepool contract and the environment reference; this page does not restate them.

Choose fixed replicas or autoscaling

With no autoscaler annotations, MachinePool.spec.replicas is the sole source of desired capacity, exactly as in the example above: change it to resize the group.

To let the module’s own native autoscaling policy own the desired count instead, set both annotations Cluster API’s autoscaler contract defines, on the MachinePool:

apiVersion: cluster.x-k8s.io/v1beta2
kind: MachinePool
metadata:
  name: my-cluster-workers
  namespace: team-a
  annotations:
    cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "3"
    cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "10"
spec:
  clusterName: my-cluster
  template:
    spec:
      clusterName: my-cluster
      version: v1.31.4
      bootstrap:
        configRef:
          apiGroup: bootstrap.cluster.x-k8s.io
          kind: KubeadmConfig
          name: my-cluster-workers
      infrastructureRef:
        apiGroup: infrastructure.cluster.x-k8s.io
        kind: TerraformMachinePool
        name: my-cluster-workers

Both annotations must be present, parse as non-negative integers, and satisfy min ≤ max, or the pool reports AutoscalingActive=False/AutoscalingAnnotationsInvalid and applies without autoscaling. Valid, the pool reports AutoscalingActive=True/ReplicasManagedByModule: the module owns the group’s desired count and its own scaling policy (target tracking, scheduled, or whatever it implements), and the controller claims the cluster.x-k8s.io/replicas-managed-by annotation on the MachinePool so Cluster API stops treating spec.replicas as authoritative. On every reconcile the controller then writes the group’s observed desired capacity back to MachinePool.spec.replicas, emitting a ReplicasWrittenBack event when it changes; see Annotations, Labels and Finalizers for both keys. Leave spec.replicas unset in this mode: Cluster API defaults and clamps it from the annotations for the first apply, and the write-back takes over from there. Removing both annotations returns spec.replicas to being authoritative and releases replicas-managed-by.

The Kubernetes Cluster Autoscaler itself does not drive these pools: its clusterapi cloud provider requires MachinePool Machines, which CAPTF does not implement. Running it against a CAPTF pool is unsupported, since its spec.replicas patches would be overwritten by the write-back above.

Add an autoscaled pool to a generated cluster

This walks through adding an autoscaled MachinePool to a cluster generated from the default flavor (Templates and ClusterClass), since none of the shipped flavors creates one on their own.

Generate the default flavor with no workers; a MachineDeployment is still created, with spec.replicas: 0, rather than omitted:

clusterctl generate cluster my-cluster --infrastructure terraform \
  --target-namespace team-a \
  --kubernetes-version v1.31.4 \
  --control-plane-machine-count 1 --worker-machine-count 0 \
  | kubectl apply -f -

Add the pool alongside it: a MachinePool with the autoscaler annotations, its KubeadmConfig (the flavor’s bootstrap provider is kubeadm), and the TerraformMachinePool:

apiVersion: cluster.x-k8s.io/v1beta2
kind: MachinePool
metadata:
  name: my-cluster-workers
  namespace: team-a
  annotations:
    cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "2"
    cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "5"
spec:
  clusterName: my-cluster
  template:
    spec:
      clusterName: my-cluster
      version: v1.31.4
      bootstrap:
        configRef:
          apiGroup: bootstrap.cluster.x-k8s.io
          kind: KubeadmConfig
          name: my-cluster-workers
      infrastructureRef:
        apiGroup: infrastructure.cluster.x-k8s.io
        kind: TerraformMachinePool
        name: my-cluster-workers
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta2
kind: KubeadmConfig
metadata:
  name: my-cluster-workers
  namespace: team-a
spec:
  joinConfiguration: {}
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachinePool
metadata:
  name: my-cluster-workers
  namespace: team-a
  labels:
    cluster.x-k8s.io/cluster-name: my-cluster
spec:
  source:
    image: ghcr.io/example/machinepool-module:v0.1.0
  identityRef:
    name: aws-prod

spec.replicas is left unset on the MachinePool: with both autoscaler annotations present and valid, Cluster API defaults and clamps it from min-size/max-size for the first apply, and the pool’s own write-back takes over from there (see Choose fixed replicas or autoscaling above). Apply the three objects, then confirm as in Confirm it worked below.

Set the membership refresh interval

Between applies, the controller runs a refresh to pick up members joining or leaving the group, on spec.membershipRefreshIntervalSeconds (15–86400 seconds; unset or 0 means 60):

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachinePool
spec:
  membershipRefreshIntervalSeconds: 30

A new member is unschedulable until its provider ID reaches spec.providerIDList, so a shorter interval gets new nodes ready for workloads sooner, at the cost of more frequent Jobs. The controller also refreshes right after every apply and, while the group has not converged (spec.providerIDList’s length differs from status.replicas), every 30 seconds, or the configured interval instead when it is shorter.

Check the group’s members

kubectl get terraformmachinepool <name> -n <namespace>

shows the owning cluster, MachinePool, desired replicas and the Ready condition. spec.providerIDList is every non-terminated member, sorted and deduplicated; status.replicas is the desired capacity as of the last refresh; status.instances is the module’s own per-member detail (provider ID, an optional instance ID, addresses, failure domain and health state), capped at 1000 entries:

kubectl get terraformmachinepool <name> -n <namespace> \
  -o jsonpath='{.status.instances}'

See the API reference for every status field. A pool reports provisioned, and the Ready condition (mirrored onto the MachinePool’s InfrastructureReady) true, once its state carries a successful apply and its module reports a health state other than pending; unlike a fixed-replica machine, that latch does not depend on spec.providerIDList being non-empty, since a pool may legitimately scale to zero.

Drift on a pool

A pool’s drift check cannot be disabled: spec.drift.intervalSeconds: 0 falls back to the manager’s default interval rather than turning checks off, because the drift Job’s own refresh is what feeds a plan; without it, a cloud-side scaling change would never register as drift. With spec.drift.action: Remediate, a detected difference re-applies the pool’s current inputs; with autoscaling enabled the module is responsible for excluding its own desired-count attribute from that plan, or every cloud-side scale reports as drift. See Drift for setting the interval and action, and Drift and Health for how a check runs and feeds health.

Delete a MachinePool

Deleting the MachinePool deletes its KubeadmConfig and TerraformMachinePool with it. Nothing blocks a TerraformMachinePool’s deletion: it destroys the group from its durable inputs and removes its finalizer once the destroy Job succeeds, whether or not the owning MachinePool or Cluster still exist. See the reconcile lifecycle for how deletion and finalizers work across every kind.

Confirm it worked

kubectl get terraformmachinepool <name> -n <namespace>

Ready reads True once the group is provisioned, and Replicas shows the desired capacity. kubectl get machinepool <name> -n <namespace> shows the same replica count and InfrastructureReady=True once Cluster API has copied spec.providerIDList and status.replicas across, which happens only once the workload cluster is reachable.

See also

• The Kinds for how a TerraformMachinePool’s mutability differs from a TerraformMachine’s.
• The machinepool contract for every input and output a module role must implement.
• Drift and Drift and Health.
• Templates and ClusterClass for a TerraformMachinePoolTemplate used through a ClusterClass topology.
• Module Variables and Tuning Jobs.
• API Reference, Conditions, Events and Annotations, Labels and Finalizers.

Plan Approval

TerraformCluster guards two things before it changes infrastructure: a plan that deletes or replaces a resource always waits for an approval, and with spec.applyPolicy: Manual every plan waits for one. This page covers both guards, how they interact, and what they do not cover.

Before you begin

• A TerraformCluster whose apply you want to guard or preview.
• kubectl access to annotate it and to read the source container’s logs of its Jobs.

The destructive-plan guard

A TerraformCluster is mutable: a new image tag, an edited field, or drift Remediate re-applies its module. A module release that renames a resource, or a different module pointed at the same state, can plan to destroy infrastructure nobody asked to destroy. So every TerraformCluster apply, including a drift remediation, runs its plan first and stops before applying if any resource’s planned actions include a delete: a removal, or a replacement in either order. Nothing changes and the state is untouched. status.lastRun.error.kind is blocked, ApplyJobSucceeded turns False/DestructivePlanBlocked naming the affected addresses and actions, and a DestructivePlanBlocked event is emitted once per blocked Job. See Conditions and Events for the exact reasons.

While the newest apply of the current inputs hash is blocked, no apply of that hash starts; the controller re-checks at least every ten minutes. A blocked drift remediation keeps drift and health checks running; a blocked input change pauses them, since they would render the unapplied inputs. Changing the inputs, which produces a new hash, starts a new guarded apply at once, and so does approving the blocked hash. A blocked Job counts toward neither the retry backoff nor the remediation failure cap.

Approving a destructive plan

Read the plan first: kubectl logs job/<name> -c source has its human-readable output, and the condition message lists what it deletes or replaces. If the change is intended, approve the inputs hash named in the condition:

kubectl annotate terraformcluster <name> -n <namespace> \
  captf.io/approve-destructive-plan=<inputs-hash> --overwrite

<name> and <namespace> are the TerraformCluster’s; <inputs-hash> is the hash from the condition message.

The approval is by inputs hash: the hash of everything the apply renders, the image reference and every module input (see Job Inputs). It approves exactly those inputs, not the object. Any later change produces a new hash, which the annotation does not name, so the next destructive plan is blocked again without anyone removing the approval.

The approval is consumed once: when an apply of the approved hash succeeds, the controller removes the annotation. A drift remediation re-applies the inputs the state already records, so without this, an approval of an input change would also cover a later destructive remediation of the same inputs; instead, that remediation is blocked again and needs its own approval.

The approval covers the plan computed when the approved apply runs, not the plan you read: something that changed since is covered too. lifecycle { prevent_destroy = true } in the module remains the stronger control for a resource that must never be replaced: it fails even an approved plan. Anyone who can annotate the TerraformCluster can approve; that is the same set of people who can change spec.source.image, which already decides what the module does (see Security Model). See captf.io/approve-destructive-plan for the annotation’s full definition.

A guarded apply whose plan has no changes is never blocked: it stops after the plan step, and the Job succeeds without an apply step, adopting the rendered inputs hash and pinning the image digest as a full apply would; see Reconcile Lifecycle.

Plan preview: applyPolicy Manual

The destructive-plan guard only stops deletes and replacements; an in-place change to a load balancer, a network or a security group still applies on the next edit. Teams that review every infrastructure change set spec.applyPolicy: Manual on the TerraformCluster or its TerraformClusterTemplate. The default, Automatic, keeps the behavior above unchanged. applyPolicy is mutable: switching back to Automatic applies whatever was waiting, and clears status.plan.

1. Plan. When the controller would apply new inputs, it starts a plan Job instead (status.activeJob.operation: plan): init, validate, plan and read the plan back, with no apply step.

2. Review. The runner reports the plan’s counts, the address and action of each changed resource (at most 50, never a value), and a plan hash over the sorted, non-no-op changes. The controller writes this to status.plan (inputsHash, job, planHash, add, change, destroy, resources, truncated, createdAt), sets ApplyJobSucceeded to Unknown/PlanAwaitingApproval with the exact approve command, and emits one PlanReady event with the counts. The Job’s log has the human-readable plan.

3. Wait. Nothing applies and no further Job starts. The controller re-checks at least every ten minutes; a changed annotation or new inputs re-trigger it at once. The condition is Unknown, not False: waiting never turns Ready false.

4. Approve. Set the annotation from the condition message to the plan hash in status.plan.planHash:

   kubectl annotate terraformcluster <name> -n <namespace> \
     captf.io/approve-plan=<plan-hash> --overwrite
   

   <name> and <namespace> are the TerraformCluster’s; <plan-hash> is status.plan.planHash.

5. Apply. The apply Job starts (PlanApproved event), plans again, and applies only if the new plan’s hash is the approved one. If anything changed since the plan was reviewed, it stops before applying: status.lastRun.error.kind is plan-changed, the new plan replaces status.plan, ApplyJobSucceeded turns Unknown/PlanChanged with a new approve command, and a PlanChanged event is emitted. The apply then waits for the approval of the new hash. A changed plan counts toward neither the retry backoff nor the remediation failure cap.

6. Done. When the approved apply succeeds, the controller removes captf.io/approve-plan, clears status.plan, and emits PlanApplied.

captf_plan_approvals_total{result="approved"|"changed"} counts applied approvals and changed plans; see Metrics.

Gated: every apply but the first, which covers an input change (image, spec field, module variable, a Cluster input), a drift remediation, and the retry of a failed apply. Not gated: the first apply of a new cluster (there is no state yet, so nothing to break), deletion, refresh, and drift checks. A plan without changes needs no approval: its hash is that of the empty change list, and the apply proceeds at once, though it plans again first and stops if the plan is no longer empty.

With the destructive-plan guard

Approving a plan also approves its deletes and replacements: they are listed in status.plan.resources, which the reviewer saw, and the apply runs only that exact plan. So under Manual, the captf.io/approve-destructive-plan annotation is never needed; the destructive-plan guard is still armed for an apply that has no plan approval, for example right after switching back to Automatic. lifecycle { prevent_destroy = true } still fails an approved plan.

Caveats

• An approval names a plan hash, not an inputs hash: it approves a set of changed addresses and actions, not the values that change. Two plans that touch the same resources hash the same, whatever the new values are. Review the plan’s log, not only its hash.
• An approval is consumed only when the approved apply succeeds. One left behind, because the plan changed or the inputs changed before the apply ran, approves any later plan with the same hash, for example a recurring drift on the same resource. Remove a stale approval with kubectl annotate terraformcluster <name> -n <namespace> captf.io/approve-plan-.
• status.plan is status, not durable state: after clusterctl move, which does not restore status, the controller plans again, and an approval still on the annotation applies if the new plan hashes the same.
• In a GitOps setup, the annotation is set by a person, or by a pipeline after its own review of status.plan and the plan Job’s log. It is not part of the desired state in Git: a controller that syncs annotations from Git would re-add a consumed approval.

What is not guarded

Neither guard applies to a TerraformMachine or a TerraformMachinePool. A machine is immutable: Cluster API replaces it with a new object instead of changing it in place, so there is nothing for the destructive-plan guard to catch, and applyPolicy does not exist outside TerraformCluster. A machine pool is mutable and its spec.variables and spec.variablesFrom change over its whole life, like a TerraformCluster does, but its applies are never guarded: it has no applyPolicy field, and the destructive-plan guard only watches TerraformCluster applies. See Machine Pools and the reconcile lifecycle.

Confirm it worked

• After approving a destructive plan, kubectl describe terraformcluster <name> -n <namespace> shows ApplyJobSucceeded back to True and the captf.io/approve-destructive-plan annotation gone.
• After approving a plan under Manual, status.plan is empty and ApplyJobSucceeded is True/ApplySucceeded.

See also

• Drift for drift.action: Remediate, which the destructive-plan guard also covers.
• Reconcile Lifecycle for where these guards sit in the reconcile flow.
• Annotations, Labels and Finalizers for every annotation this page uses.
• Conditions and Events for the exact reasons.

Drift

CAPTF periodically refreshes and plans a TerraformCluster, TerraformMachine or TerraformMachinePool to check whether the infrastructure still matches its module’s inputs. This page covers how to set the check interval and action for each kind, disable checks where that is possible, read the results, and remediate what a check finds. See Drift and Health for how the checks work and how they feed the object’s health.

Before you begin

• A provisioned TerraformCluster, TerraformMachine or TerraformMachinePool.
• kubectl access to edit the object and read its status.

Set the drift interval

Every kind checks for drift on a timer: spec.drift.intervalSeconds. Left unset, it uses the manager’s --drift-default-interval flag (30 minutes by default; see Manager Flags).

On a TerraformCluster:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformCluster
spec:
  drift:
    intervalSeconds: 900

A TerraformMachine and a TerraformMachinePool inherit an interval from the owning TerraformCluster’s spec.defaults.drift.intervalSeconds when they set none of their own:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformCluster
spec:
  defaults:
    drift:
      intervalSeconds: 1800

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachine
spec:
  drift:
    intervalSeconds: 900

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachinePool
spec:
  drift:
    intervalSeconds: 900

A machine’s or pool’s own intervalSeconds always wins over the inherited default. spec.drift is operational policy and stays mutable for the object’s whole life, unlike the fields that define the machine or pool itself (see The Kinds for what is mutable per kind).

Set the drift action

spec.drift.action decides what happens when a check finds changes: Report (the default) only records the finding, Remediate applies the object’s current inputs to remove it. Both a TerraformCluster and a TerraformMachinePool accept action:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformCluster
spec:
  drift:
    action: Remediate

A TerraformMachine’s drift is always reported, never remediated: the underlying instance is immutable infrastructure, replaced by a rollout rather than patched in place, so spec.drift has no action field. spec.defaults.drift on the TerraformCluster has no action field either: only a machine’s or pool’s own spec.drift.action sets it.

Disable drift checks

Setting intervalSeconds: 0 disables drift checks on a TerraformCluster or a TerraformMachine. Disabling them also stops every health sample taken after provisioning, since health is otherwise read only from a completed refresh or drift Job: the InfrastructureHealthy condition stops updating (a TerraformMachine still refreshes independently while its spec.remediation.annotateMachine is true; see Machine Remediation).

A TerraformMachinePool’s drift cannot be disabled; see Drift and health for why.

Read the results

DriftDetected reports the last check’s outcome:

• Unknown/DriftNotChecked before the first check completes.
• False/NoDrift when the last check found no changes.
• True/DriftReported when it found changes and the action is Report.
• True/DriftPending when it found changes, the action is Remediate, and the remediation has not started or failed.
• True/DriftRemediating while a remediation apply runs.

kubectl get terraformcluster <name> -n <namespace> \
  -o jsonpath='{.status.conditions[?(@.type=="DriftDetected")]}'

Its message names the drift Job and, when changes were found, the counts of resources to add, change and destroy. status.lastRun.drift carries the same counts as separate fields plus up to 20 affected resource addresses, and status.lastDriftCheck is when the last successful check completed. DriftJobSucceeded separately reports whether the check (or refresh) Job itself succeeded, independent of what it found. See Conditions and API Reference for every reason and field.

Remediate drift

With action: Remediate set as above, a TerraformCluster or TerraformMachinePool re-applies its current inputs whenever a check finds changes, retried with the same backoff as any other apply (see The reconcile lifecycle) and capped: after failedJobsHistoryLimit (job tuning) failed remediation applies since the last successful check, DriftDetected stays DriftPending until the next check succeeds. A TerraformCluster remediation apply also goes through the destructive-plan guard (or plan approval under applyPolicy: Manual); the guard does not apply to a TerraformMachinePool (see Plan Approval). See Drift and Health for exactly how a remediation runs and what each DriftDetected reason during it means.

A TerraformMachine’s drift is never remediated this way: an unhealthy instance is instead signaled to Cluster API for replacement. See Machine Remediation.

Confirm it worked

After a check completes, status.lastDriftCheck advances and DriftDetected reads False/NoDrift or True with a reason above. After a Remediate apply succeeds, DriftDetected returns to False/NoDrift and status.lastRun.drift is empty again.

See also

• Drift and Health for how checks and health sampling work.
• Machine Remediation for handling an unhealthy TerraformMachine.
• Plan Approval for the destructive-plan guard a TerraformCluster remediation apply goes through.
• Machine Pools for the pool’s separate membership refresh schedule.
• Conditions and Manager Flags.

Machine Remediation

A TerraformMachine reports its instance’s health on the InfrastructureHealthy condition. By itself that only ever turns the owning Machine’s InfrastructureReady condition False; nothing acts on it unless something is watching. spec.remediation optionally asks Cluster API to replace an unhealthy instance, through a MachineHealthCheck and the Machine’s owner. This page covers configuring it, how it reaches a replacement, and what it does to the Machine along the way. See Drift and Health for how health itself is computed.

Before you begin

• A provisioned TerraformMachine whose owning Machine is selected by a MachineHealthCheck with an unhealthyMachineConditions check on InfrastructureReady.
• kubectl access to edit the TerraformMachine and read its owning Machine.

Enable remediation

spec.remediation.annotateMachine defaults to false, meaning CAPTF never touches the Machine. Set it to true to have CAPTF request remediation once an instance has been unhealthy for long enough:

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachine
spec:
  remediation:
    annotateMachine: true

spec.remediation is operational policy: it stays mutable for the machine’s whole life, unlike spec.source and spec.identityRef, which are fixed at creation.

Tune the threshold and sampling interval

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: TerraformMachine
spec:
  remediation:
    annotateMachine: true
    unhealthyThreshold: 5
    healthCheckIntervalSeconds: 120

• unhealthyThreshold (1-100, default 3) is how many consecutive unhealthy health samples are needed before CAPTF signals remediation. A sample is one completed refresh or drift Job; a single transient reading is never enough on its own, since once a MachineHealthCheck acts on the signal, the Machine is replaced. A terminated instance skips the threshold; see Terminated instances below.
• healthCheckIntervalSeconds (60-86400, default 300) is how often a provisioned instance is refreshed to resample its health while annotateMachine is true, independent of spec.drift.intervalSeconds. With annotateMachine set to false it has no effect, and health is then re-read only at the drift (or refresh) cadence set by spec.drift.intervalSeconds; see Drift for that setting, including how intervalSeconds: 0 stops health sampling entirely once annotateMachine is false.

How it works with a MachineHealthCheck

1. Health outside running/healthy, read after provisioning, turns InfrastructureHealthy False, which turns Ready False, which Cluster API mirrors into the owning Machine’s InfrastructureReady condition.
2. A MachineHealthCheck with spec.checks.unhealthyMachineConditions: [{type: InfrastructureReady, status: "False", timeoutSeconds: <N>}] selecting the Machine starts its own timeout once InfrastructureReady turns False.
3. When the timeout (or, with annotateMachine, the annotation below) marks the Machine unhealthy, MachineHealthCheck sets the Machine’s OwnerRemediated condition. Only the Machine’s owner acts on that condition: a MachineDeployment’s MachineSet, or a control-plane provider that implements remediation, such as KubeadmControlPlane or RKE2ControlPlane. A single-replica control plane refuses to remediate itself.
4. The owner that acts replaces the Machine (and, through its deletion, the TerraformMachine) with a new one rather than repairing it in place: spec.source is immutable, so there is nothing for CAPTF to patch.

annotateMachine does not replace this flow; it feeds it a faster, more specific signal than the timeout alone (see the next section).

The remediate-machine annotation

With annotateMachine: true, once unhealthyThreshold consecutive samples are unhealthy, degraded or stopped (or the instance is terminated), CAPTF sets cluster.x-k8s.io/remediate-machine on the owning Machine. This asks the MachineHealthCheck reconciler to treat the Machine as unhealthy at once, bypassing unhealthyMachineConditions’ timeout; remediation still goes through the owner as described above. CAPTF marks the annotation as its own with a second annotation, captf.io/remediation-requested, whose value is why it was set, for example InstanceUnhealthy for 5 consecutive samples or the instance is terminated.

CAPTF removes both annotations once the instance reads Healthy again and the Machine is not being deleted, withdrawing a request that has not yet been acted on. It never removes cluster.x-k8s.io/remediate-machine when captf.io/remediation-requested is absent: an annotation set by someone else is left alone.

See Annotations, Labels and Finalizers for both annotations’ full definitions.

Terminated instances

A terminated instance skips unhealthyThreshold entirely: CAPTF sets the annotations on the very first sample that reads the instance terminated, since there is nothing to wait for and no risk of a transient reading.

Confirm it worked

• kubectl get machine <name> -n <namespace> -o jsonpath='{.metadata.annotations}' shows cluster.x-k8s.io/remediate-machine and captf.io/remediation-requested once a request is made, and neither once the instance recovers or is replaced.
• A RemediationRequested event on the TerraformMachine marks a new request, and RemediationWithdrawn marks a withdrawal; see Events.
• captf_remediation_requests_total{action="requested"|"withdrawn"} counts both; see Metrics.

See also

• Drift and Health for how health samples are produced.
• Drift for spec.drift.intervalSeconds, which paces health sampling when annotateMachine is false.
• API Reference for MachineRemediation’s full field list.
• Conditions for InfrastructureHealthy’s reasons.

Tuning Jobs

Every operation CAPTF runs — apply, destroy, drift, refresh, restore and plan — is one Kubernetes Job. spec.jobs on a TerraformCluster, TerraformMachine or TerraformMachinePool tunes that Job: its resources, deadlines, history, environment, pull secrets, ServiceAccount and security contexts. Every field is optional and has a built-in default, so you only set the fields you need to change. For the full field list, types and validation see JobPolicy in the API reference; for the Job’s exact shape (containers, mounts, args) see Job Environment.

Before you begin

• Know which object’s spec.jobs you are changing. A TerraformCluster also has spec.defaults.jobs, which its TerraformMachines and TerraformMachinePools inherit; see Inheriting from a cluster’s defaults below.
• A TerraformCluster’s own spec.jobs does not inherit spec.defaults.jobs: defaults are for its machines and pools, never for the cluster itself.

Resources

spec.jobs.resources sets the main container’s resources as a whole; when unset the controller applies its own default (250m CPU / 512Mi memory requested, 2Gi memory limit, deliberately no CPU limit — throttling a slow apply is worse than a slow apply). Raise it when a module pulls large provider plugins or holds a large plan in memory; the init container that copies the runner binary is not configurable, since it never varies with the module. See Default resources for the exact values.

Deadlines and lock waits

• activeDeadlineSeconds bounds the whole Job, at most one day. Defaults to 3600 (one hour). Raise it for a module with a slow apply or many resources; a Job that hits its deadline is interrupted with SIGTERM, the same as a pod eviction or deletion. For an apply, destroy or plan Job, ApplyJobSucceeded is set False with reason JobDeadlineExceeded; for a drift or refresh Job, DriftJobSucceeded is set False with reason DriftJobDeadlineExceeded; for a restore Job, RestoreJobSucceeded is set False with reason RestoreFailed. See Conditions.
• lockTimeoutSeconds is passed to the runtime as -lock-timeout. Defaults to 300 (five minutes). Raise it when Jobs commonly queue behind each other on the same state lock.
• When a policy sets both, lockTimeoutSeconds must be less than activeDeadlineSeconds: the webhook rejects a policy where a lock wait alone could fill the whole deadline. This check looks at one policy at a time, so it does not see a machine’s or pool’s field-wise merge with a cluster’s spec.defaults.jobs — setting lockTimeoutSeconds on a machine while only spec.defaults.jobs.activeDeadlineSeconds bounds it is not caught at admission.

History limits

successfulJobsHistoryLimit and failedJobsHistoryLimit cap how many finished Jobs of each kind are kept, both defaulting to 3. Each is scoped per object and per operation: apply, destroy, drift, refresh, restore and plan are pruned independently, so lowering one does not shrink another’s history. The newest Job of an operation is always kept, even at a limit of 0 — for a failed operation, until a newer Job of the same operation succeeds. failedJobsHistoryLimit also bounds retry backoff, since backoff looks at the same retained failures; see The reconcile lifecycle for how backoff is computed.

Environment variables

spec.jobs.env adds environment variables to the main container. Entries named TF_* or KUBE_* are reserved for the runner and the Job’s own environment (TF_IN_AUTOMATION, KUBE_NAMESPACE, and so on): an entry using one of those names is silently dropped instead of applied. See spec.jobs.env rejected names for the full list of names the Job already sets.

Image pull secrets

spec.jobs.imagePullSecrets covers both the role image (spec.source.image) and the runner’s own init image, so one list is enough even when they come from different registries.

ServiceAccount

spec.jobs.serviceAccountName overrides the runner ServiceAccount. Left unset, the controller creates and uses captf-runner, bound to the static captf-runner ClusterRole. An override must already exist and carry the label captf.io/runner=true; without it, no Job is created and RunnerRBACReady reports False with reason ServiceAccountNotOptedIn (see Conditions). Setting up and opting in your own ServiceAccount, and the RBAC CAPTF manages around it, is covered in RBAC.

Security contexts

spec.jobs.securityContext sets the main container’s securityContext. The container holds cloud credentials, so the webhook rejects privileged: true, allowPrivilegeEscalation: true and any capabilities.add on it, whatever else the policy sets. Every other field you set is applied on top of the controller’s defaults (no privilege escalation, every capability dropped, a read-only root filesystem), except capabilities: setting your own replaces the default drop: [ALL] wholesale, so include ALL in your own drop list to keep every capability dropped. runAsNonRoot is left to you, since some role images built from a hashicorp/terraform base still run as root.

spec.jobs.podSecurityContext sets the Job pod’s securityContext. The controller defaults its seccompProfile to RuntimeDefault and its fsGroup to the runner’s UID, so a non-root image user can read the identity credential files (mode 0440) through that group; set your own fsGroup to override it, or your own runAsNonRoot/runAsUser to run the main container as a specific non-root user. For the trust boundary these defaults protect — why the Job is treated like any other pod that holds cloud credentials — see The security model.

Inheriting from a cluster’s defaults

A TerraformMachine’s or TerraformMachinePool’s spec.jobs is merged field by field with its TerraformCluster’s spec.defaults.jobs: a field the machine or pool sets wins, an unset one falls back to the cluster’s default, and a field neither sets gets the built-in default above. Two fields merge instead of falling back as a whole:

• env is merged by name — the machine’s or pool’s entries first, then any of the cluster default’s entries whose name they do not already use.
• imagePullSecrets is the union of both lists, without duplicates, the machine’s or pool’s first.

resources, securityContext and podSecurityContext are each replaced as a whole: setting any one of them on the machine or pool drops the cluster default’s value for that field entirely, rather than merging individual keys inside it. See What a cluster passes to its machines and pools for how this fits the rest of spec.defaults.

Defaults are resolved at reconcile time and never persisted, so raising a cluster’s spec.defaults.jobs reaches every existing machine and pool on their next reconcile.

Confirm it worked

kubectl get job -n <namespace> -l captf.infrastructure.cluster.x-k8s.io/owner-name=<object-name>
kubectl get job -n <namespace> <job-name> -o jsonpath='{.spec.activeDeadlineSeconds}{"\n"}{.spec.template.spec.serviceAccountName}{"\n"}'

<namespace> and <object-name> are the namespace and name of the TerraformCluster, TerraformMachine or TerraformMachinePool you changed; <job-name> is one Job name from the first command’s output. Compare the Job’s spec.template.spec.containers[0].resources, securityContext and env against what you set; a field you expected to change but that still shows the built-in default usually means it was set on the wrong object, or dropped by the merge rules above.

See also

• API Reference for every spec.jobs field, its default and its validation.
• Job Environment for the Job’s fixed fields, mounts, security contexts and runner args.
• RBAC for the runner ServiceAccount, its RoleBinding and the opt-in sweep.
• The security model for the trust boundary a Job’s security context protects.
• The reconcile lifecycle for retry backoff and how a Job’s operation is chosen.

Module Contract

The interface between the CAPTF controller and the Terraform/OpenTofu modules it runs. A module targets exactly one contract version; the image contract is in Image Contract.

v1alpha1

cluster-api-provider-terraform is a Cluster API infrastructure provider that runs Terraform/OpenTofu child modules written to its contract, packaged as OCI images together with the runtime. Existing root modules do not drop in: the provider owns the root module, the state backend and the provider installation.

Read this first. A module written to this contract is a child module. It must not declare a backend: the controller generates the root module, configures the state backend and supplies the provider installation (the image’s mirror or the registry), passes exactly the contract inputs and reads the contract outputs back from state.

Status: Provisional: frozen for implementation, and may still change until the first real module has provisioned a cluster (see CHANGELOG.md). This contract’s text was checked against the OpenTofu v1.11.5 docs; that is a documentation-verification snapshot, not a runtime requirement — the reference images (image-contract.md) ship a newer OpenTofu.

A module is a Terraform/OpenTofu module that implements one role (cluster, machine, machinepool). It is delivered as an OCI image that bundles the module code and the runtime binary (image-contract.md): the image is the artifact, the version and the runnable object; there is no separate module source and no separate runtime image. The controller never calls a module directly: it generates a root module inside the Job that (a) configures the kubernetes backend, (b) calls the module at /captf/module with exactly the contract inputs, and (c) re-exports the contract outputs at the root so they land in state. Everything the controller learns about infrastructure comes from those root outputs.

Files:

Versioning

• The contract version is a string: v1alpha1. It is independent of the CRD API version and of the provider release.
• The image tag is the module version. spec.source.image (registry/repo:tag or @sha256:…) is the only version a CR carries; a new module version is a new image, referenced by a new Terraform*Template. The controller pins the digest after the first run (see image-contract.md “Versioning and pinning”).
• A module does not declare its contract version or role in a way the controller acts on; there is no manifest file. The role is implied by which CRD references the image, and the contract version is the one the controller release generates against (injected as captf_contract, recorded in object status). The OCI labels io.captf.contract and io.captf.role are informational (see image-contract.md). tfcapi-lint takes --role and --contract on the command line and checks the labels against them.
• Compat policy:
  • Within one version: only additive, optional changes (new optional inputs with defaults; new optional outputs). Never rename or retype.
  • Making an input required, removing an output, or changing a type means a new contract version.
  • The controller supports N and N-1 contract versions concurrently; the generated root differs per version.
  • v1alpha* versions carry no compat guarantee between alphas; this policy starts at v1beta1.

Naming rules

• Reserved prefix captf_. Every controller-injected input and every contract-defined local starts with captf_. A module MUST NOT declare a variable or output with this prefix except those defined by the contract.
• Contract outputs are not prefixed (provider_id, addresses, and so on) because they are part of the module’s public interface. Their names are reserved per role; a module MUST declare every required output of its role.
• Module inputs are the contract inputs plus user variables. Anything module-specific (instance type, machine image, region) is either fixed inside the module or a user variable: an ordinary module variable the object sets through spec.variables/spec.variablesFrom (see common.md “User variables”). A module SHOULD give every non-contract variable a default, so it plans when no user variable is set.
• Extra, non-contract outputs are allowed on any role but are never read by the controller. The only cluster-to-machine/pool handoff is the cluster role’s exports output (see cluster.md and captf_cluster_outputs in common.md).
• Tagging is mandatory. Every role receives captf_tags (see common.md) and MUST apply it to the cloud resources it creates; the CAPI provider best practices require a mechanism for identifying a provider’s cloud objects.

Trust boundary

spec.source.image is executed, unpoliced, as a Job under the runner ServiceAccount, which can read every Secret in the object’s namespace and runs with the identity’s cloud credentials mounted: referencing an image is equivalent to granting its publisher that access. See Security Model for the full threat model; this contract only requires that module authors and operators treat an image reference accordingly.

Null semantics

• Required outputs must be declared; their value may be null until known. null means “not yet known”, never “empty”.
• The controller treats null for a required output as “keep waiting” (the object stays unprovisioned, condition OutputsValid=Unknown/OutputsPending), unless the role spec says otherwise (for example control_plane_endpoint on cluster may stay null forever when CAPI supplies the endpoint).
• An absent output (not declared in the module) is a hard error: OutputsValid=False/OutputsMissing, with no retry until the module changes.
• Empty collections are meaningful and distinct from null (addresses = [] means provisioned with no addresses).

Type conventions

• Types are Terraform type constraints. Object attributes listed as optional use optional(type) / optional(type, default) (Terraform ≥1.3 / OpenTofu; both in scope; see the Terraform v1.3.0 changelog and OpenTofu docs: type constraints).
  • output blocks take no type constraint: their arguments are value, description, sensitive, ephemeral, depends_on, deprecated, plus precondition blocks (see OpenTofu docs: outputs). Output “types” in this contract are the shape the controller decodes, and any optional(..., default) on an output is applied by the controller, not by Terraform.
• All outputs are read from state JSON; the controller decodes by the contract’s expected shape, not by the type recorded in state. Extra attributes on objects are ignored; missing required attributes are an error.
• Strings that map to Kubernetes fields must satisfy the target field’s validation: provider_id 1–512 chars (see infra-machine.md “InfraMachine: provider ID”); failure-domain names 1–256 chars, no pattern (see cluster_types.go FailureDomain.Name); address types from the CAPI MachineAddressType enum (see common_types.go). The controller rejects invalid values with OutputsInvalid rather than writing them.
• sensitive = true on a module output marks the value sensitive, and sensitivity propagates to anything derived from a sensitive value (for example bootstrap_data). A root output "x" { value = module.role.x } referencing a sensitive value fails plan with “Output refers to sensitive values” unless the root output itself sets sensitive = true (reproduced with OpenTofu v1.11.5; see OpenTofu docs: outputs “sensitive”). Sensitive values are still stored in state in cleartext (same page). The generated root marks every re-exported output "sensitive": true, so a module may mark any contract output sensitive without failing plan. This has no effect on the controller: the state Secret still stores the value in cleartext, and sensitive only changes CLI display.

Injection mechanics (for spec readers)

• The generated root lives at /captf/work/root inside the Job and calls the module as module "role" { source = "../../module" }, that is, /captf/module inside the image. Terraform/OpenTofu local paths must begin with ./ or ../, so the absolute form /captf/module is not used. It is a local path, so init fetches no module from anywhere (see image-contract.md “Fixed paths”). Providers come from /captf/providers when the image ships a mirror, else from the registry.
• Contract inputs are supplied via terraform.tfvars.json in the generated root (auto-loaded; see OpenTofu docs: variables) and forwarded to the module call by name. Bootstrap data is marked sensitive = true on the root variable.
• Every contract input is a root variable named identically to the module input; the root forwards x = var.x. Nothing else is passed to the module call.
• Contract outputs are re-exported at root as output "x" { value = module.role.x, sensitive = true }: every re-export is marked sensitive, so a module marking any contract output sensitive never fails plan (see Type conventions above).
• The rendered terraform.tfvars.json of the newest apply is kept in a Secret, written when that apply Job starts (a failed apply may already have created resources from those inputs, so destroy must use them), owned by the Terraform* object (captf-inputs-<kindshort>-<name>), together with the resolved image digest (captf.io/image-digest) and the identity. Drift and destroy render from that Secret, never from the live CAPI objects: destroy must work after the bootstrap Secret, the owning Machine/MachinePool, or the Cluster are gone, and drift on an immutable machine must see exactly the inputs it was applied with. The per-run Secret handed to a Job is deleted when the Job completes. The state Secret stores every value (including bootstrap_data, sensitivity notwithstanding) in cleartext for as long as the object exists; modules that can avoid keeping bootstrap data in state (for example by hashing it into a trigger) SHOULD.
• State lives in the kubernetes backend, in the default workspace (the only one CAPTF ever uses), locked with a Lease (see OpenTofu docs: kubernetes backend and Terraform State for Secret naming, locks and backups).
• What triggers a re-apply (mutable roles): a change in the hash of a canonical struct of the spec-derived inputs. Rendered inputs are hashed inputs: object metadata (uid, labels, annotations, generation) and fields the controller itself writes back (spec.providerID, spec.providerIDList, spec.controlPlaneEndpoint) are neither rendered nor hashed, so a clusterctl move, a label edit or the controller’s own status-driven writes never re-apply anything, and a drift Remediate (which renders the current hashed set) cannot apply them either (see common.md “What is hashed”).

Roles and CAPI mapping (summary)

Delivery pipeline for a module author: tfcapi-lint module <dir> --role <role>, then build the image (see image-contract.md for the reference Containerfiles), then push, then tfcapi-lint image <ref> --role <role>, then reference the tag from a Terraform*Template. Lint runs on the source directory before the build; the image check confirms the layout after it.

Conditions: health feeds InfrastructureHealthy, which is an explicit input to the Ready summary. Deleting is also in Ready’s ForConditionTypes for every kind, declared in NegativePolarityConditionTypes{Deleting}, so Ready=False while a destroy runs (the CAPI v1beta2 convention). Drift results (DriftDetected), drift-Job outcomes, Paused and DeletionBlocked are not, so Ready — mirrored by CAPI into the Cluster’s/Machine’s InfrastructureReady — reflects infrastructure only. Ready is Unknown while an object waits on dependencies, False/Provisioning from apply start, and True when provisioned (see machine.md “Ready timeline”).

A destroy that fails permanently is resolved by cleaning up out of band and removing the finalizer by hand, after backing up or un-owning the tfstate-* Secrets (they are garbage-collected with the object). There is no skip-destroy annotation.

Common

Applies to every role. Inputs here are injected by the controller; modules MUST declare them (they may ignore them). Outputs here MUST be declared by every role.

Inputs

variable "captf_contract" { type = string }

variable "captf_cluster" {
  type = object({
    name      = string
    namespace = string
  })
}

variable "captf_object" {
  type = object({
    kind      = string        # TerraformCluster | TerraformMachine | TerraformMachinePool
    name      = string
    namespace = string
  })
}

variable "captf_cluster_outputs" {
  type    = any
  default = null
}

variable "captf_tags" { type = map(string) }

captf_cluster_outputs

The value of the cluster role’s exports output (cluster.md), read from the cluster state. Nothing else from the cluster state is injected.

• Not passed to the cluster role. Its generated root declares no such variable and its tfvars carry no key, so a cluster module either leaves it undeclared or declares it with default = null, as the skeleton below does. A declaration without a default fails validate.
• {} when the TerraformCluster is externally managed (the cluster.x-k8s.io/managed-by annotation; the provider then never runs the cluster module, so there is no cluster state to read).
• A machine/pool whose owning Cluster’s spec.infrastructureRef is not kind TerraformCluster is never rendered at all: it sets DependenciesReady=False/ClusterNotTerraform and does nothing, since there is no cluster state to read exports from.
• This is how a machine/pool module gets network ids, security groups, subnet ids and similar cluster-wide values. The cluster module puts them in its exports output; the controller reads the cluster state and injects exactly that value. exports may be marked sensitive: the generated root re-exports every contract output with "sensitive": true regardless, so a sensitive exports never fails plan, and sensitivity has no effect on the controller — the state Secret still stores the value in cleartext; only CLI display is affected (see README.md “Type conventions”). Machine/pool apply is gated on the cluster being provisioned and exports readable, so the value is complete by then.
• Because captf_cluster_outputs is any, a change in its content is not treated as a spec change for machines (immutable) but is for pools and clusters (a value change triggers a re-apply). See each role’s Lifecycle section.

captf_tags

Fixed keys the controller always sets: captf.io/cluster (Cluster name), captf.io/namespace, captf.io/kind (the Terraform* kind), captf.io/name (the Terraform* object name), captf.io/managed-by = captf, and captf.io/template.

• captf.io/template is the value of the object’s cluster.x-k8s.io/cloned-from-name annotation — the template it was cloned from by external.GenerateTemplate (util.go, common_types.go) — when present, else "". The key is always present: it is read from the Terraform* object’s own metadata, so an object created directly gets "". It is hashed like the other keys even though annotations in general are not: GenerateTemplate writes it once at clone time, but for a TerraformCluster or TerraformMachinePool managed by topology, a ClusterClass rebase onto a differently named template rewrites the annotation through SSA, which re-applies that object once; for immutable machines it is pinned at first apply.
• Modules MUST apply these tags to every cloud resource they create that supports tags or labels, so resources can be attributed and garbage-collected out of band. This follows the CAPI provider best practices: a tagging/labeling mechanism for identifying cloud objects “MUST always be provided” (best-practices.md; security-guidelines.md, housekeeping).

What is hashed

The controller re-applies mutable kinds when the hash of the rendered inputs changes. Rendered inputs are hashed inputs: the hash covers a canonical struct of exactly the spec-derived inputs the module receives, and nothing is rendered outside it (the one special case is pool replicas while autoscaling.enabled, rendered as the module’s own observed desired capacity; see machinepool.md).

captf_cluster and captf_object therefore carry only kind, name and namespace:

• no uid — a clusterctl move re-creates objects with new UIDs, and hashing them would re-apply every cluster and pool after a move;
• no generation — it changes whenever the controller itself writes spec.providerID, providerIDList or controlPlaneEndpoint;
• no annotations — they toggle around every Job (clusterctl.cluster.x-k8s.io/block-move) or are edited by GitOps tools;
• no labels — a metadata edit must never reconcile live infrastructure.

Cloud tags come only from captf_tags, whose keys are fixed. Drift for mutable kinds renders the current hashed set; immutable machines render drift and destroy from the durable inputs Secret below.

Durable inputs

The rendered inputs (including bootstrap_data) are stored in a Secret owned by the Terraform* object (captf-inputs-<kindshort>-<name>), so drift and destroy of immutable machines re-feed exactly the values used at apply, and the Secret moves with the object. The resolved image digest (captf.io/image-digest, see image-contract.md “Versioning and pinning”) and the identity are pinned there too, so an immutable machine’s drift and destroy always run the exact image that created it.

User variables

Everything a module is parameterized by beyond the contract inputs (instance type, disk size, subnets, a database password) is an ordinary Terraform variable of the module. The object’s owner sets it through spec.variables (inline JSON) or spec.variablesFrom (ConfigMaps and Secrets labeled captf.io/variables=true) on the TerraformCluster, TerraformMachine or their templates; see Module Variables for the operator side. For module authors:

• Declare each one as an ordinary variable, with a type and a default. The generated root passes a user variable as a named argument of module "role", declared in the root without a type, so your declaration’s type converts the value: a number variable accepts 40 inline and "40" from a ConfigMap. A variable nobody sets gets its default; without one, every object that does not set it fails plan on the missing argument. tfcapi-lint reports such a variable as input/user-variable-default (a warning: a module may deliberately require it).
• Mark secrets sensitive = true in the module. The root declares a variable sensitive only when its value came from a Secret; your declaration makes the value sensitive inside the module whatever its source. A value that arrives sensitive stays sensitive in everything derived from it: an output built from it must be declared sensitive = true (or plan fails with “Output refers to sensitive values”), and it cannot drive for_each. Either way the value is stored in cleartext in the inputs Secrets and in the state, like every input (see Module Variables); credentials the module runs with come from the identity, never from a variable.
• Names. User variables are Terraform identifiers (^[a-zA-Z_][a-zA-Z0-9_-]*$). Reserved, and rejected before anything is rendered: every captf_ name, every contract input of the module’s role (machine_name on a machine, control_plane_initialized on a cluster, and so on) and the module meta-arguments source, version, providers, count, for_each, depends_on, lifecycle, locals. Do not rely on a user variable to carry a contract input’s value.
• Unknown names fail the apply. A variable the object sets and the module does not declare is rejected by Terraform/OpenTofu (Unsupported argument; for the JSON root, Extraneous JSON object property). The controller does not check names against the image.
• Hashing. User variables are part of the inputs hash only when the object sets some, so modules and objects without them hash exactly as before. For a TerraformCluster a changed variable re-applies (guarded by the destructive-plan approval, see cluster.md); a TerraformMachine reads its variables until it is provisioned, and its later runs use the ones pinned in its durable inputs Secret, like every other input.
• The role input schemas (schemas/*-inputs.json) describe the contract inputs only; the rendered terraform.tfvars.json carries the user variables next to them.

Outputs

output "health" {
  value = {
    state   = "running"   # closed enum: pending | running | degraded | stopped | terminated | unknown
    healthy = true
    message = null        # optional string, surfaced in condition message
    reasons = []          # optional list(string), machine-readable
  }
}

state is a closed enum. Any other string is OutputsValid=False/OutputsInvalid.

Controller-side semantics. health is written to the InfrastructureHealthy condition. InfrastructureHealthy is one of the explicitly listed inputs to the Ready summary; so is Deleting (negative polarity, so Ready=False while a destroy runs). Drift results, drift-Job outcomes, Paused and DeletionBlocked are not, so Ready — and therefore the Cluster/Machine InfrastructureReady mirror that MachineHealthCheck acts on — reflects infrastructure health, not controller housekeeping.

Once provisioned, the Machine controller treats an empty InfraMachine spec.providerID as “waiting” and logs it; it is not a hard error, but clearing it would stall the Machine (machine_controller_phases.go reconcileInfrastructure), so a terminated reading keeps providerID rather than clearing it.

• health is re-read on every refresh (apply -refresh-only, which updates state and root output values to match remote objects; see OpenTofu docs: cli/commands/plan/ “Planning Modes”): the pending refresh loop above, every drift tick, and for pools the membership refresh (see machinepool.md). For immutable machines that is the only way it changes after apply. Modules that cannot observe health return {state="running", healthy=true} (the no-op module does this) or unknown.
• Out-of-band termination. If, after provisioning, a refresh makes provider_id come back null (the resource vanished from state) or the refresh itself fails because the instance is gone, the controller treats that as terminated: InfrastructureHealthy=False/InstanceTerminated, spec.providerID kept. Modules SHOULD still report terminated explicitly where the cloud API exposes it, because detection through a vanished resource is slower (drift interval plus a Job) and CAPI’s Node-based checks will usually fire first.
• A group with zero members (pool role, replicas == 0) reports {state="running", healthy=true}.
• Only the machine role triggers remediation. Cluster/pool health feeds conditions only.

Role-specific inputs

kubernetes_version is a role input, not a common one: for machines/pools it comes from Machine.spec.version / MachinePool.spec.template.spec.version (authoritative for nodes); for the cluster role it comes from Cluster.spec.topology.version (null without ClusterClass; see cluster.md). cluster_network, control_plane_initialized and the non-module control_plane_endpoint (including one set later by a control-plane provider) are cluster-role inputs (see cluster.md); machine/pool modules get endpoint-derived values through exports.

Cluster Role

Implements the CAPI InfraCluster contract (infra-cluster.md) for a TerraformCluster. One workspace/state per TerraformCluster. Typically owns the cluster-wide substrate: network, subnets, security groups, load balancer / control-plane endpoint, shared images/keys. Values that machine/pool modules need are handed over through the exports output, injected into them as captf_cluster_outputs.

Inputs

Common inputs apply (common.md), except captf_cluster_outputs: it is not passed to this role. The skeleton below declares it with default = null, which is safe; a declaration without a default fails validate.

control_plane_endpoint (input)

Value = Cluster.spec.controlPlaneEndpoint whenever the annotation captf.io/endpoint-source is not module — whether the user set it before the first apply or a control-plane provider (hosted control planes) sets it later (cluster_controller_phases.go). While the Cluster field is not yet valid (APIEndpoint.IsValid, host and port both set), the input falls back to a valid TerraformCluster.spec.controlPlaneEndpoint (the two are equal once CAPI copies ours, cluster_controller_phases.go, so the copy never changes the hash); else null. It is always null when endpoint-source=module.

Non-null means “use this endpoint, don’t create one”: the contract allows the endpoint to come from the user, the control plane provider, or the infra provider (infra-cluster.md “InfraCluster: control plane endpoint”). Shape = APIEndpoint{host 1–512 chars, port 1–65535} (cluster_types.go).

Provenance (captf.io/endpoint-source): user is written before the first apply when either field holds a valid endpoint; module is written when the controller first copies a non-null module output (from an apply whose rendered input was null) into TerraformCluster.spec.controlPlaneEndpoint; otherwise the annotation is absent. Once written it never changes. With module the input stays null for the life of the object even after CAPI copies the module’s endpoint to Cluster.spec: feeding it back would tell the module to drop the load balancer it created, and would change the inputs hash. In the inputs hash: an endpoint set later by a control-plane provider therefore re-applies the cluster once, which is how the module learns it.

kubernetes_version (input)

Cluster.spec.topology.version (cluster_types.go); null when the Cluster has no topology (no ClusterClass). For modules that provision managed/hosted control planes or version-dependent cluster resources. Passed verbatim (it may carry a distro suffix such as +rke2r1; strip it as the machine role does). In the inputs hash, so a topology upgrade re-applies the cluster.

control_plane_initialized (input)

Cluster.status.initialization.controlPlaneInitialized (cluster_types.go; nil maps to false). In the inputs hash, so the cluster re-applies exactly once when the control plane comes up (CAPI latches the field).

The controller latches it too: the rendered value is true when the Cluster field is true or the last rendered value in the durable inputs Secret (captf-inputs-c-<name>, which moves) is true; it is never rendered false after an apply with true. Cluster status is not moved by clusterctl move, so without this a first reconcile on the target could render false and destroy every gated resource.

Modules that need a live workload API server (in-cluster add-ons, cloud-controller secrets, DNS records pointing at registered nodes) gate those resources on it, for example count = var.control_plane_initialized ? 1 : 0.

cluster_network (input)

Cluster.spec.clusterNetwork (pods.cidrBlocks, services.cidrBlocks, serviceDomain, apiServerPort; cluster_types.go ClusterNetwork, NetworkRanges); null when the Cluster sets none of them; missing attributes are null or [].

api_server_port is not the kube-apiserver bind port with KCP — CABPK/KCP never read Cluster.spec.clusterNetwork.apiServerPort; the real bind port is KubeadmConfig localAPIEndpoint.bindPort (default 6443, workload_cluster.go). By convention the module’s LB backend port is cluster_network.api_server_port ?? 6443, and templates must keep apiServerPort and bindPort equal — the controller has no way to enforce this.

Outputs

Common outputs apply (health).

control_plane_endpoint (output)

Maps to TerraformCluster.spec.controlPlaneEndpoint → Cluster.spec.controlPlaneEndpoint, surfaced once InfraCluster initialization completes (infra-cluster.md). It is written only when all of the following hold: the output is non-null; the field is still empty; and the apply’s rendered control_plane_endpoint input was null (the first such write records captf.io/endpoint-source=module).

It is written only when the output is valid: CAPI’s copy-to-Cluster.spec path only fires once APIEndpoint.IsValid() is true, which requires both host and port to be set (cluster_types.go), so a half-set output (only one of the two present) is treated as OutputsValid=False/OutputsInvalid, not partially written. CAPI copies the InfraCluster endpoint only while Cluster.spec.controlPlaneEndpoint is not yet valid, so later changes are never propagated (cluster_controller_phases.go reconcileInfrastructure); CAPI’s Cluster admission webhook has no immutability rule for it (cluster.go).

Once emitted, the value MUST be stable for the life of the object. CAPI never updates Cluster.spec.controlPlaneEndpoint after the first valid copy (cluster_controller_phases.go), so a module that later replaces its load balancer (a new DNS name or IP) breaks every existing kubeconfig and the apiserver certificate SANs, with no way for CAPI to notice or recover. Keeping it stable is the module author’s responsibility first: the controller blocks, but cannot fix, a planned replacement.

Every TerraformCluster apply (a re-apply, a new image tag, a drift remediation) stops before a plan that deletes or replaces any resource, sets ApplyJobSucceeded=False/DestructivePlanBlocked with the affected addresses, and applies it only once the captf.io/approve-destructive-plan annotation names that apply’s inputs hash; see Plan Approval for the destructive-plan guard. With spec.applyPolicy: Manual every re-apply instead waits for the approval of its plan, which covers its deletes; see Plan Approval for plan preview. An operator can still approve a plan that replaces the endpoint’s resource; lifecycle { prevent_destroy = true } on the resource behind the endpoint turns such a plan into a failed Job even when approved, instead of a lost cluster.

When no user endpoint exists, the module MUST emit this output no later than the apply that makes the cluster provisioned (see EndpointAvailable below).

failure_domains (output)

Maps to TerraformCluster.status.failureDomains (v1beta2 list shape: listType=map, listMapKey=name, MaxItems=100; FailureDomain{name 1–256 chars, controlPlane *bool, attributes map[string]string}; infra-cluster.md “InfraCluster: failure domains”, cluster_types.go). CAPI’s controlPlane has no default; optional(bool, true) is our default, applied by the controller. null and [] both clear the field.

exports (output)

Injected verbatim into machine/pool modules as captf_cluster_outputs. May be marked sensitive: the generated root re-exports it "sensitive": true regardless, so this has no effect on the controller (see README.md “Type conventions”). Put secrets in the identity, not here. Convention: publish the load balancer target/backend-pool id(s) here for control-plane machine modules to register against (see machine.md “Control-plane machines”).

Provisioned rule

status.initialization.provisioned is the v1beta2 contract field: status.initialization.provisioned = true when the state Secret carries the inputs-hash annotation of a successful apply, the required outputs are valid, and health.state != "pending".

Latched. The formula applies only until it first holds; once provisioned is true it stays true for the object’s life, whatever later health or outputs say (CAPI never flips infrastructureProvisioned back; machine_controller_phases.go “should not flip back”).

There is no endpoint gate on provisioned: a null control_plane_endpoint output with no endpoint on the Cluster is allowed, because hosted-control-plane providers set the endpoint only after infrastructure is provisioned. Provisioned is derived from state, never from Job history, so it is rebuilt identically after clusterctl move (status is not moved; the target evaluates the formula once, then latches).

Hairpin reachability

The control-plane endpoint MUST be reachable from the first control-plane node itself, not only from the management cluster and other nodes. kubeadm’s getKubeConfigSpecsBase (kubernetes/kubernetes release-1.33, cmd/kubeadm/app/phases/kubeconfig/kubeconfig.go around lines 582-626) sets the server URL of admin.conf, super-admin.conf and kubelet.conf to GetControlPlaneEndpoint(cfg.ControlPlaneEndpoint, &cfg.LocalAPIEndpoint) — that is, the endpoint, not the node’s local address — while only controller-manager.conf/scheduler.conf use GetLocalAPIEndpoint. Because kubeadm’s post-init phases and the first node’s own kubelet talk to the workload cluster through admin.conf/kubelet.conf, the load balancer or VIP that backs control_plane_endpoint must hairpin: the control-plane instance that is itself a backend must be able to reach the frontend it was just registered behind.

EndpointAvailable condition

Cluster only, not part of Ready: False/WaitingForEndpoint once provisioned is true and neither the module’s control_plane_endpoint output nor Cluster.spec.controlPlaneEndpoint is a valid APIEndpoint (host and port both set); True/EndpointAvailable otherwise.

With KCP this case is not cosmetic: KCP creates no Machine at all until Cluster.spec.controlPlaneEndpoint.IsValid() (kubeadmcontrolplane_controller.go — “Waiting for Cluster spec.controlPlaneEndpoint to be set”), so a module that outputs null with no user-provided endpoint silently stalls the cluster forever with no other signal. tfcapi-lint warns (output/endpoint-never-set) when a module’s control_plane_endpoint output is a literal null with no conditional path to a value.

Lifecycle

• Mutable. Any change to spec.source.image (a pull-policy change alone only affects how the next Job pulls, and starts none), a non-module control_plane_endpoint (including one set later by a control-plane provider), cluster_network, kubernetes_version (Cluster.spec.topology.version), or control_plane_initialized flipping true starts a new apply Job (detected by the hash of the rendered spec-derived inputs; labels, uid, and controller-written fields are neither rendered nor hashed, see common.md “What is hashed”).
• Externally managed. A TerraformCluster carrying cluster.x-k8s.io/managed-by is skipped entirely: no Jobs, no status writes (infra-cluster.md “Externally managed infrastructure”). Machines and pools of that Cluster receive captf_cluster_outputs = {}.
• Drift. On spec.drift.intervalSeconds: apply -refresh-only first (so outputs, health and exports refresh and the plan compares against current remote values), then plan -detailed-exitcode (0 = no changes, 1 = error, 2 = changes; see OpenTofu docs: cli/commands/plan/). Exit 2 sets DriftDetected=True; with action: Report (the default) that is all, and an operator decides; with action: Remediate the controller applies, re-rendering the current inputs. A changed exports value re-applies pools (mutable) but not machines (immutable). Drift results and drift-Job outcomes never feed Ready, and after first provisioning neither does a failed re-apply: it is ApplyJobSucceeded=False plus DriftDetected, never Ready=False, because Cluster InfrastructureReady=False would suspend every MachineHealthCheck of the cluster (machinehealthcheck_targets.go).
• Delete ordering (enforced by the controller). Destroy is blocked with DeletionBlocked=True/DependentsExist (requeue) while any TerraformMachine or TerraformMachinePool labeled cluster.x-k8s.io/cluster-name=<cluster> (ClusterNameLabel, common_types.go) exists in the namespace, since their modules depend on cluster resources via exports. CAPI’s own Cluster deletion already removes Machines/MachinePools first — it requeues while descendants, including MachinePools, exist, then deletes the control plane object, then the InfraCluster (cluster_controller.go reconcileDelete, clusterDescendants) — so this guard only covers out-of-band deletions of the TerraformCluster. Then a destroy Job runs; success drops the state Secret(s) and Lease, and removes the finalizer. Module authors may rely on this ordering.
• The module MUST be idempotent under repeated apply with unchanged inputs (plan must be empty). tfcapi-lint can’t check this, and no automated test does yet; the module author must.

Control-plane provider requirements

What a control-plane provider needs from the cluster module, and how KubeadmControlPlane (KCP) and RKE2ControlPlane (RCP) differ. The KCP column draws on the KubeadmControlPlane integration guide; the RCP column draws on the RKE2ControlPlane integration guide.

See the KubeadmControlPlane and RKE2ControlPlane guides for the reasoning and exact citations behind each row.

Minimal skeleton

A block written on one line may hold at most one argument (OneLineBlock; tofu validate reports “Invalid single-argument block definition” on a violation), so a variable that needs both type and default (like captf_cluster_outputs below) must use the multi-line block form:

variable "captf_contract"        { type = string }
variable "captf_cluster"         { type = object({ name = string, namespace = string }) }
variable "captf_object"          { type = object({ kind = string, name = string, namespace = string }) }
variable "captf_cluster_outputs" {
  type    = any
  default = null
}
variable "captf_tags" { type = map(string) }

variable "control_plane_endpoint" {
  type    = object({ host = string, port = number })
  default = null
}

variable "kubernetes_version" {
  type    = string
  default = null
}
variable "control_plane_initialized" { type = bool }

variable "cluster_network" {
  # Every attribute is always present when cluster_network is non-null
  # (an unset CIDR list renders as [], an unset scalar as null), so the
  # type matches the Inputs table exactly: none of the four is optional().
  type = object({
    pods            = list(string)
    services        = list(string)
    service_domain  = string
    api_server_port = number
  })
  default = null
}

# Tag every cloud resource this module creates with captf_tags (common.md
# "captf_tags"); this stub only has to reference it, not create anything.
resource "terraform_data" "tags" { input = var.captf_tags }

output "control_plane_endpoint" { value = var.control_plane_endpoint != null ? var.control_plane_endpoint : { host = "...", port = 6443 } }
output "failure_domains"        { value = [] }
output "health"                 { value = { state = "running", healthy = true, message = null, reasons = [] } }

# Handed to machine/pool modules as captf_cluster_outputs:
output "exports" { value = { network_id = "...", subnet_ids = ["..."] } }

Machine Role

Implements the CAPI InfraMachine contract (infra-machine.md) for a TerraformMachine. One workspace/state per TerraformMachine. The module creates exactly one node instance and reports its provider ID, addresses, and health. The machine is immutable (spec.source and spec.identityRef cannot change): apply once, refresh/drift for health, destroy on delete.

Inputs

Common inputs apply (common.md), including captf_cluster_outputs.

bootstrap_data

Base64 (standard alphabet, padded) of the raw bytes of the value key of the Secret named by Machine.spec.bootstrap.dataSecretName. The bootstrap contract requires a single key value (bootstrap-config.md “BootstrapConfig: data secret”; dataSecretName in machine_types.go Bootstrap).

The controller base64-encodes value when rendering tfvars, always and for every bootstrap provider: CAPRKE2 with gzipUserData: true writes value as raw gzip bytes, which are not valid UTF-8 and so cannot be a JSON/HCL string (see the RKE2ControlPlane guide); encoding unconditionally keeps one rule instead of sniffing content. The input stays sensitive = true.

Modules use one of two patterns:

• pass it unchanged to a user_data_base64-style argument (for example AWS aws_instance.user_data_base64 or aws_launch_template.user_data, which both take base64); or
• base64decode(var.bootstrap_data) where the argument wants the plain payload — only valid when the payload is UTF-8, that is not gzipped: base64decode fails on non-UTF-8 bytes, so a binary payload must go to a base64-taking argument instead.

Apply waits until the Secret exists (the contract workflow also exits while dataSecretName is nil; infra-machine.md “Typical InfraMachine reconciliation workflow”); dataSecretName: "" (the field allows MinLength=0, machine_types.go) is treated exactly like nil (WaitingForBootstrapData).

For control-plane Machines the payload embeds the cluster CA and service account private keys uncompressed (controlplane_init.go, controlplane_join.go), so it is both larger and more sensitive than a worker’s. Size limits (for example AWS’s 16 KiB user-data) and keeping keys out of readable instance metadata are module business: gzip the payload, or stage it in a secret store and pass only a small stub through bootstrap_data/user-data.

bootstrap_format

cloud-config or ignition, from the bootstrap Secret’s format key; cloud-config when absent.

format is not part of the bootstrap contract, which specifies only value (bootstrap-config.md). It is written by the kubeadm bootstrap provider (kubeadmconfig_controller.go writes value and format; enum cloud-config;ignition in kubeadmconfig_types.go Format). Other bootstrap providers need not write it. CAPRKE2 also always writes both keys, with the same enum (rke2config_controller.go; see the RKE2ControlPlane guide). format describes the decoded payload; with CAPRKE2 gzipUserData: true the decoded bytes are gzip of that format.

failure_domain (input)

Machine.spec.failureDomain (1–256 chars; machine_types.go). The InfraMachine MUST be placed in this failure domain (infra-machine.md “InfraMachine: failure domain”).

kubernetes_version (input)

Machine.spec.version (optional, 1–256 chars; machine_types.go). The value may carry a control-plane-provider-specific distro suffix: with RKE2 it is vX.Y.Z+rke2rN (for example v1.31.4+rke2r1), copied verbatim from RKE2ControlPlane.spec.version to Machine.spec.version (see the RKE2ControlPlane guide). The controller passes this input to the module verbatim, unstripped; a module that uses it for an image lookup or a semver comparison MUST strip the +… build-metadata suffix itself (see the RKE2ControlPlane guide).

control_plane (input)

true when the owning Machine carries the cluster.x-k8s.io/control-plane label (MachineControlPlaneLabel, machine_types.go; CAPI’s own util.IsControlPlaneMachine checks only for this label’s presence, util.go). There is no role string in MachineSpec.

Outputs

Common outputs apply (health).

provider_id (output)

Maps to TerraformMachine.spec.providerID → Machine.spec.providerID. Must equal the Node’s spec.providerID exactly (see “Node providerID matching” below). 1–512 chars (infra-machine.md “InfraMachine: provider ID”; MinLength=1, MaxLength=512 markers in machine_types.go). "" is treated as null.

Written once; a later non-null change is OutputsValid=False/ProviderIDChanged. If it turns null after provisioning, the instance is treated as terminated (see common.md “Out-of-band termination”) and spec.providerID is kept. We latch it — never clear it once set — because CAPI does not: the Machine controller recopies Machine.spec.providerID from TerraformMachine.spec.providerID on every reconcile rather than latching it once (machine_controller_phases.go), and clearing our side would blank the Machine’s field and stop address/failure-domain copying from the InfraMachine on that same path.

addresses (output)

Maps to TerraformMachine.status.addresses → Machine.status.addresses (infra-machine.md “InfraMachine: addresses”). type is one of Hostname, ExternalIP, InternalIP, ExternalDNS, InternalDNS (MachineAddressType enum, common_types.go); address is 1–256 chars; at most 256 entries (MachineAddress and Machine.status.addresses markers, common_types.go and machine_types.go). The controller validates every mapped output against these CRD markers before patching status; a violation is OutputsValid=False/OutputsInvalid, and nothing is written.

Canonical order. Before writing status the controller sorts the module’s list by type precedence InternalIP, InternalDNS, ExternalIP, ExternalDNS, Hostname, then by address. CAPI itself copies the list verbatim (machine_controller_phases.go), so an unsorted module output would otherwise reach Machine.status.addresses in whatever order the module emitted it. Rationale: RKE2’s registrationMethod: internal-first takes the first address in list order whose type is InternalIP or ExternalIP (this is what the code does, not “internal preferred” as its own docs claim); internal-only-ips needs an InternalIP present, and external-only-ips needs an ExternalIP present (see the RKE2ControlPlane guide) — a fixed, sorted order makes that first-match deterministic instead of depending on module authoring order.

failure_domain (output)

Maps to TerraformMachine.status.failureDomain → Machine.status.failureDomain (the actual placement; infra-machine.md “InfraMachine: failure domain”). When the failure_domain input is non-null the output MUST equal it: the contract says the InfraMachine MUST be placed in the requested domain, and the controller enforces it — a mismatch is OutputsValid=False/FailureDomainMismatch and Ready=False, so KCP/MachineDeployment spreading can never be silently broken. The output adds information only when no domain was requested.

interruptible (output)

Maps to TerraformMachine.status.interruptible (bool); true for spot/preemptible instances. CAPI reads status.interruptible from the InfraMachine and, when true, sets the Node label cluster.x-k8s.io/interruptible (InterruptibleLabel; machine_controller_noderef.go), which termination handlers and schedulers key on. The output must be declared because the generated root re-exports every contract output unconditionally; a module without spot support declares output "interruptible" { value = false }. Null is written as false.

Provisioned rule

status.initialization.provisioned = true when the state Secret carries the inputs-hash annotation of a successful apply, provider_id is non-null, and health.state != "pending"; derived from state only, never from Job history, so it is rebuilt after clusterctl move (evaluated once on the target, then latched).

Latched. The formula applies only until it first holds; once provisioned is true it stays true for the object’s life, whatever later health or outputs say (CAPI never flips infrastructureProvisioned back; machine_controller_phases.go “should not flip back”). A later null provider_id or non-running health is reported on InfrastructureHealthy (for example InstanceTerminated), never by un-provisioning.

While pending, the controller refreshes regardless of drift.intervalSeconds, 30s after the last refresh and backing off to at most 5m (not a field; see common.md). After a successful apply it refreshes once more unless the apply’s own outputs are valid with a health.state other than pending or unknown. Ready=True additionally requires health.healthy and state == "running".

Ready timeline

What CAPI mirrors into Machine.status.conditions[InfrastructureReady], which MachineHealthCheck can act on:

Unknown rather than False while waiting matters for MHC: unhealthyMachineConditions timeouts count from the condition’s lastTransitionTime, and a reason change does not reset it. Workers created alongside a slow control plane can wait a long time for bootstrap data; starting the False window only at apply start means an MHC InfrastructureReady=False timeout measures apply time, not control-plane bring-up. After provisioning, Ready is computed from InfrastructureHealthy and deletion only; drift results, drift-Job failures, identity or RBAC problems never turn it False (they are separate conditions), so a credentials outage cannot trigger fleet-wide remediation.

Lifecycle

• Immutable. Changes to spec.source and spec.identityRef after creation are rejected by the webhook, and spec.providerID can only be set once, by the controller. Those changes come via MachineDeployment/KCP rollout. spec.jobs, spec.drift and spec.remediation are operational policy and stay mutable: they change how the controller treats the machine (a Job deadline, drift checks, auto-remediation), never what was applied. captf_cluster_outputs changes do not trigger re-apply: the rendered inputs of the first apply (including bootstrap_data and captf_cluster_outputs) are stored in the object-owned captf-inputs-<kindshort>-<name> Secret and re-fed unchanged on drift and destroy; the effective image and identity are pinned there too, so a later change to TerraformCluster.spec.defaults (for example a ClusterClass in-place update) never changes what runs against an existing machine. Bootstrap data is read once. The kubeadm bootstrap provider does not rewrite a Machine’s bootstrap Secret after creation — it only extends the join token’s TTL in the workload cluster until the Node joins (kubeadmconfig_controller.go refreshBootstrapTokenIfNeeded) — so “read once” loses nothing. Pools are different (see machinepool.md).

• Gating. Apply waits for: the owner Machine (looked up through the Machine ownerRef, util.GetOwnerMachine, then the Cluster through cluster.x-k8s.io/cluster-name; a TerraformMachine never has a Cluster ownerRef); Cluster.status.initialization.infrastructureProvisioned (cluster_types.go); the cluster’s exports readable (WaitingForClusterExports); and the bootstrap Secret (WaitingForBootstrapData; the contract workflow also exits while dataSecretName is nil, and "" counts as nil). The owning Cluster’s spec.infrastructureRef must be kind TerraformCluster; otherwise DependenciesReady=False/ClusterNotTerraform and nothing is done. Once those pass, the spec.variablesFrom sources must exist and be labeled (DependenciesReady=False/VariablesSourceNotFound) with valid keys and values (False/VariablesInvalid); see common.md “User variables”. While gated, Ready=Unknown (timeline above).

• Retry. A state Secret that exists without the inputs-hash annotation (a failed or interrupted first apply) means the apply is retried with backoff even though the spec is immutable; each attempt gets a distinct Job name (attempt counter).

• Refresh/drift. On spec.drift.intervalSeconds: apply -refresh-only then plan -detailed-exitcode. Drift is reported only (DriftDetected=True, a negative-polarity condition kept out of Ready); never auto-applied for machines (MachineDriftPolicy has no action field: there is nothing to remediate against for an immutable machine). The refresh updates addresses and health. A failed drift Job sets DriftJobSucceeded=False and nothing else.

• Health → remediation. health outside running/healthy after provisioning sets InfrastructureHealthy=False, then Ready=False/NotReady, mirrored to Machine.status.conditions[InfrastructureReady]=False, which a MachineHealthCheck can act on. See Machine Remediation for configuring spec.remediation, the unhealthy threshold and sampling interval, and how a MachineHealthCheck reaches a replacement.

  health is purely cloud-side: the controller does not cross-check Machine.status.nodeRef/nodeInfo (Node existence, machine_controller_noderef.go); a missing or NotReady Node is MHC’s nodeStartupTimeout/Node-condition checks’ job.

• Delete. Deletion of a TerraformMachine is normally initiated by the Machine controller after drain and pre-terminate hooks. Ordering is CAPI’s: drain (bounded by Machine.spec.deletion.nodeDrainTimeoutSeconds) and volume detach (nodeVolumeDetachTimeoutSeconds) finish before the TerraformMachine is deleted, so destroy never races a running drain; the Node object is deleted by core-machine only after the TerraformMachine is gone (bounded by nodeDeletionTimeoutSeconds; machine_types.go, machine_controller.go). No contract input carries these timeouts.

  The webhook allows a direct delete when the object has no Machine ownerRef at all (an orphan: KCP deletes the InfraMachine itself when KubeadmConfig or Machine creation failed, helpers.go), when the owning Machine is itself deleting, or when the request carries clusterctl.cluster.x-k8s.io/delete-for-move; it refuses deletion otherwise, so drain and the KCP etcd-membership hook are never bypassed by deleting the infra object directly.

  Then a destroy Job runs, rendered from the captf-inputs-<kindshort>-<name> Secret (never from the bootstrap Secret or the Cluster, which may already be gone); on success it drops the state, the Lease and the inputs Secret, and removes the finalizer. If the owning Machine or Cluster is gone the same path runs. If there is no state Secret and no active Job (deleted while still gated), the finalizer is removed without a Job. If destroy fails, the object stays with ApplyJobSucceeded=False/DestroyFailed (Ready=False) and retries with backoff; a Lease whose holder Job no longer exists is force-unlocked automatically. There is no skip-destroy annotation: a permanently failing destroy is resolved by cleaning up out of band and removing the finalizer by hand, after backing up or un-owning the tfstate-* Secrets, which are otherwise garbage-collected with the object (see the stuck-destroy runbook). A stuck control-plane machine destroy blocks KCP remediation, scale and upgrade until resolved (remediation.go).

• The module MUST accept bootstrap_data opaquely: it is base64 of the bootstrap payload (Inputs), passed to a base64-taking user-data argument or base64decode()d into a plain one; the module MUST NOT need to parse the payload.

• Taints and labels. Machine.spec.taints and Machine labels are applied to the Node by core-machine (machine_types.go; machine_controller_noderef.go), not by the infrastructure; the machine role has no node_labels input (pools do, see machinepool.md); no role has a taints input.

• In-place updates (the CAPI InPlaceUpdates feature gate, alpha) are unsupported: no field is mutable, and the webhook rejects the MachineSet’s SSA patch; the rollout path is a new Machine.

Node providerID matching (module authors)

CAPI links a Machine to its Node only when Node.spec.providerID equals Machine.spec.providerID exactly (machine_controller_noderef.go). Our provider_id output is copied to the Machine verbatim, so the module MUST emit the same string the Node will carry:

• With a cloud-controller-manager (CCM): the CCM sets Node.spec.providerID in its own format (for example aws:///<zone>/<instance-id>, azure:///subscriptions/..., openstack:///<uuid>, vsphere://<uuid>). Emit exactly that format.
• Without a CCM: the kubelet must be started with --provider-id=<value> (via KubeadmConfig nodeRegistration.kubeletExtraArgs), and the value must be derivable on the instance at boot (an instance-metadata lookup, or a per-machine value baked into the module’s user-data wrapper), because bootstrap_data is shared per template and opaque to the module.
• Not supported in v1: a controller-side patch of Node.spec.providerID once the Node registers, as CAPD does (machine.go), would need a workload-cluster client and a manager flag; the controller does not implement it. In v1 the CCM or kubelet --provider-id must set it.
• RKE2: CAPRKE2 itself never sets kubelet provider-id, node-ip or node-name (the fields exist in its config but nothing in the repo writes them; see the RKE2ControlPlane guide). Without a CCM, the image MUST set agentConfig.kubelet.extraArgs: [provider-id=<same format>] from instance metadata so the kubelet-registered value matches our provider_id output exactly. This matters more under RKE2 than KCP: every RKE2 join (control plane or worker) requires RCP.status.availableServerIPs to be non-empty, which requires at least one Ready control-plane Machine, and a Machine only becomes Ready once its Node exists with a matching providerID — so a providerID mismatch on the first control-plane Machine blocks every subsequent join, not just that one Machine’s own readiness.

Autoscale-from-zero (InfraMachineTemplate.status.capacity / nodeInfo; infra-machine.md “InfraMachineTemplate: support cluster autoscaling from zero”) is supported through the image: instance size is fixed inside the module, so the image declares it with the OCI labels io.captf.capacity and io.captf.node-info (see image-contract.md “OCI labels”), and a TerraformMachineTemplate reconciler copies them into status.capacity/status.nodeInfo. An image without the labels leaves both fields unset; the Cluster Autoscaler’s capacity.cluster-autoscaler.kubernetes.io/* annotations on the MachineDeployment/MachineSet remain the fallback.

Control-plane machines

When control_plane = true, the module is responsible for registering the instance in the control-plane load balancer’s backend or target group. The ids it needs (load balancer, target group, backend pool — whatever the cluster module’s cloud exposes) come from captf_cluster_outputs (see cluster.md exports, which describes the publishing side of this same convention). Registration is done in the machine module’s own Terraform state, not the cluster module’s, so destroy on the machine deregisters it as an ordinary part of tearing down that state (infra-machine.md).

Ordering matters: the instance MUST be in the load balancer backend before kubeadm init/kubeadm join finish on it, because the contract requires the endpoint to be reachable through the load balancer during control-plane bring-up (status.go; see the KubeadmControlPlane guide on reachability) — Cluster.status.initialization.controlPlaneInitialized never latches if the first control-plane node can’t be reached at the endpoint it just joined. In practice this means the module’s apply must register-then-boot (or register-then-poll-healthy) rather than boot-then-register as an afterthought.

For worker Machines (control_plane = false) no load balancer registration applies; captf_cluster_outputs is still read for any other cluster-level values the module needs.

Minimal skeleton

A block written on one line may hold at most one argument (OneLineBlock; tofu validate reports “Invalid single-argument block definition” on a violation), so a variable that needs both type and sensitive, or type and default, must use the multi-line block form below.

variable "captf_contract"        { type = string }
variable "captf_cluster"         { type = object({ name = string, namespace = string }) }
variable "captf_object"          { type = object({ kind = string, name = string, namespace = string }) }
variable "captf_cluster_outputs" {
  type    = any
  default = null
}
variable "captf_tags" { type = map(string) }

variable "machine_name" { type = string }
variable "bootstrap_data" { # base64 of the bootstrap Secret's `value`; e.g. user_data_base64 = var.bootstrap_data
  type      = string
  sensitive = true
}
variable "bootstrap_format" { type = string }
variable "failure_domain" {
  type    = string
  default = null
}
variable "kubernetes_version" {
  type    = string
  default = null
}
variable "control_plane" { type = bool }

# Tag every cloud resource this module creates with captf_tags (common.md
# "captf_tags"); this stub only has to reference it, not create anything.
resource "terraform_data" "tags" { input = var.captf_tags }

output "provider_id"    { value = "noop:///${var.captf_object.namespace}/${var.captf_object.name}" }
output "addresses"      { value = [{ type = "InternalIP", address = "10.0.0.1" }] }
output "failure_domain" { value = var.failure_domain }
output "interruptible"  { value = false }
output "health"         { value = { state = "running", healthy = true, message = null, reasons = [] } }

MachinePool Role

Implements the CAPI InfraMachinePool contract (infra-machinepool.md; MachinePool types in machinepool_types.go) for a TerraformMachinePool. One workspace/state per TerraformMachinePool. The module manages a native scaling group (an ASG, VMSS, MIG, instance pool, or similar) and reports the group id, the list of provider IDs currently in it, and the desired capacity. Mutable: re-applied on spec or replica changes.

The v1 controller implements fixed replicas and native autoscaling (autoscaling.enabled). It does not watch the bootstrap Secret: a rotated token reaches the pool at its next reconcile, at the latest one membership-refresh interval later (see “Bootstrap rotation” below). Drift relies entirely on the module’s own ignore_changes: the drift Job already runs apply -refresh-only then plan -detailed-exitcode for every kind, and the controller does no plan-JSON filtering of the result — an autoscaled pool’s module MUST put its desired-count attribute under lifecycle { ignore_changes = [...] } (see Lifecycle and the autoscaling input) or a cloud-side scale reports as drift.

MachinePool Machines (status.infrastructureMachineKind; optional, infra-machinepool.md “MachinePoolMachines support”) are out of scope. Two consequences follow: there is no drain on scale-down, replacement or version roll — drain exists only for Machines (infra-machinepool.md “MachinePoolMachines support … draining a node before scale down”), so modules SHOULD use native lifecycle hooks or termination handlers instead — and MachineHealthCheck never selects pool instances.

Inputs

Common inputs apply (common.md), including captf_cluster_outputs.

replicas (input)

The desired capacity of the group.

• With autoscaling.enabled = false: MachinePool.spec.replicas, authoritative and in the inputs hash (*int32, “Defaults to 1”; the pointer distinguishes an explicit 0 from unset, and 0 is allowed).
• With autoscaling.enabled = true: TerraformMachinePool.status.replicas (the observed desired capacity from the last refresh) once one exists, else MachinePool.spec.replicas for the first apply before any refresh has run — either way clamped into [autoscaling.min, autoscaling.max] so a render never asks the cloud for an out-of-range desired count. The clamp applies only to this rendered input; the write-back (Lifecycle) always writes the raw observed value, unclamped.

replicas is excluded from the inputs hash while autoscaling.enabled, so a non-replica-triggered apply (bootstrap rotation, module change, exports change) never fights the native autoscaler, and an observed-count change alone never triggers InputsChanged; a fixed-replica pool hashes replicas exactly as before.

bootstrap_data (input)

Base64 of the raw bytes of the value key of the bootstrap Secret named by MachinePool.spec.template.spec.bootstrap.dataSecretName (template is MachineTemplateSpec; see bootstrap-config.md), encoded by the controller for every bootstrap provider exactly as for the machine role (see machine.md bootstrap_data: CAPRKE2 gzipUserData: true makes value raw gzip bytes that cannot be a string).

Modules pass it to a base64-taking launch-configuration argument (for example aws_launch_template.user_data, which expects base64) or base64decode() it for a plain-text argument (UTF-8 payloads only). Hashed by content; dataSecretName: "" is treated as nil (WaitingForBootstrapData).

Rotates. For MachinePools the kubeadm bootstrap provider re-creates the join token when it is past half its TTL (default TTL 15m, checked every TTL/3) and rewrites the same Secret in place, roughly every 7.5 minutes (token.go refreshBootstrapTokenIfNeeded/recreateBootstrapToken; kubeadmconfig_controller.go storeBootstrapData). See Lifecycle for what that requires from modules.

failure_domains (input)

MachinePool.spec.failureDomains (at most 100 items, each 1–256 chars). [] means the module chooses. To make that choice informed, the cluster’s failure-domain names are injected as cluster_failure_domains (below); the full attributes maps are only available through exports.

cluster_failure_domains (input)

Names read from the cluster’s own state, off the failure_domains output (see cluster.md) at render time — not from TerraformCluster.status.failureDomains — so a pool module can spread across all of them when failure_domains is [] without depending on exports conventions. Sourcing it from state rather than status keeps it stable across clusterctl move (status is not moved, and is empty until the target’s first cluster reconcile); it stays in the inputs hash.

kubernetes_version (input)

MachinePool.spec.template.spec.version. A change MUST roll the instances (see Lifecycle). As with the machine role, the value may carry a control-plane-provider-specific distro suffix (RKE2: vX.Y.Z+rke2rN, for example v1.31.4+rke2r1); the controller passes it to the module verbatim, and a module that needs it for an image lookup or a semver comparison MUST strip the +… suffix itself (see machine.md and the RKE2ControlPlane guide).

node_labels (input)

MachinePool.spec.template.metadata.labels verbatim (absent maps to {}). Pool instances have no Machine objects, so core CAPI never syncs labels onto their Nodes; the module MUST render these into kubelet registration (--node-labels) itself. node_labels is in the inputs hash, so an edit re-applies: new members pick it up, and existing members keep their registration.

The precise requirement and why it is not simple. bootstrap_data is opaque (see machine.md “the module MUST NOT need to parse the payload”) and, depending on the bootstrap provider and its settings, may be plain cloud-config, plain Ignition, or gzip of either. The module cannot edit the kubelet flags inside that payload without parsing it, which the contract forbids. The realistic options, in order of how much of bootstrap_data they need to understand:

• Cloud-config, uncompressed (bootstrap_format == "cloud-config", CAPRKE2 gzipUserData unset or false): wrap bootstrap_data and a second, module-generated part in a multipart/mixed MIME message (for example Terraform’s cloudinit_config data source, or an equivalent built by hand) so cloud-init runs both parts at boot. The second part writes a kubelet drop-in with --node-labels:

  data "cloudinit_config" "node" {
    gzip          = false
    base64_encode = true
  
    part {
      content_type = "text/cloud-config"
      # bootstrap_data is base64 of the raw payload (machine.md); decode it
      # only because this part must stay unmodified UTF-8 cloud-config, not
      # to parse or edit it.
      content = base64decode(var.bootstrap_data)
    }
  
    part {
      content_type = "text/cloud-config"
      content = yamlencode({
        write_files = [{
          path    = "/etc/systemd/system/kubelet.service.d/20-node-labels.conf"
          content = "[Service]\nEnvironment=\"KUBELET_EXTRA_ARGS=--node-labels=${join(",", [for k, v in var.node_labels : "${k}=${v}"])}\"\n"
        }]
      })
    }
  }
  # data.cloudinit_config.node.rendered is base64 multipart/mixed; pass it
  # to the same user-data argument bootstrap_data alone would have gone to.
  

  RKE2’s agent config has the same shape through a second part that drops a file under /etc/rancher/rke2/config.yaml.d/*.yaml with a node-label: list, merged by the RKE2 agent at boot alongside the bootstrap part’s own /etc/rancher/rke2/config.yaml.

• Ignition (bootstrap_format == "ignition") or a gzipped payload (CAPRKE2 gzipUserData: true): neither is multipart-mixable the same way — Ignition has its own merge/append config mechanism, and a gzipped payload must be decompressed, edited or merged, and recompressed (or passed through a launch-configuration field that decompresses it itself). These need format-aware handling specific to the format; there is no one wrapper that covers them.

• A module MAY instead declare, in its own documentation, only the bootstrap_format/compression combinations it supports, and fail loudly (a precondition, or an unsupported-value error) on the others, rather than silently dropping node_labels.

Labels the kubelet may not self-assign (the node-role.kubernetes.io/* and other restricted kubernetes.io/k8s.io prefixes under NodeRestriction) are the module’s to filter.

There is no taints input: the v1.14.2 MachinePool webhook rejects any spec.template.spec.taints (“taints feature for MachinePools is not yet implemented”, machinepool.go; MachineTaintPropagation is off by default, feature.go).

autoscaling (input)

Parsed from the MachinePool annotations cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size and -max-size (AutoscalerMinSizeAnnotation/AutoscalerMaxSizeAnnotation, common_types.go). This is the pool’s only autoscaling switch; there is no separate drift flag.

enabled=true with both min/max parsed if and only if both annotations are present and 0 ≤ min ≤ max; otherwise {enabled=false, min=0, max=0} (no annotations sets AutoscalingActive=False/AutoscalingDisabled), plus, when an annotation is present but the pair is incomplete, unparsable or min > max, the condition AutoscalingActive=False/AutoscalingAnnotationsInvalid (the pool still applies without autoscaling; AutoscalingActive never feeds Ready).

How this differs from CAPI’s own reading of the same annotations. CAPI already reads these annotations on MachinePools, but only in a narrower path than ours: the MachinePool admission webhook defaults and clamps spec.replicas from them only when spec.replicas is nil (an explicit value, including 0, is left alone), requires both annotations to be present, rejects the create/update outright if either annotation is unparsable, and does not reject min > max — our controller does, via AutoscalingAnnotationsInvalid (machinepool.go, calculateMachinePoolReplicas, called from Default) — which is what makes them a sound source for the initial size. Our controller does not rely on the webhook: it reads the annotations itself on every reconcile for autoscaling.min/.max, and treats a missing or invalid annotation as enabled=false plus AutoscalingActive=False/AutoscalingAnnotationsInvalid rather than rejecting anything.

enabled means the module owns the desired count and its scaling policy: it sets the group’s min/max to these values, configures whatever native scaling policy it wants (target tracking, scheduled, and so on), and MUST put the group’s desired-count attribute under lifecycle { ignore_changes = [...] } so an apply never resets what the cloud autoscaler decided.

On the CAPI side the controller then claims cluster.x-k8s.io/replicas-managed-by: captf on the MachinePool when the annotation is absent or "false" (ReplicasManagedByAnnotation; CAPI counts the annotation as set for any value except the literal string "false", hasTruthyAnnotationValue, helpers.go, called by ReplicasManagedByExternalAutoscaler — a foreign truthy value is left alone, never overwritten), and, on every reconcile pass on which status.replicas is known and differs from MachinePool.spec.replicas, writes the raw observed desired capacity back to MachinePool.spec.replicas. The InfraMachinePool docs make this the provider’s job (machine-pool.md “It is the provider’s responsibility to update Cluster API’s Spec.Replicas property to the value observed”); see Lifecycle “Write-back” for the exact rules. With the annotation set, the MachinePool controller reports phase Scaling instead of ScalingUp/ScalingDown (machinepool_controller_phases.go).

The Kubernetes Cluster Autoscaler does not act on these pools: its clusterapi provider requires MachinePool Machines (README.md), which this role excludes. Running it against a CAPTF pool anyway is unsupported, because its spec.replicas patches would be overwritten by the write-back.

Outputs

Common outputs apply (health).

provider_id (output)

Maps to TerraformMachinePool.spec.providerID — the scaling-group id; optional in the contract and not used by core CAPI, 1–512 chars (infra-machinepool.md “InfraMachinePool: providerID”). May stay null for group-less implementations.

provider_id_list (output)

Maps to TerraformMachinePool.spec.providerIDList → MachinePool.spec.providerIDList (mandatory; at most 10000 items, each 1–512 chars; infra-machinepool.md “InfraMachinePool: providerIDList”). Rules:

1. It MUST contain every non-terminated member of the group regardless of health — pending, starting, standby, rebooting or unhealthy instances included — because CAPI deletes the Node of any providerID that leaves the list (machinepool_controller_noderef.go deleteRetiredNodes; only an empty list with status.replicas != 0 is guarded, machinepool_controller_phases.go). Listing only “running” or “healthy” instances kills live Nodes.

2. Each entry MUST equal the Node’s spec.providerID exactly (machinepool_types.go providerIDList “must match the provider IDs as seen on the node objects”; see machine.md “Node providerID matching”), otherwise Nodes never get a nodeRef, keep the node.cluster.x-k8s.io/uninitialized:NoSchedule taint that the kubeadm bootstrap provider applies, and are eventually deleted.

3. Order is irrelevant to CAPI: the controller sorts and de-duplicates before writing, since CAPI compares the list with reflect.DeepEqual and any reorder churns status (CAPD sorts too). tfcapi-lint still warns (output/provider-id-list-shape) when the output’s own expression is not wrapped in sort() or distinct(), because a module is applied and refreshed independently of the controller’s write path: an unsorted expression makes provider_id_list reorder between identical apply -refresh-only runs, which is a plan diff (and, for a module whose desired-count is not ignore_changesd, spurious drift) even though the controller’s own write is stable. Wrap the output’s value expression itself — the check inspects that expression directly, not a local it reads from:

   output "provider_id_list" {
     value = sort(local.raw_ids) # or distinct(...), or both
   }
   

   value = local.sorted_ids does not satisfy the check even when sorted_ids is itself sort(...)-derived, and JSON-syntax modules are not scanned at all. Either sort() or distinct() alone satisfies it; the controller sorts and de-duplicates regardless of which the module uses.

replicas (output)

The group’s desired capacity as observed at refresh, not a count of running instances. Maps to TerraformMachinePool.status.replicas → MachinePool.status.replicas (infra-machinepool.md “InfraMachinePool: replicas”; read in machinepool_controller_phases.go). Without MachinePool Machines, CAPI sets the MachinePool’s readyReplicas/availableReplicas equal to this value (machinepool_controller_status.go); only the deprecated v1beta1 readyReplicas is Node-based.

Outside a scaling transition it MUST equal length(provider_id_list); the controller derives status.replicas from this output and serializes 0 explicitly (a missing value would make CAPI keep the old count and block scale-to-zero forever, machinepool_controller_phases.go).

instances (output)

Maps to TerraformMachinePool.status.instances — optional, provider-defined shape, not used by core CAPI (infra-machinepool.md “InfraMachinePool: instances”; the contract’s own example shape is {addresses, instanceName, providerID, version, ready}, ours differs, which is allowed).

One Go type, MachinePoolInstance{ProviderID, InstanceID, Addresses []MachineAddress, FailureDomain, State}, maps field by field: provider_id→providerID, instance_id→instanceID, addresses→addresses (validated like the machine role’s), failure_domain→failureDomain, state→state (the health.state enum).

Size. providerIDList alone may reach 10000×512 bytes, and an object must stay under the etcd request limit of roughly 1.5 MiB, so the controller caps status.instances at 1000 entries (reported as OutputsValid=True/InstancesTruncated when it does), and the documented practical pool size is at most 2000 members.

Provisioned rule

status.initialization.provisioned = true — and, for as long as CAPI v1.14 reads it, status.ready = true; the MachinePool controller decides provisioning solely from status.ready (machinepool_controller_phases.go via external.IsReady, and CAPD still sets it) — when the state Secret carries the inputs-hash annotation of a successful apply and health.state != "pending". provider_id is not required, since the contract makes it optional. provider_id_list may legitimately be empty when replicas == 0, and a 0-member group reports running/healthy (see common.md).

Latched. The formula applies only until it first holds; once true, provisioned and status.ready stay true for the object’s life, whatever later health or outputs say. It is derived from state only, so it is rebuilt after clusterctl move (the formula is re-evaluated once on the target, then latched again); CAPI does not latch its copy, so the MachinePool shows infrastructureProvisioned=false briefly after a move until the first reconcile on the target.

Separately, CAPI copies TerraformMachinePool.spec.providerIDList/status.replicas to the MachinePool only after ClusterCache.GetClient succeeds — that is, only once the workload cluster is reachable (machinepool_controller_phases.go); until then the TerraformMachinePool side is correct but the MachinePool does not reflect it.

Lifecycle

• Mutable. Re-apply on: a spec.source change (image, pull policy); a replicas change on the MachinePool (unless autoscaling.enabled, where replicas is excluded from the inputs hash and a user edit of MachinePool.spec.replicas is overwritten by the write-back, surfaced as an event); a failure_domains change; a node_labels change (applied like bootstrap_data: a launch-configuration update, no instance replacement); a kubernetes_version change; a bootstrap_data change; or a captf_cluster_outputs content change. Every apply, including a drift Remediate, re-renders the current inputs; nothing is replayed from an older run.

• Bootstrap rotation. Because the kubeadm bootstrap provider rewrites the pool’s bootstrap Secret roughly every 7.5 minutes (bootstrap_data above), a pool is re-applied at that cadence for the life of the pool. The controller does not watch the bootstrap Secret — it is not captf.io/managed, so it is not in the cache — so a rotated token reaches the pool at its next reconcile, at the latest one membership-refresh interval later, and is then coalesced into the next apply. Module rules that make this cheap and safe:

  • a bootstrap_data change MUST be applied as an in-place update of the group’s launch configuration (a launch template version, instance template, or VMSS model) so that new members get the current token, and MUST NOT replace existing instances;
  • a kubernetes_version change MUST roll (replace) the instances, because a ClusterClass upgrade completes only when the Nodes’ kubelet versions match (upgrade.go IsMachinePoolUpgrading; CAPD’s DevMachinePool rolls on version only and never on bootstrap data, dockermachinepool_backend.go);
  • modules SHOULD expose no other trigger that replaces instances on an apply with unchanged kubernetes_version.

  Each rotation re-applies the pool, so a module that never converges (see below) is applied roughly every 7.5 minutes indefinitely, on top of its 30s refresh cadence.

• Membership refresh. provider_id_list, replicas, instances and health are only as fresh as the last apply or refresh, and a Node joining the group is unschedulable until its providerID appears in MachinePool.spec.providerIDList — CAPI removes the kubeadm-applied node.cluster.x-k8s.io/uninitialized:NoSchedule taint only then (machinepool_controller_noderef.go). The controller therefore runs apply -refresh-only on its own membership-refresh interval, spec.membershipRefreshIntervalSeconds (default 60, 15–86400, may not be 0), and additionally right after every apply and repeatedly (every 30s) while health.state == "pending" or length(provider_id_list) != replicas — a module that never converges is therefore refreshed every 30s indefinitely. drift.intervalSeconds: 0 is rejected by the CRD schema (minimum 1) for a pool’s own field, by contrast with a machine’s or the cluster’s drift.intervalSeconds, where 0 is a valid setting that disables drift; an inherited defaults.drift.intervalSeconds: 0 (unset) falls back to the controller’s default drift interval (--drift-default-interval, 30m) rather than to 0. Drift may be sparse but membership never is.

• Drift order. Drift for a pool is the same job every other kind runs — apply -refresh-only first, then plan -detailed-exitcode (internal/runner/plan.go) — and the controller does no plan-JSON filtering of the result: whatever the refreshed plan reports is drift. This is why the desired-count lifecycle { ignore_changes = [...] } in the autoscaling input is load-bearing, not optional: with autoscaling.enabled, the module owns the desired count, and a module that does not ignore_changes it reports every cloud-side scale as drift. With Remediate a detected diff re-applies the pool; with autoscaling.enabled = false that scales the group back to MachinePool.spec.replicas — the only mode where the module doesn’t already ignore_changes the desired count.

• Write-back (autoscaling.enabled). On every reconcile pass, one patch (SyncReplicas): the controller claims cluster.x-k8s.io/replicas-managed-by: captf on the MachinePool when the annotation is absent or "false" (a foreign truthy value already means another controller manages it, and is left untouched), and when status.replicas is known and differs from MachinePool.spec.replicas, writes the raw observed value (unclamped by [min,max], unlike the rendered replicas input) to spec.replicas in the same patch. A ReplicasWrittenBack Normal event is emitted on the TerraformMachinePool whenever a write happens. With autoscaling.enabled = false, the controller removes replicas-managed-by only if it still carries captf (never a foreign value), and never writes spec.replicas. Without the write-back, CAPI would report ScalingUp/ScalingDown forever and upToDateReplicas would follow the stale spec (machinepool_controller_status.go). A ClusterClass-managed MachinePool MUST leave replicas unset in the topology for this to work — the topology controller would otherwise reassert it every reconcile, fighting the write-back.

  Module note. Because lifecycle { ignore_changes } cannot be conditional on a variable, a module that supports both autoscaling.enabled = true and = false from the same scaling-group resource should, when disabled, pin the group’s min_size/max_size to var.replicas — not to var.autoscaling.min/.max, which are 0 when disabled — so the same ignore_changes block still lets the group track replicas exactly.

• Health. health feeds conditions only; there is no remediation for pools in v1 (MHC selects Machines, and pool replicas only have Machines under MachinePool Machines, which this role excludes; infra-machinepool.md “MachinePoolMachines support”). Instances terminated by the cloud simply leave provider_id_list at the next refresh, and CAPI deletes their Nodes.

• Delete. A destroy Job renders from the object-owned captf-inputs-<kindshort>-<name> Secret, not from the bootstrap Secret: CAPI deletes the bootstrap config and the InfraMachinePool in the same pass (machinepool_controller_phases.go reconcileDeleteExternal), so the bootstrap Secret is usually already gone when destroy runs. Then the controller drops state, the Lease and the inputs Secret, and removes the finalizer.

  Node cleanup is partial. On MachinePool deletion, CAPI deletes only the Nodes whose providerIDs are absent from the last-seen providerIDList (reconcileDeleteNodes → deleteRetiredNodes, machinepool_controller_noderef.go); Nodes still listed survive and are removed by the cloud-controller-manager once their instances are gone, or remain as orphans without a CCM. A module MAY empty provider_id_list on the final refresh before destroy, but the controller does not depend on it.

Minimal skeleton

A block written on one line may hold at most one argument (OneLineBlock; tofu validate reports “Invalid single-argument block definition” on a violation), so a variable that needs both type and sensitive, or type and default, must use the multi-line block form below.

variable "captf_contract"        { type = string }
variable "captf_cluster"         { type = object({ name = string, namespace = string }) }
variable "captf_object"          { type = object({ kind = string, name = string, namespace = string }) }
variable "captf_cluster_outputs" {
  type    = any
  default = null
}
variable "captf_tags" { type = map(string) }

variable "machinepool_name" { type = string }
variable "replicas" { type = number }
variable "bootstrap_data" { # base64 of the bootstrap Secret's `value`; e.g. launch template user_data = var.bootstrap_data
  type      = string
  sensitive = true
}
variable "bootstrap_format" { type = string }
variable "node_labels" { type = map(string) }
variable "failure_domains" { type = list(string) } # always set, may be []; no default (input/default)
variable "cluster_failure_domains" { type = list(string) } # always set, may be []; no default (input/default)
variable "kubernetes_version" {
  type    = string
  default = null
}
variable "autoscaling" { type = object({ enabled = bool, min = number, max = number }) }

# Tag every cloud resource this module creates with captf_tags (common.md
# "captf_tags"); this stub only has to reference it, not create anything.
resource "terraform_data" "tags" { input = var.captf_tags }

locals {
  # Unsorted; provider_id_list must wrap sort()/distinct() directly in its
  # own output expression: see "provider_id_list order" below.
  raw_ids = [for i in range(var.replicas) : "noop:///${var.captf_object.namespace}/${var.captf_object.name}/${i}"]
}

output "provider_id"      { value = "noop-group:///${var.captf_object.namespace}/${var.captf_object.name}" }
output "provider_id_list" { value = sort(local.raw_ids) }
output "replicas"         { value = var.replicas }
output "instances"        { value = [for id in sort(local.raw_ids) : { provider_id = id, state = "running" }] }
output "health"           { value = { state = "running", healthy = true, message = null, reasons = [] } }

A real module with autoscaling.enabled sets the group’s min_size/max_size from var.autoscaling, its desired_capacity from var.replicas, and declares lifecycle { ignore_changes = [desired_capacity] } on the group resource; output "replicas" then reads the group’s current desired capacity from the resource (refreshed by apply -refresh-only), not var.replicas.

Per-instance state

instances[*].state uses the health.state enum (see common.md) so per-instance conditions can be derived later.

Deriving group health from mixed instance states. The controller does not aggregate instances[*].state into health itself: health is copied to the InfrastructureHealthy condition, and instances is copied to status.instances (capped, see “Size” above), with no cross-referencing between the two — a module that reports a healthy health and a list full of degraded instances gets a healthy InfrastructureHealthy and a degraded-looking status.instances, since nothing else reconciles the difference. A pool module SHOULD derive health from the instances it is about to report:

• healthy = true, state = "running" when the group has reached its desired capacity and every counted instance’s own state is running;
• healthy = false, state reflecting the worst instance (for example degraded if any instance is degraded, else stopped, and so on down the enum) with reasons naming the affected instances, when some instances are not running;
• state = "pending" only while the group itself does not yet exist or has no members to report at all — not merely while some members are still starting, since length(provider_id_list) != replicas already forces the controller’s fast (30s) refresh loop on its own (see “Membership refresh” below), so a module does not need health.state = "pending" to get that cadence during a scale-up or scale-down.

This is a SHOULD, not a MUST: a module that cannot observe per-instance health may still report {state="running", healthy=true} as long as the group itself is healthy (see common.md).

Changelog

Changes to the module contract, newest first. Within v1alpha* there is no compatibility guarantee (see README.md “Versioning”); every change is still recorded here.

Unreleased

Added

• Machinepool role, including native autoscaling (additive within v1alpha1; machinepool.md, README.md “Roles and CAPI mapping”): the v1 controller now reconciles TerraformMachinePool. New API kinds TerraformMachinePool and TerraformMachinePoolTemplate, and new field TerraformMachinePoolSpec.membershipRefreshIntervalSeconds (default 60, 15–86400, rejected below 15 by the CRD schema, not the webhook). New OutputsValid reason InstancesTruncated, set when status.instances is capped at 1000 entries. Manager flag --terraformmachinepool-concurrency (default 10). The controller does not watch the bootstrap Secret (a rotated token reaches the pool at its next reconcile, at the latest one membership-refresh interval later) and does not support MachinePool Machines.
• Machinepool autoscaling (additive within v1alpha1; machinepool.md “Inputs” autoscaling/replicas, “Lifecycle” “Drift order” and “Write-back”): the autoscaling input is parsed from the MachinePool’s cluster-api-autoscaler-node-group-min-size/-max-size annotations; when enabled, replicas renders status.replicas (falling back to MachinePool.spec.replicas before the first observation) clamped into [min,max], is excluded from the inputs hash, and the controller writes the raw observed value back to MachinePool.spec.replicas and claims cluster.x-k8s.io/replicas-managed-by: captf (never a foreign truthy value), removing its own claim when autoscaling is disabled again. New condition AutoscalingActive (informational, pool-only, never in Ready; reasons ReplicasManagedByModule, AutoscalingDisabled, AutoscalingAnnotationsInvalid). New event ReplicasWrittenBack. New RBAC: patch on cluster.x-k8s.io machinepools (write-back needs it; get/list/watch already covered reads). Drift for a pool already ran apply -refresh-only then plan like every other kind; nothing changed there — it is the module’s lifecycle { ignore_changes } on the desired-count attribute that keeps a cloud-side scale from reporting as drift, and that responsibility is now load-bearing rather than theoretical. Upgrade behavior: a MachinePool that already carries both autoscaler annotations before the upgrade starts autoscaled and re-applies once right after it — its autoscaling input flips from {enabled=false,min=0,max=0} to the parsed, enabled value, which is hashed, so the inputs hash changes and the pool re-applies.
• Plan preview and approval (the module contract is unchanged; see Plan Approval): TerraformCluster and TerraformClusterTemplate gain spec.applyPolicy (Automatic, the default applied at reconcile, or Manual; mutable, not hashed). Under Manual every apply but the first (no state yet), including a drift remediation and a retry, first runs a new operation, plan (added to the Operation enum): a Job that runs init, validate, plan -out and show -json under the run lease only and applies nothing. Its plan (counts, up to 50 "<address> (<action>)" entries, never values, and a plan hash p1:<sha256> of the sorted address|actions of every change) goes to the new status.plan (inputsHash, job, planHash, add, change, destroy, resources, truncated, createdAt), and the apply waits until the new captf.io/approve-plan annotation names that hash. The approved apply (runner flag --expect-plan) plans again and applies only an identical plan; otherwise it stops with the new RunErrorKind plan-changed and the new plan waits for its own approval. A plan without changes never waits. Approving a plan also approves its deletes; the destructive-plan annotation is not needed on top. The annotation is removed and status.plan cleared once the approved apply succeeds. New ApplyJobSucceeded Unknown reasons PlanAwaitingApproval and PlanChanged; events PlanReady, PlanApproved, PlanApplied, PlanChanged; metric captf_plan_approvals_total{kind,result} (approved, changed) and captf_jobs_total result plan_changed; Job annotations captf.io/approved-plan, captf.io/plan-changed and captf.io/plan-unreadable.
• State backups and restore (the module contract is unchanged; see Terraform State and State Restore): every new state serial the manager reads is copied verbatim into captf-state-backup-<suffix>-<serial> Secrets (plus -part-N per chunk), owned by the Terraform* object and labeled for clusterctl move; the newest --state-backups (default 5, 0 disables) are kept. TerraformCluster and TerraformMachine gain status.stateBackups (up to 16 {serial, takenAt, bytes}). The captf.io/restore-state: "<serial>" annotation starts a new operation, restore (added to the Operation enum of status.activeJob and status.lastRun): a Job that runs init, state push -force of the backup and state list, under the run lease (and the cluster write lease for a TerraformCluster). It precedes apply, drift and refresh but not deletion; the annotation is removed on success, and a failure is not retried for the same serial. New condition RestoreJobSucceeded (never in Ready) with reasons StateRestored, RestoreFailed, RestoreBackupNotFound and the three lease waits; events StateBackedUp, StateRestored, StateRestoreFailed; metrics captf_state_backups_total{kind,result} and captf_state_restores_total{kind,result}; runner steps state-push and state-list. A TerraformCluster’s restore takes part in the cluster operation gate like its apply.
• User variables (see common.md “User variables” and Module Variables): TerraformCluster and TerraformMachine (and both templates) gain spec.variables, an inline JSON object, and spec.variablesFrom, up to 16 ConfigMaps or Secrets labeled captf.io/variables=true, each optional and read in format String (default) or JSON. Sources merge in list order, a later one winning; inline variables win over all. Each variable becomes a named argument of module "role", declared in the generated root without a type (sensitive = true when its value came from a Secret) and valued in terraform.tfvars.json; a name the module does not declare fails the apply. Reserved: captf_ names, the role’s contract inputs and the module meta-arguments. The inputs hash covers the variables only when an object sets some, so existing objects keep their h2 hash and do not re-apply. On a TerraformCluster a changed variable or source re-applies; on a TerraformMachine both fields are immutable and read only until the machine is provisioned. New DependenciesReady False reasons VariablesSourceNotFound and VariablesInvalid. The manager’s ClusterRole gains get, list and watch on configmaps, and it runs a second, label-scoped cache (captf.io/variables=true, data stripped) for the source watches.
• Run leases (the module contract is unchanged; see The Reconcile Lifecycle): the manager holds a coordination.k8s.io/v1 Lease captf-run-<suffix> per object before it creates a Job, so a stale Job cache, a leader-election handover or two manager instances with overlapping --watch-filter values cannot run two Jobs of one object. With the new --cluster-operation-gate (default true) a TerraformCluster’s apply or destroy and its machines’ applies and destroys never run at once (cluster write Lease captf-cluster-<hash>); refresh and drift are not gated. New ApplyJobSucceeded Unknown reasons WaitingForRunLease, WaitingForClusterOperation and WaitingForMachineOperations (DriftJobSucceeded gains WaitingForRunLease), the Normal events of the same names once per wait, and captf_lease_waits_total{kind,reason}. The manager’s ClusterRole gains create and update on leases.
• Events for every stage (the module contract is unchanged; see Observability “Events”): the runner now reports its progress as events.k8s.io/v1 Events on the object that owns the Job (RunStarted, StepStarted, StepSucceeded, StepFailed, PlanSummary, ResourcesChanged, RunFinished; reporting controller captf.io/runner), best effort and never failing the run. Job pods get two new runner flags, --event-object and --job-name; the manager’s new --runner-events (default true) turns them off. The captf-runner ClusterRole gains events.k8s.io events create. The manager emits new reasons once per transition: JobSucceeded, JobInterrupted, JobDeadlineExceeded, StuckJobDeleted, DestructivePlanApprovalConsumed, DeletionStarted, FinalizerRemoved, Paused, Resumed, ProviderIDSet, ControlPlaneEndpointSet, FailureDomainsChanged, InputsChanged, StateAdopted, StateLost, StateLocked (both formerly StateUnreadable), DriftResolved, DriftRemediationStarted, InstanceHealthy, InstanceUnhealthy, MirrorCreated, MirrorRemoved, IdentitySecretFound, IdentitySecretNotFound, CapacityResolved, and ConditionChanged for any other owned condition’s status or reason change. JobFailed now also covers failed drift and refresh Jobs; JobCreated names the attempt, the image and why the Job started.
• Destructive-plan guard: a TerraformCluster apply, including a drift remediation, runs plan -out, show -json and an apply of the saved plan, and stops before a plan whose actions include delete (a removal or a replacement) unless the captf.io/approve-destructive-plan annotation names the inputs hash it renders. New status.lastRun.error.kind: blocked, reason ApplyJobSucceeded=False/DestructivePlanBlocked, event DestructivePlanBlocked and captf_jobs_total{result="blocked"}. The image’s runtime must support plan -out=<file> and show -json <planfile> (see Image Contract). Machine applies are unchanged.
• Controller remediation additions (the module contract is unchanged):
  • New remediation.healthCheckIntervalSeconds (60–86400, default 300): with annotateMachine, a provisioned machine is refreshed at that interval to sample health, independent of drift. A health sample is one completed refresh or drift Job, not a new state serial.
  • New condition reasons: StateReadable=False/StateLost (a provisioned object’s state is gone or lost its inputs hash; no Job runs), StateReadable=False/StateLocked (the state lock is held by something other than the object’s runner) and ApplyJobSucceeded=False/InputsTooLarge. DriftDetected starts as Unknown/DriftNotChecked.
  • captf_job_attempts records the retry number of a successful Job (failures of that op since its last success, plus one); status.activeJob.attempt is the Job’s sequence number.
• tfcapi-lint’s input/extra error is replaced by the input/user-variable-default warning (a non-contract variable without a default is legitimate when every object is meant to set it through spec.variables/spec.variablesFrom).

Changed

• The contract pages (README.md, common.md, cluster.md, machine.md, machinepool.md and this changelog) were reorganized for readability: long paragraphs were split into sections, table cells were shortened with their detail moved into a section below each table, and source citations became links. The contract itself did not change: every MUST/SHOULD/MAY requirement, every schema and every skeleton is unchanged.
• Fewer Jobs per bring-up and in steady state (the module contract is unchanged; see Machine Remediation, Plan Approval and The Reconcile Lifecycle): a TerraformMachine no longer refreshes after a successful apply whose own outputs are valid with a health.state other than pending or unknown; that reading is the post-apply health sample (status.lastRefresh is the apply’s finish, and it counts once toward status.unhealthySamples). While health is pending the refresh backs off 30s, 1m, 2m, 4m, then every 5m (plus the UID jitter) instead of a fixed 30s; TerraformCluster and TerraformMachine gain status.pendingRefreshes, the consecutive pending samples since the last other reading or apply (status only; it restarts after clusterctl move). A guarded or approved (--expect-plan) apply whose plan has no changes skips the apply step and succeeds right after the plan, with zero changes and no ResourcesChanged event, so its leases go as soon as it is bookkept. The example bring-up (one cluster, six machines, one no-change re-apply) drops from 14 Jobs to 8.
• Controller remediation behavior (the module contract is unchanged):
  • The cluster.x-k8s.io/remediate-machine annotation CAPTF set (marked captf.io/remediation-requested) is removed once the instance reads Healthy and the Machine is not being deleted.
  • The first drift check runs one interval after the last successful apply; drift and health deadlines get a per-object jitter of up to 10%.
  • A failed apply is retried even when the inputs equal the state’s hash. An image change clears the pinned digest in the durable inputs Secret.
• TerraformMachine.spec.drift and spec.defaults.drift are a MachineDriftPolicy with intervalSeconds only: machine drift is always reported. The cluster’s spec.drift.action defaults to Report, not Remediate.
• Machine jobs and drift policies are merged field by field over the cluster’s spec.defaults (env by name, imagePullSecrets as a union), instead of replacing them as a whole. spec.defaults no longer applies to the TerraformCluster itself.
• TerraformMachine.spec.jobs, drift and remediation are mutable; source and identityRef stay immutable.
• jobs.lockTimeoutSeconds is optional as a pointer; jobs.activeDeadlineSeconds (at most 86400) and remediation.unhealthyThreshold default when unset (0), not through a pointer. A policy that sets both lockTimeoutSeconds and activeDeadlineSeconds must keep the lock timeout below the deadline. jobs.securityContext may not set privileged, allowPrivilegeEscalation or capabilities.add.
• TerraformCluster.spec.identityRef is required; machines fall back to spec.defaults.identityRef, then to spec.identityRef. spec.controlPlaneEndpoint is immutable once it has a host.
• TerraformClusterIdentity.spec.allowedNamespaces: {} is rejected; write selector: {} to allow every namespace. The identity has a status (conditions, namespaces), filled by the manager.
• status.lastRun.error.tail is now summary (at most 512 bytes): the runner’s summary of the failure, not raw stderr.
• captf_cluster_outputs is not passed to the cluster role. Previously the text said it was null. The generated cluster root declares no such variable, and its tfvars carry no key. A cluster module may leave it undeclared or declare it with default = null, as the skeletons do. A declaration without a default fails validate. Updated common.md, cluster.md, and the cluster-inputs.json description; the schema still accepts null.
• Consequence of the inputs hash: the hash accepts only integer numbers. The cluster role’s exports value becomes every machine’s captf_cluster_outputs input, which is hashed. So a fractional or exponent number anywhere in exports (for example {"ratio": 0.5}) makes those machines’ inputs unhashable, and they cannot apply. Module authors should export numbers as integers or strings. The controller reports it as OutputsValid=False/OutputsInvalid on exports. No tfcapi-lint check catches this today. The alternative, a canonical float encoding in the hash, was not chosen, because the design calls for integers only and a float that does not round-trip exactly would change the hash silently.
• API field names: the drift and job durations are integer seconds, following the Cluster API v1beta2 convention and the kube-api-linter nodurations rule. spec.drift.interval is now spec.drift.intervalSeconds, spec.drift.refreshInterval is spec.drift.refreshIntervalSeconds, and jobs.lockTimeout is jobs.lockTimeoutSeconds. Defaults are unchanged (1800, pool-only 60 with a minimum of 15, and 300 seconds). References in these documents were renamed.

Removed

• jobs.backoffLimit (Jobs always get backoffLimit: 0) and jobs.ttlSecondsAfterFinished (Jobs never get a TTL): the controller owns retries, and backoff, digest pinning and conditions are derived from the Jobs it retains.
• spec.drift.refreshIntervalSeconds (it did nothing), spec.source.imagePullSecrets (use jobs.imagePullSecrets, which covers the source and runner images) and spec.source.command (the image contract fixes /captf/runtime; the image’s /captf/runtime is now the only executable the runner starts). The durable inputs Secret no longer carries captf.io/command, and the inputs hash no longer covers a command: the scheme is now h2, so every provisioned mutable object re-applies once after the upgrade.
• The five no-op mutating webhooks; only validating webhooks remain. Unused condition reasons RBACPending, StateReadPending and CapacityResolving.

Fixed

• machinepool.md’s “Minimal skeleton” gave failure_domains and cluster_failure_domains a default = []; both are always set (non-null) by the controller, so tfcapi-lint’s input/default check warned on either one, and --strict failed. The defaults are removed; the inputs are documented as always set (may be []), matching modules/noop/machinepool. Every role’s “Minimal skeleton” (cluster.md, machine.md, machinepool.md) now also references captf_tags (an empty terraform_data stub), since the skeletons are now actually run through tfcapi-lint module --role <role> --strict, which otherwise warns input/tags-unused on all three.
• cluster.md’s “Minimal skeleton” typed cluster_network’s attributes with optional(...), while the Inputs table types them as required and the controller always renders all four (an unset CIDR list as [], an unset scalar as null, never omitted). Both forms pass tfcapi-lint’s input/type check, but the skeleton now matches the table.
• The output/provider-id-list-shape check registers at both error (the output is missing or not a list) and warning (the expression is not sorted and deduplicated); reference/tfcapi-lint-cli.md listed it as two separate rows with the same description and no way to tell them apart. It is now one row whose Severity cell names both.
• machinepool.md said provider_id_list’s order was irrelevant (true for the controller’s own write, which sorts and de-duplicates) without mentioning that tfcapi-lint separately checks the output’s own expression for sort()/distinct(), and the skeleton’s comment pointed at modules/noop/machinepool/outputs.tf instead of explaining why. Both are now explained inline, with the pattern shown directly.
• machinepool.md’s node_labels said only that the module “MUST render these into kubelet registration … through the bootstrap/agent config it controls,” without saying how, while bootstrap_data is opaque and may be gzipped or Ignition. The realistic options (a multipart/mixed cloud-config extra part, with a worked example; Ignition and gzipped payloads needing format-aware handling; or a module declaring only the formats it supports) are now spelled out. No new MUST or SHOULD is added.
• runtime-environment.md did not say that a provider’s own debug logging (TF_LOG) cannot be turned on for a Job: it is dropped like every other TF_* name, and spec.jobs.env rejects it outright. The page now says so and points at running the pinned image locally against copies of the rendered inputs (the stuck-destroy runbook’s recipe) with TF_LOG set, as the supported way to get provider debug output.
• runtime-environment.md did not say that a network provider mirror or a private registry cannot be configured at run time: TF_CLI_CONFIG_FILE is runner-owned and every other TF_* name is dropped. The page now states this and points at a filesystem mirror baked into the image (/captf/providers) as the supported path.
• machinepool.md and common.md described health, instances and their controller-side effects individually, without saying how a pool module should combine mixed instances[*].state values into one group health — the controller does not aggregate them itself. machinepool.md “Per-instance state” now states what the controller does (nothing: both outputs are copied independently) and what a module SHOULD do to derive health from instances (a new SHOULD, not a MUST).
• image-contract.md recommended the org.opencontainers.image.* labels but the reference Containerfiles (docs/book/src/module-author/examples/Containerfile.terraform and .opentofu) never set them. Both now take IMAGE_SOURCE, IMAGE_REVISION and IMAGE_VERSION build args (empty by default) and set the three labels from them.

v1alpha1 (provisional freeze)

Frozen for implementation: Go types and the controller are built against this version. It may still change until the first real module has provisioned a cluster (the contract is frozen for good only once one real cloud module has created a cluster); every such change is recorded here.

Scope

• README.md, common.md, cluster.md, machine.md, machinepool.md and schemas/ in this directory.
• Image Contract and the reference Containerfiles (docs/book/src/module-author/examples/, modules/noop/).
• The control-plane guides in Control-Plane Integration.

Changed at the freeze

• Provider mirror wildcard. The runner writes include = ["*/*/*"] and exclude = ["*/*/*"], not */*. Verified with the pinned runtimes Terraform v1.16.4 and OpenTofu v1.12.6, and separately with OpenTofu v1.11.5: */* matches providers on the default registry host only, so a provider from any other host fell through to direct; */*/* matches every host.
• Remediation wording (machine.md “Health → remediation”). Remediation happens for Machines whose owner acts on MachineOwnerRemediated: a MachineSet or a control-plane provider that implements remediation (KubeadmControlPlane, RKE2ControlPlane), not only a MachineSet or KubeadmControlPlane.

Added at the freeze

• Reserved paths. /captf/bin (the injected runner) and /captf/config (the per-run Secret) are mount points; an image MUST NOT ship anything there. Added to the image contract’s “Fixed paths” table and the tfcapi-lint image checklist.
• Machine-checkable JSON Schemas (draft 2020-12) in schemas/: definitions.json, cluster-inputs.json, cluster-outputs.json, machine-inputs.json and machine-outputs.json, with a valid and an invalid example for each. The schemas change nothing in the contract; they encode it.
  • Inputs are closed at every object level (module inputs are exactly the contract inputs); outputs are open (extra, non-contract outputs are ignored).
  • Outputs are validated after the controller’s normalization: "" for an ID such as provider_id becomes null first, so the schema requires 1–512 characters for a non-null provider_id.
  • Limits come from the CAPI v1.14.2 API markers, including three the role documents do not spell out: service_domain 1–253 characters, at most 100 CIDR blocks of 1–43 characters each, api_server_port 1–65535.
  • Not expressible in the schemas and enforced by the controller: unique failure-domain names, the canonical address order, and a machine’s failure_domain output equal to its input when one was requested.
• The image contract’s reference Containerfiles create /captf/providers before running providers mirror. Neither terraform providers mirror (v1.16.4, in the reference build) nor tofu providers mirror (checked with OpenTofu v1.11.5 on the build host) creates the target directory when the module requires no providers, so the previous recipe failed with lstat /captf/providers: no such file or directory for a provider-less module (found building the reference images).

Decisions

• Operator decisions: native pool autoscaling, runner Secret access accepted and documented (see Security Model), static credential Secrets only in v1.
• Every candidate CAPI mapping considered was decided; the adopted ones are in the role documents (e.g. address order, interruptible, cluster kubernetes_version, control_plane_initialized, endpoint provenance, empty dataSecretName, captf.io/template).
• The role documents’ claims were checked against CAPI source and resolved. Resolutions for the items raised in the RKE2ControlPlane guide:
  • binary bootstrap data: resolved by always-base64 bootstrap_data;
  • providerID under RKE2: no contract change; the machine module makes the kubelet provider-id match (CCM or kubelet extra args), as the guide and checklist say;
  • a second listener on 9345: a cluster-module convention, not a contract field, since CAPI has nowhere to put it;
  • the InternalIP guarantee: no contract change; no tfcapi-lint check exists for it in internal/lint;
  • the version string: stays verbatim; modules strip +rke2rN (as machine.md says), because normalizing would change the inputs hash;
  • remediation wording: fixed above;
  • CAPRKE2 on CAPI v1.14: needs a compatibility smoke test against CAPI v1.14.2 before RKE2ControlPlane is relied on in production;
  • a nondeterministic join target: upstream behavior, documented in the guide.
• The contract is authoritative on captf_contract: every role receives it (common.md), and the renderer renders it for every role.
• Proposals considered and not adopted for common.md, cluster.md, machine.md and machinepool.md (extra tags, captf_identity, health.observed_at, a resources inventory output, cluster tags/dns_zone, machine image/instance_type/disk/ssh_authorized_keys/machine_uid, failure_domain_attributes, bootstrap_data_hash, instance_id, pool rolling_update and ready_replicas): none loosens the closed input schemas (schemas/) or is needed by a shipped module; several are fixed in the module instead because the CRDs have no free-form per-object variables outside spec.variables/spec.variablesFrom.

Image Contract

The OCI image is the deliverable a module author ships. This page is the normative image contract for the v1alpha1 module contract: the fixed paths CAPTF’s runner looks for, the labels it and tfcapi-lint read, the user the image should run as, and multi-arch publishing. tfcapi-lint image checks an image against this contract. What the module actually sees when the runner executes it — the generated root, the environment, the commands run and their order — is on Runtime Environment.

One image bundles one role module (cluster, machine or machinepool) and the runtime that runs it (tofu or terraform). There is no separate module source and no separate runtime image: the image tag is the module version, the image is what spec.source.image references, and the image is what gets pinned, moved, rolled out and audited. The controller runs it as a Job with its own runner binary as the entrypoint; the image itself never needs a shell.

Fixed paths

The runner discovers a module by fixed path, not by label or registry metadata, and fails the Job at start (error.kind: image-layout) if a required path is missing or unusable.

Everything else in the image is the author’s business: CA certificates, git for provider blocks that shell out, or a helper binary a local-exec provisioner calls.

Provider mirror layout

Build /captf/providers with terraform providers mirror <dir> or tofu providers mirror <dir>, for every platform the image publishes (for example -platform=linux_amd64 -platform=linux_arm64). Both runtimes accept either layout it can produce: the packed layout (HOST/NAMESPACE/TYPE/terraform-provider-TYPE_VERSION_TARGET.zip plus .json index files) or the unpacked layout (HOST/NAMESPACE/TYPE/VERSION/TARGET/).

providers mirror creates its target directory only when it writes at least one provider, so a module that requires none needs the mirror stage to create /captf/providers itself (see the reference Containerfiles below) or the later COPY --from=mirror fails. It also fails with “Module not installed” when the module calls local modules that are not yet installed, so run terraform get / tofu get first, in the same stage; that installs modules only, never providers.

Providers are optional: an image without /captf/providers still works, but needs registry egress at init and is slower and non-hermetic. The reference images in this repository ship a mirror. How the runner uses the mirror at run time is on Runtime Environment.

OCI labels

Labels are metadata only: the runner never reads them for behavior. tfcapi-lint image checks them for consistency with --role/--contract.

The reference Containerfiles below take these three as IMAGE_SOURCE, IMAGE_REVISION and IMAGE_VERSION build args (empty by default), so a build pipeline sets them with --build-arg from the source repository URL, the commit, and the tag being built.

Capacity labels (machine role only, optional) let TerraformMachineTemplate support Cluster Autoscaler scale-from-zero: the module fixes the instance type, so the image is the only place that knows the node’s size. Set both identically on every platform of a multi-arch index; an image without them leaves status.capacity/status.nodeInfo unset.

A module whose instance type varies needs one image per instance type to use these labels; pool images may carry them, but they are ignored.

User

Any UID works for the runner, but recommend a non-root USER (for example 65532) so the Job can run under a namespace that enforces the Pod Security restricted profile. Files and directories under /captf/module and, when present, /captf/providers must be readable, and directories traversable, by that UID. Credential files are mounted mode 0440, so a non-root image user reads them through the pod’s fsGroup, not through ownership. See Security Model for how the pod’s own security context defaults interact with the image’s USER.

Multi-arch

Publish a multi-arch manifest (linux/amd64, linux/arm64) or pin the management cluster’s node architecture to the platform the image ships: tfcapi-lint image checks the linux/amd64 platform by default, and --platform/--all-platforms select others. A provider mirror must carry a package for the target platform it is checked against.

Versioning and pinning

The image tag is the module version: spec.source.image is registry/repo:tag or registry/repo@sha256:…, and a new module version is a new tag referenced by a new Terraform*Template. The controller pins the digest it actually ran after the first successful apply and, for immutable machines, uses that digest for every later drift and destroy Job; see Security Model for the full mechanics and why it matters.

Contract version is not declared in the image: labels are informational. The controller injects the contract it generates against as captf_contract, and tfcapi-lint takes --contract on the command line.

Trust boundary

Referencing an image is equivalent to granting its publisher the runner’s Secret access and the resolved identity’s cloud credentials in that namespace: see Security Model for the full trust boundary and what it means for review and tenancy.

Building an image

Lint the module first, then build, then lint the pushed image:

tfcapi-lint module ./cluster --role cluster --strict
podman build -t "$IMAGE" .
tfcapi-lint image "$IMAGE" --role cluster --strict

The reference images below ship as examples/Containerfile.terraform and examples/Containerfile.opentofu, each taking ARG ROLE and ARG RUNTIME_VERSION.

Reference: Terraform base

hashicorp/terraform is Alpine with git, openssh and CA certificates, its binary at /bin/terraform, and ENTRYPOINT ["/bin/terraform"]; the runner replaces that entrypoint, so it has no effect. The image has no USER (root); the Containerfile below adds one.

# Reference source image, Terraform base (docs/book/src/module-author/image-contract.md "Reference:
# Terraform base"). Run from your module's root directory:
#
#   podman build -f Containerfile.terraform --build-arg ROLE=cluster \
#     --build-arg IMAGE_SOURCE=https://github.com/<org>/<repo> \
#     --build-arg IMAGE_REVISION="$(git rev-parse HEAD)" \
#     --build-arg IMAGE_VERSION=<tag> -t <registry>/<repo>:<tag> .
#
# The image tag is the module version. Lint first:
#   tfcapi-lint module . --role cluster --strict

ARG RUNTIME_VERSION=1.16.4
# The org.opencontainers.image.* labels below (image-contract.md "OCI
# labels"): leave these unset for a local/test build, or pass them from
# your CI pipeline (source repo URL, commit SHA, the image tag).
ARG IMAGE_SOURCE=""
ARG IMAGE_REVISION=""
ARG IMAGE_VERSION=""

# Optional but recommended: hermetic provider mirror for the platforms you
# publish. Needs registry egress at build time; drop this stage (and the
# COPY --from=mirror below) for a non-hermetic image.
FROM docker.io/hashicorp/terraform:${RUNTIME_VERSION} AS mirror
WORKDIR /src
COPY . /src
# get: `providers mirror` refuses a module whose nested local modules are not
# installed; get installs them (no providers), in this stage only.
# mkdir: `providers mirror` does not create the target when the module
# requires no providers, and the COPY --from=mirror below needs it.
RUN terraform get \
 && mkdir -p /captf/providers \
 && terraform providers mirror -platform=linux_amd64 -platform=linux_arm64 /captf/providers

FROM docker.io/hashicorp/terraform:${RUNTIME_VERSION}
ARG ROLE=cluster
ARG RUNTIME_VERSION
ARG IMAGE_SOURCE
ARG IMAGE_REVISION
ARG IMAGE_VERSION
COPY --from=mirror /captf/providers /captf/providers
COPY . /captf/module
RUN ln -s /bin/terraform /captf/runtime \
 && adduser -D -u 65532 captf \
 && chown -R 65532:65532 /captf
USER 65532
LABEL io.captf.contract="v1alpha1" \
      io.captf.role="${ROLE}" \
      io.captf.runtime="terraform" \
      io.captf.runtime.version="${RUNTIME_VERSION}" \
      org.opencontainers.image.source="${IMAGE_SOURCE}" \
      org.opencontainers.image.revision="${IMAGE_REVISION}" \
      org.opencontainers.image.version="${IMAGE_VERSION}"

Mirroring providers needs registry egress at build time; drop the mirror stage (and its COPY --from=mirror) for a non-hermetic image.

Reference: OpenTofu base

opentofu:*-minimal is FROM scratch with only the tofu binary — no CA certificates, no shell, no git — which is why the Containerfile below copies it into an Alpine stage instead of using it directly as the final base. The full (non-minimal) OpenTofu image refuses to be used as a FROM base.

# Reference source image, OpenTofu base (docs/book/src/module-author/image-contract.md "Reference:
# OpenTofu base"). Run from your module's root directory:
#
#   podman build -f Containerfile.opentofu --build-arg ROLE=machine \
#     --build-arg IMAGE_SOURCE=https://github.com/<org>/<repo> \
#     --build-arg IMAGE_REVISION="$(git rev-parse HEAD)" \
#     --build-arg IMAGE_VERSION=<tag> -t <registry>/<repo>:<tag> .
#
# The image tag is the module version. Lint first:
#   tfcapi-lint module . --role machine --strict
#
# The full ghcr.io/opentofu/opentofu image refuses to be a FROM base
# (ONBUILD RUN exit 1); use the -minimal tag via COPY --from as below.

ARG RUNTIME_VERSION=1.12.6
# The org.opencontainers.image.* labels below (image-contract.md "OCI
# labels"): leave these unset for a local/test build, or pass them from
# your CI pipeline (source repo URL, commit SHA, the image tag).
ARG IMAGE_SOURCE=""
ARG IMAGE_REVISION=""
ARG IMAGE_VERSION=""

FROM ghcr.io/opentofu/opentofu:${RUNTIME_VERSION}-minimal AS tofu

# Optional but recommended: hermetic provider mirror for the platforms you
# publish. Needs registry egress at build time; drop this stage (and the
# COPY --from=mirror below) for a non-hermetic image.
FROM docker.io/library/alpine:3.22 AS mirror
RUN apk add --no-cache ca-certificates
COPY --from=tofu /usr/local/bin/tofu /usr/local/bin/tofu
WORKDIR /src
COPY . /src
# get: `providers mirror` refuses a module whose nested local modules are not
# installed; get installs them (no providers), in this stage only.
# mkdir: `providers mirror` does not create the target when the module
# requires no providers, and the COPY --from=mirror below needs it.
RUN tofu get \
 && mkdir -p /captf/providers \
 && tofu providers mirror -platform=linux_amd64 -platform=linux_arm64 /captf/providers

FROM docker.io/library/alpine:3.22
ARG ROLE=machine
ARG RUNTIME_VERSION
ARG IMAGE_SOURCE
ARG IMAGE_REVISION
ARG IMAGE_VERSION
RUN apk add --no-cache ca-certificates \
 && adduser -D -u 65532 captf
COPY --from=tofu /usr/local/bin/tofu /captf/runtime
COPY --from=mirror /captf/providers /captf/providers
COPY . /captf/module
RUN chown -R 65532:65532 /captf
USER 65532
LABEL io.captf.contract="v1alpha1" \
      io.captf.role="${ROLE}" \
      io.captf.runtime="tofu" \
      io.captf.runtime.version="${RUNTIME_VERSION}" \
      org.opencontainers.image.source="${IMAGE_SOURCE}" \
      org.opencontainers.image.revision="${IMAGE_REVISION}" \
      org.opencontainers.image.version="${IMAGE_VERSION}"

A distroless/static:nonroot final stage also works, and needs no shell, if the module needs no other tool: tofu is statically linked.

Checklist for tfcapi-lint image

Summarized: /captf/module present and lints clean for the role; /captf/runtime present and executable; /captf/providers, if present, follows the mirror layout and covers every required_providers entry for the platform being checked; labels, if present, agree with --role/--contract; the capacity labels, if present, are valid JSON of the shapes above; no files under the reserved paths; config.User is non-root (running as root is a warning, not an error). See tfcapi-lint CLI: checks for every check’s ID, severity and the roles it applies to.

See also

• Runtime Environment
• Module Contract
• tfcapi-lint
• Security Model

Runtime Environment

This page describes what a module sees once CAPTF actually runs it: the working directory, the environment, the provider mirror, the commands the runner runs and in what order, and how a failure is reported. It complements Image Contract, which is the static shape of the image itself.

For every operation, the runner replaces the image’s ENTRYPOINT/CMD with its own binary, checks the image’s layout (below), prepares a working directory, then runs /captf/runtime (the image’s tofu or terraform) as a child process, one command at a time, streaming its output to the Job’s log.

Working directory and the generated root

Before running anything, the runner checks that /captf/module holds at least one .tf, .tf.json, .tofu or .tofu.json file at its top level and that /captf/runtime is executable; either failure stops the Job with error.kind: image-layout before any command runs.

It then builds a generated root at /captf/work/root from the files mounted read-only at /captf/config (the per-run Secret): main.tf.json, which declares a partial kubernetes backend (init’s -backend-config= flags complete it) and calls the image’s module as module "role" { source = "../../module" } (resolving to /captf/module, a local path, so init never fetches the module from anywhere), and terraform.tfvars.json, the rendered contract and user variables. A restore’s root is the backend block alone: it calls no module and declares no variable, since state push and state list need neither. Every command in this page runs with /captf/work/root as its working directory; the runner never uses -chdir. What the module actually receives through these two files — the contract inputs, user variables and where each value comes from — is on Job Inputs.

The runner never invokes a shell: it execs /captf/runtime directly. A module’s local-exec provisioner needs a shell in the image, or its own interpreter, to run at all: the reference images on Image Contract ship one, but a distroless/static final stage does not. The image’s root filesystem is read-only by default (readOnlyRootFilesystem: true); the only writable paths are /captf/work (this generated root, the plan files and TF_DATA_DIR) and /tmp. A provider that writes anywhere else needs the module or the image to relocate it, typically through an environment variable such as a cache directory setting.

Environment set and dropped

The runner starts from the container’s own environment (the identity’s envFrom, the image’s ENV, and spec.jobs.env) and rewrites it before running any command: HOME is always forced to /captf/work and TF_DATA_DIR to /captf/work/.terraform (note that the working directory of every command is /captf/work/root, one level below); TMPDIR is kept if already set (the Job sets it to /tmp) and otherwise defaults to /captf/work/tmp; and every other TF_* and KUBE_* variable is dropped — logged by name, never by value — except the three the Job itself sets (TF_IN_AUTOMATION, TF_INPUT, KUBE_NAMESPACE). This stops a leaked TF_WORKSPACE, TF_CLI_ARGS_*, TF_VAR_*, TF_LOG or KUBE_* value from moving state, rewriting a step’s flags, overriding a rendered input, or logging provider traffic and credentials. See Job Environment for the full variable list, a worked example of what is kept and dropped, and the names spec.jobs.env may not set at all.

There is no way to turn on provider debug logging in a Job. TF_LOG falls under the dropped TF_* names above; spec.jobs.env cannot set it either, since any name starting with TF_ or KUBE_ is rejected outright (with an event) rather than passed through. A module author debugging a provider needs to reproduce the run outside a Job: pull the pinned image, and run its /captf/runtime binary by hand against a copy of the rendered root and inputs, with TF_LOG set in that shell. The stuck-destroy runbook walks through getting those inputs (the durable inputs Secret’s main.tf.json/terraform.tfvars.json, the image digest, and the identity’s credentials) and invoking the image’s binary directly; the same recipe works for a debug run, add -e TF_LOG=DEBUG (or TRACE) to the docker run/podman run invocation.

Provider mirror and CLI configuration

When the image ships /captf/providers, the runner writes /captf/work/cli.tfrc:

provider_installation {
  filesystem_mirror {
    path    = "/captf/providers"
    include = ["*/*/*"]
  }
  direct {
    exclude = ["*/*/*"]
  }
}

and sets TF_CLI_CONFIG_FILE to that path, so init installs every provider from the image and never contacts a registry; init fails clearly instead of silently downloading a provider missing from the mirror. Without /captf/providers, the runner sets neither, and init falls back to the runtime’s default direct installation, which needs registry egress. TF_PLUGIN_CACHE_DIR is never set: it must not coincide with a filesystem mirror.

A network mirror or a private registry cannot be configured at run time. TF_CLI_CONFIG_FILE is runner-owned, written fresh for every Job as shown above (or left unset), and spec.jobs.env cannot override it or any other TF_* name (see “Environment set and dropped”); there is no field that lets an object or its defaults point init at a network_mirror block or a private registry host’s credentials. The supported path is a filesystem mirror baked into the image at /captf/providers (see Image Contract): build it from whatever upstream, mirror or private registry the image’s build pipeline can reach, and ship the result. There is no run-time equivalent.

Commands, and their order

version -json runs first, once per Job, to record the runtime version for the result document; its outcome is informational and never fails the Job. init, plan, apply and destroy (including -refresh-only) carry -input=false -no-color -lock-timeout=<n>s (init also carries a -backend-config= flag for each value the generated root’s partial kubernetes backend needs); validate and show carry -json -no-color instead; state push carries only -force -lock-timeout=<n>s; and state list and force-unlock take neither. Every command runs against the generated root in order, and a non-zero exit (outside the codes a step accepts, such as plan’s 2 for “changes present”) stops the Job at that step. When the Job carries a stale lock ID to clear, force-unlock -force <id> runs immediately after init (force-unlock needs an initialized backend); it accepts an already-unlocked state as success, since a retried pod or another holder may have cleared it already.

<tfvars> is terraform.tfvars.json, in the generated root. An apply whose plan exits 0 (no changes at all, not even to outputs) skips its apply step entirely: the Job ends successfully without applying anything. show -json’s output is never written to the Job’s log, since a plan document carries every input value; every other command’s output streams to the log as it runs. validate’s JSON diagnostics are the exception: they are captured for the failure summary below and still written to the log, since they carry no input values. state list’s output (resource addresses only) is logged too. Approving a destructive plan or a Manual-policy preview is covered in Plan Approval; the counts, add/change/ destroy resource lists and the drift check itself are covered in Drift and Health.

Lock timeout and stop timeout

init, plan, apply, destroy and state push all carry -lock-timeout, defaulting to 300 seconds and overridable per object with spec.jobs.lockTimeoutSeconds (Tuning Jobs). On SIGTERM — a deletion, a drain, or the Job’s activeDeadlineSeconds (default 3600 seconds) — the runner sends /captf/runtime SIGTERM, never its provider plugins, and gives it up to 570 seconds (the pod’s 600-second termination grace period, less a margin the runner needs to write the result) to finish in-flight provider calls, write state and release the backend lock before SIGKILL. A run stopped this way reports error.kind: interrupted, not a module failure.

What a failed step reports

A run’s status.lastRun.error never carries raw process output. Its kind is one of image-layout, step, interrupted, blocked (a guarded apply stopped before a plan that deletes or replaces a resource) or plan-changed (an approved apply whose new plan no longer matches the approved hash); step names the command that failed, when there is one. A blocked or plan-changed result changes nothing. A plan-changed result also carries a plan summary in status.plan; a blocked result’s summary appears in the object’s condition message instead. Both are covered in Plan Approval.

The failure summary itself is built from the failing step’s own output, never a raw stderr dump: for validate, from its -json diagnostics; for every other step, from the Error: diagnostic header lines in its stderr (ANSI escape codes stripped first, since a provider or a local-exec child is not bound by -no-color). When neither yields a line, it falls back to step <name> exited <code>; see the Job's logs. The summary is capped at 512 bytes; the process exit code is the failing step’s own code, or 1 when the runner itself could not start or was killed. The full stderr always reaches the Job’s own pod log, uncapped, whether or not it contributed to the summary.

Output size limits

See also

• Image Contract
• Job Environment
• Job Inputs
• Plan Approval
• Tuning Jobs

tfcapi-lint

tfcapi-lint checks a Terraform or OpenTofu module, and the OCI image built from it, against the CAPTF module contract (module contract) and the image contract, without running init, plan or apply. It is for module authors, and for CI pipelines that build module images. It is released alongside the provider, with the same version.

Install

Every release attaches one binary per platform, plus a checksum file:

base="https://github.com/captf-io/cluster-api-provider-terraform/releases/download/<version>"
curl -fsSLO "${base}/tfcapi-lint-<os>-<arch>"
curl -fsSLO "${base}/tfcapi-lint-checksums.txt"
sha256sum --check --ignore-missing tfcapi-lint-checksums.txt
install -m 0755 "tfcapi-lint-<os>-<arch>" /usr/local/bin/tfcapi-lint
tfcapi-lint version

• <version> is the provider release you deploy, for example v0.1.0.
• <os> and <arch> pick one row of the table above, for example linux and amd64. On macOS, verify the checksum with shasum -a 256 -c --ignore-missing tfcapi-lint-checksums.txt instead.

Match the tfcapi-lint release to the controller you deploy against: tfcapi-lint version --json reports the contract versions it lints against, ["v1alpha1"].

go install .../cmd/tfcapi-lint@<version> does not work: the repository is a Go workspace, and cmd/tfcapi-lint depends on the api module, which only the workspace resolves. Build from source instead (below), or use a release binary.

Roles

Every module implements exactly one role, and --role on module and image is required and takes one of cluster, machine or machinepool, matching the TerraformCluster, TerraformMachine and TerraformMachinePool contracts. tfcapi-lint checks the module or image against that role’s inputs, outputs and checks only; see tfcapi-lint CLI for which checks apply to which roles.

Lint a module

tfcapi-lint module --role machine ./machine

This reads the .tf, .tf.json, .tofu and .tofu.json files under ./machine directly; it needs neither terraform nor tofu installed, and never contacts a registry. A clean module prints an empty finding list and an all-zero summary.

Lint an image

tfcapi-lint image --role machine registry.example.com/acme/machine:v1.0.0

This pulls the image manifest and its layers, and checks the fixed paths and labels the image contract requires, without running the image. <image-ref> is a registry reference; oci:<dir> reads a local OCI image layout instead, such as one written by podman save --format oci-dir or skopeo copy ... oci:<dir>, with no registry or daemon involved. By default it checks the linux/amd64 platform of a multi-platform image; --platform os/arch picks a different one, and --all-platforms checks every platform the image publishes.

Registry credentials

tfcapi-lint image authenticates the same way docker and podman do, through go-containerregistry’s default keychain: it reads ~/.docker/config.json, or $DOCKER_CONFIG/config.json when that variable is set; if neither exists, it falls back to a Podman-style config at $REGISTRY_AUTH_FILE or $XDG_RUNTIME_DIR/containers/auth.json. With none of those present, the pull is anonymous. Log in with docker login or podman login against the registry before linting a private image; --insecure allows a plain-HTTP registry for a local or air-gapped registry that has none.

Strict mode and allowed warnings

--strict treats a warning the same as an error for the exit code, so a module or image that is merely clean today does not silently pick up new warnings later. --allow-warning <id> (repeatable, or a comma-separated list) downgrades one check ID’s warnings to informational findings; it never touches errors. Use it for a deliberate, reviewable exception, for example a module whose provider cannot tag anything: --allow-warning input/tags-unused. Run with --strict by default, and add --allow-warning only for checks you have decided not to act on.

--json prints a report with a findings array (id, severity, file, line, message) and a summary, instead of one line of text per finding. See tfcapi-lint CLI for every flag, and tfcapi-lint CLI: checks for every check ID, its severity and the roles it applies to.

Exit codes

tfcapi-lint uses its exit code to signal a CI step’s pass or fail; see tfcapi-lint CLI: exit codes for the full list. In short: 0 is clean, 1 is at least one error (or, under --strict, at least one warning), 2 means the module could not be parsed or the image could not be pulled, and 3 is a usage error.

In CI

Lint the module before building the image, then lint the built image before pushing it: the source is checked before the build, and the built layout after it.

tfcapi-lint module --role machine --strict ./module
podman build -t "$IMAGE" .
podman save --format oci-dir -o "$RUNNER_TEMP/image" "$IMAGE"
tfcapi-lint image --role machine --strict "oci:$RUNNER_TEMP/image"
podman push "$IMAGE"

To check exactly what was pushed, for example a multi-platform index built and pushed by a separate step, lint the pushed reference instead of the local layout:

tfcapi-lint image --role machine --strict --all-platforms "$IMAGE"

Building from source

make release-lint-snapshot builds all five release assets and the checksum file into dist/ with GoReleaser, stamped with a snapshot version; make release-lint does the same from the current git tag. See Releasing for how these assets reach a GitHub release.

See also

• tfcapi-lint CLI for every flag, every check ID and the exit codes.
• Module contract and image contract for what the checks enforce.
• Runtime environment for what a module sees once CAPTF actually runs it.

Control-Plane Integration

CAPTF’s cluster and machine modules are infrastructure only: they create networks, load balancers and instances, and report state back through the contract. Bringing up Kubernetes on top of that infrastructure is the job of a Cluster API control-plane provider — KubeadmControlPlane (KCP) or RKE2ControlPlane (RCP) — plus its bootstrap provider.

Those providers read specific fields from your modules and write specific fields back, on a specific schedule. Getting the details wrong produces a cluster that hangs rather than one that fails loudly.

This guide set distills the verified requirements into what you need as a module author:

• kubeadm.md — everything specific to KubeadmControlPlane / the kubeadm bootstrap provider (CABPK).
• rke2.md — everything specific to RKE2ControlPlane / the RKE2 bootstrap provider (CAPRKE2).
• checklist.md — every port, health check and ordering rule from both guides, organized by topic, for a build-time or review-time reference.

Both guides cite CAPI/CAPRKE2 source paths and the contract’s own module role pages: the cluster role, the machine role and the common contract. Where this guide set and the contract state the same requirement, the contract’s wording is authoritative.

The shared creation sequence

Both control-plane providers drive the same shape of sequence against a CAPTF cluster module and one or more CAPTF machine modules. The sequence below is the kubeadm case; RKE2’s differences are called out inline and detailed in rke2.md, in what it reads from your modules and lifecycle constraints.

1. A user (or a ClusterClass topology) creates the Cluster, TerraformCluster, control-plane object (KubeadmControlPlane or RKE2ControlPlane) and TerraformMachineTemplate.
2. The CAPI Cluster controller reconciles the infrastructureRef, setting its owner reference to the TerraformCluster.
3. The TerraformCluster controller runs the cluster module’s first apply. At this point control_plane_endpoint is rendered null and control_plane_initialized is rendered false. The module returns control_plane_endpoint, failure_domains, exports and health.
4. The controller copies control_plane_endpoint onto TerraformCluster.spec.controlPlaneEndpoint and failure_domains onto TerraformCluster.status.failureDomains; CAPI in turn copies them onto Cluster.spec.controlPlaneEndpoint and Cluster.status.initialization.infrastructureProvisioned = true.
5. The control-plane provider gates all further work on: a valid Cluster.spec.controlPlaneEndpoint and infrastructureProvisioned = true. Neither KCP nor RCP creates a single Machine before both are true (see what KubeadmControlPlane reads and what RKE2ControlPlane reads).
6. For control-plane machine i = 1..N, in strict order — the provider creates machine i+1 only after machine i has a nodeRef:
   1. The control-plane provider creates the bootstrap config (init for i=1, join for i>1) and the Machine (labeled cluster.x-k8s.io/control-plane) plus a TerraformMachine cloned from the template.
   2. The bootstrap provider writes the bootstrap Secret (value, format); Machine.spec.bootstrap.dataSecretName is set.
   3. The TerraformMachine controller reconciles machine i, gated on: the cluster’s infrastructureProvisioned, the cluster’s exports being readable, and the bootstrap Secret existing.
   4. The machine module’s apply runs once (immutable): it receives bootstrap_data (base64), control_plane = true, captf_cluster_outputs (the cluster’s exports), failure_domain and kubernetes_version. In this same apply the module creates the instance and registers it in the control-plane LB backend, or does neither of the latter under Pattern C below. It returns provider_id, addresses, failure_domain, interruptible and health.
   5. The TerraformMachine controller writes spec.providerID and status.addresses on the TerraformMachine, and marks it provisioned and Ready. The CAPI Machine controller then copies both onto Machine.spec.providerID and Machine.status.addresses.
   6. cloud-init on the instance runs kubeadm init (i=1) or kubeadm join (i>1); post-init phases go through the endpoint (hairpin — see kubeadm.md’s networking section). Under RKE2’s default registration method the join target is the endpoint host itself, but the join cannot proceed until at least one Ready control-plane Machine already exists, since RCP.status.availableServerIPs stays empty without one (see what RKE2ControlPlane reads and rke2.md’s lifecycle constraints).
   7. The Node registers with spec.providerID set (by a cloud-controller-manager or kubelet --provider-id); the Machine controller matches nodeRef by an exact providerID match.
   8. When i = 1 and the control plane finishes initializing, Cluster.status.initialization.controlPlaneInitialized flips true. This triggers exactly one cluster re-apply, for resources gated on a live workload API server.
7. Workers follow the same machine path (MachineDeployment → MachineSet → Machine → TerraformMachine, control_plane = false), gated on ControlPlaneInitialized because the bootstrap provider writes worker join data only after that. A worker fleet backed by a MachinePool instead goes straight from MachinePool to TerraformMachinePool — one infrastructure object for the whole group, with no per-replica Machine or TerraformMachine; see Machine Pools.

LB-membership patterns

The contract requires a control-plane machine module to register (and, on destroy, deregister) its instance with the control-plane load balancer (see machine.md’s control-plane machines), but leaves how up to the module. Three patterns are sanctioned here.

Pattern A: attachment resource in the machine module’s state

The machine module creates the instance and, in the same apply, an explicit attachment/registration resource (for example an LB target-group attachment) pointed at the target-group or backend-pool id it received through captf_cluster_outputs (see common.md’s outputs). Because the attachment lives in the machine module’s own state, destroy deregisters it as an ordinary part of tearing that state down.

This is the pattern the contract documents directly: registration is “done in the machine module’s own Terraform state, not the cluster module’s” (machine.md’s control-plane machines, citing capi/docs/book/src/developer/providers/contracts/infra-machine.md:633,645). The instance MUST be registered before kubeadm init/kubeadm join finishes (KCP) or before the Machine is Ready (RKE2) — see kubeadm.md’s module checklist and rke2.md’s module checklist.

Typical fit: clouds whose load balancer exposes an explicit target-group/backend-pool attach API (for example AWS ALB/NLB target-group attachments, Azure Load Balancer backend-pool membership).

This is design guidance, not yet tested against a real control plane.

Pattern B: selector/tag-based backend pools

The load balancer derives its backend membership itself, from a tag or label selector query, rather than from an explicit per-instance attach call — “target group by tag/label”. The machine module’s only job is to apply the right tags to the instance (via captf_tags, see common.md’s inputs, or an additional module-defined tag); the load balancer’s own membership scan does the rest, and removing the instance (destroy) removes it from the pool.

The same ordering requirement applies as under Pattern A: the instance MUST be a member of the backend before kubeadm init/join finishes on it (KCP) or before the Machine is Ready (RKE2) — see kubeadm.md’s module checklist and rke2.md’s module checklist.

Typical fit: clouds or load balancers whose backend pool is defined by a selector/tag query against an instance group or autoscaling group, rather than an explicit attach call.

This is design guidance, not yet tested against a real control plane.

Pattern C: kube-vip-style VIP

No separate load-balancer resource is created by the cluster module at all; a virtual IP is run by the control-plane nodes themselves (implicit via kube-vip or similar), so there is “nothing LB-wise” for the machine module to do. The cluster module still MUST emit a valid control_plane_endpoint before any Machine is created — the same gate applies regardless of how the endpoint is realized (see what KubeadmControlPlane reads and kubeadm.md’s networking section; what RKE2ControlPlane reads and rke2.md’s networking section). Under this pattern the backend-membership and per-backend health-check rows of checklist.md do not apply, because there is no separate LB backend to join.

Typical fit: bare-metal, on-premises, or otherwise L2-reachable environments without a managed load balancer in front of the control plane.

This is design guidance, not yet tested against a real control plane.

See also

• The module contract — the normative cluster and machine roles this guide set builds on.
• Security model — the trust boundary a control-plane Job runs inside.
• The Kinds — how TerraformCluster and TerraformMachine map to Cluster API objects.

KubeadmControlPlane

What KubeadmControlPlane (KCP) and the kubeadm bootstrap provider (CABPK) require of your CAPTF cluster and machine modules, and why this is the case.

Citations below prefixed capi/ are files in the CAPI v1.14.2 tree (https://github.com/kubernetes-sigs/cluster-api/tree/v1.14.2). See also the shared creation sequence and the requirements checklist for every requirement in one place.

What KubeadmControlPlane reads from your modules

• Cluster.spec.controlPlaneEndpoint. KCP has no endpoint field of its own; it only reads the Cluster’s. It gates all Machine creation on this field being valid (host != "" and port != 0) — see networking and load balancer. The endpoint can come from the user, your TerraformCluster (copied once while the Cluster field is not yet valid), or a control-plane provider; KCP never supplies one itself (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/kubeadmcontrolplane_controller.go:316,439; capi/core/reconcilers/cluster/cluster_controller_phases.go:210-238).
• Machine.status.addresses. Not read by KCP or CABPK at all. Only the core Machine controller copies it from the InfraMachine; apiserver certificate SANs come from kubeadm’s own node-IP detection, not from CAPI addresses (capi/core/reconcilers/machine/machine_controller_phases.go:339-346).
• providerID / nodeRef. The Machine controller links a Node to a Machine only on an exact providerID match (see ../contract/v1alpha1/machine.md “Node providerID matching”). KCP then gates almost everything on the resulting nodeRef: etcd-member matching, static-pod health, and scale-up/scale-down preflight all require every existing control-plane Machine to have one (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/preflight.go:193-203; capi/controlplane/kubeadm/pkg/workload_cluster_conditions.go:136,390,434).
• Failure domains. KCP only sees Cluster.status.failureDomains entries with controlPlane == true; a nil controlPlane counts as false. This is why the cluster module’s failure_domains[].control_plane default of true matters (capi/controlplane/kubeadm/pkg/control_plane.go:171-184).
• cluster_network.api_server_port. Not read by CABPK or KCP at all in CAPI v1.14.2. The real kube-apiserver bind port is the KubeadmConfig localAPIEndpoint.bindPort (default 6443) (capi/controlplane/kubeadm/pkg/workload_cluster.go:281-291). Treat api_server_port ?? 6443 as a documentation-only convention, not a wired-through value (see networking and load balancer).
• InfraMachineTemplate rotation. KCP triggers a rollout when a Machine’s InfraMachine carries a cluster.x-k8s.io/cloned-from-name/ -groupkind annotation different from the current machineTemplate.spec.infrastructureRef. Template content is never diffed — CAPTF’s template immutability already matches this expectation (capi/controlplane/kubeadm/pkg/filters.go:183-227).

What KCP and CABPK write

• Bootstrap Secret. Named after the KubeadmConfig (== the Machine name). Keys value and format are always both written; format defaults to cloud-config (Ignition requires the alpha feature gate KubeadmBootstrapFormatIgnition, off by default) (capi/bootstrap/kubeadm/reconcilers/kubeadmconfig/kubeadmconfig_controller.go:1405-1437; capi/api/bootstrap/kubeadm/v1beta2/kubeadmconfig_types.go:27-36,117-120; capi/feature/feature.go:108). See ../contract/v1alpha1/machine.md bootstrap_format for how CAPTF surfaces this.
• Payload contents. Control-plane init and join payloads embed the cluster CA, etcd CA, service-account and front-proxy key material as files, uncompressed (capi/bootstrap/kubeadm/pkg/cloudinit/controlplane_init.go:63, controlplane_join.go:61). See module checklist for what this means for your module.
• Labels and hooks. Every KCP Machine, InfraMachine and KubeadmConfig gets cluster.x-k8s.io/cluster-name, cluster.x-k8s.io/control-plane: "" and a pre-terminate.delete.hook.machine.cluster.x-k8s.io/kcp-cleanup: "" annotation (capi/controlplane/kubeadm/pkg/desiredstate/desired_state.go:324-340,110-135). CAPTF’s own metadata-update path already allows these (label/annotation edits are excluded from TerraformMachine’s immutability checks).
• KubeadmControlPlane.status.initialization.controlPlaneInitialized. Latches true once the kubeadm-config ConfigMap is readable through the endpoint — the reason the endpoint’s hairpin reachability matters (see networking and load balancer) (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/status.go:176-210). This mirrors into Cluster.status.initialization.controlPlaneInitialized and drives one cluster re-apply (capi/core/reconcilers/cluster/cluster_controller_status.go:80-81,450-551).
• Conditions. KCP writes Initialized, Available, CertificatesAvailable, EtcdClusterHealthy, ControlPlaneComponentsHealthy, MachinesReady, MachinesUpToDate, RollingOut, ScalingUp/ScalingDown, Remediating, Deleting, plus per-Machine pod/etcd-member conditions (capi/api/controlplane/kubeadm/v1beta2/kubeadm_control_plane_types.go:74-421,504-517). None of these read anything from your module beyond what is described above.

Lifecycle constraints

• Init ordering. KCP creates nothing until the Cluster is both infrastructureProvisioned and has a valid endpoint; CABPK waits on the same gate. The first control-plane Machine gets init data; every other Machine (control-plane or worker) waits on Cluster.status.initialization.controlPlaneInitialized (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/kubeadmcontrolplane_controller.go:316,413,439; capi/bootstrap/kubeadm/reconcilers/kubeadmconfig/kubeadmconfig_controller.go:289-298,352-368,466-495).
• Join is serialized. Scale-up preflight requires certificates available, no Machine deleting, and every existing control-plane Machine already healthy with a nodeRef. A failed preflight requeues after 15s, so control-plane joins are serialized on your apply time plus kubeadm join time (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/scale.go:60-97; preflight.go:61-218).
• Rollout. maxSurge is 0 or 1 (default 1): with 1, KCP scales up then down, one Machine at a time; with 0 it scales down first (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/update.go:103-120). Rotation is by creating a new TerraformMachineTemplate name; edits to an existing template’s content are never detected.
• Deleting blocks everything. While any control-plane Machine has a deletion timestamp, KCP pauses scale-up, scale-down, rollout and remediation (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/preflight.go:111-122). A slow or stuck machine-module destroy therefore freezes the whole control plane, not just that Machine.
• Remediation gates and retries. Post-initialization remediation needs more than one replica, no Machine still provisioning without a Node, no Machine deleting, and preserved etcd quorum; one remediation runs at a time (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/remediation.go:138-313). remediation.maxRetry defaults to unlimited and retryPeriodSeconds defaults to 0 (retry immediately) (capi/api/controlplane/kubeadm/v1beta2/kubeadm_control_plane_types.go:631-673). See module checklist for the template defaults this motivates.
• No built-in control-plane MHC. KCP does its own remediation and is not configured through a MachineHealthCheck object. Ship a plain MachineHealthCheck selecting cluster.x-k8s.io/control-plane: "" (or a ClusterClass controlPlane.healthCheck) if you want MHC-driven remediation in addition to KCP’s own (capi/docs/book/src/tasks/automated-machine-management/healthchecking.md:76-99).
• Version skew. KubeadmControlPlane.spec.version must be valid semver with a v prefix, and an update may move at most one minor version (admission webhook, capi/controlplane/kubeadm/webhooks/admission/kubeadmcontrolplane.go:322-328,619-662).
• Cluster deletion. Workers and MachinePools are deleted first; once only control-plane Machines remain, KCP deletes all of them in parallel, with no drain (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/kubeadmcontrolplane_controller.go:720-813). Expect concurrent destroy Jobs on cluster teardown.
• Failed create cleanup. KCP creates the InfraMachine before the Machine object. If KubeadmConfig or Machine creation then fails, KCP deletes the InfraMachine directly, with no Machine owner reference at all (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/helpers.go:160-219). CAPTF’s delete webhook already allows a delete with no Machine owner reference for exactly this reason (see ../contract/v1alpha1/machine.md “Delete”) — nothing further is required of your module.

Networking and load balancer

• Endpoint before any Machine. KCP creates no Machine at all until Cluster.spec.controlPlaneEndpoint.IsValid() (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/kubeadmcontrolplane_controller.go:438-457).
• Frontend port. control_plane_endpoint.port, any value 1-65535.
• Backend port. The kube-apiserver bind port, i.e. KubeadmConfig localAPIEndpoint.bindPort (default 6443). CABPK/KCP never read Cluster.spec.clusterNetwork.apiServerPort, so treat api_server_port ?? 6443 as a convention your templates must keep in sync with bindPort yourselves (capi/controlplane/kubeadm/pkg/workload_cluster.go:281-291).
• Reachability, including hairpin. The endpoint must be reachable from the management cluster, from every control-plane and worker node (join discovery, kubelet), and from the first control-plane Machine itself. This last point is the hairpin requirement: kubeadm’s getKubeConfigSpecsBase sets the server URL of admin.conf, super-admin.conf and kubelet.conf to the control-plane endpoint, not the node’s own local address, so the node that is itself an LB backend must be able to reach the frontend it was just registered behind (see ../contract/v1alpha1/cluster.md “Hairpin reachability”, citing kubeadm’s cmd/kubeadm/app/phases/kubeconfig/kubeconfig.go ~L582-626, kubernetes/kubernetes release-1.33).
• Backend membership. Your machine module MUST register (and on destroy deregister) each control-plane instance in the LB backend from its own state (capi/docs/book/src/developer/providers/contracts/infra-machine.md:633,645; ../contract/v1alpha1/machine.md “Control-plane machines”). The instance MUST be in the backend before kubeadm init/kubeadm join finishes on it, or controlPlaneInitialized never latches (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/status.go:195-207).
• Health check. Either plain TCP, or HTTPS /readyz//healthz without certificate verification, works; the check must be able to go green with a single backend during init. CAPD’s reference LB uses haproxy option httpchk GET /healthz with check-ssl verify none against backend 6443 (capi/test/infrastructure/docker/internal/loadbalancer/config.go:81-83).
• Security groups / firewall. Backend port (bindPort) from the LB, from all nodes, and from the management cluster; TCP 2379-2380 and 10250 between control-plane nodes; 10250 from control-plane nodes to workers. KCP itself reaches etcd only through an apiserver pod port-forward, so the management cluster needs no direct etcd or kubelet port (capi/controlplane/kubeadm/pkg/etcd_client_generator.go:53; capi/controlplane/kubeadm/pkg/proxy/dial.go:72-120).

Module checklist

Cluster module

• MUST output a valid control_plane_endpoint (host and port) no later than the apply that makes the cluster provisioned, when no user endpoint is given — there is no other source with KCP (capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/kubeadmcontrolplane_controller.go:316,439; see what KubeadmControlPlane reads). tfcapi-lint’s output/endpoint-never-set check warns on a module that never emits one (see cluster.md’s EndpointAvailable condition and the tfcapi-lint CLI reference).
• MUST keep the emitted control_plane_endpoint stable for the life of the object: CAPI never updates Cluster.spec.controlPlaneEndpoint after the first valid copy (capi/core/reconcilers/cluster/cluster_controller_phases.go:217-228), so replacing the LB behind it (new DNS name or IP) breaks every kubeconfig and the API server certificate SANs. The controller neither detects nor prevents such a replacement; lifecycle { prevent_destroy = true } on the resource behind the endpoint turns it into a failed Job instead of a lost cluster (../contract/v1alpha1/cluster.md control_plane_endpoint output).
• MUST create a stable LB or VIP reachable from the management cluster, all nodes, and the control-plane nodes themselves (hairpin, see networking and load balancer).
• MUST use a frontend on control_plane_endpoint.port and a backend on api_server_port ?? 6443, kept equal to the KubeadmConfig bindPort your templates configure (capi/controlplane/kubeadm/pkg/workload_cluster.go:281-291; see networking and load balancer).
• SHOULD publish the LB target/backend-pool id in exports for machine modules to register against — a CAPTF convention built on the contract’s exports field, not a CAPI-mandated shape (../contract/v1alpha1/cluster.md exports).
• SHOULD leave failure_domains[].control_plane at its default true (capi/controlplane/kubeadm/pkg/control_plane.go:171-184; see what KubeadmControlPlane reads); a nil value is invisible to KCP.

Machine module

• MUST register the control-plane instance in the LB target/backend-pool from captf_cluster_outputs, in the module’s own state, before kubeadm init/join finishes on it (capi/docs/book/src/developer/providers/contracts/infra-machine.md:633,645; capi/controlplane/kubeadm/reconcilers/kubeadmcontrolplane/status.go:195-207; see networking and load balancer and one of the three patterns in README.md’s LB-membership patterns).
• MUST accept bootstrap_data opaquely as base64 and MUST NOT need to parse it, per ../contract/v1alpha1/machine.md bootstrap_data.
• Control-plane payloads embed cluster CA and service-account private key material and are not compressed by CABPK (see what KCP and CABPK write). Payload size limits (for example a cloud’s user-data cap) and keeping key material out of readable instance metadata are the module’s responsibility: gzip the payload, or stage it in a secret store and pass only a small stub through bootstrap_data/user-data.
• MUST emit provider_id exactly matching the Node’s spec.providerID; KCP gates join preflight, etcd-member matching and remediation on the resulting nodeRef (see what KubeadmControlPlane reads).
• addresses are informational only for KCP — no KCP or CABPK code reads them.
• MUST make failure_domain output equal the failure_domain input when the input is non-null (../contract/v1alpha1/machine.md failure_domain output).

Templates and health checks

• Bring-up time for N control-plane replicas is approximately N × (apply + boot + join), because every join is serialized on the previous Machine’s nodeRef (see lifecycle constraints). A rollout adds (apply + join + destroy) per replica.
• Ship KubeadmControlPlane.spec.remediation.maxRetry (e.g. 3) and a non-zero retryPeriodSeconds in your templates: the unbounded defaults turn a control-plane module whose apply always fails into an endless create/destroy loop of cloud resources (see lifecycle constraints).
• Ship a control-plane MachineHealthCheck (selector cluster.x-k8s.io/control-plane: "", or a ClusterClass controlPlane.healthCheck). Its InfrastructureReady=False timeout MUST exceed apply time plus one drift interval — the same rule as any other machine module (../contract/v1alpha1/machine.md “Health → remediation”).

Open questions

• Bootstrap payload size and secrecy. The contract leaves the gzip/stub strategy to the module (see module checklist); there is no CAPTF-side size or secrecy check today.
• Unbounded remediation retry. Combined with a deterministically failing module, KCP’s default unlimited maxRetry causes cloud-resource churn. The module checklist’s template-default recommendation is the only mitigation; there is no CAPTF-side guard against it.

See also

• README.md — the shared creation sequence and the LB-membership patterns.
• rke2.md — the same requirements under RKE2ControlPlane.
• checklist.md — every requirement from both guides in one place.
• The machine role — the normative contract this guide builds on.

RKE2ControlPlane

What RKE2ControlPlane (RCP) and the RKE2 bootstrap provider (CAPRKE2) require of your CAPTF cluster and machine modules, and why this is the case.

Citations below prefixed rke2/ are paths in rancher/cluster-api-provider-rke2 at tag v0.25.2; rke2docs/ is rancher/rke2-docs (main branch); capi/ is the CAPI v1.14.2 tree (https://github.com/kubernetes-sigs/cluster-api/tree/v1.14.2). See also the shared creation sequence and the requirements checklist for every requirement in one place.

Version and compatibility

CAPRKE2 v0.25.2 is built against sigs.k8s.io/cluster-api v1.13.5 (rke2/go.mod:33) and its v1beta2 API is the contract version CAPTF also targets. Its own e2e suite runs only against CAPI core v1.12.11 and v1.13.5 (rke2/test/e2e/config/e2e_conf.yaml:22,33), and its getting-started guide pins CAPI core to v1.13.5 (rke2/docs/book/src/01_user/01_getting-started.md:56).

CAPTF runs CAPI v1.14.2. The contract shape is unchanged, but CAPI v1.14 core is untested by CAPRKE2 v0.25.2 — run a compatibility smoke test before relying on it in production; see open questions.

What RKE2ControlPlane reads from your modules

• Cluster.status.initialization.infrastructureProvisioned. RCP creates nothing — not even certificates — until this is true; RKE2Config generates no bootstrap data until it is true either (rke2/controlplane/internal/controllers/rke2controlplane_controller.go:376-394; rke2/bootstrap/internal/controllers/rke2config_controller.go:171-182).

• Cluster.spec.controlPlaneEndpoint. A hard gate: RCP creates no Machine at all — including the first — while the endpoint is not valid (host and port both set) (rke2/controlplane/internal/controllers/rke2controlplane_controller.go:421-440). Only the endpoint’s host feeds the RKE2 tls-san list and the init node’s ServerURL (https://<host>:9345); the endpoint’s port is ignored for the supervisor join URL, which is always 9345 (rke2/pkg/rke2/config.go:350; rke2/bootstrap/internal/controllers/rke2config_controller.go:66,455-473).

• Cluster.spec.clusterNetwork. pods/services.cidrBlocks map to cluster-cidr/service-cidr when non-empty (rke2/pkg/rke2/config.go:209-215). serviceDomain and api_server_port are not read at all: cluster domain comes from RKE2’s own serverConfig.clusterDomain, and the API server is always on 6443 on the node (rke2/pkg/rke2/config.go:224-225). Do not derive an LB backend port from api_server_port under RKE2 beyond using it as the frontend port if your templates choose to.

• Failure domains. Same rule as KCP: only entries with controlPlane: true are used for scale-up/scale-down placement; a nil controlPlane counts as false (rke2/pkg/rke2/control_plane.go:176-190).

• Machine.status.addresses. Read only for three of the five registrationMethod values, and only from Machines whose Ready condition is already True (the registrationMethod table below; rke2/controlplane/internal/controllers/status.go:80,152,158-177). The contract’s canonical output order for addresses (InternalIP, InternalDNS, ExternalIP, ExternalDNS, Hostname — see ../contract/v1alpha1/machine.md addresses) is what makes RCP’s first-match logic deterministic; your module’s job is only to emit the address type(s) each method requires (below), not to order them — the controller does that.

• Machine.status.nodeRef. Used for etcd leader-move/member-removal ordering and remediation; HasHealthyMachineStillProvisioning = a healthy Machine with no Node yet (rke2/pkg/rke2/workload_cluster_etcd.go:40-45; rke2/pkg/rke2/control_plane.go:527-529).

• registrationMethod (immutable once set, enforced by webhook):

  Method	status.availableServerIPs =	Address types your module must supply	Join URL
  control-plane-endpoint (default, "" ≡ this)	[endpoint.host]	none	https://<endpoint host>:9345
  address	[spec.registrationAddress]	none	https://<registrationAddress>:9345
  internal-first	first InternalIP or ExternalIP per Ready CP Machine, in list order	InternalIP and/or ExternalIP	https://<ip>:9345
  internal-only-ips	first InternalIP per Ready CP Machine	InternalIP	https://<ip>:9345
  external-only-ips	first ExternalIP per Ready CP Machine	ExternalIP	https://<ip>:9345

  (rke2/pkg/registration/registration.go:46,68-140; rke2/controlplane/api/v1beta2/rke2controlplane_types.go:97-100; rke2/controlplane/api/v1beta2/rke2controlplane_webhook.go:142-145,211-214.) A Ready Machine with no address of the required type produces the RCP status error “ready but they have no IP Address available” (rke2/controlplane/internal/controllers/status.go:175-177); see open questions.

• RKE2ControlPlane.spec.version. Must match (v\d\.\d{2}\.\d+\+rke2r\d)|^$ and is copied verbatim to Machine.spec.version (rke2/controlplane/api/v1beta2/rke2controlplane_types.go:79-81; rke2/controlplane/internal/controllers/scale.go:566-567,612). See ../contract/v1alpha1/machine.md kubernetes_version and the module checklist.

What RCP and RKE2Config write

• Bootstrap Secret. Name = RKE2Config name; keys value and format are always both written (there is no case where format is absent); format defaults to cloud-config (rke2/bootstrap/internal/controllers/rke2config_controller.go:1040-1061; rke2/bootstrap/api/v1beta2/rke2config_webhook.go:79-80). See ../contract/v1alpha1/machine.md bootstrap_format.
• Ignition is supported end to end (init, CP join, worker), via Butane → Ignition 3.3 (rke2/bootstrap/internal/controllers/rke2config_controller.go:552-556,798-802,927-931).
• gzipUserData: true. For cloud-config, value becomes raw gzip bytes (not base64, not UTF-8 text); format stays cloud-config. For Ignition, the gzip is wrapped inside the Ignition config instead (rke2/bootstrap/internal/controllers/rke2config_controller.go:1015-1037). See machine.md’s bootstrap_data input for how the contract resolves the binary-payload question, and the module checklist for what it means for your module.
• Cluster Secrets. <cluster>-ca, -cca (client CA), -peer-etcd, -etcd, -kubeconfig, -token. No -sa, no -proxy (contrast with KCP’s secret set, kubeadm.md’s what KCP and CABPK write) (rke2/pkg/secret/certificates.go:52-80,168-190,382-384).
• Machine labels/annotations. Template labels plus forced cluster.x-k8s.io/cluster-name, cluster.x-k8s.io/control-plane: ""; a pre-terminate.delete.hook.machine.cluster.x-k8s.io/rke2-cleanup annotation on every control-plane Machine (rke2/controlplane/internal/controllers/scale.go:650-666,577-578).
• RCP status (v1beta2). initialization.controlPlaneInitialized (true once the workload kube-system/rke2-serving Secret exists, or once every owned Machine is Ready); availableServerIPs; conditions including EtcdClusterHealthy, ControlPlaneComponentsHealthy, Remediating (rke2/controlplane/internal/controllers/status.go:98-194; rke2/controlplane/api/v1beta2/rke2controlplane_types.go:262-316).
• Node identity. CAPRKE2 never sets kubelet provider-id, node-ip or node-name itself; the fields exist in its config but nothing in the repo writes them (rke2/pkg/rke2/config.go:501-503). See the module checklist and machine.md’s node providerID matching — RKE2.