# Multi-Tenancy

CAPTF can serve several tenants from one management cluster. The unit of isolation is the **namespace**: one tenant, one identity and one workload cluster per namespace. This page explains what that isolation is made of, where it is thin, and how to lay out a tenant. It builds on [Identities and Credentials](<https://captf.io/docs/user-guide/identities/index.md>), [RBAC](<https://captf.io/docs/operator-guide/rbac/index.md>) and the [Security Model](<https://captf.io/docs/concepts/security-model/index.md>), and links to them instead of repeating them.

## The model

A tenant is a namespace that holds that tenant’s Cluster API and `Terraform*` objects, their Secrets, and the Jobs CAPTF runs for them. What a tenant can do is decided by three things:

| Layer | Controls | Where |
| --- | --- | --- |
| **Kubernetes RBAC** | Who may create or edit `Terraform*` objects, approve plans and read Secrets | Roles in the tenant namespace |
| **The identity** | Which cloud credentials a namespace may use | `TerraformClusterIdentity.spec.allowedNamespaces` |
| **The namespace itself** | What a Job can read, how much it can use, where it can connect | Pod Security labels, quotas, network policy |

> [!WARNING]
>
> **A module runs with the namespace’s Secret access and the identity’s credentials**
>
> The point to hold on to is that **a module is code that runs with the namespace’s Secret access and the identity’s cloud credentials**. Whoever controls `spec.source.image` of an object, or can edit one, controls both. Everything below follows from that.

## Identities and `allowedNamespaces`

A `TerraformClusterIdentity` is cluster-scoped. It names a credentials Secret (`spec.secretRef`, in any namespace) and says which namespaces may use it.

| `allowedNamespaces` | Meaning |
| --- | --- |
| unset | No namespace may use it |
| `list` | Exactly the named namespaces |
| `selector` | Namespaces whose labels match |
| `selector: {}` | **Every** namespace, including ones not yet created |
| `list` and `selector` | The union |
| `{}` | Rejected: it would read as “no one” or “everyone” |

Protections the admission webhook adds:

- **Creating** an identity, **changing** `spec.secretRef` and **widening** `allowedNamespaces` each run a `SubjectAccessReview`: the requester must be allowed to `get` the named Secret. A role that may manage identities but cannot read the Secret cannot point an identity at it, or grant more namespaces access to it. Narrowing is not re-checked.
- **Deleting** an identity is refused while any object resolves to it or any namespace still holds its mirror.

Practical rules:

- Give each tenant its **own identity**, with credentials scoped to that tenant’s cloud account or role, and allow only that tenant’s namespace. Never use `selector: {}` for an identity with real credentials.
- Keep the source Secret in a **platform namespace** tenants cannot read. The tenant’s namespace holds only the mirror.
- Do not give tenants `create`, `update` or `patch` on `terraformclusteridentities`. The `SubjectAccessReview` limits what a requester can point an identity at, but identities are a platform object.

### The mirror per namespace

For each identity a namespace uses, the manager keeps one Secret, `captf-creds-<identity>`, in that namespace: a copy of the source Secret. All the namespace’s objects that use the identity share it, and each is an owner of it. The Job mounts it into the module’s container as environment variables and read-only files.

- **Revocation.** Drop a namespace from `allowedNamespaces` and the manager deletes the mirror there, whatever still references it, and starts no new Job for the namespace’s objects. A destroy waits, with `IdentityNotAllowed`, until access returns, or the object is abandoned. See [Revoke access](<https://captf.io/docs/user-guide/identities/#revoke-access>).
- **The mirror is a Secret the tenant’s workloads can read.** Anyone with `get` on Secrets in the namespace can read the credentials. Scope the credentials, not just the access.
- **Rotation is not instant.** A change to the source reaches the mirror on the manager’s next reconcile. See [Rotate credentials](<https://captf.io/docs/user-guide/identities/#rotate-credentials>).

## The runner reads every Secret in its namespace

> [!CAUTION]
>
> **Any module image can read, replace or delete every Secret in its namespace**
>
> The Job runs as the `captf-runner` ServiceAccount, bound by a RoleBinding in each namespace to a ClusterRole with `get`, `list`, `create`, `update` and `delete` on Secrets, unscoped by name. The Kubernetes state backend needs that: it lists and writes its own chunks, and RBAC cannot name Secrets that do not exist yet. The ServiceAccount token is mounted in the Job pod, and the state backend uses it. As a result, **any module image can read, replace or delete every Secret in its namespace**: other objects’ state and inputs, the credential mirrors, and Cluster API Secrets such as a cluster’s kubeconfig and CA. See [What the runner can read, and why](<https://captf.io/docs/concepts/security-model/#what-the-runner-can-read-and-why>).

Consequences for a platform team:

- **Do not put two trust domains in one namespace.** Two tenants, or a trusted platform cluster and an untrusted module author, in one namespace give each the other’s state and credentials.
- **An untrusted or experimental module gets its own namespace**, with its own identity whose credentials can do no more than that experiment needs.
- **The manager itself reads Secrets cluster-wide**, as any provider that uses the Kubernetes state backend does. Protect the manager’s namespace and its image supply chain accordingly.

### Override ServiceAccounts

`spec.jobs.serviceAccountName` can name another ServiceAccount for a Job. It must exist and carry the label `captf.io/runner=true`, or the object reports `RunnerRBACReady=False`/`ServiceAccountNotOptedIn` and runs nothing. The label is consent: whoever may label a ServiceAccount opts it in. The manager then adds it to the namespace’s `captf-runner` RoleBinding, and prunes subjects no object uses. This is for a namespace administrator who needs a Job to run as a ServiceAccount with extra cloud-side permissions (workload identity), not a way to narrow the runner: the binding gives the Secret rights to every subject. See [Custom ServiceAccounts](<https://captf.io/docs/operator-guide/rbac/#custom-serviceaccounts-and-the-runner-opt-in>).

## Who writes specs and who approves

Approval is Kubernetes RBAC, by design: the approval annotations (`captf.io/approve-plan`, `captf.io/approve-destructive-plan`) and the manual-action annotations (`captf.io/restore-state`, `captf.io/abandon-infrastructure`) are authorized by who may `patch` the object, and the webhooks do not restrict them. RBAC cannot separate a patch that sets an annotation from one that edits the spec, so whoever holds `patch` on a `TerraformCluster` can do both.

The pattern for a tenant, from [Operating the gates](<https://captf.io/docs/concepts/approvals/operating/#who-can-approve>):

- **Approvers** hold `patch` on `terraformclusters` in the tenant namespace.
- **Spec changes** come from a pipeline’s ServiceAccount, through a reviewed change (GitOps). People hold read-only access otherwise.
- **Machines and pools are never gated**, so the split protects only the cluster module. See [Limits](<https://captf.io/docs/concepts/approvals/limits/index.md>).
- An optional admission policy that guards the annotations is outside CAPTF; see that page.

Which fields a spec writer can change also differs by kind. A `TerraformMachine`’s source, identity and variables are immutable; its `jobs`, `drift` and `remediation` policy is not. A `TerraformCluster` and a `TerraformMachinePool` are fully mutable by whoever has `update`.

## Quotas, limits and network policy

- **Pod Security.** The Job pod meets `baseline` by default. Label each tenant namespace with the level you enforce. For `restricted`, the module image must run as non-root. See [Pod security](<https://captf.io/docs/concepts/security-model/#pod-security>).
- **ResourceQuota.** A tenant’s quota has to allow what CAPTF creates: Jobs and pods (finished Jobs are retained until pruned, 3 successful and 3 failed per op by default), Secrets (the state and its chunks, up to five backups of each, the durable inputs, the plan key, the mirror, and a per-run Secret per running Job) and Leases (a run lease per object, a write lease per Cluster and a state lock per object). The default Job pod requests 250m CPU and 512Mi, sets a 2Gi memory limit and **no CPU limit**. If the quota covers `limits.cpu`, add a `LimitRange` that defaults a CPU limit, or pods without one are refused (Kubernetes behavior). CAPTF has no quota-specific condition: a refusal appears as a reconcile error or a Job that never starts. See [Production Readiness](<https://captf.io/docs/operator-guide/production-readiness/#runner-jobs>).
- **Network policy.** The manager’s policy does not cover Job pods. Copy `job-egress-sample.yaml` into each tenant namespace and add the cloud and registry egress that tenant’s modules need. Egress is the control that limits where the identity’s credentials can be sent. See [Network exposure](<https://captf.io/docs/concepts/security-model/#network-exposure>).
- **Several managers.** One manager watches one namespace or all of them, never a chosen set, and the leader-election lease name is fixed. To give groups of tenants their own manager instance, use `--watch-filter` and label the objects, and read [Namespace scoping and `--watch-filter`](<https://captf.io/docs/operator-guide/configuration/#namespace-scoping-and---watch-filter>) first: the webhook still admits every namespace, and an object no instance watches is never reconciled.

## An example tenant

The platform team creates, per tenant `acme`:

1. A **namespace** `tenant-acme`, labeled for Pod Security (`pod-security.kubernetes.io/enforce: baseline`).
2. A **credentials Secret** `acme-cloud` in the platform namespace `captf-credentials`, holding credentials scoped to Acme’s cloud account.
3. An **identity**:

   ```yaml
   apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
   kind: TerraformClusterIdentity
   metadata:
     name: acme
   spec:
     secretRef:
       name: acme-cloud
       namespace: captf-credentials
     allowedNamespaces:
       list:
         - tenant-acme
   ```
4. A **ResourceQuota** and **LimitRange** sized as above, and a **NetworkPolicy** for Job pods from the sample.
5. **Roles** in `tenant-acme`. The pipeline’s ServiceAccount authors clusters and machines; approvers are a smaller group with `patch` on `terraformclusters`. This is the pipeline’s Role:

   ```yaml
   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: captf-spec-pipeline
     namespace: tenant-acme
   rules:
   - apiGroups: ["infrastructure.cluster.x-k8s.io"]
     resources:
     - terraformclusters
     - terraformmachines
     - terraformmachinepools
     - terraformclustertemplates
     - terraformmachinetemplates
     - terraformmachinepooltemplates
     verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
   - apiGroups: ["cluster.x-k8s.io"]
     resources: ["clusters", "machinedeployments", "machinepools"]
     verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
   - apiGroups: ["batch"]
     resources: ["jobs"]
     verbs: ["get", "list", "watch"]
   - apiGroups: [""]
     resources: ["pods", "pods/log", "events"]
     verbs: ["get", "list", "watch"]
   ```

   Note what is **absent**: no `secrets` rule. The pipeline does not read the credential mirror or the state through the API, although the modules they run can (see above). The approvers’ Role is the one in [Operating the gates](<https://captf.io/docs/concepts/approvals/operating/#who-can-approve>), bound to a smaller group.

Bind the Roles to groups with RoleBindings in the tenant namespace. Give the tenant no `ClusterRole` that reaches `terraformclusteridentities`.

> [!NOTE]
>
> **See also**
>
> - [Identities and Credentials](<https://captf.io/docs/user-guide/identities/index.md>).
> - [RBAC](<https://captf.io/docs/operator-guide/rbac/index.md>) and [Secrets](<https://captf.io/docs/operator-guide/secrets/index.md>).
> - [Security Model](<https://captf.io/docs/concepts/security-model/index.md>) and [Security considerations](<https://captf.io/docs/concepts/secret-management/security/index.md>).
> - [Approvals and Gates](<https://captf.io/docs/concepts/approvals/index.md>).
> - [Production Readiness](<https://captf.io/docs/operator-guide/production-readiness/index.md>).