# Runbooks

If you already have a condition and a reason, [Troubleshooting by Condition](<https://captf.io/docs/operator-guide/troubleshooting/index.md>) looks it up directly.

Each runbook below covers one symptom: what it looks like, why it happens, how to check, how to fix it, and how to confirm the fix worked. Alerts and condition reasons are cross-references, not a substitute for reading the page: start from whichever alert fired or condition you see, but read the whole runbook before you act.

| Runbook | Covers | Alerts and conditions |
| --- | --- | --- |
| [Failing Jobs](<https://captf.io/docs/operator-guide/runbooks/job-failures/index.md>) | A `TerraformCluster`, `TerraformMachine` or `TerraformMachinePool` whose Jobs keep failing. | [`CAPTFJobFailing`](<https://captf.io/docs/reference/alerts/#captfjobfailing>), [`CAPTFNoRecentSuccess`](<https://captf.io/docs/reference/alerts/#captfnorecentsuccess>); `ApplyJobSucceeded=False` and `DriftJobSucceeded=False` |
| [Stuck Destroy](<https://captf.io/docs/operator-guide/runbooks/stuck-destroy/index.md>) | A `destroy` Job that cannot succeed, and how to remove the object’s finalizer safely. | [`CAPTFDestroyStuck`](<https://captf.io/docs/reference/alerts/#captfdestroystuck>); `ApplyJobSucceeded=False`/`DestroyFailed` |
| [Unreadable State](<https://captf.io/docs/operator-guide/runbooks/state-unreadable/index.md>) | A state Secret CAPTF cannot parse. | [`CAPTFStateUnreadable`](<https://captf.io/docs/reference/alerts/#captfstateunreadable>); `StateReadable=False`/`StateCorrupt`, `StateInconsistent` or `StateEncrypted` |
| [State Restore](<https://captf.io/docs/operator-guide/runbooks/state-restore/index.md>) | Restoring a Terraform or OpenTofu state from a CAPTF-managed backup. | `StateReadable=False`/`StateLost`; `RestoreJobSucceeded` |
| [Stale State Lock](<https://captf.io/docs/operator-guide/runbooks/stale-lock/index.md>) | Clearing a state lock left behind by a killed or evicted runner. | [`CAPTFForceUnlocks`](<https://captf.io/docs/reference/alerts/#captfforceunlocks>); `StateReadable=False`/`StateLocked` |
| [Size Limits](<https://captf.io/docs/operator-guide/runbooks/size-limits/index.md>) | A state or rendered inputs approaching the Secret size limit. | [`CAPTFStateNearSecretLimit`](<https://captf.io/docs/reference/alerts/#captfstatenearsecretlimit>), [`CAPTFInputsNearLimit`](<https://captf.io/docs/reference/alerts/#captfinputsnearlimit>); `ApplyJobSucceeded=False`/`InputsTooLarge` |
| [Slow Jobs](<https://captf.io/docs/operator-guide/runbooks/slow-jobs/index.md>) | Jobs that take a long time to run, or a long time to start. | [`CAPTFJobSlow`](<https://captf.io/docs/reference/alerts/#captfjobslow>), [`CAPTFJobQueueSlow`](<https://captf.io/docs/reference/alerts/#captfjobqueueslow>) |
| [Reconcile Errors](<https://captf.io/docs/operator-guide/runbooks/reconcile-errors/index.md>) | The controller itself failing to reconcile. | [`CAPTFReconcileErrors`](<https://captf.io/docs/reference/alerts/#captfreconcileerrors>) |
| [Identities and Credentials](<https://captf.io/docs/operator-guide/runbooks/identity-and-credentials/index.md>) | An object that cannot resolve or mirror its `TerraformClusterIdentity`. | `IdentityAllowed=False`, `CredentialsMirrored=False` |
| [Webhook Unavailable](<https://captf.io/docs/operator-guide/runbooks/webhook-unavailable/index.md>) | Writes to a `Terraform*` object failing because the admission webhook cannot be reached. | None |
| [Total State Loss and Import](<https://captf.io/docs/operator-guide/runbooks/total-state-loss/index.md>) | The state is gone with no backup: rebuild it from a workstation, or abandon and recreate the object with `import` blocks. | `StateReadable=False`/`StateLost` |
| [clusterctl move](<https://captf.io/docs/operator-guide/runbooks/move/index.md>) | Moving a Cluster’s `Terraform*` objects with `clusterctl move`: the procedure, what does and does not come along, and cleaning up what a move leaves behind. | None |

For how deletion works and a flowchart from a stuck object to the right runbook, see [Deletion and Teardown](<https://captf.io/docs/concepts/deletion/index.md>). For why an object has no Job and is waiting, see [Nothing Is Happening](<https://captf.io/docs/concepts/jobs/troubleshooting/index.md>).

Every runbook above applies to `TerraformCluster`, `TerraformMachine` and `TerraformMachinePool` alike unless it says otherwise.

> [!NOTE]
>
> **See also**
>
> - [Observability](<https://captf.io/docs/operator-guide/observability/index.md>) — the metrics and alerts these runbooks are reached from.
> - [Conditions reference](<https://captf.io/docs/reference/conditions/index.md>) — every condition type and reason named above.