pal-e-docs

Convention: Apply-Before-Merge (Deprecated)

convention-apply-before-merge Convention

active convention
Project

Convention: Apply-Before-Merge (Deprecated)

The pattern where you run tofu apply locally before merging a PR. This convention documents a deprecated pattern and its break-glass exception. All infrastructure changes now flow through CI: merge = deploy.

History

During platform bootstrap (2026-02 through 2026-03-14), infrastructure PRs were applied locally before merge. This was necessary because:
  • No CI pipeline existed for tofu plan or tofu apply
  • The operator needed to verify the plan output before committing to a merge
  • Some resources (Keycloak, CNPG clusters) required iterative apply-fix cycles that couldn't be predicted from plan output alone
This pattern was retired by plan-pal-e-platform Phase 6.3 (plan-on-PR) and Phase 6.4 (apply-on-merge).

Why Deprecated

Problem Impact CI Fix
State lock contention Two sessions ran <code>tofu apply</code> simultaneously (2026-03-14 incident), blocking each other CI serializes all applies — one pipeline at a time
Laptop SPOF Only Lucas's machine could deploy; no deploy if laptop offline Woodpecker runs from cluster — no laptop dependency
No deploy audit trail Manual applies leave no record of what was applied and when Pipeline logs, Forgejo PR comments with plan output, commit history
DORA measurement gap Deployment Frequency unmeasurable — no event to count Pipeline runs = deployments. Measurable. Timestamped.
Drift between PR and applied state PR could be merged without the apply ever running, or apply could drift from PR content Plan runs on PR (6.3), apply runs on merge (6.4) — always in sync

The New Pattern

After Phase 6.4 is live:
  • Developer creates branch, makes changes, opens PR
  • Woodpecker runs tofu validate + tofu plan on PR
  • Plan output is posted as Forgejo PR comment
  • QA (or human) reviews the plan output alongside the code diff
  • Lucas approves and merges
  • Woodpecker runs tofu apply -auto-approve on merge to main
  • Betty Sue verifies pipeline success before marking phase complete
Nobody runs tofu apply locally. The CI pipeline is the single writer.

Break-Glass Procedure

For emergency infrastructure changes when CI is broken or unavailable. This is an L0 action — always requires Lucas's explicit approval.
  • Announce: Post in Telegram group: "Manual apply in progress — hold all merges to pal-e-platform"
  • Plan: Run tofu plan locally, save output to file
  • Apply: Run tofu apply -lock-timeout=5m
  • Verify: Confirm resources created/updated as expected
  • Commit: Push any state or config changes
  • Announce: Post in Telegram: "Manual apply complete, merges unblocked"
  • Document: Create a TODO note: why break-glass was needed, what was applied, what CI issue blocked the normal path

When Break-Glass is Appropriate

  • CI pipeline itself is broken and can't self-heal (chicken-and-egg: can't deploy the fix via CI)
  • Security emergency requiring immediate infrastructure change
  • Bootstrap of entirely new infrastructure that CI can't reach yet (new cluster, new provider)
  • State corruption recovery that requires manual tofu import or tofu state rm

What is NOT Break-Glass

  • "I want to see it work before merging" — that's what the plan-on-PR comment is for
  • "CI is slow" — patience, not a manual apply
  • "I'm already SSH'd into the cluster" — convenience is not an emergency
  • plan-pal-e-platform — Phase 6.3 (plan-on-PR) and Phase 6.4 (apply-on-merge)
  • convention-agent-autonomy-levels — manual deploy is L0
  • sop-ci-pipeline-recovery — what to do when the pipeline fails
  • phase-pal-e-agency-9-ci-driven-operating-model — parent phase
  • convention-cross-pillar-triggers — the feedback loop this convention feeds into