Review: Apply 5+ pending terraform changes (ArgoCD migrations)

review-521-2026-03-28 Doc

review needs-refinement

Verdict: NEEDS_REFINEMENT

Template Completeness

  • [x] Type — Task
  • [x] User Story — clear who/what/why
  • [x] Context — present but stale (describes drift that no longer exists)
  • [x] Scope — Task-appropriate replacement for File Targets
  • [x] File Targets — also present (redundant with Scope for Task type, not harmful)
  • [x] Acceptance Criteria — 5 criteria, but AC #4 references non-existent service
  • [x] Test Expectations — verifiable kubectl commands
  • [ ] Constraints — missing. High-risk repo_credentials replacement not documented.
  • [ ] Checklist — missing
  • [x] Related — dependencies and PRs listed
  • [ ] Lineage — missing
  • [ ] Repo — missing explicit repo declaration (implied by Forgejo issue placement)

Traceability

  • [x] story:superuser-deploy — superuser deployment story
  • [x] arch:argocd — ArgoCD architecture component
  • [x] arch:terraform — Terraform architecture component
  • [x] Forgejo issue — forgejo_admin/pal-e-services#39, open

File Targets

Task type — file targets are informational, not prescriptive. Verified anyway:
  • [x] terraform/main.tf — verified exists at ~/pal-e-services/terraform/main.tf
  • [x] terraform/k3s.tfvars — verified exists at ~/pal-e-services/terraform/k3s.tfvars
Targets are specific enough for an agent — this is a tofu apply task, not a code change.

Repo Placement

Issue filed on forgejo_admin/pal-e-services — correct. The tofu apply runs from this repo's terraform/ directory. Single-repo scope. No mismatch.

Dependencies

  • [x] depends:#224 — board item #224 ("tofu-state backup CronJob failures") is in done column. Dependency satisfied.
  • [x] PR #34 (CNPG re-establish) — merged on main (commit 8442d07)
  • [x] PR #37 (remove :80) — merged on main (commit 6b4267d)
  • [x] PR #38 (remove dead funnel) — merged on main (commit dc771c7)
  • [x] PR #40 (CMP sidecar memory) — merged on main (commit 72df991), not referenced in issue but already landed
No unresolved dependencies blocking execution.

Acceptance Criteria

5 acceptance criteria in the issue. Assessment:
  • [x] AC1: "tofu plan shows 0 changes after apply" — testable, specific
  • [x] AC2: "ArgoCD dashboard shows all apps in Synced state" — testable via kubectl get applications -n argocd
  • [x] AC3: "Image Updater logs show successful tag detection" — testable via kubectl logs
  • [ ] AC4: "pal-e-app deploys from pal-e-deployments overlay" — INVALID. pal-e-app does not exist in terraform services. It was renamed to pal-e-docs-app. No overlays/pal-e-app/ directory exists in pal-e-deployments.
  • [x] AC5: "gcal-scheduler deploys from pal-e-deployments overlay" — testable, but already true (no plan change for gcal-scheduler)
AC4 will cause agent confusion. AC5 is a no-op verification (already done). Missing AC for CNPG cluster creation and pal-e-mail ArgoCD app — the two most significant new resources.

Blast Radius

CRITICAL: Issue body is stale. The drift described in the Context section does not match the current tofu plan output. Multiple items described as pending changes have already been applied or never existed.
Stale claims in issue Context:
  • "gcal-scheduler: k8s to overlays" — already migrated, no plan change
  • "pal-e-app: k8s to overlays" — pal-e-app does not exist in terraform. Renamed to pal-e-docs-app.
  • "Image Updater write-back for 3 apps" — only pal-e-mail is new
  • "mcd-tracker-app image-list fix" — not in current plan, already resolved
Actual current plan (4 add, 9 change, 1 destroy):
  • CREATE: argocd_application.service["pal-e-mail"] — new ArgoCD app pointing to overlays/pal-e-mail/prod (overlay verified to exist)
  • CREATE: kubernetes_manifest.cnpg_cluster — CNPG cluster from PR #34
  • CREATE: kubernetes_manifest.cnpg_scheduled_backup — CNPG scheduled backup
  • REPLACE (destroy+create): argocd_repository_credentials.forgejo — removing :80 port suffix forces recreation. HIGH RISK: briefly disconnects ArgoCD from Forgejo. All apps may show "Unknown" until new creds propagate.
  • UPDATE: 8x harbor_creds secrets — removing stale argocd.argoproj.io/instance labels (cosmetic)
  • UPDATE: kubernetes_ingress_v1.service_funnel["pal-e-docs-app"] — ingress change
  • OUTPUT: service_urls removing pal-e-app + westsidekingsandqueens, adding pal-e-docs-app
Rollback: tofu apply is partially reversible. CNPG cluster creation and repo creds replacement are the highest-risk changes. If repo creds replacement fails mid-way, manual ArgoCD intervention may be needed.

Decomposition Assessment

Apply the three-thing limit and five-minute rule:
  • Discrete changes: 3 categories (new resources, repo creds replacement, label cleanup) — at the limit but acceptable since it is a single tofu apply command
  • Agent time estimate: under 5 minutes — run plan, apply, verify. Single command execution.
  • Independent subtasks: No. All changes are in one terraform state and must be applied atomically.
No decomposition needed. This is a single tofu apply, not a code change. However, the issue body must be corrected first so the agent verifies the right things.

Recommendation

  • [BODY] Remove all references to pal-e-app — this service does not exist in terraform. Replace AC #4 with "pal-e-mail ArgoCD app created and syncing from pal-e-deployments overlay"
  • [BODY] Rewrite Context section with current tofu plan output: 4 add (pal-e-mail app, CNPG cluster, CNPG backup, repo creds replacement), 9 update (8 harbor-creds label cleanup + 1 ingress), 1 destroy (old repo creds with :80)
  • [BODY] Remove stale items from Context: gcal-scheduler migration (already done), mcd-tracker-app image-list fix (already done), Image Updater write-back for 3 apps (only pal-e-mail is new)
  • [BODY] Add Constraints section: "The argocd_repository_credentials replacement will destroy and recreate Forgejo creds. Verify all apps re-sync after apply. Have kubectl ready to manually patch if creds recreation fails."
  • [BODY] Update Scope step 2: "Verify pal-e-deployments has correct overlay for pal-e-mail" (not gcal-scheduler/pal-e-app)
  • [BODY] Add AC: "CNPG cluster and scheduled backup created successfully" and "pal-e-mail ArgoCD app created and syncing"
  • [BODY] Update title to "Apply pending terraform changes (pal-e-mail app, CNPG, repo creds cleanup)"
  • [LABEL] Add arch:cnpg label — CNPG cluster creation is a significant part of this apply