pal-e-docs

Review: ArgoCD CMP sidecar fails to render kustomize+SOPS overlays (re-review)

review-525-2026-03-29-v2 Doc

review ready
Project

Verdict: READY

Re-review of board item #525 after refinements. Type corrected to bug, file targets corrected to pal-e-services, full bug template sections added, tofu -lock=false convention applied. Ticket is actionable.

Template Completeness

Checked against template-issue-bug:
  • [x] Type — Bug
  • [x] Lineage — Board, Story, Arch references present (non-standard format but more informative than template default)
  • [x] Repo — forgejo_admin/pal-e-services correctly identified with file location
  • [x] What Broke — Clear error message (CMP sidecar EOF), affected apps listed (4 apps)
  • [x] Repro Steps — 4 concrete steps with observable outcomes
  • [x] Expected Behavior — Clear success criteria (tofu apply succeeds, apps Synced)
  • [x] Environment — Cluster (prod), namespace (argocd), service version context
  • [x] Acceptance Criteria — 4 testable criteria
  • [x] Related — Project, blocking issues, discovery context
  • [x] File Targets — Present (extra section beyond bug template, adds value)
  • [x] Test Expectations — Present (extra section, adds concrete verification commands)
  • [x] Constraints — Present with tofu/-lock=false convention correctly applied
  • [x] Checklist — Present

Traceability

  • [x] story:superuser-deploy — Present on board item. Verified on project-pal-e-platform user-stories table: "I can deploy infrastructure changes via tofu plan/apply and see them succeed in Woodpecker CI without manual intervention."
  • [x] arch:argocd — Present on board item. Note arch-argocd does not exist in pal-e-docs (search returned no results).
  • [x] arch:k8s-deploy — Present on board item. Note arch-k8s-deploy does not exist in pal-e-docs (search returned no results).
  • [x] Forgejo issue — #225 on forgejo_admin/pal-e-platform, state: open

File Targets

  • [x] ~/pal-e-services/terraform/main.tf:35-191 — Verified: line 35 starts resource "helm_release" "argocd", line 93 repoServer block, line 114 extraContainers with cmp-sops (line 116), volumes at lines 169-191. Line references accurate.
  • [x] CMP plugin config (configs.cmp block) — Verified at lines 51-76: kustomize-sops plugin with generate command (sh -c, sops decrypt, kustomize build) and discover config (fileName = "*.enc.yaml").
  • [x] ~/pal-e-platform/terraform/modules/ci/ exclusion — Confirmed: Grep found zero CMP/SOPS references in pal-e-platform/terraform. Correctly excluded.
  • [x] ~/pal-e-services/terraform/variables.tf:37 — Contains sops_age_private_key variable. Not listed as target but may be relevant during investigation (read-only).

Repo Placement

Forgejo issue filed on forgejo_admin/pal-e-platform (#225) but fix is entirely in forgejo_admin/pal-e-services. This is acceptable — the issue was discovered during platform validation, and the board is board-pal-e-platform which tracks cross-repo platform concerns. The agent must open the PR on pal-e-services, not pal-e-platform. The issue body correctly identifies the target repo as forgejo_admin/pal-e-services.

Dependencies

  • [x] Board item #521 "Apply 5+ pending terraform changes (ArgoCD migrations)" — in todo column with depends:#224. Related but does not block this ticket. This ticket (#525) should resolve first since it unblocks CMP functionality.
  • [x] pal-e-services #39 (ArgoCD source migrations) — open, blocked by this bug per issue body. Documented.
  • [x] pal-e-app #88 (CI validation) — blocked by this bug per issue body. Documented.
  • [x] pal-e-services #28 (ArgoCD auto-sync fights manual deploys) — open, potentially related symptom. Not documented as dependency in this ticket.
  • [x] pal-e-services #17 (repo-server memory bump) — open, potentially related (memory limits could cause EOF). Not documented but worth investigation during fix.

Acceptance Criteria

4 criteria, all testable by an agent:
  • [x] tofu apply for pal-e-services succeeds — agent can run tofu plan -lock=false to verify convergence
  • [x] ArgoCD dashboard shows all apps Synced — verifiable via kubectl get applications -n argocd
  • [x] CMP sidecar logs show success — verifiable via kubectl -n argocd logs for repo-server cmp-sops container
  • [x] No regression in existing working ArgoCD apps — verifiable by checking all app sync statuses
Test expectations section adds two concrete checks: pod 2/2 Running with no restarts, and tofu plan shows 0 changes. Specific and automatable.

Blast Radius

CMP sidecar configuration is contained entirely in pal-e-services/terraform/main.tf. No CMP-related code exists in pal-e-platform (confirmed via Grep). All 4 affected ArgoCD-managed apps (pal-e-docs, gcal-scheduler, pal-e-app, platform-validation) are documented in the issue. No sibling services have independent CMP configurations. Rollback is straightforward — revert the Helm values change and reapply.

Decomposition Assessment

Three-thing limit: 1 primary file target (main.tf), investigation-only targets for kubectl commands. Single repo (pal-e-services).
Five-minute rule: 4 acceptance criteria, 1 file to modify, debugging + fix + verify cycle. Fits in a single agent pass.
No independent subtasks to parallelize. No decomposition needed.

Recommendation

  • [SCOPE] Create architecture note arch-argocd for the ArgoCD component in pal-e-docs. Not a blocker — documentation gap.
  • [SCOPE] Create architecture note arch-k8s-deploy for the k8s deployment component in pal-e-docs. Not a blocker — documentation gap.
No action needed on the ticket itself. Scope is solid and actionable.