pal-e-docs

Validation: Register ISS dev environment as terraform-managed service

validation-svc192-2026-07-26b Doc

validation fail
Project

Verdict: FAIL

Ticket

svc#192 (board item #1892 on board-iss) — Register ISS dev environment as terraform-managed service. Merged via PR #195 (commit 6df8aa6).

Environment

Prod cluster (k3s, default context). Repo: pal-e-services (terraform). Validation tiers: Tier 1 (local) + Tier 3 (prod).

Checks

# Criterion How Verified Result Evidence
1 Namespace exists on cluster matching chosen service key (iss-dev) <code>kubectl get ns | grep iss</code> FAIL Only <code>intelligentstaffingsystems</code> (prod) exists. No <code>iss-dev</code> namespace. <code>tofu plan</code> shows <code>kubernetes_namespace_v1.service["iss-dev"] will be created</code>.
2 ArgoCD application is Synced and Healthy <code>kubectl get applications.argoproj.io -n argocd</code> FAIL No <code>iss-dev</code> ArgoCD app exists. <code>tofu plan</code> shows <code>argocd_application.service["iss-dev"] will be created</code>.
3 <code>curl -sI https://dev.intelligentstaffingsystems.ai</code> returns 200 <code>curl -sI</code> FAIL Returns <code>HTTP/2 502</code> (Caddy, no backend).
4 No Harbor project or robot accounts for dev service Code review + <code>tofu plan</code> output PASS <code>grep -E 'harbor.*iss-dev' plan</code> returns empty. <code>local.services_with_image_updater</code> correctly filters out iss-dev (<code>image_updater = false</code>).
5 No image updater annotations on dev ArgoCD app Code review of <code>services.tf</code> PASS Annotations block is conditional: <code>each.value.image_updater ? merge(...) : {}</code>. iss-dev entry has <code>image_updater = false</code> in k3s.tfvars.
6 <code>terraform plan</code> is clean after apply <code>tofu plan -lock=false -var-file=k3s.tfvars</code> FAIL Plan: 14 to add, 7 to change, 4 to destroy. <code>tofu apply</code> has NOT been run. Includes changes from PR #195 (iss-dev), #196 (staging), and #201 (prod image_updater disable).

Tier 1 (Local) Summary

  • Code review: PASS — image_updater = optional(bool, true) added to variables.tf; services.tf gates Harbor resources behind local.services_with_image_updater; ArgoCD annotations conditional on the flag. iss-dev entry in k3s.tfvars is correct (port 80, funnel false, image_updater false, source_path points to dev overlay).
  • CI pipeline #255: SUCCESS — push to main for merge commit 6df8aa6.
  • tofu plan: iss-dev resources planned correctly (namespace + ArgoCD app only, no Harbor).

Tier 3 (Prod) Summary

  • iss-dev namespace: Does not exist.
  • iss-dev ArgoCD app: Does not exist.
  • dev.intelligentstaffingsystems.ai: HTTP 502.

Regression Check

ISS prod is unaffected by the merge:
  • intelligentstaffingsystems ArgoCD app: Synced / Healthy
  • Pod intelligentstaffingsystems-69c4f699d6-vp89s: 1/1 Running, 0 restarts
  • curl -sI https://intelligentstaffingsystems.ai: HTTP 200

Root Cause

Merged does not equal applied. PR #195 was merged to main (commit 6df8aa6) and CI passed, but tofu apply was never executed. The code changes are correct; the deployment step was missed. Manual pipeline runs (#206, #216, #217) all predate the merge.

Action Required

Run tofu apply -var-file=k3s.tfvars in pal-e-services/terraform. Note: the plan includes changes from three merged PRs (#195 iss-dev, #196 staging, #201 prod image_updater disable). All should be reviewed before a single apply. After apply, re-validate AC 1-3 and 6.

Discovered Issues

Multiple merged PRs (#195, #196, #201) are stacked unapplied. This creates a compound apply that is harder to validate per-ticket. Consider running apply after each merge, or at minimum per-sprint.