pal-e-docs

Review: Create ISS staging environment with auto-deploy on merge to main

review-1893-2026-07-20 Review

review needs-refinement
Project

Verdict: NEEDS_REFINEMENT

Template Completeness

  • [x] Type -- Feature
  • [x] Lineage -- references #192 and #194
  • [x] Repo -- ldraney/pal-e-services
  • [x] User Story -- As a developer / staging auto-deploy / verify before prod
  • [x] Context -- explains current risk (auto-deploy to prod), staging rationale
  • [x] File Targets -- 4 targets across 2 repos + DNS
  • [x] Feature Flag -- none (appropriate for infra work)
  • [x] Acceptance Criteria -- 7 items
  • [x] Test Expectations -- 4 items with run command
  • [x] Constraints -- 4 constraints
  • [x] Checklist -- standard PR gates
  • [x] Related -- project-iss, #192, #194
  • [x] Dependency Chain -- bonus section, well-documented
  • [x] Keycloak -- bonus section, clear instructions

Traceability

  • [x] story:dev-environment label -- verified in project-iss user-stories section (row: "dev-environment", backing "#192 / #193 / #194 + docs/pipeline.md")
  • [x] story note verified -- found in project-iss user-stories table
  • [x] arch:argocd label -- references ArgoCD deployment pattern
  • [x] arch note verified -- arch-argocd note exists in pal-e-docs (slug: arch-argocd, "ArgoCD + Image Updater Deployment Pattern")
  • [x] Forgejo issue -- ldraney/pal-e-services#193, open

File Targets

  • [x] terraform/k3s.tfvars.example (pal-e-services) -- verified: contains services map (line 517, existing ISS entry) and service_databases map (line 551, existing iss entry). Agent needs to add staging service entry and staging database entry.
  • [x] terraform/keycloak.tf (pal-e-services) -- verified: uses for_each = var.keycloak_clients. Issue correctly says "no structural changes needed" -- Keycloak client URIs are managed in k3s.tfvars, not keycloak.tf.
  • [x] terraform/databases.tf (pal-e-services) -- verified: uses for_each = var.service_databases. Issue correctly says "no structural changes needed".
  • [x] overlays/intelligentstaffingsystems/staging/ (pal-e-deployments) -- verified: does not exist yet (expected, to be created). Prod overlay exists at overlays/intelligentstaffingsystems/prod/ with kustomization.yaml, deployment-patch.yaml, secrets.enc.yaml, README.md -- staging should mirror this structure.
  • [ ] DNS -- ISSUE: Issue says "add A record via DNS provider (GoDaddy)" but DNS records are terraform-managed in pal-e-platform/terraform/dns.tf. Existing pattern: godaddy_dns_record.iss_dev (line 64) uses a CNAME pointing to intelligentstaffingsystems.ai, not an A record. Staging should follow the same CNAME pattern. This is a 3rd repo not listed in File Targets.
  • [ ] Caddy reverse proxy -- MISSING: The custom domain routing path (per arch-argocd) requires Caddy on the Hetzner edge VPS to proxy staging.intelligentstaffingsystems.ai to intelligentstaffingsystems-staging.tail5b443a.ts.net. Caddy is managed by SaltStack (hetzner-edge module uses salt_master_ip). The issue does not mention Caddy/Salt configuration. Without this, DNS alone will not route traffic to staging.

Repo Placement

Issue is filed on pal-e-services. Work actually spans 3 repos:
  • pal-e-services -- terraform k3s.tfvars (service entry, database entry, Keycloak client URIs)
  • pal-e-deployments -- staging overlay (acknowledged in issue)
  • pal-e-platform -- DNS record in dns.tf + Caddy/Salt config (NOT acknowledged in issue)
The issue does not clarify whether the agent should create PRs in all 3 repos or how cross-repo coordination works.

Dependencies

  • #192 (dev environment, board item 1892) -- backlog, independent. Confirmed parallel.
  • #194 (prod manual promotion, board item 1894) -- backlog, blocked by this ticket. Confirmed: both open.
  • No in-progress blockers found on board-iss.
  • Dependency chain is well-documented in the issue body.

Acceptance Criteria

7 ACs, all verifiable by an agent:
  • AC1 (namespace) -- kubectl get ns intelligentstaffingsystems-staging
  • AC2 (ArgoCD synced) -- argocd app get intelligentstaffingsystems-staging
  • AC3 (image updater) -- check annotations on ArgoCD application CR
  • AC4 (staging URL) -- curl -sI https://staging.intelligentstaffingsystems.ai
  • AC5 (databases) -- kubectl or psql verification
  • AC6 (Keycloak) -- verify redirect URIs via Keycloak admin API
  • AC7 (DNS) -- dig staging.intelligentstaffingsystems.ai
All criteria are testable. AC4 depends on Caddy/Salt configuration which is not scoped in the issue (see File Targets issues).

Blast Radius

  • var.services addition: Low risk. New entry follows arch-argocd's var.services pattern (7 resources via for_each). Does not modify existing services.
  • Keycloak client URIs: Medium risk. Adding staging redirect URIs to the shared intelligentstaffingsystems client. Misconfiguration could affect prod/dev auth flows. However, the pattern matches the existing dev URIs already added.
  • Image updater overlap: During the gap between #193 (this) and #194 (prod manual promotion), BOTH staging and prod will auto-deploy on new images. This is documented and expected but worth noting -- any broken merge to main hits prod immediately during this period.
  • Network policies: pal-e-platform's network-policies.tf has explicit entries for the intelligentstaffingsystems namespace. The new intelligentstaffingsystems-staging namespace may need its own network policy entries. Not mentioned in the issue.
  • Stale image_repo in k3s.tfvars.example: The example file says image_repo = "intelligentstaffingsystems/intelligentstaffingsystems" (line 519), but the production overlay references harbor.tail5b443a.ts.net/intelligentstaffingsystems/app. Per the "Image Repo Mismatch Fix" memory note, the actual k3s.tfvars was fixed to intelligentstaffingsystems/app. The issue's constraint perpetuates the stale value. The agent must use the correct value (intelligentstaffingsystems/app) for staging.

Decomposition Assessment

File targets: 4+ files across 3 repos (pal-e-services + pal-e-deployments + pal-e-platform). ACs: 7 (exceeds threshold of 5). Estimated time: 5-10 minutes for an experienced agent. Per user preference (memory: "No Decomposition"), no decomposition recommended. The work is cohesive -- it follows the arch-argocd "Onboarding a New Service" pattern, and splitting it would create coordination overhead. A single agent with platform knowledge can execute this as one unit.

Recommendations

  • [BODY] Fix image_repo reference in Constraints section: intelligentstaffingsystems/intelligentstaffingsystems should be intelligentstaffingsystems/app (matches prod overlay and image repo mismatch fix). Also update k3s.tfvars.example line 519 to reflect the actual value.
  • [BODY] Fix DNS file target: DNS is terraform-managed in pal-e-platform/terraform/dns.tf, not added manually via GoDaddy. Add pal-e-platform/terraform/dns.tf to File Targets. Follow the existing CNAME pattern (cf. godaddy_dns_record.iss_dev at line 64) rather than an A record.
  • [BODY] Add Caddy/Salt configuration to scope: staging.intelligentstaffingsystems.ai needs a Caddy reverse proxy entry on the Hetzner edge VPS to route to intelligentstaffingsystems-staging.tail5b443a.ts.net. This is managed by SaltStack. Without it, AC4 (staging URL serves app) will fail. Document whether this is handled in this ticket or a separate one.
  • [BODY] Clarify cross-repo PR strategy: Work spans pal-e-services, pal-e-deployments, and pal-e-platform. State whether the agent should create PRs in all repos, and specify the order of operations (e.g., pal-e-services terraform first, then pal-e-deployments overlay, then pal-e-platform DNS).