Review: Add Ingress/TLS + wire Keycloak env vars for production deploy

Verdict: APPROVED

Previous Findings Resolution

• [x] Missing Lineage section -- NOW ADDED with upstream (pal-e-services#173 tofu apply) and downstream (#93 verify site live)
• [x] Missing Acceptance Criteria -- NOW ADDED with 6 verifiable conditions
• [x] Keycloak env var placement unspecified -- NOW CLARIFIED: ConfigMap for KEYCLOAK_URL, KEYCLOAK_REALM, KEYCLOAK_CLIENT_ID; Secret for KEYCLOAK_CLIENT_SECRET
• [x] redirect-middleware.yaml unclear -- NOW ANNOTATED as conditional ("only if needed; Tailscale funnels may handle HTTPS redirect natively")
• [x] Undocumented tofu apply dependency -- NOW IN LINEAGE as upstream blocker
• [x] [SCOPE] Create arch-k8s-deploy note -- NOW EXISTS in pal-e-docs (comprehensive k8s deployment architecture)
• [x] [SCOPE] Create arch-keycloak note -- NOW EXISTS in pal-e-docs (Keycloak OIDC architecture)

Template Completeness

• [x] Type -- Feature
• [x] Lineage -- present (upstream: pal-e-services#173, downstream: #93, sprint: 12)
• [x] Repo -- prediction-assistant
• [x] User Story -- present
• [x] Context -- present, includes Keycloak env var placement and ingress clarification
• [x] File Targets -- 7 targets (3 new, 4 existing)
• [x] Feature Flag -- None (acceptable for infra work)
• [x] Acceptance Criteria -- 6 items, all verifiable
• [x] Test Expectations -- 3 items
• [x] Constraints -- present
• [x] Checklist -- present (Code complete checked)
• [x] Related -- present

Traceability

• [x] story:platform-setup label -- Platform Setup
• [x] story note verified -- found in project-prediction-assistant user-stories section (key: platform-setup, role: Developer)
• [x] arch:k8s-deploy label -- Kubernetes Deployment
• [x] arch note verified -- arch-k8s-deploy exists in pal-e-docs (created 2026-07-03, comprehensive with diagram, directory structure, components)
• [x] arch:keycloak label -- Keycloak OIDC
• [x] arch note verified -- arch-keycloak exists in pal-e-docs (client pattern, config location, Terraform-managed)
• [x] Forgejo issue -- ldraney/prediction-assistant#105, closed (consistent with validation column -- implementation merged)

File Targets

• [x] k8s/overlays/prod/ingress.yaml (new) -- verified: does not exist, correctly marked as new
• [x] k8s/overlays/prod/cluster-issuer.yaml (new) -- verified: does not exist, correctly marked as new
• [x] k8s/overlays/prod/redirect-middleware.yaml (new, conditional) -- verified: does not exist. Now annotated as conditional on Tailscale funnel behavior. Previous concern RESOLVED.
• [x] k8s/overlays/prod/kustomization.yaml -- verified: exists (73 lines, has image pin, patches, labels). New resources will need adding.
• [x] k8s/base/configmap.yaml -- verified: exists with RAILS_ENV, RAILS_LOG_TO_STDOUT, RAILS_SERVE_STATIC_FILES, RAILS_MAX_THREADS, PORT. No Keycloak vars present (confirms gap).
• [x] k8s/base/deployment.yaml -- verified: exists with envFrom configMapRef and individual secretKeyRef entries (DATABASE_URL, SECRET_KEY_BASE, KALSHI_*). No Keycloak env vars. Pattern clear for adding KEYCLOAK_CLIENT_SECRET as another secretKeyRef.
• [x] k8s/base/worker-deployment.yaml -- verified: exists with same env pattern as deployment.yaml. No Keycloak env vars.

Repo Placement

Dependencies

• Upstream (documented): pal-e-services#173 "tofu apply" (board item 1772, todo column, sprint:12) -- provisions prediction-assistant-secrets with KEYCLOAK_CLIENT_SECRET. Documented in Lineage. RESOLVED from previous review.
• Downstream (documented): prediction-assistant#93 "Verify prediction-assistant.com is live" (board item 1774, todo column, sprint:12). Documented in Lineage.
• Transitive (implicit, not blocking): pal-e-services#171 "Add Keycloak realm + OIDC client" (board item 1768, validation column, sprint:11) -- realm must exist before env vars are meaningful. Covered transitively through #173.

Acceptance Criteria

• [x] AC1: "Tailscale Ingress resource created and assigned hostname" -- verifiable via kustomize build output
• [x] AC2: "HTTPS redirect applied" -- annotated with flexibility (funnel config OR middleware). Verifiable by checking resource/annotation presence.
• [x] AC3: "Keycloak env vars wired: ConfigMap for KEYCLOAK_URL, KEYCLOAK_REALM, KEYCLOAK_CLIENT_ID" -- verifiable by inspecting configmap.yaml
• [x] AC4: "KEYCLOAK_CLIENT_SECRET wired from prediction-assistant-secrets Secret" -- verifiable by checking deployment manifest for secretKeyRef
• [x] AC5: "Pods can start with all required env vars present" -- post-deploy verification, reasonable
• [x] AC6: "kustomize build k8s/overlays/prod/ succeeds" -- verifiable by agent

Blast Radius

• ConfigMap changes via envFrom propagate to both web and worker deployments -- both listed as file targets. Covered.
• Adding Keycloak vars to base/ affects both prod and dev overlays. Dev overlay could need different Keycloak URL/realm. Minor future concern -- not a blocker for this ticket.
• No sibling services in this repo.
• Existing env var pattern (individual secretKeyRef entries) is well-established -- adding KEYCLOAK_CLIENT_SECRET follows the same pattern as DATABASE_URL, SECRET_KEY_BASE, and KALSHI_* keys.

Decomposition Assessment

Recommendation