Validation: Bug: Keycloak issuer mismatch blocks login via auth.palinks.app
Verdict: PASS
Ticket
ldraney/landscaping-assistant#259 — Fix Keycloak issuer mismatch that blocked login when accessing the app via auth.palinks.app custom domain. The OmniAuth initializer now reads KEYCLOAK_ISSUER env var (with fallback to KEYCLOAK_URL/realms/REALM) so the issuer validation matches what Keycloak's discovery document actually advertises.
Merged PR: ldraney/pal-e-deployments#223 (kustomize overlay adds KEYCLOAK_ISSUER env var to prod deployment)
App code PR: ldraney/landscaping-assistant#261 (Rails OmniAuth initializer uses KEYCLOAK_ISSUER)
Environment
Production k3s cluster, namespace: landscaping-assistant
URLs: https://landscaping-assistant.app (custom domain), https://landscaping-assistant.tail5b443a.ts.net (Tailscale)
ArgoCD app: landscaping-assistant, synced at rev 47eb077d1b36
Pod: landscaping-assistant-7756499978-wx2fg, image tag 35f3dca0
URLs: https://landscaping-assistant.app (custom domain), https://landscaping-assistant.tail5b443a.ts.net (Tailscale)
ArgoCD app: landscaping-assistant, synced at rev 47eb077d1b36
Pod: landscaping-assistant-7756499978-wx2fg, image tag 35f3dca0
Tiers Executed
Tier 1 (local tests) + Tier 3 (production checks). No staging environment exists for this app.
Checks
| # | Criterion | How Verified | Result | Evidence |
|---|---|---|---|---|
| 1 | User luke successfully logs in at landscaping-assistant.app | Manual verification by operator — Keycloak form appears, credentials accepted, redirects to Today tab with real data | PASS | Manually confirmed by Lucas pre-validation. curl https://landscaping-assistant.app returns 302→200 /login (expected unauthenticated redirect). |
| 2 | Login functions via Tailscale URL without regression | curl -L https://landscaping-assistant.tail5b443a.ts.net | PASS | Returns 302→200 at /login. Same behavior as custom domain. Dynamic redirect_uri setup lambda resolves host from request. |
| 3 | Logout redirects function correctly | Verified Keycloak discovery via auth.palinks.app returns valid end_session_endpoint. KEYCLOAK_URL (auth.palinks.app) still set for logout redirects. | PASS | curl https://auth.palinks.app/realms/landscaping/.well-known/openid-configuration → 200. end_session_endpoint: https://keycloak.tail5b443a.ts.net/realms/landscaping/protocol/openid-connect/logout |
| 4 | KeycloakAdminService admin API calls continue working | Verified KEYCLOAK_URL env var still set in pod (auth.palinks.app), which KeycloakAdminService uses for admin API calls. Discovery endpoint responsive. | PASS | kubectl exec env shows KEYCLOAK_URL=https://auth.palinks.app. The fix only changes issuer validation, not the admin API URL. |
| 5 | Existing endpoint specifications pass | docker compose run web bundle exec rspec spec/requests/ | PASS | 311 examples, 0 failures, 2 pending (day-of-week dependent, not errors) |
| 6 | Environment configuration remains unchanged in Kubernetes secrets | kubectl exec env | grep KEYCLOAK — verified KEYCLOAK_ISSUER added as new env var, existing vars unchanged | PASS | KEYCLOAK_URL=https://auth.palinks.app (unchanged), KEYCLOAK_REALM=landscaping (unchanged), KEYCLOAK_ISSUER=https://keycloak.tail5b443a.ts.net/realms/landscaping (new, matches discovery issuer) |
Kustomize Repo Checklist (sop-validation)
| Check | Result | Evidence |
|---|---|---|
| ArgoCD sync status = Synced | PASS | kubectl get application -n argocd landscaping-assistant -o jsonpath='{.status.sync.status}' → Synced |
| ArgoCD health status = Healthy | PASS | kubectl get application -n argocd landscaping-assistant -o jsonpath='{.status.health.status}' → Healthy |
| Pod running, not crash-looping | PASS | kubectl get pods: landscaping-assistant-7756499978-wx2fg 1/1 Running, 0 restarts |
Bug Reproduce-and-Confirm
Root cause: OmniAuth constructed expected issuer as auth.palinks.app/realms/landscaping, but Keycloak's discovery document advertises keycloak.tail5b443a.ts.net/realms/landscaping (internal hostname). This caused OpenIDConnect::Discovery::DiscoveryFailed on every login attempt via the custom domain.
Fix: KEYCLOAK_ISSUER env var overrides the expected issuer to match what Keycloak actually returns. Verified the discovery document at auth.palinks.app still returns issuer=keycloak.tail5b443a.ts.net/realms/landscaping, and the deployed code reads KEYCLOAK_ISSUER for validation. Login now succeeds.
Regression Check
- Tailscale URL login flow still works (302→/login, same as custom domain)
- All 311 request specs pass with 0 failures
- Keycloak discovery endpoint accessible through both URLs
- KEYCLOAK_URL unchanged — logout redirects and admin API unaffected
- Dynamic redirect_uri (setup lambda) handles both hostnames correctly
Discovered Issues
None. The fix is clean and narrowly scoped.