pal-e-docs

Scope Review: #88 -- Deploy Headlamp for visual cluster management

review-1311-2026-06-04 Review

review needs-refinement
Project

Verdict: NEEDS_REFINEMENT

Board item: #1311 on board-landscaping-observability (backlog, 2 points)
Forgejo issue: ldraney/landscaping-assistant #88
Labels: type:feature, arch:platform, story:observability, parent:43, devops

Template Completeness

  • [x] Type -- Feature
  • [x] Lineage -- Child of #43, Phase 4 of observability roadmap
  • [ ] Repo -- Says forgejo_admin/pal-e-platform but actual owner is ldraney (forgejo_admin 301-redirects to ldraney). Minor inaccuracy.
  • [x] User Story -- "As a platform operator / I want a visual web UI / So that I can see pod health... and provide cluster access via RBAC"
  • [x] Context -- References 26 Grafana dashboards, CNCF status, explains the gap clearly
  • [ ] File Targets -- Three files listed but paths do not match the actual repo structure (see File Targets section below)
  • [x] Acceptance Criteria -- Four testable criteria
  • [x] Test Expectations -- Two smoke checks (acceptable for 2-point infra ticket)
  • [x] Constraints -- Keycloak OIDC, Tailscale funnel, read-only RBAC
  • [x] Checklist -- Standard PR/tests/scope items
  • [x] Related -- References project-pal-e-platform and parent #43

Traceability

  • [x] story:observability -- present on board item labels
  • [x] arch:platform -- present on board item labels
  • [x] Forgejo issue #88 -- exists and is open
  • [x] Parent #43 -- exists and is open (Observability & DORA metrics stack [PARENT])
  • [x] Roadmap alignment -- confirmed as Phase 4 in docs/observability-roadmap.md

File Targets

Issue lists:
  • [ ] terraform/headlamp.tf -- DOES NOT EXIST. No flat .tf files at terraform root beyond main.tf, variables.tf, providers.tf, outputs.tf, versions.tf, network-policies.tf.
  • [ ] terraform/tailscale.tf -- DOES NOT EXIST. Tailscale funnels live in terraform/modules/networking/main.tf.
  • [ ] terraform/keycloak.tf -- DOES NOT EXIST. Keycloak resources live in terraform/modules/keycloak/main.tf.
Actual repo structure uses modules:
  • terraform/modules/monitoring/ -- Prometheus, Grafana, Loki, blackbox, DORA dashboards
  • terraform/modules/networking/ -- Tailscale operator, ACLs, 9 existing funnels (Grafana, Forgejo, Woodpecker, Harbor, MinIO, Keycloak, etc.)
  • terraform/modules/keycloak/ -- Keycloak k8s deployment (Deployment, Service, PVC, ConfigMap only -- no OIDC client resources)
  • terraform/main.tf -- Module orchestration with ~80 moved blocks from monolith refactor
Assessment: File targets are wrong. An agent following them will create files in the wrong directory. Correct targets depend on a design decision: does Headlamp go into the monitoring module or get its own headlamp module? The funnel goes in networking. The OIDC client approach needs clarification (see Dependencies).

Repo Placement

Correct repo: ldraney/pal-e-platform. This is a platform-level Helm deployment with Tailscale networking and Keycloak auth -- all concerns live in pal-e-platform. Single-repo scope is appropriate. No cross-repo work needed (Headlamp reads the k8s API directly, no app instrumentation required).
The ### Repo field should be corrected from forgejo_admin/pal-e-platform to ldraney/pal-e-platform.

Dependencies

  • [x] Headlamp Helm chart -- available at https://kubernetes-sigs.github.io/headlamp/, chart name headlamp/headlamp, version 0.42.0. Note: the roadmap references headlamp-k8s/headlamp which is the old GitHub org; chart is now under kubernetes-sigs.
  • [x] Tailscale funnel pattern -- 9 existing funnels in networking module using kubernetes_ingress_v1 with tailscale.com/funnel annotation. Pattern is well-established and trivially extensible.
  • [x] Keycloak deployment -- running (v26.0.7, k8s Deployment in keycloak module).
  • [ ] Keycloak OIDC client management -- UNRESOLVED. The keycloak module uses only the kubernetes provider (k8s Deployment/Service/PVC). There are no keycloak_openid_client resources using the Keycloak Terraform provider. OIDC clients for existing services (Grafana, Forgejo) appear to be configured manually via the Keycloak admin UI. The Headlamp chart supports OIDC natively via config.oidc values (clientID, clientSecret, issuerURL, scopes, usePKCE), so the chart side is ready -- but the ticket must clarify how the Keycloak-side client is created.

Acceptance Criteria

  • "Headlamp deployed and accessible via Tailscale funnel" -- testable: curl the funnel URL, expect HTTP 200.
  • "Visual overview of all namespaces, pods, and resource usage" -- testable only via manual UI check. Acceptable for a UI deployment ticket.
  • "Log streaming per pod works from the UI" -- manual verification. Acceptable.
  • "OIDC login via Keycloak (or service account for initial deploy)" -- the parenthetical fallback is good but creates ambiguity about whether OIDC is in scope or deferred. Should pick one path for this ticket and make the other a follow-up.

Blast Radius

  • Files touched: 2-4 files in pal-e-platform (Helm release, funnel ingress, optionally OIDC client, module wiring)
  • Services affected: None -- Headlamp is additive. It reads the k8s API; it does not modify existing services.
  • Failure modes: Bad RBAC could expose cluster-admin via web UI (the constraint says read-only, which mitigates this). Broken Tailscale funnel would only affect Headlamp access.
  • Rollback: tofu destroy -target=module.monitoring.helm_release.headlamp (or equivalent) -- straightforward Helm uninstall.
  • Risk: LOW. Purely additive infrastructure with no dependencies from existing services.

Decomposition Assessment

Three-thing limit: This ticket has 3 discrete changes (Helm release, Tailscale funnel, OIDC client). At the limit but acceptable if OIDC scope is clarified.
Five-minute rule: An agent with correct file targets can execute this in under 5 minutes. The Tailscale funnel is a copy-paste of an existing pattern. The Helm release follows established module patterns.
Parallelization: Not needed -- all changes are in one repo, one PR.
Assessment: Scope is appropriate for a 2-point ticket. No decomposition needed.

Recommendation

  • Fix File Targets. Replace the three flat paths with the correct modular paths. Decide whether Headlamp goes in terraform/modules/monitoring/main.tf (alongside Grafana) or gets a new terraform/modules/headlamp/ module. The monitoring module is the natural fit given Headlamp is in the observability roadmap.
  • Clarify OIDC implementation path. Three options: (a) Add the keycloak Terraform provider and define the OIDC client as IaC (sets a new pattern for all future OIDC clients), (b) document manual Keycloak admin UI steps in the PR, (c) defer OIDC to a follow-up ticket and use a ServiceAccount + ClusterRoleBinding (read-only) for initial deploy. Pick one and update the AC accordingly.
  • Fix Repo owner from forgejo_admin/pal-e-platform to ldraney/pal-e-platform.
  • (Optional) Update the Helm chart repo reference in docs/observability-roadmap.md from headlamp-k8s/headlamp to kubernetes-sigs/headlamp to reflect the current ownership.