pal-e-docs

TODO: Wire ALL woodpecker secrets into terraform helm values (DB password + agent secret + API token + encryption key)

todo-woodpecker-secrets-terraform Doc

todo done
Project
CRITICAL — recurring failure class. Every Woodpecker DB migration breaks 4 secrets across 5 consumers. This has happened twice now.

Root Cause

Woodpecker generates a random jwt-secret in server_configs table on every fresh DB. This invalidates all previously-issued API tokens. The WOODPECKER_ENCRYPTION_KEY env var can make this persistent, but it's not set.

Affected Secrets (4)

  • woodpecker_db_password — connection string in helm values has empty password. CNPG secret has the real one.
  • woodpecker_agent_secret — env var has no value. Agent can't auth to server.
  • woodpecker_api_token — JWT signed with old key. Breaks DORA exporter, MCP server, CI secrets.
  • WOODPECKER_ENCRYPTION_KEY — NOT SET. This is the root cause. Must be a persistent value in helm env vars so the JWT signing key survives DB migrations.

Affected Consumers (5)

  • k3s.tfvars — woodpecker_api_token
  • ~/.mcp.json — WOODPECKER_TOKEN (Woodpecker MCP server)
  • k8s secret dora-exporter in monitoring namespace — WOODPECKER_TOKEN
  • Woodpecker CI repo secrets — any repo using the API token
  • Helm values — WOODPECKER_DATABASE_DATASOURCE, WOODPECKER_AGENT_SECRET

Permanent Fix (all 4 items)

  • Add WOODPECKER_ENCRYPTION_KEY to helm values (persistent JWT signing key — prevents future token invalidation)
  • Wire woodpecker_db_password into the datasource URL template
  • Wire woodpecker_agent_secret into the agent env var via valueFrom
  • Add all 3 variables + encryption key to Makefile's TF_SECRET_VARS
  • Create SOP: "Woodpecker DB Migration" — checklist that includes token rotation + consumer updates

Temporary Fix (2026-03-15)

  • kubectl set env patches for DB password and agent secret (will be overwritten on next tofu apply)
  • API token still stale — requires manual UI regeneration, then update in k3s.tfvars + ~/.mcp.json + dora-exporter secret
Repo: forgejo_admin/pal-e-platform
Source: phase-platform-16-alert-tuning (16e) + Phase 17 DORA pipeline investigation