pal-e-docs

Architecture: CI Pipeline (shared Woodpecker pattern)

arch-ci-pipeline Doc

architecture active platform ci
Project

CI Pipeline (shared Woodpecker pattern)

All platform services ship through the same Woodpecker → Harbor → pal-e-deployments → ArgoCD loop. This note is the architectural anchor that the arch:ci-pipeline ticket label points at.

Motivation

Every Python service in the platform (basketball-api, pal-e-docs, westside-contracts, westside-app, westside-streamlit, gcal-scheduler, westside-ai-assistant, etc.) uses exactly the same CI loop shape. Re-inventing it per service is waste. Documenting it once here gives new-service tickets a concrete arch reference to cite, and gives reviewers a canonical shape to compare against.

Pipeline steps

  • Checkout + setup — default Woodpecker checkout, set up Python/uv as needed
  • Test — run the repo's test suite (pytest, typically). Tests must pass before build.
  • Build image — Kaniko builds from the repo's Dockerfile, tags with the short SHA, pushes to Harbor at harbor.tail5b443a.ts.net/{project}/{service}:{sha}
  • Update kustomize tag — a step that updates the image tag in pal-e-deployments/overlays/{service}/kustomization.yaml and pushes a commit directly to pal-e-deployments/main. This step runs on both success and failure of the build step per lesson from commit f17b49b (2026-04-09) — previously the update step was gated on build success, which left the overlay stale when a build partially succeeded.
  • ArgoCD sync — ArgoCD image updater or manual sync picks up the new tag in the overlay and applies to the cluster

Reference implementations

  • ~/basketball-api/.woodpecker.yaml — canonical reference for Python FastAPI services with pytest. See line ~60 for the update-kustomize-tag step.
  • ~/pal-e-docs/.woodpecker.yaml — reference for a SvelteKit + Python hybrid service.
  • ~/westside-ai-assistant/.woodpecker.yaml — reference for an Ollama-backed Python service.

Constraints (learned the hard way)

  • YAML parse validation is mandatory. See feedback_yaml_parse_validation. Three repos were broken by unquoted colons in .woodpecker.yaml. Dev agents writing new pipelines must run python -c "import yaml; yaml.safe_load(open('.woodpecker.yaml'))" as part of the PR checklist.
  • The update-kustomize-tag step runs on both success and failure (when: status: [success, failure]) — never gate it only on success.
  • Harbor credentials come from a per-service Woodpecker secret, NOT a shared global secret. Each new service needs its own Harbor robot account scoped to its project namespace.
  • Kaniko build context must exclude .venv/, node_modules/, and .git/ via .dockerignore — forgetting this inflates the build context to GB-scale and times out.

New-service onboarding checklist

  • Add Dockerfile to the repo (see reference repos for language-appropriate shape)
  • Add .dockerignore
  • Add .woodpecker.yaml copied from a reference implementation, adapted to service name
  • Create Harbor robot account via harbor-robot.sh or the Harbor UI, scoped to the service's Harbor project
  • Add Harbor credentials as Woodpecker repo-scoped secrets
  • Create the kustomize overlay in pal-e-deployments/overlays/{service}/
  • Create the ArgoCD Application pointing at the overlay
  • First push triggers the pipeline — verify all 5 steps succeed end-to-end
  • feedback_ci_pipeline_lessons — 12 root causes fixed in the CI pipeline over time
  • feedback_yaml_parse_validation — mandatory YAML parse check
  • sop-ci-pipeline-recovery — triage runbook for CI failures
  • sop-platform-tf-changes — infra PR validation gate (requires kustomize build evidence)
  • Reference repo: forgejo_admin/basketball-api