pal-e-mail

Vision

Extract all email concerns from app repos into a platform service. Any project calls POST /send. The service handles templates, rendering, Gmail OAuth, and logging. Same tier as Keycloak (auth), MinIO (storage), Woodpecker (CI) — platform infrastructure that all projects consume.

Projects & Repos Touched

Context

Email sending embedded in basketball-api (825 lines of Python: inline HTML, Gmail OAuth, brand wrappers, logging). March 21 jersey email session hit every anti-pattern: ConfigMap template delivery, read-only secret mount blocking token refresh, IPv6-only DNS breaking Gmail API, manual image deploys. Stopgap built (MJML templates in westside-emails, thin sender in basketball-api) but architecture is wrong — email logic shouldn't live in app repos.

What already exists:

• gmail-sdk-ldraney v0.1.3 on Forgejo PyPI — OAuth token load/refresh/save, Gmail API send
• westside-emails repo — MJML templates with partials, build script
• MinIO CDN — assets bucket, public-read for images
• minio-api — deployed, upload endpoints
• _brand_wrapper() in basketball-api — branded email shell
• EmailLog model in basketball-api — logs sent emails

Previous Plan

None. First plan for pal-e-mail project.

Depends On

No plan dependencies. gmail-sdk and MinIO CDN already deployed.

Decisions Made

Phases

• Phase 1: Service Scaffold + Deploy (COMPLETED — pod healthy, pipeline green, ArgoCD synced)
• Phase 2: Core Send API (COMPLETED — PR #4 merged, 28 tests, POST /send + GET /log)
• Phase 2b: Scheduled Send (NOT STARTED — send_at field, scheduled_emails table, CronJob worker, cancel endpoint)
• Phase 2c: Sender Registry + Self-Service Re-Auth (NOT STARTED — senders table in Postgres, onboard/status/reauth endpoints, token health monitoring)
• Phase 3: Template CI Pipeline (NOT STARTED — can parallel with Phase 2b/2c)
• Phase 4: Basketball-api Migration (NOT STARTED — depends on 2+3)
• Phase 5: Admin Dashboard + Monitoring (NOT STARTED — depends on 2)

Key Files

Verification

• [ ] curl https://pal-e-mail.tail5b443a.ts.net/healthz → 200
• [ ] POST /send with template mode → email received with images
• [ ] POST /send with custom HTML → email received in brand layout
• [ ] Push to westside-emails → template at CDN URL within 60s
• [ ] basketball-api sends all email types through pal-e-mail
• [ ] GET /log?sender=westside returns send history
• [ ] Gmail OAuth token refreshes automatically
• [ ] Second sender onboarded without code changes

Epilogue

From Phase 1:
Discovered scope:

• pal-e-platform#143 — ArgoCD DERP hairpin: all ArgoCD apps should use internal Forgejo URL (http://forgejo-http.forgejo.svc.cluster.local). ArgoCD repo-server can't reach external Tailscale funnel URLs reliably. Image Updater also affected (can't reach Harbor). Patched pal-e-mail app to use internal URL; others still use external. Same class of issue as Woodpecker CI clone fix.
• Forgejo NetworkPolicy missing argocd namespace in ingress allow list — kubectl patched, terraform fix pending in network-policies.tf
• First-deploy image tag must be full 40-char SHA (CI pushes full SHA via ${CI_COMMIT_SHA}, not 7-char short)

• Missing [build-system] table in pyproject.toml
• Module-level engine creation in database.py (standard but noted for test isolation)
• default vs server_default on EmailLog.status
• Missing ArgoCD .argocd-source-* path exclude in .woodpecker.yaml
• Duplicate httpx dep (in both deps and dev deps)

• pal-e-platform#144 — MinIO network policy: add pal-e-mail namespace to MinIO ingress allow list. Without this, template CDN fetching from minio.minio.svc.cluster.local:9000 is blocked. Same DERP hairpin class — must use internal URL.

• body_html in brand_wrapper() inserted raw — add docstring clarifying trust boundary (callers responsible for sanitization; current callers are CDN templates and API-provided HTML)
• html_to_plain_text() doesn't decode HTML entities (— stays literal). Future improvement for email preview quality.
• Stale noqa: E501 on brand_wrapper signature (template.py:45) — line is under 120 chars, suppression can be removed
• pytest-asyncio in dev dependencies (pyproject.toml:19) — not used, all code is sync. Remove.
• data: dict[str, str] = {} in SendRequest — safe in Pydantic v2 but Field(default_factory=dict) is more explicit

Related

• service-onboarding-sop — the deployment SOP this plan follows
• convention-kustomize-overlay — kustomize overlay pattern
• deployment-lessons — operational lessons from prior service deploys
• plan-wkq — Westside plan (Phase 11 jersey email was the catalyst)

Vision

Centralized, multi-tenant email service for the Pal-E platform. Any project sends branded emails via one API call. Email content lives here — not in app repos. Sending infrastructure (Gmail OAuth, token refresh, delivery logging) is platform-level, not duplicated per project. Templates compile from MJML repos and deploy to MinIO CDN. Custom composed emails wrap in project-specific brand layouts. Self-hosted, zero SaaS dependencies.

User Stories

Consumer Projects

Projects that send email through pal-e-mail:

Plan

Active: plan-pal-e-mail (to be created)

Board

board-pal-e-mail (to be created)

Status

Phase 2 complete. Core Send API merged (PR #4). POST /send supports template mode (CDN fetch + {{var}} replacement), custom HTML mode (brand wrapper), and plain text. GET /log provides cross-project queryable email history with filtering and pagination. 28 tests, Woodpecker CI green. Phase 1 deployed: pod healthy, ArgoCD synced, Postgres with email_log table, Gmail OAuth PVC. MinIO network policy applied (pal-e-platform#144 — merged). Template CDN fetching unblocked.

Milestones

None yet.

Architecture

System Overview

Any App API                          pal-e-mail service
  POST /send ──────────────────────► FastAPI
  {sender, to, template, data}         │
                                       ├─► Fetch template from MinIO CDN (public URL)
                                       │   https://minio-api.ts.net/assets/email-templates/{project}/
                                       │
                                       ├─► String replace {{variables}}
                                       │
                                       ├─► Load Gmail OAuth creds for sender
                                       │   /secrets/gmail/{sender}.json
                                       │
                                       ├─► Gmail API send
                                       │
                                       └─► Log to email_log table (project, to, template, status)

Key Architectural Decisions

• Templates on MinIO CDN, not ConfigMaps — MJML CI compiles and uploads to assets/email-templates/{project}/. Public-read. Service fetches via HTTP at send time. No k8s volume mounts, no pod restarts for template changes.
• minio-api for uploads, CDN for reads — CI uploads templates through authenticated minio-api. Service reads via public CDN URL. Clean separation.
• Multi-tenant Gmail OAuth — one token file per sender identity. Service loads the right credentials based on sender field. Tokens on writable volume for refresh.
• Template OR custom HTML — POST /send accepts either template (fetched from CDN) or html (raw HTML, wrapped in brand layout). Supports both templated and ad-hoc emails.
• No Jinja2, no template engine — simple {{variable}} string replacement. MJML handles the rendering. Python handles the data injection.
• Westside-emails migrates in — existing westside-emails repo continues to own MJML source. CI uploads compiled HTML to MinIO. pal-e-mail fetches from there.

Repos

Inbox

Query: list_board_items(board_slug="board-pal-e-mail", column="backlog")