pal-e-docs

DORA Framework: Platform Axiom

dora-framework Doc

active reference platform axiom
Project

DORA Framework: The Measure of a DORA Elite AI Enterprise

Status: Re-baselined 2026-03-14 with Prometheus data (DORA exporter + Grafana dashboard LIVE). Previous manual baseline: 2026-03-01. Confidence upgraded from Low-Medium to Medium-High.

The Axiom

DORA is the reason this platform exists. Every plan, every capability in the maturity matrix, every SOP exists to move one of four numbers. If it doesn't move a DORA metric, it doesn't matter.
The maturity matrix is the means. DORA is the measure.
The platform thesis: one human architect + AI agents operating within an enterprise-grade enforcement system can achieve and sustain DORA Elite delivery performance. The scoping pipeline (projects → plans → phases → kanban items → issues) eliminates coordination overhead. The enforcement architecture (conventions → SOPs → hooks) makes compliance deterministic. The four metrics prove or disprove this claim. Everything else is commentary.

The Four Metrics

Metric What It Measures Why It Matters for This Platform
<strong>Deployment Frequency (DF)</strong> How often code reaches production Proves the agent workforce can ship continuously
<strong>Lead Time for Changes (LT)</strong> Time from commit to production Proves the pipeline eliminates human bottlenecks
<strong>Change Failure Rate (CFR)</strong> % of deployments causing failures Proves quality gates (QA agents, CI, review loops) work
<strong>Mean Time to Recovery (MTTR)</strong> Time from failure detection to recovery Proves the system is resilient, not just fast

DORA Bands (Industry Standard)

Band DF LT CFR MTTR
<strong>Elite</strong> On-demand (multiple/day) &lt; 1 day 0-5% &lt; 1 hour
<strong>High</strong> Once/day to once/week 1 day - 1 week 5-10% &lt; 1 day
<strong>Medium</strong> Once/week to once/month 1 week - 1 month 11-15% 1 day - 1 week
<strong>Low</strong> &lt; once/month 1 month - 6 months 16-30% 1 week+

What "Deployment" Means Per Project

This is the critical definition. DORA only works if "deployment" is defined consistently.
Project Deployment Event Pipeline Fully Automated?
<strong>pal-e-platform</strong> Woodpecker CI: plan-on-PR, <code>tofu apply</code> on merge PR → Woodpecker plan → merge → Woodpecker apply <strong>Yes</strong> (since Phase 6, 2026-03-14)
<strong>pal-e-services</strong> <code>tofu apply</code> completes successfully Manual (laptop) — CI planned No — planned
<strong>pal-e-deployments</strong> ArgoCD sync completes from Kustomize change Git push → ArgoCD auto-sync Yes
<strong>pal-e-docs</strong> Successful Woodpecker build → Harbor push → ArgoCD sync → pod running Push → Woodpecker → Harbor → ArgoCD Yes
<strong>basketball-api</strong> Successful Woodpecker build → Harbor push → ArgoCD sync Push → Woodpecker → Harbor → ArgoCD Yes
<strong>westside-app</strong> Container image deployed via ArgoCD Manual build+push → ArgoCD (CI broken — K8s backend bug) Partial
<strong>MCP services</strong> Container image deployed via ArgoCD (same as pal-e-docs pattern) Push → Woodpecker → Harbor → ArgoCD Yes

Baseline: 2026-03-01 (Manual Measurement — Historical)

Measurement period: pal-e-docs: 2026-02-23 to 2026-02-27 (4 active days). pal-e-platform: 2026-02-19 to 2026-03-01 (6 active days, 20 commits).
Method: git log --first-parent main for commit counts, Woodpecker MCP list_pipelines for CI data, bug notes in pal-e-docs for incidents.
This section preserved for historical comparison. See Re-Baseline 2026-03-14 below for current data.

App DORA (pal-e-docs — our most mature pipeline)

Raw data:
  • 40 total first-parent commits to main in 4 days
  • 15 were ArgoCD Image Updater auto-commits ("build: automatic update") — excluded from deployment count
  • 25 human-triggered commits to main in 4 days = 6.25/day
  • 17 Woodpecker push pipelines visible in API: 10 success, 7 failure
  • 7 CI failures were all during playwright/ruff setup period (pipelines #32-#36, #42, #54, #63)
  • 1 production incident: pal-e-docs Alembic crash (CrashLoopBackOff from bad migration)
Metric Measured Value DORA Band Confidence Notes
Deployment Frequency 10 successful deploys in 4 days (2.5/day) <strong>Elite</strong> High — from Woodpecker pipeline data "Deploy" = successful Woodpecker push build that reaches production via ArgoCD
Lead Time PR open → merge → deploy in &lt;1 hour typically <strong>Elite</strong> Medium — estimated from git timestamps, not measured precisely Agent creates PR, QA reviews, merge triggers build + ArgoCD sync. Need PR timestamp data for precise measurement.
Change Failure Rate (CI-gated) 7 failed push pipelines / 17 total = 41% <strong>Low</strong> High — from Woodpecker data These are CI gate catches, not production outages. Most during playwright/ruff setup.
Change Failure Rate (production) 1 production incident / 10 successful deploys = 10% <strong>High</strong> High — from bug notes + pipeline count 1 Alembic crash out of 10 deployments that reached production.
MTTR Follow-up fix commits within hours <strong>High</strong> Low — estimated, no alerting to measure detection time No automated detection. MTTR starts when Lucas notices. Recovery is fast once detected.

Infra DORA (pal-e-platform) — 2026-03-01

Raw data:
  • 20 commits to main across 6 active days (Feb 19, 20, 23, 25, 27, Mar 1)
  • No CI pipeline at this time — all deploys were manual tofu apply
  • Actual tofu apply frequency: ESTIMATED at ~2-3/week (no apply log existed)
  • 2 production incidents: Grafana CrashLoopBackOff, pal-e-docs Alembic crash
Metric Measured Value DORA Band Confidence Notes
Deployment Frequency ~2-3 tofu apply per week (estimated) <strong>Medium</strong> Low — no apply log, this is an estimate Commits are frequent (3.3/day) but actual infra deploys are batched
Lead Time Hours to days (PR → review → manual apply) <strong>Medium</strong> Low — estimated The manual apply step is the bottleneck. No timestamp data.
Change Failure Rate 2 incidents / ~20 commits = ~10% <strong>High</strong> Medium — incident count is exact, commit count is exact, but not all commits are deploys Grafana CrashLoopBackOff + pal-e-docs Alembic crash
MTTR Hours (manual detection, manual remediation) <strong>Medium</strong> Low — no alerting, detection time unknown No alerting = MTTR starts when Lucas notices

Infra DORA (pal-e-services) — 2026-03-01

Raw data:
  • 17 commits to main across Feb 19 - Mar 1
  • No documented production incidents
  • Same manual apply bottleneck as pal-e-platform
Metric Measured Value DORA Band Confidence Notes
Deployment Frequency ~1-2 tofu apply per week (estimated) <strong>Medium</strong> Low — estimate Same manual apply bottleneck
Lead Time Hours to days <strong>Medium</strong> Low — estimate Same pattern
Change Failure Rate 0 documented incidents <strong>Elite</strong> Low — small sample, no incidents may reflect luck not quality No recorded failures
MTTR N/A N/A N/A No incidents to measure

Agent DORA (unique to pal-e) — 2026-03-01

Traditional DORA measures a team. Agent DORA measures the AI agent workforce.
IMPORTANT: These were estimates based on observation, not measured from data. See Re-Baseline 2026-03-14 for real numbers.
Metric Estimated Value Confidence Notes
PRs shipped per day (across all repos) ~3-5/day during active sessions Low — estimate from observation Needs PR timestamp data from Forgejo API
Rework rate (QA review iterations) ~1-2 iterations per PR Low — estimate Needs PR comment/review event data
Plan-to-ship time (phase active → merged) 1-3 sessions (hours to days) Low — estimate Needs plan phase timestamps from pal-e-docs
Autonomy ratio ~80% autonomous, 20% Lucas decision gates Low — qualitative estimate Agents own: plan → issue → branch → code → PR → review. Lucas owns: approve merge, approve plan, tofu apply.

Re-Baseline: 2026-03-14 (Prometheus Data)

Measurement period: All-time cumulative from repo creation (~2026-02-19) through 2026-03-14 (23 active days).
Method: Prometheus metrics from DORA exporter (dora_deployments_total, dora_pr_merges_total, dora_pr_lead_time_seconds_bucket). Exporter pulls from Forgejo API (PR data) and Woodpecker API (pipeline data). Scraped every 60s by Prometheus ServiceMonitor.
What changed since 2026-03-01:
  • DORA exporter deployed — real-time metrics collection from Forgejo + Woodpecker APIs
  • Grafana DORA dashboard live — 4 metric panels with per-repo drill-down
  • pal-e-platform CI pipeline operational — plan-on-PR, apply-on-merge (Phase 6 completed)
  • Alerting operational — Telegram + Slack receivers, pod/node/target health rules
  • Woodpecker migrated to Postgres (CNPG) — reliable pipeline data storage
  • 262 PRs merged across 30 repos (up from ~40 at first baseline)

Platform-Wide PR Velocity (from dora_pr_merges_total)

Repo PRs Merged Lead Time p50 Lead Time p95
<strong>pal-e-docs</strong> 50 9 min 5.0 hours
<strong>claude-custom</strong> 41 11 min 1.9 hours
<strong>basketball-api</strong> 36 6 min 3.2 hours
<strong>pal-e-platform</strong> 30 12 min 6.0 hours
<strong>pal-e-docs-mcp</strong> 17 8 min 34 min
<strong>pal-e-docs-sdk</strong> 14 8 min 5.2 hours
<strong>platform-validation</strong> 8 2 min 5 min
<strong>pal-e-app</strong> 7 10 min 6.6 hours
<strong>westside-app</strong> 5 10 min 3.5 hours
Other 21 repos 54 ~20 min varies
<strong>TOTAL</strong> <strong>262</strong> <strong>~10 min (core)</strong> <strong>~4 hours (core)</strong>

App DORA (pal-e-docs — re-baselined)

Metric Measured Value DORA Band Confidence Source
Deployment Frequency 50 PRs merged / 20 days = 2.5/day <strong>Elite</strong> High — from <code>dora_pr_merges_total</code> Merge = deploy (Woodpecker → Harbor → ArgoCD)
Lead Time p50 = 9 min, p95 = 5.0 hours <strong>Elite</strong> <strong>High</strong> — from <code>histogram_quantile(dora_pr_lead_time_seconds_bucket)</code> PR open → merge timestamp from Forgejo API. Now measured, not estimated.
Change Failure Rate (production) 1 incident (Alembic crash) / 50 deploys = 2% <strong>Elite</strong> High — incident count from bug notes, deploy count from Prometheus Zero production incidents since Alembic crash was fixed.
MTTR Detection: &lt;5 min (alerting). Fix: &lt;1 hour (agent ships patch) <strong>Elite</strong> Medium — alerting operational but limited incident sample PodRestartStorm + OOMKilled alerts → Telegram. Detection is now automated.

Infra DORA (pal-e-platform — re-baselined)

Key change: Woodpecker CI now operational (Phase 6 completed 2026-03-14). Plan-on-PR validates tofu plan before merge. Apply-on-merge runs tofu apply automatically. Merge = deploy.
Metric Measured Value DORA Band Confidence Source
Deployment Frequency 30 PRs merged / 23 days = 1.3/day <strong>Elite</strong> High — from <code>dora_pr_merges_total</code> Merge = deploy now (Woodpecker apply-on-merge). Was Medium (manual apply bottleneck).
Lead Time p50 = 12 min, p95 = 6.0 hours <strong>Elite</strong> <strong>High</strong> — from <code>histogram_quantile(dora_pr_lead_time_seconds_bucket)</code> Was Medium (hours to days). CI eliminated manual apply bottleneck.
Change Failure Rate 1 CI failure / 4 completed Woodpecker pipelines = 25% <strong>Medium</strong> Low — very small sample (fresh Woodpecker DB, only 4 pipelines since migration) From <code>dora_deployments_total</code>. Sample too small to be meaningful — will stabilize over next 2 weeks.
MTTR Detection: automated (alerting). Recovery: same session. <strong>High</strong> Medium — alerting exists, limited incident sample post-alerting Telegram alerts for pod health, node health, target down. Was Medium (no alerting).

Infra DORA (pal-e-services — unchanged)

No CI pipeline yet. Same manual apply bottleneck. Numbers unchanged from 2026-03-01 baseline. Will improve when pal-e-services gets Woodpecker CI.

Agent DORA (re-baselined with real data)

The DORA exporter now collects PR data from the Forgejo API. These are real measurements, not estimates.
Metric Measured Value DORA Band Confidence Source
PRs shipped per day (all repos) 262 total / 23 days = <strong>11.4/day</strong> <strong>Elite</strong> High — from <code>dora_pr_merges_total</code> Was estimated "~3-5/day." Actual is 2-3x higher. Includes agent + human PRs.
PR Lead Time p50 (core repos) <strong>~10 minutes</strong> <strong>Elite</strong> High — from <code>dora_pr_lead_time_seconds_bucket</code> Agent creates PR → QA reviews → merge. Median under 15 min for all core repos.
Rework rate Still estimated: ~1-2 iterations per PR High Low — no automated collection yet Needs PR comment/review event data from Forgejo API
Autonomy ratio ~90% autonomous (up from 80%) (new) Medium — qualitative but informed by CI automation tofu apply now automated (was manual gate). Lucas gates: approve merge, approve plan.

Composite DORA Standing

Dimension DF LT CFR MTTR Overall Band Confidence
App Pipeline Elite Elite Elite Elite <strong>Elite</strong> High (all metrics from Prometheus)
Infra Pipeline Elite Elite Medium High <strong>High</strong> Medium (CFR sample too small, will stabilize)
Agent Velocity Elite Elite High High <strong>Elite</strong> Medium (DF+LT from data, CFR+MTTR estimated)
<strong>Platform Overall</strong> <strong>Elite</strong> <strong>Elite</strong> <strong>High</strong> <strong>High</strong> <strong>High-Elite</strong> <strong>Medium-High</strong>
Honesty check: Confidence upgraded from Low-Medium to Medium-High. DF and LT are now measured from Prometheus with high confidence. CFR for infra pipeline has a very small sample (4 Woodpecker pipelines since DB migration) — this will stabilize over the next 2 weeks. MTTR is rated based on alerting capability (operational) but limited post-alerting incident sample. The biggest remaining gap: CFR needs more pipeline data and production incident tracking.
Delta from 2026-03-01 baseline:
Dimension Was Now Change
App Pipeline High <strong>Elite</strong> ↑ CFR improved (2% vs 10%), MTTR now automated
Infra Pipeline Medium <strong>High</strong> ↑↑ DF and LT jumped from Medium to Elite (CI automated)
Agent Velocity High <strong>Elite</strong> ↑ Real data shows 11.4 PRs/day, not 3-5
Platform Overall Medium-High <strong>High-Elite</strong> ↑↑ Every dimension improved
Confidence Low-Medium <strong>Medium-High</strong> ↑↑ Prometheus data replaces estimates

What Moves Each Number

Every existing plan maps to DORA metrics. This is the bridge between the maturity matrix (means) and DORA (measure).
Plan DF LT CFR MTTR How
<code>plan-tf-ci-team-hardening</code> ++ +++ + + Automated apply eliminates manual bottleneck (LT). CI gates catch bad changes (CFR). Pipeline enables frequent deploys (DF).
<code>plan-platform-observability</code> + +++ Alerting enables fast detection (MTTR). Dashboards surface regressions (CFR). Deployment protection prevents outages (CFR).
<code>plan-kustomize-service-bases</code> ++ ++ + Standardized deploys reduce onboarding friction (DF, LT). Consistent bases reduce misconfiguration (CFR).
<code>plan-network-security-hardening</code> ++ NetworkPolicies and ACLs are guardrails that prevent blast radius from bad changes (CFR).
<code>plan-environment-isolation</code> + +++ Dev cluster lets you break things safely. Promotion gates prevent bad changes reaching prod (CFR).
<code>plan-woodpecker-mcp</code> + + + Agent-operated CI enables autonomous pipeline management (DF, LT, MTTR).
<code>plan-schema-api-mcp</code> + + Mature APIs reduce integration errors (CFR). Better MCP = better agent autonomy (LT).
<code>plan-knowledge-system-consolidation</code> + + Better knowledge = agents start faster (LT). Queryable incidents improve future MTTR.
<code>plan-mcp-gateway-migration</code> + + + Centralized MCP reduces service sprawl (DF). Gateway pattern reduces per-service failure modes (CFR).

DORA Targets

Dimension Baseline (Mar 01) Current (Mar 14) Target Band Target Date Key Enabler
App Pipeline High <strong>Elite ✓</strong> Elite Q2 2026 <strong>TARGET MET.</strong> Stabilize CFR below 5% — currently at 2%.
Infra Pipeline Medium <strong>High ✓</strong> Elite Q2 2026 <strong>Original target (High) MET.</strong> New target: Elite. Need CFR data to stabilize (more pipeline runs).
Agent Velocity High <strong>Elite ✓</strong> Elite Q3 2026 <strong>TARGET MET EARLY.</strong> 11.4 PRs/day measured. Need rework rate data for full picture.
<strong>Platform Overall</strong> <strong>Medium-High</strong> <strong>High-Elite</strong> <strong>Elite</strong> <strong>Q2 2026</strong> <strong>On track.</strong> Infra CFR needs more data. Synthetic monitoring (Phase 14) will close MTTR gap.

The Agent DORA Dimension

Traditional DORA measures a human team. This platform's thesis is that AI agents ARE the team. Agent DORA extends the framework:
Agent Metric Analogous DORA Metric What It Measures Data Source Status
PRs shipped per day Deployment Frequency Agent throughput Forgejo API (PR merge events) <strong>LIVE</strong> — <code>dora_pr_merges_total</code>
PR lead time Lead Time Full cycle: PR open → merge Forgejo API (PR timestamps) <strong>LIVE</strong> — <code>dora_pr_lead_time_seconds_bucket</code>
Plan-to-ship time Lead Time Full cycle: plan → issue → code → review → merge pal-e-docs plan phases + Forgejo PR timestamps NOT YET — needs cross-system correlation
Rework rate Change Failure Rate QA review iterations before clean pass Forgejo PR comments + review events NOT YET — needs PR review event collection
Autonomy ratio (new) % of pipeline requiring human intervention Count human gates vs autonomous steps Qualitative (~90%)
Agent incident rate MTTR How often agent-created code causes incidents Bug notes in pal-e-docs linked to agent PRs NOT YET — needs bug-to-PR linking
Why this matters: Agent DORA is now partially measurable. The DORA exporter provides real PR velocity and lead time data across all 30 repos. The remaining gaps (rework rate, plan-to-ship time, agent incident rate) require Forgejo PR review events and cross-system correlation with pal-e-docs. These are future phase candidates.

Measurement Automation Roadmap

Phase What Gets Automated Status Depends On
Manual baseline Git log + Woodpecker API queries. Calculated per session. <strong>COMPLETED</strong> (2026-03-01) Nothing
DORA Exporter Prometheus metrics: <code>dora_deployments_total</code>, <code>dora_pr_merges_total</code>, <code>dora_pr_lead_time_seconds_bucket</code>. Scraped every 60s. <strong>LIVE</strong> (Phase 4) Forgejo API + Woodpecker API
Grafana DORA Dashboard 4 metric panels with per-repo drill-down. Historical trends. <strong>LIVE</strong> (Phase 4) DORA Exporter
TF CI Pipeline Infra DORA: pipeline data → Prometheus metrics. Merge = deploy. <strong>COMPLETED</strong> (Phase 6) Woodpecker CI + Postgres
Alerting Pod health, node health, target down → Telegram + Slack. <strong>LIVE</strong> (Phase 3) Prometheus + Alertmanager
Synthetic monitoring HTTP uptime probes for all Tailscale funnel endpoints. <strong>PLANNED</strong> (Phase 14) Blackbox Exporter
Agent DORA expansion Rework rate, plan-to-ship time, agent incident rate. FUTURE Forgejo PR review events + pal-e-docs API correlation

Measurement Methodology

Current method (2026-03-14):
  • Deployment Frequency: dora_pr_merges_total from Prometheus (sourced from Forgejo API). For repos with CI, merge = deploy. Also dora_deployments_total from Woodpecker pipeline events. Both metrics available per-repo via Grafana dashboard.
  • Lead Time: histogram_quantile(0.5, dora_pr_lead_time_seconds_bucket) for p50, histogram_quantile(0.95, ...) for p95. Computed from PR open → merge timestamps via Forgejo API. Measured precisely, not estimated.
  • Change Failure Rate: dora_deployments_total{status="failure"} / sum(dora_deployments_total) from Woodpecker pipeline data. Production CFR also tracked via bug notes in pal-e-docs. Note: Woodpecker DB was migrated to Postgres on 2026-03-14, so pipeline data is limited to post-migration runs.
  • MTTR: Detection time now automated via Prometheus alerting (PodRestartStorm, OOMKilled, DiskPressure, TargetDown). Recovery time still tracked manually via incident notes. Full automation requires incident management integration (Phase 12).
  • Agent metrics: PR velocity and lead time are automated via DORA exporter. Rework rate and plan-to-ship time still manual.
Remaining gaps:
  • Woodpecker pipeline data is limited (fresh DB since Postgres migration 2026-03-14) — CFR will stabilize over 2 weeks
  • No synthetic uptime monitoring — MTTR for endpoint-level failures requires Blackbox Exporter (Phase 14)
  • Agent rework rate not collected — needs Forgejo PR review event integration
  • No SLO/error budget framework — planned in Phase 16
  • No distributed tracing — APM gap, planned in Phase 17
  • plan-pal-e-agency — A DORA Elite AI Enterprise Operating Model
  • project-pal-e-agency — project page with architecture diagrams and enforcement architecture
  • agent-workflow — the operating model (affects Agent DORA metrics)
  • plan-pal-e-platform — Platform Hardening (observability enables DORA measurement)
  • platform-maturity-matrix — the means (capabilities). DORA is the measure.
  • milestone-2026-03-14-woodpecker-postgres-dora-pipeline — milestone documenting the Woodpecker migration + DORA pipeline deployment
  • todo-token-metrics-dora-correlation — correlate token usage with DORA and boards