pal-e-docs

Review: board-landscaping-observability#1313 -- Pyrra + Falco scope review

review-1313-2026-06-04 Doc

review block
Project

Verdict: BLOCK

Board item: board-landscaping-observability#1313 (backlog, 5 points)
Forgejo issue: ldraney/landscaping-assistant #90 -- "Deploy Pyrra for SLO tracking and Falco for runtime security"
Reviewer: QA Agent
Date: 2026-06-04

Template Completeness

Checked against template-issue-feature:
  • [x] Type -- Feature
  • [x] Lineage -- Child of #43, Phase 6 of observability roadmap
  • [ ] Repo -- States forgejo_admin/pal-e-platform but issue is filed in ldraney/landscaping-assistant. Mismatch.
  • [ ] User Story -- Combines two unrelated capabilities in one "I want" clause. "SLO tracking with error budgets" and "runtime container security monitoring" are distinct user needs for distinct personas (SRE vs Security).
  • [x] Context -- Present. Acknowledges "Two independent capabilities bundled" but does not justify why.
  • [ ] File Targets -- Three targets listed, all in pal-e-platform. SLO YAML location is ambiguous: "pal-e-platform or pal-e-deployments" is a question, not a spec.
  • [x] Acceptance Criteria -- 5 criteria covering both components.
  • [x] Test Expectations -- 3 items, all kubectl-based manual verification. No automated tests.
  • [ ] Constraints -- "SLO targets need to be decided per-service before deployment" is an unresolved prerequisite, not a constraint. This is a blocker hiding as a constraint.
  • [x] Checklist -- Present.
  • [x] Related -- References project-pal-e-platform and parent #43.

Traceability

  • [x] story:observability -- present on board item labels
  • [x] arch:platform -- present on board item labels
  • [x] Forgejo issue -- #90 exists and is open
  • [x] parent:43 -- present on board item labels, matches Lineage
Traceability triangle is intact for the board item. However, the traceability to the platform plan is broken (see Decomposition Assessment below).

File Targets

  • [ ] terraform/pyrra.tf (pal-e-platform) -- Cannot verify from landscaping-assistant repo. File would be new. Parent directory terraform/ exists in pal-e-platform per project page.
  • [ ] terraform/falco.tf (pal-e-platform) -- Same. New file in pal-e-platform.
  • [ ] SLO YAML definitions -- Location undecided ("pal-e-platform or pal-e-deployments"). Agent cannot act on this without a decision.
Assessment: Targets are in a different repo than where the issue is filed. An agent dispatched against landscaping-assistant would not find these files.

Repo Placement

PROBLEM: The issue is filed in ldraney/landscaping-assistant but all work targets are in forgejo_admin/pal-e-platform. This is a cross-repo mismatch. The issue body even says Repo: forgejo_admin/pal-e-platform. Either:
  • Move the issue to pal-e-platform (where the Terraform files live), or
  • Make #90 a tracking issue in landscaping-assistant with child issues in pal-e-platform (but this adds unnecessary indirection for what is fundamentally platform work)

Dependencies

Per plan-pal-e-platform, the two components have distinct dependency chains:
  • [ ] SLO engine (Phase 16) depends on Phase 14 (synthetic monitoring) and Phase 15 (DORA re-baseline) -- status of both is unclear from available data
  • [ ] Falco (Phase 20b) depends on Phase 10 (vulnerability scanning -- COMPLETED) and Phase 19 (Kyverno -- NOT STARTED)
The plan explicitly states "Tier 1.5 gates Tier 2." Falco is Tier 2. Phase 19 (Kyverno) is a prerequisite for Phase 20 and is NOT STARTED. The issue does not acknowledge or justify skipping this dependency.

Acceptance Criteria

Evaluating each AC:
  • "Pyrra deployed and generating PrometheusRules from SLO definitions" -- Testable via kubectl get prometheusrules. Specific enough.
  • "Grafana dashboard shows error budget remaining and burn rate" -- Testable but requires manual verification. No dashboard JSON path specified.
  • "At least one SLO defined (e.g., landscaping-assistant 99.5% availability)" -- Testable. But the constraint says "SLO targets need to be decided per-service before deployment" -- so this AC depends on an unresolved prerequisite.
  • "Falco DaemonSet running and detecting test anomaly" -- Testable via kubectl exec test. Specific enough.
  • "Falco alerts route through Alertmanager to Telegram" -- Testable. Requires Alertmanager config changes not listed in file targets.
AC quality is adequate individually, but conflated across two independent capabilities.

Blast Radius

  • Repos touched: 1-2 (pal-e-platform, possibly pal-e-deployments for SLO YAMLs)
  • Services affected: All (Falco DaemonSet runs on every node; SLO definitions reference all monitored services)
  • Risk: Falco DaemonSet is medium-risk -- it monitors syscalls on every node. A misconfigured Falco can generate alert storms or consume node resources. SLO engine is low-risk -- namespace-scoped deployment.
  • Rollback: Both are independent Helm releases, so rollback is straightforward per-component. But bundling them in one PR makes selective rollback harder.

Decomposition Assessment

Three-thing limit: VIOLATED. This ticket has at least 5 discrete changes:
  • Deploy Pyrra/Sloth Helm release
  • Write SLO YAML definitions
  • Create Grafana SLO dashboard
  • Deploy Falco Helm release (DaemonSet)
  • Configure Falco alerting through Alertmanager
Five-minute rule: VIOLATED. An agent would need substantial time to: resolve the Sloth/Pyrra decision, decide SLO YAML location, write SLO definitions for services, create Grafana dashboard JSON, deploy and test two independent Helm releases, and configure Alertmanager routing.
Critical conflict -- Sloth vs Pyrra:
  • The platform plan Phase 16 (phase-pal-e-platform-16-slo-error-budgets) is titled "SLO Governance (Sloth)" and explicitly scopes deploying Sloth.
  • The plan-pal-e-platform summary references "SLO governance/Sloth (16)".
  • The validation pipeline diagram in project-pal-e-platform shows "Sloth -- SLO YAML to Recording Rules".
  • This issue says "Pyrra uses Prometheus-native SLOs (not Sloth)" -- directly contradicting the plan.
  • The docs/observability-roadmap.md in landscaping-assistant references Pyrra.
Two sources of truth disagree. An agent cannot execute this without a human decision.
Recommendation: Split into two issues aligned with existing plan phases:
  • Issue A: SLO engine -- aligned with Phase 16. Filed in pal-e-platform. Resolve Sloth vs Pyrra first.
  • Issue B: Falco runtime security -- aligned with Phase 20b. Filed in pal-e-platform. Acknowledge Kyverno (Phase 19) dependency.

Helm Chart Verification

  • Pyrra: Official Helm chart repo is pyrra-dev/helm-charts (not pyrra-dev/pyrra which is the project repo). Chart name: pyrra. ArtifactHub listing exists.
  • Falco: Official Helm chart repo is falcosecurity/charts (not falcosecurity/falco which is the project repo). Chart name: falco. Current version 9.0.0 confirmed.
The observability-roadmap.md references project repos, not chart repos. Terraform helm_release blocks will need the correct chart repository URLs.

Recommendation

  • Resolve Sloth vs Pyrra. Human decision required. Update the losing side's documentation (either Phase 16 + project-pal-e-platform validation diagram, or observability-roadmap.md + this issue).
  • Split into two issues aligned with existing plan phases: Phase 16 (SLO engine) and Phase 20b (Falco). File both in forgejo_admin/pal-e-platform.
  • Address dependency chains in each new issue. Phase 16 depends on 14+15. Phase 20b depends on 10+19. Either complete prerequisites or document why they can be skipped.
  • Decide SLO YAML location before the SLO issue moves forward. "pal-e-platform or pal-e-deployments" is not actionable.
  • Add Alertmanager config to file targets for the Falco issue -- routing Falco alerts requires Alertmanager config changes not currently listed.
  • plan-pal-e-platform -- authoritative platform hardening plan
  • phase-pal-e-platform-16-slo-error-budgets -- Phase 16: SLO Governance (Sloth)
  • phase-platform-20-security-deepening -- Phase 20: Security Deepening (Falco is subphase 20b)
  • template-issue-feature -- issue template validated against
  • template-review -- review template used for this note