pal-e-docs

Review: PrometheusRule alerts (blocked on #19 + #15)

review-1293-2026-06-02 Review

review needs-refinement
Project

Verdict: NEEDS_REFINEMENT

Template Completeness

  • [x] Type -- Feature
  • [x] Lineage
  • [x] Repo -- ldraney/pal-e-deployments
  • [x] User Story
  • [x] Context
  • [x] File Targets
  • [x] Acceptance Criteria
  • [x] Test Expectations
  • [x] Constraints
  • [x] Checklist
  • [x] Related
All template sections present. Issue is well-structured.

Traceability

  • [x] story:observability label -- present on board item
  • [ ] story note MISSING -- [SCOPE] The observability story does not exist in the project-landscaping-assistant user-stories table. Current stories are: address-lookup, property-crud, service-tracking, special-notes, work-queue, weekly-tracking, schedule-upload. Create user story entry on project-landscaping-assistant.
  • [x] arch:k8s-deploy label -- present on board item
  • [ ] arch note MISSING -- [SCOPE] No arch-k8s-deploy note found in pal-e-docs. Create architecture note arch-k8s-deploy for the K8s deployment component.
  • [x] Forgejo issue -- ldraney/landscaping-assistant#17, state: open
  • [x] parent:43 label -- parent is board item #1283 "Observability & DORA metrics stack" (backlog)

File Targets

  • [ ] overlays/landscaping-assistant/prod/prometheusrule.yaml -- ISSUE: File does not exist yet (expected, this is a new file). However, this file lives in ldraney/pal-e-deployments, NOT in ldraney/landscaping-assistant. See Repo Placement below.
  • [x] overlays/landscaping-assistant/prod/kustomization.yaml -- verified: exists in pal-e-deployments. Currently lists ../../../bases/standard as a resource with JSON patches for Deployment and Service renaming. Does not yet reference prometheusrule.yaml.

Repo Placement

MISMATCH CONFIRMED. The issue body correctly states ### Repo: ldraney/pal-e-deployments and the file targets are paths within that repo. However, the Forgejo issue is filed on ldraney/landscaping-assistant (issue #17). The board item links to landscaping-assistant/issues/17. An implementing agent working in the landscaping-assistant checkout will not find the overlays/ directory -- it does not exist in that repo.
[BODY] The Forgejo issue must be moved to ldraney/pal-e-deployments, or a new issue created there and the board item's forgejo_issue_url updated. Forgejo does not support cross-repo issue moves, so the practical fix is: create a new issue on pal-e-deployments, close this one with a reference, and update the board item.

Dependencies

  • #19 (Add /metrics endpoint) -- state: closed. Board item #1290 is in done. This dependency is resolved. PR #47 merged.
  • #15 (Fix ServiceMonitor label mismatch) -- state: open. Board item #1291 is in todo. This dependency is NOT resolved. The PrometheusRule alerts are useless without a working ServiceMonitor because Prometheus cannot scrape metrics. This is a hard blocker.
  • The title "blocked on #19 + #15" is partially stale -- #19 is done. Title should be updated to reflect only the remaining blocker (#15).

Acceptance Criteria

  • [x] "PrometheusRule resource created with alerts for high 5xx error rate, p95 latency spike, zero available pods" -- verifiable via kustomize build output
  • [x] "Alerts scoped to landscaping-assistant namespace via label matchers" -- verifiable
  • [x] "Alert severity labels use platform convention" -- verifiable but vague: the convention (warning vs critical) is stated in the AC itself, which is good
  • [x] "kustomize build renders the PrometheusRule cleanly" -- verifiable with run command
Issue with metric names: The AC says "high 5xx error rate (>5% over 5m)" and "p95 latency spike (>2s over 5m)". The actual yabeda-rails metrics exposed are rails_requests_total (counter with status tag) and rails_request_duration_seconds (histogram). The issue does not specify the PromQL expressions or reference the actual metric names. An agent will need to know:
  • Error rate: sum(rate(rails_requests_total{status=~"5.."}[5m])) / sum(rate(rails_requests_total[5m]))
  • Latency: histogram_quantile(0.95, sum(rate(rails_request_duration_seconds_bucket[5m])) by (le))
  • Pod availability: kube_deployment_status_replicas_available{deployment="landscaping-assistant"} (from kube-state-metrics, not yabeda)
[BODY] Add the actual PromQL expressions and metric names to the issue body so the implementing agent does not have to guess.

Blast Radius

Low blast radius. This creates a new standalone YAML file and adds one line to kustomization.yaml. No existing resources are modified. The only risk is alert noise from poorly tuned thresholds, which the Constraints section already acknowledges.

Decomposition Assessment

2 file targets in 1 repo, 4 acceptance criteria. Well within the 5-minute rule. No decomposition needed.

Recommendations

  • [BODY] Move or re-create the issue on ldraney/pal-e-deployments. The file targets live there, not in landscaping-assistant. Update board item #1293 forgejo_issue_url accordingly.
  • [BODY] Add PromQL expressions with actual yabeda-rails metric names (rails_requests_total, rails_request_duration_seconds_bucket, kube_deployment_status_replicas_available) to the issue body so the implementing agent has exact expressions to use.
  • [BODY] Update the title and context to remove the resolved #19 dependency. Only #15 (ServiceMonitor fix) remains as a blocker.
  • [SCOPE] Create user story entry "observability" on project-landscaping-assistant user-stories section. Multiple board items (8 items) reference story:observability but the story is not defined.
  • [SCOPE] Create architecture note arch-k8s-deploy for the Kubernetes deployment component in pal-e-docs.