Hyprland Observability

hyprland-observability

Notes

Project Page 1
  • Hyprland Observability project-hyprland-observability

    Vision

    Full observability of the Hyprland workstation — host metrics, GPU thermals, and compositor state — feeding into the existing Prometheus/Grafana stack on the pal-e cluster. The workstation should be as visible as any cluster node or CI agent.

    Additionally, this board tracks the Pal-E World gaming platform — cloud game streaming via GOW containers and Sunshine/Moonlight, with the Hyprland workstation as both the host and the dogfooding client.

    User Stories

    Key Story Note Role Success Metric
    host-health TBD Platform operator CPU, memory, disk, network metrics visible in Grafana with 30s resolution
    gpu-thermals TBD Gamer / developer GPU temp, utilization, VRAM, clock speeds in Grafana; alert on thermal throttle
    wm-state TBD Power user Active windows, workspace usage, focused app tracked over time
    dora-verified TBD Platform operator DORA dashboard populated with real data from Forgejo + Woodpecker
    gaming-stability TBD Gamer Steam/Palworld runs stable without overlay crashes or VRAM exhaustion
    pal-e-world project-pal-e-world Gamer / platform operator Palworld playable via Moonlight streaming from GOW containers on k8s; session switching automated via Salt

    Architecture

    Three layers of metrics collection on the Arch workstation, all scraped by in-cluster Prometheus via Tailscale:

    1. node-exporter — standard host metrics (CPU, memory, disk, network). Same pattern as the Mac build agent.
    2. nvidia-gpu-exporter — GPU-specific metrics via nvidia-smi (temp, utilization, VRAM, clocks).
    3. hyprland-exporter (custom) — compositor metrics via hyprctl (window count, workspace usage, focused app).

    All three expose /metrics on distinct ports, scraped via additionalScrapeConfigs in the kube-prometheus-stack Helm values (pal-e-platform monitoring module).

    DORA metrics are already deployed — dora-exporter runs in-cluster scraping Forgejo and Woodpecker. Needs verification only.

    Board

    See board-hyprland-observability

    Status

    Project created 2026-05-29. DORA exporter deployed but unverified. Pal-E World sprint started 2026-06-27 — 7 tickets in backlog/todo.

    Milestones

    Palworld 1.0: July 10, 2026 — server must be updated to REST API before this date.

    Repos

    Repo Platform Role Status
    ~/.config/hypr Forgejo Hyprland config + client-side Moonlight docs Active
    ~/pal-e-platform Forgejo Salt states (nvidia, sunshine, gaming-session), monitoring module Active
    ldraney/palworld-server Forgejo OpenTofu IaC for GOW pod + dedicated server Archived (unarchive: #468)
    TBD: hyprland-exporter Forgejo Custom Prometheus exporter for Hyprland compositor metrics Not started
Doc 2
  • Architecture: Steam on Wayland arch-steam-wayland

    Steam on Wayland (Hyprland)

    Steam runs natively on Hyprland via XWayland. Games launch through Proton (GE-Proton or Experimental) with DXVK translating D3D11 to Vulkan.

    Known Issues

    • IPC pipe crash: Steam overlay triggers Assert( fatal stalled cross-thread pipe ) at pipes.cpp:900. Workaround: disable overlay per-game via OverlayAppEnable=0 in localconfig.vdf
    • libgamemode.so warning: Non-fatal — Steam runtime can't see host gamemode libs. Packages are installed and functional.
    • Zombie windows: Dead game windows stay registered in Hyprland, blocking IPC socket. Recovery requires killing Hyprland.

    Config

    • Window rules: steam_app_* → workspace 9, fullscreen, immediate (VRR tearing passthrough)
    • Launch options: STEAM_RUNTIME_PREFER_HOST_LIBRARIES=0 gamemoderun mangohud %command%
    • Overlay: disabled for crash-prone titles
  • User Story: Gaming Stability story-gaming-stability

    Gaming Stability

    As a gamer on Arch Linux with Hyprland, I want Steam/Proton games (especially Palworld) to run without crashing the compositor or locking up the IPC socket, so I can game without needing to kill and restart my desktop session.

    Key Outcomes

    • Palworld runs for extended sessions without Steam overlay IPC pipe crashes
    • If a game crashes, the zombie window is cleaned up without blocking Hyprland IPC
    • Crash recovery procedures are documented and tested
    • Observability commands are documented for GPU, Steam, Hyprland, and audio diagnostics
    • Workstation config is managed as IaC for reproducible setup
Review 1
  • Verdict: NEEDS_REFINEMENT

    Template Completeness

    • [x] Type -- Feature
    • [x] Lineage -- present (standalone, discovered during archbox setup)
    • [x] Repo -- ldraney/hyprland-config
    • [x] User Story -- present and well-formed
    • [x] Context -- present with background and key decisions
    • [x] File Targets -- present with 10 create/modify targets and 2 do-not-touch targets
    • [x] Feature Flag -- present (none)
    • [x] Acceptance Criteria -- 5 criteria present
    • [x] Test Expectations -- present with dry-run, pipeline, and lint checks
    • [x] Constraints -- present (Salt patterns, NVIDIA pinning, idempotency, pacman/yay)
    • [x] Checklist -- present
    • [x] Related -- present

    Traceability

    • [ ] story:gaming-stability label -- MISSING from project page. The project-hyprland-observability user-stories section lists: host-health, gpu-thermals, wm-state, dora-verified. "gaming-stability" is not among them. [SCOPE] Create user story entry "gaming-stability" on project-hyprland-observability user-stories section.
    • [ ] arch:iac label -- arch note MISSING. search_notes("arch-iac") returned no results. [SCOPE] Create architecture note arch-iac for component iac.
    • [x] Forgejo issue -- https://forgejo.tail5b443a.ts.net/ldraney/hyprland-config/issues/9, state: open

    File Targets

    • [ ] salt/ directory -- does NOT exist yet (to be created). Verified: correct, this is new work.
    • [ ] salt/top.sls -- does NOT exist yet (to be created). OK for feature ticket.
    • [ ] salt/packages/init.sls -- does NOT exist yet (to be created). OK.
    • [ ] salt/nvidia/init.sls -- does NOT exist yet (to be created). OK.
    • [ ] salt/steam/init.sls -- does NOT exist yet (to be created). OK.
    • [ ] salt/pipewire/init.sls -- does NOT exist yet (to be created). OK.
    • [ ] salt/dotfiles/init.sls -- does NOT exist yet (to be created). OK.
    • [ ] salt/pillar/ -- does NOT exist yet (to be created). OK.
    • [ ] .woodpecker.yml -- does NOT exist yet (to be created). OK.
    • [x] Makefile -- EXISTS at repo root. Currently has install, setup, console-rotate targets. Will be extended with Salt targets. Verified.
    • [x] hyprland.conf -- listed as do-not-touch. NOTE: file does not exist; the repo uses hyprland.lua instead. The ticket references hyprland.conf but the actual config is hyprland.lua. [BODY] Fix do-not-touch reference: hyprland.conf should be hyprland.lua.
    • [x] docs/architecture.md -- listed as do-not-touch (update separately). File exists. OK.

    Repo Placement

    Issue is filed on ldraney/hyprland-config. The fix lives in the same repo. Salt states and Woodpecker CI config will be added to this repo alongside existing dotfiles. The ticket mentions pal-e-platform as prior art for Salt patterns but does NOT propose changes there. Repo placement is correct -- single repo scope.

    Dependencies

    • Item #1619 (Fix Steam/Palworld crash: IPC pipe failure + docs overhaul, 3pts, backlog) -- shares story:gaming-stability label. Not a hard dependency but related work. Both are in backlog, no blocking relationship.
    • External dependency: Salt must be installed on the workstation (not tracked on board).
    • External dependency: Woodpecker CI must be accessible on the home network (ticket states it is already deployed).
    • No documented dependencies in the issue body. External dependencies should be noted.

    Acceptance Criteria

    5 acceptance criteria present. Assessment:

    • AC1 (salt-call state.apply) -- verifiable via dry-run command provided in Test Expectations. However, full verification requires Salt to be installed and a minion configured, which is outside agent scope.
    • AC2 (Woodpecker CI runs on push) -- partially verifiable. Agent can create .woodpecker.yml but cannot verify Woodpecker actually triggers. Needs manual verification.
    • AC3 (add package to state, installs on next apply) -- verifiable in principle but requires running Salt. Testable via dry-run.
    • AC4 (NVIDIA drift detection) -- specific and verifiable. Good criterion.
    • AC5 (fresh Arch install provision) -- not verifiable by agent. Requires a fresh machine. This is an aspirational criterion, acceptable for a feature ticket.

    Criteria are reasonable for infrastructure work. Most can be partially verified via dry-run.

    Blast Radius

    Searched for Salt and Woodpecker references in the repo -- none found. This is greenfield work with no existing patterns to conflict with. The Makefile will be extended (not replaced), so existing make targets (install, setup, console-rotate) must be preserved.

    The ticket references hyprland.conf as do-not-touch, but the actual config file is hyprland.lua. The salt/dotfiles/init.sls state will need to reference hyprland.lua, not hyprland.conf.

    Decomposition Assessment

    10 file targets across 1 repo. 5 acceptance criteria. Estimated agent work: this is substantial -- creating an entire Salt state tree (7+ SLS files), a Woodpecker CI pipeline, and extending the Makefile. Estimated at >5 minutes of agent work.

    NEEDS DECOMPOSITION -- 10 new files across multiple Salt state modules, plus CI pipeline, plus Makefile extension. Recommend splitting into:

    • Sub-ticket 1: Salt foundation (top.sls, pillar, packages/init.sls, Makefile salt targets) -- core state tree
    • Sub-ticket 2: Salt application states (nvidia, steam, pipewire, dotfiles) -- individual state modules
    • Sub-ticket 3: Woodpecker CI pipeline (.woodpecker.yml, lint/syntax validation)

    Route to skill-decompose-ticket.

    Recommendation

    • [BODY] Fix do-not-touch reference: hyprland.conf should be hyprland.lua (file does not exist as hyprland.conf)
    • [SCOPE] Create user story entry "gaming-stability" on project-hyprland-observability user-stories section
    • [SCOPE] Create architecture note arch-iac for component iac
    • [DECOMPOSE] 10 file targets, 5 AC, estimated >5 min agent work. Route to skill-decompose-ticket with 3-sub-ticket split: Salt foundation, Salt app states, Woodpecker CI.
Board 1