pal-e-docs

Post-Move Network Recovery — Archbox at New Location

todo-post-move-network-recovery Doc

open todo infra networking
Project

Context

Archbox moved to a new physical location (Xfinity network). LAN IP changed from 10.0.0.217 to 10.0.0.149 (DHCP). Tailscale overlay is up and cluster is running, but several issues need attention to restore full production.

What's Working

  • Tailscale is UP — archbox online at 100.110.151.59, all 20+ funnel proxies running
  • K3s cluster healthy — single node Ready, most pods running
  • Tailscale Operator — running in-cluster, all funnel ingresses created
  • Public URLs responding — forgejo (200), grafana (302), woodpecker (200), argocd (200)
  • Salt firewall rules — SSH uses 10.0.0.0/24 CIDR, new IP still in range
  • Terraform — no hardcoded IPs, routes through Tailscale domain

Action Items

P0 — Fix systemd-resolved for Tailscale DNS

Tailscale reports: setLinkDNS: Could not activate remote peer 'org.freedesktop.resolve1'. systemd-resolved is disabled/dead. DNS falls back to NetworkManager with Xfinity resolvers (75.75.75.75). MagicDNS may not work from the host.
  • Enable and start systemd-resolved
  • Verify Tailscale MagicDNS works after
  • Confirm no conflict with NetworkManager DNS

P1 — Investigate pal-e-docs 404 on funnel

Pod is running, /healthz returns 200, but root / returns 404 via the public funnel URL. MCP API calls returning 502. This may be an app-level routing issue or funnel path misconfiguration.
  • Check funnel ingress target path
  • Check if pal-e-docs app expects a specific base path
  • Verify MCP connectivity from Claude Code
  • basketball-api — ErrImagePull
  • mcd-tracker — ErrImagePull
  • mcd-tracker-app — CrashLoopBackOff
  • westside-app — ImagePullBackOff
  • ollama — UnexpectedAdmissionError
Likely Harbor auth or missing image pushes. Triage after networking is solid.

P3 — Update archbox IP in memory/docs

  • Update MEMORY.md: 10.0.0.217 → 10.0.0.149 (note: DHCP, may change again)
  • Consider setting a DHCP reservation on the Xfinity router for stability

P4 — Salt master/minion decision

Both salt-master and salt-minion are disabled/dead. Not blocking anything currently. Decide whether to re-enable for host config management or continue managing manually.

What Does NOT Need Changing

  • Salt firewall pillar — /24 CIDR covers new IP
  • Terraform — no LAN IP references
  • Tailscale operator/funnels — IP-independent overlay
  • K3s networking — flannel/cni0 operational