pal-e-docs

GPU Freedom — Offload LLM to Groq, Free GPU for Gaming

story-pal-e-world-gpu-freedom User Story

active user-story
Project

story: GPU Freedom

Role

Developer / Gamer (Lucas)

Key

gpu-freedom

Want

As a developer who games on archbox, I want LLM inference offloaded from local Ollama to Groq API

So That

So that the GPU is free for Steam/Sunshine gaming without losing semantic search or platform LLM features

Acceptance Criteria

  • Ollama scaled to 0 replicas in k8s, GPU free (nvidia-smi shows no Ollama)
  • Groq API handles embedding generation for pal-e-docs semantic search
  • Semantic search returns relevant results via Groq
  • Cost stays under $1.00/day for normal platform usage
  • Keyword search fallback still works as graceful degradation

Success Metric

GPU is fully available for gaming (0% Ollama usage) while semantic search quality is maintained at parity with local Ollama.
No architecture notes yet for pal-e-world — this is the first story.
  • board-pal-e-world — project board
  • pal-e-platform #486 — Scale down k8s Ollama
  • pal-e-api #283 — Migrate semantic search to Groq API