pal-e-docs

Phase 8: pal-e-docs SDK + MCP Rewrite + Integration Tests

phase-postgres-8-mcp-optimization Phase

Project
Goal: Rebuild the MCP tool surface to exploit Postgres search and block-structured content. Eliminate the list→get→get pattern. Minimize tokens per interaction.
Owner: Dev agent
Repo: pal-e-docs-mcp (primary), pal-e-docs (API endpoints)
Depends on: Phase 5 (tsvector), Phase 6 (pgvector), Phase 7 (blocks)

Why

The current MCP is a dumb HTTP passthrough — 26 tools, no search, no granular access. Finding information costs 12+ tool calls and ~15,000 tokens. Updating a note requires sending the entire html_content blob back (~5,000 tokens) even to fix one sentence. This phase cuts that to 2-3 calls and ~800 tokens.

Current Tool Inventory (26 tools)

Module Tools Problem
notes (5) list_notes, get_note, create_note, update_note, delete_note, get_note_revisions list returns no content, get returns ALL content. No middle ground. update requires full html_content replacement.
sprints (8) create/get/list/update sprint, add/move/remove item, get_board, get_backlog, bulk_move Reasonable. Board queries are efficient.
links (2) get_note_links, update_note_links Fine.
repos (3) list_repos, create_repo, update_repo Fine.
projects (2) list_projects, create_project Fine.
tags (1) list_tags Fine.

New Tools

Tool Depends On What It Does Token Impact
<code>search_notes(query)</code> Phase 5 Full-text search via tsvector. Returns matching notes with ts_headline snippets (just the relevant paragraph). Replaces list→get→get pattern. 12 calls → 1 call.
<code>semantic_search(query)</code> Phase 6 Vector similarity search via pgvector. Finds conceptually related notes without keyword match. Finds notes that keyword search misses.
<code>get_note_toc(slug)</code> Phase 7 Returns just headings + anchor IDs for a note. No content. ~50 tokens vs ~5,000 for get_note.
<code>get_block(slug, anchor_id)</code> Phase 7 Returns one section of a note by anchor ID. ~200 tokens vs ~5,000 for get_note.
<code>update_block(slug, anchor_id, content)</code> Phase 7 Updates one section without touching the rest. No need to send full document back. ~100 tokens vs ~5,000 for update_note.
<code>create_block(slug, block_type, content, after_anchor)</code> Phase 7 Insert a new block at a specific position in a note. Targeted insertion without full rewrite.
<code>delete_block(slug, anchor_id)</code> Phase 7 Remove a specific section from a note. Targeted deletion without full rewrite.

Modifications to Existing Tools

Tool Change
<code>get_note</code> Add optional <code>include_content=false</code> param. Default true for backward compat. When false, returns metadata + TOC only.
<code>update_note</code> Keep working with full html_content for backward compat. Internally, the backend decomposes HTML into blocks and recompiles.
<code>list_notes</code> Add optional <code>query</code> param that uses tsvector search when provided. Graceful upgrade — existing tag/type filters still work.

Cross-Cutting Optimizations

  • Response size limits: Add max_tokens param to search tools. Truncate long responses with "... (truncated, use get_block for full section)"
  • Batch queries: get_notes(slugs="sop-a,sop-b,sop-c") — fetch multiple note summaries in one call instead of N calls
  • Context-aware responses: Search results include anchor IDs so the agent can jump directly to the relevant section

Example: Before vs After

Steps

  • Add search_notes tool wrapping GET /notes/search?q=... (Phase 5 API)
  • Add semantic_search tool wrapping GET /notes/semantic-search?q=... (Phase 6 API)
  • Add block tools: get_note_toc, get_block, update_block, create_block, delete_block (Phase 7 API)
  • Add include_content param to get_note
  • Add query param to list_notes
  • Add batch endpoint: get_notes(slugs=...)
  • Add max_tokens response truncation to search tools
  • Update session-start hook to use search instead of list→get pattern
  • Update CLAUDE.md to document new tool patterns
  • Deprecation notice on raw html_content in update_note docstring

Migration Path

All existing tools keep working unchanged. New tools are additive. The old pattern (list→get→update with full HTML) still works but is discouraged. Block tools are the preferred path for reads and writes. Session-start hooks updated last, after tools are proven stable.

Success Metric

Average token cost per knowledge lookup drops from ~15,000 to <1,000. Average token cost per note update drops from ~5,000 to <500.