pal-e-docs
Project not found.

Phase 7e: Per-Block Search + Content Hashing

phase-postgres-7e-block-search-optimization Phase

Project
Goal: Add per-block tsvector search indexing and content hashing for compiled pages. This is the bridge between Phase 7 (blocks) and Phase 6 (embeddings) — once blocks have their own search vectors, they can also have their own embedding vectors.
Owner: Dev agent
Repo: pal-e-docs
Depends on: Phase 7d (block API working)
Parent phase: Phase 7 (Block-Structured Content Model)

Why This Phase

Phase 5 added tsvector search at the note level. With blocks, we can now search at the section level — "find the section about credentials" returns a specific block, not a whole note. This is also the prerequisite for Phase 6: the embedding vector(768) column will go on the blocks table, right next to the tsvector column.

Deliverables

1. Per-Block Search Vector

  • Add search_vector tsvector column to blocks table
  • Add GIN index on blocks.search_vector
  • Trigger: auto-update search_vector when block content changes
  • Weight: block content (B), parent note title (A) — so searching "postgres plan" still ranks blocks from postgres-related notes higher

2. Section-Level Search API

  • Update GET /notes/search to optionally return block-level results
  • Or add GET /blocks/search?q=... endpoint
  • Response: slug, note title, anchor_id, block_type, snippet, rank
  • Agent can jump directly from search result to get_block(slug, anchor_id)

3. Content Hashing

  • compiled_pages.content_hash = SHA-256 of all block content
  • On note update: compare hash before recompiling. Skip recompile if unchanged.
  • On block write: always recompile (single block changed = new hash)
  • Saves CPU on bulk operations and prevents unnecessary revision creation

4. Update search_notes MCP Tool

  • Add optional granularity param: "note" (default, backward compat) or "block"
  • Block-level results include anchor_id so agent can fetch just that section

Acceptance Criteria

  • Block-level search returns section-level results with anchor IDs
  • Search for "credentials" returns the specific section of sop-secrets-management that discusses credentials, not the whole note
  • Content hashing prevents unnecessary recompiles
  • search_notes MCP tool supports both note and block granularity
  • Backfill: all existing blocks have search_vectors populated

Bridge to Phase 6

After this phase, the blocks table has: content jsonb, search_vector tsvector, and will gain embedding vector(768) in Phase 6. The same table, progressively smarter — the database-side intelligence pattern at work.
  • phase-postgres-7d-api-mcp-tools — blocks API must be working
  • phase-postgres-6-vector-search — this phase unblocks per-block embeddings
  • phase-postgres-5-fulltext-search — note-level search pattern we're extending to blocks
  • concept-phase5-database-side-intelligence — the architectural pattern