pal-e-docs

Phase 7: Block-Structured Content Model

phase-postgres-7-block-content Phase

Project
Goal: Replace HTML blob storage with typed content blocks. Enable compiled pages, per-section search, stable anchors, deterministic rendering, and flexible note hierarchy.
Owner: Dev agent
Repo: pal-e-docs, pal-e-docs-mcp
Depends on: Phase 5 (full-text search) — COMPLETED
Status: IN PROGRESS — 7a and 7b COMPLETED. 7c IN PROGRESS (script merged, backfill not yet run). 7f-1 COMPLETED.

Why

HTML blobs are opaque. You can't search within them meaningfully, can't deep-link to sections, can't reuse content across pages, can't generate a TOC, and can't embed per-section for AI retrieval. Blocks fix all of this. See benchmark-phase7-block-baseline for the quantitative case.

Key Data (from baseline)

  • 256 notes, 1.09M chars (~272K tokens total corpus)
  • 84% have 4+ sections — would benefit from block-level access
  • Plans avg 9,441 chars — the largest type, most read, biggest savings target
  • 6 orphaned docs can't nest under phases due to type restriction
  • Estimated savings: 92% per targeted read, 96% per update, 95% session startup

Sub-Phases

# Sub-Phase Slug Depends On Status Deliverable
7a Schema + Hierarchy Relaxation <code>phase-postgres-7a-schema-hierarchy</code> Phase 5 <strong>COMPLETED</strong> blocks + compiled_pages tables, any-to-any hierarchy
7b Parser + Compiler <code>phase-postgres-7b-parser-compiler</code> 7a <strong>COMPLETED</strong> HTML→blocks parser (6 types), blocks→HTML compiler, 105 tests. PR #97.
7c Backfill Migration <code>phase-postgres-7c-backfill-migration</code> 7a, 7b <strong>IN PROGRESS</strong> (script merged via PR #101, backfill not yet run) All 256 notes converted to blocks, compiled pages populated
7d Block API + MCP Tools <code>phase-postgres-7d-api-mcp-tools</code> 7c NOT STARTED 6 API endpoints, 5 MCP tools (get_toc, get/update/create/delete_block)
7e Compiled Page Architecture <code>phase-postgres-7e-compiled-pages</code> 7d NOT STARTED Compiled page format, API, source-of-truth cutover
7f Doc Cleanup + SOP Hardening <code>phase-postgres-7f-doc-cleanup-sop</code> 7e NOT STARTED (sub-phase 7f-1 COMPLETED) Doc debt cleanup, post-merge automation, SOP review
Note: The original 7e (Per-Block Search + Hashing, phase-postgres-7e-block-search-optimization) has been renumbered. The sub-phases above reflect the current sequence.

Dependency Chain




          
  
User Stories



          
  

    

        
          

              #
              Story
              Sub-Phase
          

        
      
          

              A3
              As an agent, I can read one section of a note without fetching the entire document
              7d
          

          

              A4
              As an agent, I can update one section without rewriting the entire document
              7d
          

          

              A5
              As an agent, I can see a note's TOC and jump to the right section
              7d
          

          

              K1
              Concept docs, benchmarks, incidents nest under the phase they belong to
              7a + 7c
          

          

              H1
              As Lucas, I can see a table of contents on long documents
              7c + browse frontend
          

      
    

  



          
  
Backward Compatibility



          
    
html_content stays as a computed/cached field populated by the compiler. Existing MCP tools that read/write html_content continue to work unchanged. New tools work with blocks directly. Zero-breaking-change migration.



          
  


          
  
    
      
  • benchmark-phase7-block-baseline — quantitative baseline before blocks
    • 
      
  • decision-phase6-vector-search-architecture — per-block embedding depends on blocks
    • 
      
  • concept-phase5-database-side-intelligence — the database-side intelligence pattern this extends
    •