QA Report: Phase 7c Backfill (2026-03-07)

qa-phase7c-backfill-2026-03-07 Doc

Project

Summary

Backfill script (scripts/backfill_blocks.py, merged in PR #101) executed against production via kubectl cp + kubectl exec. All 274 notes parsed into blocks and compiled pages populated. Zero data loss. Zero visual regressions.

Database State After Backfill

Metric	Value
Notes processed	<strong>274</strong>
Total blocks created	<strong>5,197</strong>
Compiled pages created	<strong>274</strong> (100% coverage)
Notes with TOC	<strong>247</strong> (90%)
Notes without TOC	<strong>27</strong> (no headings)
Avg blocks/note	<strong>19.0</strong>
Max blocks/note	<strong>84</strong> (<code>plan-2026-02-28-woodpecker-mcp</code>)
Empty notes (0 blocks)	<strong>1</strong> (<code>convention-dockerfile-pypi-pattern</code> — genuinely empty)

Block Type Distribution

Type	Count	What It Captures
heading	2,087	Section headers — TOC entries + anchor IDs
paragraph	1,723	Body text
list	876	Bullet/numbered lists
table	369	Structured data
code	127	Code blocks, pre-formatted content
mermaid	15	Dependency diagrams

Round-Trip Fidelity Analysis

Every note was parsed (HTML → blocks) then compiled (blocks → HTML) and compared against the original html_content.

Category	Count	Verdict
Identical (byte-for-byte)	15	Perfect match
Anchor-only	3	Compiler adds <code>id=</code> to headings (expected, correct)
Whitespace + entity + br normalization	228	Cosmetic only — zero visual impact
True semantic diff	28	Investigated — all benign (see below)

True Semantic Diff Root Causes

All 28 "true semantic" diffs traced to exactly two causes:

Table formatting (22 notes): Compiler outputs <tr>\n<th> instead of <tr><th>. HTML renders identically in browsers — only source formatting differs.
Bare <pre> → <pre><code> (9 notes): Compiler wraps bare <pre> content in <code>. Browser rendering identical; actually more standards-compliant.

Verdict: Zero visual regressions. Zero data loss. Zero broken notes.

Cosmetic Diff Breakdown (the other 228)

— → — — HTML entity normalization to Unicode. Renders identically.
<br> → <br/> — Self-closing tag normalization. XHTML-compliant.
Blank line removal — Compiler doesn't insert blank lines between block-level elements. No visual impact.
Table cell whitespace — Newlines inside <tr>/<td> tags. HTML whitespace rules make these invisible.

Token Reduction Preview (Phase 7d Payoff)

With the blocks and TOC data now in the database, Phase 7d's get_note_toc and get_block MCP tools will deliver:

Single Note (the main plan)

Method	Chars	~Tokens	Reduction
<code>get_note</code> (current)	9,668	~2,417	—
<code>get_note_toc</code> (Phase 7d)	710	~177	92.7%
<code>get_block</code> (Vision only)	1,275	~318	86.8%

Session Startup (4 active plans injected every conversation)

Plan	Current	TOC Only	Reduction
<code>plan-2026-02-26-tf-modularize-postgres</code>	9,668	710	93%
<code>plan-2026-03-01-pal-e-sprints</code>	3,693	400	89%
<code>plan-2026-03-03-sprint-workflow-automation</code>	10,378	792	92%
<code>plan-2026-02-25-platform-observability</code>	2,138	384	82%
<strong>Total</strong>	<strong>25,877</strong>	<strong>2,286</strong>	<strong>91.2%</strong>

~5,900 tokens freed per session start. Compounds across every get_note call in a session (20-30 reads typical = tens of thousands of tokens saved).

Execution Details

Script: scripts/backfill_blocks.py (merged PR #101, nit fixes PR #103)
Method: kubectl cp into pod, kubectl exec with DATABASE_URL=$PALDOCS_DATABASE_URL
Runtime: 2.0 seconds for 274 notes
Idempotent: Safe to re-run (deletes existing blocks/compiled_page per note before re-inserting)
Env var note: Pod uses PALDOCS_DATABASE_URL, not DATABASE_URL. Run with: sh -c 'DATABASE_URL="$PALDOCS_DATABASE_URL" python /tmp/backfill_blocks.py'

Edge Cases

convention-dockerfile-pypi-pattern — 0 blocks, empty html_content. Compiled page exists with empty HTML and empty TOC. Correct behavior.
Notes with 1 block: phase-postgres-1-tf-modularize, todo-move-mcp-migration-plan-project — verified as simple single-paragraph notes.