Captured 2026-03-06, before any search capability exists.
Session Startup Cost (Betty Sue)
Before a single user message, every Betty Sue session consumes:
| Component |
Chars |
Est. Tokens |
% of Total |
| Session injection (personality, SOPs list, plans, instructions) |
~6,100 |
~1,525 |
12.5% |
| CLAUDE.md (global) |
~900 |
~225 |
1.8% |
| CLAUDE.md (project) |
~1,200 |
~300 |
2.5% |
| MEMORY.md |
~5,500 |
~1,375 |
11.3% |
| 4 mandatory get_note calls (full HTML blobs) |
~35,000 |
~8,750 |
<strong>71.9%</strong> |
| <strong>Total before first user message</strong> |
<strong>~48,700</strong> |
<strong>~12,175</strong> |
|
Key finding: 72% of startup cost is fetching 4 full HTML documents that the agent may not need. With search, startup could fetch summaries/snippets on demand instead of preloading full blobs.
Ad-Hoc Query Cost (5 representative queries)
BEFORE (pre-search, 2026-03-06)
| Query |
Description |
MCP Calls |
Response Chars |
Found? |
| Q1 |
Secrets management SOP |
2 |
~7,660 |
YES |
| Q2 |
Postgres restore procedure |
2 |
~7,374 |
YES |
| Q3 |
Agent workflow operation |
2 |
~8,954 |
YES |
| Q4 |
Sprint workflow automation status |
2 |
~9,670 |
YES |
| Q5 |
Repos in postgres migration |
3 |
~10,089 |
YES |
| <strong>Totals</strong> |
|
<strong>11</strong> |
<strong>~43,747</strong> |
|
| <strong>Averages</strong> |
|
<strong>2.2</strong> |
<strong>~8,749</strong> |
|
| Query |
Description |
MCP Calls |
Response Chars |
Found? |
| Q1 |
Secrets management SOP |
1 |
~2,800 |
YES (#1 result, rank 0.997) |
| Q2 |
Postgres restore procedure |
1 |
~2,400 |
YES (#1 result, rank 0.756) |
| Q3 |
ArgoCD deployment |
1 |
~2,600 |
YES (10 ranked results) |
| Q4 |
Sprint workflow automation |
1 |
~2,600 |
YES (#1 result, rank 1.0) |
| Q5 |
Woodpecker CI pipeline |
1 |
~2,400 |
YES (#1 result, rank 1.0) |
| <strong>Totals</strong> |
|
<strong>5</strong> |
<strong>~12,800</strong> |
|
| <strong>Averages</strong> |
|
<strong>1.0</strong> |
<strong>~2,560</strong> |
|
Comparison
| Metric |
Before |
After |
Improvement |
| MCP calls (5 queries) |
11 |
5 |
<strong>55% reduction</strong> |
| Avg calls per query |
2.2 |
1.0 |
<strong>55% reduction</strong> |
| Total response chars |
~43,747 |
~12,800 |
<strong>71% reduction</strong> |
| Avg chars per query |
~8,749 |
~2,560 |
<strong>71% reduction</strong> |
| Est. tokens per query |
~2,187 |
~640 |
<strong>71% reduction</strong> |
| Requires tag/slug knowledge |
Yes |
No |
Natural language queries |
| Cross-cutting search |
Impossible |
Enabled |
New capability |
Key Observations (After)
- 1 call per query, always. No more list→get loops. search_notes returns ranked results with snippets directly.
- 71% token reduction — slightly under the 80-90% estimate because each search returns 10 results with snippets. Using
limit=3 for targeted queries would reduce further.
- Natural language works. "secrets management", "ArgoCD deployment", "woodpecker CI pipeline" all return the right notes as top results. No tag/slug knowledge needed.
- Ranking is accurate. Q1 returns SOP: Secrets Management at rank 0.997. Q4 returns the sprint workflow automation plan at rank 1.0. The weighted tsvector (title A, content B, slug C) is working as designed.
- Snippets provide context. Each result includes a headline snippet with
**bold** match highlighting. Often enough to answer the question without fetching the full note.
Remaining Optimization Opportunities
- Session startup: 4 mandatory get_note calls still consume ~8,750 tokens. Could replace with search-on-demand, but requires session hook redesign.
- Limit tuning: Default limit=10 returns more than needed for targeted queries. Agents should use limit=3-5 for specific lookups.
- Phase 6 (pgvector): Semantic search will find related concepts even without keyword overlap — e.g., "how do we handle database credentials" → secrets management SOP.
- Phase 7 (blocks): Section-level retrieval would return just the relevant section of a note, not the whole document or even a snippet.
phase-postgres-5-fulltext-search — the phase this benchmarks (COMPLETED)concept-phase5-database-side-intelligence — architecture rationale