12 Commits
Author SHA1 Message Date
Richie 68c9693711 feat(config): add new scoring parameters for phrase extraction and matching 2026-07-24 11:38:51 -04:00
Richie 6a0e71a30d feat(ebook-search): implement async file path resolution and EPUB discovery 2026-07-24 11:38:51 -04:00
Richie 8073144e2b feat(prompts): implement TOML-backed prompt loading and refactor message generation 2026-07-24 11:38:51 -04:00
Richie 94f18722e4 feat(search): enhance phrase matching and reranking logic with improved async handling 2026-07-24 11:38:51 -04:00
Richie 9f126fedc7 feat(ebook-search): implement async phrase matching for chunks and add ChunkPhraseHit model 2026-07-24 11:38:51 -04:00
Richie e010756e09 refactor(ebook-search): simplify search and phrase matching 2026-07-24 11:38:51 -04:00
Richie 0028237579 perf(ebook-search): run phrase detection in parallel with retrieval
Move protected phrase detection into the retrieval gather so it runs
concurrently with vector and BM25 candidates instead of sequentially
before them. Make the search API accept real bool form fields for
rerank/phrase_matching, gate phrase matching on both the request and
config kill switch, and reflow log f-strings for readability.
2026-07-24 11:38:51 -04:00
Richie 58be234d7f fix(protected-phrases): isolate phrase generation per book
Run full-book candidate generation inside worker-owned sessions so each book commits independently during backfills. Abort recalculation when a book has no indexed chapters to preserve existing phrase data, and update admin/UI tests for the new generation flow.
2026-07-24 11:38:50 -04:00
Richie 565119ee45 test(ebook): cover protected phrases and migrate suite to async
Add test_protected_phrases.py covering phrase-matching behavior in the
RAG engine, and update the existing ebook_search tests to use the async
SQLAlchemy engine/session (create_async_engine, AsyncSession) and async
HTTP paths.
2026-07-24 11:38:50 -04:00
Richie 09d963ba34 fix(ebook-search): skip comment lines in gold query loader and realign tests
treefmt / nix fmt (pull_request) Successful in 10s
pytest / pytest (pull_request) Successful in 31s
build_systems / build-brain (pull_request) Successful in 52s
build_systems / build-bob (pull_request) Successful in 52s
build_systems / build-jeeves (pull_request) Successful in 2m43s
build_systems / build-leviathan (pull_request) Successful in 59s
build_systems / build-rhapsody-in-green (pull_request) Successful in 1m5s
load_gold_queries now skips blank and `//` comment lines so the committed
section separator in queries.jsonl no longer breaks dataset/load-test loading.

Update tests left stale by the search refactor (6bc3011):
- pass the now-required rank_constant to reciprocal_rank_fusion
- expect bm25_candidates to receive the full query and drop the removed
  "BM25 query preparation" timing step
- assert reranking is enabled by default
2026-06-21 14:45:31 -04:00
Richie 6ae1ff1f5c feat(ebook-search): add load-test CLI for the search service
Add a Typer CLI script that drives POST /search on a running server at a
configurable concurrency and reports latency percentiles (p50/p90/p95/p99),
throughput, and HTTP status distribution. Queries are drawn from the shared
eval JSONL set so load testing and evaluation exercise the same questions.
2026-06-18 12:39:55 -04:00
Richie dbc6b5b53b test(ebook-search): organize tests under dedicated package
Move ebook search tests into tests/ebook_search and standardize mocking on pytest-mock.
2026-06-16 21:47:40 -04:00