Convert the ebook-search web app to async end to end and add concurrency
to the protected-phrase extraction and judging pipeline so large books no
longer block the event loop or the UI.
ORM / infra:
- Add get_async_postgres_engine and factor shared URL/connect_args building
into build_postgres_url (reused by the sync and async engine builders)
- Add async FastAPI session helpers (get_async_db, AsyncDbSession) with
expire_on_commit=False to avoid implicit IO under asyncio
App:
- Use AsyncEngine/AsyncSession throughout routes, search, ingest, embeddings,
answer, rerank and LLM calls; convert handlers to async
- Share a single httpx.AsyncClient in app state for LLM requests; size the
connection pool for concurrent phrase-judging workers
- Add judge_tasks: run per-book judging as tracked background tasks so a
book already being judged isn't double-queued
Protected phrases:
- Add a process pool (pool.py) and worker-count config
(extraction/judge book/phrase workers) to parallelize candidate generation
and judging
- Split admin actions into all/missing variants for generation and judging
Config:
- Add protected_phrase_extraction_workers, phrase_judge_book_workers,
phrase_judge_phrase_workers
Add a Typer CLI script that drives POST /search on a running server at a
configurable concurrency and reports latency percentiles (p50/p90/p95/p99),
throughput, and HTTP status distribution. Queries are drawn from the shared
eval JSONL set so load testing and evaluation exercise the same questions.
Richie