Expose the protected phrase extraction pipeline through the web UI:
- Admin routes: POST /admin/build-phrases, /admin/generate-ngrams, and
/admin/judge-ngrams, each wrapping the protected_phrases.lib backfill
helpers, committing on success, rolling back and rendering an error
partial on failure, and reporting per-book/candidate/mention counts.
- Book detail page: show candidate, judged, and protected phrase counts,
list top candidate n-grams (with kept/rejected status) and protected
phrases, and add a POST /books/{id}/recalculate-phrases action that
clears and regenerates candidates, then redirects back with a status
message.
- Admin template: add Generate/Judge n-gram buttons.
Also reflows admin.html to 2-space HTML formatting.
Refactor protected phrase handling from a single module into a
python/ebook_search/protected_phrases package covering extraction,
storage, and runtime matching. Phrase filtering is now data-driven via
bundled TOML files: ignored_phrases, bad_starts, bad_ends, and
most_common_words.
Add phrase-tuning settings to EbookSearchConfig so candidate generation,
scoring, LLM judging, and matching are configurable rather than hardcoded:
token bounds, entity token limit, raw n-gram min count, frequency and
chapter-spread score thresholds, candidate/LLM/target caps, confidence
threshold, nesting defaults, and the phrase hit boost.
The gitea runner containers have no docker access, so build the test
env with uv from the existing lockfile and run pytest directly:
- test_ebook_search workflow: uv sync --locked + uv run pytest, with
UV_PYTHON_DOWNLOADS=never so uv uses the nix-provided python 3.14
- add uv to the runner hostPackages (needs a jeeves rebuild to apply)
- ignore nested **/.venv in .dockerignore (uv sync creates one in
python/ebook_search/docker)
- document the uv test commands in the docker README; the docker test
image remains for validating the image itself
Add a self-contained docker/ package for running the ebook search app
against the existing Postgres database on jeeves:
- Dockerfile: python:3.14-slim image, non-root user, runs the FastAPI
app on port 8070
- docker-compose.yml: service definition with library volume mount,
BM25 index volume, .env loading, and a /health healthcheck
- containers.py: Typer CLI (ebook-search-containers) for build/start/
stop/restart/logs/ps lifecycle management
- README.md: usage and configuration docs
load_gold_queries now skips blank and `//` comment lines so the committed
section separator in queries.jsonl no longer breaks dataset/load-test loading.
Update tests left stale by the search refactor (6bc3011):
- pass the now-required rank_constant to reciprocal_rank_fusion
- expect bm25_candidates to receive the full query and drop the removed
"BM25 query preparation" timing step
- assert reranking is enabled by default
Add a Typer CLI script that drives POST /search on a running server at a
configurable concurrency and reports latency percentiles (p50/p90/p95/p99),
throughput, and HTTP status distribution. Queries are drawn from the shared
eval JSONL set so load testing and evaluation exercise the same questions.
Richie