8 Commits

Include renames

Author	SHA1	Message	Date
Richie	1841ac69fa	fix(protected-phrases): isolate phrase generation per book ... Run full-book candidate generation inside worker-owned sessions so each book commits independently during backfills. Abort recalculation when a book has no indexed chapters to preserve existing phrase data, and update admin/UI tests for the new generation flow.	2026-07-12 13:34:04 -04:00
Richie	76a9c70a47	feat(ebook): migrate to async DB/HTTP and parallelize phrase pipeline ... Convert the ebook-search web app to async end to end and add concurrency to the protected-phrase extraction and judging pipeline so large books no longer block the event loop or the UI. ORM / infra: - Add get_async_postgres_engine and factor shared URL/connect_args building into build_postgres_url (reused by the sync and async engine builders) - Add async FastAPI session helpers (get_async_db, AsyncDbSession) with expire_on_commit=False to avoid implicit IO under asyncio App: - Use AsyncEngine/AsyncSession throughout routes, search, ingest, embeddings, answer, rerank and LLM calls; convert handlers to async - Share a single httpx.AsyncClient in app state for LLM requests; size the connection pool for concurrent phrase-judging workers - Add judge_tasks: run per-book judging as tracked background tasks so a book already being judged isn't double-queued Protected phrases: - Add a process pool (pool.py) and worker-count config (extraction/judge book/phrase workers) to parallelize candidate generation and judging - Split admin actions into all/missing variants for generation and judging Config: - Add protected_phrase_extraction_workers, phrase_judge_book_workers, phrase_judge_phrase_workers	2026-07-12 13:34:04 -04:00
Richie	338ff60f80	Add models and database persistence for protected phrase extraction ... - Introduced dataclasses for phrase candidates, judgments, and matches in `models.py`. - Implemented database operations for candidate and protected phrases in `store.py`, including loading, saving, and deleting phrases. - Enhanced text normalization functions in `text_normalization.py` with detailed docstrings. - Refactored search functionality to utilize new models and methods for detecting protected phrases.	2026-07-12 13:34:04 -04:00
Richie	fff50084c1	feat(ebook): add admin and book-detail UI for protected phrase pipeline ... Expose the protected phrase extraction pipeline through the web UI: - Admin routes: POST /admin/build-phrases, /admin/generate-ngrams, and /admin/judge-ngrams, each wrapping the protected_phrases.lib backfill helpers, committing on success, rolling back and rendering an error partial on failure, and reporting per-book/candidate/mention counts. - Book detail page: show candidate, judged, and protected phrase counts, list top candidate n-grams (with kept/rejected status) and protected phrases, and add a POST /books/{id}/recalculate-phrases action that clears and regenerates candidates, then redirects back with a status message. - Admin template: add Generate/Judge n-gram buttons. Also reflows admin.html to 2-space HTML formatting.	2026-07-12 13:34:04 -04:00
Richie	a9daa60c17	moved TC001 expetop out of pyproject.toml	2026-06-16 18:31:40 -04:00
Richie	4640ebf8ce	converted addmin.py and page.py to DbSession and AppConfig	2026-06-15 22:01:42 -04:00
Richie	eb76edb740	reworked ebook_search routers	2026-06-14 15:41:09 -04:00
Richie	6715bbf0a5	build api and frountend	2026-06-14 15:40:04 -04:00