Richie/dotfiles
1
0
Fork 0
Code Issues 1 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
971 Commits 49 Branches 0 Tags
ceb2dbb2b34af75a6d9ea43149911d460de44ced
Commit Graph
8 Commits
Author SHA1 Message Date
Richie ceb2dbb2b3 refactor(ebook-search): simplify search and phrase matching
build_systems / build-rhapsody-in-green (pull_request) Successful in 1m3s
Details
treefmt / nix fmt (pull_request) Successful in 5s
Details
pytest / pytest (pull_request) Successful in 28s
Details
test ebook search / test-ebook-search (pull_request) Failing after 35s
Details
build_systems / build-bob (pull_request) Successful in 51s
Details
build_systems / build-brain (pull_request) Successful in 50s
Details
build_systems / build-jeeves (pull_request) Successful in 2m20s
Details
2026-07-15 15:25:11 -04:00
Richie bfb3463fd0 feat(ebook): migrate to async DB/HTTP and parallelize phrase pipeline 
Convert the ebook-search web app to async end to end and add concurrency
to the protected-phrase extraction and judging pipeline so large books no
longer block the event loop or the UI.

ORM / infra:
- Add get_async_postgres_engine and factor shared URL/connect_args building
  into build_postgres_url (reused by the sync and async engine builders)
- Add async FastAPI session helpers (get_async_db, AsyncDbSession) with
  expire_on_commit=False to avoid implicit IO under asyncio

App:
- Use AsyncEngine/AsyncSession throughout routes, search, ingest, embeddings,
  answer, rerank and LLM calls; convert handlers to async
- Share a single httpx.AsyncClient in app state for LLM requests; size the
  connection pool for concurrent phrase-judging workers
- Add judge_tasks: run per-book judging as tracked background tasks so a
  book already being judged isn't double-queued

Protected phrases:
- Add a process pool (pool.py) and worker-count config
  (extraction/judge book/phrase workers) to parallelize candidate generation
  and judging
- Split admin actions into all/missing variants for generation and judging

Config:
- Add protected_phrase_extraction_workers, phrase_judge_book_workers,
  phrase_judge_phrase_workers
 2026-07-12 17:51:06 -04:00
Richie d03daeb66d feat(ebook): implement phrase matching functionality and UI enhancements 2026-07-12 17:49:52 -04:00
Richie f8b5ba82a6 feat(ebook): add protected phrase extraction library with config-driven tuning 
Refactor protected phrase handling from a single module into a
python/ebook_search/protected_phrases package covering extraction,
storage, and runtime matching. Phrase filtering is now data-driven via
bundled TOML files: ignored_phrases, bad_starts, bad_ends, and
most_common_words.

Add phrase-tuning settings to EbookSearchConfig so candidate generation,
scoring, LLM judging, and matching are configurable rather than hardcoded:
token bounds, entity token limit, raw n-gram min count, frequency and
chapter-spread score thresholds, candidate/LLM/target caps, confidence
threshold, nesting defaults, and the phrase hit boost.
 2026-07-12 17:49:52 -04:00
Richie 6bc30115d9 fix(ebook-search): code clean up to impove reliablty and readabilty 2026-06-18 12:45:56 -04:00
Richie f71ae7d2c6 added guardrails.py to constrain responses and added validation to config.py 2026-06-15 21:57:38 -04:00
Richie 68b3a38b81 converting to pydantic-settings 2026-06-14 21:29:45 -04:00
Richie 26ff1f0fd3 added answer.py and config 2026-06-14 15:40:04 -04:00