Commit Graph
8 Commits
Author SHA1 Message Date
Richie 63e0b0dd3b feat(extraction): add cached YAKE extractor for improved performance 2026-07-12 17:49:52 -04:00
Richie 167aedcefb feat(ebook): add junk tokens for improved phrase matching 2026-07-12 17:49:52 -04:00
Richie bcb8b7b169 Add models and database persistence for protected phrase extraction
- Introduced dataclasses for phrase candidates, judgments, and matches in `models.py`.
- Implemented database operations for candidate and protected phrases in `store.py`, including loading, saving, and deleting phrases.
- Enhanced text normalization functions in `text_normalization.py` with detailed docstrings.
- Refactored search functionality to utilize new models and methods for detecting protected phrases.
2026-07-12 17:49:52 -04:00
Richie 8d8c8cd3dd refactor(protected-phrases): extract config and text normalization helpers 2026-07-12 17:49:52 -04:00
Richie 77692b1414 feat(ebook): update protected phrases with additional tokens and phrases 2026-07-12 17:49:52 -04:00
Richie f4a498e45e feat(ebook): enhance phrase judgment logging with failure tracking 2026-07-12 17:49:52 -04:00
Richie d03daeb66d feat(ebook): implement phrase matching functionality and UI enhancements 2026-07-12 17:49:52 -04:00
Richie f8b5ba82a6 feat(ebook): add protected phrase extraction library with config-driven tuning
Refactor protected phrase handling from a single module into a
python/ebook_search/protected_phrases package covering extraction,
storage, and runtime matching. Phrase filtering is now data-driven via
bundled TOML files: ignored_phrases, bad_starts, bad_ends, and
most_common_words.

Add phrase-tuning settings to EbookSearchConfig so candidate generation,
scoring, LLM judging, and matching are configurable rather than hardcoded:
token bounds, entity token limit, raw n-gram min count, frequency and
chapter-spread score thresholds, candidate/LLM/target caps, confidence
threshold, nesting defaults, and the phrase hit boost.
2026-07-12 17:49:52 -04:00