Commit Graph
8 Commits
Author SHA1 Message Date
Richie 9bc0d67962 feat(extraction): add cached YAKE extractor for improved performance 2026-07-16 13:09:34 -04:00
Richie d6d90489db feat(ebook): add junk tokens for improved phrase matching 2026-07-16 13:09:34 -04:00
Richie 7b97f31fec Add models and database persistence for protected phrase extraction
- Introduced dataclasses for phrase candidates, judgments, and matches in `models.py`.
- Implemented database operations for candidate and protected phrases in `store.py`, including loading, saving, and deleting phrases.
- Enhanced text normalization functions in `text_normalization.py` with detailed docstrings.
- Refactored search functionality to utilize new models and methods for detecting protected phrases.
2026-07-16 13:09:34 -04:00
Richie 6284d72af3 refactor(protected-phrases): extract config and text normalization helpers 2026-07-16 13:09:34 -04:00
Richie 37d0ae487a feat(ebook): update protected phrases with additional tokens and phrases 2026-07-16 13:09:34 -04:00
Richie ac54159496 feat(ebook): enhance phrase judgment logging with failure tracking 2026-07-16 13:09:34 -04:00
Richie 3cf50c860e feat(ebook): implement phrase matching functionality and UI enhancements 2026-07-16 13:09:34 -04:00
Richie 2427ef05a2 feat(ebook): add protected phrase extraction library with config-driven tuning
Refactor protected phrase handling from a single module into a
python/ebook_search/protected_phrases package covering extraction,
storage, and runtime matching. Phrase filtering is now data-driven via
bundled TOML files: ignored_phrases, bad_starts, bad_ends, and
most_common_words.

Add phrase-tuning settings to EbookSearchConfig so candidate generation,
scoring, LLM judging, and matching are configurable rather than hardcoded:
token bounds, entity token limit, raw n-gram min count, frequency and
chapter-spread score thresholds, candidate/LLM/target caps, confidence
threshold, nesting defaults, and the phrase hit boost.
2026-07-16 13:09:34 -04:00