Richie
6c8a4bfea7
Refactor protected phrase handling from a single module into a python/ebook_search/protected_phrases package covering extraction, storage, and runtime matching. Phrase filtering is now data-driven via bundled TOML files: ignored_phrases, bad_starts, bad_ends, and most_common_words. Add phrase-tuning settings to EbookSearchConfig so candidate generation, scoring, LLM judging, and matching are configurable rather than hardcoded: token bounds, entity token limit, raw n-gram min count, frequency and chapter-spread score thresholds, candidate/LLM/target caps, confidence threshold, nesting defaults, and the phrase hit boost.
31 lines
270 B
TOML
31 lines
270 B
TOML
tokens = [
|
|
"a",
|
|
"an",
|
|
"and",
|
|
"any",
|
|
"as",
|
|
"at",
|
|
"be",
|
|
"because",
|
|
"but",
|
|
"by",
|
|
"can",
|
|
"could",
|
|
"do",
|
|
"for",
|
|
"from",
|
|
"have",
|
|
"if",
|
|
"of",
|
|
"or",
|
|
"some",
|
|
"than",
|
|
"the",
|
|
"these",
|
|
"this",
|
|
"to",
|
|
"will",
|
|
"with",
|
|
"would",
|
|
]
|