Richie
6c8a4bfea7
Refactor protected phrase handling from a single module into a python/ebook_search/protected_phrases package covering extraction, storage, and runtime matching. Phrase filtering is now data-driven via bundled TOML files: ignored_phrases, bad_starts, bad_ends, and most_common_words. Add phrase-tuning settings to EbookSearchConfig so candidate generation, scoring, LLM judging, and matching are configurable rather than hardcoded: token bounds, entity token limit, raw n-gram min count, frequency and chapter-spread score thresholds, candidate/LLM/target caps, confidence threshold, nesting defaults, and the phrase hit boost.
103 lines
1.1 KiB
TOML
103 lines
1.1 KiB
TOML
words = [
|
|
"the",
|
|
"be",
|
|
"to",
|
|
"of",
|
|
"and",
|
|
"a",
|
|
"in",
|
|
"that",
|
|
"have",
|
|
"I",
|
|
"it",
|
|
"for",
|
|
"not",
|
|
"on",
|
|
"with",
|
|
"he",
|
|
"as",
|
|
"you",
|
|
"do",
|
|
"at",
|
|
"this",
|
|
"but",
|
|
"his",
|
|
"by",
|
|
"from",
|
|
"they",
|
|
"we",
|
|
"say",
|
|
"her",
|
|
"she",
|
|
"or",
|
|
"an",
|
|
"will",
|
|
"my",
|
|
"one",
|
|
"all",
|
|
"would",
|
|
"there",
|
|
"their",
|
|
"what",
|
|
"so",
|
|
"up",
|
|
"out",
|
|
"if",
|
|
"about",
|
|
"who",
|
|
"get",
|
|
"which",
|
|
"go",
|
|
"me",
|
|
"when",
|
|
"make",
|
|
"can",
|
|
"like",
|
|
"time",
|
|
"no",
|
|
"just",
|
|
"him",
|
|
"know",
|
|
"take",
|
|
"people",
|
|
"into",
|
|
"year",
|
|
"your",
|
|
"good",
|
|
"some",
|
|
"could",
|
|
"them",
|
|
"see",
|
|
"other",
|
|
"than",
|
|
"then",
|
|
"now",
|
|
"look",
|
|
"only",
|
|
"come",
|
|
"its",
|
|
"over",
|
|
"think",
|
|
"also",
|
|
"back",
|
|
"after",
|
|
"use",
|
|
"two",
|
|
"how",
|
|
"our",
|
|
"work",
|
|
"first",
|
|
"well",
|
|
"way",
|
|
"even",
|
|
"new",
|
|
"want",
|
|
"because",
|
|
"any",
|
|
"these",
|
|
"give",
|
|
"day",
|
|
"most",
|
|
"us",
|
|
]
|