Add models and database persistence for protected phrase extraction
- Introduced dataclasses for phrase candidates, judgments, and matches in `models.py`. - Implemented database operations for candidate and protected phrases in `store.py`, including loading, saving, and deleting phrases. - Enhanced text normalization functions in `text_normalization.py` with detailed docstrings. - Refactored search functionality to utilize new models and methods for detecting protected phrases.
This commit is contained in:
@@ -20,8 +20,7 @@ from python.ebook_search.bm25_corpus import (
|
||||
score_bm25_corpus,
|
||||
)
|
||||
from python.ebook_search.embeddings import MODEL_DIMENSIONS, embed_query, get_embedding_table
|
||||
from python.ebook_search.protected_phrases.lib import (
|
||||
HydratedPhraseMatch,
|
||||
from python.ebook_search.protected_phrases.matching import (
|
||||
detect_protected_phrases_for_query,
|
||||
phrase_hits_for_chunks,
|
||||
)
|
||||
@@ -40,6 +39,7 @@ if TYPE_CHECKING:
|
||||
from sqlalchemy.engine import Engine
|
||||
|
||||
from python.ebook_search.config import EbookSearchConfig
|
||||
from python.ebook_search.protected_phrases.models import HydratedPhraseMatch
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user