Compare commits

..
37 Commits
Author SHA1 Message Date
Richie 19201fa13c feat(chat): add optional response format parameter to request_chat_completion
treefmt / nix fmt (pull_request) Successful in 5s
pytest / pytest (pull_request) Successful in 30s
test ebook search / test-ebook-search (pull_request) Successful in 36s
build_systems / build-brain (pull_request) Successful in 50s
build_systems / build-bob (pull_request) Successful in 53s
build_systems / build-rhapsody-in-green (pull_request) Successful in 1m2s
build_systems / build-jeeves (pull_request) Successful in 2m10s
2026-07-16 13:50:45 -04:00
Richie 699b9322ee feat(ingest): ensure session rollback on ingestion error and log mention count 2026-07-16 13:50:35 -04:00
Richie 13efd3de0d feat(judge_ngrams): enhance phrase judgment with alias filtering and strict boolean handling 2026-07-16 13:50:12 -04:00
Richie 60c9fac079 feat(extraction): add deduplication limit parameter to YAKE extractor 2026-07-16 13:35:15 -04:00
Richie 802a781195 feat(config): add new scoring parameters for phrase extraction and matching 2026-07-16 13:09:34 -04:00
Richie f00df43213 feat(ebook-search): implement async file path resolution and EPUB discovery 2026-07-16 13:09:34 -04:00
Richie 4024672327 feat(prompts): implement TOML-backed prompt loading and refactor message generation 2026-07-16 13:09:34 -04:00
Richie 2f303fe7a8 feat(search): enhance phrase matching and reranking logic with improved async handling 2026-07-16 13:09:34 -04:00
Richie 44a9eb6713 feat(dependencies): update database engine dependencies to support async operations 2026-07-16 13:09:34 -04:00
Richie 7b19e7841a feat(ebook-search): implement async phrase matching for chunks and add ChunkPhraseHit model 2026-07-16 13:09:34 -04:00
Richie 0d347e20a7 refactor(ebook-search): simplify search and phrase matching 2026-07-16 13:09:34 -04:00
Richie ed1ea4546a perf(ebook-search): run phrase detection in parallel with retrieval
Move protected phrase detection into the retrieval gather so it runs
concurrently with vector and BM25 candidates instead of sequentially
before them. Make the search API accept real bool form fields for
rerank/phrase_matching, gate phrase matching on both the request and
config kill switch, and reflow log f-strings for readability.
2026-07-16 13:09:34 -04:00
Richie 49b93c79f6 Refactor logging statements to use f-strings for improved readability and consistency across the codebase. This change enhances the clarity of log messages by directly embedding variable values, making it easier to trace and debug application behavior. 2026-07-16 13:09:34 -04:00
Richie 4a97f5a2d0 feat(dependencies): update sqlalchemy to use asyncio and add aiosqlite and pytest-asyncio 2026-07-16 13:09:34 -04:00
Richie 5925aa301d fix(protected-phrases): isolate phrase generation per book
Run full-book candidate generation inside worker-owned sessions so each book commits independently during backfills. Abort recalculation when a book has no indexed chapters to preserve existing phrase data, and update admin/UI tests for the new generation flow.
2026-07-16 13:09:34 -04:00
Richie 623c73acd0 feat(admin): simplify phrase generation by removing missing phrases endpoint 2026-07-16 13:09:34 -04:00
Richie 5bff9e82fc refactor(ebook): remove spaCy-ner attributes from PhraseCandidate and related functions 2026-07-16 13:09:34 -04:00
Richie ef9ad1a6d6 feat(dependencies): update sqlalchemy to use asyncio and add aiosqlite and pytest-asyncio to dev dependencies 2026-07-16 13:09:34 -04:00
Richie 3e2cd550c6 test(ebook): cover protected phrases and migrate suite to async
Add test_protected_phrases.py covering phrase-matching behavior in the
RAG engine, and update the existing ebook_search tests to use the async
SQLAlchemy engine/session (create_async_engine, AsyncSession) and async
HTTP paths.
2026-07-16 13:09:34 -04:00
Richie 82fae3874b feat(ebook): add additional tokens to junk tokens configuration 2026-07-16 13:09:34 -04:00
Richie 12924cde0c feat(vscode): add new words to spell checker configuration 2026-07-16 13:09:34 -04:00
Richie ddc8421ec2 feat(ebook): improve search UX with grid actions and Enter-to-submit
Add a two-column grid layout for the admin protected-phrases actions
and submit the search form on Enter (Shift+Enter for newline).
2026-07-16 13:09:34 -04:00
Richie 1f5e527cbc feat(ebook): migrate to async DB/HTTP and parallelize phrase pipeline
Convert the ebook-search web app to async end to end and add concurrency
to the protected-phrase extraction and judging pipeline so large books no
longer block the event loop or the UI.

ORM / infra:
- Add get_async_postgres_engine and factor shared URL/connect_args building
  into build_postgres_url (reused by the sync and async engine builders)
- Add async FastAPI session helpers (get_async_db, AsyncDbSession) with
  expire_on_commit=False to avoid implicit IO under asyncio

App:
- Use AsyncEngine/AsyncSession throughout routes, search, ingest, embeddings,
  answer, rerank and LLM calls; convert handlers to async
- Share a single httpx.AsyncClient in app state for LLM requests; size the
  connection pool for concurrent phrase-judging workers
- Add judge_tasks: run per-book judging as tracked background tasks so a
  book already being judged isn't double-queued

Protected phrases:
- Add a process pool (pool.py) and worker-count config
  (extraction/judge book/phrase workers) to parallelize candidate generation
  and judging
- Split admin actions into all/missing variants for generation and judging

Config:
- Add protected_phrase_extraction_workers, phrase_judge_book_workers,
  phrase_judge_phrase_workers
2026-07-16 13:09:34 -04:00
Richie 9bc0d67962 feat(extraction): add cached YAKE extractor for improved performance 2026-07-16 13:09:34 -04:00
Richie d6d90489db feat(ebook): add junk tokens for improved phrase matching 2026-07-16 13:09:34 -04:00
Richie 6c5fcbd0fc feat(orm): add pool_size parameter to get_postgres_engine for connection management 2026-07-16 13:09:34 -04:00
Richie 7b97f31fec Add models and database persistence for protected phrase extraction
- Introduced dataclasses for phrase candidates, judgments, and matches in `models.py`.
- Implemented database operations for candidate and protected phrases in `store.py`, including loading, saving, and deleting phrases.
- Enhanced text normalization functions in `text_normalization.py` with detailed docstrings.
- Refactored search functionality to utilize new models and methods for detecting protected phrases.
2026-07-16 13:09:34 -04:00
Richie 51077cdc92 feat(ebook): add phrase matching display and update search result structure 2026-07-16 13:09:34 -04:00
Richie 6284d72af3 refactor(protected-phrases): extract config and text normalization helpers 2026-07-16 13:09:34 -04:00
Richie 37d0ae487a feat(ebook): update protected phrases with additional tokens and phrases 2026-07-16 13:09:34 -04:00
Richie ac54159496 feat(ebook): enhance phrase judgment logging with failure tracking 2026-07-16 13:09:34 -04:00
Richie 3cf50c860e feat(ebook): implement phrase matching functionality and UI enhancements 2026-07-16 13:09:34 -04:00
Richie 088e58df0b feat(ebook): add admin and book-detail UI for protected phrase pipeline
Expose the protected phrase extraction pipeline through the web UI:

- Admin routes: POST /admin/build-phrases, /admin/generate-ngrams, and
  /admin/judge-ngrams, each wrapping the protected_phrases.lib backfill
  helpers, committing on success, rolling back and rendering an error
  partial on failure, and reporting per-book/candidate/mention counts.
- Book detail page: show candidate, judged, and protected phrase counts,
  list top candidate n-grams (with kept/rejected status) and protected
  phrases, and add a POST /books/{id}/recalculate-phrases action that
  clears and regenerates candidates, then redirects back with a status
  message.
- Admin template: add Generate/Judge n-gram buttons.

Also reflows admin.html to 2-space HTML formatting.
2026-07-16 13:09:34 -04:00
Richie 2427ef05a2 feat(ebook): add protected phrase extraction library with config-driven tuning
Refactor protected phrase handling from a single module into a
python/ebook_search/protected_phrases package covering extraction,
storage, and runtime matching. Phrase filtering is now data-driven via
bundled TOML files: ignored_phrases, bad_starts, bad_ends, and
most_common_words.

Add phrase-tuning settings to EbookSearchConfig so candidate generation,
scoring, LLM judging, and matching are configurable rather than hardcoded:
token bounds, entity token limit, raw n-gram min count, frequency and
chapter-spread score thresholds, candidate/LLM/target caps, confidence
threshold, nesting defaults, and the phrase hit boost.
2026-07-16 13:09:34 -04:00
Richie 41978f631e fix(ebook): enhance EPUB ingestion with error handling and incrmental commits 2026-07-16 13:09:34 -04:00
Richie f49f866575 feat(ebook): add phrase metadata tables for protected phrase matching
Introduce four ORM models and their Alembic migration to support
phrase-based query matching in the ebook RAG engine:

- EbookCandidatePhrase: high-recall phrase candidates extracted per book,
  with source flags (ngram/yake/spacy/capitalized/metadata), scoring, and
  LLM judge results.
- EbookProtectedPhrase: phrases accepted by the LLM judge, with canonical
  id, importance, and nesting controls.
- EbookPhraseAlias: normalized aliases mapping to protected phrases.
- EbookChunkPhraseMention: precomputed phrase occurrences within chunks.

Export the new models from python.orm.richie and add a JSON_DOCUMENT
helper (JSON with JSONB postgres variant) for storing sample contexts.
2026-07-16 13:09:34 -04:00
Richie 247e951a27 Remove legacy HTML templates and API service configuration for contact management system
treefmt / nix fmt (pull_request) Successful in 5s
pytest / pytest (pull_request) Successful in 32s
test ebook search / test-ebook-search (pull_request) Successful in 35s
build_systems / build-brain (pull_request) Successful in 50s
build_systems / build-bob (pull_request) Successful in 51s
build_systems / build-rhapsody-in-green (pull_request) Successful in 1m5s
build_systems / build-jeeves (pull_request) Successful in 2m26s
treefmt / nix fmt (push) Successful in 5s
pytest / pytest (push) Successful in 29s
test ebook search / test-ebook-search (push) Successful in 30s
build_systems / build-brain (push) Successful in 36s
build_systems / build-bob (push) Successful in 36s
build_systems / build-rhapsody-in-green (push) Successful in 49s
build_systems / build-jeeves (push) Successful in 2m9s
- Deleted base template (base.html) and all related contact templates (contact_detail.html, contact_form.html, contact_list.html, graph.html, need_list.html).
- Removed partial templates for managing contacts and needs (contact_table.html, manage_needs.html, manage_relationships.html, need_items.html).
- Eliminated contact API service configuration (contact_api.nix) from the NixOS setup.
2026-07-15 15:28:15 -04:00
21 changed files with 48 additions and 1810 deletions
-1
View File
@@ -1 +0,0 @@
"""FastAPI applications."""
-56
View File
@@ -1,56 +0,0 @@
"""FastAPI interface for Contact database."""
from __future__ import annotations
import logging
from contextlib import asynccontextmanager
from typing import TYPE_CHECKING, Annotated
import typer
import uvicorn
from fastapi import FastAPI
from python.api.routers import contact_router, views_router
from python.common import configure_logger
from python.fastapi_tools import ZstdMiddleware
from python.orm.common import get_postgres_engine
if TYPE_CHECKING:
from collections.abc import AsyncIterator
logger = logging.getLogger(__name__)
def create_app() -> FastAPI:
"""Create and configure the FastAPI application."""
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
"""Manage application lifespan."""
app.state.engine = get_postgres_engine()
yield
app.state.engine.dispose()
app = FastAPI(title="Contact Database API", lifespan=lifespan)
app.add_middleware(ZstdMiddleware)
app.include_router(contact_router)
app.include_router(views_router)
return app
def serve(
host: Annotated[str, typer.Option("--host", "-h", help="Host to bind to")],
port: Annotated[int, typer.Option("--port", "-p", help="Port to bind to")] = 8000,
log_level: Annotated[str, typer.Option("--log-level", "-l", help="Log level")] = "INFO",
) -> None:
"""Start the Contact API server."""
configure_logger(log_level)
app = create_app()
uvicorn.run(app, host=host, port=port)
if __name__ == "__main__":
typer.run(serve)
-6
View File
@@ -1,6 +0,0 @@
"""API routers."""
from python.api.routers.contact import router as contact_router
from python.api.routers.views import router as views_router
__all__ = ["contact_router", "views_router"]
-481
View File
@@ -1,481 +0,0 @@
"""Contact API router."""
from pathlib import Path
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from pydantic import BaseModel
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from python.fastapi_tools.db import DbSession # noqa: TC001 this is a FastAPI needed at runtime
from python.orm.richie.contact import Contact, ContactRelationship, Need, RelationshipType
TEMPLATES_DIR = Path(__file__).parent.parent / "templates"
templates = Jinja2Templates(directory=TEMPLATES_DIR)
def _is_htmx(request: Request) -> bool:
"""Check if the request is from HTMX."""
return request.headers.get("HX-Request") == "true"
class NeedBase(BaseModel):
"""Base schema for Need."""
name: str
description: str | None = None
class NeedCreate(NeedBase):
"""Schema for creating a Need."""
class NeedResponse(NeedBase):
"""Schema for Need response."""
id: int
model_config = {"from_attributes": True}
class ContactRelationshipCreate(BaseModel):
"""Schema for creating a contact relationship."""
related_contact_id: int
relationship_type: RelationshipType
closeness_weight: int | None = None
class ContactRelationshipUpdate(BaseModel):
"""Schema for updating a contact relationship."""
relationship_type: RelationshipType | None = None
closeness_weight: int | None = None
class ContactRelationshipResponse(BaseModel):
"""Schema for contact relationship response."""
contact_id: int
related_contact_id: int
relationship_type: str
closeness_weight: int
model_config = {"from_attributes": True}
class RelationshipTypeInfo(BaseModel):
"""Information about a relationship type."""
value: str
display_name: str
default_weight: int
class GraphNode(BaseModel):
"""Node in the relationship graph."""
id: int
name: str
current_job: str | None = None
class GraphEdge(BaseModel):
"""Edge in the relationship graph."""
source: int
target: int
relationship_type: str
closeness_weight: int
class GraphData(BaseModel):
"""Complete graph data for visualization."""
nodes: list[GraphNode]
edges: list[GraphEdge]
class ContactBase(BaseModel):
"""Base schema for Contact."""
name: str
age: int | None = None
bio: str | None = None
current_job: str | None = None
gender: str | None = None
goals: str | None = None
legal_name: str | None = None
profile_pic: str | None = None
safe_conversation_starters: str | None = None
self_sufficiency_score: int | None = None
social_structure_style: str | None = None
ssn: str | None = None
suffix: str | None = None
timezone: str | None = None
topics_to_avoid: str | None = None
class ContactCreate(ContactBase):
"""Schema for creating a Contact."""
need_ids: list[int] = []
class ContactUpdate(BaseModel):
"""Schema for updating a Contact."""
name: str | None = None
age: int | None = None
bio: str | None = None
current_job: str | None = None
gender: str | None = None
goals: str | None = None
legal_name: str | None = None
profile_pic: str | None = None
safe_conversation_starters: str | None = None
self_sufficiency_score: int | None = None
social_structure_style: str | None = None
ssn: str | None = None
suffix: str | None = None
timezone: str | None = None
topics_to_avoid: str | None = None
need_ids: list[int] | None = None
class ContactResponse(ContactBase):
"""Schema for Contact response with relationships."""
id: int
needs: list[NeedResponse] = []
related_to: list[ContactRelationshipResponse] = []
related_from: list[ContactRelationshipResponse] = []
model_config = {"from_attributes": True}
class ContactListResponse(ContactBase):
"""Schema for Contact list response."""
id: int
model_config = {"from_attributes": True}
router = APIRouter(prefix="/api", tags=["contacts"])
@router.post("/needs", response_model=NeedResponse)
def create_need(need: NeedCreate, db: DbSession) -> Need:
"""Create a new need."""
db_need = Need(name=need.name, description=need.description)
db.add(db_need)
db.commit()
db.refresh(db_need)
return db_need
@router.get("/needs", response_model=list[NeedResponse])
def list_needs(db: DbSession) -> list[Need]:
"""List all needs."""
return list(db.scalars(select(Need)).all())
@router.get("/needs/{need_id}", response_model=NeedResponse)
def get_need(need_id: int, db: DbSession) -> Need:
"""Get a need by ID."""
need = db.get(Need, need_id)
if not need:
raise HTTPException(status_code=404, detail="Need not found")
return need
@router.delete("/needs/{need_id}", response_model=None)
def delete_need(need_id: int, request: Request, db: DbSession) -> dict[str, bool] | HTMLResponse:
"""Delete a need by ID."""
need = db.get(Need, need_id)
if not need:
raise HTTPException(status_code=404, detail="Need not found")
db.delete(need)
db.commit()
if _is_htmx(request):
return HTMLResponse("")
return {"deleted": True}
@router.post("/contacts", response_model=ContactResponse)
def create_contact(contact: ContactCreate, db: DbSession) -> Contact:
"""Create a new contact."""
need_ids = contact.need_ids
contact_data = contact.model_dump(exclude={"need_ids"})
db_contact = Contact(**contact_data)
if need_ids:
needs = list(db.scalars(select(Need).where(Need.id.in_(need_ids))).all())
db_contact.needs = needs
db.add(db_contact)
db.commit()
db.refresh(db_contact)
return db_contact
@router.get("/contacts", response_model=list[ContactListResponse])
def list_contacts(
db: DbSession,
skip: int = 0,
limit: int = 100,
) -> list[Contact]:
"""List all contacts with pagination."""
return list(db.scalars(select(Contact).offset(skip).limit(limit)).all())
@router.get("/contacts/{contact_id}", response_model=ContactResponse)
def get_contact(contact_id: int, db: DbSession) -> Contact:
"""Get a contact by ID with all relationships."""
contact = db.scalar(
select(Contact)
.where(Contact.id == contact_id)
.options(
selectinload(Contact.needs),
selectinload(Contact.related_to),
selectinload(Contact.related_from),
)
)
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
return contact
@router.patch("/contacts/{contact_id}", response_model=ContactResponse)
def update_contact(
contact_id: int,
contact: ContactUpdate,
db: DbSession,
) -> Contact:
"""Update a contact by ID."""
db_contact = db.get(Contact, contact_id)
if not db_contact:
raise HTTPException(status_code=404, detail="Contact not found")
update_data = contact.model_dump(exclude_unset=True)
need_ids = update_data.pop("need_ids", None)
for key, value in update_data.items():
setattr(db_contact, key, value)
if need_ids is not None:
needs = list(db.scalars(select(Need).where(Need.id.in_(need_ids))).all())
db_contact.needs = needs
db.commit()
db.refresh(db_contact)
return db_contact
@router.delete("/contacts/{contact_id}", response_model=None)
def delete_contact(contact_id: int, request: Request, db: DbSession) -> dict[str, bool] | HTMLResponse:
"""Delete a contact by ID."""
contact = db.get(Contact, contact_id)
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
db.delete(contact)
db.commit()
if _is_htmx(request):
return HTMLResponse("")
return {"deleted": True}
@router.post("/contacts/{contact_id}/needs/{need_id}")
def add_need_to_contact(
contact_id: int,
need_id: int,
db: DbSession,
) -> dict[str, bool]:
"""Add a need to a contact."""
contact = db.get(Contact, contact_id)
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
need = db.get(Need, need_id)
if not need:
raise HTTPException(status_code=404, detail="Need not found")
if need not in contact.needs:
contact.needs.append(need)
db.commit()
return {"added": True}
@router.delete("/contacts/{contact_id}/needs/{need_id}", response_model=None)
def remove_need_from_contact(
contact_id: int,
need_id: int,
request: Request,
db: DbSession,
) -> dict[str, bool] | HTMLResponse:
"""Remove a need from a contact."""
contact = db.get(Contact, contact_id)
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
need = db.get(Need, need_id)
if not need:
raise HTTPException(status_code=404, detail="Need not found")
if need in contact.needs:
contact.needs.remove(need)
db.commit()
if _is_htmx(request):
return HTMLResponse("")
return {"removed": True}
@router.post(
"/contacts/{contact_id}/relationships",
response_model=ContactRelationshipResponse,
)
def add_contact_relationship(
contact_id: int,
relationship: ContactRelationshipCreate,
db: DbSession,
) -> ContactRelationship:
"""Add a relationship between two contacts."""
contact = db.get(Contact, contact_id)
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
related_contact = db.get(Contact, relationship.related_contact_id)
if not related_contact:
raise HTTPException(status_code=404, detail="Related contact not found")
if contact_id == relationship.related_contact_id:
raise HTTPException(status_code=400, detail="Cannot relate contact to itself")
# Use provided weight or default from relationship type
weight = relationship.closeness_weight
if weight is None:
weight = relationship.relationship_type.default_weight
db_relationship = ContactRelationship(
contact_id=contact_id,
related_contact_id=relationship.related_contact_id,
relationship_type=relationship.relationship_type.value,
closeness_weight=weight,
)
db.add(db_relationship)
db.commit()
db.refresh(db_relationship)
return db_relationship
@router.get(
"/contacts/{contact_id}/relationships",
response_model=list[ContactRelationshipResponse],
)
def get_contact_relationships(
contact_id: int,
db: DbSession,
) -> list[ContactRelationship]:
"""Get all relationships for a contact."""
contact = db.get(Contact, contact_id)
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
outgoing = list(db.scalars(select(ContactRelationship).where(ContactRelationship.contact_id == contact_id)).all())
incoming = list(
db.scalars(select(ContactRelationship).where(ContactRelationship.related_contact_id == contact_id)).all()
)
return outgoing + incoming
@router.patch(
"/contacts/{contact_id}/relationships/{related_contact_id}",
response_model=ContactRelationshipResponse,
)
def update_contact_relationship(
contact_id: int,
related_contact_id: int,
update: ContactRelationshipUpdate,
db: DbSession,
) -> ContactRelationship:
"""Update a relationship between two contacts."""
relationship = db.scalar(
select(ContactRelationship).where(
ContactRelationship.contact_id == contact_id,
ContactRelationship.related_contact_id == related_contact_id,
)
)
if not relationship:
raise HTTPException(status_code=404, detail="Relationship not found")
if update.relationship_type is not None:
relationship.relationship_type = update.relationship_type.value
if update.closeness_weight is not None:
relationship.closeness_weight = update.closeness_weight
db.commit()
db.refresh(relationship)
return relationship
@router.delete("/contacts/{contact_id}/relationships/{related_contact_id}", response_model=None)
def remove_contact_relationship(
contact_id: int,
related_contact_id: int,
request: Request,
db: DbSession,
) -> dict[str, bool] | HTMLResponse:
"""Remove a relationship between two contacts."""
relationship = db.scalar(
select(ContactRelationship).where(
ContactRelationship.contact_id == contact_id,
ContactRelationship.related_contact_id == related_contact_id,
)
)
if not relationship:
raise HTTPException(status_code=404, detail="Relationship not found")
db.delete(relationship)
db.commit()
if _is_htmx(request):
return HTMLResponse("")
return {"deleted": True}
@router.get("/relationship-types")
def list_relationship_types() -> list[RelationshipTypeInfo]:
"""List all available relationship types with their default weights."""
return [
RelationshipTypeInfo(
value=rt.value,
display_name=rt.display_name,
default_weight=rt.default_weight,
)
for rt in RelationshipType
]
@router.get("/graph")
def get_relationship_graph(db: DbSession) -> GraphData:
"""Get all contacts and relationships as graph data for visualization."""
contacts = list(db.scalars(select(Contact)).all())
relationships = list(db.scalars(select(ContactRelationship)).all())
nodes = [GraphNode(id=c.id, name=c.name, current_job=c.current_job) for c in contacts]
edges = [
GraphEdge(
source=rel.contact_id,
target=rel.related_contact_id,
relationship_type=rel.relationship_type,
closeness_weight=rel.closeness_weight,
)
for rel in relationships
]
return GraphData(nodes=nodes, edges=edges)
-345
View File
@@ -1,345 +0,0 @@
"""HTMX server-rendered view router."""
from pathlib import Path
from typing import Annotated, Any
from fastapi import APIRouter, Form, HTTPException, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from fastapi.templating import Jinja2Templates
from sqlalchemy import select
from sqlalchemy.orm import Session, selectinload
from python.fastapi_tools.db import DbSession # noqa: TC001 this is a FastAPI needed at runtime
from python.orm.richie.contact import Contact, ContactRelationship, Need, RelationshipType
TEMPLATES_DIR = Path(__file__).parent.parent / "templates"
templates = Jinja2Templates(directory=TEMPLATES_DIR)
router = APIRouter(tags=["views"])
FAMILIAL_TYPES = {
"parent",
"child",
"sibling",
"grandparent",
"grandchild",
"aunt_uncle",
"niece_nephew",
"cousin",
"in_law",
}
FRIEND_TYPES = {"best_friend", "close_friend", "friend", "acquaintance", "neighbor"}
PARTNER_TYPES = {"spouse", "partner"}
PROFESSIONAL_TYPES = {"mentor", "mentee", "business_partner", "colleague", "manager", "direct_report", "client"}
CONTACT_STRING_FIELDS = (
"name",
"legal_name",
"suffix",
"gender",
"current_job",
"timezone",
"profile_pic",
"bio",
"goals",
"social_structure_style",
"safe_conversation_starters",
"topics_to_avoid",
"ssn",
)
CONTACT_INT_FIELDS = ("age", "self_sufficiency_score")
def _group_relationships(relationships: list[ContactRelationship]) -> dict[str, list[ContactRelationship]]:
"""Group relationships by category."""
groups: dict[str, list[ContactRelationship]] = {
"familial": [],
"partners": [],
"friends": [],
"professional": [],
"other": [],
}
for rel in relationships:
if rel.relationship_type in FAMILIAL_TYPES:
groups["familial"].append(rel)
elif rel.relationship_type in PARTNER_TYPES:
groups["partners"].append(rel)
elif rel.relationship_type in FRIEND_TYPES:
groups["friends"].append(rel)
elif rel.relationship_type in PROFESSIONAL_TYPES:
groups["professional"].append(rel)
else:
groups["other"].append(rel)
return groups
def _build_contact_name_map(database: Session, contact: Contact) -> dict[int, str]:
"""Build a mapping of contact IDs to names for relationship display."""
related_ids = {rel.related_contact_id for rel in contact.related_to}
related_ids |= {rel.contact_id for rel in contact.related_from}
related_ids.discard(contact.id)
if not related_ids:
return {}
related_contacts = list(database.scalars(select(Contact).where(Contact.id.in_(related_ids))).all())
return {related.id: related.name for related in related_contacts}
def _get_relationship_type_display() -> dict[str, str]:
"""Build a mapping of relationship type values to display names."""
return {rel_type.value: rel_type.display_name for rel_type in RelationshipType}
async def _parse_contact_form(request: Request) -> dict[str, Any]:
"""Parse contact form data from a multipart/form request."""
form_data = await request.form()
result: dict[str, Any] = {}
for field in CONTACT_STRING_FIELDS:
value = form_data.get(field, "")
result[field] = str(value) if value else None
for field in CONTACT_INT_FIELDS:
value = form_data.get(field, "")
result[field] = int(value) if value else None
result["need_ids"] = [int(value) for value in form_data.getlist("need_ids")]
return result
def _save_contact_from_form(database: Session, contact: Contact, form_result: dict[str, Any]) -> None:
"""Apply parsed form data to a Contact and save associated needs."""
need_ids = form_result.pop("need_ids")
for key, value in form_result.items():
setattr(contact, key, value)
if need_ids:
contact.needs = list(database.scalars(select(Need).where(Need.id.in_(need_ids))).all())
else:
contact.needs = []
@router.get("/", response_class=HTMLResponse)
@router.get("/contacts", response_class=HTMLResponse)
def contact_list_page(request: Request, database: DbSession) -> HTMLResponse:
"""Render the contacts list page."""
contacts = list(database.scalars(select(Contact)).all())
return templates.TemplateResponse(request, "contact_list.html", {"contacts": contacts})
@router.get("/contacts/new", response_class=HTMLResponse)
def new_contact_page(request: Request, database: DbSession) -> HTMLResponse:
"""Render the new contact form page."""
all_needs = list(database.scalars(select(Need)).all())
return templates.TemplateResponse(request, "contact_form.html", {"contact": None, "all_needs": all_needs})
@router.post("/htmx/contacts/new")
async def create_contact_form(request: Request, database: DbSession) -> RedirectResponse:
"""Handle the create contact form submission."""
form_result = await _parse_contact_form(request)
contact = Contact()
_save_contact_from_form(database, contact, form_result)
database.add(contact)
database.commit()
database.refresh(contact)
return RedirectResponse(url=f"/contacts/{contact.id}", status_code=303)
@router.get("/contacts/{contact_id}", response_class=HTMLResponse)
def contact_detail_page(contact_id: int, request: Request, database: DbSession) -> HTMLResponse:
"""Render the contact detail page."""
contact = database.scalar(
select(Contact)
.where(Contact.id == contact_id)
.options(
selectinload(Contact.needs),
selectinload(Contact.related_to),
selectinload(Contact.related_from),
)
)
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
contact_names = _build_contact_name_map(database, contact)
grouped_relationships = _group_relationships(contact.related_to)
all_contacts = list(database.scalars(select(Contact)).all())
all_needs = list(database.scalars(select(Need)).all())
available_needs = [need for need in all_needs if need not in contact.needs]
return templates.TemplateResponse(
request,
"contact_detail.html",
{
"contact": contact,
"contact_names": contact_names,
"grouped_relationships": grouped_relationships,
"all_contacts": all_contacts,
"available_needs": available_needs,
"relationship_types": list(RelationshipType),
},
)
@router.get("/contacts/{contact_id}/edit", response_class=HTMLResponse)
def edit_contact_page(contact_id: int, request: Request, database: DbSession) -> HTMLResponse:
"""Render the edit contact form page."""
contact = database.scalar(select(Contact).where(Contact.id == contact_id).options(selectinload(Contact.needs)))
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
all_needs = list(database.scalars(select(Need)).all())
return templates.TemplateResponse(request, "contact_form.html", {"contact": contact, "all_needs": all_needs})
@router.post("/htmx/contacts/{contact_id}/edit")
async def update_contact_form(contact_id: int, request: Request, database: DbSession) -> RedirectResponse:
"""Handle the edit contact form submission."""
contact = database.get(Contact, contact_id)
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
form_result = await _parse_contact_form(request)
_save_contact_from_form(database, contact, form_result)
database.commit()
return RedirectResponse(url=f"/contacts/{contact_id}", status_code=303)
@router.post("/htmx/contacts/{contact_id}/add-need", response_class=HTMLResponse)
def add_need_to_contact_htmx(
contact_id: int,
request: Request,
database: DbSession,
need_id: Annotated[int, Form()],
) -> HTMLResponse:
"""Add a need to a contact and return updated manage-needs partial."""
contact = database.scalar(select(Contact).where(Contact.id == contact_id).options(selectinload(Contact.needs)))
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
need = database.get(Need, need_id)
if not need:
raise HTTPException(status_code=404, detail="Need not found")
if need not in contact.needs:
contact.needs.append(need)
database.commit()
database.refresh(contact)
return templates.TemplateResponse(request, "partials/manage_needs.html", {"contact": contact})
@router.post("/htmx/contacts/{contact_id}/add-relationship", response_class=HTMLResponse)
def add_relationship_htmx(
contact_id: int,
request: Request,
database: DbSession,
related_contact_id: Annotated[int, Form()],
relationship_type: Annotated[str, Form()],
) -> HTMLResponse:
"""Add a relationship and return updated manage-relationships partial."""
contact = database.scalar(select(Contact).where(Contact.id == contact_id).options(selectinload(Contact.related_to)))
if not contact:
raise HTTPException(status_code=404, detail="Contact not found")
related_contact = database.get(Contact, related_contact_id)
if not related_contact:
raise HTTPException(status_code=404, detail="Related contact not found")
rel_type = RelationshipType(relationship_type)
weight = rel_type.default_weight
relationship = ContactRelationship(
contact_id=contact_id,
related_contact_id=related_contact_id,
relationship_type=relationship_type,
closeness_weight=weight,
)
database.add(relationship)
database.commit()
database.refresh(contact)
contact_names = _build_contact_name_map(database, contact)
return templates.TemplateResponse(
request,
"partials/manage_relationships.html",
{"contact": contact, "contact_names": contact_names},
)
@router.post("/htmx/contacts/{contact_id}/relationships/{related_contact_id}/weight")
def update_relationship_weight_htmx(
contact_id: int,
related_contact_id: int,
database: DbSession,
closeness_weight: Annotated[int, Form()],
) -> HTMLResponse:
"""Update a relationship's closeness weight from HTMX range input."""
relationship = database.scalar(
select(ContactRelationship).where(
ContactRelationship.contact_id == contact_id,
ContactRelationship.related_contact_id == related_contact_id,
)
)
if not relationship:
raise HTTPException(status_code=404, detail="Relationship not found")
relationship.closeness_weight = closeness_weight
database.commit()
return HTMLResponse("")
@router.post("/htmx/needs", response_class=HTMLResponse)
def create_need_htmx(
request: Request,
database: DbSession,
name: Annotated[str, Form()],
description: Annotated[str, Form()] = "",
) -> HTMLResponse:
"""Create a need via form data and return updated needs list."""
need = Need(name=name, description=description or None)
database.add(need)
database.commit()
needs = list(database.scalars(select(Need)).all())
return templates.TemplateResponse(request, "partials/need_items.html", {"needs": needs})
@router.get("/needs", response_class=HTMLResponse)
def needs_page(request: Request, database: DbSession) -> HTMLResponse:
"""Render the needs list page."""
needs = list(database.scalars(select(Need)).all())
return templates.TemplateResponse(request, "need_list.html", {"needs": needs})
@router.get("/graph", response_class=HTMLResponse)
def graph_page(request: Request, database: DbSession) -> HTMLResponse:
"""Render the relationship graph page."""
contacts = list(database.scalars(select(Contact)).all())
relationships = list(database.scalars(select(ContactRelationship)).all())
graph_data = {
"nodes": [{"id": contact.id, "name": contact.name, "current_job": contact.current_job} for contact in contacts],
"edges": [
{
"source": rel.contact_id,
"target": rel.related_contact_id,
"relationship_type": rel.relationship_type,
"closeness_weight": rel.closeness_weight,
}
for rel in relationships
],
}
return templates.TemplateResponse(
request,
"graph.html",
{
"graph_data": graph_data,
"relationship_type_display": _get_relationship_type_display(),
},
)
-198
View File
@@ -1,198 +0,0 @@
<!DOCTYPE html>
<html lang="en" data-theme="light">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{% block title %}Contact Database{% endblock %}</title>
<script src="https://unpkg.com/htmx.org@2.0.4"></script>
<style>
:root {
--color-bg: #f5f5f5;
--color-bg-card: #ffffff;
--color-bg-hover: #f0f0f0;
--color-bg-muted: #f9f9f9;
--color-bg-error: #ffe0e0;
--color-text: #333333;
--color-text-muted: #666666;
--color-text-error: #cc0000;
--color-border: #dddddd;
--color-border-light: #eeeeee;
--color-border-lighter: #f0f0f0;
--color-primary: #0066cc;
--color-primary-hover: #0055aa;
--color-danger: #cc3333;
--color-danger-hover: #aa2222;
--color-tag-bg: #e0e0e0;
--shadow: 0 1px 3px rgba(0, 0, 0, 0.1);
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
line-height: 1.5;
color: var(--color-text);
background-color: var(--color-bg);
}
[data-theme="dark"] {
--color-bg: #1a1a1a;
--color-bg-card: #2d2d2d;
--color-bg-hover: #3d3d3d;
--color-bg-muted: #252525;
--color-bg-error: #4a2020;
--color-text: #e0e0e0;
--color-text-muted: #a0a0a0;
--color-text-error: #ff6b6b;
--color-border: #404040;
--color-border-light: #353535;
--color-border-lighter: #303030;
--color-primary: #4da6ff;
--color-primary-hover: #7dbfff;
--color-danger: #ff6b6b;
--color-danger-hover: #ff8a8a;
--color-tag-bg: #404040;
--shadow: 0 1px 3px rgba(0, 0, 0, 0.3);
}
* { box-sizing: border-box; }
body { margin: 0; background: var(--color-bg); color: var(--color-text); }
.app { max-width: 1000px; margin: 0 auto; padding: 20px; }
nav { display: flex; align-items: center; gap: 20px; padding: 15px 0; border-bottom: 1px solid var(--color-border); margin-bottom: 20px; }
nav a { color: var(--color-primary); text-decoration: none; font-weight: 500; }
nav a:hover { text-decoration: underline; }
.theme-toggle { margin-left: auto; }
main { background: var(--color-bg-card); padding: 20px; border-radius: 8px; box-shadow: var(--shadow); }
.header { display: flex; justify-content: space-between; align-items: center; margin-bottom: 20px; }
.header h1 { margin: 0; }
a { color: var(--color-primary); }
a:hover { text-decoration: underline; }
.btn { display: inline-block; padding: 8px 16px; border: 1px solid var(--color-border); border-radius: 4px; background: var(--color-bg-card); color: var(--color-text); text-decoration: none; cursor: pointer; font-size: 14px; margin-left: 8px; }
.btn:hover { background: var(--color-bg-hover); }
.btn-primary { background: var(--color-primary); border-color: var(--color-primary); color: white; }
.btn-primary:hover { background: var(--color-primary-hover); }
.btn-danger { background: var(--color-danger); border-color: var(--color-danger); color: white; }
.btn-danger:hover { background: var(--color-danger-hover); }
.btn-small { padding: 4px 8px; font-size: 12px; }
.btn:disabled { opacity: 0.6; cursor: not-allowed; }
table { width: 100%; border-collapse: collapse; }
th, td { padding: 12px; text-align: left; border-bottom: 1px solid var(--color-border-light); }
th { font-weight: 600; background: var(--color-bg-muted); }
tr:hover { background: var(--color-bg-muted); }
.error { background: var(--color-bg-error); color: var(--color-text-error); padding: 10px; border-radius: 4px; margin-bottom: 20px; }
.tag { display: inline-block; background: var(--color-tag-bg); padding: 2px 8px; border-radius: 12px; font-size: 12px; color: var(--color-text-muted); }
.add-form { display: flex; gap: 10px; margin-top: 15px; flex-wrap: wrap; }
.add-form select, .add-form input { padding: 8px; border: 1px solid var(--color-border); border-radius: 4px; min-width: 200px; background: var(--color-bg-card); color: var(--color-text); }
.form-group { margin-bottom: 20px; }
.form-group label { display: block; font-weight: 500; margin-bottom: 5px; }
.form-group input, .form-group textarea, .form-group select { width: 100%; padding: 10px; border: 1px solid var(--color-border); border-radius: 4px; font-size: 14px; background: var(--color-bg-card); color: var(--color-text); }
.form-group textarea { resize: vertical; }
.form-row { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; }
.checkbox-group { display: flex; flex-wrap: wrap; gap: 15px; }
.checkbox-label { display: flex; align-items: center; gap: 5px; cursor: pointer; }
.form-actions { display: flex; gap: 10px; margin-top: 30px; padding-top: 20px; border-top: 1px solid var(--color-border-light); }
.need-form { background: var(--color-bg-muted); padding: 20px; border-radius: 4px; margin-bottom: 20px; }
.need-items { list-style: none; padding: 0; }
.need-items li { display: flex; justify-content: space-between; align-items: flex-start; padding: 15px; border: 1px solid var(--color-border-light); border-radius: 4px; margin-bottom: 10px; }
.need-info p { margin: 5px 0 0; color: var(--color-text-muted); font-size: 14px; }
.graph-container { width: 100%; }
.graph-hint { color: var(--color-text-muted); font-size: 14px; margin-bottom: 15px; }
.selected-info { margin-top: 15px; padding: 15px; background: var(--color-bg-muted); border-radius: 8px; }
.selected-info h3 { margin: 0 0 10px; }
.selected-info p { margin: 5px 0; color: var(--color-text-muted); }
.legend { margin-top: 20px; padding: 15px; background: var(--color-bg-muted); border-radius: 8px; }
.legend h4 { margin: 0 0 10px; font-size: 14px; }
.legend-items { display: flex; flex-wrap: wrap; gap: 15px; }
.legend-item { display: flex; align-items: center; gap: 8px; font-size: 12px; color: var(--color-text-muted); }
.legend-line { width: 30px; border-radius: 2px; }
.id-card { width: 100%; }
.id-card-inner { background: linear-gradient(135deg, #0a0a0f 0%, #1a1a2e 50%, #0a0a0f 100%); background-image: radial-gradient(white 1px, transparent 1px), linear-gradient(135deg, #0a0a0f 0%, #1a1a2e 50%, #0a0a0f 100%); background-size: 50px 50px, 100% 100%; color: #fff; border-radius: 12px; padding: 25px; min-height: 500px; position: relative; overflow: hidden; }
.id-card-header { display: flex; justify-content: space-between; align-items: flex-start; margin-bottom: 15px; }
.id-card-header-left { flex: 1; }
.id-card-header-right { display: flex; flex-direction: column; align-items: flex-end; gap: 10px; }
.id-card-title { font-size: 2.5rem; font-weight: 700; margin: 0; color: #fff; text-shadow: 2px 2px 4px rgba(0,0,0,0.5); }
.id-profile-pic { width: 80px; height: 80px; border-radius: 8px; object-fit: cover; border: 2px solid rgba(255,255,255,0.3); }
.id-profile-placeholder { width: 80px; height: 80px; border-radius: 8px; background: linear-gradient(135deg, #4ecdc4 0%, #44a8a0 100%); display: flex; align-items: center; justify-content: center; border: 2px solid rgba(255,255,255,0.3); }
.id-profile-placeholder span { font-size: 2rem; font-weight: 700; color: #fff; text-shadow: 1px 1px 2px rgba(0,0,0,0.3); }
.id-card-actions { display: flex; gap: 8px; }
.id-card-actions .btn { background: rgba(255,255,255,0.1); border-color: rgba(255,255,255,0.3); color: #fff; }
.id-card-actions .btn:hover { background: rgba(255,255,255,0.2); }
.id-card-body { display: grid; grid-template-columns: 1fr 1.5fr; gap: 30px; }
.id-card-left { display: flex; flex-direction: column; gap: 8px; }
.id-field { font-size: 1rem; line-height: 1.4; }
.id-field-block { margin-top: 15px; font-size: 0.95rem; line-height: 1.5; }
.id-label { color: #4ecdc4; font-weight: 500; }
.id-card-right { display: flex; flex-direction: column; gap: 20px; }
.id-bio { font-size: 0.9rem; line-height: 1.6; color: #e0e0e0; }
.id-relationships { margin-top: 10px; }
.id-section-title { font-size: 1.5rem; margin: 0 0 15px; color: #fff; border-bottom: 1px solid rgba(255,255,255,0.2); padding-bottom: 8px; }
.id-rel-group { margin-bottom: 12px; font-size: 0.9rem; line-height: 1.6; }
.id-rel-label { color: #a0a0a0; }
.id-rel-group a { color: #4ecdc4; text-decoration: none; }
.id-rel-group a:hover { text-decoration: underline; }
.id-rel-type { color: #888; font-size: 0.85em; }
.id-card-warnings { margin-top: 30px; padding-top: 20px; border-top: 1px solid rgba(255,255,255,0.2); display: flex; flex-wrap: wrap; gap: 20px; }
.id-warning { display: flex; align-items: center; gap: 8px; font-size: 0.9rem; color: #ff6b6b; }
.warning-dot { width: 8px; height: 8px; background: #ff6b6b; border-radius: 50%; flex-shrink: 0; }
.warning-desc { color: #ccc; }
.id-card-manage { margin-top: 20px; background: var(--color-bg-muted); border-radius: 8px; padding: 15px; }
.id-card-manage summary { cursor: pointer; font-weight: 600; font-size: 1.1rem; padding: 5px 0; }
.id-card-manage[open] summary { margin-bottom: 15px; border-bottom: 1px solid var(--color-border-light); padding-bottom: 10px; }
.manage-section { margin-bottom: 25px; }
.manage-section h3 { margin: 0 0 15px; font-size: 1rem; }
.manage-relationships { display: flex; flex-direction: column; gap: 10px; margin-bottom: 15px; }
.manage-rel-item { display: flex; align-items: center; gap: 12px; padding: 10px; background: var(--color-bg-card); border-radius: 6px; flex-wrap: wrap; }
.manage-rel-item a { font-weight: 500; min-width: 120px; }
.weight-control { display: flex; align-items: center; gap: 8px; font-size: 12px; color: var(--color-text-muted); }
.weight-control input[type="range"] { width: 80px; cursor: pointer; }
.weight-value { min-width: 20px; text-align: center; font-weight: 600; }
.manage-needs-list { list-style: none; padding: 0; margin: 0 0 15px; }
.manage-needs-list li { display: flex; align-items: center; gap: 12px; padding: 10px; background: var(--color-bg-card); border-radius: 6px; margin-bottom: 8px; }
.manage-needs-list li .btn { margin-left: auto; }
.htmx-indicator { display: none; }
.htmx-request .htmx-indicator { display: inline; }
.htmx-request.htmx-indicator { display: inline; }
@media (max-width: 768px) {
.id-card-body { grid-template-columns: 1fr; }
.id-card-title { font-size: 1.8rem; }
.id-card-header { flex-direction: column; gap: 15px; }
}
</style>
</head>
<body>
<div class="app">
<nav>
<a href="/contacts">Contacts</a>
<a href="/graph">Graph</a>
<a href="/needs">Needs</a>
<button class="btn btn-small theme-toggle" onclick="toggleTheme()">
<span id="theme-label">Dark</span>
</button>
</nav>
<main id="main-content">
{% block content %}{% endblock %}
</main>
</div>
<script>
function toggleTheme() {
const html = document.documentElement;
const current = html.getAttribute('data-theme');
const next = current === 'light' ? 'dark' : 'light';
html.setAttribute('data-theme', next);
localStorage.setItem('theme', next);
document.getElementById('theme-label').textContent = next === 'light' ? 'Dark' : 'Light';
}
(function() {
const saved = localStorage.getItem('theme') || 'light';
document.documentElement.setAttribute('data-theme', saved);
document.getElementById('theme-label').textContent = saved === 'light' ? 'Dark' : 'Light';
})();
</script>
</body>
</html>
-204
View File
@@ -1,204 +0,0 @@
{% extends "base.html" %}
{% block title %}{{ contact.name }}{% endblock %}
{% block content %}
<div class="id-card">
<div class="id-card-inner">
<div class="id-card-header">
<div class="id-card-header-left">
<h1 class="id-card-title">I.D.: {{ contact.name }}</h1>
</div>
<div class="id-card-header-right">
{% if contact.profile_pic %}
<img src="{{ contact.profile_pic }}" alt="{{ contact.name }}'s profile" class="id-profile-pic">
{% else %}
<div class="id-profile-placeholder">
<span>{{ contact.name[0]|upper }}</span>
</div>
{% endif %}
<div class="id-card-actions">
<a href="/contacts/{{ contact.id }}/edit" class="btn btn-small">Edit</a>
<a href="/contacts" class="btn btn-small">Back</a>
</div>
</div>
</div>
<div class="id-card-body">
<div class="id-card-left">
{% if contact.legal_name %}
<div class="id-field">Legal name: {{ contact.legal_name }}</div>
{% endif %}
{% if contact.suffix %}
<div class="id-field">Suffix: {{ contact.suffix }}</div>
{% endif %}
{% if contact.gender %}
<div class="id-field">Gender: {{ contact.gender }}</div>
{% endif %}
{% if contact.age %}
<div class="id-field">Age: {{ contact.age }}</div>
{% endif %}
{% if contact.current_job %}
<div class="id-field">Job: {{ contact.current_job }}</div>
{% endif %}
{% if contact.social_structure_style %}
<div class="id-field">Social style: {{ contact.social_structure_style }}</div>
{% endif %}
{% if contact.self_sufficiency_score is not none %}
<div class="id-field">Self-Sufficiency: {{ contact.self_sufficiency_score }}</div>
{% endif %}
{% if contact.timezone %}
<div class="id-field">Timezone: {{ contact.timezone }}</div>
{% endif %}
{% if contact.safe_conversation_starters %}
<div class="id-field-block">
<span class="id-label">Safe con starters:</span> {{ contact.safe_conversation_starters }}
</div>
{% endif %}
{% if contact.topics_to_avoid %}
<div class="id-field-block">
<span class="id-label">Topics to avoid:</span> {{ contact.topics_to_avoid }}
</div>
{% endif %}
{% if contact.goals %}
<div class="id-field-block">
<span class="id-label">Goals:</span> {{ contact.goals }}
</div>
{% endif %}
</div>
<div class="id-card-right">
{% if contact.bio %}
<div class="id-bio">
<span class="id-label">Bio:</span> {{ contact.bio }}
</div>
{% endif %}
<div class="id-relationships">
<h2 class="id-section-title">Relationships</h2>
{% if grouped_relationships.familial %}
<div class="id-rel-group">
<span class="id-rel-label">Familial:</span>
{% for rel in grouped_relationships.familial %}
<a href="/contacts/{{ rel.related_contact_id }}">{{ contact_names[rel.related_contact_id] }}</a><span class="id-rel-type">({{ rel.relationship_type|replace("_", " ")|title }})</span>{% if not loop.last %}, {% endif %}
{% endfor %}
</div>
{% endif %}
{% if grouped_relationships.partners %}
<div class="id-rel-group">
<span class="id-rel-label">Partners:</span>
{% for rel in grouped_relationships.partners %}
<a href="/contacts/{{ rel.related_contact_id }}">{{ contact_names[rel.related_contact_id] }}</a>{% if not loop.last %}, {% endif %}
{% endfor %}
</div>
{% endif %}
{% if grouped_relationships.friends %}
<div class="id-rel-group">
<span class="id-rel-label">Friends:</span>
{% for rel in grouped_relationships.friends %}
<a href="/contacts/{{ rel.related_contact_id }}">{{ contact_names[rel.related_contact_id] }}</a>{% if not loop.last %}, {% endif %}
{% endfor %}
</div>
{% endif %}
{% if grouped_relationships.professional %}
<div class="id-rel-group">
<span class="id-rel-label">Professional:</span>
{% for rel in grouped_relationships.professional %}
<a href="/contacts/{{ rel.related_contact_id }}">{{ contact_names[rel.related_contact_id] }}</a><span class="id-rel-type">({{ rel.relationship_type|replace("_", " ")|title }})</span>{% if not loop.last %}, {% endif %}
{% endfor %}
</div>
{% endif %}
{% if grouped_relationships.other %}
<div class="id-rel-group">
<span class="id-rel-label">Other:</span>
{% for rel in grouped_relationships.other %}
<a href="/contacts/{{ rel.related_contact_id }}">{{ contact_names[rel.related_contact_id] }}</a><span class="id-rel-type">({{ rel.relationship_type|replace("_", " ")|title }})</span>{% if not loop.last %}, {% endif %}
{% endfor %}
</div>
{% endif %}
{% if contact.related_from %}
<div class="id-rel-group">
<span class="id-rel-label">Known by:</span>
{% for rel in contact.related_from %}
<a href="/contacts/{{ rel.contact_id }}">{{ contact_names[rel.contact_id] }}</a>{% if not loop.last %}, {% endif %}
{% endfor %}
</div>
{% endif %}
</div>
</div>
</div>
{% if contact.needs %}
<div class="id-card-warnings">
{% for need in contact.needs %}
<div class="id-warning">
<span class="warning-dot"></span>
Warning: {{ need.name }}
{% if need.description %}<span class="warning-desc"> - {{ need.description }}</span>{% endif %}
</div>
{% endfor %}
</div>
{% endif %}
</div>
<details class="id-card-manage">
<summary>Manage Contact</summary>
<div class="manage-section">
<h3>Manage Relationships</h3>
<div id="manage-relationships" class="manage-relationships">
{% include "partials/manage_relationships.html" %}
</div>
{% if all_contacts %}
<form hx-post="/htmx/contacts/{{ contact.id }}/add-relationship"
hx-target="#manage-relationships"
hx-swap="innerHTML"
class="add-form">
<select name="related_contact_id" required>
<option value="">Select contact...</option>
{% for other in all_contacts %}
{% if other.id != contact.id %}
<option value="{{ other.id }}">{{ other.name }}</option>
{% endif %}
{% endfor %}
</select>
<select name="relationship_type" required>
<option value="">Select relationship type...</option>
{% for rel_type in relationship_types %}
<option value="{{ rel_type.value }}">{{ rel_type.display_name }}</option>
{% endfor %}
</select>
<button type="submit" class="btn btn-primary">Add Relationship</button>
</form>
{% endif %}
</div>
<div class="manage-section">
<h3>Manage Needs/Warnings</h3>
<div id="manage-needs">
{% include "partials/manage_needs.html" %}
</div>
{% if available_needs %}
<form hx-post="/htmx/contacts/{{ contact.id }}/add-need"
hx-target="#manage-needs"
hx-swap="innerHTML"
class="add-form">
<select name="need_id" required>
<option value="">Select a need...</option>
{% for need in available_needs %}
<option value="{{ need.id }}">{{ need.name }}</option>
{% endfor %}
</select>
<button type="submit" class="btn btn-primary">Add Need</button>
</form>
{% endif %}
</div>
</details>
</div>
{% endblock %}
-115
View File
@@ -1,115 +0,0 @@
{% extends "base.html" %}
{% block title %}{{ "Edit " + contact.name if contact else "New Contact" }}{% endblock %}
{% block content %}
<div class="contact-form">
<h1>{{ "Edit Contact" if contact else "New Contact" }}</h1>
{% if contact %}
<form method="post" action="/htmx/contacts/{{ contact.id }}/edit">
{% else %}
<form method="post" action="/htmx/contacts/new">
{% endif %}
<div class="form-group">
<label for="name">Name *</label>
<input id="name" name="name" type="text" value="{{ contact.name if contact else '' }}" required>
</div>
<div class="form-row">
<div class="form-group">
<label for="legal_name">Legal Name</label>
<input id="legal_name" name="legal_name" type="text" value="{{ contact.legal_name or '' }}">
</div>
<div class="form-group">
<label for="suffix">Suffix</label>
<input id="suffix" name="suffix" type="text" value="{{ contact.suffix or '' }}">
</div>
</div>
<div class="form-row">
<div class="form-group">
<label for="age">Age</label>
<input id="age" name="age" type="number" value="{{ contact.age if contact and contact.age is not none else '' }}">
</div>
<div class="form-group">
<label for="gender">Gender</label>
<input id="gender" name="gender" type="text" value="{{ contact.gender or '' }}">
</div>
</div>
<div class="form-group">
<label for="current_job">Current Job</label>
<input id="current_job" name="current_job" type="text" value="{{ contact.current_job or '' }}">
</div>
<div class="form-group">
<label for="timezone">Timezone</label>
<input id="timezone" name="timezone" type="text" value="{{ contact.timezone or '' }}">
</div>
<div class="form-group">
<label for="profile_pic">Profile Picture URL</label>
<input id="profile_pic" name="profile_pic" type="url" placeholder="https://example.com/photo.jpg" value="{{ contact.profile_pic or '' }}">
</div>
<div class="form-group">
<label for="bio">Bio</label>
<textarea id="bio" name="bio" rows="3">{{ contact.bio or '' }}</textarea>
</div>
<div class="form-group">
<label for="goals">Goals</label>
<textarea id="goals" name="goals" rows="3">{{ contact.goals or '' }}</textarea>
</div>
<div class="form-group">
<label for="social_structure_style">Social Structure Style</label>
<input id="social_structure_style" name="social_structure_style" type="text" value="{{ contact.social_structure_style or '' }}">
</div>
<div class="form-group">
<label for="self_sufficiency_score">Self-Sufficiency Score (1-10)</label>
<input id="self_sufficiency_score" name="self_sufficiency_score" type="number" min="1" max="10" value="{{ contact.self_sufficiency_score if contact and contact.self_sufficiency_score is not none else '' }}">
</div>
<div class="form-group">
<label for="safe_conversation_starters">Safe Conversation Starters</label>
<textarea id="safe_conversation_starters" name="safe_conversation_starters" rows="2">{{ contact.safe_conversation_starters or '' }}</textarea>
</div>
<div class="form-group">
<label for="topics_to_avoid">Topics to Avoid</label>
<textarea id="topics_to_avoid" name="topics_to_avoid" rows="2">{{ contact.topics_to_avoid or '' }}</textarea>
</div>
<div class="form-group">
<label for="ssn">SSN</label>
<input id="ssn" name="ssn" type="text" value="{{ contact.ssn or '' }}">
</div>
{% if all_needs %}
<div class="form-group">
<label>Needs/Accommodations</label>
<div class="checkbox-group">
{% for need in all_needs %}
<label class="checkbox-label">
<input type="checkbox" name="need_ids" value="{{ need.id }}"
{% if contact and need in contact.needs %}checked{% endif %}>
{{ need.name }}
</label>
{% endfor %}
</div>
</div>
{% endif %}
<div class="form-actions">
<button type="submit" class="btn btn-primary">Save</button>
{% if contact %}
<a href="/contacts/{{ contact.id }}" class="btn">Cancel</a>
{% else %}
<a href="/contacts" class="btn">Cancel</a>
{% endif %}
</div>
</form>
</div>
{% endblock %}
-14
View File
@@ -1,14 +0,0 @@
{% extends "base.html" %}
{% block title %}Contacts{% endblock %}
{% block content %}
<div class="contact-list">
<div class="header">
<h1>Contacts</h1>
<a href="/contacts/new" class="btn btn-primary">Add Contact</a>
</div>
<div id="contact-table">
{% include "partials/contact_table.html" %}
</div>
</div>
{% endblock %}
-198
View File
@@ -1,198 +0,0 @@
{% extends "base.html" %}
{% block title %}Relationship Graph{% endblock %}
{% block content %}
<div class="graph-container">
<div class="header">
<h1>Relationship Graph</h1>
</div>
<p class="graph-hint">Drag nodes to reposition. Closer relationships have shorter, darker edges.</p>
<canvas id="graph-canvas" width="900" height="600"
style="border: 1px solid var(--color-border); border-radius: 8px; background: var(--color-bg); cursor: grab;">
</canvas>
<div id="selected-info"></div>
<div class="legend">
<h4>Relationship Closeness (1-10)</h4>
<div class="legend-items">
<div class="legend-item">
<span class="legend-line" style="background: hsl(220, 70%, 40%); height: 4px; display: inline-block;"></span>
<span>10 - Very Close (Spouse, Partner)</span>
</div>
<div class="legend-item">
<span class="legend-line" style="background: hsl(220, 70%, 52%); height: 3px; display: inline-block;"></span>
<span>7 - Close (Family, Best Friend)</span>
</div>
<div class="legend-item">
<span class="legend-line" style="background: hsl(220, 70%, 64%); height: 2px; display: inline-block;"></span>
<span>4 - Moderate (Friend, Colleague)</span>
</div>
<div class="legend-item">
<span class="legend-line" style="background: hsl(220, 70%, 72%); height: 1px; display: inline-block;"></span>
<span>2 - Distant (Acquaintance)</span>
</div>
</div>
</div>
</div>
<script>
(function() {
const RELATIONSHIP_DISPLAY = {{ relationship_type_display|tojson }};
const graphData = {{ graph_data|tojson }};
const canvas = document.getElementById('graph-canvas');
const ctx = canvas.getContext('2d');
const width = canvas.width;
const height = canvas.height;
const centerX = width / 2;
const centerY = height / 2;
const nodes = graphData.nodes.map(function(node) {
return Object.assign({}, node, {
x: centerX + (Math.random() - 0.5) * 300,
y: centerY + (Math.random() - 0.5) * 300,
vx: 0,
vy: 0
});
});
const nodeMap = new Map(nodes.map(function(node) { return [node.id, node]; }));
const edges = graphData.edges.map(function(edge) {
const sourceNode = nodeMap.get(edge.source);
const targetNode = nodeMap.get(edge.target);
if (!sourceNode || !targetNode) return null;
return Object.assign({}, edge, { sourceNode: sourceNode, targetNode: targetNode });
}).filter(function(edge) { return edge !== null; });
let dragNode = null;
let selectedNode = null;
const repulsion = 5000;
const springStrength = 0.05;
const baseSpringLength = 150;
const damping = 0.9;
const centerPull = 0.01;
function simulate() {
for (const node of nodes) { node.vx = 0; node.vy = 0; }
for (let i = 0; i < nodes.length; i++) {
for (let j = i + 1; j < nodes.length; j++) {
const dx = nodes[j].x - nodes[i].x;
const dy = nodes[j].y - nodes[i].y;
const dist = Math.sqrt(dx * dx + dy * dy) || 1;
const force = repulsion / (dist * dist);
const fx = (dx / dist) * force;
const fy = (dy / dist) * force;
nodes[i].vx -= fx; nodes[i].vy -= fy;
nodes[j].vx += fx; nodes[j].vy += fy;
}
}
for (const edge of edges) {
const dx = edge.targetNode.x - edge.sourceNode.x;
const dy = edge.targetNode.y - edge.sourceNode.y;
const dist = Math.sqrt(dx * dx + dy * dy) || 1;
const normalizedWeight = edge.closeness_weight / 10;
const idealLength = baseSpringLength * (1.5 - normalizedWeight);
const displacement = dist - idealLength;
const force = springStrength * displacement;
const fx = (dx / dist) * force;
const fy = (dy / dist) * force;
edge.sourceNode.vx += fx; edge.sourceNode.vy += fy;
edge.targetNode.vx -= fx; edge.targetNode.vy -= fy;
}
for (const node of nodes) {
node.vx += (centerX - node.x) * centerPull;
node.vy += (centerY - node.y) * centerPull;
}
for (const node of nodes) {
if (node === dragNode) continue;
node.x += node.vx * damping;
node.y += node.vy * damping;
node.x = Math.max(30, Math.min(width - 30, node.x));
node.y = Math.max(30, Math.min(height - 30, node.y));
}
}
function getEdgeColor(weight) {
const normalized = weight / 10;
return 'hsl(220, 70%, ' + (80 - normalized * 40) + '%)';
}
function draw() {
ctx.clearRect(0, 0, width, height);
for (const edge of edges) {
const lineWidth = 1 + (edge.closeness_weight / 10) * 3;
ctx.strokeStyle = getEdgeColor(edge.closeness_weight);
ctx.lineWidth = lineWidth;
ctx.beginPath();
ctx.moveTo(edge.sourceNode.x, edge.sourceNode.y);
ctx.lineTo(edge.targetNode.x, edge.targetNode.y);
ctx.stroke();
const midX = (edge.sourceNode.x + edge.targetNode.x) / 2;
const midY = (edge.sourceNode.y + edge.targetNode.y) / 2;
ctx.fillStyle = '#666';
ctx.font = '10px sans-serif';
ctx.textAlign = 'center';
const label = RELATIONSHIP_DISPLAY[edge.relationship_type] || edge.relationship_type;
ctx.fillText(label, midX, midY - 5);
}
for (const node of nodes) {
const isSelected = node === selectedNode;
const radius = isSelected ? 25 : 20;
ctx.beginPath();
ctx.arc(node.x, node.y, radius, 0, Math.PI * 2);
ctx.fillStyle = isSelected ? '#0066cc' : '#fff';
ctx.fill();
ctx.strokeStyle = '#0066cc';
ctx.lineWidth = 2;
ctx.stroke();
ctx.fillStyle = isSelected ? '#fff' : '#333';
ctx.font = '12px sans-serif';
ctx.textAlign = 'center';
ctx.textBaseline = 'middle';
const name = node.name.length > 10 ? node.name.slice(0, 9) + '\u2026' : node.name;
ctx.fillText(name, node.x, node.y);
}
}
function animate() {
simulate();
draw();
requestAnimationFrame(animate);
}
animate();
function getNodeAt(x, y) {
for (const node of nodes) {
const dx = x - node.x;
const dy = y - node.y;
if (dx * dx + dy * dy < 400) return node;
}
return null;
}
canvas.addEventListener('mousedown', function(event) {
const rect = canvas.getBoundingClientRect();
const node = getNodeAt(event.clientX - rect.left, event.clientY - rect.top);
if (node) {
dragNode = node;
selectedNode = node;
const infoDiv = document.getElementById('selected-info');
let html = '<div class="selected-info"><h3>' + node.name + '</h3>';
if (node.current_job) html += '<p>Job: ' + node.current_job + '</p>';
html += '<a href="/contacts/' + node.id + '">View details</a></div>';
infoDiv.innerHTML = html;
}
});
canvas.addEventListener('mousemove', function(event) {
if (!dragNode) return;
const rect = canvas.getBoundingClientRect();
dragNode.x = event.clientX - rect.left;
dragNode.y = event.clientY - rect.top;
});
canvas.addEventListener('mouseup', function() { dragNode = null; });
canvas.addEventListener('mouseleave', function() { dragNode = null; });
})();
</script>
{% endblock %}
-31
View File
@@ -1,31 +0,0 @@
{% extends "base.html" %}
{% block title %}Needs{% endblock %}
{% block content %}
<div class="need-list">
<div class="header">
<h1>Needs / Accommodations</h1>
<button class="btn btn-primary" onclick="document.getElementById('need-form').toggleAttribute('hidden')">Add Need</button>
</div>
<form id="need-form" hidden
hx-post="/htmx/needs"
hx-target="#need-items"
hx-swap="innerHTML"
hx-on::after-request="if(event.detail.successful) this.reset()"
class="need-form">
<div class="form-group">
<label for="name">Name *</label>
<input id="name" name="name" type="text" placeholder="e.g., Light Sensitive, ADHD" required>
</div>
<div class="form-group">
<label for="description">Description</label>
<textarea id="description" name="description" placeholder="Optional description..." rows="2"></textarea>
</div>
<button type="submit" class="btn btn-primary">Create</button>
</form>
<div id="need-items">
{% include "partials/need_items.html" %}
</div>
</div>
{% endblock %}
@@ -1,33 +0,0 @@
{% if contacts %}
<table>
<thead>
<tr>
<th>Name</th>
<th>Job</th>
<th>Timezone</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for contact in contacts %}
<tr id="contact-row-{{ contact.id }}">
<td><a href="/contacts/{{ contact.id }}">{{ contact.name }}</a></td>
<td>{{ contact.current_job or "-" }}</td>
<td>{{ contact.timezone or "-" }}</td>
<td>
<a href="/contacts/{{ contact.id }}/edit" class="btn">Edit</a>
<button class="btn btn-danger"
hx-delete="/api/contacts/{{ contact.id }}"
hx-target="#contact-row-{{ contact.id }}"
hx-swap="outerHTML"
hx-confirm="Delete this contact?">
Delete
</button>
</td>
</tr>
{% endfor %}
</tbody>
</table>
{% else %}
<p>No contacts yet.</p>
{% endif %}
@@ -1,14 +0,0 @@
<ul class="manage-needs-list">
{% for need in contact.needs %}
<li id="contact-need-{{ need.id }}">
<strong>{{ need.name }}</strong>
{% if need.description %}<span> - {{ need.description }}</span>{% endif %}
<button class="btn btn-small btn-danger"
hx-delete="/api/contacts/{{ contact.id }}/needs/{{ need.id }}"
hx-target="#contact-need-{{ need.id }}"
hx-swap="outerHTML">
Remove
</button>
</li>
{% endfor %}
</ul>
@@ -1,23 +0,0 @@
{% for rel in contact.related_to %}
<div class="manage-rel-item" id="rel-{{ contact.id }}-{{ rel.related_contact_id }}">
<a href="/contacts/{{ rel.related_contact_id }}">{{ contact_names[rel.related_contact_id] }}</a>
<span class="tag">{{ rel.relationship_type|replace("_", " ")|title }}</span>
<label class="weight-control">
<span>Closeness:</span>
<input type="range" min="1" max="10" value="{{ rel.closeness_weight }}"
hx-post="/htmx/contacts/{{ contact.id }}/relationships/{{ rel.related_contact_id }}/weight"
hx-trigger="change"
hx-include="this"
name="closeness_weight"
hx-swap="none"
oninput="this.nextElementSibling.textContent = this.value">
<span class="weight-value">{{ rel.closeness_weight }}</span>
</label>
<button class="btn btn-small btn-danger"
hx-delete="/api/contacts/{{ contact.id }}/relationships/{{ rel.related_contact_id }}"
hx-target="#rel-{{ contact.id }}-{{ rel.related_contact_id }}"
hx-swap="outerHTML">
Remove
</button>
</div>
{% endfor %}
@@ -1,21 +0,0 @@
{% if needs %}
<ul class="need-items">
{% for need in needs %}
<li id="need-item-{{ need.id }}">
<div class="need-info">
<strong>{{ need.name }}</strong>
{% if need.description %}<p>{{ need.description }}</p>{% endif %}
</div>
<button class="btn btn-danger"
hx-delete="/api/needs/{{ need.id }}"
hx-target="#need-item-{{ need.id }}"
hx-swap="outerHTML"
hx-confirm="Delete this need?">
Delete
</button>
</li>
{% endfor %}
</ul>
{% else %}
<p>No needs defined yet.</p>
{% endif %}
+1
View File
@@ -92,6 +92,7 @@ class EbookSearchConfig(BaseSettings):
phrase_max_tokens: int = 5
phrase_max_entity_tokens: int = 8
phrase_yake_top_k: int = 1000
phrase_yake_dedup_limit: float = 0.85
phrase_raw_ngram_min_count: int = 2
phrase_raw_count_score_threshold: int = 3
phrase_raw_count_high_score_threshold: int = 10
+2 -1
View File
@@ -159,13 +159,14 @@ async def ingest_file(session: AsyncSession, path: Path | AsyncPath, config: Ebo
await session.flush()
chunk_index = add_chapter_chunks(session, source, chapter, parsed_chapter, chunk_index, config)
await session.commit()
mention_count = await index_chunk_phrase_mentions_for_book(session, source.id, config)
await session.commit()
logger.info(
f"ebook_ingest_file_complete {source.id=} {resolved_path=} chapters={len(parsed.chapters)} {chunk_index=} "
f"{mention_count=}"
)
except Exception:
await session.rollback()
logger.exception(f"ebook_ingest_file_error {path=}")
return False
else:
+5 -5
View File
@@ -174,6 +174,8 @@ async def request_chat_completion(
client: httpx.AsyncClient,
config: EbookSearchConfig,
messages: Sequence[dict[str, str]],
*,
response_format: dict[str, object] | None = None,
) -> str:
"""Request a chat completion over a shared async client.
@@ -181,6 +183,7 @@ async def request_chat_completion(
client (httpx.AsyncClient): Shared async client whose connection pool bounds concurrency.
config (EbookSearchConfig): Runtime settings supplying the endpoint, model, and auth.
messages (Sequence[dict[str, str]]): OpenAI-style chat messages.
response_format (dict[str, object] | None): Optional OpenAI-compatible structured output constraint.
Returns:
str: The assistant message text.
@@ -192,11 +195,8 @@ async def request_chat_completion(
response = await client.post(
f"{config.vllm_base_url.rstrip('/')}/chat/completions",
headers=auth_headers(config.vllm_api_key),
json={
"model": config.chat_model,
"messages": list(messages),
"temperature": 0,
},
json={"model": config.chat_model, "messages": list(messages), "temperature": 0}
| ({"response_format": response_format} if response_format is not None else {}),
timeout=config.chat_timeout_seconds,
)
response.raise_for_status()
@@ -154,7 +154,7 @@ def extract_raw_ngrams_by_chapter(
@lru_cache(maxsize=2)
def get_yake_extractor(max_ngram: int, top_k: int) -> KeywordExtractor:
def get_yake_extractor(max_ngram: int, top_k: int, dedup_limit: float) -> KeywordExtractor:
"""Return a cached YAKE extractor for the given settings.
Constructing a ``KeywordExtractor`` loads the language's stopword list from disk, so it is
@@ -163,11 +163,12 @@ def get_yake_extractor(max_ngram: int, top_k: int) -> KeywordExtractor:
Args:
max_ngram (int): Maximum n-gram size to extract.
top_k (int): Maximum number of keyphrases to request.
dedup_limit (float): Deduplication similarity threshold.
Returns:
KeywordExtractor: A shared extractor instance for the given settings.
"""
return KeywordExtractor(lan="en", n=max_ngram, dedupLim=0.85, top=top_k)
return KeywordExtractor(lan="en", n=max_ngram, dedupLim=dedup_limit, top=top_k)
def extract_yake_candidates(
@@ -183,7 +184,11 @@ def extract_yake_candidates(
Returns:
dict[str, PhraseCandidate]: Candidates keyed by normalized phrase, with YAKE scores.
"""
extractor = get_yake_extractor(config.phrase_max_tokens, config.phrase_yake_top_k)
extractor = get_yake_extractor(
config.phrase_max_tokens,
config.phrase_yake_top_k,
config.phrase_yake_dedup_limit,
)
out: dict[str, PhraseCandidate] = {}
for phrase_text, yake_score in extractor.extract_keywords(book_text):
normalized = normalize_candidate_phrase(phrase_text, config)
@@ -5,7 +5,7 @@ from __future__ import annotations
import asyncio
import json
import logging
import re
from dataclasses import replace
from time import perf_counter
from typing import TYPE_CHECKING
@@ -45,9 +45,6 @@ if TYPE_CHECKING:
from python.ebook_search.protected_phrases.models import PhraseCandidate
from python.orm.richie import EbookProtectedPhrase
JSON_OBJECT_RE = re.compile(r"\{.*\}", re.DOTALL)
logger = logging.getLogger(__name__)
@@ -262,7 +259,9 @@ async def judge_candidate_async(
Returns:
LLMJudgment: The parsed judgment.
"""
content = await request_chat_completion(client, config, build_judge_messages(candidate))
content = await request_chat_completion(
client, config, build_judge_messages(candidate), response_format={"type": "json_object"}
)
return parse_llm_judgment(content, config)
@@ -287,12 +286,23 @@ async def persist_book_judgments(
book_started_at = perf_counter()
async with AsyncSession(engine, expire_on_commit=False) as session:
try:
normalized_book_text = normalize_text(await load_book_text(session, source_id))
protected: list[EbookProtectedPhrase] = []
for candidate_id, candidate, judgment, promote in judged:
candidate_row = await save_candidate_to_db(session, source_id, None, candidate, judgment=judgment)
filtered_judgment = replace(
judgment,
aliases=tuple(
alias for alias in judgment.aliases if alias_occurs_in_book(alias, normalized_book_text)
),
)
candidate_row = await save_candidate_to_db(
session, source_id, None, candidate, judgment=filtered_judgment
)
if promote:
protected.append(
await upsert_protected_phrase(session, source_id, None, candidate, judgment, candidate_row)
await upsert_protected_phrase(
session, source_id, None, candidate, filtered_judgment, candidate_row
)
)
logger.info(
f"ebook_candidate_phrase_judgment_candidate_complete {source_id=} {candidate_id=} "
@@ -395,14 +405,14 @@ def parse_llm_judgment(content: str, config: EbookSearchConfig) -> LLMJudgment:
if not isinstance(aliases, list | tuple):
aliases = ()
return LLMJudgment(
keep=bool(body.get("keep", False)),
keep=strict_bool(body.get("keep"), default=False),
canonical=optional_text(body.get("canonical")),
category=optional_text(body.get("category")),
aliases=tuple(str(alias) for alias in aliases if isinstance(alias, str) and alias.strip()),
confidence=clamped_float(body.get("confidence"), default=0.0),
importance=clamped_float(body.get("importance"), default=0.5),
allow_nested=bool(body.get("allow_nested", config.phrase_default_allow_nested)),
suppress_children=bool(body.get("suppress_children", config.phrase_default_suppress_children)),
allow_nested=strict_bool(body.get("allow_nested"), default=config.phrase_default_allow_nested),
suppress_children=strict_bool(body.get("suppress_children"), default=config.phrase_default_suppress_children),
reason=optional_text(body.get("reason")),
)
@@ -419,14 +429,13 @@ def extract_json_object(content: str) -> str:
Raises:
ValueError: If no JSON object is found in the response.
"""
stripped = content.strip()
if stripped.startswith("{") and stripped.endswith("}"):
return stripped
match = JSON_OBJECT_RE.search(stripped)
if match is None:
msg = "LLM phrase judge response did not contain a JSON object"
raise ValueError(msg)
return match.group(0)
return content.strip()
def alias_occurs_in_book(alias: str, normalized_book_text: str) -> bool:
"""Return whether a normalized alias occurs as a complete phrase in the source book."""
alias_norm = normalize_text(alias)
return bool(alias_norm) and f" {alias_norm} " in f" {normalized_book_text} "
def optional_text(value: object) -> str | None:
@@ -444,6 +453,11 @@ def optional_text(value: object) -> str | None:
return stripped or None
def strict_bool(value: object, *, default: bool) -> bool:
"""Return a JSON boolean, falling back when the value has another type."""
return value if isinstance(value, bool) else default
def clamped_float(value: object, *, default: float) -> float:
"""Coerce a JSON number into the 0.0 to 1.0 range.
-43
View File
@@ -1,43 +0,0 @@
{
pkgs,
inputs,
...
}:
{
networking.firewall.allowedTCPPorts = [
8069
];
systemd.services.contact-api = {
description = "Contact Database API";
after = [
"postgresql.service"
"network.target"
];
requires = [ "postgresql.service" ];
wantedBy = [ "multi-user.target" ];
environment = {
PYTHONPATH = "${inputs.self}";
POSTGRES_DB = "richie";
POSTGRES_HOST = "/run/postgresql";
POSTGRES_USER = "richie";
POSTGRES_PORT = "5432";
};
serviceConfig = {
Type = "simple";
ExecStart = "${pkgs.my_python}/bin/python -m python.api.main --host 192.168.90.40 --port 8069";
Restart = "on-failure";
RestartSec = "5s";
StandardOutput = "journal";
StandardError = "journal";
NoNewPrivileges = true;
ProtectSystem = "strict";
ProtectHome = "read-only";
PrivateTmp = true;
ReadOnlyPaths = [
"${inputs.self}"
];
};
};
}