100 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Richie	4c71508e73	feat(ebook): enhance phrase judgment logging with failure tracking	2026-07-24 11:38:50 -04:00
Richie	70bf8627a2	feat(ebook): implement phrase matching functionality and UI enhancements	2026-07-24 11:38:50 -04:00
Richie	a12e7461c5	feat(ebook): add admin and book-detail UI for protected phrase pipeline ... Expose the protected phrase extraction pipeline through the web UI: - Admin routes: POST /admin/build-phrases, /admin/generate-ngrams, and /admin/judge-ngrams, each wrapping the protected_phrases.lib backfill helpers, committing on success, rolling back and rendering an error partial on failure, and reporting per-book/candidate/mention counts. - Book detail page: show candidate, judged, and protected phrase counts, list top candidate n-grams (with kept/rejected status) and protected phrases, and add a POST /books/{id}/recalculate-phrases action that clears and regenerates candidates, then redirects back with a status message. - Admin template: add Generate/Judge n-gram buttons. Also reflows admin.html to 2-space HTML formatting.	2026-07-24 11:38:50 -04:00
Richie	6c8a4bfea7	feat(ebook): add protected phrase extraction library with config-driven tuning ... Refactor protected phrase handling from a single module into a python/ebook_search/protected_phrases package covering extraction, storage, and runtime matching. Phrase filtering is now data-driven via bundled TOML files: ignored_phrases, bad_starts, bad_ends, and most_common_words. Add phrase-tuning settings to EbookSearchConfig so candidate generation, scoring, LLM judging, and matching are configurable rather than hardcoded: token bounds, entity token limit, raw n-gram min count, frequency and chapter-spread score thresholds, candidate/LLM/target caps, confidence threshold, nesting defaults, and the phrase hit boost.	2026-07-24 11:38:50 -04:00
Richie	fbb1ebfd56	fix(ebook): enhance EPUB ingestion with error handling and incrmental commits	2026-07-24 11:38:50 -04:00
Richie	9ba8200673	feat(ebook): add phrase metadata tables for protected phrase matching ... Introduce four ORM models and their Alembic migration to support phrase-based query matching in the ebook RAG engine: - EbookCandidatePhrase: high-recall phrase candidates extracted per book, with source flags (ngram/yake/spacy/capitalized/metadata), scoring, and LLM judge results. - EbookProtectedPhrase: phrases accepted by the LLM judge, with canonical id, importance, and nesting controls. - EbookPhraseAlias: normalized aliases mapping to protected phrases. - EbookChunkPhraseMention: precomputed phrase occurrences within chunks. Export the new models from python.orm.richie and add a JSON_DOCUMENT helper (JSON with JSONB postgres variant) for storing sample contexts.	2026-07-24 11:38:50 -04:00
Richie	add7a6a848	feat(gems): add multiplayer gem game with custom content packs ... - implement FastAPI and HTMX lobby and game interfaces - support up to four human and AI-controlled players - add configurable rules, victory conditions, and expansion modules - support validated JSON cards, patrons, objectives, and outposts	2026-07-22 20:47:03 -04:00
Richie	247e951a27	Remove legacy HTML templates and API service configuration for contact management system ... - Deleted base template (base.html) and all related contact templates (contact_detail.html, contact_form.html, contact_list.html, graph.html, need_list.html). - Removed partial templates for managing contacts and needs (contact_table.html, manage_needs.html, manage_relationships.html, need_items.html). - Eliminated contact API service configuration (contact_api.nix) from the NixOS setup.	2026-07-15 15:28:15 -04:00
Richie	8c3de690c9	feat(ebook-search): set up uv cache management and pruning service	2026-07-12 17:26:55 -04:00
Richie	91575e3ab2	feat(ebook-search): update test workflow to use nix for dependency management	2026-07-12 17:10:47 -04:00
Richie	4799054601	ci(ebook-search): run tests via uv on the runner instead of docker ... The gitea runner containers have no docker access, so build the test env with uv from the existing lockfile and run pytest directly: - test_ebook_search workflow: uv sync --locked + uv run pytest, with UV_PYTHON_DOWNLOADS=never so uv uses the nix-provided python 3.14 - add uv to the runner hostPackages (needs a jeeves rebuild to apply) - ignore nested **/.venv in .dockerignore (uv sync creates one in python/ebook_search/docker) - document the uv test commands in the docker README; the docker test image remains for validating the image itself	2026-07-12 16:40:56 -04:00
Richie	e9636d19de	feat(ebook-search): add containerized test environment	2026-07-12 16:12:40 -04:00
Richie	2518ec8551	feat(python-env): remove unused dependencies and clean up package list	2026-07-12 16:12:40 -04:00
Richie	f5ec88f1e5	feat(ebook): add Docker packaging and lifecycle tooling ... Add a self-contained docker/ package for running the ebook search app against the existing Postgres database on jeeves: - Dockerfile: python:3.14-slim image, non-root user, runs the FastAPI app on port 8070 - docker-compose.yml: service definition with library volume mount, BM25 index volume, .env loading, and a /health healthcheck - containers.py: Typer CLI (ebook-search-containers) for build/start/ stop/restart/logs/ps lifecycle management - README.md: usage and configuration docs	2026-07-12 16:12:40 -04:00
Richie	f4c4b11ff8	updated dependencies and added .dockerignore	2026-07-12 16:12:40 -04:00
Richie	37f37c41ac	feat(common): add get_repo_dir function and corresponding tests	2026-07-12 14:24:05 -04:00
Richie	80a521f297	refactor: extract signal_alert into its own module ... Move signal_alert out of python/common.py into a dedicated python/signal_alert.py module and update its importers (validate_system.py, snapshot_manager.py) to the new path. Relocate the signal_alert tests from tests/test_common.py into tests/test_signal_alert.py, repatching python.signal_alert.logger and python.signal_alert.Apprise to match the new module.	2026-07-12 13:16:28 -04:00
Richie	4a410dbdf8	feat(postgres): add trust authentication for richie on 172.16.0.0/12	2026-07-12 13:06:44 -04:00
Richie	138b79bf97	feat(flake): update lock file	2026-07-12 12:49:57 -04:00
Richie	47f77443dd	feat(networking): configure unused network ports to prevent errors and optimize boot time	2026-07-09 10:45:08 -04:00
Richie	8548f4ff64	feat(programs): add codebase-memory-mcp to home packages	2026-07-02 16:08:12 -04:00
Richie	682960b4c9	feat(installer): refactor NixOS configurations to dynamically generate host systems	2026-07-02 14:33:45 -04:00
Richie	ddea13b93a	feat(installer): enhance NixOS installer with SSH access and encryption password input	2026-07-02 14:17:49 -04:00
Richie	b252b0bf6e	feat(installer): add one-file installer build and custom install ISO ... Replace old_installer.py with a curses TUI installer packaged as a one-file PyInstaller binary (python/installer/build.py, wrapped by python/installer/package.nix). The default .#installer is patched to run on foreign Linux live media; the new .#installer-nixos variant keeps its Nix store interpreter so it runs on NixOS. Add systems/iso, a minimal NixOS install CD with kernel 6.18 and ZFS 2.4 matching the deployed systems and the installer on PATH; build it with nix build .#iso. Shared logging and subprocess helpers move out of common.py into python/logging_config.py and python/process.py.	2026-07-02 14:17:49 -04:00
Richie	fbf288649a	feat(haproxy): simplify Gitea rate limiting and fix rule ordering ... - Drop the session-cookie rate-limit bypass: Gitea hands the cookie to anonymous visitors too, so any cookie-accepting crawler could exempt itself after one request - Loosen the general Gitea limit from 10 to 50 req/10s since logged-in users are now rate-limited as well - Serve robots.txt above the rate-limit rules so throttled crawlers can always read the Crawl-delay, and capture Host/User-Agent before the deny so 429s are logged with them - Match the compare/diff cap on every repo case-insensitively instead of only /Richie/dotfiles - Remove the Jellyfin health check; it never ran (server had no check flag) and is not wanted	2026-07-01 16:54:07 -04:00
Richie	db199f884d	feat(comms): add Slack to communication packages	2026-07-01 12:14:23 -04:00
Richie	c6645eb37c	feat(home-assistant): remove status indicator configuration and related YAML file	2026-07-01 00:21:05 -04:00
Richie	3984081753	feat(cleanup): remove leviathan and elise configurations and references	2026-06-29 16:24:21 -04:00
Richie	03f610feed	feat(ui): refactor HTML templates for consistent structure and improved styling	2026-06-29 10:23:13 -04:00
Richie	8804a8df83	added rate limits	2026-06-27 23:01:03 -04:00
Richie	8daac4d98b	feat(grafana): add HAProxy requests dashboard and Richie Postgres datasource ... Visualize the ingested HAProxy request logs in Grafana. - add a `richie-postgres` datasource connecting to the Richie DB over the local Unix socket (/run/postgresql; pg_hba trust for user richie) - order grafana after postgresql.service - add the "HAProxy Requests" dashboard: request rate by backend, response -time percentiles (p50-p99), top user-agents/IPs/endpoints, status mix	2026-06-27 23:01:03 -04:00
Richie	7606ddc514	feat(haproxy-logs): ingest HAProxy request logs into Richie DB ... Add a pipeline to load HAProxy `option httplog` lines into the Richie database so bot/crawler traffic can be analyzed. - model: HaproxyRequest mirroring the httplog format, with a unique line_hash dedup key and indexes on common filter columns - migration: create the haproxy_request table (unique line_hash + indexes) - haproxy_logs package: - parser: httplog line -> columns, strips the journald prefix and hashes the normalized line - ingest: batched, idempotent insert that skips rows whose line_hash already exists, so re-ingesting the same logs is a no-op - cli: ingest-only `haproxy-logs` command reading stdin or a file - tests: parsing of a real GPTBot line and idempotent re-ingestion	2026-06-27 23:01:03 -04:00
Richie	137bd8ed2f	adding robots.txt	2026-06-27 23:01:03 -04:00
Richie	f936253ad5	flake update	2026-06-22 11:31:47 -04:00
Richie	fc01fb9d0b	Merge branch 'main' into feature/book-search-engine	2026-06-21 15:04:43 -04:00
Richie	09d963ba34	fix(ebook-search): skip comment lines in gold query loader and realign tests ... load_gold_queries now skips blank and `//` comment lines so the committed section separator in queries.jsonl no longer breaks dataset/load-test loading. Update tests left stale by the search refactor (6bc3011): - pass the now-required rank_constant to reciprocal_rank_fusion - expect bm25_candidates to receive the full query and drop the removed "BM25 query preparation" timing step - assert reranking is enabled by default	2026-06-21 14:45:31 -04:00
Richie	855af0fdff	updating 10-Primary to enp97s0f1	2026-06-21 14:25:02 -04:00
Richie	fcb69cc68b	adding words to spell check	2026-06-18 12:55:46 -04:00
Richie	6bc30115d9	fix(ebook-search): code clean up to impove reliablty and readabilty	2026-06-18 12:45:56 -04:00
Richie	6ae1ff1f5c	feat(ebook-search): add load-test CLI for the search service ... Add a Typer CLI script that drives POST /search on a running server at a configurable concurrency and reports latency percentiles (p50/p90/p95/p99), throughput, and HTTP status distribution. Queries are drawn from the shared eval JSONL set so load testing and evaluation exercise the same questions.	2026-06-18 12:39:55 -04:00
Richie	dbc6b5b53b	test(ebook-search): organize tests under dedicated package ... Move ebook search tests into tests/ebook_search and standardize mocking on pytest-mock.	2026-06-16 21:47:40 -04:00
Richie	a9daa60c17	moved TC001 expetop out of pyproject.toml	2026-06-16 18:31:40 -04:00
Richie	241a42b20d	updated front end	2026-06-16 00:00:13 -04:00
Richie	4640ebf8ce	converted addmin.py and page.py to DbSession and AppConfig	2026-06-15 22:01:42 -04:00
Richie	0c9583c1cc	moved config = load_config() to lifespan	2026-06-15 21:59:51 -04:00
Richie	f71ae7d2c6	added guardrails.py to constrain responses and added validation to config.py	2026-06-15 21:57:38 -04:00
Richie	2e68c83021	added health endpoints	2026-06-15 21:21:01 -04:00
Richie	b126987b63	added AppConfig and AppEngine	2026-06-15 21:08:55 -04:00
Richie	68b3a38b81	converting to pydantic-settings	2026-06-14 21:29:45 -04:00
Richie	a5d7c3be4f	fixed fomat issue	2026-06-14 15:42:05 -04:00
Richie	2995a75748	fixed test	2026-06-14 15:41:09 -04:00
Richie	dce1838163	opning ports for testing	2026-06-14 15:41:09 -04:00
Richie	121eb979a4	added a index for the VEctor DB	2026-06-14 15:41:09 -04:00
Richie	c88315e9b6	improved BM25 write	2026-06-14 15:41:09 -04:00
Richie	50795ab7fc	added ZstdMiddleware to ebook_search	2026-06-14 15:41:09 -04:00
Richie	e9a80a0308	added vector_engine to fix name postgres name space issue	2026-06-14 15:41:09 -04:00
Richie	eb76edb740	reworked ebook_search routers	2026-06-14 15:41:09 -04:00
Richie	c53afb3c70	made fastapi tools	2026-06-14 15:41:09 -04:00
Richie	8301db39e5	added proper cache invalidation to load_bm25_corpus	2026-06-14 15:40:31 -04:00
Richie	45a9e90524	updated tests	2026-06-14 15:40:31 -04:00
Richie	dd67de3993	improved reranking weights	2026-06-14 15:40:31 -04:00
Richie	6e3635ca01	fixed duplicat enrichment	2026-06-14 15:40:31 -04:00
Richie	11cbe31152	improved queary for vector search	2026-06-14 15:40:31 -04:00
Richie	5544dfa61c	add .ebook_search_bm25 to gitignore	2026-06-14 15:40:31 -04:00
Richie	911df63513	updated python	2026-06-14 15:40:31 -04:00
Richie	07eb170b34	setup tests	2026-06-14 15:40:04 -04:00
Richie	6715bbf0a5	build api and frountend	2026-06-14 15:40:04 -04:00
Richie	26ff1f0fd3	added answer.py and config	2026-06-14 15:40:04 -04:00
Richie	666ea97754	added __init__	2026-06-14 15:40:04 -04:00
Richie	e01c687625	made llm_interface.py	2026-06-14 15:40:04 -04:00
Richie	82a367a2b6	added rerank	2026-06-14 15:40:04 -04:00
Richie	ad1834c537	built ingest	2026-06-14 15:40:04 -04:00
Richie	aed1e14d95	built rag search setup	2026-06-14 15:40:04 -04:00
Richie	0a2d4c08cb	set up embedding system	2026-06-14 15:40:04 -04:00
Richie	db98bd3559	built BM25 search foundation	2026-06-14 15:40:04 -04:00
Richie	cdded5da12	added ebook embedding to orm	2026-06-14 15:40:04 -04:00
Richie	d022251a58	removed hedgedoc	2026-06-14 15:40:04 -04:00
Richie	e1ef4de6a3	adding embedding Models to jeeves	2026-06-14 15:40:04 -04:00
Richie	5c230a267c	ran treefmt	2026-06-14 13:53:02 -04:00
Richie	5215d66d40	adding noqa to DbSession	2026-06-14 13:50:49 -04:00
Richie	7ad198416b	added TYPE_CHECKING to contact main.py	2026-06-14 13:50:27 -04:00
Richie	1461c2552a	removed van-inventory from pyproject.toml	2026-06-14 13:48:17 -04:00
Richie	736717c2f8	added TYPE_CHECKING to middleware.py	2026-06-14 13:48:02 -04:00
Richie	ab2521867e	added TYPE_CHECKING to dependencies.py	2026-06-14 13:47:50 -04:00
Richie	8e0ab4190b	added TYPE_CHECKING to heater main.py	2026-06-14 13:46:40 -04:00
Richie	734fd7641e	moved fetch_weather to masked lat lon and	2026-06-14 13:46:03 -04:00
Richie	e898e08c48	added noqa to validate system	2026-06-14 13:41:50 -04:00
Richie	d916ea903c	removed dead code	2026-06-14 13:39:55 -04:00
Richie	d8e916dbe6	fixed type bug in get_snapshots	2026-06-14 13:38:14 -04:00
Richie	48e9f0199d	deleting van_inventory	2026-06-14 13:37:55 -04:00
Richie	a526420c8d	fixed un needed noqa's	2026-06-14 12:58:50 -04:00
Richie	41e3e265af	cleaned up audiobook.py mapped_column	2026-06-14 12:53:54 -04:00
Richie	38a17f6146	cleaned uo python dependencies	2026-06-14 12:49:22 -04:00
Richie	fe48d4c1ad	removing splendor	2026-06-14 12:00:39 -04:00
Richie	9290cb46ee	updated series_index to float and added UniqueConstraint to audiobook and audiobook_author	2026-06-13 22:29:56 -04:00
Richie	acd3f2d3ac	fixed omnibus for audio books	2026-06-13 22:29:56 -04:00
Richie	08e716f66a	deleted frontend dir	2026-06-13 22:29:56 -04:00
Richie	d197731af4	added llm_tool_calling.py	2026-06-13 22:29:56 -04:00
Richie	1ffc48bb02	built workflow	2026-06-13 22:29:56 -04:00
Richie	b6395ef18f	Add catalog.py for manually adding authors and series to the database.	2026-06-13 22:29:56 -04:00