996 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
gitea-actions[bot]	cf9f18cc27	chore: update flake.lock	2026-08-07 20:02:17 -04:00
Richie	c1cee8dbcd	adding hardware design tools	2026-08-05 21:28:39 -04:00
Richie	bd50bd8262	removing cuda workaround	2026-08-03 23:06:44 -04:00
Richie	47e753f5b9	fix(flake.lock): update locked versions and hashes for dependencies	2026-08-03 21:44:06 -04:00
Richie	48a7e3a54c	feat(zfs): enhance command handling with run_zfs and run_zpool functions	2026-07-30 12:49:18 -04:00
Richie	cc166df90f	fix(open_webui): update OLLAMA_API_BASE_URL to use HTTPS	2026-07-27 21:10:48 -04:00
Richie	4384853430	feat(t3_code): add T3 Code AppImage launcher and desktop entry	2026-07-27 21:10:41 -04:00
Richie	ed8b653997	feat(richie user env): adding app_image_path configuration	2026-07-27 20:23:52 -04:00
Richie	aff7398f8d	fix: rename 'models' to 'modelsDir' for consistency in llms.nix configurations	2026-07-27 20:15:35 -04:00
Richie	e259526c38	feat(settings): add configuration to disable YAML schema detection for specific workflow files	2026-07-25 18:44:13 -04:00
Richie	e57895cc6e	patch cuda	2026-07-25 14:21:07 -04:00
Richie	89e24c45a0	chore: remove poetry from Python packages list	2026-07-25 13:59:24 -04:00
Richie	2e7b51ce1d	flake update	2026-07-25 13:47:36 -04:00
Richie	8eee5faf72	feat(chat): add optional response format parameter to request_chat_completion	2026-07-24 11:38:51 -04:00
Richie	c135821534	feat(ingest): ensure session rollback on ingestion error and log mention count	2026-07-24 11:38:51 -04:00
Richie	62bcc4e156	feat(judge_ngrams): enhance phrase judgment with alias filtering and strict boolean handling	2026-07-24 11:38:51 -04:00
Richie	31ecad881f	feat(extraction): add deduplication limit parameter to YAKE extractor	2026-07-24 11:38:51 -04:00
Richie	68c9693711	feat(config): add new scoring parameters for phrase extraction and matching	2026-07-24 11:38:51 -04:00
Richie	6a0e71a30d	feat(ebook-search): implement async file path resolution and EPUB discovery	2026-07-24 11:38:51 -04:00
Richie	8073144e2b	feat(prompts): implement TOML-backed prompt loading and refactor message generation	2026-07-24 11:38:51 -04:00
Richie	94f18722e4	feat(search): enhance phrase matching and reranking logic with improved async handling	2026-07-24 11:38:51 -04:00
Richie	e510c94b95	feat(dependencies): update database engine dependencies to support async operations	2026-07-24 11:38:51 -04:00
Richie	9f126fedc7	feat(ebook-search): implement async phrase matching for chunks and add ChunkPhraseHit model	2026-07-24 11:38:51 -04:00
Richie	e010756e09	refactor(ebook-search): simplify search and phrase matching	2026-07-24 11:38:51 -04:00
Richie	0028237579	perf(ebook-search): run phrase detection in parallel with retrieval ... Move protected phrase detection into the retrieval gather so it runs concurrently with vector and BM25 candidates instead of sequentially before them. Make the search API accept real bool form fields for rerank/phrase_matching, gate phrase matching on both the request and config kill switch, and reflow log f-strings for readability.	2026-07-24 11:38:51 -04:00
Richie	8f1a69529c	Refactor logging statements to use f-strings for improved readability and consistency across the codebase. This change enhances the clarity of log messages by directly embedding variable values, making it easier to trace and debug application behavior.	2026-07-24 11:38:51 -04:00
Richie	5210f00587	feat(dependencies): update sqlalchemy to use asyncio and add aiosqlite and pytest-asyncio	2026-07-24 11:38:51 -04:00
Richie	58be234d7f	fix(protected-phrases): isolate phrase generation per book ... Run full-book candidate generation inside worker-owned sessions so each book commits independently during backfills. Abort recalculation when a book has no indexed chapters to preserve existing phrase data, and update admin/UI tests for the new generation flow.	2026-07-24 11:38:50 -04:00
Richie	29a51eb1b8	feat(admin): simplify phrase generation by removing missing phrases endpoint	2026-07-24 11:38:50 -04:00
Richie	da78914a9f	refactor(ebook): remove spaCy-ner attributes from PhraseCandidate and related functions	2026-07-24 11:38:50 -04:00
Richie	c852f9136a	feat(dependencies): update sqlalchemy to use asyncio and add aiosqlite and pytest-asyncio to dev dependencies	2026-07-24 11:38:50 -04:00
Richie	565119ee45	test(ebook): cover protected phrases and migrate suite to async ... Add test_protected_phrases.py covering phrase-matching behavior in the RAG engine, and update the existing ebook_search tests to use the async SQLAlchemy engine/session (create_async_engine, AsyncSession) and async HTTP paths.	2026-07-24 11:38:50 -04:00
Richie	56c9bb2520	feat(ebook): add additional tokens to junk tokens configuration	2026-07-24 11:38:50 -04:00
Richie	f69c84e7b6	feat(vscode): add new words to spell checker configuration	2026-07-24 11:38:50 -04:00
Richie	5c826088c5	feat(ebook): improve search UX with grid actions and Enter-to-submit ... Add a two-column grid layout for the admin protected-phrases actions and submit the search form on Enter (Shift+Enter for newline).	2026-07-24 11:38:50 -04:00
Richie	2706c4417d	feat(ebook): migrate to async DB/HTTP and parallelize phrase pipeline ... Convert the ebook-search web app to async end to end and add concurrency to the protected-phrase extraction and judging pipeline so large books no longer block the event loop or the UI. ORM / infra: - Add get_async_postgres_engine and factor shared URL/connect_args building into build_postgres_url (reused by the sync and async engine builders) - Add async FastAPI session helpers (get_async_db, AsyncDbSession) with expire_on_commit=False to avoid implicit IO under asyncio App: - Use AsyncEngine/AsyncSession throughout routes, search, ingest, embeddings, answer, rerank and LLM calls; convert handlers to async - Share a single httpx.AsyncClient in app state for LLM requests; size the connection pool for concurrent phrase-judging workers - Add judge_tasks: run per-book judging as tracked background tasks so a book already being judged isn't double-queued Protected phrases: - Add a process pool (pool.py) and worker-count config (extraction/judge book/phrase workers) to parallelize candidate generation and judging - Split admin actions into all/missing variants for generation and judging Config: - Add protected_phrase_extraction_workers, phrase_judge_book_workers, phrase_judge_phrase_workers	2026-07-24 11:38:50 -04:00
Richie	38c01ec121	feat(extraction): add cached YAKE extractor for improved performance	2026-07-24 11:38:50 -04:00
Richie	a9311a2f9e	feat(ebook): add junk tokens for improved phrase matching	2026-07-24 11:38:50 -04:00
Richie	584b209dfa	feat(orm): add pool_size parameter to get_postgres_engine for connection management	2026-07-24 11:38:50 -04:00
Richie	4861f58f27	Add models and database persistence for protected phrase extraction ... - Introduced dataclasses for phrase candidates, judgments, and matches in `models.py`. - Implemented database operations for candidate and protected phrases in `store.py`, including loading, saving, and deleting phrases. - Enhanced text normalization functions in `text_normalization.py` with detailed docstrings. - Refactored search functionality to utilize new models and methods for detecting protected phrases.	2026-07-24 11:38:50 -04:00
Richie	34e7823517	feat(ebook): add phrase matching display and update search result structure	2026-07-24 11:38:50 -04:00
Richie	11b5d5db3c	refactor(protected-phrases): extract config and text normalization helpers	2026-07-24 11:38:50 -04:00
Richie	8e2ca365c2	feat(ebook): update protected phrases with additional tokens and phrases	2026-07-24 11:38:50 -04:00
Richie	4c71508e73	feat(ebook): enhance phrase judgment logging with failure tracking	2026-07-24 11:38:50 -04:00
Richie	70bf8627a2	feat(ebook): implement phrase matching functionality and UI enhancements	2026-07-24 11:38:50 -04:00
Richie	a12e7461c5	feat(ebook): add admin and book-detail UI for protected phrase pipeline ... Expose the protected phrase extraction pipeline through the web UI: - Admin routes: POST /admin/build-phrases, /admin/generate-ngrams, and /admin/judge-ngrams, each wrapping the protected_phrases.lib backfill helpers, committing on success, rolling back and rendering an error partial on failure, and reporting per-book/candidate/mention counts. - Book detail page: show candidate, judged, and protected phrase counts, list top candidate n-grams (with kept/rejected status) and protected phrases, and add a POST /books/{id}/recalculate-phrases action that clears and regenerates candidates, then redirects back with a status message. - Admin template: add Generate/Judge n-gram buttons. Also reflows admin.html to 2-space HTML formatting.	2026-07-24 11:38:50 -04:00
Richie	6c8a4bfea7	feat(ebook): add protected phrase extraction library with config-driven tuning ... Refactor protected phrase handling from a single module into a python/ebook_search/protected_phrases package covering extraction, storage, and runtime matching. Phrase filtering is now data-driven via bundled TOML files: ignored_phrases, bad_starts, bad_ends, and most_common_words. Add phrase-tuning settings to EbookSearchConfig so candidate generation, scoring, LLM judging, and matching are configurable rather than hardcoded: token bounds, entity token limit, raw n-gram min count, frequency and chapter-spread score thresholds, candidate/LLM/target caps, confidence threshold, nesting defaults, and the phrase hit boost.	2026-07-24 11:38:50 -04:00
Richie	fbb1ebfd56	fix(ebook): enhance EPUB ingestion with error handling and incrmental commits	2026-07-24 11:38:50 -04:00
Richie	9ba8200673	feat(ebook): add phrase metadata tables for protected phrase matching ... Introduce four ORM models and their Alembic migration to support phrase-based query matching in the ebook RAG engine: - EbookCandidatePhrase: high-recall phrase candidates extracted per book, with source flags (ngram/yake/spacy/capitalized/metadata), scoring, and LLM judge results. - EbookProtectedPhrase: phrases accepted by the LLM judge, with canonical id, importance, and nesting controls. - EbookPhraseAlias: normalized aliases mapping to protected phrases. - EbookChunkPhraseMention: precomputed phrase occurrences within chunks. Export the new models from python.orm.richie and add a JSON_DOCUMENT helper (JSON with JSONB postgres variant) for storing sample contexts.	2026-07-24 11:38:50 -04:00
Richie	add7a6a848	feat(gems): add multiplayer gem game with custom content packs ... - implement FastAPI and HTMX lobby and game interfaces - support up to four human and AI-controlled players - add configurable rules, victory conditions, and expansion modules - support validated JSON cards, patrons, objectives, and outposts	2026-07-22 20:47:03 -04:00