Registers the tortoise machine in flake.nix and adds its system
configuration with a placeholder hardware config (ZFS root pool,
x86_64) to be replaced by the installer-generated hardware-configuration.nix.
Replace old_installer.py with a curses TUI installer packaged as a
one-file PyInstaller binary (python/installer/build.py, wrapped by
python/installer/package.nix). The default .#installer is patched to run
on foreign Linux live media; the new .#installer-nixos variant keeps its
Nix store interpreter so it runs on NixOS.
Add systems/iso, a minimal NixOS install CD with kernel 6.18 and ZFS 2.4
matching the deployed systems and the installer on PATH; build it with
nix build .#iso. Shared logging and subprocess helpers move out of
common.py into python/logging_config.py and python/process.py.
Introduce four ORM models and their Alembic migration to support
phrase-based query matching in the ebook RAG engine:
- EbookCandidatePhrase: high-recall phrase candidates extracted per book,
with source flags (ngram/yake/spacy/capitalized/metadata), scoring, and
LLM judge results.
- EbookProtectedPhrase: phrases accepted by the LLM judge, with canonical
id, importance, and nesting controls.
- EbookPhraseAlias: normalized aliases mapping to protected phrases.
- EbookChunkPhraseMention: precomputed phrase occurrences within chunks.
Export the new models from python.orm.richie and add a JSON_DOCUMENT
helper (JSON with JSONB postgres variant) for storing sample contexts.
Visualize the ingested HAProxy request logs in Grafana.
- add a `richie-postgres` datasource connecting to the Richie DB over the
local Unix socket (/run/postgresql; pg_hba trust for user richie)
- order grafana after postgresql.service
- add the "HAProxy Requests" dashboard: request rate by backend, response
-time percentiles (p50-p99), top user-agents/IPs/endpoints, status mix
Add a pipeline to load HAProxy `option httplog` lines into the Richie
database so bot/crawler traffic can be analyzed.
- model: HaproxyRequest mirroring the httplog format, with a unique
line_hash dedup key and indexes on common filter columns
- migration: create the haproxy_request table (unique line_hash + indexes)
- haproxy_logs package:
- parser: httplog line -> columns, strips the journald prefix and
hashes the normalized line
- ingest: batched, idempotent insert that skips rows whose line_hash
already exists, so re-ingesting the same logs is a no-op
- cli: ingest-only `haproxy-logs` command reading stdin or a file
- tests: parsing of a real GPTBot line and idempotent re-ingestion
load_gold_queries now skips blank and `//` comment lines so the committed
section separator in queries.jsonl no longer breaks dataset/load-test loading.
Update tests left stale by the search refactor (6bc3011):
- pass the now-required rank_constant to reciprocal_rank_fusion
- expect bm25_candidates to receive the full query and drop the removed
"BM25 query preparation" timing step
- assert reranking is enabled by default
Add a Typer CLI script that drives POST /search on a running server at a
configurable concurrency and reports latency percentiles (p50/p90/p95/p99),
throughput, and HTTP status distribution. Queries are drawn from the shared
eval JSONL set so load testing and evaluation exercise the same questions.
Richie