Richie/dotfiles

reducing bot traffic #25

Merged

Richie merged 4 commits from feature/reducing-bot-traffic into main

2026-06-27 23:22:11 -04:00

Author	SHA1	Message	Date
Richie	8804a8df83	added rate limits build_systems / build-brain (pull_request) Successful in 51s Details build_systems / build-leviathan (pull_request) Successful in 58s Details pytest / pytest (pull_request) Successful in 26s Details build_systems / build-rhapsody-in-green (pull_request) Successful in 1m3s Details treefmt / nix fmt (pull_request) Successful in 5s Details build_systems / build-jeeves (pull_request) Successful in 2m32s Details build_systems / build-bob (pull_request) Failing after 13m10s Details treefmt / nix fmt (push) Successful in 4s Details pytest / pytest (push) Failing after 17m2s Details build_systems / build-rhapsody-in-green (push) Failing after 17m2s Details build_systems / build-leviathan (push) Failing after 17m2s Details build_systems / build-jeeves (push) Failing after 17m2s Details build_systems / build-brain (push) Failing after 17m2s Details build_systems / build-bob (push) Failing after 17m2s Details	2026-06-27 23:01:03 -04:00
Richie	8daac4d98b	feat(grafana): add HAProxy requests dashboard and Richie Postgres datasource Visualize the ingested HAProxy request logs in Grafana. - add a `richie-postgres` datasource connecting to the Richie DB over the local Unix socket (/run/postgresql; pg_hba trust for user richie) - order grafana after postgresql.service - add the "HAProxy Requests" dashboard: request rate by backend, response -time percentiles (p50-p99), top user-agents/IPs/endpoints, status mix	2026-06-27 23:01:03 -04:00
Richie	7606ddc514	feat(haproxy-logs): ingest HAProxy request logs into Richie DB Add a pipeline to load HAProxy `option httplog` lines into the Richie database so bot/crawler traffic can be analyzed. - model: HaproxyRequest mirroring the httplog format, with a unique line_hash dedup key and indexes on common filter columns - migration: create the haproxy_request table (unique line_hash + indexes) - haproxy_logs package: - parser: httplog line -> columns, strips the journald prefix and hashes the normalized line - ingest: batched, idempotent insert that skips rows whose line_hash already exists, so re-ingesting the same logs is a no-op - cli: ingest-only `haproxy-logs` command reading stdin or a file - tests: parsing of a real GPTBot line and idempotent re-ingestion	2026-06-27 23:01:03 -04:00
Richie	137bd8ed2f	adding robots.txt	2026-06-27 23:01:03 -04:00