Richie
1d1bafbd30
feat(haproxy-logs): ingest HAProxy request logs into Richie DB
...
Add a pipeline to load HAProxy `option httplog` lines into the Richie
database so bot/crawler traffic can be analyzed.
- model: HaproxyRequest mirroring the httplog format, with a unique
line_hash dedup key and indexes on common filter columns
- migration: create the haproxy_request table (unique line_hash + indexes)
- haproxy_logs package:
- parser: httplog line -> columns, strips the journald prefix and
hashes the normalized line
- ingest: batched, idempotent insert that skips rows whose line_hash
already exists, so re-ingesting the same logs is a no-op
- cli: ingest-only `haproxy-logs` command reading stdin or a file
- tests: parsing of a real GPTBot line and idempotent re-ingestion
2026-06-23 21:13:20 -04:00
Richie
121eb979a4
added a index for the VEctor DB
2026-06-14 15:41:09 -04:00
Richie
cdded5da12
added ebook embedding to orm
2026-06-14 15:40:04 -04:00
Richie
9290cb46ee
updated series_index to float and added UniqueConstraint to audiobook and audiobook_author
treefmt / nix fmt (push) Successful in 5s
build_systems / build-bob (push) Successful in 32s
build_systems / build-leviathan (push) Successful in 41s
build_systems / build-rhapsody-in-green (push) Successful in 44s
pytest / pytest (push) Successful in 27s
build_systems / build-brain (push) Successful in 31s
build_systems / build-jeeves (push) Successful in 2m28s
pytest / pytest (pull_request) Successful in 26s
build_systems / build-bob (pull_request) Successful in 47s
treefmt / nix fmt (pull_request) Successful in 5s
build_systems / build-brain (pull_request) Successful in 44s
build_systems / build-leviathan (pull_request) Successful in 52s
build_systems / build-rhapsody-in-green (pull_request) Successful in 58s
build_systems / build-jeeves (pull_request) Successful in 2m27s
2026-06-13 22:29:56 -04:00
Richie
aff6f4e1bd
adding audiobook data to DB
2026-06-13 22:29:56 -04:00
Richie
5d3a851137
deleting data_science code
...
build_systems / build-bob (pull_request) Successful in 46s
build_systems / build-leviathan (pull_request) Successful in 53s
build_systems / build-rhapsody-in-green (pull_request) Successful in 57s
build_systems / build-jeeves (pull_request) Successful in 2m33s
build_systems / build-brain (pull_request) Successful in 46s
treefmt / nix fmt (push) Successful in 4s
build_systems / build-brain (push) Successful in 29s
pytest / pytest (push) Successful in 23s
build_systems / build-bob (push) Successful in 31s
build_systems / build-leviathan (push) Successful in 38s
build_systems / build-rhapsody-in-green (push) Successful in 43s
build_systems / build-jeeves (push) Successful in 2m29s
treefmt / nix fmt (pull_request) Successful in 6s
pytest / pytest (pull_request) Successful in 23s
this code was moved to https://gitea.tmmworkshop.com/Nornsight/weave
2026-06-13 21:14:42 -04:00
Richie
e05e5c77bc
deleting signal bot
2026-06-13 21:09:34 -04:00
Richie
261ff139f7
removed ds table from richie DB
2026-03-29 15:54:54 -04:00
Richie
e368402eea
adding LegislatorSocialMedia
2026-03-29 15:54:54 -04:00
Richie
c5babf8bad
ran treefmt
2026-03-29 15:54:54 -04:00
Richie
ca62cc36a7
adding congress data to new DS DB
2026-03-29 15:54:54 -04:00
Richie
e40ab757ca
making more generic exception handling
2026-03-29 15:54:54 -04:00
Richie
f2084206b6
adding tables for 2023
2026-03-29 15:54:54 -04:00
Richie
50e764146a
added ingest_posts.py
2026-03-29 15:54:54 -04:00
Richie
ea97b5eb19
adding 2026 partitions
2026-03-29 15:54:54 -04:00
Richie
1ef2512daa
adding post table
2026-03-29 15:54:54 -04:00
Richie
42330ec186
changed to sa.Enum
2026-03-18 19:29:59 -04:00
Richie
3f4373d1f6
fixed tests and treeftm
2026-03-18 19:29:59 -04:00
Richie
976c3f9d3e
move signal bot to its own DB
2026-03-18 19:29:59 -04:00
Richie
7d2fbaea43
added bot class and rbac style auth with dynamic help msg base on roles
2026-03-18 19:29:59 -04:00
Richie
f1e394565d
migrated to tanasty and added dead letter queue
2026-03-14 11:49:44 -04:00
Richie
a6fbbd245f
fixed safety number logic
2026-03-14 11:49:44 -04:00
Richie
7ad321e5e2
moved device registry to postgresql
2026-03-14 11:49:44 -04:00
Richie
3dadb145b7
added congress data to database
2026-03-14 11:49:44 -04:00
Richie
b7bce0bcb9
created alembic revision for van_inventory
2026-03-09 07:18:01 -04:00
Richie
ec80bf1c5f
added commit to env.py
2026-03-09 07:18:01 -04:00
Richie
18f149b831
ran treefmt
2026-03-09 07:18:01 -04:00
Richie
69f5b87e5f
setup multy db suport
2026-03-09 07:18:01 -04:00
Richie
8afa4fce6c
added Contact api and data model
2026-01-22 21:26:38 -05:00
Richie
0109167b10
base of sqlalchemy alembic
2026-01-11 11:41:19 -05:00
Richie