From 5ba86fdaea0dbdabb110c3d168f070bb077c9bc4 Mon Sep 17 00:00:00 2001 From: Richie Cahill Date: Thu, 1 Oct 2026 16:53:04 -0400 Subject: [PATCH] removed README.md --- common/optional/x86-64-v3/patches/README.md | 38 ----- .../x86-64-v3/patches/abseil/README.md | 59 ------- .../x86-64-v3/patches/gnutls/README.md | 160 ------------------ .../x86-64-v3/patches/prometheus/README.md | 66 -------- .../x86-64-v3/patches/pytest-xdist/README.md | 79 --------- .../x86-64-v3/patches/scipy/README.md | 70 -------- python/ebook_search/docker/README.md | 77 --------- python/gems/README.md | 54 ------ systems/brain/docker/docker_networks.md | 3 - systems/jeeves/monitoring/README.md | 33 ---- systems/portal-1/README.md | 83 --------- 11 files changed, 722 deletions(-) delete mode 100644 common/optional/x86-64-v3/patches/README.md delete mode 100644 common/optional/x86-64-v3/patches/abseil/README.md delete mode 100644 common/optional/x86-64-v3/patches/gnutls/README.md delete mode 100644 common/optional/x86-64-v3/patches/prometheus/README.md delete mode 100644 common/optional/x86-64-v3/patches/pytest-xdist/README.md delete mode 100644 common/optional/x86-64-v3/patches/scipy/README.md delete mode 100644 python/ebook_search/docker/README.md delete mode 100644 python/gems/README.md delete mode 100644 systems/brain/docker/docker_networks.md delete mode 100644 systems/jeeves/monitoring/README.md delete mode 100644 systems/portal-1/README.md diff --git a/common/optional/x86-64-v3/patches/README.md b/common/optional/x86-64-v3/patches/README.md deleted file mode 100644 index 2aa8a6b..0000000 --- a/common/optional/x86-64-v3/patches/README.md +++ /dev/null @@ -1,38 +0,0 @@ -# Package patches - -Each package follows the [GnuTLS layout](gnutls/README.md): - -- `default.nix` applies the patch through the package overlay. -- A descriptive `.patch` file contains the standalone upstream source change. -- `README.md` explains the problem, scope, reproduction, upstream status, - Nix integration, and recorded validation limits. -- Companion `verify-*` tools live beside the patch when needed; otherwise - the README gives commands for the package's existing tests. - -Keep package-specific evidence in its directory. Patch headers explain the -change independently of Nix, and `default.nix` preserves existing patches. - -| Package | Repair | -| --- | --- | -| [Abseil](abseil/README.md) | Public BMI2 header in Electron, Deno, and Signal's vendored copies | -| [GnuTLS](gnutls/README.md) | Wait for the UDP server socket before connecting | -| [Prometheus](prometheus/README.md) | Complete parsing before inspecting the test editor state | -| [pytest-xdist](pytest-xdist/README.md) | Check worker replacements and allow startup on loaded builders | -| [SciPy](scipy/README.md) | Account for floating-point rounding in STFT tests | - -## Local NixOS integration - -The [`x86-64-v3` optional module](../default.nix) -imports this directory's [`default.nix`](default.nix) directly, so the patch -overlay applies only to hosts using that package set. Prometheus patches its -separate assets derivation; Python packages use `pythonPackagesExtensions`. - -The [pytest-xdist directory](pytest-xdist/README.md) also owns its outer-worker -limit and remote-worker event timeout. These package overrides add no skipped -tests. Existing nixpkgs exclusions remain separate from these repairs. - -The review used Python 3.14.7 and the pinned x86-64-v3 package set. Host-flake -evaluation verified patch wiring, Python install checks, and Prometheus's -reference to the patched assets. No complete NixOS rebuild was performed. -Individual READMEs distinguish package builds, focused tests, and checks that -have not been run. diff --git a/common/optional/x86-64-v3/patches/abseil/README.md b/common/optional/x86-64-v3/patches/abseil/README.md deleted file mode 100644 index ff3843c..0000000 --- a/common/optional/x86-64-v3/patches/abseil/README.md +++ /dev/null @@ -1,59 +0,0 @@ -# Abseil BMI2 public header - -Vendored Abseil includes `bmi2intrin.h` directly when `__BMI2__` is enabled. -Compilers reject that internal header without the umbrella-header setup. -`bmi2-public-header.patch` includes `immintrin.h` instead, allowing builds -that enable BMI2 through `-march=x86-64-v3`. - -## Scope and behavior - -The patch changes one include in -`third_party/abseil-cpp/absl/container/internal/raw_hash_set.h`. -`default.nix` applies it to Electron 43's unwrapped package, Deno's -`librusty_v8`, and Signal's WebRTC dependency. It also supplies the patched -Electron package to Signal. These overrides apply only to `x86-64-v3`. - -The shared file path is relative to each vendoring project's source root, -not the root of a standalone Abseil checkout. No hash-table algorithm or -test exclusion changes. - -## Reproduction and focused checks - -From this directory, check and apply the patch to each vendored source tree: - -```sh -patch --dry-run --fuzz=0 -d /path/to/vendor-source -p1 < bmi2-public-header.patch -patch --fuzz=0 -d /path/to/vendor-source -p1 < bmi2-public-header.patch -``` - -A small compiler check isolates the header requirement. With GCC or Clang -on x86-64, compile `#include ` using `-march=x86-64-v3`; the -compiler rejects the direct include. Changing it to `#include ` -should compile. The full consumer builds below check integration with their -actual toolchains. - -## Upstream status - -Abseil addressed this issue through -[PR #2071](https://github.com/abseil/abseil-cpp/pull/2071), imported by its -upstream workflow. That change uses `x86gprintrin.h`; this local variant uses -the public `immintrin.h` umbrella header for the vendored toolchains. -Keep the workaround until all three bundled copies include a compatible fix. -This file is a local adaptation, not a verbatim copy of the upstream diff. - -## Local NixOS integration and build results - -[`../default.nix`](../default.nix) merges this directory's overlay fragment -because it repairs multiple packages. From the repository root, the consumer -build commands are: - -```sh -nix build --no-link -L .#nixosConfigurations.jeeves.pkgs.deno -nix build --no-link -L .#nixosConfigurations.jeeves.pkgs.electron_43 -nix build --no-link -L .#nixosConfigurations.jeeves.pkgs.signal-desktop -``` - -The earlier extraction checked the vendored header snapshots and evaluated -all three patch attachments. Those records do not establish successful full -consumer rebuilds. No new compiler or consumer build was run for the layout -change; the patch and override are unchanged. diff --git a/common/optional/x86-64-v3/patches/gnutls/README.md b/common/optional/x86-64-v3/patches/gnutls/README.md deleted file mode 100644 index c286028..0000000 --- a/common/optional/x86-64-v3/patches/gnutls/README.md +++ /dev/null @@ -1,160 +0,0 @@ -# GnuTLS UDP server readiness - -Under load, the test client can start before `gnutls-serv` binds its UDP -socket, and the first handshake fails with `Connection refused`. -`serv-udp.sh` currently waits a fixed four seconds; elapsed time does not -establish server readiness. `udp-server-readiness.patch` replaces that wait -with polling for the local IPv4 UDP endpoint. - -## Scope and waiting behavior - -The patch changes the existing `wait_udp_server()` and adds a new -`check_if_udp_port_bound()` beside it in `tests/scripts/common.sh`. -`serv-udp.sh` is its only caller in 3.8.13. The TCP helpers `wait_server()` -and `wait_for_port()`, including their existing sleeps, are unchanged. -Both original DTLS handshake checks remain unchanged. - -Each iteration checks process liveness and the socket **before sleeping**. -A ready socket returns immediately. An unsuccessful check sleeps two -seconds only if another attempt remains: at most 90 attempts, consistent -with the existing `wait_server()` budget implemented by `wait_for_port()`, -with no sleep after the final check. Server exit fails early; exhausting the -budget fails and terminates the server. No handshake is retried, and no -protocol timeout is changed. Once bound, the kernel can queue datagrams -while the server is scheduled; the probe itself sends no packets. - -The existing `have_port_finder()` prefers `ss`, then `netstat`. If neither -exists, it prints `neither ss nor netstat found` and exits **77 (skip)**. -In the normal test flow, port selection calls it before launching a server. -The probe runs in a subshell so that, even if this skip occurs after launch, -the waiting helper can terminate and reap the server before exiting 77. - -## Why an IPv4 socket is expected - -This is specific to the server used by this test, not a general rule that -IPv6 sockets cannot serve IPv4 clients. The client explicitly uses -`127.0.0.1`. The server's `--udp` path calls `udp_server()`, which calls -`listen_socket(..., SOCK_DGRAM)`. That function iterates the wildcard -addresses returned by `getaddrinfo(NULL, ..., AI_PASSIVE)`: - -| Server build / Linux setting | Binding behavior | -| --- | --- | -| IPv6 enabled, `net.ipv6.bindv6only=0` | Requests `IPV6_V6ONLY=1` on the IPv6 socket, binds `[::]:PORT`, and separately binds `0.0.0.0:PORT`. It overrides the system's dual-stack default. | -| IPv6 enabled, `net.ipv6.bindv6only=1` | The same explicit socket option and separate IPv4/IPv6 binds. | -| `HAVE_IPV6` undefined | Skips every address family except `AF_INET`; only the IPv4 wildcard is attempted. | - -`udp_server()` uses `wait_for_connection()`, which puts **every listener** -from that list into `select()` and returns a readable socket for `recvfrom()`; -it does not permanently choose one socket based on `getaddrinfo()` order. - -The first two cases were traced with the actual GnuTLS 3.8.13 binary in -separate Linux network namespaces: `setsockopt(IPV6_V6ONLY, [1])` and both -UDP binds returned success under each setting. The no-IPv6 case was checked -in source, not by building a second binary. The same bind implementation -was checked directly on GitLab master. - -Thus, successful normal startup for this invocation provides an explicit -IPv4 socket; a lone IPv6 wildcard is not the expected success path. -There is one portability caveat: upstream discards the return value of -`setsockopt(IPV6_V6ONLY)`. On a platform where that call fails and the server -ends up with only a dual-stack socket, this helper would time out despite -IPv4 reachability. Such a platform needs additional handling before this -patch can claim support. Blindly accepting every IPv6 wildcard would also -accept IPv6-only sockets before the separate IPv4 bind finishes. - -Source: [`src/serv.c`, `listen_socket()`](https://gitlab.com/gnutls/gnutls/-/blob/master/src/serv.c#L937), -[`src/udp-serv.c`](https://gitlab.com/gnutls/gnutls/-/blob/master/src/udp-serv.c), -and [`tests/serv-udp.sh`](https://gitlab.com/gnutls/gnutls/-/blob/master/tests/serv-udp.sh). - -## Port matching and ownership limit - -Only `-an` is passed to the socket-listing tool: BSD `netstat -u` selects -Unix-domain sockets, whereas Linux `netstat -u` selects UDP. The parser -handles the extra state column in `ss`, Linux colon-separated endpoints, -and BSD dot-separated endpoints, including `*.PORT`. It matches the full -local port and rejects TCP, IPv6 entries, peer ports, and longer numbers. - -A live PID plus a bound port does **not** prove that PID owns the socket. -Existing `GETPORT` selection checks for an unused port and uses a test -port-lock directory; `launch_bare_server()` also calls -`wait_for_free_port()` before starting the process. These are advisory: -the launcher does not enforce the latter's result, and another process -can bind between the check and launch. The patch does not close that race -or add nonportable PID parsing. An unrelated process can satisfy the -socket check; the real handshakes remain the functional check and may -fail (or reach the wrong server). This is a startup-order fix, not a -socket-ownership guarantee. - -## Reproduction and focused checks - -Apply the patch to an unpacked source tree, then run the companion checks -with Python's standard library and a shell: - -```sh -patch --fuzz=0 -d /path/to/gnutls -p1 < udp-server-readiness.patch -SHELL=/bin/sh python3 verify-readiness.py /path/to/gnutls/tests/scripts/common.sh -v -``` - -Set `NETSTAT=/path/to/netstat` to exercise one outside `PATH`. The checks -cover Linux/BSD output samples, false matches, immediate readiness, -missing tools, process exit, timeout cleanup, and real IPv4 UDP sockets -whose bind is delayed six seconds. The missing-tools fixture is skipped -if an absolute fallback `ss` path cannot be hidden with `PATH`. Native -BSD execution remains untested. - -To reproduce with GnuTLS itself, run `tests/serv-udp.sh` with `SERV` pointing -to a wrapper that sleeps six seconds, then `exec`s `gnutls-serv` with all -arguments. Set `CLI` to the matching `gnutls-cli`, `srcdir` to the source -`tests` directory, and `abs_top_builddir` to a writable build directory. -With GnuTLS 3.8.13, the original helper failed the first handshake with -`Connection refused`; the patched helper passed both with the same binaries. - -## GnuTLS submission - -Development and merge requests are on [GitLab](https://gitlab.com/gnutls/gnutls). -[`CONTRIBUTING.md` on master](https://gitlab.com/gnutls/gnutls/-/blob/master/CONTRIBUTING.md) -was read directly for this review. It requires the contributor's DCO -`Signed-off-by`, successful and failure test coverage, consistent coding -style, and adequate documentation; GitLab CI runs for merge requests. -Its commenting guidance asks for comments explaining non-obvious behavior -or protocol expectations. It does not prescribe an additional special -test-suite comment. The patch now explains its IPv4 binding assumption -next to the probe. - -The submission will contain the shell patch, without the Python verifier -or a new Python test dependency. The existing `serv-udp.sh` supplies the -functional success check. Running it through the six-second startup -wrapper supplies a reproducible regression case: it fails before the fix -and passes after it. The local verifier was used to validate socket-output -parsing and the helper's success, process-exit, skip-cleanup, and timeout -branches. Those branch checks are local evidence, not new automated -coverage in the upstream suite; the MR must state that distinction. - -No dedicated unit-test harness for these shell helpers was found in the -3.8.13 tests inspected. That does not establish that Python cannot be used -upstream; keeping this submission dependency-free is a scope choice. Use -the existing test and before/after reproduction as the submission's -coverage argument, retaining the platform limitations above. Apply the -patch in an upstream checkout and include those results with the -contributor's own sign-off. No MR or sign-off has been created. - -## Local NixOS integration and build results - -`overlays/default.nix` imports the `overlays/patches` overlay, which loads -`gnutls/default.nix` to apply the patch and keep `serv-udp.sh` enabled. -The patch itself has no Nix dependencies and applies to 3.8.13 and GitLab -master without fuzz. - -The final patch was rebuilt with: - -```sh -nix build --no-link -L .#nixosConfigurations.jeeves.pkgs.gnutls -``` - -That x86-64-v3 build passed: 927 tests, 796 passes, 131 existing skips, -zero failures/errors, and `PASS: serv-udp.sh`. The patch bytes in the built -derivation were compared with the repository artifact; both have SHA-256 -`59013d47fd446f2dd065012a2259ccc1898fedc8a053a630e13efa0076368760`. -All seven local checks passed, including skip cleanup and exactly 90 -probes with 89 sleeps on timeout. The six-second before/after reproduction -was also repeated successfully with the final helper. diff --git a/common/optional/x86-64-v3/patches/prometheus/README.md b/common/optional/x86-64-v3/patches/prometheus/README.md deleted file mode 100644 index be7399d..0000000 --- a/common/optional/x86-64-v3/patches/prometheus/README.md +++ /dev/null @@ -1,66 +0,0 @@ -# Prometheus complete test parsing - -CodeMirror gives editor-state creation a 20 ms synchronous parsing budget. -The shared `createEditorState()` test helper can therefore return an -incomplete syntax tree when the process is descheduled. The completion and -vector-matching tests immediately inspect that tree. - -## Scope and behavior - -`complete-test-parsing.patch` changes only -`module/codemirror-promql/src/test/utils-test.ts` inside `web/ui`. It completes -the small test expression with `ensureSyntaxTree(..., Infinity)` and publishes -the completed parse through an empty transaction so `syntaxTree(state)` sees -it. Failure to obtain a tree raises an error. - -The original assertions remain enabled, including `autocomplete topk params 2` -and `foo * on(test,blub) bar`. The unlimited budget applies to the test helper; -production editor parsing budgets are unchanged. - -## Reproduction and focused checks - -Use a disposable Prometheus 3.14.0 checkout. The patch root is `web/ui`, matching -the Nix assets derivation. From this directory: - -```sh -patch --fuzz=0 -d /path/to/prometheus/web/ui -p1 < complete-test-parsing.patch -cd /path/to/prometheus/web/ui -pnpm install --frozen-lockfile -pnpm --filter @prometheus-io/lezer-promql build -pnpm --filter @prometheus-io/codemirror-promql test -``` - -To force the scheduling condition, temporarily append this clock to -`module/codemirror-promql/setupJest.cjs` in the disposable checkout: - -```js -let parseClock = 0; -Date.now = () => (parseClock += 25); -``` - -Each clock read crosses the editor's initial parsing budget. Against the -original helper, the hybrid and vector suites have 186 failures, including -both locally excluded cases. With the patch, all 386 CodeMirror tests pass -under that same clock. Remove the injected clock before normal builds. - -## Upstream status - -This is a standalone test-helper patch for Prometheus 3.14.0. No upstream -submission was made during this work. Recheck the helper when updating -Prometheus or CodeMirror, including how an ensured parse becomes visible -through the editor state. - -## Local NixOS integration and build results - -[`../default.nix`](../default.nix) loads `default.nix`, which patches the -separate assets derivation. It updates both `passthru.assets` and the main -Prometheus build's reference to those assets. From the repository root: - -```sh -nix build --no-link -L .#nixosConfigurations.jeeves.pkgs.prometheus.assets -``` - -The full x86-64-v3 assets build passed with the normal clock, including the -CodeMirror and UI suites. Host-flake evaluation confirmed that the main -Prometheus derivation refers to these patched assets. The Go server package -was not rebuilt for this test-helper change. diff --git a/common/optional/x86-64-v3/patches/pytest-xdist/README.md b/common/optional/x86-64-v3/patches/pytest-xdist/README.md deleted file mode 100644 index 07fd295..0000000 --- a/common/optional/x86-64-v3/patches/pytest-xdist/README.md +++ /dev/null @@ -1,79 +0,0 @@ -# pytest-xdist test fixes - -With two workers and a restart limit of three, the fourth worker crash -requests shutdown while another test can still be running. That test may -also crash. The original queued-work test requires exactly four failures, -even though five failures can occur without exceeding the replacement limit. - -## Scope and behavior - -`concurrent-worker-crashes.patch` changes the assertions in -`TestNodeFailure.test_max_worker_restart_tests_queued` in -`testing/acceptance_test.py`. It requires exactly three replacements, four or -five failed tests, the failed-tests exit status, the limit message, and no -internal error. It retains the two-worker workload and ten queued tests. - -`worker-startup-timeout.patch` changes the remote-test helper's event timeout -from 10 to 60 seconds so loaded builders have time to start workers. The -helper returns immediately when an event arrives and still has a bounded wait. - -The existing nixpkgs pytest-9 compatibility patches remain in place. -Production scheduling and worker-restart behavior are unchanged. - -## Reproduction and focused checks - -Use a disposable pytest-xdist 3.8.0 checkout with its test dependencies and -the nixpkgs pytest-9 compatibility patches where required. From this directory: - -```sh -patch --fuzz=0 -d /path/to/pytest-xdist -p1 < concurrent-worker-crashes.patch -patch --fuzz=0 -d /path/to/pytest-xdist -p1 < worker-startup-timeout.patch -cd /path/to/pytest-xdist -python -m pytest testing/acceptance_test.py \ - -k test_max_worker_restart_tests_queued -q -python -m pytest testing/test_remote.py -q -``` - -Twenty unmodified runs passed during the review. To force the failing -schedule, modify the generated crashing test in a disposable checkout to -accept `worker_id`: make `gw3` wait for a marker created by `gw4`, and make -`gw4` pause 0.1 seconds after creating the marker. Then both have in-flight -tests when shutdown starts. Bound the marker wait so a reproduction failure -cannot hang the suite. The original assertion fails on five reported -failures; the patched test passes. - -## Worker startup and outer concurrency - -[`default.nix`](default.nix) runs the outer suite with one worker to limit -nested process pools. This is a Nix test-runner setting; the source timeout -change lives in [`worker-startup-timeout.patch`](worker-startup-timeout.patch). - -A separate reproduction inserts an 11-second `pytest_sessionstart` delay -into the child created by `test_basic_collect_and_runtests` in -`testing/test_remote.py`. The original 10-second channel wait fails; the -60-second wait passes. This bounds waits for test worker events, including -startup, rather than changing a product deadline. - -## Upstream status - -These are standalone test patches for pytest-xdist 3.8.0. No upstream submission -was made during this work. Recheck the allowed in-flight failures, replacement -count, and remote-test wait when updating the scheduler or worker behavior. - -## Local NixOS integration and build results - -[`../default.nix`](../default.nix) loads `default.nix` through -`pythonPackagesExtensions`. From the repository root: - -```sh -nix build --no-link -L .#nixosConfigurations.jeeves.pkgs.python314Packages.pytest-xdist -``` - -After consolidating the settings in this directory, the full x86-64-v3 package -build passed 185 tests, with 6 existing skips and 10 expected failures. Nix -evaluation confirmed the same outer-worker limit and preserved existing -patches, with the timeout now applied as a source patch. - -The earlier forced concurrent-crash and delayed-startup reproductions passed -after their fixes; the focused crash test also passed after formatting its -assertion. diff --git a/common/optional/x86-64-v3/patches/scipy/README.md b/common/optional/x86-64-v3/patches/scipy/README.md deleted file mode 100644 index dab2d2b..0000000 --- a/common/optional/x86-64-v3/patches/scipy/README.md +++ /dev/null @@ -1,70 +0,0 @@ -# SciPy STFT test tolerances - -The x86-64-v3 build can produce small floating-point residuals in inverse-STFT -comparisons and scaling round trips. The original bounds reject these results, -including residuals around `4e-17` where a round trip expects zero for a signal -with amplitude 2. - -## Scope and behavior - -`stft-test-tolerances.patch` changes only the signal tests: - -- The inverse-STFT comparison in `_scipy_spectral_test_shim.py` uses - `max(1e-7, 2 * np.finfo(x.dtype).eps)` as its relative tolerance. Float64 - keeps the original bound, and the existing i686 override remains. -- Three scaling round trips in `test_spectral.py` gain an absolute tolerance - of one epsilon for the input dtype, allowing small residuals near zero. - -The tests remain enabled, and the production STFT implementation is unchanged. - -## Reproduction and focused checks - -From this directory, apply the patch to a disposable SciPy 1.18.0 checkout: - -```sh -patch --fuzz=0 -d /path/to/scipy -p1 < stft-test-tolerances.patch -``` - -Build and install that tree with SciPy's test dependencies. From outside the -source directory, run the installed tests: - -```sh -python -m pytest --pyargs scipy.signal.tests.test_spectral \ - -k 'roundtrip_float32 or roundtrip_scaling' -q -``` - -Use the same compiler flags and numerical libraries for before/after runs. -The earlier reproduction called `TestSTFT.test_roundtrip_float32` and -`TestSTFT.test_roundtrip_scaling` against the x86-64-v3 libraries, then loaded -patched copies of the test modules. Both failed with the original bounds -and passed with the adjusted bounds. - -## Upstream status - -[SciPy issue #25488](https://github.com/scipy/scipy/issues/25488) records -related test failures with architecture-specific compiler flags. It is -context for the local tolerance repair; this exact patch has not been -submitted upstream during this work. - -## Local NixOS integration and build results - -[`../default.nix`](../default.nix) loads [`default.nix`](default.nix) through -`pythonPackagesExtensions`, preserving the package's existing patches. -The override also covers SciPy used to test other Python dependencies, -including pgvector in portal's shared Python environment. - -From the repository root: - -```sh -nix build --no-link -L .#nixosConfigurations.portal-1.pkgs.python314Packages.scipy -``` - -The original remote build of patched SciPy 1.18.0 passed 87,723 tests, with -8,342 skips, 300 expected failures, and 22 unexpected passes. The patch and -override have been restored byte-for-byte from commit `24cbf74f`; those counts -describe the earlier full build. - -Restoration checks confirmed that the patch applies to the pinned source -without fuzz, portal's evaluated SciPy retains its existing patch and install -checks, and pgvector uses the patched SciPy. A full package or system rebuild -was not repeated for this restoration. diff --git a/python/ebook_search/docker/README.md b/python/ebook_search/docker/README.md deleted file mode 100644 index 24a35b9..0000000 --- a/python/ebook_search/docker/README.md +++ /dev/null @@ -1,77 +0,0 @@ -# Ebook Search Docker - -Run the EPUB search app against the existing Postgres database on `jeeves`: - -```sh -python -m python.ebook_search.docker.containers start --library-path /path/to/epubs --build -``` - -All ebook-search Docker files live in this directory: - -- `Dockerfile` — multi-stage: `test` (runs pytest) and `runtime` (default target, the app image) -- `docker-compose.yml` -- `containers.py` — Typer lifecycle CLI -- `pyproject.toml` / `uv.lock` — the container's uv-locked dependencies - -The app listens on `http://localhost:8070`. - -Useful lifecycle commands: - -```sh -python -m python.ebook_search.docker.containers build -python -m python.ebook_search.docker.containers start --library-path /path/to/epubs -python -m python.ebook_search.docker.containers test -python -m python.ebook_search.docker.containers logs -python -m python.ebook_search.docker.containers ps -python -m python.ebook_search.docker.containers stop -``` - -Direct compose usage from the repo root: - -```sh -docker compose -f python/ebook_search/docker/docker-compose.yml ps -``` - -## Dependencies - -The image builds its environment with uv from `pyproject.toml` + `uv.lock` in this -directory — this is the source of truth for the container's dependencies. To add or -update a dependency, edit `pyproject.toml` here and regenerate the lock (uv is -available in the `ebook-search` dev shell): - -```sh -nix develop .#ebook-search -c uv lock --project python/ebook_search/docker -``` - -## Tests - -The main pytest suite excludes `tests/ebook_search` (its dependencies are no longer -in the nix dev shell). The `test ebook search` CI workflow runs them in a uv env -built from the lockfile in this directory — same commands work locally from the -repo root (the `--override-ini` drops the main suite's ignore): - -```sh -uv sync --locked --project python/ebook_search/docker -uv run --project python/ebook_search/docker --no-sync pytest tests/ebook_search --override-ini addopts="-n auto -ra" -``` - -They can also run inside the Docker `test` image, which validates the image itself: - -```sh -python -m python.ebook_search.docker.containers test -``` - -or the raw docker equivalent: - -```sh -docker build --file python/ebook_search/docker/Dockerfile --target test --tag ebook-search:test . -docker run --rm ebook-search:test -``` - -## Configuration - -The compose service loads the repo root `.env` into the container via `env_file`. - -Mount your EPUB directory by setting `EBOOK_LIBRARY_HOST_PATH` in an env file or on the command line. The container sees it as `/library`, and `EBOOK_SEARCH_LIBRARY_PATHS` is set to `/library` inside the container. - -Database connection settings are controlled by `RICHIE_DB`, `RICHIE_HOST`, `RICHIE_PORT`, `RICHIE_USER`, and `RICHIE_PASSWORD`. The default host is `jeeves`. diff --git a/python/gems/README.md b/python/gems/README.md deleted file mode 100644 index 83646d8..0000000 --- a/python/gems/README.md +++ /dev/null @@ -1,54 +0,0 @@ -# Gems - -Gems is a server-rendered, turn-based resource-engine game for one to four human or AI players. It uses FastAPI, -Jinja, HTMX, server-sent events, and SQLite. - -The application deliberately contains no playable card deck, patron/governor set, objective set, official artwork, -or copied rulebook text. A room host must upload a content pack they are entitled to use before starting a game. - -## Run locally - -```shell -uv run gems --host 127.0.0.1 --port 8082 -``` - -The default database and installation key are created under `.gems/`. The following environment variables override -runtime behavior: - -- `GEMS_DATABASE_PATH` -- `GEMS_KEY_PATH` -- `GEMS_PUBLIC_ORIGIN` -- `GEMS_SECURE_COOKIES` -- `GEMS_HOST` -- `GEMS_PORT` - -## Content packs - -The current schema is available from a running server at `/schemas/content-pack-v1.json`. A pack defines exactly -five normal resources, one wild resource, cards, and optional patrons, objectives, and outposts. `patrons` is the -canonical field name; `governors` is accepted as an input alias. - -Cards may use only the built-in, bounded effect vocabulary: - -- `none` -- `virtual_wild` -- `copy_bonus` -- `copy_and_claim` -- `multi_bonus` -- `claim_free` -- an optional discard-cards alternate cost - -Unknown fields, resource references, executable expressions, HTML, artwork URLs, and files larger than 512 KiB are -rejected. The normalized pack is private to its room and becomes immutable when play starts. - -## Neutral module mapping - -Gems calls the four optional mechanics Objectives, Outposts, Eastern Decks, and Fortifications. Lobby presets combine -these mechanics into the familiar base, objective-race, objective-plus-outpost, eastern-plus-fortification, and -all-module configurations. Component identities and values always come from the uploaded pack. - -## Jeeves - -The NixOS module runs one Uvicorn worker on `127.0.0.1:8002`, stores state in -`/zfs/media/services/gems`, and publishes it through HAProxy at `https://gems.tmmworkshop.com`. The DNS record must -point to Jeeves before ACME can issue the certificate. diff --git a/systems/brain/docker/docker_networks.md b/systems/brain/docker/docker_networks.md deleted file mode 100644 index c612f26..0000000 --- a/systems/brain/docker/docker_networks.md +++ /dev/null @@ -1,3 +0,0 @@ -# docker_networks - -docker network create -d bridge web diff --git a/systems/jeeves/monitoring/README.md b/systems/jeeves/monitoring/README.md deleted file mode 100644 index 56df1fe..0000000 --- a/systems/jeeves/monitoring/README.md +++ /dev/null @@ -1,33 +0,0 @@ -# Monitoring - -## Vultr API metrics - -The `vultr-exporter` service reads its API token from: - -```text -/zfs/storage/secrets/services/vultr-exporter -``` - -Create the file on Jeeves as root with the following contents: - -```text -API_KEY= -``` - -The token needs read access to the Vultr Account and Billing APIs. Unrelated -resource collectors are disabled in the packaged exporter. - -Restrict the file to root and ensure the public egress IP used by Jeeves is -allowed for the token in the Vultr API settings: - -```console -sudo chown root:root /zfs/storage/secrets/services/vultr-exporter -sudo chmod 600 /zfs/storage/secrets/services/vultr-exporter -``` - -The exporter listens on `127.0.0.1:9188`; it is scraped by the local -`prometheus-main` service every five minutes and is not exposed through the -host firewall. - -Portal-1 exposes its node exporter only through `tailscale0` on port `9100`. -Jeeves reaches it using the Portal-1 Tailscale hostname. diff --git a/systems/portal-1/README.md b/systems/portal-1/README.md deleted file mode 100644 index c7b1bbd..0000000 --- a/systems/portal-1/README.md +++ /dev/null @@ -1,83 +0,0 @@ -# portal_1 - -Minimal NixOS target for a Vultr VM, installed with nixos-anywhere. The Nix -flake target is `portal_1`; the machine hostname is `portal-1` because DNS -hostnames cannot contain underscores. - -## Before deploying - -1. Confirm the VM's system disk is `/dev/vda`. If it is not, update both - references in `disk-config.nix`. -2. Confirm the SSH public key in `default.nix` is the key that should have - administrator access. -3. Boot the VM into a NixOS installer or another nixos-anywhere-compatible - Linux rescue environment with root SSH access. Keep this environment - running while completing the SOPS bootstrap below. - -## Bootstrap SOPS - -Use the rescue environment's SSH host key as the permanent portal identity. -Replace `VM_IP` below: - -```console -ssh root@VM_IP 'cat /etc/ssh/ssh_host_ed25519_key.pub' | \ - nix shell nixpkgs#ssh-to-age --command ssh-to-age -``` - -This prints an `age1...` recipient; it does not copy the private key. Add the -recipient to `.sops.yaml`: - -```yaml -- &system_portal_1 age1... -``` - -Then add `*system_portal_1` to the age recipients for -`users/secrets.yaml`. Re-encrypt the existing file for the new recipient and -add the Tailscale key: - -```console -nix shell nixpkgs#sops --command sops updatekeys users/secrets.yaml -nix shell nixpkgs#sops --command sops users/secrets.yaml -``` - -Add the OAuth client secret from the `Auth Keys: Write` credential in the SOPS -editor and save it: - -```yaml -tailscale_auth_key: tskey-client-... -``` - -## Deploy - -From the repository root, replace `VM_IP` with the VM's public IP: - -```console -nix run github:nix-community/nixos-anywhere -- \ - --copy-host-keys --flake .#portal_1 root@VM_IP -``` - -This repartitions `/dev/vda`, so anything already on that disk is erased. The -layout reserves 8 GiB for swap and assigns the remaining space to the root -filesystem. - -`--copy-host-keys` preserves the same private SSH host key at -`/etc/ssh/ssh_host_ed25519_key` on the installed system. SOPS-Nix converts that -key to an age identity during activation. After the reboot, connect as -`richie` and verify that automatic Tailscale enrollment succeeded: - -```console -ssh -p 278 richie@VM_IP -sudo tailscale status -``` - -The installed OpenSSH service listens on port 278. Port 22 is served by -Endlessh and will not provide an SSH login. - -HAProxy uses the same frontend, routing, and rate-limiting configuration as -Jeeves. Portal manages the ACME certificates for the existing public domains; -their DNS records must resolve to Portal for HTTP-01 issuance and renewal. - -The application backends still use Jeeves' original `127.0.0.1` addresses. -Replace them with the corresponding Tailscale addresses before directing -application traffic through Portal. Ports 80 and 443 are allowed through the -firewall.