Richie
50d4aa383f
2.9 KiB
pytest-xdist concurrent worker crashes
With two workers and a restart limit of three, the fourth worker crash requests shutdown while another test can still be running. That test may also crash. The original queued-work test requires exactly four failures, even though five failures can occur without exceeding the replacement limit.
Scope and behavior
concurrent-worker-crashes.patch changes the assertions in
TestNodeFailure.test_max_worker_restart_tests_queued in
testing/acceptance_test.py. It requires exactly three replacements, four or
five failed tests, the failed-tests exit status, the limit message, and no
internal error. It retains the two-worker workload and ten queued tests.
The existing nixpkgs pytest-9 compatibility patches remain in place. Production scheduling and worker-restart behavior are unchanged.
Reproduction and focused checks
Use a disposable pytest-xdist 3.8.0 checkout with its test dependencies and the nixpkgs pytest-9 compatibility patches where required. From this directory:
patch --fuzz=0 -d /path/to/pytest-xdist -p1 < concurrent-worker-crashes.patch
cd /path/to/pytest-xdist
python -m pytest testing/acceptance_test.py \
-k test_max_worker_restart_tests_queued -q
Twenty unmodified runs passed during the review. To force the failing
schedule, modify the generated crashing test in a disposable checkout to
accept worker_id: make gw3 wait for a marker created by gw4, and make
gw4 pause 0.1 seconds after creating the marker. Then both have in-flight
tests when shutdown starts. Bound the marker wait so a reproduction failure
cannot hang the suite. The original assertion fails on five reported
failures; the patched test passes.
Remaining resource settings
../../test-exclusions.nix runs the outer suite
with one worker and sets the inner-worker wait to 60 seconds. These settings
limit nested process pools and allow worker startup on loaded builders.
A separate reproduction inserts an 11-second pytest_sessionstart delay
into the child created by test_basic_collect_and_runtests in
testing/test_remote.py. The original 10-second channel wait fails; the
60-second wait passes. This is a worker-startup bound, not a product deadline.
Upstream status
This is a standalone test patch for pytest-xdist 3.8.0. No upstream submission was made during this work. Recheck the allowed in-flight failures and replacement count when updating the scheduler or shutdown behavior.
Local NixOS integration and build results
../default.nix loads default.nix through
pythonPackagesExtensions. From the repository root:
nix build --no-link -L .#nixosConfigurations.jeeves.pkgs.python314Packages.pytest-xdist
The patched x86-64-v3 package passed 185 tests, with 6 existing skips and 10 expected failures. The forced concurrent-crash reproduction passed after the fix, and the focused test passed again after formatting the assertion.