Skip to content

Run the fake FXServer tests on ports the system gives, so that two runs on one host no longer collide - #56

Merged
ImBaphomettt merged 2 commits into
mainfrom
free-test-ports
Nov 11, 2026
Merged

ImBaphomettt merged 2 commits into
mainfrom
free-test-ports

Conversation

@ImBaphomettt

Copy link
Copy Markdown
Member

Two commits, tests only. The tests of crates/fake-fxserver/tests/ bound fixed loopback ports: 41920 to 41928 for supervisor.rs, pre_launch.rs and spawn.rs, 41926 to 41928 for runtime.rs, 41940 to 41942 for changes.rs, 41960 to 41962 and 41956 for artifacts.rs and archive.rs. Two checkouts that ran the suite at the same moment took each other's ports: AddrInUse in archive.rs, a BindError then "the status never came" or "the event never came" in the others. Each test now takes a port the system gives, and holds it until it ends.

What changed

A port the system gives, held until the test ends (b0e06aa)

  • crates/fake-fxserver/tests/common/mod.rs, which the test files include as mod common;: Port::free() binds 127.0.0.1:0, reads the port it was given, then claims it by binding the same port on 127.0.0.2, held until the test drops it. The system may give a port again as soon as nothing listens on it, and FXServer only listens once it has started: a test of another run given the same port meanwhile cannot claim it, and asks for another.
  • Ports 40120 to 40150 are passed over: the cfg scan warns of an endpoint on one of txAdmin's ports, and Linux gives them out.
  • archive.rs: fake_archive_says_it_listens_only_on_a_port_of_its_own still holds the port with another listener before fake-archive is started on it, now on a port it claimed.
  • spawn.rs: each test that gives the fake an endpoint takes a Port. a_taken_port_is_a_bind_error_then_a_fatal_exit holds 127.0.0.1:0 itself and expects the bind error on that address.

The labs (b102618)

  • Ports::new(n) lends ports to n labs of a file at a time: 9 in supervisor.rs and pre_launch.rs, 3 in runtime.rs, changes.rs and artifacts.rs. Those are the number of ports each file had, so no more servers run at once than before.
  • common::panel(): the panel address the labs give Observer, which the cfg checks only compare with, is a port held for the whole run instead of 127.0.0.1:41975, so that no server is given it.
  • artifacts.rs: fake-archive serves on a port claimed for the whole run of the binary instead of 41956.
  • backups.rs is unchanged: 3306 and 3307 there are values of a connection string that fake-dump copies into its dump. Nothing binds or connects to them.

Why

Several checkouts of the repository on one machine run cargo test or cargo xtask check at the same time. Their fake-fxserver tests failed on each other's ports and passed when run again alone, so a check could take several tries.

Notes for review

  • Why 127.0.0.2: a port the system has just given is free on 127.0.0.1, but nothing keeps another run from being given it again before the fake binds it. Holding the same port on a second loopback address leaves 127.0.0.1 to the fake and makes a second claim fail. Measured on Windows before writing it: 127.0.0.1:P binds while 127.0.0.2:P is held, a second bind of 127.0.0.2:P fails with AddrInUse (10048), and 2,000 claims take 81 ms.
  • Each file keeps the number of labs it ran at once, which its timings were set for.
  • d0bf382 moved the test server of observer-artifacts from an ephemeral port to five fixed ones, because WSL hands out ephemeral ports from 32768, "a range that holds ports other servers on the same machine are known to use". This branch does the opposite for fake-fxserver, as many of the workspace's tests already do (observer/tests/run.rs, observer-sftp, observer-backups). Under WSL is where to look first if it misbehaves.
  • Left as they are: the test server of observer-artifacts (41950 to 41954) and the browser tests' ports.
  • tests/runtime.rs is also changed by other branches waiting to merge: the second to merge will meet a small conflict in its use lines.

Verified

  • Before, at main (41ac843): the archive test binary started twice at the same moment; one run failed with AddrInUse (10048) at archive.rs:31, the failure the other checkouts met.
  • After, the same three times in a row: both runs passed each time.
  • After: cargo test -p fake-fxserver --no-fail-fast started twice at the same moment, three rounds in a row. All six runs passed, 130 tests each, with no BindError, AddrInUse or "never came" in their output. Their timestamped output shows the same test binaries running at the same moments in both runs: in the first round, supervisor.rs started at 04:44:38 in both.
  • b0e06aa on its own: cargo fmt -p fake-fxserver -- --check, cargo clippy -p fake-fxserver --all-targets -- -D warnings, cargo test -p fake-fxserver --test archive --test spawn.
  • On Windows, at b102618: cargo xtask check, that is fmt, icons, clippy with -D warnings with and without test-catalog, 3,223 Rust tests passed and none failed, bindings unchanged, and in web/ format, lint, type check, 758 tests and the build, under Node 26.13.
  • A first cargo xtask check had stopped at crashes::tests::a_dump_still_growing_past_the_window_counts_as_in_progress in observer-fxserver, which this branch does not touch. That test checks a 500 ms window, and it failed while the machine's CPU was at 100% with other checkouts checking at the same time. Run alone it then passed five times out of five, as did its whole binary, and the second cargo xtask check passed whole.
  • GitHub Actions does not start the jobs at the moment: on Let server resources hold a stop until they have saved, in contract 7 #55 the Panel job ended in 3 seconds, "not started because recent account payments have failed or your spending limit needs to be increased", and the two others were skipped. The checks above are the CI's Rust and panel steps, run locally on Windows.

Not verified

  • Linux: no WSL distribution is installed on this machine, so neither the claim on 127.0.0.2 nor the suite ran there. Linux routes all of 127.0.0.0/8 to the loopback, but that was not tried.
  • macOS has only 127.0.0.1 by default and would refuse the claim. FXServer runs on Windows and Linux only.
  • cargo xtask e2e: this branch changes no browser test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant