Skip to content

Unit tests built once per CI shard; customs@v0.3.1 - #516

Merged
winding-lines merged 1 commit into
mainfrom
customs-build-once
Nov 16, 2026
Merged

winding-lines merged 1 commit into
mainfrom
customs-build-once

Conversation

@winding-lines

Copy link
Copy Markdown
Contributor

What and why

customs 0.3.1 adds mojo-build-per-test (magmalake/customs#7). On main it reports four tasks that build each test file into a binary of its own: test-executor-core (25 files), test-executor-workers (4), test-scheduler-ui (9) and test-data-tables (7). The code those files share (morca, dista, postgres…) was compiled once per file.

scripts/unit-tests.sh now builds one binary per shard, following census's scripts/test.sh:

  • Build. scripts/unit-tests.sh build SHARD builds build/unit-tests-SHARD from a generated main under build/unit-tests-gen/. It goes through scripts/mojo-build.sh, so the build cache, MOJO_BUILD_CACHE_DIR on sai's runners and the ouromo split-cache path keep working.
  • Links. build/<name> is the path each pixi task always built (build/test-steps, build/test-control…), and is now a link to the shard binary. The binary dispatches on basename(argv[0]). The binaries and links stay directly in build/, so executable_dir() resolves to the same directory as before.
  • Run. scripts/unit-tests.sh run NAME runs build/NAME and checks the number of tests that ran against the file's def test_*. Tests are registered by name, not found with __functions_in_module(), which comes back incomplete in an imported module.
  • Unchanged: every task name, the three CI shards, the macOS (m4) job's task list, and the wrappers around each test (scripts/with-postgres.sh and the shell test that follows it). Each test file still runs as its own process.
  • Shards: executor-core 21 files, executor-workers 3 (tests/hcloud.sh now runs test-hcloud from it), scheduler-ui 8, data-tables 5. data-tables-source is test_tables and test_lake_batch built with -D MORCA_LAKE_SOURCE=1, as test-tables-source and test-lake-batch-source built them.
  • customs pin moves to v0.3.1. With this change customs 0.3.1 reports nothing here, so no warn is set.

Built on their own, as before

  • tests/test_source.mojo: its -D MORCA_SOURCE and -D MORCA_COMMIT would become every test's source in a shared binary.
  • tests/test_append_files.mojo: its tests take, as an argument, the dista runtime that its main starts before any connection, so they can't be run by name from another main.
  • tests/test_datacache.mojo: tests/dataview.sh also runs it as a tool (test-datacache plant …).

Test files changed

  • test_formal_pins, test_formal_migrate and test_formal_fence: main now runs the tests with TestSuite instead of calling each one.
  • test_replace, test_formal_retry and test_run_token: the MORCA_WAREHOUSE setup moved into before_tests(), which their own main and the shard binary both call.
  • In all six, the tests run in the same order as before (the order of the file, which was the order main called them in).

Left alone

Registry hashes: unchanged

These are build/<p> describe | jq .registry_hash on sai (Linux x86-64), after each run's build-shared:

Pipeline Before After
toy 0xa3bcc571ad7c6939 0xa3bcc571ad7c6939
daily 0x81ead348d0a21c1f 0x81ead348d0a21c1f
fanout 0x1f0ebf1f2a93d993 0x1f0ebf1f2a93d993

Tests run and passed per file, before and after

Measured on sai (16-core Linux x86-64), default environment for the three shards and data environment for test-data-tables. Before is main at 42a82b3, after is this branch.

Shard Task Test file Before: run / passed After: run / passed Built as
test-executor-core test-cgroups tests/test_cgroup.mojo 19 / 19 19 / 19 unit-tests-executor-core
test-executor-core test-crash tests/test_crash.mojo 7 / 7 7 / 7 unit-tests-executor-core
test-executor-core test-cron tests/test_cron.mojo 12 / 12 12 / 12 unit-tests-executor-core
test-executor-core test-formal-fence tests/test_formal_fence.mojo 1 / 1 1 / 1 unit-tests-executor-core
test-executor-core test-formal-migrate tests/test_formal_migrate.mojo 1 / 1 1 / 1 unit-tests-executor-core
test-executor-core test-formal-pins tests/test_formal_pins.mojo 2 / 2 2 / 2 unit-tests-executor-core
test-executor-core test-graph tests/test_graph.mojo 3 / 3 3 / 3 unit-tests-executor-core
test-executor-core test-highlight tests/test_highlight.mojo 5 / 5 5 / 5 unit-tests-executor-core
test-executor-core test-locate tests/test_locate.mojo 3 / 3 3 / 3 unit-tests-executor-core
test-executor-core test-logs tests/test_logs.mojo 2 / 2 2 / 2 unit-tests-executor-core
test-executor-core test-recommend tests/test_recommend.mojo 15 / 15 15 / 15 unit-tests-executor-core
test-executor-core test-resources tests/test_resources.mojo 12 / 12 12 / 12 unit-tests-executor-core
test-executor-core test-sample tests/test_sample.mojo 10 / 10 10 / 10 unit-tests-executor-core
test-executor-core test-source tests/test_source.mojo 5 / 5 5 / 5 own build
test-executor-core test-state tests/test_state.mojo 19 / 19 19 / 19 unit-tests-executor-core
test-executor-core test-step-cards tests/test_step_cards.mojo 6 / 6 6 / 6 unit-tests-executor-core
test-executor-core test-steps tests/test_steps.mojo 23 / 23 23 / 23 unit-tests-executor-core
test-executor-core test-store tests/test_store.mojo 25 / 25 25 / 25 unit-tests-executor-core
test-executor-core test-tasks tests/test_tasks.mojo 3 / 3 3 / 3 unit-tests-executor-core
test-executor-core test-timeline tests/test_timeline.mojo 4 / 4 4 / 4 unit-tests-executor-core
test-executor-core test-usage tests/test_usage.mojo 11 / 11 11 / 11 unit-tests-executor-core
test-executor-core test-warehouse tests/test_warehouse.mojo 9 / 9 9 / 9 unit-tests-executor-core
test-executor-workers test-hcloud tests/test_hcloud.mojo 6 / 6 6 / 6 unit-tests-executor-workers
test-executor-workers test-placement tests/test_control.mojo 6 / 6 6 / 6 unit-tests-executor-workers
test-executor-workers test-pressure tests/test_pressure.mojo 14 / 14 14 / 14 unit-tests-executor-workers
test-scheduler-ui test-config tests/test_config.mojo 17 / 17 17 / 17 unit-tests-scheduler-ui
test-scheduler-ui test-fill tests/test_fill.mojo 11 / 11 11 / 11 unit-tests-scheduler-ui
test-scheduler-ui test-metrics tests/test_metrics.mojo 6 / 6 6 / 6 unit-tests-scheduler-ui
test-scheduler-ui test-remote tests/test_remote.mojo 11 / 11 11 / 11 unit-tests-scheduler-ui
test-scheduler-ui test-toml tests/test_toml.mojo 10 / 10 10 / 10 unit-tests-scheduler-ui
test-scheduler-ui test-upgrade tests/test_upgrade.mojo 4 / 4 4 / 4 unit-tests-scheduler-ui
test-scheduler-ui test-watchdog tests/test_watchdog.mojo 12 / 12 12 / 12 unit-tests-scheduler-ui
test-scheduler-ui test-worker-api tests/test_worker.mojo 12 / 12 12 / 12 unit-tests-scheduler-ui
test-data-tables (data env) test-append-files tests/test_append_files.mojo 5 / 5 5 / 5 own build
test-data-tables (data env) test-formal-retry tests/test_formal_retry.mojo 1 / 1 1 / 1 unit-tests-data-tables
test-data-tables (data env) test-lake-batch tests/test_lake_batch.mojo 8 / 8 8 / 8 unit-tests-data-tables
test-data-tables (data env) test-lake-batch-source tests/test_lake_batch.mojo 8 / 8 8 / 8 unit-tests-data-tables-source
test-data-tables (data env) test-replace tests/test_replace.mojo 7 / 7 7 / 7 unit-tests-data-tables
test-data-tables (data env) test-run-token tests/test_run_token.mojo 3 / 3 3 / 3 unit-tests-data-tables
test-data-tables (data env) test-tables tests/test_tables.mojo 12 / 12 12 / 12 unit-tests-data-tables
test-data-tables (data env) test-tables-source tests/test_tables.mojo 12 / 12 12 / 12 unit-tests-data-tables-source
total 362 passed 362 passed

"Before" for formal_* / replace / run_token / append_files counts their PASS test_… lines; "after" counts their TestSuite summary (append_files is unchanged). The shell tests' ok lines are also the same before and after: core 55, workers 73, scheduler-ui 160, data-tables 27.

Timings (sai; empty Mojo compile cache and empty build/ for each shard)

Shard Before After
test-executor-core 817 s 667 s
test-executor-workers 712 s 696 s
test-scheduler-ui 812 s 766 s
test-data-tables (data env) 1398 s 1157 s

The shard binaries built in 84 s (core), 17 s (workers), 37 s (scheduler-ui) and 85 s (data-tables). Most of each shard's time is its shell tests and its build-shared builds, which this PR doesn't touch.

Not checked locally

  • The nightly environment, and macOS. This PR's CI runs macOS on the m4 runner; nightly runs on main.
  • test-data-sensors and test-data-keyed. They have no unit-test builds, so this change doesn't touch them.
  • pixi run -e fmt format-check passes on sai.

no-changelog: this is test infrastructure, and nothing changes for a user.

🤖 Generated with Claude Code

customs 0.3.1's mojo-build-per-test (magmalake/customs#7) reported four
tasks: test-executor-core (25 test files built one by one),
test-executor-workers (4), test-scheduler-ui (9) and the data
environment's test-data-tables (7). Each test file was a binary of its
own, so the code they share compiled once per file.

scripts/unit-tests.sh builds one binary per shard,
build/unit-tests-<shard>, through scripts/mojo-build.sh, so the build cache,
MOJO_BUILD_CACHE_DIR and the ouromo path keep working. build/<name>, the
path each pixi task always built (build/test-steps), is a link to it, and
the binary runs the tests of the file its name says. Each test file still
runs as its own process, from the same task, under the same wrappers
(scripts/with-postgres.sh) and before the same shell tests. Tests are
registered by name (census's scripts/test.sh), and `unit-tests.sh run`
checks the number that ran against the file's `def test_*`. Task names,
shard boundaries and CI are unchanged.

Shards: executor-core 21 files, executor-workers 3 (tests/hcloud.sh now
runs test-hcloud from it), scheduler-ui 8, data-tables 5, and
data-tables-source 2 (test_tables and test_lake_batch with
-D MORCA_LAKE_SOURCE=1, as test-tables-source and test-lake-batch-source
built them).

Built on their own, as before:
- tests/test_source.mojo: its -D MORCA_SOURCE and -D MORCA_COMMIT would be
  every test's source in a shared binary.
- tests/test_append_files.mojo: its tests take the dista runtime its main
  starts before any connection, so they can't be run by name.
- tests/test_datacache.mojo: tests/dataview.sh also runs it as a tool
  (`test-datacache plant ...`).

test_formal_pins, test_formal_migrate and test_formal_fence now run their
tests with TestSuite instead of calling each one; test_replace,
test_formal_retry and test_run_token set MORCA_WAREHOUSE in before_tests().
The order their tests run in is unchanged (the order of the file).

The test and example pipelines' multicall binary (scripts/pipelines.sh)
is untouched. Their registry hashes are unchanged (sai, `build/<p>
describe`): toy 0xa3bcc571ad7c6939, daily 0x81ead348d0a21c1f, fanout
0x1f0ebf1f2a93d993.

Co-Authored-By: Claude Opus <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017TycsxF3PkSX49fnv7cSZd
@winding-lines winding-lines added no-changelog Needs no changelog fragment (CI, docs or tests only) and removed no-changelog Needs no changelog fragment (CI, docs or tests only) labels Nov 16, 2026
@winding-lines
winding-lines merged commit 614701c into main Nov 16, 2026
40 of 43 checks passed
@winding-lines
winding-lines deleted the customs-build-once branch November 16, 2026 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog Needs no changelog fragment (CI, docs or tests only)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant