Repository navigation
Unit tests built once per CI shard; customs@v0.3.1 - #516
Merged
Merged
Conversation
customs 0.3.1's mojo-build-per-test (magmalake/customs#7) reported four tasks: test-executor-core (25 test files built one by one), test-executor-workers (4), test-scheduler-ui (9) and the data environment's test-data-tables (7). Each test file was a binary of its own, so the code they share compiled once per file. scripts/unit-tests.sh builds one binary per shard, build/unit-tests-<shard>, through scripts/mojo-build.sh, so the build cache, MOJO_BUILD_CACHE_DIR and the ouromo path keep working. build/<name>, the path each pixi task always built (build/test-steps), is a link to it, and the binary runs the tests of the file its name says. Each test file still runs as its own process, from the same task, under the same wrappers (scripts/with-postgres.sh) and before the same shell tests. Tests are registered by name (census's scripts/test.sh), and `unit-tests.sh run` checks the number that ran against the file's `def test_*`. Task names, shard boundaries and CI are unchanged. Shards: executor-core 21 files, executor-workers 3 (tests/hcloud.sh now runs test-hcloud from it), scheduler-ui 8, data-tables 5, and data-tables-source 2 (test_tables and test_lake_batch with -D MORCA_LAKE_SOURCE=1, as test-tables-source and test-lake-batch-source built them). Built on their own, as before: - tests/test_source.mojo: its -D MORCA_SOURCE and -D MORCA_COMMIT would be every test's source in a shared binary. - tests/test_append_files.mojo: its tests take the dista runtime its main starts before any connection, so they can't be run by name. - tests/test_datacache.mojo: tests/dataview.sh also runs it as a tool (`test-datacache plant ...`). test_formal_pins, test_formal_migrate and test_formal_fence now run their tests with TestSuite instead of calling each one; test_replace, test_formal_retry and test_run_token set MORCA_WAREHOUSE in before_tests(). The order their tests run in is unchanged (the order of the file). The test and example pipelines' multicall binary (scripts/pipelines.sh) is untouched. Their registry hashes are unchanged (sai, `build/<p> describe`): toy 0xa3bcc571ad7c6939, daily 0x81ead348d0a21c1f, fanout 0x1f0ebf1f2a93d993. Co-Authored-By: Claude Opus <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017TycsxF3PkSX49fnv7cSZd
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
customs 0.3.1 adds
mojo-build-per-test(magmalake/customs#7). On main it reports four tasks that build each test file into a binary of its own:test-executor-core(25 files),test-executor-workers(4),test-scheduler-ui(9) andtest-data-tables(7). The code those files share (morca, dista, postgres…) was compiled once per file.scripts/unit-tests.shnow builds one binary per shard, following census'sscripts/test.sh:scripts/unit-tests.sh build SHARDbuildsbuild/unit-tests-SHARDfrom a generated main underbuild/unit-tests-gen/. It goes throughscripts/mojo-build.sh, so the build cache,MOJO_BUILD_CACHE_DIRon sai's runners and theouromosplit-cache path keep working.build/<name>is the path each pixi task always built (build/test-steps,build/test-control…), and is now a link to the shard binary. The binary dispatches onbasename(argv[0]). The binaries and links stay directly inbuild/, soexecutable_dir()resolves to the same directory as before.scripts/unit-tests.sh run NAMErunsbuild/NAMEand checks the number of tests that ran against the file'sdef test_*. Tests are registered by name, not found with__functions_in_module(), which comes back incomplete in an imported module.scripts/with-postgres.shand the shell test that follows it). Each test file still runs as its own process.tests/hcloud.shnow runstest-hcloudfrom it), scheduler-ui 8, data-tables 5. data-tables-source is test_tables and test_lake_batch built with-D MORCA_LAKE_SOURCE=1, astest-tables-sourceandtest-lake-batch-sourcebuilt them.warnis set.Built on their own, as before
tests/test_source.mojo: its-D MORCA_SOURCEand-D MORCA_COMMITwould become every test's source in a shared binary.tests/test_append_files.mojo: its tests take, as an argument, the dista runtime that its main starts before any connection, so they can't be run by name from another main.tests/test_datacache.mojo:tests/dataview.shalso runs it as a tool (test-datacache plant …).Test files changed
test_formal_pins,test_formal_migrateandtest_formal_fence:mainnow runs the tests with TestSuite instead of calling each one.test_replace,test_formal_retryandtest_run_token: theMORCA_WAREHOUSEsetup moved intobefore_tests(), which their ownmainand the shard binary both call.maincalled them in).Left alone
scripts/pipelines.shand the test pipelines' multicall binary (CI: the test pipelines as one multi-call binary (#447) #448, CI: fold kill, worker-procs and pool-spread into other tests; -D pipelines into the multicall binary (#447) #450).scripts/check_layers.pyand the_cli/_lakehost/_rust_ffilayout (Remove the import cycles between _internal and the public modules #511).Registry hashes: unchanged
These are
build/<p> describe | jq .registry_hashon sai (Linux x86-64), after each run'sbuild-shared:Tests run and passed per file, before and after
Measured on sai (16-core Linux x86-64), default environment for the three shards and data environment for
test-data-tables. Before is main at 42a82b3, after is this branch.unit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-coreunit-tests-executor-workersunit-tests-executor-workersunit-tests-executor-workersunit-tests-scheduler-uiunit-tests-scheduler-uiunit-tests-scheduler-uiunit-tests-scheduler-uiunit-tests-scheduler-uiunit-tests-scheduler-uiunit-tests-scheduler-uiunit-tests-scheduler-uiunit-tests-data-tablesunit-tests-data-tablesunit-tests-data-tables-sourceunit-tests-data-tablesunit-tests-data-tablesunit-tests-data-tablesunit-tests-data-tables-source"Before" for formal_* / replace / run_token / append_files counts their
PASS test_…lines; "after" counts their TestSuite summary (append_files is unchanged). The shell tests'oklines are also the same before and after: core 55, workers 73, scheduler-ui 160, data-tables 27.Timings (sai; empty Mojo compile cache and empty
build/for each shard)The shard binaries built in 84 s (core), 17 s (workers), 37 s (scheduler-ui) and 85 s (data-tables). Most of each shard's time is its shell tests and its
build-sharedbuilds, which this PR doesn't touch.Not checked locally
test-data-sensorsandtest-data-keyed. They have no unit-test builds, so this change doesn't touch them.pixi run -e fmt format-checkpasses on sai.no-changelog: this is test infrastructure, and nothing changes for a user.🤖 Generated with Claude Code