Checks a Mojo project's imports, and how it builds its tests. Six rules, in two groups. The full guide, with examples and how to fix each finding, is on magmalake.org/s/customs.
imports
import-cycle: no import cycles. A module imports only modules that don't import it back, directly or through others. Mojo compiles a cycle, so the compiler won't catch one. Every import counts: one inside a function, and one of a package's__init__.mojo(so a module ofpkgthat doesfrom pkg import x, whilepkg/__init__.mojoimports that module, is a cycle).import-through-reexport: import a name from the file that defines it.from m import x, andm.xafterimport m(orfrom pkg import m), name the module wherexis defined, not one that only imports it and passes it on. A package's__init__.mojois the one place a re-export may be imported from.unused-import: every name a file imports is used in it. Outside a package's__init__.mojo, whose imports are what it re-exports. A module that only passes names on (a compatibility shim) becomes a package:shim.mojo->shim/__init__.mojo, with the same import path. Since 0.4.0. To keep an import anyway, as Python's tools do:
tests
functions-in-imported-module: a file that calls__functions_in_module()isn't imported. In Mojo 1.2.0 that list is incomplete when its file is imported rather than built as the main file. A test file whosemainisTestSuite.discover_tests[__functions_in_module()]().run(), imported and run by another file (to build every test into one binary), can run some of its tests or none, and exit 0. The finding names the importing line. Build the tests once by registering them by name instead, as census'sscripts/test.shdoes: it generates the main underbuild/(which customs skips), imports eachdef test_*by name, and checks the count each file runs.mojo-run-per-test: no command runsmojo runon more than 3 test files. Eachmojo runcompiles the library code the tests share again. In morca-dev/census, 35mojo run -I src tests/test_*.mojotook 665 s on a 16-core Linux machine (building each file separately, 663 s), and one binary of every test about 210 s; in CI the step went from 550 s to 256 s. The finding names the task, CI step or script, and the files it runs. See How it reads tasks, workflows and scripts for what it sees.mojo-build-per-test: no command runsmojo buildon more than 3 test files. Building each test file into a binary of its own costs whatmojo runper file does (663 s against 665 s in census). It countsmojo builditself and a wrapper that passes its arguments on to it: a shell function (compile() { mojo build "$@"; }) or a script (scripts/mojo-build.sh -I src tests/test_x.mojo -o build/test-x). Building several programs (pipelines, a CLI, a test's driver) isn't a finding: only test files count. Since 0.3.1.
Every rule is deny unless configured.
A # noqa comment on an import's line allows the rules reported there,
as flake8
and ruff read one: # noqa alone allows all of them, and
# noqa: NAME, NAME only the rules or groups named
(# noqa: unused-import, # noqa: imports). It's read anywhere in the
comment, in any case. On the first line of a parenthesized import it covers
every name in it; on a name's own line, that name. It works for
unused-import, import-through-reexport and
functions-in-imported-module, not for import-cycle: a cycle is fixed,
or allowed for the whole project in the configuration.
flake8's codes (F401) aren't customs's names, and allow nothing.
from .copy import CopyIn # noqa: unused-import
from arrow_mlake.arrow import ( # noqa: unused-import
bit_get,
bit_set,
)
from json.cpu import parse as parse # kept: a redundant alias$ customs
src/morca/_internal/ui/layout.mojo:16: import cycle: morca._internal.ui.layout -> morca._internal.web.page -> morca._internal.ui.layout [import-cycle]
src/morca/_internal/ui/layout.mojo:16: from morca._internal.web.page import Page
src/morca/_internal/web/page.mojo:8: from morca._internal.ui.layout import with_header_help
src/morca/_internal/usage.mojo:46: from morca._internal.store import Usage: `Usage` is defined in morca._internal.db.run_rows, and morca._internal.store only re-exports it; import it from morca._internal.db.run_rows [import-through-reexport]
...
customs: 259 modules, 384 files: 3 import cycles, 146 imports through a re-export, 106 unused imports, 0 imports of a file that calls __functions_in_module(), 0 commands that run `mojo run` on more than 3 files, 0 commands that run `mojo build` on more than 3 files
customs [-I DIR]... [-A|-W|-D NAME]... [-v] [PROJECT]
- PROJECT (default
.): its modules are the.mojofiles underPROJECT/src(orPROJECTwithout asrc), named asmojo build -I srcnames them:src/a/b.mojoisa.b,src/a/__init__.mojothe packagea. A file is a module only inside packages, every directory above it having an__init__.mojo, as Mojo imports them. Its other.mojofiles (tests, examples, binaries) are checked as importers. Hidden directories,build,targetandnode_modulesare skipped. -I DIR: modules of another project, asmojo build -I DIR. Names imported from them are checked; their own imports aren't reported.-A NAME,-W NAME,-D NAME: allow (don't check), warn or deny a rule or a group, over the configuration file.-v: also notes what it can't place: each name that isn't found at the top level of the module it's imported from (a definition inside acomptime if, or a local variable that shadows a module's name), and eachmojo runwhose files it can't tell. Notes don't fail the run.
Each finding ends with its rule's name. It exits 1 when a deny rule finds
anything (a warn rule's findings are printed as warning: and don't
fail), and 2 on a usage, configuration or read error. Under GitHub Actions
(GITHUB_ACTIONS=true) it also prints each finding as an ::error (or
::warning) annotation, so the lines show on a pull request.
In the project's pixi.toml, after clippy's style: a level per rule or per
group, and each rule's numbers as flat kebab-case keys named after it.
[tool.customs]
mojo-run-per-test-threshold = 3 # report a command that runs more test files
mojo-build-per-test-threshold = 3 # report a command that builds more test files
[tool.customs.lints]
imports = "deny" # group: import-cycle, import-through-reexport, unused-import
tests = "deny" # group: functions-in-imported-module, mojo-run-per-test, mojo-build-per-test
# The examples' tests are separate programs: they can't share a binary.
mojo-run-per-test = "warn" # a rule's own level wins over its group's- Levels:
denyreports and fails,warnreports and passes,allowdoesn't check. Both groups aredenyby default. - A rule's own level wins over its group's, in the file and on the command
line; the command line (
-A,-W,-D) wins over the file. lintsmay also be an inline table (lints = { tests = "warn" }) or dotted keys (lints.tests = "warn").- An unknown option, rule, group or level is an error (exit 2), so a
misspelt one doesn't go unnoticed. pixi itself ignores the
[tool]table.
customs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: magmalake/customs@v0.4.0The action downloads the release of its own tag (or version:, or the
latest when its ref isn't a version) for the runner's platform and runs it
with args: (default .). Findings show as annotations on the pull request.
The Mojo repos of magmalake run customs on each pull request, as a job their one required check waits for, as in zstd.mojo:
# Imports: no cycles between modules, and each name imported from the
# module that defines it (github.com/magmalake/customs).
customs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: magmalake/customs@v0.4.0
ci:
needs: [release, release-check, test, lint-bench, customs]Adding it found two kinds of fixes. In postgres.mojo, a cycle:
src/postgres/connection.mojo:103: import cycle: postgres.connection -> postgres.copy -> postgres.connection
src/postgres/connection.mojo:103: from .copy import CopyIn
src/postgres/copy.mojo:60: from .connection import _ConnCell
copy needed the connection's shared cell and the helpers that raise
errors on it, and connection needed CopyIn; the cell and helpers moved,
unchanged, into a module below both. In
parquet.mojo, imports
through a compatibility shim:
tests/carrow_check.mojo:42: from parquet.arrow import bit_get: `bit_get` is defined in arrow_mlake.arrow, and parquet.arrow only re-exports it; import it from arrow_mlake.arrow
The shim exists for consumers, so it became a package
(parquet/arrow/__init__.mojo), where a re-export is allowed, and the
import path stayed the same.
It's a scanner, not a parser: string literals and comments are blanked
first, then from … import … (with parentheses or backslashes over lines)
and import … statements are read wherever they are, and a module's names
are what it defines at column 0 with def, fn, struct, trait,
comptime, alias or var. A name it can't place is a note, never a
finding.
For functions-in-imported-module, an import names a file when it's one of
the project's modules, or a file beside the importer (import test_a in
tests/all.mojo is tests/test_a.mojo, as Mojo finds it next to the file
it builds). A file that only mentions __functions_in_module() in a comment
or a string doesn't count.
mojo-run-per-test and mojo-build-per-test are a scanner too. They
read, at PROJECT:
- the tasks of
pixi.toml:[tasks],[feature.*.tasks]and theirtarget.*forms, as strings,{ cmd = …, depends-on = … }tables or[tasks.NAME]tables; a task is counted with itsdepends-onchain; - the
run:steps of.github/workflows/*.ymland*.yaml; - the shell scripts those run (
sh x.sh,bash x.sh,./x.sh,scripts/x.sh,source x.sh), the code ofbash -c '…'and of$(…), and the tasks run withpixi run NAME; and what those run in turn.
In each, the commands are cut into words as a shell would, comments and
here-documents dropped, and a mojo run X.mojo, or mojo X.mojo (the
same), counts X.mojo. Inside for f in WORDS; do … done, mojo run "$f"
(or tests/test_$f.mojo) counts each word, a glob (tests/test_*.mojo)
counting each file it matches, as does $(ls GLOB); so do variables set
to such words. mojo build X.mojo (and ouromo build) counts the same way
for mojo-build-per-test, and so does a call to a shell function or a
script that passes its arguments on to mojo build ("$@"): the call's
arguments are what is built, and a function's $1... are its arguments.
Only test files are counted: under a directory test or tests, or named
test_*, and, when the file exists, with a def test_ at its top level.
A program under tests/ (a driver a shell test starts, a file that must
fail to compile) isn't one. A task, step or script is reported when what
it runs, leaving out the files of what it runs that is reported already,
is more than the threshold: the place to change.
Where it can't tell, it doesn't report. A mojo run whose file it can't
work out (a variable it doesn't know, a matrix value ${{ … }}, a task
argument {{ … }}, xargs, while read, find) is a note with -v, not
a count; so is a mojo build. It doesn't follow cd, make or just,
scripts named by a variable, a script's own arguments (only a function's),
a task's cwd, nor pyproject.toml's [tool.pixi].
echo and printf lines are skipped, so a mojo run in a message isn't
counted.
cargo build --release # target/release/customs
cargo testIt has no dependencies. The release builds Linux binaries for musl, which
are static, and macOS ones, which link only the system's libSystem: each
is one file to download and run.