Repository navigation
feat(abi): generate the Zig and Rust wire enums from the normative Idris2 ABI (#120) - #811
Merged
Merged
Conversation
…ris2 ABI (#120) The 16-connector wire contract existed in five hand-maintained copies that agreed only by discipline. This makes three of them generated from the normative Idris2 ABI, so drift becomes impossible rather than merely detected. New: - src/Hypatia/ABI/Gen.idr + src/abi/hypatia-abi-gen.ipkg -- an Idris2 executable that IMPORTS Hypatia.ABI.Types and EVALUATES it. It never parses .idr text. Because Gen is the mainfile, --total is genuinely re-checked for it. - ffi/zig/src/connector_generated.zig - clients/rust/hypatia-client/src/connector_generated.rs - ffi/connectors.json is now generated (its "Golden source-of-truth" claim contradicted the ABI-authority ruling and is gone) - `just abi-gen` - .github/workflows/abi-codegen-drift.yml -- regenerates into a temp dir and diffs. Never regenerate-in-place-and-`git diff`: measured, that reads GREEN when the generator silently no-ops. - .github/workflows/zig.yml zigIdent/rustIdent live in Gen, not in the ABI -- the normative module must not know about its consumers. Both are total with one clause per constructor, so adding a connector without naming it is a BUILD failure, not a CI diff. The hand-written files keep their tests. A generated test re-asserts its own source and proves nothing. Also closes instance 21 of the guard/consumer trap: the Elixir guard read src/abi/Types.idr, a byte-identical copy compiled by nothing, so the live ABI could drift and the guard would stay green. It now reads src/Hypatia/ABI/Types.idr. A mutant proved the fix: renaming a connector in the live module reddens the guard while the dead copy is untouched. The guard also gained a wire-id check. A mutant showed the name-only version stayed GREEN through an id swap -- names and their order are unchanged by a renumber, so a name-only guard sleeps through a C ABI break. ffi/zig/build.zig now roots unified-api-adapter.zig. It was reachable from no test root, so its comptime 16-connector assertion and its four tests had never executed anywhere. `zig build test` goes from 3 tests to 7. Measurements worth recording: - `zig build test` is RED on main for pre-existing reasons: three of four roots fail to compile against the Zig 0.16 std (16 call sites, six removed or relocated symbols) plus an `opaque` type with fields. zig.yml therefore runs the maximal green subset, not the whole build. - `cargo test --workspace` is RED on main for an unrelated pre-existing reason: integration/Cargo.toml requires testcontainers ^0.28 AND testcontainers-modules ^0.15, which needs testcontainers ^0.27. That is unsatisfiable at the manifest level, so no lockfile can fix it and hypatia-client never gets compiled. Its five tests were verified against the generated enum in a standalone crate: 5 passed, 0 failed. - Consequently, before this PR NEITHER "independent pin" on the wire contract executed anywhere. The drift gate is currently the only executing pin, and zig.yml makes the comptime assertion the second. actions.lock: two workflow keys added by hand, purely additive (6 insertions, 0 deletions). Both actions were already present, so no new dependency entry. `gh actions-lock --verify-local` goes from "2 of 32 workflows failed: 4 not-pinned" to "All 32 workflows have complete lockfile coverage". Fix mode was not used -- it has previously de-pinned SHAs and invented invalid refs. Every gate was mutation-tested; all reds were drift or assertion failures, not compile errors: 1. hand-edit a generated Zig wire name -> gate RED, guard RED 2. swap wire ids 3<->4 in the live ABI, do not regenerate -> gate RED (3 of 3 files), guard RED 3. add a 17th connector without naming it in Gen -> generator BUILD fails (not covering) then name it and regenerate -> Zig comptime RED, Rust count_is_sixteen + from_id_round_trip RED 4. generator replaced by a silent no-op exiting 0 -> gate prints "connectors emitted: 0" / "files compared: 0 of 3" and fails Positive control on a clean tree: gate rc=0 printing 16 / 3 of 3, guard 4/4, zig 4/4, rust 5/5, `just abi-gen` byte-identical and idempotent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0113HQM9LVGkNCzU1WwkJZSV
Contributor
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (14)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
| zig version | ||
| test "$(zig version)" = "${ZIG_VERSION}" | ||
|
|
||
| - name: Run the wire-contract tests |
This was referenced Sep 22, 2026
Contributor
|
❌ Failed to create Coding Agent finishing-touch task. Please try again. |
hyperpolymath
added a commit
that referenced
this pull request
Sep 22, 2026
…#817) ## The gate I shipped in #811 is red on `main` #811 merged with its own new `ABI codegen drift` check failing. This fixes it forward. The gate is loudly red rather than silently green, so it is not a vacuous gate — but a gate that has never passed is not yet evidence of anything, and left red it trains people to ignore it. ### The cause, measured ``` INTERNAL ERROR: Can't find data file chez/support.ss in any of ["/home/runner/.idris2/idris2-0.7.0/support/chez/support.ss", ".../base-0.7.0/data/chez/support.ss", .../contrib-0.7.0/data/chez/support.ss"] ``` Compiling an Idris2 **executable** needs the Chez support tree at `$(idris2 --libdir)/support/chez/support.ss`. Type-checking and building a **library** does not. `verify-proofs.yml` only ever does the latter — `idris2 --check` over the proof files, `idris2 --build` on two library packages — and never links a binary. So the cache this job deliberately shares with it (same key `idris2-${IDRIS2_VERSION}-${runner.os}-2`, chosen so the gate would not repeat a 20-minute bootstrap) was populated **without** that tree. **Sharing a warm cache imports its omissions as well as its contents.** That is the generalisable lesson here, and it is why my local build passed while CI's failed: locally the support tree exists under pack, so the local success was never evidence about CI. ### The fix A step between "Verify Idris is on PATH" and "Build the generator" that: 1. resolves `$(idris2 --libdir)/support` and checks for `chez/support.ss`; 2. if absent, shallow-clones the Idris2 source at the **pinned** `IDRIS2_VERSION` and copies `support/` into place — `support/` is plain data (Scheme and C sources), needs no build, so this costs seconds, not a bootstrap; 3. **asserts the file is present by name** before continuing. Step 3 matters independently of step 2. A missing support tree previously surfaced as an opaque `INTERNAL ERROR` three steps later, inside the generator build, where it read like a generator bug. Now it fails where it is caused. ### Also in this PR `zig.yml` extracted the test count with `grep … || true`, which the `Hypatia` workflow scanner flagged at `zig.yml:99` as swallowing a non-zero exit. `set -o pipefail` already made a genuine `zig test` failure fatal, so nothing was actually masked — but a suppressed exit code is indistinguishable from a masked failure to a reader or a scanner, so this is a rewrite rather than an argument. `grep` exits 1 on no-match, which under `set -e` needs suppressing; **awk exits 0 whether or not it matched**, so the no-match case needs no suppression at all and the count check below it becomes the only thing that can fail the step. Measured, not assumed — the extraction was run against the real `zig test` output and three mutant logs: | Input | Extracted | Gate | |---|---|---| | real run (`All 4 tests passed.`) | `4` | green | | `All 0 tests passed.` | `0` | **red** | | no summary line at all | `0` | **red** | | empty log | `0` | **red** | | `3 of 7 tests passed.` | `7` | green | `awk` returns rc=0 on the no-match cases, which is the whole point of the change. The one remaining `|| true` in `abi-codegen-drift.yml` — a diagnostic `find` inside an already-failing branch — is replaced by a `[ -d ]` guard for the same reason. There is now no `|| true` in either workflow. ### Not caused by this change `governance / Actions lockfile verify` is red on every PR because #810 bumped 7 actions without refreshing `.github/workflows/actions.lock` (`taiki-e/install-action` v2.87.13→v2.87.15, `ruby/setup-ruby` v1.323.0→v1.324.0, `docker/setup-buildx-action` v4.4.0→v4.4.1, and a `github/codeql-action` SHA). Every finding is on a workflow this PR does not touch. Filed separately — folding an unrelated repo-wide fault into this PR would make both harder to review. The `Check` / `Test` / `Clippy` / `Cargo check + clippy + fmt` reds are pre-existing on `main` (the unsatisfiable `testcontainers` constraint, #814), as are the three `governance /` reds. ### Acceptance This PR is not done when it is merged; it is done when `abi-codegen-drift` reports **jobs > 0** and prints its denominator: ``` connectors emitted: 16 files compared: 3 of 3 ``` Refs #120 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_0113HQM9LVGkNCzU1WwkJZSV Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 22, 2026
hyperpolymath
added a commit
that referenced
this pull request
Sep 22, 2026
… pass (#819) ## Why this exists PR #817 was squash-merged at head `c82e9aa`, which carried only the **first half** of the fix. The second commit landed on the branch after the merge and is not in `main`. Measured, not assumed: ``` $ gh api repos/hyperpolymath/hypatia/commits/5060161.../check-runs completed failure abi-codegen-drift ``` `abi-codegen-drift` — the gate #811 shipped — **is still red on `main`**. This PR carries the remainder. ## What was actually wrong Running (not merely building) an Idris2 executable needs two support artefacts, and neither was present under `$(idris2 --libdir)` on the runner: | artefact | when it is needed | symptom | |---|---|---| | `<libdir>/support/chez/support.ss` | compile | `INTERNAL ERROR: Can't find data file chez/support.ss` | | `<libdir>/lib/libidris2_support.so` | run | `(while loading libidris2_support.so) cannot open shared object file` | #817 cured the first by copying `support/` in as data. That is not sufficient — `libidris2_support.so` is a **built C library**, not data, and the Chez backend *copies it into the executable's `_app` directory at link time*. Fixing an incomplete installation file-by-file reveals one layer per CI round trip, which is exactly how the second failure happened. So this uses the tree's own targets, which are complete by construction: ```sh make -C "$src" support make -C "$src" install-support PREFIX="$(idris2 --prefix)" ``` It builds `support/` only, never the compiler — seconds, not a bootstrap. ### ⚠ It corrects the cause #817 recorded #817 says library builds do not need the support tree, so the shared cache never had it. **That is false**, and the evidence was already in the repo. Idris2's top-level Makefile: ``` install: install-idris2 install-support install-libs ``` so `verify-proofs.yml`'s `sudo make install PREFIX=/usr/local` **does** install support — under `/usr/local/idris2-0.7.0/`. The binary resolves `idris2 --libdir` to `/home/runner/.idris2/idris2-0.7.0`, and that job's cache `path:` list names *both* prefixes. It is a **prefix mismatch**. Library work (`--check`, `--build` on the two proof packages) never needs the difference, which is why only a job linking an executable ever noticed. The fix is correct either way because it anchors on `$(idris2 --libdir)` / `$(idris2 --prefix)` rather than on a hard-coded prefix — but the comment in the workflow now says what was measured instead of the tidier story. ## Two further changes **Assert on the consumer's artefact.** #817 asserted the *installer's* output and the job still died at run time. This asserts `build/abi-gen/exec/hypatia-abi-gen_app/libidris2_support.so` — the thing that actually has to load it — because a build missing that copy still exits 0, and the failure then surfaces one step later inside the comparison, where it reads as a drift failure rather than a toolchain fault. Plus a `--help` smoke test: the cheapest proof the binary runs, and it writes nothing, so it cannot mask a drift failure by leaving output behind. **`Gen.idr`: `--help` exits 0.** It exited 1, so under `set -e` the smoke test would have failed and tempted an `|| true` — the exact pattern the `Hypatia` scanner flags and which is indistinguishable from a masked failure. Missing arguments are still a usage error and still exit 1. ## Verification done locally before pushing - `--help` / `-h` → rc=0; no-args and one-arg → rc=1. - Regeneration prints `connectors emitted: 16` / `files written: 3 of 3` with **zero diff** against all three tracked targets. - Mutants, against the gate's own comparison logic: | case | compared | drifted | verdict | |---|---|---|---| | positive control (clean tree) | 3 of 3 | 0 | **green** | | hand-edit generated Zig (`arrow_flight`→`arrowflight`) | 3 of 3 | 1 | **red** | | swap wire ids 3↔4 in the live `Types.idr`, no regen | 3 of 3 | 3 | **red** | | empty output dir | 0 of 3 | 3 | **red on the denominator** | The third is the decisive one: it proves the gate reads the **normative source**, not merely that the generated copies agree with each other. ## Note on caching The repair is deliberately **not persisted** — `Post Cache` is skipped on a cache hit, so this job never mutates the shared entry and redoes the clone+build each run. That is the intended property and should not be "optimised" away; two workflows writing one cache key is how the poisoned `-1` cache happened. ## Not blockers (pre-existing on `main`, per the standing ruling) `governance / Actions lockfile verify` (#818, from #810) · `Check` / `Test` / `Clippy` / `Cargo check + clippy + fmt` (#814) · `governance / {Code quality + docs, Validate Hypatia Baseline, Workflow security linter}` · `CodeQL Security Analysis` startup failure. None are touched by this diff. Refs #120, #811, #817 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_0113HQM9LVGkNCzU1WwkJZSV Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 22, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #120.
The 16-connector wire contract existed in five hand-maintained copies that
agreed only by discipline. This makes three of them generated from the
normative Idris2 ABI, so drift becomes impossible rather than merely detected.
What generates what
src/Hypatia/ABI/Gen.idr+src/abi/hypatia-abi-gen.ipkgbuild an Idris2executable that imports
Hypatia.ABI.Typesand evaluates it. It neverparses
.idrtext. BecauseGenis themainfile,--totalis genuinelyre-checked for it (Idris2 only re-checks
--totalfor the mainfile).ffi/zig/src/connector_generated.zigsrc/Hypatia/ABI/Types.idrclients/rust/hypatia-client/src/connector_generated.rsffi/connectors.jsonffi/connectors.jsonpreviously called itself the "Golden source-of-truth",which contradicts the ABI-authority ruling. That claim is gone.
zigIdent/rustIdentlive inGen, not in the ABI — the normative modulemust not know about its consumers. Both are total with one clause per
constructor, so adding a connector without naming it is a build failure,
which is strictly stronger than a CI diff.
The hand-written
unified-api-adapter.zigandconnector.rskeep their owntests. A generated test re-asserts its own source and proves nothing.
just abi-genregenerates. Every generated file carries// ABI source blob: <git hash-object src/Hypatia/ABI/Types.idr>.⚠ Deliberate consequence: any byte change to
Types.idr, commentsincluded, reddens the gate until
just abi-genis re-run and committed.Touching the normative ABI should force you to look at what it generated.
Three things this PR found, stated plainly
1. Before this PR, neither "independent pin" on the wire contract executed
anywhere.
comptime16-connector assertion and the four tests inunified-api-adapter.zigwere reachable from no test root.build.zigrooted
main.zig,json_writer.zig,file_ops.zigandtest/integration_test.zig; none import the adapter. They had never run,locally or in CI. This PR adds the fourth root —
zig build testgoes from3 tests to 7.
cargo test --workspaceis red onmainfor an unrelated pre-existingreason:
integration/Cargo.tomlrequirestestcontainers ^0.28andtestcontainers-modules ^0.15, which needstestcontainers ^0.27. That isunsatisfiable at the manifest level, so no lockfile can fix it and
hypatia-clientnever gets compiled. Its five tests were verified againstthe generated enum in a standalone crate: 5 passed, 0 failed.
So
abi-codegen-driftis currently the only executing pin on the wirecontract, and
zig.ymlmakes thecomptimeassertion the second. That is animprovement on nothing rather than "one of three independent pins".
2. The Elixir drift guard was reading a dead file. It bound its golden to
src/abi/Types.idr— a byte-identical copy that is compiled by nothing (thelive module is
src/Hypatia/ABI/Types.idr;src/abi/hypatia-abi.ipkgsetssourcedir = "..", which resolves tosrc/). The live ABI could drift and theguard would stay green. This is instance 21 of the guard-asks-a-different-
question-than-its-consumer trap. Repointed, and a mutant proves the fix:
renaming a connector in the live module reddens the guard while the dead copy
is untouched.
The guard also gained a wire-id check. A mutant showed the name-only
version stayed green through an id swap — names and their order are
unchanged by a renumber, so a name-only guard sleeps through a C ABI break.
3.
zig build testis red onmainfor pre-existing reasons unrelated tothis PR: three of four roots fail to compile against the Zig 0.16 std — 16 call
sites across six removed or relocated symbols (
std.fs.cwd,std.posix.getenv,std.fs.openFileAbsolute,std.fs.createFileAbsolute,std.fs.deleteFileAbsolute,std.time.timestamp) plus anopaquetypedeclared with fields at
main.zig:46.zig.ymltherefore runs the maximalgreen subset (
zig teston the adapter), not the whole build. Filed as its ownissue rather than folded in here.
The gate
.github/workflows/abi-codegen-drift.yml, nopaths:filter, so it fireson every PR and can be made required later without the filter/required
deadlock. (
verify-proofs.ymlfilters onsrc/abi/**,src/Hypatia/ABI/**and
verify/**and lists none of the generated targets, so a hand-edit toa generated file would not have fired it.)
It regenerates into
$RUNNER_TEMPand diffs — never in place withgit diff. Measured, in an isolated repo: a no-op generator +git diffproduces an empty diff, rc=0, reads GREEN. The temp-dir design on the same
no-op prints
connectors emitted: 0/files compared: 0 of 3and fails.It prints its denominator and fails on a zero one. It does not assert
N == 16— that would be a sixth hand-written16; the count is the proof'sjob (
connectorCount = Refl).It reuses
verify-proofs.yml's Idris2 install verbatim, including the samecache key so it shares that warm cache.
Mutants — a passing gate proves nothing until one dies
Every red below was verified to be a drift or assertion failure, not a compile
error.
GenrustFile,variant is not coveringcomptimeRED; Rustcount_is_sixteen+from_id_round_tripREDconnectors emitted: 0/files compared: 0 of 3, REDPositive control, clean tree: gate rc=0 printing
connectors emitted: 16/files compared: 3 of 3; guard 4/4;zig test4/4;zig build test7/7;Rust 5/5;
just abi-genbyte-identical and idempotent.actions.lockHand-edited, purely additive: 6 insertions, 0 deletions — two workflow keys
in alphabetical position. Both actions already had dependency entries, so none
was added. Tags, not SHAs, because that is the format every one of the other 30
workflow entries uses; a lone SHA-pinned workflow would be inconsistent with
the lock in use. Fix/rewrite mode was not used — it has previously
de-pinned 24 SHAs and invented invalid
uses:refs.gh actions-lock --verify-local(offline, writes nothing) goes from2 of 32 workflows failed: 4 not-pinnedtoAll 32 workflows have complete lockfile coverage.This gate lands advisory
rules/branches/mainon this repo returns[]— zero effective rules, provenwith a positive control (a repo returning 8 rule types) and a 404 negative
control.
Optimus-Branchexists but isenforcement: disabled. So nothinghere can block anything until a ruleset requires it. Giving it teeth is an
owner action, tracked separately.
⚠ Do not enable
Optimus-Branchfor this — it carries five requiredcontexts whose reporting on hypatia PRs is unmeasured, and a required context
that never reports is how a default branch deadlocks.
Follow-ups (issues, not blockers)
src/abi/*.idrmodules and correct the ~15 prosereferences — the agreed immediately-following PR.
integration/testcontainers conflict. NoteDEBT-REGISTER CI-1blames"a stale
dtolnay/rust-toolchain@stablepin … 13 jobs die atSet up job" —that is now wrong; the jobs start and die at cargo resolution.
mise.tomlpinszig = "latest"— a floating pin.verification/PROOF-STATUS.adocis stale.docs/abi/ABI-AUTHORITY.mdshould repoint tosrc/Hypatia/ABI/Types.idr.🤖 Generated with Claude Code
https://claude.ai/code/session_0113HQM9LVGkNCzU1WwkJZSV
Residue, filed as issues (added post-merge)
Per the standing rule that a new finding is an issue, not a merge blocker —
each with explicit acceptance criteria:
ffi/zigto the Zig 0.16 std. 3 of the 4 test roots do notcompile; 16 call sites (
std.fs.cwd×5,std.posix.getenv×3,std.fs.deleteFileAbsolute×3,std.fs.openFileAbsolute×2,std.fs.createFileAbsolute×2,std.time.timestamp×1), plusopaquetypescannot have fields at
main.zig:46. This is whyzig.ymlroots onlyunified-api-adapter.zigrather thanzig build test.mise.tomlpinszig = "latest", disagreeing withzig.yml'spinned
0.16.0.cargo test --workspaceis red onmain: unsatisfiabletestcontainers ^0.28vstestcontainers-modules ^0.15→^0.27. Recordsthat DEBT-REGISTER CI-1 names the wrong cause — it blames a stale
dtolnay/rust-toolchain@stablepin and says "13 jobs die atSet up job";the jobs actually start and die at cargo resolution.
src/abi/*.idrmodules and repoint the~15 prose references. The agreed immediate follow-up to this PR.
verification/PROOF-STATUS.adocis stale: wrong path, wrong LOC(140 vs 242), wrong proofs.
docs/abi/ABI-AUTHORITY.mdatsrc/Hypatia/ABI/Types.idrand re-digest.Discovered after merge:
ABI codegen driftgate shipped red: compilingan Idris2 executable needs the Chez support tree, which the cache shared with
verify-proofs.ymlnever carried because that workflow only buildslibraries. Fixed forward.
governance / Actions lockfile verifyis red on every PR becausechore(deps): bump the actions group with 7 updates #810 bumped 7 actions without refreshing
actions.lock. Not caused by thisPR; every finding is on a workflow it does not touch.