From 29ac96ecdff907f10a4933fc52a80bffe5a9c616 Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 07:04:36 +0800 Subject: [PATCH 01/14] docs: the PR CI acceleration and toolchain specification design record --- ...-and-the-toolchain-specification-design.md | 954 ++++++++++++++++++ .agents/docs/README.md | 4 +- 2 files changed, 957 insertions(+), 1 deletion(-) create mode 100644 .agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md diff --git a/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md b/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md new file mode 100644 index 00000000..14f97da2 --- /dev/null +++ b/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md @@ -0,0 +1,954 @@ +--- +subject: design +status: active +--- + +# PR CI acceleration and the toolchain specification (#756, #757, #669) + +- Date: 2026-10-02. Status: proposed for review; revision 2. Nothing in this record is implemented. +- Base: `main` at `4d81d062` (2026.10.1.3). +- Inputs: + - CI jobs 110514498085, 110514498447 and 110514498760 on `main`. + - Issues #756, #757 and #669, with #669's 2026-10-01 measurement. + - A census of every place mcpp states a toolchain version. + - The packaging pipeline of the `gcc` and `llvm` payloads, and the upstream state of GCC 16.2 and + LLVM 23. + - The GitHub Actions record of seven head commits: four pull-request heads (#758, #754, #752, #745) and + three pushes to `main` (`4d81d062`, `68e49981`, `23c9590b`). Every job, step, queue time, cache lookup + and ninja count of these runs was measured. + +**Revision 2.** Revision 1 proposed moving the default toolchains to GCC 16.2 and LLVM 23. Review on +2026-10-02 settled the toolchain work as a specification only: SPEC-009 is written, and no version moves +in this work (Part V keeps the analysis for the move that will follow it). The subject of this record +therefore became the time a pull request waits for CI. That time is dominated by work the CI repeats, +and the triage of the three red jobs that started this record was the first evidence of it. + +Reading order: +- Part I triages the inputs. +- Part II is the main design: where PR CI time goes, the rules that remove the repetition, and an optional + external lane. +- Part III states the defects the next release repairs. +- Part IV is the toolchain specification, as approved. +- Part V keeps the toolchain candidates for later. +- Part VI orders the work, Part VII lists what is not yet measured, and Part VIII asks the open questions. + +## 0. Decisions + +### 0.1 Settled in review (2026-10-02) + +| # | Decision | +|---|---| +| T1 | The toolchain specification (Part IV) is published as a new specification, SPEC-009; SPEC-006 keeps identity, selection and the payload contract, and its §7 becomes a reference to SPEC-009 | +| T2 | No default toolchain moves in this work. GCC 16.2 and LLVM 23 are candidates (Part V), each subject to the gate of TS-10.5 (G1 to G6, tolerance 10 percent) | +| T3 | LLVM moves only to 23.1.3, once it passes the gate; 23.1.2 is measured as an early reading; `23.1.2-r1` only on explicit request | +| T4 | A default that an earlier mcpp recorded on a first run is announced once when a newer default exists, and the user chooses to move or keep it (TS-11.2); a fresh install announces nothing | +| T5 | When LLVM moves, the bare-metal and iOS rows lag with the reason recorded | +| T6 | GCC 16.2 is a candidate only if Bug 126577 does not reproduce in mcpp's own build and suite; otherwise the GCC line waits for 16.3 | +| T7 | The GCC recipe and builder live in `xlings-res/gcc`, as `musl-gcc`'s do | +| T8 | The macOS e2e step limit rises now; Part II replaces it with measured shards | + +### 0.2 Requested now + +| # | Decision | Recommendation | +|---|---|---| +| D1 | Each commit builds mcpp once per host, and every other job consumes that build (§2.4.1) | Yes | +| D2 | One top-level workflow orchestrates the stages; per-area work moves into reusable workflows (§2.4.2) | Yes | +| D3 | A pull request that changes only documentation runs the documentation checks and nothing else (§2.4.3) | Yes | +| D4 | Only pushes to `main` write caches; one job writes each key; the `target/` caches are removed (§2.4.4) | Yes | +| D5 | The e2e suite is sharded by measured duration, and a shard's step limit follows from its budget (§2.4.5) | Yes | +| D6 | Every e2e test runs on some host or carries a recorded reason, checked in CI; the `llvm` capability is granted (§2.4.6) | Yes | +| D7 | Legs that are known red run on `main` and on dispatch, not on pull requests (§2.4.7) | Yes | +| D8 | An external lane under `speak-agent` is designed now and built only if the rule of §2.5.5 holds after D1 to D7 | Yes | +| D9 | The next release repairs #757 and #756 (Part III) | Yes | +| D10 | The single line table of TS-3 waits for the first toolchain move | Yes | + +## Part I. Triage + +### 1.1 The three CI jobs + +| Job | Leg | Failing step | Cause | Class | +|---|---|---|---|---| +| 110514498085 | `ci-macos`, `xcode-27` (known red #669) | Test: non-module C++23 compilation | `ld64.lld` 22.1.8 cannot load `libc++.tbd` and `libSystem.tbd` of the Xcode 27 SDK (`unknown architecture`, `arm64e.x1`); every undefined symbol after it is a consequence | external, tracked by #669 | +| 110514498447 | `ci-macos-e2e`, `xcode-27` (known red #669) | Build mcpp from source | the same signature while linking `bin/mcpp` | external, tracked by #669 | +| 110514498760 | `ci-macos-e2e`, `macos-15` | E2E suite | the step reached its 25-minute limit at test 867 of 877; every earlier test passed, and test 867 was still printing readings when it was stopped | a time budget in this repository | + +The third job is not a hang. The suite started at 18:30:52 and was stopped at 18:55:54. The last green +run of the same suite (36895492498, the head of #755) took 21 min 01 s, so the budget left 19 percent +for runner variance, and this runner used more. The workflow's header still states that the suite +takes about 3.5 minutes on macOS and is therefore not sharded (`ci-macos-e2e.yml:6-7`); that was true +when it was written and is now false by a factor of six. + +None of the three is a usage error or a non-conforming project. + +### 1.2 #756: a provider's `.ixx` host module is invisible to the fast path + +**Conformance.** The reproducer conforms to the manifest specification. `.ixx` is opt-in through +`[build] module_extensions` (the built-in table is `.cppm` only, `modules/source-kind/src/source_kind.cppm:121`); +`host-module = true` is the Cargo-inherited kebab key and `module_extensions` the mcpp-owned snake key +(SPEC-004, naming table); `[lib] path` and `mcpp::define` (`src/build/hostprogram.cppm:48`) exist. The +workaround the report describes, declaring `.ixx` in the consumer, is excluded: the consumer has no +`.ixx` source, and the dead-entry warning it then receives is correct. + +**Mechanism.** `fast_path_identity` builds one extension table from the consumer's effective manifest +(`src/build/execute.cppm:1418-1437`). `dep_sources_newer_than` (`execute.cppm:1172-1215`) classifies +every file under every path-dependency root with that table. A `.ixx` file that only its own package +declares is classified as not affecting the graph's shape and is skipped. The host module is compiled +into the consumer's build program, which no edge of `build.ninja` names, so ninja cannot see the edit +either, and the fast path replays the old graph with the old build program. + +Classification is a property of the package that owns the file; the sweep asks a different package. +The mechanism does not depend on the platform, although the report was made on Windows. + +**Verdict.** An engine defect. Repair in §3.2. + +### 1.3 #757: an upgraded engine replays a graph that names the removed one + +**Conformance.** Upgrading mcpp through xlings and removing the previous version is ordinary use. + +**Mechanism.** The emitter writes the absolute path of the running engine into `build.ninja` for the +`__action` wrapper and for `stage` (`src/build/ninja_backend.cppm:3502-3512`), and states why that is +safe: "A version change regenerates the file (the version is in the fingerprint)." The fingerprint does +contain the version (`modules/toolchain-model/src/fingerprint.cppm:125`), but no fast path computes a +fingerprint. `try_fast_build` (`execute.cppm:1594`), `try_fast_workspace_build` (`:1749`) and +`try_fast_run` (`:1884`) select a recorded entry by target, profile, cache mode, features and toolchain +request, and then compare the recorded fingerprint with the recorded directory's basename, which is the +record compared with itself. `BuildCacheEntry` (`execute.cppm:150`) records no engine identity. The +emitter assumed a property that the fast path never promised. When the old executable still exists the +failure is silent instead: a newer front end drives the actions of an older engine. + +**A structural observation.** The three fast paths carry 23, 30 and 30 hand-written declines. A +property added to one must be added to the others by hand, and every field of `BuildCacheEntry` carries +its own paragraph explaining what its absence means. #757 is the case in which no path received the +property at all. + +**Verdict.** An engine defect. Repair in §3.1. Recognising `CreateProcess failed` in +`is_stale_ninja_failure` is rejected: it is a criterion over error text, and a launch failure of a +program the user declared must not be retried as a stale graph. + +### 1.4 #669: the macOS 27 link + +- The Xcode 27 SDKs list `arm64e.x1-macos` and `arm64e.x1-maccatalyst` in their `.tbd` stubs. + `ld64.lld` up to 23.1.2 rejects them (`unknown architecture` in 22.1.8, `unknown target` in 23.1.2). +- The fix, llvm/llvm-project#222721, merged to `main` on 2026-09-11. It reached `release/23.x` on + 2026-09-29 as `532fa5afbe2b`, `8f747d63ffb2` (lldb) and `ee66426152f9` (no ABI break), after + `llvmorg-23.1.2` was tagged on 2026-09-22. The 23.1.2 announcement schedules 23.1.3 for Tuesday + 2026-10-06. +- Measured (speak-agent/llvm-macos27-lab, run 36878561820): an `ld64.lld` built from `release/23.x` at + `21ef2ddb8060`, with every other component taken from the official 23.1.2 package, links and runs C, + C++23 and `import std` on the `xcode-27` image (macOS 27.0, Xcode 27.0, SDK 27.0). The stock 23.1.2 + `ld64.lld` fails there; both link on `macos-15` (SDK 15.5). The Command Line Tools `MacOSX26.5.sdk` + does not list the slice, so the failure follows the SDK, not the operating system. +- Other distributors carry the backport as a patch: Homebrew's `llvm` formula from 23.1.1_1, and + hermetic-llvm for 22 and 23. Apple's own linker reads the slice (third-party reports, not Apple + documentation). + +It is neither an mcpp defect nor the SDK-selection defect that xim-pkgindex#858 repaired. + +### 1.5 What the census found + +1. **The host defaults are older than the title of this work presumes.** macOS and Windows with MSVC + default to `llvm@20.1.7` (`modules/toolchain-model/src/triple.cppm:1045`, `:1049`); Linux on an + architecture other than x86_64 defaults to `gcc@15.1.0-musl` (`:1057`). `llvm@22.1.8` appears only as + the pin of 17 target rows and as mcpp's own `[toolchain] macos`. +2. **The macOS default cannot build mcpp.** libc++ 20's `std` module does not expose the comparison of + `directory_iterator`, so `llvm@20.1.7` cannot compile mcpp's sources (`.github/workflows/ci-linux.yml:278-284`); + mcpp's own manifest therefore builds with 22.1.8 while it gives its users 20.1.7. +3. **Defaults are stated in two tables and copied widely.** Table A is `pins::kFirstRun*` with duplicate + `kSuggest*` literals (`triple.cppm:1034-1081`); table B is `kKnownTargets[].pin` (`triple.cppm:447-845`). + Beyond them: 21 literals in nine source files, `mcpp.toml`, 175 rows of `tests/matrix/expected.tsv`, + seven workflows, one action, six CI tools, at least eight e2e scripts, examples, and 33 documentation + files. Two couplings are checked: `kFirstRunWinGnu` against the `x86_64-windows-gnu` row + (`tests/unit/test_windows_defaults.cpp:55-61`), and the host default against four documentation + statements (`.github/tools/check_default_toolchain_docs.py`). +4. **Two capability probes name payload versions.** `tests/e2e/run_all.sh:70-71` grants `musl` only if + `musl-gcc/15.1.0` is installed, and `:78-79` grants `mingw-cross` only for `mingw-cross-gcc/16.1.0`. + When the default moves, sixteen tests skip and report nothing. +5. **A first-run default is written once.** `src/build/prepare/toolchain.cpp:1607` persists it to + `config.toml`, and nothing revisits it. A default that moves therefore reaches fresh homes only. +6. **The pack ABI tag carries the compiler major** (`src/pack/abi_tag.cppm:215-224`). GCC 16.1 to 16.2 + keeps `gcc16-libstdcxx16`; LLVM 22 to 23 changes `clang22-libcxx22` to `clang23-libcxx23`, and a + prebuilt artefact with the old tag is refused (`src/pack/prebuilt.cppm:117`). `mcpp-index`'s + `main` holds no such artefact today. +7. **The GCC payload has no recipe that runs anywhere but one machine.** 16.1.0 was built by hand from + the `fromsource` recipe (`make -j8`, about 28 minutes), then stripped and its `specs` rewritten by + steps no repository scripts; SPEC-006 §5.1 already records the gap. `musl-gcc` and + `aarch64-linux-musl-gcc` have dispatchable builder workflows in `xlings-res`; `mingw-cross-gcc` and the + Windows-host Canadian cross are manual; `mingw-gcc` mirrors winlibs, which publishes 16.2.0. + +## Part II. PR CI acceleration + +### 2.1 What was measured + +Seven head commits were measured from the GitHub Actions API and the job logs. The four pull-request heads +are the final heads of #758 (`51b66cec`), #754 (`71d09f0b`), #752 (`72e238dc`) and #745 (`ac4beb88`). +The three pushes to `main` are `4d81d062`, `68e49981` (documentation only) and `23c9590b`. Only runs with +event `pull_request` or `push` are counted. The release workflow, which a tag push starts beside the CI of +the same commit, is reported separately. The repository has no required status check, so a pull request +waits for the last job to finish; the four pull requests were merged between 32 s and 183 s after their +last job ended. + +| | #758 | #754 | #752 | #745 | `4d81d062` | `68e49981` (docs) | `23c9590b` | +|---|---|---|---|---|---|---|---| +| jobs | 47 | 45 | 46 | 46 | 35 | 35 | 35 | +| wall clock, first attempt (min) | 49.6 | 42.0 | 37.1 | 35.7 | 59.5 | 42.9 | 27.0 | +| wall clock with reruns (min) | 79.5 | 42.0 | 37.1 | 50.7 | 59.5 | 42.9, left red | 27.0 | +| wall clock without queueing, modelled (min) | 36.3 | 28.6 | 26.7 | 25.5 | 59.4 | 33.5 | 27.0 | +| runner minutes | 568 | 477 | 467 | 501 | 478 | 436 | 417 | +| builds of mcpp from source | 37 + 6 | 35 + 6 | 36 + 6 | 36 + 6 | 30 + 6 | 30 + 6 | 30 + 6 | + +The second figure in the last row counts the deliberate rebuilds after `mcpp clean` or with another +toolchain. + +Where the runner minutes go, as a share of each commit's total: + +| category | share | +|---|---| +| building mcpp (primary builds, then the deliberate rebuilds) | 34-40 percent, then 8-11 percent: 44-49 percent together | +| the e2e suite | 18-22 percent | +| `mcpp test` | 6-7 percent | +| setup actions | 4-14 percent; the high values are `Install wine` (below) | +| cache saves | 1-5 percent | +| everything else | 13-17 percent | + +Median duration of the primary build step, over all seven commits: + +| host | median | range | +|---|---|---| +| Linux | 6.6 min | 4.1 to 16.3 min; the maximum is the aarch64 fresh install | +| Windows | 4.8 min | 2.2 to 6.0 min | +| macOS | 2.7 min | 1.9 to 4.9 min | + +### 2.2 What the measurements show + +**F1. Every job builds mcpp, and no job reuses another's build.** +- Thirty to thirty-seven jobs per commit compile mcpp from source with the pinned bootstrap. +- Only three hand-offs of a built binary exist: ci-windows `build-test` to `no-msvc-fallback` + (`ci-windows.yml:113-165`), the openkal `run` legs, and the Windows-to-Linux cross build. +- The present shape was chosen deliberately (`ci-linux.yml:8-15`), on the premise that a warm rebuild costs + about 2.5 minutes, which is cheaper than serialising the legs behind a shared artifact. The premise no + longer holds: the measured median is 6.6 minutes on Linux, and F2 shows that the rebuild is not warm. + +**F2. The `target/` cache makes no build incremental.** +- On the documentation-only commit `68e49981`, the compiled inputs were identical to the previous commit's, + and 14 jobs restored their `target/` by exact key. In the Linux `build + unit tests` job of that commit, + ninja still ran 830 of 830 edges. +- On #758, a restore-key hit ran 834 of 834. +- In #758's `toolchain: gcc`, the first build (with 94 MB of restored `target/`) took 362.75 s. The cold + rebuild after `mcpp clean` took 381 s, only 5 percent more. +- Build steps after an exact hit are no faster than after a miss (Linux 488 s against 340 s, where the six + exact hits also missed the sandbox; Windows 293 s against 324 s). +- The cause is not measured. Checkout giving every source a modification time newer than the restored + outputs is the likely one. + +**F3. The caches evict each other.** +- The repository held 10.78 GB of caches against a 10 GB limit. +- A pull-request run saves 34 to 42 caches, of 17 MB to 1.6 GB each; `target/` caches alone range from + 46 MB to 3.3 GB. 26 job identifiers form 26 `target/` lineages on Linux. +- In 155 job logs: + - the sandbox cache missed 44 percent of lookups and the xlings cache 36 percent; + - a Windows sandbox key that hit at 17:02 and 17:11 was gone by 17:44; + - after #758's saves, the next push to `main` missed 20 of 26 sandbox lookups. +- Parallel jobs race to save one key: two to twenty-seven "Unable to reserve cache" failures per commit. +- A cancelled run saves nothing, and every CI workflow cancels in progress, including on `main`. + +**F4. The 20-job limit is reached by this repository alone.** +- On each of the seven commits, the peak number of running jobs was exactly 20; macOS reached its own + limit of 5 on three of them. +- Sixty of the 67 Linux and Windows jobs that queued 90 s or more started within 8 s of another of our + jobs finishing, which is the signature of a slot cap. +- Queueing added 9 to 13 minutes of wall clock on five of the seven commits. +- macOS jobs queued up to 12.7 minutes. 15 of the 18 macOS jobs that queued over two minutes carry + GitHub's capacity notice, so part of the macOS wait is GitHub's capacity rather than ours. + +**F5. The e2e suites are the last jobs, and they hit their limit.** +- An e2e shard was the last job on three of the seven commits (Windows 2/2 at 27 to 34 minutes; Linux 2/2 + at 28.6). +- The 25-minute step limit was reached on three of the seven commits, and a fourth failed a test that + passed on rerun. Two were rerun, at +30 and +15 minutes of wall clock; two stayed red on `main`. +- On Windows, about 15 tests of over 30 seconds take 58 to 64 percent of a 15 to 18 minute shard. The same + tests take about one second on Linux, for example `36_llvm_toolchain` and + `687_the_vendored_xlings_probe_is_an_argument_vector`. +- Shards are assigned round-robin (`tests/e2e/run_all.sh:379-414`), and the two shards differ by 2.7 + minutes on average on Linux and 3.6 on Windows. +- macOS is not sharded: one job of 242 tests runs 17 to 21 minutes. The workflow header still gives + 3.5 minutes (`ci-macos-e2e.yml:6-7`). + +**F6. A documentation-only commit costs as much as a code commit.** +- No workflow has a `paths-ignore`. +- `68e49981`, which changed two files under `.agents/docs`, ran 35 jobs and 36 builds in 436 runner + minutes, against 417 for the code commit before it, and it ended red on a Windows shard timeout. + +**F7. One evicted cache costs up to fifty minutes.** +- When the `wine-debs` cache is evicted, `apt` downloads the Wine packages at stalled speeds: + - 25.8 minutes on #758; + - 49.9 minutes on `4d81d062`, where libwine (105 MB) alone took 841 s; + - 40 seconds when the cache hits. +- On two of the seven commits this made `mingw-cross-wine` the last job (59.5 minutes of wall on + `4d81d062`). + +**F8. Known-red legs spend macOS slots on pull requests.** +- The two `xcode-27` legs fail on every commit (#669), after 2 to 4 minutes, each holding a macOS slot. +- A pull request cannot change their outcome unless it is about #669. + +**F9. Seven e2e tests never run in CI.** +- `run_all.sh` lists `llvm` as a capability a test may require (`run_all.sh:263`), but no line grants it. +- Of the 32 tests that require it, 25 are invoked directly by name in a dedicated job. +- Seven are invoked nowhere: 134, 135, 136, 137, 741, 875 and 876. Among them are 875 and 876, the tests + #755 added for a toolchain named by path and for the toolchain phase. +- Their skip lines read like legitimate ones. + +### 2.3 Rules + +These rules govern PR CI. They are stated so that a later change of the CI can be checked against them. + +| Rule | Statement | +|---|---| +| R1 | A commit builds mcpp from source once per host; every job that needs the commit's mcpp consumes that build. A job builds mcpp again only when the rebuild is what it tests: another toolchain, a cold build, a cross target. | +| R2 | What a job runs follows from what the commit changed. A change that cannot affect a job's outcome does not start the job. | +| R3 | A cache has one writer, the `main` lineage. Pull requests restore; they do not save. A cache that does not shorten the job that restores it is removed. | +| R4 | A job's time limit follows from its measured duration; a limit that is reached without a hang is a budget defect, not a test failure. | +| R5 | Every test runs on at least one host, or carries a recorded reason why no host runs it; CI checks this. | +| R6 | The scarcest resource is a macOS slot (5 per organisation). A macOS job runs only work that needs macOS. | +| R7 | A pull request waits only for work that can change its outcome. | + +### 2.4 The design inside the repository + +#### 2.4.1 One build per host (R1) + +Each commit has one build job per host: + +| host | runner | +|---|---| +| Linux x86_64 | ubuntu-24.04 | +| Linux aarch64 | ubuntu-24.04-arm | +| macOS arm64 | macos-15 | +| Windows x86_64 | windows-latest | + +A build job: +1. restores the sandbox; +2. runs the bootstrap; +3. builds the commit's mcpp; +4. packs it with `mcpp pack`, as ci-windows `build-test` already does for `no-msvc-fallback` + (`ci-windows.yml:68-119`), so that the artifact carries its own runtime closure; +5. uploads the result as an artifact. + +Every consumer downloads and unpacks that artifact and uses it as `$MCPP`. A consumer still restores the +sandbox when its tests need toolchain payloads. The `find … -newer mcpp.toml` and newest-binary selections +that protect against a stale cached binary (`ci-target-matrix.yml:59-63`, `ci-windows.yml:43-48`, +`openkal-cross.yml:150-180`) are removed, because the binary no longer comes from a restored `target/`. + +Jobs whose subject is a rebuild keep it, and start from the artifact instead of a bootstrap build: +- the GCC cold rebuild; +- the musl and LLVM builds; +- the cross builds; +- the macOS rebuild with the LLVM default; +- the Windows LLVM rebuild. + +This removes the bootstrap build that each of them pays first. + +`mcpp test` keeps a job of its own per host in the first wave. It compiles the code under test and the +144 unit-test programs itself, so the artifact would spare it only the bootstrap build, at the price of +waiting for the build stage. On Linux it is the longest job today: 24.7 minutes, of which 8.2 are the +build and 15.4 the tests. If it becomes the critical path, it splits into the root suite and the +per-member suites (`ci-linux.yml:196-218`). + +The target-matrix and openkal legs on `macos-14` and `windows-2022` consume the artifact of their host +family. For `macos-14` this holds only if the artifact's minimum macOS version is at most 14 (Part VII). +Whether these legs must build with `--dev` is not recorded anywhere; they move to the artifact unless a +reason is found (Part VII). The target matrix's `invariants` and `scan` become one job per host, because +`scan` already waits for `invariants` on the same host (`ci-target-matrix.yml:233`). + +Counted per pull-request commit, the change is: +- builds: from 41 to 43 down to about 16, which is four artifact builds, three `mcpp test` jobs, and about + nine rebuilds that are a job's subject; +- build runner minutes: from about 235 down to about 90. + +#### 2.4.2 One workflow, stages in order (R1, R7) + +An artifact is shared without extra machinery only within one workflow run. The CI therefore becomes +one top-level workflow, `ci.yml`, with three stages: + +1. `changes`: classify the commit's paths (§2.4.3), in seconds. +2. `docs`: the text checks that need no binary. They are the 14 checks `build-test` runs today before + its bootstrap (`ci-linux.yml:55-158`). +3. `build`: a matrix over the four hosts. + +The present per-area workflows become reusable workflows that the top-level workflow calls with the +artifact names: Linux, Linux e2e, Windows, Windows e2e, macOS, macOS e2e, iOS, target matrix, cross +build, openkal cross, and the MSVC xlings test. Their jobs keep their names, so that the checks a reader +knows remain recognisable. Path-gated workflows (`ci-aarch64-fresh-install`, +`measure-windows-tool-crt`, `pypi-publish`) stay separate. + +The wall clock of a pull request becomes the sum of three stages: + +| stage | duration | +|---|---| +| `changes` | under a minute | +| the slowest build | about 10 minutes: Linux setup, a 6.6-minute build, packing, upload | +| the slowest consumer | setup, plus the longest shard of §2.4.5 | + +The first wave of jobs is the four builds, the docs job and the three `mcpp test` jobs. The first wave +today is all 46 jobs at once, which is what saturates the 20 slots (F4). + +#### 2.4.3 What a change starts (R2, R7) + +`changes` sorts the changed paths into classes: + +| class | paths | starts | +|---|---|---| +| documentation | `**/*.md`, `docs/**`, `.agents/**`, `LICENSE*` | the `docs` stage | +| documentation that a check reads | a changed documentation path that a script under `tests/` or `.github/tools/` names, found by searching for it; today this includes `docs/01-getting-started.md`, `docs/20-toolchains.md` and `docs/03-examples.md`, with their `docs/zh/` copies | the `docs` stage, plus the Linux build and every test and tool that names the path, among them the default-toolchain check (`check_default_toolchain_docs.py`) and e2e 616 | +| everything else | | the whole CI | + +A commit that changes nothing outside the documentation class costs one docs job of about two minutes, +instead of 35 jobs and 436 runner minutes (F6). No check is required by the repository's rules, so a +workflow that does not start leaves no check pending. + +The second class is derived by a search, not listed by hand, so that a test added later to read a +document is found without editing the classifier. The classes are deliberately coarse. A finer selection, such as running only the e2e tests a change +could affect, is rejected. A differential selection cannot see a test that is red on both sides, and the +repository has paid for that before. + +#### 2.4.4 Caches with one writer (R3) + +- **Who writes.** Sandbox (`~/.mcpp`) and xlings (`~/.xlings`) caches are saved only by the build job of + each host, and only on a push to `main` (`actions/cache/save` after the build). +- **Who reads.** Every other job, and every pull-request job, uses `actions/cache/restore`. A pull request + reads the `main` lineage, which GitHub scopes to every branch, and adds nothing to the store. +- **Saves per run.** From 34 to 42 down to at most three per host (sandbox, xlings and the e2e durations of + §2.4.5), each key with one writer, on `main` only. +- **The `target/` caches are removed.** F2 shows they do not shorten a build, and at up to 3.3 GB each they + are the main cause of F3. They return only if the build becomes incremental across checkouts. That needs + a measured cause (Part VII) and a fix that keeps the stale-object hazards already recorded in + `.agents/docs/2026-05-15-stdcompat-restat-e2e.md` and `cross-build-test.yml:416-434` out of the result. +- **Concurrency.** `cancel-in-progress` becomes `${{ github.event_name == 'pull_request' }}`. A push to + `main` runs to completion and writes its caches; a superseded pull-request run is still cancelled. +- **Wine.** The Wine packages come from one pinned archive, published once as a release asset of this + repository, instead of from the `apt` mirrors with a cache that eviction removes (F7). The archive's + sha256 is checked before installation. + +#### 2.4.5 Shards by measured duration (R4) + +- `run_all.sh` already reports each test's duration (`run_all.sh:452-462`). On a push to `main`, each shard + writes its durations under a key of its own, and the next pull request merges the shards of its host. +- A pull request assigns tests to shards greedily by duration, longest first. This is the plan recorded + in `.agents/docs/todos/2026-06-24-e2e-suite-sharding.md`, of which only round-robin was built. +- A test without a recorded duration counts as the host's median. +- Assignment changes only balance, never which tests run, so a stale timing file cannot hide a test. + +Once §2.4.1 lands, a shard no longer pays a build, so more shards cost only their setup. The proposed +counts: + +| host | shards | each shard | +|---|---|---| +| Linux | 3 | about 10 minutes | +| Windows | 3 | about 12 minutes, the long tests spread first | +| macOS | 2 | about 10 minutes | + +Each shard's step limit is twice its budgeted duration, so a limit is reached only by a hang (R4). The +25-minute limits that F5 measured being reached become 20 to 24 minutes on shards half as long. + +The 15 slow Windows tests are a finding in their own right. A test that takes one second on Linux and +up to 104 seconds on Windows measures something about Windows, and Part VII lists it for investigation. +It is not a reason to remove them from the suite. + +#### 2.4.6 No test is silently absent (R5) + +- `run_all.sh` grants `llvm` when the sandbox holds an LLVM payload with `clang++`, as it already does for + `scan-deps` and `import-std-libcxx` (`run_all.sh:221-230`). The Linux e2e shards install the LLVM + payload beside GCC. +- Every shard writes the list of tests it ran and skipped, with the reason for each skip, as a step + summary and as a small artifact. +- A final `e2e-coverage` job collects the lists of every host. It fails if a test ran on no host and is not + listed in `tests/e2e/never-in-ci.tsv`, which states for each such test why no hosted runner can run it. +- The seven tests of F9 must either run or be listed. On Linux, 134, 135, 136, 137, 741, 875 and 876 are + expected to run once `llvm` is granted. + +The run_all.sh comment at `:290-305` records why a hard-required capability was rejected: a token cannot +tell a misconfigured runner from a platform that lacks the capability. The coverage job answers the +question that the token could not, because it knows every host's list. + +#### 2.4.7 macOS slots (R6, R7) + +- The `xcode-27` legs, which are known red, run on pushes to `main` and on dispatch, and on a pull request + only when the pull request carries the label `macos-27`. A pull request that addresses #669 sets the + label. +- The macOS legs of the target matrix and of openkal consume the macOS artifact. macOS builds of mcpp per + commit fall from eight to one. +- The number of macOS jobs per pull-request commit changes less than their minutes do: from 9 or 10 to + about 8. + - The build. + - Two e2e shards. + - `ci-macos` integration, which keeps its fresh sandbox (`setup-macos-llvm/action.yml:7-9` states that + a fresh sandbox is what it proves). + - iOS. + - The target matrix as one job. + - openkal's build and run legs. +- What falls is the time each macOS job holds a slot: no job but the build compiles mcpp, and the two + known-red legs leave pull requests. + +### 2.5 An external lane under `speak-agent` (optional) + +#### 2.5.1 What it would buy + +The 20-job and 5-macOS limits apply per account or organisation, not per repository. `mcpp-community` is +on the Free plan. A repository under `speak-agent`, a machine account, runs against a second pool of +the same size. One repository is enough, because more repositories under one account add no capacity. +The lane can therefore raise the parallelism of a pull request. It cannot shorten a job. + +#### 2.5.2 Constraints + +- **Terms of Service.** GitHub allows one free account per person, plus a machine account that is used + only for automated tasks; `speak-agent` is one. The Actions terms forbid using hosted runners for + activity unrelated to the testing of the software project associated with the repository in which the + Actions run. A lane repository whose stated purpose is to test `mcpp-community/mcpp`, which runs only + mcpp's tests at mcpp's commits, satisfies the wording. Spreading one project's CI over two accounts to + exceed one plan's concurrency is not addressed by the terms, and that is a risk this record cannot + resolve. The lane is therefore optional. It has an off switch, and the repository's own CI stays + complete without it. +- **Forks.** A `pull_request` run from a fork receives no secrets, so it cannot dispatch the lane. Fork + pull requests run every leg inside the repository. +- **Authority.** A push to `main` always runs every leg inside the repository. The lane serves pull + requests only. + +#### 2.5.3 Shape + +- **Repository.** `speak-agent/mcpp-ci-lane`. It is separate from the validation labs, whose `main` + holds only rules and a guard; the lane's workflows live on its `main`, because they are dispatched there. +- **Dispatch.** A job in mcpp's CI (seconds, no slot held while the lane runs) calls the lane's + `workflow_dispatch` with the commit, the pull request number and the legs. The token is a fine-grained + token of `speak-agent` with Actions write on the lane repository only, stored as an mcpp secret. +- **Lane run.** It checks out `mcpp-community/mcpp` at that commit (public, no token). It builds once per + host as in §2.4.1, and runs the legs as consumers. +- **Test jobs.** They have `permissions: {}` and no secret. +- **Report.** A last job, which runs no code from the commit, writes one commit status per leg to the mcpp + commit (`lane/macos-e2e-1` and so on), linking to the lane run. Its token is a second fine-grained token + with commit-status write on `mcpp-community/mcpp` only. That job reads its inputs from + `needs..result` only. +- **In mcpp.** When the lane is on (a repository variable), the legs it carries are skipped on same-repo + pull requests and run as usual everywhere else. + +#### 2.5.4 Costs + +- Two places to read logs. +- Two secrets to rotate. +- A cold cache lineage in the lane until its own runs warm it. +- Contention with the validation labs, which use the same `speak-agent` pool. + +#### 2.5.5 When to build it + +Only if, after D1 to D7 have landed, the median first-attempt wall clock of five consecutive code pull +requests exceeds 30 minutes, and queueing accounts for more than a fifth of it. The first legs to move +are then the macOS and Windows e2e shards, because macOS slots are the scarcest and Windows shards the +longest. + +### 2.6 Expected effect and how it is checked + +An estimate, from the medians of §2.1, to be replaced by measurement: + +| | today (median of the 4 pull requests) | after D1 to D7 | +|---|---|---| +| from-source builds of mcpp | 42 | about 16 | +| runner minutes | 489 | about 330 | +| first-attempt wall clock | 39.6 min | 25 to 28 min, of which about 10 is the build stage | +| macOS jobs, and macOS builds of mcpp | 9 to 10, and 7 | about 8, and 1 | +| caches saved | 34 to 42 | none on a pull request; on `main`, the sandbox and xlings caches of each host and one durations key per shard | +| a documentation-only commit | 35 jobs, 436 runner minutes | 1 to 3 jobs, under 15 runner minutes | + +The work is accepted when five consecutive code pull requests after it show all of the following. The +measurement uses the same scripts as §2.1. +- A median first-attempt wall clock of at most 28 minutes. +- At most 16 from-source builds per commit. +- No e2e step reaching its limit. +- A green `e2e-coverage` job. +- Cache usage below 8 GB. +- A documentation-only pull request finishing under 5 minutes. + +## Part III. Defects repaired by the next release + +### 3.1 #757: the engine's identity is part of a build record + +1. `BuildCacheEntry` records the engine that wrote the graph: its version (`MCPP_VERSION`) and the + canonical path of its executable (`self_exe_path()`, the same function the emitter uses). An entry + without them declines once: "the recorded build predates the engine identity". +2. **One admission predicate.** The gates common to the three fast paths are one function: the engine + identity, the record's version fields, the toolchain request, the runtime binding, the graph's + existence, mode and request tag, and the freshness sweeps. `try_fast_build`, + `try_fast_workspace_build` and `try_fast_run` call it and add only their own gates (run targets, a + runner, the run tier, the selection). A field added later is then checked in one place, which is the + structural repair of the shape §1.3 describes. +3. Criteria: an e2e test builds with an engine copied to one path, deletes it, and builds again with the + same binary at another path; the second build regenerates the graph and succeeds, and no command in + the new graph names the old path. A unit test of the predicate covers a recorded version that differs + from the running one, and an entry that lacks the fields. + +### 3.2 #756: each path dependency is classified by its own package + +1. When the plan records the path-dependency roots (`src/build/prepare/plan.cpp:139-177`), it records with + each root that package's own `module_extensions` and `device_extensions`, from its effective manifest. +2. `dep_sources_newer_than` classifies the files under each root with that root's table. The record's + block is count-prefixed like `depSourceRoots=`, and an entry without the tables declines once. +3. Criteria: an e2e test with a provider that declares `.ixx` and a consumer that declares nothing. + Editing the provider's host module re-runs the consumer's build program, the program's output + changes, and no dead-entry warning is printed. + +## Part IV. The mcpp toolchain specification (SPEC-009, as approved) + +This part is normative. It is to be published as SPEC-009 in `docs/specs/`, in that directory's +language and format, with an implementation status on each rule. SPEC-006 continues to define what a +toolchain is, how one is named and selected, and what a payload must contain; SPEC-009 defines how the +set of supported toolchains changes over time. Terms follow RFC 2119. + +### TS-1 Terms + +| Term | Meaning | +|---|---| +| Family | `gcc`, `llvm`, `msvc`, `emsdk`, `android-ndk` (SPEC-006 §2.1) | +| Line | a family and a major version: GCC 16, LLVM 23 | +| Release | one upstream version of a line: 16.2.0, 23.1.3 | +| Payload | an installable tree for a release, a host and a variant: glibc `gcc`, native `musl-gcc`, cross `-linux-musl-gcc`, `mingw-gcc`, `mingw-cross-gcc`, `llvm` | +| Row | one row of the target matrix (`kKnownTargets`), or a host default | +| Default | the release a row resolves when nothing is declared | +| Line table | the one table in the engine that states every row's default (TS-3) | +| Self-host toolchain | the toolchains mcpp's own `mcpp.toml` declares | + +### TS-2 Support tiers + +| Tier | Which releases | Obligation | +|---|---|---| +| Default | the release the line table names for a row | the full e2e suite on that row's CI host; the build of mcpp itself where that host builds mcpp; release artefacts are built with it | +| Supported | the previous Default of each row, and releases the line table lists explicitly | the acceptance programs (SPEC-006 §6.2) in CI whenever the line table or one of its payloads changes; a regression is a defect | +| Available | every other release in the index | no CI. Reports are accepted. A known defect is stated as a capability of the payload (`.mcpp-toolchain.json`), never as a version branch in the engine | + +Each family's floor, the oldest line whose `import std` mcpp accepts, MUST be stated in the line +table. Today the floor is implicit. + +### TS-3 One line table + +1. Every default and every row pin MUST be read from one table in the engine. For each row it states the + family, the release, the variant, the tier, and, for a row whose release differs from its family's + Default, the reason and the condition under which the difference ends. +2. Help text, install suggestions, error messages and `mcpp self env --format json` MUST be formatted + from the table. A second literal of a default inside the engine is a defect. +3. Everything outside the engine that names a default (documentation, workflows, tests, examples and + mcpp's own manifest) MUST either read it from `mcpp self env --format json` or be compared with it by + a CI check. A literal that is neither read nor checked is a defect. +4. The index's `latest` is not a default. mcpp pins exact releases, and moving `latest` changes nothing + mcpp builds. + +### TS-4 A line moves as a whole + +1. On one host, the rows of one family SHOULD resolve one release. A row MAY lag only with a reason in + the line table, and the reason MUST be revisited at the next move. +2. A release enters the Default of a row only when every payload that row needs exists at that release, + on both mirrors (TS-9). The payloads of one GCC line (glibc, native musl, cross musl, mingw) are one + line for this purpose. + +### TS-5 Provenance + +1. A payload MUST derive from an upstream release: an official tag and its official source or binary + archive, whose sha256 the payload's description records (SPEC-006 §4.5). +2. A payload MUST be produced by a recipe in a repository, run by a CI job or by a script a CI job could + run. A manual step is not part of a recipe (SPEC-006 §5.1). +3. A patched payload, an upstream release plus upstream commits that no release carries yet, MAY be + published only when all of the following hold: + - every patch is a commit on the upstream release branch, or on its main branch with an open + backport, cited by hash; + - it is published as a revision of that release (`revision = N`, assets under `-r`), never + under the official asset name, and its description lists the patches; + - it is placed on the rows that need it and on no other; + - it has an exit: when an upstream release carries the patches, the Default moves to that release, + and the revision remains in the index only for those who pinned it. +4. The engine MUST NOT work around a toolchain defect by changing the platform's inputs (selecting an + older SDK, rewriting SDK files, substituting another linker) unless that is the row's declared design. + A toolchain defect is repaired by a toolchain release, or by a patched payload under rule 3. + +### TS-6 What it means that mcpp supports a line + +1. The engine MUST derive what a toolchain can do from the payload, not from its version: the standard + library module from the library's manifest or layout, the scanner from the driver, flags from probes + measured on the row. A comparison of version numbers is permitted only for a defect registered under + TS-7, or in a language-feature table whose rows cite the release notes. +2. On every row where a line is Default, the engine MUST pass the acceptance programs (a program using + ``, `` and ``; `import std`; `import std.compat`), the e2e suite, and, where the + host builds mcpp, the build of mcpp itself. +3. Every output that encodes a release MUST be reviewed at a move: the pack ABI tag (compiler major), + BMI and cache identity (the version enters the fingerprint and every cache key, so a move needs no + epoch change), and diagnostics that quote a version. + +### TS-7 The compiler-defect register + +1. Every engine behaviour, and every shape of mcpp's own sources, that exists because of a compiler + defect MUST have an entry: family, first release observed, last release verified, upstream report, + a minimal reproduction under `tests/` that fails while the defect exists, and the sites that depend + on it. +2. At every move of a line, every reproduction runs against the new release, and the entry records the + result. +3. A workaround is removed only when every Supported release is past the fix. + +Entries known today, from source comments and earlier records: an instantiation dropped along a module +import chain (GCC 16.1); a `FILE`-typed entity in a module interface breaking a later +`#include ` (GCC PR 99000, open); a nested `std::map` member truncating a BMI (GCC 16.1); a new, +widely imported module with standard types in its interface poisoning downstream BMIs (GCC 16.1); a +segmentation fault on a new interface unit (GCC 16.1, `src/build/prepare.cppm:37-50`); a miscompile with +a full BMI, which is why clang uses the two-phase reduced interface (clang 22.1.8, +`src/build/ninja_backend.cppm:1331`); a crash on an inline helper in a module purview (clang 20.1.7 on +Windows); the 27.0 SDK leaving `INFINITY` and `NAN` to `` under modules +(`src/toolchain/hostflags.cppm:162-181`); the missing `directory_iterator` comparison in libc++ 20's +`std`; an ICE on `AMDGPUAsmParser.cpp` (GCC 16.1, the reason `llvm-dev` is built with GCC 15.1). + +### TS-8 Host platform releases + +1. A new operating system or SDK release that a row's users will meet gets a CI leg as soon as a hosted + image exists, before that image becomes the runner default. +2. A leg that is red for an external cause carries `known_red: '#'`, the workflow assertion keeps + the issue open, and the leg leaves the list when the issue closes. +3. The repair follows TS-5. + +### TS-9 Mirrors + +A version row MUST NOT reach the index before every asset it names is present on the GLOBAL and CN +mirrors and verified on both by a GET (status 200, byte size and sha256). An upload of more than about +8 MiB to the CN mirror is made from a host inside the CN network. + +### TS-10 Moving a Default + +1. The upstream release exists. +2. Its payloads are built by their recipes (TS-5) and pass the admission of SPEC-006 §6; each records its + inputs. +3. The assets are on both mirrors and verified (TS-9). +4. The index gains the version rows; `latest` is unchanged. +5. The engine is verified on every row that will take the release as Default, and the release passes + the gate below. The payloads MAY be named by path (SPEC-006 §2.2.1), so that this step does not wait + for step 4. +6. One mcpp pull request moves the line table and every reader and checked copy of it; mcpp is + released. +7. The index's `latest` moves after that release has shipped. +8. The tiers shift: the previous Default becomes Supported, and the line before it becomes Available. + +A move is reverted by reverting step 6. Payloads and index rows stay. + +**TS-10.5 The gate: a new release does not make the module experience worse.** A candidate release R is +compared with the row's current Default D on the same mcpp commit, on the same runner image, in the +same job, so that the comparison spans only the change of release. R passes when all six hold: + +| Gate | Criterion | +|---|---| +| G1 | every e2e test that passes with D passes with R, and no test that runs with D is skipped with R | +| G2 | where the row builds mcpp, mcpp builds itself with R, and that binary passes the suite | +| G3 | the acceptance programs of TS-6.2 build and run | +| G4 | every TS-7 reproduction is run, and none of G1 to G3 needs a new workaround or a reshaped source; a defect that does is a failure of the gate unless the review accepts it with a register entry | +| G5 | the scanned module graph (each unit's provided and required modules) of mcpp's own sources and of the e2e module fixtures is identical under R and D | +| G6 | the cold and the warm build of mcpp and of the `bench/` projects take no more than 10 percent longer with R, as the median of three runs, and the BMIs are no more than 10 percent larger | + +A release that fails the gate stays Available. The next release of the same line becomes the +candidate. + +### TS-11 Machines that already have a default + +1. A release that a user or a project states (`[toolchain]`, `--toolchain`, `MCPP_TOOLCHAIN`, + `mcpp toolchain default`) MUST NOT move. +2. A default that mcpp wrote on a first run is not a statement by the user, and its record MUST say so. + - On a home with no recorded default, a first run installs and records the line table's answer, as + it does today, and states nothing more. + - The notice exists for one case only: the home holds a default recorded by an earlier mcpp's first + run, and the running mcpp's line table names a newer release for that row. + - In that case mcpp MUST state once the recorded and the newer release, together with the two + commands that settle the question: `mcpp toolchain default ` moves the record, and + `mcpp toolchain default --keep` keeps it. Either command turns the record into a statement by the + user. + - The notice does not block, does not prompt, and is not repeated. Until the user answers, the + recorded release stays in use. +3. Payloads already installed stay; removing them is the user's decision. + +### TS-12 mcpp builds with what it gives + +mcpp's own manifest MUST use the Default release of each row it is built on. A divergence is a line-table +entry with a reason and an exit. + +### TS-13 Conformance checks + +The rules a program can check MUST be checked in CI: + +| Check | Rule | +|---|---| +| C1 | no default literal in `src/` or `modules/` outside the line table (TS-3.2) | +| C2 | every documentation statement of a default, including the target-row tables of `docs/21` and the README, matches `mcpp self env --format json` (TS-3.3) | +| C3 | workflows, actions and tests take versions from the line table; a capability probe asks for a family, not a version (TS-3.3) | +| C4 | `mcpp.toml`'s `[toolchain]` equals the line table or a recorded divergence (TS-12) | +| C5 | a `known_red` leg names an open issue (TS-8; exists) | +| C6 | the TS-7 reproductions run on every Default row | +| C7 | the gate of TS-10.5 is one workflow that takes a row and a candidate payload, and the pull request of TS-10.6 cites its run | + +## Part V. The toolchain candidates, kept for the move that follows this work + +Nothing in this part is done in this work (T2). It states where each line would go and what each move +needs, so that a candidate that passes the gate (TS-10.5) can move without another design. + +### 5.1 The line table if both candidates pass + +| Row | Today | After | Payloads it needs | +|---|---|---|---| +| Linux x86_64 (host default) | `gcc@16.1.0` | `gcc@16.2.0` | `gcc` 16.2.0 | +| Linux, other architectures (host default) | `gcc@15.1.0-musl` | `gcc@16.2.0-musl` | `musl-gcc` 16.2.0 (native) | +| Windows without MSVC (host default) | `gcc@16.1.0` | `gcc@16.2.0` | `mingw-gcc` 16.2.0 | +| Windows with MSVC (host default) | `llvm@20.1.7` | `llvm@23.1.3` | `llvm` 23.1.3 windows | +| macOS (host default) | `llvm@20.1.7` | `llvm@23.1.3` | `llvm` 23.1.3 macosx-arm64 | +| `x86_64-linux-musl`, `aarch64-linux-musl`, `x86_64-windows-gnu` | `gcc@16.1.0` | `gcc@16.2.0` | `musl-gcc`, `aarch64-linux-musl-gcc`, `mingw-gcc`, `mingw-cross-gcc` 16.2.0 | +| `x86_64-windows-musl` | `llvm@22.1.8` | `llvm@23.1.3` | `llvm` 23.1.3 | +| bare-metal rows (12) and iOS rows (3) | `llvm@22.1.8` | `llvm@22.1.8`, lagging (T5) | picolibc sysroots carry compiler-rt builtins built with 22.1.8; iOS rows need `llvm.libcxx` and `llvm.compiler-rt-builtins` packages at 23 (`src/build/prepare/scan.cpp:631-632`) | +| mcpp's own `[toolchain]` | `gcc@16.1.0`, `macos = llvm@22.1.8`, `windows = llvm@20.1.7`, musl `gcc@16.1.0-musl` | the Default of each row | none beyond the above | + +Not moved: `android-ndk` and `emsdk` rows; the `bench/` pins, which are fixed on purpose so that a +benchmark does not report a toolchain change as an engine change; fixtures that pin a release to test a +specific behaviour. + +The move from 20.1.7 to 23.1.3 on macOS and Windows also removes finding 2 of §1.5: the macOS default +will be able to build mcpp. + +### 5.2 GCC 16.2 + +**Upstream.** GCC 16.2.0 was released on 2026-08-07, a bug-fix release with 672 commits since 16.1.0. +The commit range does not touch the p1689 output, the module mapper or the driver options mcpp uses. +It repairs nine module defects (among them 124953, failed to load pendings; 124981, an undefined +reference for an instantiation streamed from a BMI; 125768 and 126209, ICEs). libstdc++ adds the symbol +version `GLIBCXX_3.4.36`. + +**The risk that decides T6.** Bug 126577, a corrupt CMI when `-fmodule-mapper`, a re-exported partition, +a `chrono` alias and `unique_ptr` meet, is a regression of the 16 branch after 16.1.0, present in 16.2.0, +and retargeted to 16.3. mcpp passes `-fmodule-mapper` to GCC for every unit with a module scope +(`src/build/plan.cppm:3402`). The rule: if measurement M1 reproduces it, the GCC Default stays at +16.1.0 and 16.2.0 is Available, until a 16.3 release or a TS-5 revision carrying the fix. + +**Payloads.** + +| Payload | How it is produced this round | +|---|---| +| `gcc` 16.2.0 (glibc) | the `fromsource` recipe at 16.2.0, with the strip and the `specs` canonicalisation turned into a script (the musl builders' strip loop is the model), run by a builder workflow in `xlings-res/gcc` in the shape of `xlings-res/musl-gcc`'s; the description records the configure line, the source sha256 and the build's C library | +| `musl-gcc`, `aarch64-linux-musl-gcc` 16.2.0 | dispatch of the existing builder workflows with `gcc_ver=16.2.0` | +| `mingw-gcc` 16.2.0 | mirror of winlibs `16.2.0posix-14.0.0-ucrt-r2` | +| `mingw-cross-gcc` 16.2.0 | a builder workflow for the recipe that 16.1.0 followed by hand (GCC, binutils 2.44, mingw-w64 v12 CRT); if it is not ready, the `x86_64-windows-gnu` row lags under TS-4 | + +Out of this round: `riscv64-linux-musl-gcc`, the Windows-host `x86_64-linux-musl-gcc`, `gcc-runtime` +(only 15.1.0 exists), and rebuilding `llvm-dev` and `libllvm` with 16.2 (they are built with 15.1 +because of the 16.1 ICE in TS-7). + +### 5.3 LLVM 23 + +**Upstream.** 23.1.0, 23.1.1 and 23.1.2 were released on 2026-08-25, 09-08 and 09-22; 23.1.3 is +scheduled for 2026-10-06 and is expected to carry the `arm64e.x1` repair, which reached +`release/23.x` after the 23.1.2 tag. Release archive names and layouts are unchanged from 22.1.8; 23 +adds `.tar.zst` siblings and `.msi` installers. The Windows archive carries no +libc++ and no `std` module, as before, so `import std` on that row continues to come from the MSVC +STL. + +**What changes for a build tool**, from the 23 release notes and the sources: + +- P1857R3 changes how `module` and `import` directives are lexed during dependency discovery. The + scanner's output must be measured (M2, §5.5). +- `__has_feature(modules)` is no longer true with only `-std=c++20`. The SDK 27 `INFINITY`/`NAN` + workaround may therefore become unnecessary on 23; it stays harmless, because its values equal the + SDK's, and its TS-7 entry is re-measured. +- `export` inside a module implementation partition is now rejected. +- libc++ removed many transitive includes, and `std.cppm` now includes `` and + ``. +- The libc++ `[abi:...]` tag follows `_LIBCPP_VERSION` and changes with every patch release. The + fingerprint and the cache keys already contain the version. +- Reduced BMIs became the default in 22. `-fmodule-output` and `-fmodules-reduced-bmi` remain. + `GetStdModuleManifestPath` is unchanged between 22.1.8 and 23.1.2. + +**Payloads.** The carve script (`xim-pkgindex/.agents/tools/build-llvm-subpkg.sh`) applies unchanged to +the three official archives. The Linux carve still injects `libatomic` from a GCC on the build host +(23.1.2's `libc++.so.1` still names `libatomic.so.1`, and the archive carries none). That GCC should be +16.2 once its payload exists. `llvm-tools` and `mcpp-vscode-clangd` take the same version. The index's +parity test requires the new version on the linux, macosx and windows sections together. The existing +22.1.8 and 20.1.7 rows stay, because index tests name their assets and pinned users rely on them. + +### 5.4 The macOS 27 link + +The repair is the official 23.1.3 (TS-5.4, T3). Four days separate this record from its scheduled tag, +and the `xcode-27` legs are known red without blocking anything. Every LLVM row therefore moves to +23.1.3 when 23.1.3 passes the gate; the 23.1.2 measurement only shows early whether the line is likely +to pass. + +A patched payload is available under TS-5.3 if it is asked for: `llvm@23.1.2` revision 1 for the macOS +row. It would be the official 23.1.2 archive carved as usual, with `ld64.lld` and `llvm-otool` rebuilt +from `release/23.x` at a recorded commit containing `532fa5afbe2b` and `ee66426152f9`, published as +`23.1.2-r1` assets whose description names the commits. It is not part of the plan unless 23.1.3 is +delayed and macOS 27 users must be served before it. + +Rejected alternatives: selecting the Command Line Tools 26.5 SDK, rewriting the SDK's `.tbd` files, +and linking with Apple's `ld` instead of `ld64.lld`. Each changes the platform's inputs to hide a +toolchain defect (TS-5.4), and each would survive the repair as a second behaviour. + +When both `xcode-27` legs are green with the new release, #669 closes and the `known_red` entries leave +the workflows (TS-8.2). + +### 5.5 Measurements that start the move + +Each one decides something. They run when the first move begins, not in this work. + +| # | Measurement | Decides | +|---|---|---| +| M1 | mcpp built by itself, and its e2e suite, with GCC 16.2.0 on Linux x86_64; the 126577 reproducer | T6 | +| M2 | the same with the official LLVM 23.1.x on Linux x86_64, named by path: scanning, partitions, transitive includes | the engine changes of §5.3 | +| M3 | the same on Windows with LLVM 23.1.x and the MSVC STL | the Windows row | +| M4 | macOS 15 with 23.1.x; `xcode-27` with 23.1.3 | T3, #669 | +| M5 | every TS-7 reproduction against the candidates | the register's entries | +| M6 | a program referencing a `GLIBCXX_3.4.36` symbol, built with 16.2.0 and packed in each pack mode, run on a clean container | the runtime closure of the GCC row | +| M7 | a bare-metal row and an iOS row with the 23 compiler and their 22.1.8 companion packages | whether T5 can be shortened | + +## Part VI. Order of the work + +Each item is one pull request unless stated otherwise. CI changes need no mcpp release; the engine +repairs do. + +| # | Item | Contents | Depends on | +|---|---|---|---| +| 1 | SPEC-009 | Part IV in `docs/specs/`, in that directory's language and format, with an implementation status on each rule (most are "not implemented"); SPEC-006 §7 becomes a reference to it | none | +| 2 | Engine repairs | Part III; released as the next mcpp version | none | +| 3 | CI, low-risk rules | D4 (cache writers, `target/` removal, concurrency on `main`, the Wine archive), D6 (the `llvm` capability and the coverage job), D7 (known-red legs off pull requests), and the macOS step limit of T8 | none | +| 4 | CI, the build stage | D1, D2 and D3: `ci.yml`, the build matrix with packed artifacts, the reusable workflows, path classes | item 3, so that its measurements start from the new cache policy | +| 5 | CI, shards | D5: duration-based shards and their limits | item 4 | +| 6 | Measurement | §2.6, over five consecutive code pull requests | item 5 | +| 7 | External lane | §2.5, only if the rule of §2.5.5 holds | item 6 | + +Items 1, 2 and 3 run in parallel. + +The first application of SPEC-009 (Part V) is not in this list. It starts when LLVM 23.1.3 is tagged, under +a record of its own, and it begins with the measurements of §5.5 and the single line table (D10). + +## Part VII. Not yet measured + +- **Why a restored `target/` does not make a build incremental (F2).** Checkout modification times are the + likely cause, and are not verified. The answer decides whether `target/` caching can return, and in what + form. +- **Whether every consumer works with the packed artifact (§2.4.1).** The Windows hand-off proves it for + one job. The e2e suite on Linux and macOS with a packed `$MCPP` is the measurement, run once in a + validation pull request before item 4. +- **Why the target-matrix and openkal legs build with `--dev`.** Nothing records it. +- **The minimum macOS version of an mcpp built on `macos-15`.** It decides whether the `macos-14` legs can + consume it. +- **Why about 15 e2e tests take 39 to 104 seconds on Windows and about one second on Linux.** +- **The cost of the e2e-coverage job's lists**, and whether any host-specific skip is missing from them. +- **Toolchains.** + - Whether mcpp builds itself with GCC 16.2.0 and with LLVM 23 (§5.5, M1 to M3). + - The `libstdc++.modules.json` relative path in a 16.2.0 install. + - Whether the clang 23 driver changes a default that the generated `clang++.cfg` relies on. + - Whether the build-program cache and the host-module store key the engine version as well as the + compiler. If they do not, §3.1 extends to them. + +## Part VIII. Questions for review + +1. One top-level `ci.yml` with reusable workflows (recommended), or each platform workflow with a build job + of its own and artifacts shared only within it. +2. The artifact as the packed self-host build (recommended), or the raw binary next to a restored + sandbox. +3. Removing the `target/` caches now (recommended), or keeping them until F2's cause is measured. +4. The documentation classes of §2.4.3, with the second class derived by searching `tests/` and + `.github/tools/` for the changed path. +5. The shard counts of §2.4.5 (Linux 3, Windows 3, macOS 2), and step limits at twice the budget. +6. The known-red legs on pull requests only with the label `macos-27` (recommended). +7. The external lane: designed now and built under §2.5.5 (recommended), or built now. If it is built, the + repository name `speak-agent/mcpp-ci-lane`. +8. The single line table (TS-3) deferred to the first toolchain move (recommended), or built in item 2. diff --git a/.agents/docs/README.md b/.agents/docs/README.md index db02537d..f9ace8e8 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -323 records. +324 records. ## By subject @@ -30,6 +30,7 @@ Records that declare one. Everything else is listed by date below. ### design +- [PR CI acceleration and the toolchain specification (#756, #757, #669)](2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md) — active - [工具与工具链的来源：声明、编程决定、可观察](2026-10-01-tool-and-toolchain-sources-design.md) — landed - [A pack's build reported as a build, and a unit's compile independent of the member selection: triage and design (#753, #751)](2026-10-01-pack-drive-and-selection-independent-compile-design.md) — landed - [Member selection, build programs prepared once, a pack over several members, and the output streams of `mcpp run`: the plan for the release after 2026.9.30.2 (#748, #749, #750)](2026-09-30-member-selection-and-build-program-cost-plan.md) — landed @@ -114,6 +115,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-10 +- [PR CI acceleration and the toolchain specification (#756, #757, #669)](2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md) — active - [工具与工具链的来源：声明、编程决定、可观察](2026-10-01-tool-and-toolchain-sources-design.md) — landed - [A pack's build reported as a build, and a unit's compile independent of the member selection: triage and design (#753, #751)](2026-10-01-pack-drive-and-selection-independent-compile-design.md) — landed ### 2026-09 From f402012089b51a5aabb1560e9d6cb6bedd96fe7d Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 07:10:49 +0800 Subject: [PATCH 02/14] docs(specs): SPEC-009, the support and maintenance of toolchains Add SPEC-009 (docs/specs/toolchain-maintenance.md), the toolchain specification approved in review on 2026-10-02: support tiers, the single line table, whole-line moves, provenance, what supporting a line means, the compiler-defect register, host platform releases, mirrors, the order and the gate (G1 to G6, 10 percent) for moving a default, machines that already have a default, mcpp building with what it gives, and the conformance checks C1 to C7. Each rule carries an implementation status and, where the status is not "implemented", the current behaviour. SPEC-006 section 7 becomes a reference to SPEC-009 section 10 (v0.6); the order of moving a C-library binding is carried into SPEC-009 section 10.10. The specification index and both documentation indexes list SPEC-009. --- docs/README.md | 1 + docs/specs/README.md | 3 +- docs/specs/toolchain-maintenance.md | 386 ++++++++++++++++++++++++++++ docs/specs/toolchain-management.md | 10 +- docs/zh/README.md | 1 + 5 files changed, 396 insertions(+), 5 deletions(-) create mode 100644 docs/specs/toolchain-maintenance.md diff --git a/docs/README.md b/docs/README.md index 1ad37ccc..4eea17b3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -164,3 +164,4 @@ downstream tooling. - [SPEC-006 — Toolchain management: identity, origin, selection and the payload contract](specs/toolchain-management.md) - [SPEC-007 — Build plugins: configuration, construction and verification, and the runtime and planning obligations](specs/build-plugins.md) - [SPEC-008 — A library's interface: public modules, the published closure, and one interface in both forms](specs/library-interface.md) + - [SPEC-009 — The support and maintenance of toolchains: lines, defaults, provenance, moves and retirement](specs/toolchain-maintenance.md) diff --git a/docs/specs/README.md b/docs/specs/README.md index 0647d87e..7e5e77be 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -35,9 +35,10 @@ | [SPEC-003](exit-codes.md) | 退出码契约(分类、语义、稳定性承诺) | 评审中 v1.0 | 2026-09-01 | mcpp >= 2026.9.1.1 | | [SPEC-004](manifest-semantics.md) | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | 草案 v1.10 | 2026-09-28 | 条件化形状 mcpp >= 2026.8.29.1;目标轴 mcpp >= 2026.9.6.4;`linkage` 默认值 mcpp >= 2026.9.15.2;链接 flag 的词读法 mcpp >= 2026.9.26.2;条件化的 `dialect_cxxflags` 与 `-p` 的包身份 mcpp >= 2026.9.28.1;条件表按具体程度生效 mcpp >= 2026.9.28.2 | | [SPEC-005](build-database.md) | 构建数据库:`mcpp emit build-database` 的内容、取值规则与不写工程目录的保证 | 评审中 v1.6 | 2026-09-29 | mcpp >= 2026.9.15.1;v1.3 条款 mcpp >= 2026.9.26.2;v1.4 条款 mcpp >= 2026.9.27.1;v1.5 条款 mcpp >= 2026.9.28.1;v1.6 条款 mcpp >= 2026.9.29.5 | -| [SPEC-006](toolchain-management.md) | 工具链管理:身份、来源、选择与载荷契约 | 草案 v0.4 | 2026-09-28 | 逐条标注;已实现条款 mcpp >= 2026.9.24.1;§3.7 mcpp >= 2026.9.28.1;§3.7.1 mcpp >= 2026.9.28.2 | +| [SPEC-006](toolchain-management.md) | 工具链管理:身份、来源、选择与载荷契约 | 草案 v0.6 | 2026-10-02 | 逐条标注;已实现条款 mcpp >= 2026.9.24.1;§3.7 mcpp >= 2026.9.28.1;§3.7.1 mcpp >= 2026.9.28.2;§2.2.1 与 §3.3 的非缺省来源 mcpp >= 2026.10.1.3 | | [SPEC-007](build-plugins.md) | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | 草案 v0.6 | 2026-09-28 | 逐条标注;mcpp >= 2026.9.26.2;v0.3 条款 mcpp >= 2026.9.27.1;v0.4 条款 mcpp >= 2026.9.28.1;v0.5 条款 mcpp >= 2026.9.28.2;v0.6(§9)mcpp >= 2026.9.28.3 | | [SPEC-008](library-interface.md) | 库的接口:公开模块、发布闭包与两种形态的一致 | 草案 v0.1 | 2026-09-28 | 第一阶段(只警告)mcpp >= 2026.9.28.3 | +| [SPEC-009](toolchain-maintenance.md) | 工具链的支持与维护:版本线、默认值、来源、移动与退役 | 草案 v0.1 | 2026-10-02 | 逐条标注;本版只有规范,多数条款未实现 | ## 文档约定 diff --git a/docs/specs/toolchain-maintenance.md b/docs/specs/toolchain-maintenance.md new file mode 100644 index 00000000..f2ff6140 --- /dev/null +++ b/docs/specs/toolchain-maintenance.md @@ -0,0 +1,386 @@ +# SPEC-009:工具链的支持与维护 + +| 项 | 值 | +|---|---| +| 规范编号 | SPEC-009 | +| 标题 | 工具链的支持与维护:版本线、默认值、来源、移动与退役 | +| 状态 | 草案 v0.1 | +| 最后修改 | 2026-10-02 | +| 对应实现 | 逐条标注;本版只有规范,多数条款未实现 | +| 相关设计文档 | `.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md`(第 IV 部分) | +| 相关 issue | mcpp#669(macOS 27 的链接)、mcpp#685、mcpp#687、mcpp#755 | +| 使用文档 | [docs/20 - 工具链](../zh/20-toolchains.md) | + +本规范定义 mcpp 支持的工具链集合如何随时间变化:哪些版本被支持、默认值由哪一张表给出、载荷从哪里来、 +一个默认值如何移动,以及已经记录了默认值的机器如何得知新的默认值。 +[SPEC-006](toolchain-management.md) 定义工具链是什么、如何命名与选择、载荷必须包含什么,本规范不重复这些内容。 + +本规范不指定任何一个具体版本为默认值。具体版本由一张线表给出(§3);本版发布时线表尚不存在, +默认值仍由两张表分别给出(§3.1)。各节编号与设计过程中的条款编号 TS-n 一一对应。 + +用语按 RFC 2119:**必须 / 禁止**(强制)、**应当**(强烈建议)、**可以**(可选)。实现状态标记见 [规范索引](README.md)。 + +--- + +## 1. 术语 + +族与载荷沿用 [SPEC-006](toolchain-management.md) §1 与 §2.1 的定义。本节增加以下术语。 + +| 术语 | 含义 | +|---|---| +| 族(family) | `gcc`、`llvm`、`msvc`、`emsdk`、`android-ndk` | +| 线(line) | 一个族加一个主版本:GCC 16、LLVM 23 | +| 发布(release) | 一条线上的一个上游版本:16.2.0、23.1.3 | +| 载荷(payload) | 一个发布在某个宿主与变体上的可安装目录树:glibc 的 `gcc`、原生 `musl-gcc`、交叉的 `-linux-musl-gcc`、`mingw-gcc`、`mingw-cross-gcc`、`llvm` | +| 行(row) | 目标矩阵(`kKnownTargets`)的一行,或一个宿主默认值 | +| 默认(default) | 一行在没有任何声明时解析到的发布 | +| 线表(line table) | 引擎中唯一一张给出每一行默认的表(§3) | +| 自举工具链(self-host toolchain) | mcpp 自己的 `mcpp.toml` 所声明的工具链 | + +--- + +## 2. 支持分级 + +### 2.1 三个等级 未实现 + +发布按下列三个等级分级。 + +| 等级 | 范围 | 义务 | +|---|---|---| +| Default | 线表为一行所指名的发布 | 该行 CI 宿主上的完整 e2e 套件;该宿主构建 mcpp 时,mcpp 自身的构建;发布产物以它构建 | +| Supported | 每一行上一个 Default,以及线表明确列出的发布 | 线表或其载荷变化时,CI 运行验收程序(SPEC-006 §6.2);回归是缺陷 | +| Available | 索引中的其余所有发布 | 无 CI,接受报告。已知缺陷**必须**以载荷的能力(`.mcpp-toolchain.json`)陈述,**禁止**写成引擎里的版本分支 | + +当前:引擎不按发布分级。`kKnownTargets` 的 `tier`(`verified`、`preview`、`planned`)是目标行的验证级别,与本节的发布分级是两个概念。 + +### 2.2 下限 未实现 + +每个族的下限,即 mcpp 接受其 `import std` 的最老的线,**必须**写在线表中。 + +当前:下限是隐含的,没有一处陈述。 + +--- + +## 3. 一张线表 + +### 3.1 唯一的来源 未实现 + +每一个默认与每一个行的钉住版本,**必须**读自引擎中的同一张表。表为每一行写明族、发布、变体与等级; +行的发布不同于其族的 Default 时,**必须**另写明理由,以及差异在何种条件下终止。 + +当前:默认值由两张表分别给出,均在 `modules/toolchain-model/src/triple.cppm`:表 A 是 `pins::kFirstRun*`(附带重复的 `kSuggest*` 字面量), +表 B 是 `kKnownTargets[].pin`。两张表之间只有一处耦合被检查:`kFirstRunWinGnu` 与 `x86_64-windows-gnu` 行的一致 +(`tests/unit/test_windows_defaults.cpp`)。 + +### 3.2 引擎内的读者 未实现 + +帮助文本、安装建议、错误信息与 `mcpp self env --format json` **必须**由线表格式化。引擎内出现第二处默认值字面量是缺陷。 + +当前:`mcpp self env --format json` 的 `data.defaultToolchain` 取自 `pins::host_default_toolchain`(表 A)。安装建议使用 `kSuggest*` 字面量; +两张表之外,另有 21 处默认值字面量分布在九个源文件中。 + +### 3.3 引擎外的读者 部分实现 + +引擎之外凡写出默认值者(文档、工作流、测试、示例与 mcpp 自己的清单),**必须**读自 `mcpp self env --format json`, +或者由一项 CI 检查与它比对。既不读取也不比对的字面量是缺陷。 + +当前:只有宿主默认值在文档中的四条陈述被比对:`.github/tools/check_default_toolchain_docs.py` 在每个 CI 宿主上检查该宿主的行, +在 `docs/01`、`docs/20` 及其 `docs/zh/` 副本中各一条。其余读者既不读取也不比对:`mcpp.toml`、`tests/matrix/expected.tsv` 的 175 行、 +七个工作流、一个 action、六个 CI 工具、至少八个 e2e 脚本、示例,以及 33 个文档文件中的其余陈述。 + +### 3.4 索引的 latest 不是默认值 已实现 + +索引的 `latest` **禁止**作为默认值。mcpp 钉住精确的发布;移动 `latest` 不改变 mcpp 构建的任何结果。 + +--- + +## 4. 版本线整体移动 + +### 4.1 一个族在一台宿主上解析到一个发布 未实现 + +在一台宿主上,同一个族的各行**应当**解析到同一个发布。一行**可以**落后,但**必须**在线表中写明理由, +且理由**必须**在下一次移动时重新评估。 + +当前:不成立。macOS 与带 MSVC 的 Windows 默认 `llvm@20.1.7`,而 17 个目标行钉住 `llvm@22.1.8`。Linux 在 x86_64 之外的架构默认 +`gcc@15.1.0-musl`,其余 gcc 行为 16.1.0。这些落后都没有记录理由。 + +### 4.2 载荷齐备 未实现 + +一个发布进入某一行的 Default,**必须**以该行所需的每一个载荷在该发布上都存在为前提,且两个镜像上都有(§9)。 +同一条 GCC 线的载荷(glibc、原生 musl、交叉 musl、mingw)在此处视为同一条线。 + +当前:没有程序检查此条件。 + +--- + +## 5. 来源 + +### 5.1 上游发布 部分实现 + +载荷**必须**出自一个上游发布:官方标签,及其官方的源码或二进制归档;载荷的描述**必须**记录该归档的 sha256(SPEC-006 §4.5)。 + +当前:索引以 sha256 固定资产(SPEC-006 §4.6),但描述文件尚不记录上游源码的 sha256(SPEC-006 §4.5 未实现)。 + +### 5.2 配方 未实现 + +载荷**必须**由检入仓库的配方产出,由一个 CI 作业运行,或由 CI 作业可以运行的脚本运行。人工步骤不属于配方(SPEC-006 §5.1)。 + +当前:glibc 的 `gcc` 载荷由人工按 `fromsource` 配方构建,随后的 strip 与 `specs` 改写没有任何仓库中的脚本(16.1.0 如此)。 +`musl-gcc` 与 `aarch64-linux-musl-gcc` 在 `xlings-res` 中有可派发的构建工作流;`mingw-cross-gcc` 与 Windows 宿主上的 Canadian cross 是手工的; +`mingw-gcc` 转存 winlibs 的发布。 + +### 5.3 打补丁的载荷 部分实现 + +打补丁的载荷,即一个上游发布加上尚无发布携带的上游提交,**可以**发布,当且仅当以下条件全部成立: + +- 每一个补丁都是上游发布分支上的一个提交,或是主分支上带有未关闭的回移(backport)的提交,并以哈希引用; +- 它作为该发布的修订发布(`revision = N`,资产名为 `<发布>-r`),**禁止**使用官方资产名,其描述列出全部补丁; +- 它只放在需要它的行上,不放在其他行上; +- 它有退出条件:当某个上游发布携带了这些补丁,Default 移到该发布,修订只为已钉住它的使用者留在索引中。 + +当前:索引已有修订的先例(xim-pkgindex 的 glibc 2.44.3-r1,`revision` 字段与 `-rN` 资产名),机制存在;上述四项条件没有程序检查。 + +### 5.4 引擎不改动平台的输入 部分实现 + +引擎**禁止**以改变平台输入的方式绕开工具链缺陷,例如选择更旧的 SDK、改写 SDK 的文件、替换另一个链接器,除非那就是该行声明的设计。 +工具链缺陷由工具链的发布修复,或由 §5.3 的打补丁载荷修复。 + +当前:mcpp#669 没有以改动平台输入的方式处理:引擎没有为它选择更旧的 SDK、改写 SDK 的文件或替换链接器。全引擎范围的符合性没有核对,也没有检查。 + +--- + +## 6. 支持一条版本线的含义 + +### 6.1 能力取自载荷 部分实现 + +引擎**必须**从载荷推导工具链能做什么,而不是从版本:标准库模块取自库的清单或布局,扫描器取自驱动,flag 取自在该行上测得的探针。 +版本号的比较只在两种情形被允许:一项登记于 §7 的缺陷,或一张语言特性表,其每一行引用发布说明。 + +当前:标准库模块、扫描器与描述文件的字段按载荷取得(SPEC-006 §4.4、§4.5)。引擎中是否还有按版本号的分支,没有逐处核对;已知的一处是按 `cl.exe` 横幅的版本选择 std 模块的语言级别(`src/toolchain/msvc.cppm`)。 + +### 6.2 Default 行的验收 部分实现 + +在一个线是 Default 的每一行上,引擎**必须**通过验收程序(一个使用 ``、`` 与 `` 的程序;`import std`;`import std.compat`)与 e2e 套件; +该宿主构建 mcpp 时,还**必须**通过 mcpp 自身的构建。 + +当前:e2e 套件在各 CI 宿主上运行;验收程序的矩阵部分实现(SPEC-006 §6.2)。macOS 的 Default `llvm@20.1.7` 无法构建 mcpp 自身 +(其 libc++ 的 `std` 模块不暴露 `directory_iterator` 的比较),mcpp 的清单因此以 22.1.8 构建;该行的第三项验收不成立。 + +### 6.3 随发布编码的输出 部分实现 + +每一个编码了发布的输出,**必须**在移动时被评审:包的 ABI 标签(编译器主版本)、BMI 与缓存的身份(版本进入指纹与每一个缓存键,因此移动不需要更换纪元),以及引用版本的诊断。 + +当前:ABI 标签带编译器主版本(`src/pack/abi_tag.cppm`):GCC 16.1 到 16.2 保持 `gcc16-libstdcxx16`;LLVM 22 到 23 由 `clang22-libcxx22` +变为 `clang23-libcxx23`,带旧标签的预制产物被拒绝(`src/pack/prebuilt.cppm`)。版本已进入指纹(`modules/toolchain-model/src/fingerprint.cppm`)。评审本身没有清单,也没有检查。 + +--- + +## 7. 编译器缺陷登记 + +### 7.1 登记项 未实现 + +凡因一个编译器缺陷而存在的引擎行为,以及凡因此而采取的 mcpp 自身源码的形状,**必须**有一个登记项,写明:族、首次观察到的发布、最后验证过的发布、上游报告、 +`tests/` 下一个在缺陷存在时失败的最小复现,以及依赖它的位置。 + +当前:没有登记表。这些行为的原因散见于源码注释与早先的记录,不保证每一项都有最小复现。 + +### 7.2 移动时重跑 未实现 + +每一次移动一条线时,**必须**对新发布运行每一个复现,并在登记项中记录结果。 + +当前:没有登记表,也就没有可运行的集合。 + +### 7.3 移除条件 未实现 + +一个绕开缺陷的做法,**必须**在每一个 Supported 发布都越过修复之后才移除。 + +当前:没有此规则;各处绕行的移除时机由修改者判断。 + +### 7.4 已知条目 + +以下条目取自源码注释与早先的记录,待 §7.1 落地后写成正式的登记项。 + +| 缺陷 | 族与发布 | 位置 | +|---|---|---| +| 一个实例化沿模块导入链被丢弃 | GCC 16.1 | — | +| 模块接口中的 `FILE` 类型实体使其后的 `#include ` 失败 | GCC PR 99000(未关闭) | — | +| 嵌套的 `std::map` 成员使 BMI 被截断 | GCC 16.1 | — | +| 一个新的、被广泛导入且接口含标准类型的模块毒化下游的 BMI | GCC 16.1 | — | +| 新接口单元上的段错误 | GCC 16.1 | `src/build/prepare.cppm` | +| 完整 BMI 下的误编译,因此 clang 使用两阶段的精简接口 | clang 22.1.8 | `src/build/ninja_backend.cppm` | +| 模块 purview 中的内联辅助函数导致崩溃 | clang 20.1.7(Windows) | — | +| 27.0 SDK 在模块下把 `INFINITY` 与 `NAN` 留给 `` | clang 22 与 macOS 27.0 SDK | `src/toolchain/hostflags.cppm` | +| libc++ 20 的 `std` 缺少 `directory_iterator` 的比较 | libc++ 20 | `mcpp.toml` 的 `[toolchain]` | +| `AMDGPUAsmParser.cpp` 的内部编译器错误,因此 `llvm-dev` 以 GCC 15.1.0 构建 | GCC 16.1 | — | + +--- + +## 8. 宿主平台的新版本 + +### 8.1 新版本的 CI 腿 部分实现 + +一个行的使用者将会遇到的新操作系统或新 SDK 发布,**必须**在存在托管的 runner 镜像时尽早得到一条 CI 腿,早于该镜像成为 runner 的默认。 + +当前:macOS 27 在 `xcode-27` 镜像上已有 CI 腿,因 mcpp#669 为已知红色。没有一般性的清单规定哪些宿主版本须有腿。 + +### 8.2 已知红色的腿 已实现 + +因外部原因而红色的腿**必须**带 `known_red: '#'`,工作流断言(`.github/tools/check_workflow_assertions.py`)**必须**保证该 issue 未关闭, +issue 关闭时该腿**必须**离开已知红色的列表。 + +### 8.3 修复 部分实现 + +宿主平台新版本引出的工具链缺陷,其修复遵循 §5.3 与 §5.4。 + +当前:状态同 §5.3 与 §5.4。mcpp#669 尚未修复,两条 `xcode-27` 腿保持已知红色。 + +--- + +## 9. 镜像 + +### 9.1 先镜像,后入索引 未实现 + +一个版本行**禁止**在它所指名的每一个资产都出现在 GLOBAL 与 CN 两个镜像上、并在两者上经 GET 验证(状态码 200、字节数与 sha256)之前进入索引。 + +当前:没有程序检查镜像的存在与哈希,顺序由发布者保证。 + +### 9.2 CN 镜像的上传 未实现 + +向 CN 镜像上传大于约 8 MiB 的文件,**必须**从 CN 网络内的主机进行。 + +当前:没有检查;该条是一项操作约束。 + +--- + +## 10. 移动默认版本 未实现 + +移动一行的 Default **必须**按以下顺序进行。当前:没有程序串联这些步骤;没有线表,步骤 10.6 须同时修改两张表及其全部副本(§3);步骤 10.5 见该节。 + +### 10.1 上游发布 + +该上游发布已经存在。 + +### 10.2 载荷 + +其载荷由各自的配方构建(§5),通过 SPEC-006 §6 的准入,并各自记录其输入。 + +### 10.3 镜像 + +资产已在两个镜像上并通过验证(§9)。 + +### 10.4 索引 + +索引增加该版本的行;`latest` 不变。 + +### 10.5 门:新发布不使模块体验变差 + +引擎在每一个将以该发布为 Default 的行上得到验证,且该发布通过下列的门。载荷**可以**按路径命名(SPEC-006 §2.2.1),因此本步不必等待 10.4。 + +候选发布 R 与该行当前的 Default D 在同一个 mcpp 提交、同一个 runner 镜像、同一个作业中比较,使比较只跨越发布的变化。 +R 通过,当且仅当下列六项全部成立。 + +| 门 | 判据 | +|---|---| +| G1 | 凡以 D 通过的 e2e 测试,以 R 也通过;凡以 D 运行的测试,以 R 不被跳过 | +| G2 | 该行构建 mcpp 时,mcpp 以 R 构建自身,且该二进制通过套件 | +| G3 | §6.2 的验收程序能够构建并运行 | +| G4 | 运行 §7 的每一个复现,且 G1 至 G3 不需要新的绕行或重塑的源码;需要者使门失败,除非评审接受并附登记项 | +| G5 | mcpp 自身源码与 e2e 模块夹具所扫描出的模块图(每个单元提供与需要的模块),在 R 与 D 下相同 | +| G6 | mcpp 与 `bench/` 工程的冷构建和暖构建,以 R 所用时间不超过 D 的 110%(三次运行的中位数),BMI 的体积不超过 110% | + +未通过门的发布留在 Available。同一条线的下一个发布成为候选。 + +当前:没有门的工作流,G1 至 G6 均未实现;CI 不在同一作业中比较两个发布。 + +### 10.6 引擎 + +一个 mcpp 拉取请求移动线表,以及线表的每一个读者与每一个被检查的副本;随后发布 mcpp。 + +### 10.7 latest + +在该 mcpp 发布之后,索引的 `latest` 移动。 + +### 10.8 等级移位 + +此前的 Default 成为 Supported,再之前的线成为 Available。 + +### 10.9 撤销 + +移动通过回退 10.6 撤销。载荷与索引的行保留。 + +### 10.10 C 库绑定 + +本条承接 SPEC-006 §7 原有的规则:移动 C 库绑定,或某个载荷的 `latest`,之前,SPEC-006 §6.2 的矩阵**必须**在新版本上全部通过。 +状态:未实现(§6.2 的矩阵部分实现,此顺序没有检查)。 + +--- + +## 11. 已有默认值的机器 + +### 11.1 声明的发布不移动 已实现 + +使用者或项目所声明的发布(`[toolchain]`、`--toolchain`、`MCPP_TOOLCHAIN`、`mcpp toolchain default`)**禁止**移动。 + +当前:已记录的默认值只由首次运行与 `mcpp toolchain default` 写入,没有路径因线表的变化而改写它。 + +### 11.2 首次运行记录的默认值 未实现 + +mcpp 在首次运行时写下的默认值不是使用者的声明,其记录**必须**如实表明这一点。 + +- 在没有已记录默认值的主目录上,首次运行安装并记录线表的答案,一如今日,此外不说任何话。 +- 通知只为一种情形存在:主目录里是由更早的 mcpp 的首次运行记录的默认值,而正在运行的 mcpp 的线表为该行指名了更新的发布。 +- 此时 mcpp **必须**陈述一次:已记录的发布、更新的发布,以及解决这个问题的两条命令。`mcpp toolchain default <更新的发布>` 移动记录; + `mcpp toolchain default --keep` 保留记录。任何一条命令都使该记录成为使用者的声明。 +- 通知不阻塞,不提示输入,不重复。使用者作答之前,已记录的发布继续使用。 + +当前:`src/build/prepare/toolchain.cpp:1607` 在首次运行时把默认值写入 `config.toml` 的 `[toolchain] default`(Windows 的 MinGW 回退另有两处写入), +此后没有任何路径按线表重新评估它,因此移动后的默认值只到达全新的主目录。该键与 `mcpp toolchain default ` 所写的是同一个键,记录无法区分首次运行与使用者的声明; +`mcpp toolchain default --keep` 不存在。 + +### 11.3 已安装的载荷 已实现 + +已安装的载荷保留;移除它们是使用者的决定。 + +当前:移除载荷的入口只有 `mcpp toolchain remove`。 + +--- + +## 12. mcpp 用自己给出的工具链构建 + +### 12.1 清单取线表 部分实现 + +mcpp 自己的清单**必须**使用其构建所在的每一行的 Default 发布。偏离是线表中的一项,带理由与退出条件。 + +当前:`mcpp.toml` 的 `[toolchain]` 为 `default = "gcc@16.1.0"`、`macos = "llvm@22.1.8"`、`windows = "llvm@20.1.7"`。`default` 等于 Linux x86_64 的默认, +`windows` 等于带 MSVC 的 Windows 的默认;`macos` 偏离该行的默认 `llvm@20.1.7`,原因见 §6.2,但没有线表项记录理由与退出条件。 + +--- + +## 13. 一致性检查 + +程序能够检查的规则**必须**在 CI 中被检查。 + +| 检查 | 内容 | 条款 | 状态 | +|---|---|---|---| +| C1 | `src/` 与 `modules/` 中,线表之外没有默认值字面量 | §3.2 | 未实现 | +| C2 | 文档中每一处对默认值的陈述,包括 `docs/21` 的目标行表与 README,与 `mcpp self env --format json` 一致 | §3.3 | 部分实现 | +| C3 | 工作流、action 与测试的版本取自线表;能力探针询问族,不询问版本 | §3.3 | 未实现 | +| C4 | `mcpp.toml` 的 `[toolchain]` 等于线表或一项已记录的偏离 | §12 | 未实现 | +| C5 | 一条 `known_red` 的腿指名一个未关闭的 issue | §8.2 | 已实现 | +| C6 | §7 的复现在每一个 Default 行上运行 | §7 | 未实现 | +| C7 | §10.5 的门是一个以行与候选载荷为输入的工作流,§10.6 的拉取请求引用它的运行 | §10 | 未实现 | + +当前: + +- C2:`check_default_toolchain_docs.py` 只比对宿主默认值的四条陈述(§3.3);`docs/21` 的目标行表与 README 没有覆盖。 +- C3:`tests/e2e/run_all.sh` 仅当 `musl-gcc/15.1.0` 已安装时授予 `musl`,仅当 `mingw-cross-gcc/16.1.0` 已安装时授予 `mingw-cross`。 + 默认值移动后,十六个测试被跳过且不报告。 +- C5:`check_workflow_assertions.py` 要求允许失败的作业在名字中指名一个 issue,并在检查 issue 状态时报告已不再打开的 issue。 + +--- + +## 变更记录 + +| 版本 | 日期 | 变更 | +|---|---|---| +| v0.1 | 2026-10-02 | 初版 | diff --git a/docs/specs/toolchain-management.md b/docs/specs/toolchain-management.md index 6c90275a..c2a3bd4e 100644 --- a/docs/specs/toolchain-management.md +++ b/docs/specs/toolchain-management.md @@ -4,10 +4,10 @@ |---|---| | 规范编号 | SPEC-006 | | 标题 | 工具链管理:身份、来源、选择与载荷契约 | -| 状态 | 草案 v0.5 | -| 最后修改 | 2026-10-01 | +| 状态 | 草案 v0.6 | +| 最后修改 | 2026-10-02 | | 对应实现 | 逐条标注;标为「已实现」的条款对应 mcpp >= 2026.9.24.1。标为「未实现」的条款计划与下一批 LLVM 工具链一同落地,届时按实测修订本规范 | -| 相关设计文档 | `.agents/docs/2026-09-24-toolchain-selection-and-payload-trust-design.md`、`.agents/docs/2026-09-24-685-687-msvc-stl-and-toolchain-payloads.md`、`.agents/docs/2026-09-28-ecosystem-design-and-optimisation-plan.md` | +| 相关设计文档 | `.agents/docs/2026-09-24-toolchain-selection-and-payload-trust-design.md`、`.agents/docs/2026-09-24-685-687-msvc-stl-and-toolchain-payloads.md`、`.agents/docs/2026-09-28-ecosystem-design-and-optimisation-plan.md`、`.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md` | | 相关 issue | mcpp#685、mcpp#687、mcpp#718、mcpp#755 | | 使用文档 | [docs/20 - 工具链](../zh/20-toolchains.md)、[docs/32 - 编写载荷](../zh/32-authoring-a-payload.md)、[docs/91 - 工具链内部](../zh/91-toolchain-internals.md) | @@ -304,7 +304,8 @@ xim-pkgindex 的准入脚本 `verify-toolchain.sh` 对一个载荷归档做一 ## 7. 发布顺序 未实现 -移动 C 库绑定或某个载荷的 `latest` 之前,§6.2 的矩阵**必须**在新版本上全部通过。 +移动一个默认版本或一个 C 库绑定的顺序,由 [SPEC-009](toolchain-maintenance.md) §10 规定;§6.2 的矩阵是该顺序中的一道要求。 +本节不再自有条款,实现状态同 SPEC-009 §10:未实现。 --- @@ -317,3 +318,4 @@ xim-pkgindex 的准入脚本 `verify-toolchain.sh` 对一个载荷归档做一 | v0.3 | 2026-09-28 | 随 mcpp 2026.9.28.1:新增 §3.7,MSVC ABI 的 CRT 模型是目标 ABI 的性质,cl 与 clang++ 同样收到,默认 `toolchain-coupled`(mcpp#718)。 | | v0.5 | 2026-10-01 | 随 mcpp 2026.10.1.3(mcpp#755):新增 §2.2.1,工具链可由路径命名,并说明 `bootstrap` 与 `configure = "build.mcpp"`;§3.3 增加非缺省来源的陈述、汇总、记录与 `--managed-only`。 | | v0.4 | 2026-09-28 | 随 mcpp 2026.9.28.2:新增 §3.7.1,程序旁的文件由一个解析器决定;MSVC C++ 运行时是一个带版本的集合;契约决定种类;声明的运行时文件与 toolset 的版本比较;读不出的版本不作决定;action 的 `PATH` 首位是 toolset 的运行时目录(2026-09-28 设计 WS1)。 | +| v0.6 | 2026-10-02 | §7 移入 SPEC-009(工具链的支持与维护):移动默认版本或 C 库绑定的顺序、门与撤销由 SPEC-009 §10 规定,本节改为引用。 | diff --git a/docs/zh/README.md b/docs/zh/README.md index 8fac9f6b..9af4ddca 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -159,3 +159,4 @@ - [SPEC-006 —— 工具链管理：身份、来源、选择与载荷契约](../specs/toolchain-management.md) - [SPEC-007 —— 构建插件：配置、施工与校验的分工，运行时与规划期的义务](../specs/build-plugins.md) - [SPEC-008 —— 库的接口：公开模块、发布闭包与两种形态的一致](../specs/library-interface.md) + - [SPEC-009 —— 工具链的支持与维护：版本线、默认值、来源、移动与退役](../specs/toolchain-maintenance.md) From 2268c15719d738ee35c451e26b669faf35d90464 Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 07:24:00 +0800 Subject: [PATCH 03/14] ci: one build of mcpp per host, stages in one workflow, single-writer caches, timed shards and a coverage check ci.yml classifies the change, runs the documentation checks, builds mcpp once per host (build.yml) and calls the per-area workflows, which take that build through use-built-mcpp instead of building their own. Caches are restored by every job and saved only by one job per key on a push to main; target/ is no longer cached. The e2e shards are assigned by measured duration (tests/e2e/timings) and write per-test reports that check_e2e_coverage.py reads: every test runs on some host, is run by a dedicated job, or is excused with a reason. run_all.sh grants llvm on Linux and probes musl and mingw-cross by family rather than by release. --- .github/actions/bootstrap-mcpp/action.yml | 64 +-- .github/actions/setup-macos-llvm/action.yml | 15 +- .github/actions/use-built-mcpp/action.yml | 41 ++ .github/actions/use-built-mcpp/use.sh | 67 +++ .github/tools/check_e2e_coverage.py | 148 ++++++ .github/tools/classify_changes.py | 140 ++++++ .github/workflows/build.yml | 121 +++++ .github/workflows/ci-linux-e2e.yml | 163 +++--- .github/workflows/ci-linux.yml | 231 ++------- .github/workflows/ci-macos-e2e.yml | 89 ++-- .github/workflows/ci-macos-ios.yml | 30 +- .github/workflows/ci-macos.yml | 55 +-- .github/workflows/ci-target-matrix.yml | 39 +- .github/workflows/ci-windows-e2e.yml | 68 +-- .github/workflows/ci-windows-msvc-xlings.yml | 30 +- .github/workflows/ci-windows.yml | 84 +--- .github/workflows/ci.yml | 257 ++++++++++ .github/workflows/cross-build-test.yml | 162 +++--- .../workflows/measure-windows-tool-crt.yml | 2 - .github/workflows/openkal-cross.yml | 53 +- tests/e2e/coverage-exceptions.tsv | 4 + tests/e2e/run_all.sh | 106 +++- tests/e2e/timings/linux.tsv | 466 ++++++++++++++++++ tests/e2e/timings/macos.tsv | 242 +++++++++ tests/e2e/timings/windows.tsv | 182 +++++++ tests/scripts/test_check_e2e_coverage.py | 84 ++++ tests/scripts/test_classify_changes.py | 82 +++ 27 files changed, 2382 insertions(+), 643 deletions(-) create mode 100644 .github/actions/use-built-mcpp/action.yml create mode 100644 .github/actions/use-built-mcpp/use.sh create mode 100644 .github/tools/check_e2e_coverage.py create mode 100644 .github/tools/classify_changes.py create mode 100644 .github/workflows/build.yml create mode 100644 .github/workflows/ci.yml create mode 100644 tests/e2e/coverage-exceptions.tsv create mode 100644 tests/e2e/timings/linux.tsv create mode 100644 tests/e2e/timings/macos.tsv create mode 100644 tests/e2e/timings/windows.tsv create mode 100644 tests/scripts/test_check_e2e_coverage.py create mode 100644 tests/scripts/test_classify_changes.py diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 15ef29bf..5ca4e06a 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -1,13 +1,21 @@ name: bootstrap-mcpp description: > - Restore the shared CI cache lineage (mcpp sandbox + xlings + target/) and - bootstrap a released mcpp via xlings. Exports MCPP and XLINGS_BIN. + Restore the shared CI cache lineage (mcpp sandbox and xlings) and bootstrap + a released mcpp via xlings. Exports MCPP and XLINGS_BIN. - Extracted so the split CI jobs (build / toolchain legs / e2e shards / - integration) share ONE definition instead of copy-pasting a 40-line - preamble per job. Every job that uses it lands on the same cache keys, - which is what makes splitting cheap: each job restores a warm sandbox - and only pays one incremental `mcpp build`. + The caches are RESTORED here and never saved. One job per host writes them, + the build job of .github/workflows/build.yml, and only on a push to main + (rule R3 of .agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain- + specification-design.md). Before that rule every job that used this action + saved the same key on success: thirty-four to forty-two saves per pull + request against a 10 GB repository limit, parallel saves of one key racing + each other, and the main lineage evicted within forty minutes (measured + 2026-10-01). The keys are outputs so that the one writer saves exactly what + was restored. + + `target/` is no longer cached. A restored `target/` made no build + incremental: on an exact hit ninja still ran 830 of 830 edges, while the + caches themselves were up to 3.3 GB each. inputs: xlings-version: @@ -26,10 +34,20 @@ inputs: # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. default: '2026.9.30.1' - cache-target: - description: also restore/save target/ (build artifacts + BMIs) - required: false - default: 'true' + +outputs: + sandbox-key: + description: the exact key of the mcpp sandbox cache + value: ${{ steps.sandbox.outputs.cache-primary-key }} + sandbox-hit: + description: "'true' when the sandbox was restored by its exact key" + value: ${{ steps.sandbox.outputs.cache-hit }} + xlings-key: + description: the exact key of the xlings cache + value: ${{ steps.xlings.outputs.cache-primary-key }} + xlings-hit: + description: "'true' when xlings was restored by its exact key" + value: ${{ steps.xlings.outputs.cache-hit }} runs: using: composite @@ -38,8 +56,9 @@ runs: # "-release-" caches. A bare "mcpp-sandbox--" restore prefix used to # match the release sandbox too, silently swapping in a differently # populated registry (issue #120). - - name: Cache mcpp sandbox - uses: actions/cache@v4 + - name: Restore the mcpp sandbox + id: sandbox + uses: actions/cache/restore@v4 with: path: ~/.mcpp # `runner.arch` IS PART OF EVERY KEY, AND WAS NOT. @@ -70,8 +89,9 @@ runs: restore-keys: | mcpp-sandbox-${{ runner.os }}-${{ runner.arch }}-ci-xl${{ inputs.xlings-version }}- - - name: Cache xlings - uses: actions/cache@v4 + - name: Restore xlings + id: xlings + uses: actions/cache/restore@v4 with: path: ~/.xlings key: xlings-${{ runner.os }}-${{ runner.arch }}-v2-xl${{ inputs.xlings-version }}-${{ hashFiles('.xlings.json') }} @@ -207,17 +227,3 @@ runs: # Precise key on src/ + manifest so a no-source-change run lands on a full # hit; layered restore-keys let partial hits keep BMI/dyndep state for a # proper incremental build. - - name: Cache target/ (build artifacts + BMIs) - if: inputs.cache-target == 'true' - uses: actions/cache@v4 - with: - path: target - # `modules/**` belongs here as much as `src/**` does. mcpp's own - # source lives in both since the subsystem split, and a key that hashed - # only one of them would restore a target/ built from different sources - # and report success — the failure mode a cache key exists to prevent, - # arriving silently. - key: mcpp-target-${{ runner.os }}-${{ runner.arch }}-${{ github.job }}-${{ hashFiles('src/**', 'modules/**', 'tests/**', 'mcpp.toml', 'mcpp.lock') }} - restore-keys: | - mcpp-target-${{ runner.os }}-${{ runner.arch }}-${{ github.job }}- - mcpp-target-${{ runner.os }}-${{ runner.arch }}- diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index e79a10cb..b43cd7de 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -24,11 +24,22 @@ inputs: required: false default: 'macos-15' +outputs: + xlings-key: + description: the exact key of the xlings cache + value: ${{ steps.xlings.outputs.cache-primary-key }} + xlings-hit: + description: "'true' when xlings was restored by its exact key" + value: ${{ steps.xlings.outputs.cache-hit }} + runs: using: composite steps: - - name: Cache xlings - uses: actions/cache@v4 + # Restored, never saved here: the macOS build job of + # .github/workflows/build.yml is the one writer, on a push to main (rule R3). + - name: Restore xlings + id: xlings + uses: actions/cache/restore@v4 with: path: ~/.xlings key: xlings-${{ inputs.image }}-arm64-v3-xl${{ inputs.xlings-version }}-${{ hashFiles('.xlings.json') }} diff --git a/.github/actions/use-built-mcpp/action.yml b/.github/actions/use-built-mcpp/action.yml new file mode 100644 index 00000000..96ab65c6 --- /dev/null +++ b/.github/actions/use-built-mcpp/action.yml @@ -0,0 +1,41 @@ +name: use-built-mcpp +description: > + Put this commit's mcpp, built once per host by .github/workflows/build.yml, + in place of a build of the job's own (rule R1 of + .agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md). + + Run it after bootstrap-mcpp or setup-macos-llvm, which restore the sandbox + and install the released bootstrap. It exports MCPP_BOOT (that bootstrap), + MCPP and MCPP_FRESH (this commit's binary, at an absolute path), and + MCPP_VENDORED_XLINGS, and sets the given mirror on xlings and on the binary. + + The binary is the one the build job produced, not a repackaging of it, so + every consumer tests what a self-host build makes. On Linux that binary's + interpreter and runtime libraries live in payloads of the sandbox (glibc and + the GCC runtime of the toolchain mcpp.toml names). A restored sandbox holds + them; when it does not, the bootstrap installs that toolchain and the binary + is run again. A binary that still does not run fails this step. + +inputs: + host: + description: > + The host the artifact was built on: linux-x86_64, linux-aarch64, + macos-arm64 or windows-x86_64. + required: true + mirror: + description: The mirror xlings and mcpp use in this job. + required: false + default: GLOBAL + +runs: + using: composite + steps: + - name: Download this commit's mcpp (${{ inputs.host }}) + uses: actions/download-artifact@v4 + with: + name: mcpp-built-${{ inputs.host }} + path: ${{ runner.temp }}/mcpp-built + + - name: Use this commit's mcpp + shell: bash + run: bash "$GITHUB_ACTION_PATH/use.sh" "${{ inputs.host }}" "${{ inputs.mirror }}" diff --git a/.github/actions/use-built-mcpp/use.sh b/.github/actions/use-built-mcpp/use.sh new file mode 100644 index 00000000..e94da450 --- /dev/null +++ b/.github/actions/use-built-mcpp/use.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# The body of the use-built-mcpp action; see action.yml for what it provides. +# +# Usage: use.sh +set -euo pipefail + +host="$1" +mirror="$2" + +dir="$RUNNER_TEMP/mcpp-built" +case "$host" in + windows-*) exe=mcpp.exe; dir="$(cygpath -u "$dir")" ;; + *) exe=mcpp ;; +esac +bin="$dir/$exe" +if [ ! -f "$bin" ]; then + echo "::error::the artifact mcpp-built-$host holds no $exe" + ls -la "$dir" || true + exit 1 +fi +chmod +x "$bin" + +boot="${MCPP:-}" +if [ -z "$boot" ]; then + echo "::error::MCPP is unset: run bootstrap-mcpp or setup-macos-llvm before use-built-mcpp" + exit 1 +fi + +# The toolchain mcpp.toml names for this host is the one the build used, so it +# is the one whose payloads hold the binary's runtime. +manifest_toolchain() { + local key + case "$host" in + macos-*) key=macos ;; + windows-*) key=windows ;; + *) key=default ;; + esac + awk -v k="$key" ' + /^\[/ { in_tc = ($0 == "[toolchain]") ; next } + in_tc && $1 == k { gsub(/"/, "", $3); print $3; exit } + ' mcpp.toml +} + +if ! out=$("$bin" --version 2>&1); then + tc="$(manifest_toolchain)" + echo "this commit's mcpp does not run yet ($out); installing ${tc:-the default toolchain} with the bootstrap" + if [ -n "$tc" ]; then + "$boot" toolchain install "${tc%@*}" "${tc#*@}" + fi + if ! out=$("$bin" --version 2>&1); then + echo "::error::this commit's mcpp does not run on this runner: $out" + exit 1 + fi +fi +echo "this commit's mcpp: $out ($bin)" + +{ + echo "MCPP_BOOT=$boot" + echo "MCPP=$bin" + echo "MCPP_FRESH=$bin" + if [ -n "${XLINGS_BIN:-}" ]; then echo "MCPP_VENDORED_XLINGS=$XLINGS_BIN"; fi +} >> "$GITHUB_ENV" + +if [ -n "${XLINGS_BIN:-}" ]; then + "$XLINGS_BIN" config --mirror "$mirror" 2>/dev/null || true +fi +MCPP_VENDORED_XLINGS="${XLINGS_BIN:-}" "$bin" self config --mirror "$mirror" diff --git a/.github/tools/check_e2e_coverage.py b/.github/tools/check_e2e_coverage.py new file mode 100644 index 00000000..54cca913 --- /dev/null +++ b/.github/tools/check_e2e_coverage.py @@ -0,0 +1,148 @@ +#!/usr/bin/env python3 +"""Every e2e test runs somewhere, or says why it cannot (rule R5). + +WHY THIS EXISTS + +`run_all.sh` exits 0 on a skip, and a skip line reads the same whether the +host legitimately lacks a capability or the runner was set up wrong. Measured +on 2026-10-01, before this check: 24 of 564 tests ran on no host and were named +by no workflow. Seven declared `llvm`, which no line granted; three declared +`musl`, whose probe named a release (15.1.0) the runners no longer installed; +seven declared `mingw-cross`, which no shard installed. Every one of those runs +was green. + +WHAT IT CHECKS + +It reads the per-test reports the shards write (`E2E_REPORT` of run_all.sh, one +`\\t\\t\\t` line per test) and decides, for every test +under tests/e2e: + + ran some report says pass, fail or timeout; + job no report ran it, but a workflow names it (by file name, or by its + number as an `E2E_ONLY` pattern such as `239_*.sh`): a dedicated job + runs it and asserts its result itself; + excused tests/e2e/coverage-exceptions.tsv lists it with the reason no hosted + runner can run it; + uncovered none of these. The check fails. + +An exception for a test that ran, or for a test that does not exist, also fails +the check, so that the list cannot outlive the reason it records. + +With --timings-out DIR it writes `.tsv`, the measured duration of every +test that ran, merged across that host's shards, in the format run_all.sh reads +from E2E_TIMINGS; refreshing tests/e2e/timings/ is copying those files. A report +file is named `e2e-report--.tsv`. + +Usage: + check_e2e_coverage.py --reports DIR [--root DIR] [--timings-out DIR] +""" +from __future__ import annotations + +import argparse +import re +import sys +from collections import defaultdict +from pathlib import Path + +RAN = {"pass", "fail", "timeout"} +REPORT_NAME = re.compile(r"e2e-report-(?P[a-z0-9-]+?)-(?P\d+)\.tsv$") + + +def read_reports(directory: Path): + """Yield (host, shard, status, test, ms, detail) for every report line.""" + for path in sorted(directory.rglob("e2e-report-*.tsv")): + m = REPORT_NAME.search(path.name) + if not m: + continue + for line in path.read_text(encoding="utf-8", errors="replace").splitlines(): + parts = line.split("\t") + if len(parts) < 3: + continue + ms = int(parts[2]) if parts[2].isdigit() else 0 + yield m["host"], int(m["shard"]), parts[0], parts[1], ms, parts[3] if len(parts) > 3 else "" + + +def read_exceptions(path: Path) -> dict[str, str]: + out: dict[str, str] = {} + if not path.exists(): + return out + for line in path.read_text(encoding="utf-8").splitlines(): + if not line.strip() or line.startswith("#"): + continue + test, _, reason = line.partition("\t") + out[test.strip()] = reason.strip() + return out + + +def named_by_a_workflow(test: str, workflows: str) -> bool: + number = test.split("_", 1)[0] + return test in workflows or test[:-3] in workflows or f"{number}_*" in workflows + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--reports", required=True) + parser.add_argument("--root", default=".") + parser.add_argument("--timings-out") + args = parser.parse_args() + root = Path(args.root).resolve() + + tests = sorted(p.name for p in (root / "tests" / "e2e").glob("[0-9]*.sh")) + workflows = "\n".join(p.read_text(encoding="utf-8") + for p in sorted((root / ".github" / "workflows").glob("*.yml"))) + exceptions = read_exceptions(root / "tests" / "e2e" / "coverage-exceptions.tsv") + + ran: dict[str, set[str]] = defaultdict(set) + skipped: dict[str, set[str]] = defaultdict(set) + shard_ms: dict[tuple[str, int], int] = defaultdict(int) + timings: dict[str, dict[str, int]] = defaultdict(dict) + reports = 0 + for host, shard, status, test, ms, detail in read_reports(Path(args.reports)): + reports += 1 + if status in RAN: + ran[test].add(host) + shard_ms[(host, shard)] += ms + timings[host][test] = ms + elif status == "skip": + skipped[test].add(f"{host}: {detail}") + if reports == 0: + print(f"no report under {args.reports}: nothing can be said about coverage") + return 1 + + uncovered, stale = [], [] + counts = defaultdict(int) + for test in tests: + if test in ran: + counts["ran"] += 1 + if test in exceptions: + stale.append(f"{test} is excused but ran on {', '.join(sorted(ran[test]))}") + elif named_by_a_workflow(test, workflows): + counts["job"] += 1 + elif test in exceptions: + counts["excused"] += 1 + else: + uncovered.append(f"{test}: {'; '.join(sorted(skipped[test])) or 'in no report'}") + for test in exceptions: + if test not in tests: + stale.append(f"{test} is excused but does not exist") + + print(f"{len(tests)} tests: {counts['ran']} ran on a shard, {counts['job']} run by a " + f"dedicated job, {counts['excused']} excused, {len(uncovered)} uncovered") + for (host, shard), ms in sorted(shard_ms.items()): + print(f" {host} shard {shard}: {ms / 60000:.1f} min of tests") + for line in uncovered: + print(f"UNCOVERED: {line}") + for line in stale: + print(f"STALE EXCEPTION: {line}") + + if args.timings_out: + out = Path(args.timings_out) + out.mkdir(parents=True, exist_ok=True) + for host, table in timings.items(): + (out / f"{host}.tsv").write_text( + "".join(f"{t}\t{ms}\n" for t, ms in sorted(table.items())), encoding="utf-8") + return 1 if uncovered or stale else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.github/tools/classify_changes.py b/.github/tools/classify_changes.py new file mode 100644 index 00000000..b6d5a6ff --- /dev/null +++ b/.github/tools/classify_changes.py @@ -0,0 +1,140 @@ +#!/usr/bin/env python3 +"""Decide whether a change can affect anything but the documentation checks. + +Rule R2 of .agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain- +specification-design.md: what a job runs follows from what the commit changed. +Measured before this rule (2026-10-01): a commit that changed two files under +`.agents/docs` ran 35 jobs, 36 builds of mcpp and 436 runner minutes. + +THE CLASSES + + documentation `*.md` anywhere, `docs/**`, `.agents/**`, `LICENSE*`. + code everything else, and every documentation file that a script, + a test, a workflow or a source file names. The second half is + DERIVED, by searching the repository, rather than listed: a test + added later that reads a document is found without editing this + program. The search looks for the file's path and for the path of + its translation (`docs/X` and `docs/zh/X` name each other), since + a check that reads one language usually reads both. It does not + search for a bare file name: `README.md` is named by release + packaging, and a match on the name alone would make every design + record, whose index is `.agents/docs/README.md`, a code change. + +A change whose every path is documentation that nothing names starts the +documentation checks only. Anything else, an empty change and a change that +cannot be listed included, starts the whole CI. + +Usage: + classify_changes.py [--root DIR] [--github-output FILE] < changed-paths + One path per line on standard input, relative to the repository root. + Prints `code=true` or `code=false` and one line per reason; with + --github-output, appends `code=...` to that file as well. +""" +from __future__ import annotations + +import argparse +import fnmatch +import os +import sys +from pathlib import Path + +DOC_PATTERNS = ("*.md", "docs/*", ".agents/*", "LICENSE*") +SEARCHED = ("tests", ".github", "src", "modules", "tools", "mcpp.toml", "bench/src") +SKIPPED_DIRS = {".git", "target", "node_modules", "__pycache__"} + +# A document read only by these files is still documentation: this program and +# its test name paths as data, and the checks of the `docs` job in ci.yml run on +# every change, documentation-only ones included. tests/scripts/ +# test_classify_changes.py requires each check listed here to appear in ci.yml. +DOCS_JOB_CHECKS = ( + ".github/tools/check_docs_style.sh", + ".github/tools/check_docs_structure.sh", + ".github/tools/gen_agents_index.py", + ".github/tools/check_target_tiers.py", + ".github/tools/check_reason_tokens.sh", + ".github/tools/check_matrix_reasons.sh", + ".github/tools/check_version_pins.sh", +) +NOT_READERS = (".github/tools/classify_changes.py", "tests/scripts/test_classify_changes.py") \ + + DOCS_JOB_CHECKS + + +def is_documentation(path: str) -> bool: + return any(fnmatch.fnmatchcase(path, p) for p in DOC_PATTERNS) or \ + fnmatch.fnmatchcase(Path(path).name, "*.md") + + +def searched_files(root: Path): + for entry in SEARCHED: + base = root / entry + if base.is_file(): + yield base + continue + if not base.is_dir(): + continue + for dirpath, dirnames, filenames in os.walk(base): + dirnames[:] = [d for d in dirnames if d not in SKIPPED_DIRS] + for name in filenames: + if name.endswith(".md"): + continue + yield Path(dirpath) / name + + +def names_of(path: str) -> tuple[str, ...]: + """The path, and the path of its translation under docs/.""" + if path.startswith("docs/zh/"): + return path, "docs/" + path[len("docs/zh/"):] + if path.startswith("docs/"): + return path, "docs/zh/" + path[len("docs/"):] + return (path,) + + +def readers_of(root: Path, docs: list[str]) -> dict[str, list[str]]: + """Map each documentation path to the files that name it.""" + needles = {d: names_of(d) for d in docs} + found: dict[str, list[str]] = {d: [] for d in docs} + for f in searched_files(root): + if str(f.relative_to(root)).replace(os.sep, "/") in NOT_READERS: + continue + try: + text = f.read_text(encoding="utf-8", errors="ignore") + except OSError: + continue + for d, names in needles.items(): + if any(n in text for n in names): + found[d].append(str(f.relative_to(root))) + return found + + +def classify(root: Path, paths: list[str]) -> tuple[bool, list[str]]: + paths = [p.strip() for p in paths if p.strip()] + if not paths: + return True, ["the change lists no path"] + code = [p for p in paths if not is_documentation(p)] + if code: + shown = ", ".join(code[:5]) + (" ..." if len(code) > 5 else "") + return True, [f"{len(code)} path(s) outside the documentation: {shown}"] + named = {d: r for d, r in readers_of(root, paths).items() if r} + if named: + return True, [f"{d} is read by {', '.join(r[:3])}" for d, r in sorted(named.items())] + return False, [f"{len(paths)} documentation path(s), none named by a script, test or source"] + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--root", default=".") + parser.add_argument("--github-output") + args = parser.parse_args() + code, reasons = classify(Path(args.root).resolve(), sys.stdin.read().splitlines()) + value = "true" if code else "false" + print(f"code={value}") + for r in reasons: + print(f" {r}") + if args.github_output: + with open(args.github_output, "a", encoding="utf-8") as out: + out.write(f"code={value}\n") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml new file mode 100644 index 00000000..15cfc57f --- /dev/null +++ b/.github/workflows/build.yml @@ -0,0 +1,121 @@ +name: build + +# ONE BUILD OF THIS COMMIT'S MCPP PER HOST (rule R1 of +# .agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md). +# +# Called by ci.yml once per host. The binary is uploaded as the artifact +# `mcpp-built-`, and every job of the commit that needs this commit's mcpp +# takes it with .github/actions/use-built-mcpp instead of building its own. +# Measured before this workflow existed (2026-10-01): thirty to thirty-seven +# jobs per commit built mcpp from source, which was 44 to 49 percent of the +# commit's runner minutes, and none of those builds was incremental. +# +# This job is also the ONE WRITER of the host's sandbox and xlings caches (rule +# R3), and it writes them only on a push to main. Before it saves, it installs +# the toolchains its consumers need, so that the cache a pull request restores +# already holds them. + +on: + workflow_call: + inputs: + host: + description: linux-x86_64, linux-aarch64, macos-arm64 or windows-x86_64 + type: string + required: true + runs-on: + description: the runner label + type: string + required: true + prewarm: + description: > + Toolchains installed before the caches are saved on main, separated + by ';', each as `mcpp toolchain install` takes it (`gcc 16.1.0-musl`). + type: string + required: false + default: '' + +jobs: + build: + name: build mcpp (${{ inputs.host }}) + runs-on: ${{ inputs.runs-on }} + timeout-minutes: 45 + steps: + - uses: actions/checkout@v4 + + - id: boot + if: ${{ !startsWith(inputs.host, 'macos-') }} + uses: ./.github/actions/bootstrap-mcpp + + - id: mac + if: ${{ startsWith(inputs.host, 'macos-') }} + uses: ./.github/actions/setup-macos-llvm + with: + image: ${{ inputs.runs-on }} + + - name: Build mcpp from source (self-host) + shell: bash + run: | + set -euo pipefail + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true + "$MCPP" self config --mirror GLOBAL 2>/dev/null || true + "$MCPP" build + # target/ is not restored from a cache, so the tree holds exactly the + # binary this build linked. + case "${{ inputs.host }}" in windows-*) exe=mcpp.exe ;; *) exe=mcpp ;; esac + built=$(find target -type f -name "$exe" -path '*/bin/*' | grep -v '/dist/' || true) + [ "$(printf '%s\n' "$built" | grep -c .)" = 1 ] || { + echo "::error::expected one $exe under target/, found: ${built:-none}"; exit 1; } + mkdir -p dist + cp "$built" "dist/$exe" + "dist/$exe" --version + + - name: Upload this commit's mcpp + timeout-minutes: 5 + uses: actions/upload-artifact@v4 + with: + name: mcpp-built-${{ inputs.host }} + path: dist/${{ startsWith(inputs.host, 'windows-') && 'mcpp.exe' || 'mcpp' }} + if-no-files-found: error + retention-days: 3 + + # Only on main, and only for the caches this job restored: a pull request + # reads the main lineage and adds nothing to the store. + - name: Install the toolchains the consumers use + if: ${{ github.event_name == 'push' && inputs.prewarm != '' }} + shell: bash + run: | + set -euo pipefail + IFS=';' read -ra specs <<< "${{ inputs.prewarm }}" + for spec in "${specs[@]}"; do + spec="$(echo "$spec" | xargs)" + [ -n "$spec" ] || continue + echo "::group::mcpp toolchain install $spec" + # shellcheck disable=SC2086 -- a spec is ` ` + "$MCPP" toolchain install $spec + echo "::endgroup::" + done + + - name: Save the sandbox cache + if: ${{ github.event_name == 'push' && steps.boot.outcome == 'success' && steps.boot.outputs.sandbox-hit != 'true' }} + uses: actions/cache/save@v4 + with: + path: ~/.mcpp + key: ${{ steps.boot.outputs.sandbox-key }} + + - name: Save the xlings cache + if: ${{ github.event_name == 'push' && steps.boot.outcome == 'success' && steps.boot.outputs.xlings-hit != 'true' }} + uses: actions/cache/save@v4 + with: + path: ~/.xlings + key: ${{ steps.boot.outputs.xlings-key }} + + # macOS keeps no sandbox cache: setup-macos-llvm builds the sandbox from + # the freshly resolved LLVM on every run, which is what the macOS jobs + # exist to prove. Its xlings cache has this job as its writer. + - name: Save the xlings cache (macOS) + if: ${{ github.event_name == 'push' && steps.mac.outcome == 'success' && steps.mac.outputs.xlings-hit != 'true' }} + uses: actions/cache/save@v4 + with: + path: ~/.xlings + key: ${{ steps.mac.outputs.xlings-key }} diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index a26be330..b8195c3c 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -1,40 +1,31 @@ name: ci-linux-e2e -# The e2e suite (tests/e2e/run_all.sh) split out of ci-linux.yml so it runs in -# PARALLEL with the build/unit/toolchain jobs instead of tacked on after them, -# and SHARDED across two runners on top of that. Both workflows share the same -# cache lineage (mcpp sandbox + xlings + target/), so each shard restores a warm -# build and the only added wall-clock vs. the inline version is one extra warm -# `mcpp build` per runner. +# The e2e suite (tests/e2e/run_all.sh) on Linux, called by ci.yml beside +# ci-linux.yml after the Linux build. Three shards, assigned by measured +# duration (tests/e2e/timings/linux.tsv, rule R4 of the 2026-10-02 CI record), +# each running the one binary build.yml produced (use-built-mcpp, rule R1). +# Each shard writes a per-test report that the e2e-coverage job of ci.yml reads +# (rule R5). # -# Paired workflows: ci-linux.yml (build + unit + toolchain legs + integration), +# Paired workflows: ci-linux.yml (unit + toolchain legs + integration), # ci-macos.yml / ci-macos-e2e.yml, ci-windows.yml / ci-windows-e2e.yml. on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + workflow_call: jobs: e2e: - name: e2e ${{ matrix.shard }}/2 (linux x86_64, self-host) + name: e2e ${{ matrix.shard }}/3 (linux x86_64, self-host) runs-on: ubuntu-24.04 timeout-minutes: 45 strategy: fail-fast: false matrix: - shard: [1, 2] + shard: [1, 2, 3] env: - # Round-robin slice of tests/e2e (see run_all.sh). Two shards halve the - # suite's wall-clock; the split is computed on the full file list, so a - # test's shard does not move when host capabilities differ. - E2E_SHARD: ${{ matrix.shard }}/2 + E2E_SHARD: ${{ matrix.shard }}/3 + E2E_TIMINGS: tests/e2e/timings/linux.tsv + E2E_REPORT: ${{ github.workspace }}/e2e-report-linux-${{ matrix.shard }}.tsv MCPP_HOME: /home/runner/.mcpp # NOTE: do NOT force MCPP_VERBOSE here. The e2e suite includes tests that # assert mcpp's DEFAULT (quiet) output — e.g. 48_build_error_output and @@ -45,74 +36,64 @@ jobs: # `submodules: recursive` so tests/e2e/233_bench_matrix.sh can check that # each `hub`/`body` in bench/matrix.json exists in the tree it names. # Without the trees that check reads "submodule not initialised" and - # reports nothing, which is how a hub path written for the CURRENT - # layout stayed in matrix.json while the workload it named is a - # HISTORICAL mcpp -- three cells reporting `skipped` on every bench run - # and the job still green (#599). - # - # It is not free and it is not expensive: the three pinned workloads are - # 725 + 701 + 806 tracked files, under 10 MB of source in total, and - # nothing here builds them. Both shards carry it because run_all.sh - # slices the file list round-robin, so which shard holds 233 moves when - # a test is added. + # reports nothing (#599). Under 10 MB of source; every shard carries it, + # because which shard holds 233 follows from the timing table. - uses: actions/checkout@v4 with: submodules: recursive - # Same cache lineage as ci-linux.yml so this job lands on a warm - # toolchain/sandbox instead of re-installing it. - uses: ./.github/actions/bootstrap-mcpp - - name: Configure mirror + Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP" build + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 - name: E2E suite - # Per-test 600s timeout lives in tests/e2e/run_all.sh and identifies - # WHICH test hung; this caps the whole suite so a hang fails fast. + # Twice the shard's budget of about ten minutes (R4): reached only by a + # hang. The per-test 600 s limit in run_all.sh names the test that hung. timeout-minutes: 25 run: | - # Point the e2e runner at the freshly-built binary, not the - # bootstrap one. Tests cd into mktemp -d, so $MCPP must be - # absolute or the relative path breaks under the temp cwd. - # The RELEASED mcpp this job bootstrapped from, kept for e2e 252: the - # claim that an older client can still build against a package the PR - # produces is only worth making if something checks it against a real - # old binary. Captured before $MCPP is repointed at the fresh build. - export MCPP_BOOT="$MCPP" - MCPP=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - test -x "$MCPP" - export MCPP - # Tests that set MCPP_HOME to a fresh tmpdir need an xlings to - # bootstrap from; surface the xlings binary installed above. - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + set -euo pipefail + # MCPP is this commit's binary and MCPP_BOOT the released bootstrap + # (use-built-mcpp). e2e 252 needs the latter: the claim that an older + # client can build against a package this commit produces is only + # worth making against a real old binary. + export MCPP MCPP_BOOT test -x "$MCPP_VENDORED_XLINGS" # GitHub-hosted runners are outside CN; keep CI toolchain downloads on # the global mirror while mcpp's default remains CN for fresh local # sandboxes. E2E tests with their own MCPP_HOME read this variable. export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL - "$MCPP" self config --mirror "$MCPP_E2E_TOOLCHAIN_MIRROR" "$MCPP" self config # Pin the global default so test 28 (default-toolchain path) gets a - # deterministic GNU answer instead of an auto-install pick. - # - # Installed explicitly, not assumed. `toolchain default` requires the - # toolchain to be present, and nothing in this job puts it there: - # building mcpp above uses whatever first-run picks (llvm). It worked - # only because the sandbox cache happened to carry a gcc some earlier - # run had installed -- so the moment the cache key changed (an xlings - # pin bump moves the whole prefix), every e2e shard failed with - # `gcc@16.1.0 is not installed` before running a single test. - # Warm runs re-install nothing; this costs a lookup. + # deterministic GNU answer instead of an auto-install pick. Installed + # explicitly, not assumed: the restored sandbox is the build job's, + # and a toolchain that only happens to be in it is a dependency on a + # cache. "$MCPP" toolchain install gcc 16.1.0 "$MCPP" toolchain default gcc@16.1.0 - # Warm musl once so fresh-home e2e tests inherit the payload. + # The toolchains whose capabilities the suite probes (run_all.sh): + # musl, llvm (which also yields scan-deps and import-std-libcxx) and + # the Linux-hosted MinGW cross compiler. Without them their tests + # skip on every runner; measured 2026-10-01, seventeen such tests ran + # nowhere. Warm runs re-install nothing. "$MCPP" toolchain install gcc 16.1.0-musl + "$MCPP" toolchain install llvm 22.1.8 + "$MCPP" toolchain install mingw-cross 16.1.0 + XLINGS_HOME="$MCPP_HOME/registry" "$MCPP_VENDORED_XLINGS" install xim:nasm -y bash tests/e2e/run_all.sh + # One file, no glob: the report the e2e-coverage job of ci.yml reads. + - name: Upload the shard's report + if: always() + timeout-minutes: 5 + uses: actions/upload-artifact@v4 + with: + name: e2e-report-linux-${{ matrix.shard }} + path: e2e-report-linux-${{ matrix.shard }}.tsv + if-no-files-found: warn + retention-days: 7 + # ────────────────────────────────────────────────────────────────── # Bare metal: the one chain the sharded suite above cannot be trusted # to exercise. @@ -138,12 +119,9 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP" build + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 - name: Install the emulator (xim:qemu-riscv) run: | @@ -184,12 +162,8 @@ jobs: - name: Bare-metal e2e timeout-minutes: 25 run: | - MCPP=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - test -x "$MCPP" - export MCPP - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + export MCPP MCPP_VENDORED_XLINGS export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL - "$MCPP" self config --mirror "$MCPP_E2E_TOOLCHAIN_MIRROR" # llvm is the toolchain a freestanding target pins; install it # explicitly rather than relying on whatever the sandbox cache holds. "$MCPP" toolchain install llvm 22.1.8 @@ -278,12 +252,9 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP" build + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 - name: Install the NDK (xim:android-ndk) timeout-minutes: 20 @@ -296,12 +267,8 @@ jobs: - name: Android e2e (no device) timeout-minutes: 40 run: | - MCPP=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - test -x "$MCPP" - export MCPP - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + export MCPP MCPP_VENDORED_XLINGS export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL - "$MCPP" self config --mirror "$MCPP_E2E_TOOLCHAIN_MIRROR" "$MCPP" toolchain install gcc 16.1.0 "$MCPP" toolchain default gcc@16.1.0 # Run directly, not through run_all.sh, so a skip cannot pass as a @@ -374,8 +341,9 @@ jobs: # Payload cache (downloads only — the container still has no host # toolchain, which is the property under test). - - name: Cache mcpp sandbox payloads - uses: actions/cache@v4 + - name: Restore mcpp sandbox payloads + id: hermetic-cache + uses: actions/cache/restore@v4 with: path: ~/.mcpp key: mcpp-hermetic-${{ hashFiles('mcpp.toml') }} @@ -393,6 +361,8 @@ jobs: "$MCPP_BOOT" self config --mirror GLOBAL echo "MCPP_BOOT=$MCPP_BOOT" >> "$GITHUB_ENV" + # This job builds its own mcpp on purpose: the build is part of what it + # tests, a compiler-free container in which only sandbox payloads exist. - name: Build PR mcpp from source (sandbox gcc only) run: | "$MCPP_BOOT" build @@ -495,3 +465,12 @@ jobs: "ok: provides['std'] names the std-cache BMI" \ "ok: build-id is present and stable across two runs" \ "PASS: 786 the std unit in the database, emit/build agreement, provides and build-id" + + # Last, so that the payloads the steps above installed are in it. This job + # is the one writer of its cache, on main only (R3). + - name: Save mcpp sandbox payloads + if: ${{ github.event_name == 'push' && steps.hermetic-cache.outputs.cache-hit != 'true' }} + uses: actions/cache/save@v4 + with: + path: ~/.mcpp + key: ${{ steps.hermetic-cache.outputs.cache-primary-key }} diff --git a/.github/workflows/ci-linux.yml b/.github/workflows/ci-linux.yml index 448787e3..d4dade3d 100644 --- a/.github/workflows/ci-linux.yml +++ b/.github/workflows/ci-linux.yml @@ -5,30 +5,25 @@ name: ci-linux # workflow no longer depends on a previous-release tarball — the # chicken-and-egg now lives upstream in the xlings index. # -# SHAPE: four INDEPENDENT jobs, no `needs:` between them. Each restores the -# same cache lineage (see .github/actions/bootstrap-mcpp) and pays one warm -# `mcpp build` (~2.5 min) to get the PR's own binary, then does its own leg. -# That warm rebuild is far cheaper than serialising the legs behind a shared -# artifact would be: +# SHAPE: called by ci.yml after the Linux build (build.yml). No job here builds +# mcpp to obtain the commit's binary: each takes the one binary that build +# produced (.github/actions/use-built-mcpp, rule R1 of the 2026-10-02 CI +# record). A job builds mcpp only when the build is what it tests: a cold +# rebuild, another toolchain, a cross target. # -# before: build → unit → gcc → musl → llvm → xlings ≈ 18 min (one job) -# after: max(build+unit, gcc, musl+llvm, xlings) ≈ 7-8 min +# This replaced four independent jobs that each paid a "warm" rebuild, chosen +# when that rebuild cost about 2.5 minutes. Measured on 2026-10-01 it cost 6.6 +# minutes, and on an exact hit of the restored target/ ninja still ran 830 of +# 830 edges. # -# The ~18 min e2e suite is a SEPARATE workflow (ci-linux-e2e.yml, sharded) -# that runs in parallel on the same caches. +# The checks that need no binary run in the `docs` job of ci.yml. +# +# The e2e suite is ci-linux-e2e.yml, called beside this one. # # Paired workflows: ci-linux-e2e.yml, ci-macos.yml, ci-windows.yml. on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + workflow_call: env: # MCPP_HOME pinned so the cache keys below restore into the same path @@ -41,138 +36,20 @@ env: jobs: build-test: - name: build + unit tests (linux x86_64, self-host) + name: unit tests (linux x86_64, self-host) runs-on: ubuntu-24.04 timeout-minutes: 45 steps: - uses: actions/checkout@v4 - # Before the bootstrap, not after: this needs no toolchain and no mcpp, - # takes under a second, and the drift it catches (a stale xlings pin) - # would otherwise surface minutes later as an unrelated-looking - # dependency-resolution failure. Pure text extraction on purpose — it - # has to work when the build is broken. - - name: Check version / xlings pin consistency - run: bash .github/tools/check_version_pins.sh - - # Same placement and the same reason as the pin check above: adding a - # package under `modules/` means editing four files, and three of those - # omissions fail later and elsewhere — the worst of them only on a macOS - # machine that has no mcpp to rebuild with. - - name: Check modules/ wiring - run: bash .github/tools/check_modules_wiring.sh - - # Same placement, same reason: pure text, no toolchain, under a second. - # Catches the prepare.cppm decomposition growing back into one huge - # file one commit at a time — see the script's header. - - name: Check src/build/prepare* file lengths - run: bash .github/tools/check_file_lengths.sh - - # Same placement, same reason: pure text, no toolchain, under a second. - # - # This one is a HARD gate (unlike lint-ci-assertions.sh below) because it - # has no false positives left — its scope was cut to the two directories - # that walk trees mcpp does not control, and the one legitimate site - # carries a NARROW-OK rationale. See the script's header for why the - # scope is that narrow, and mcpp#516 for what it costs when it is missed. - # - # It runs on LINUX on purpose even though the bug it guards is - # Windows-only: it is text analysis, and putting it where the fast leg is - # means a violation is reported in seconds rather than after a Windows - # bootstrap. - - name: Check no walk-derived path is narrowed directly - run: bash .github/tools/check_narrow_conversions.sh - - # Same placement, same reason: pure text, no toolchain. - # - # IT PRINTS AND DOES NOT FAIL, DELIBERATELY. The three rules it carries - # each have real false positives — a control step legitimately pins one - # target, a `grep -q .` legitimately expects output — and the whole value - # of the thing is that somebody READS it. A hard gate would be routed - # around with a suppression within a month, and the suppression would then - # be the only record that a rule ever existed. - - name: Where the CI assertions live - run: bash tools/lint-ci-assertions.sh - - # A GATE, unlike the warnings above: a step asserts what its name says - # (the 2026-09-28 design, WS7, #729). A build piped into `tee` without - # pipefail, a build step whose status is discarded before a grep, and a - # job allowed to fail without an open issue are refused. The fixture - # tests run first, so a lint that stopped reading files fails here - # rather than passing every workflow. - - name: Steps assert what their names say - env: - GH_TOKEN: ${{ github.token }} - run: | - python3 tests/scripts/test_check_workflow_assertions.py - python3 .github/tools/check_workflow_assertions.py --check-open - - - name: The release canary runner runs each command under the named bash - run: python3 tests/scripts/test_release_canaries.py - - - name: The protocol table of SPEC-007 names the engine's protocol - run: python3 tests/scripts/test_protocol_table.py - - # Text-only, like the two steps around it, and it belongs here rather - # than in the target-matrix workflow: that workflow runs the matrix, and - # this asserts a property of the TABLE, which is readable without a - # single toolchain. - - name: The target matrix names every refusal - run: bash .github/tools/check_matrix_reasons.sh - - # Same placement and the same reason: pure text, no toolchain, and the - # drift it catches — a 简体中文 page that has silently fallen behind its - # English original — is invisible to every other job. - - name: Check documentation style and bilingual parity - run: bash .github/tools/check_docs_style.sh - - # The architecture half: which tree a document lives in, whether every - # path it names resolves, and whether each specification is reachable - # from all three indexes. Same placement and the same reason -- it needs - # no toolchain, and what it catches is invisible to every other job. - - name: Check documentation structure - run: bash .github/tools/check_docs_structure.sh - - # Every documented tier agrees with kKnownTargets. - # - # Four documents restate a row's tier and nothing compared them to - # the table. When wasm32-emscripten became `verified` and the - # Android rows gained tiers, docs/21 was updated and both READMEs - # were not -- so the front page said three targets were `planned` - # while the engine had built and run two of them. On its first run - # this check also found two Cortex-A rows missing from docs/21 - # entirely, which predates that work. - - name: Documented target tiers agree with the table - run: python3 .github/tools/check_target_tiers.py - - # The reason-token table in docs/50 is a MACHINE INTERFACE: mcpp-index's - # compatibility measurement reads a token out of a refusal to tell "this - # graph does not supply what the member asked for" from "the member did - # not build", and that distinction decides a published figure. A token - # the engine emits and the table omits is a promise nobody can rely on. - # - # Four tokens were added to that table by hand as "the ones it was - # missing"; a later enumeration found four more that had been absent the - # whole time. A set compared by reading is a set compared by sampling. - - name: Reason tokens agree with the engine, and with the mirror - run: bash .github/tools/check_reason_tokens.sh - - uses: ./.github/actions/bootstrap-mcpp - - name: Configure mirror + Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - # Set GLOBAL mirror via xlings directly (bootstrap mcpp may lack --mirror flag) - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP" build + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 - name: Unit + integration tests via `mcpp test` run: | - # Use freshly-built mcpp for test (it has --mirror support) - MCPP_FRESH=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - echo "MCPP_FRESH=$MCPP_FRESH" >> "$GITHUB_ENV" - "$MCPP_FRESH" self config --mirror GLOBAL "$MCPP_FRESH" test # Each package under `modules/` carries its own tests, and they are built @@ -196,17 +73,8 @@ jobs: - name: Per-subsystem tests (`mcpp test -p `) run: | set -euo pipefail - # `$MCPP_FRESH` comes from the step above, which takes the NEWEST - # mcpp anywhere under target/. That is correct in this job because - # nothing between the two steps writes another one — but `mcpp pack` - # leaves a copy under `target/dist/`, and a pack step inserted above - # would silently redirect both this loop and the suite before it to - # the packaged binary. Observed locally; kept as a note because the - # expression is shared with three other jobs and this is not the - # place to change it. - case "$MCPP_FRESH" in - */target/dist/*) echo "MCPP_FRESH points into target/dist — a pack step now runs before the tests"; exit 1 ;; - esac + # `$MCPP_FRESH` is the binary of build.yml (use-built-mcpp), not one + # found under target/. members=$(sed -n '/^\[workspace\]/,/^\[/p' mcpp.toml \ | grep -oE '"modules/[a-z0-9-]+"' | tr -d '"' | sed 's|modules/||') [ -n "$members" ] || { echo "no workspace members found in mcpp.toml"; exit 1; } @@ -217,11 +85,10 @@ jobs: echo "::endgroup::" done - # A cold, from-scratch self-host build with the manifest-pinned GCC: the - # property `build-test` cannot cover, because it builds incrementally on a - # restored target/. `mcpp test` is deliberately NOT repeated here — it would - # be the same suite, same toolchain, same driver binary as `build-test`, - # differing only in incremental state. + # A cold, from-scratch self-host build by this commit's binary with the + # manifest-pinned GCC: mcpp building itself, where build.yml has the + # bootstrap build it. `mcpp test` is deliberately NOT repeated here; it would + # be the same suite, toolchain and binary as `build-test`. toolchain-gcc: name: "toolchain: gcc (cold self-host)" runs-on: ubuntu-24.04 @@ -230,15 +97,9 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP" build - MCPP_FRESH=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - cp "$MCPP_FRESH" /tmp/mcpp-fresh - echo "MCPP=/tmp/mcpp-fresh" >> "$GITHUB_ENV" + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 - name: "Toolchain: GCC — cold rebuild with the PR binary" run: | @@ -248,8 +109,8 @@ jobs: "$MCPP" clean "$MCPP" build 2>&1 | tee build.log; grep -q "Resolved gcc@16.1.0" build.log - # The two cheap legs share one runner: each is ~1 min after the warm build, - # so a runner apiece would cost more in setup than it saves in wall-clock. + # The two rebuilds share one runner, each with this commit's binary as the + # builder: a runner apiece would cost more in setup than it saves. toolchain-cross: name: "toolchain: musl + llvm" runs-on: ubuntu-24.04 @@ -258,15 +119,9 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP" build - MCPP_FRESH=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - cp "$MCPP_FRESH" /tmp/mcpp-fresh - echo "MCPP=/tmp/mcpp-fresh" >> "$GITHUB_ENV" + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 # Auto-installs gcc@16.1.0-musl on demand (cached across runs). - name: "Toolchain: musl-gcc — build mcpp (--target)" @@ -313,15 +168,9 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP" build - MCPP_FRESH=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - cp "$MCPP_FRESH" /tmp/mcpp-fresh - echo "MCPP=/tmp/mcpp-fresh" >> "$GITHUB_ENV" + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 - name: "Integration: mcpp builds & runs xlings (openxlings/xlings)" run: | @@ -353,15 +202,9 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP" build - MCPP_FRESH=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - cp "$MCPP_FRESH" /tmp/mcpp-fresh - echo "MCPP=/tmp/mcpp-fresh" >> "$GITHUB_ENV" + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 - name: Build every example the runner can build run: | diff --git a/.github/workflows/ci-macos-e2e.yml b/.github/workflows/ci-macos-e2e.yml index 62e53072..10548091 100644 --- a/.github/workflows/ci-macos-e2e.yml +++ b/.github/workflows/ci-macos-e2e.yml @@ -1,28 +1,31 @@ name: ci-macos-e2e -# The e2e suite on macOS ARM64, split out of ci-macos.yml (where it was 3.4 of -# the job's 8.4 min) so it runs in parallel with the toolchain/integration -# work — same shape as ci-linux-e2e.yml / ci-windows-e2e.yml. +# The e2e suite on macOS ARM64, called by ci.yml beside ci-macos.yml after the +# macOS build. Two shards, assigned by measured duration +# (tests/e2e/timings/macos.tsv, rule R4 of the 2026-10-02 CI record), each +# running the one binary build.yml produced (use-built-mcpp, rule R1), and +# each writing the per-test report the e2e-coverage job of ci.yml reads. # -# Not sharded: the macOS runners finish the suite in ~3.5 min, which is under -# this workflow's setup+build floor, so a second runner would buy nothing. +# The suite took about 3.5 minutes when this workflow was split out of +# ci-macos.yml, and it was not sharded for that reason. Measured on 2026-10-01 +# it took 17 to 21 minutes in one job, and reached the 25-minute limit once. # # Paired workflows: ci-macos.yml, ci-linux-e2e.yml, ci-windows-e2e.yml. on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + workflow_call: + inputs: + known-red: + description: > + Include the legs that are known red (#669). ci.yml passes true on + main, on dispatch and on a pull request labelled `macos-27` (rule R7). + type: boolean + required: false + default: true jobs: e2e: - name: e2e suite (macOS ARM64, self-host, ${{ matrix.image }}${{ matrix.known_red != '' && format(', known red {0}', matrix.known_red) || '' }}) + name: e2e suite ${{ matrix.shard }}/${{ matrix.shards }} (macOS ARM64, self-host, ${{ matrix.image }}${{ matrix.known_red != '' && format(', known red {0}', matrix.known_red) || '' }}) # The same two images as ci-macos.yml; `xcode-27` is macOS 27 (see there). # KNOWN RED on xcode-27, along with ci-macos.yml's own job: the image's # Command Line Tools SDK ships an `arm64e.x1` .tbd stub ld64.lld 22.1.8 @@ -31,11 +34,10 @@ jobs: strategy: fail-fast: false matrix: - include: - - image: macos-15 - known_red: '' - - image: xcode-27 - known_red: '#669' + # macos-15 in two shards. The known-red leg is in the matrix only when + # the caller asks for it, and as one shard: it fails at its first link + # (#669), and two shards of it would hold two of the five macOS slots. + include: ${{ fromJSON(inputs.known-red && '[{"image":"macos-15","known_red":"","shard":1,"shards":2},{"image":"macos-15","known_red":"","shard":2,"shards":2},{"image":"xcode-27","known_red":"#669","shard":1,"shards":1}]' || '[{"image":"macos-15","known_red":"","shard":1,"shards":2},{"image":"macos-15","known_red":"","shard":2,"shards":2}]') }} runs-on: ${{ matrix.image }} # KNOWN RED, MACHINE-READABLY (the 2026-09-28 design, WS7). A leg whose # failure has a tracked external cause carries that issue in `known_red`: @@ -44,6 +46,10 @@ jobs: # the issue to be open. The leg leaves the list when #669 closes. continue-on-error: ${{ matrix.known_red != '' }} timeout-minutes: 60 + env: + E2E_SHARD: ${{ matrix.shard }}/${{ matrix.shards }} + E2E_TIMINGS: tests/e2e/timings/macos.tsv + E2E_REPORT: ${{ github.workspace }}/e2e-report-${{ matrix.image == 'macos-15' && 'macos' || matrix.image }}-${{ matrix.shard }}.tsv # NOTE: no MCPP_VERBOSE — the e2e suite asserts mcpp's default quiet # output (tests 48/53). steps: @@ -59,29 +65,21 @@ jobs: with: image: ${{ matrix.image }} - - name: Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP" build + - uses: ./.github/actions/use-built-mcpp + with: + host: macos-arm64 - name: E2E suite - # Fail-fast on hung tests instead of burning the whole job budget. - # Per-test 600s timeout lives in run_all.sh. - timeout-minutes: 25 + # Twice a shard's budget of about nine minutes (R4); the per-test 600 s + # limit in run_all.sh names a test that hangs. + timeout-minutes: 20 run: | - # The RELEASED mcpp this job bootstrapped from, kept for e2e 252: the - # claim that an older client can still build against a package the PR - # produces is only worth making if something checks it against a real - # old binary. Captured before $MCPP is repointed at the fresh build. - export MCPP_BOOT="$MCPP" - MCPP=$(find target -path "*/bin/mcpp" | head -1) - MCPP=$(cd "$(dirname "$MCPP")" && pwd)/$(basename "$MCPP") - test -x "$MCPP" - export MCPP - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + # MCPP is this commit's binary and MCPP_BOOT the released bootstrap + # (use-built-mcpp); e2e 252 checks an older client against a package + # this commit produces, so it needs a real old binary. + export MCPP MCPP_BOOT MCPP_VENDORED_XLINGS test -x "$MCPP_VENDORED_XLINGS" export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL - "$MCPP" self config --mirror "$MCPP_E2E_TOOLCHAIN_MIRROR" "$MCPP" self config # macOS default toolchain is LLVM "$MCPP" toolchain default "llvm@${MCPP_LLVM_VER}" @@ -89,6 +87,19 @@ jobs: set -o pipefail bash tests/e2e/run_all.sh 2>&1 | tee "$RUNNER_TEMP/e2e-suite.log" + # One file, no glob: the report the e2e-coverage job of ci.yml reads. A + # glob upload once hung a macOS job for ten minutes, and cancelling it lost + # the job's whole log. + - name: Upload the shard's report + if: always() + timeout-minutes: 5 + uses: actions/upload-artifact@v4 + with: + name: e2e-report-${{ matrix.image == 'macos-15' && 'macos' || matrix.image }}-${{ matrix.shard }} + path: e2e-report-${{ matrix.image == 'macos-15' && 'macos' || matrix.image }}-${{ matrix.shard }}.tsv + if-no-files-found: warn + retention-days: 7 + # Measurement legs print READING lines and pass whatever they read; the # readings are what a decision is taken from (#646 F2: whether a C++ # exception thrown in a dylib is caught by its class under the payload's @@ -108,6 +119,8 @@ jobs: # NDK is a large download the suite's per-test bound does not allow for, # and the shard does not carry it (the script declares android-ndk). - name: "Android row on a macOS host (721)" + # Once per image, not once per shard: the step installs the NDK. + if: matrix.shard == 1 # ci-lint: allow-r1: one NDK install per image; the shards share it timeout-minutes: 30 shell: bash run: | diff --git a/.github/workflows/ci-macos-ios.yml b/.github/workflows/ci-macos-ios.yml index 5d32e7d1..980191e8 100644 --- a/.github/workflows/ci-macos-ios.yml +++ b/.github/workflows/ci-macos-ios.yml @@ -14,15 +14,7 @@ name: ci-macos-ios # assertion before. on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-macos-ios-${{ github.ref }} - cancel-in-progress: true + workflow_call: jobs: # THE PREMISES, AS A PROBE THAT RUNS ON REQUEST. @@ -168,11 +160,9 @@ jobs: submodules: recursive - uses: ./.github/actions/setup-macos-llvm - - name: Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP" build - echo "MCPP_DEV=$(ls -td "$PWD"/target/*/*/bin/mcpp | head -1)" >> "$GITHUB_ENV" + - uses: ./.github/actions/use-built-mcpp + with: + host: macos-arm64 # NO TOOLCHAIN IS DECLARED, AND THAT IS WHAT MAKES THIS THE USER'S PATH. # @@ -288,7 +278,7 @@ jobs: run: | set -euo pipefail cd /tmp/iostest - "$MCPP_DEV" build --target aarch64-ios + "$MCPP_FRESH" build --target aarch64-ios /tmp/assert-artefact.sh aarch64-ios arm64 2 18.0 # `platform 7` is IOSSIMULATOR. The number is what separates this row from @@ -297,7 +287,7 @@ jobs: run: | set -euo pipefail cd /tmp/iostest - "$MCPP_DEV" build --target aarch64-ios-sim 2>&1 | tee build-sim.log + "$MCPP_FRESH" build --target aarch64-ios-sim 2>&1 | tee build-sim.log /tmp/assert-artefact.sh aarch64-ios-sim arm64 7 18.0 # THE TWO LAYERS THE PACKAGES SUPPLY, READ FROM THE REPORT AND FROM THE @@ -335,14 +325,14 @@ jobs: version = "0.1.0" TOML printf 'import std;\nint main() { std::print("x\\n"); }\n' > src/main.cpp - "$MCPP_DEV" build --target aarch64-ios-sim > mixed.log 2>&1 || { echo "FAIL: import std without llvm.libcxx no longer builds"; cat mixed.log; exit 1; } + "$MCPP_FRESH" build --target aarch64-ios-sim > mixed.log 2>&1 || { echo "FAIL: import std without llvm.libcxx no longer builds"; cat mixed.log; exit 1; } # A degradation renders its `what` text, not its domain, so the # sentence is what a log can be asked for. grep -q "links the SDK's libc++ under the toolchain payload's libc++ headers" mixed.log || { echo "FAIL: no degradation named the mixed libc++"; cat mixed.log; exit 1; } grep -q 'llvm.libcxx' mixed.log || { echo "FAIL: the degradation does not name llvm.libcxx"; cat mixed.log; exit 1; } printf '#include \n#include \nint main() { std::string s = "1-2-3"; std::puts(s.c_str()); }\n' > src/main.cpp rm -rf target - "$MCPP_DEV" build --target aarch64-ios-sim 2>&1 | tee plain.log + "$MCPP_FRESH" build --target aarch64-ios-sim 2>&1 | tee plain.log grep -q 'carries no compiler runtime for aarch64-ios-sim' plain.log || { echo "FAIL: no degradation named the missing compiler runtime"; exit 1; } grep -q "links the SDK's libc++ under" plain.log && { echo "FAIL: a program without import std was reported as mixing libc++"; exit 1; } ninja=$(ls target/aarch64-ios-sim/*/build.ninja | head -1) @@ -365,7 +355,7 @@ jobs: run: | set -uo pipefail cd /tmp/iostest - out=$("$MCPP_DEV" run --target aarch64-ios-sim 2>&1) && rc=0 || rc=$? + out=$("$MCPP_FRESH" run --target aarch64-ios-sim 2>&1) && rc=0 || rc=$? printf '%s\n' "$out" | tail -20 if [ "$rc" -ne 0 ]; then echo "FAIL: mcpp run --target aarch64-ios-sim exited $rc" @@ -384,5 +374,5 @@ jobs: run: | set -euo pipefail cd /tmp/iostest - "$MCPP_DEV" build --target x86_64-ios-sim + "$MCPP_FRESH" build --target x86_64-ios-sim /tmp/assert-artefact.sh x86_64-ios-sim x86_64 7 18.0 diff --git a/.github/workflows/ci-macos.yml b/.github/workflows/ci-macos.yml index b7c0c8e6..3167e3aa 100644 --- a/.github/workflows/ci-macos.yml +++ b/.github/workflows/ci-macos.yml @@ -3,19 +3,21 @@ name: ci-macos # macOS CI for mcpp — validates LLVM/Clang as the default macOS toolchain. # Tests the full xlings → LLVM → C++23 import std pipeline on macOS ARM64. # -# The e2e suite runs in PARALLEL in ci-macos-e2e.yml (same setup via -# .github/actions/setup-macos-llvm), not tacked onto this job. +# Called by ci.yml after the macOS build (build.yml): the steps that need this +# commit's mcpp take that build (use-built-mcpp, rule R1 of the 2026-10-02 CI +# record) instead of building one. The e2e suite is ci-macos-e2e.yml, called +# beside this one with the same setup (.github/actions/setup-macos-llvm). on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-macos-${{ github.ref }} - cancel-in-progress: true + workflow_call: + inputs: + known-red: + description: > + Include the legs that are known red (#669). ci.yml passes true on + main, on dispatch and on a pull request labelled `macos-27` (rule R7). + type: boolean + required: false + default: true jobs: macos-xlings-llvm: @@ -32,11 +34,10 @@ jobs: strategy: fail-fast: false matrix: - include: - - image: macos-15 - known_red: '' - - image: xcode-27 - known_red: '#669' + # The known-red leg is in the matrix only when the caller asks for it: + # a pull request cannot change its outcome unless it addresses #669, + # and on every other one it held a macOS slot to fail in three minutes. + include: ${{ fromJSON(inputs.known-red && '[{"image":"macos-15","known_red":""},{"image":"xcode-27","known_red":"#669"}]' || '[{"image":"macos-15","known_red":""}]') }} runs-on: ${{ matrix.image }} # KNOWN RED, MACHINE-READABLY (the 2026-09-28 design, WS7). A leg whose # failure has a tracked external cause carries that issue in `known_red`: @@ -292,25 +293,19 @@ jobs: *) echo "FAIL: unexpected platform"; exit 1 ;; esac - - name: Build mcpp from source (self-host) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP" build + - uses: ./.github/actions/use-built-mcpp + with: + host: macos-arm64 - name: Unit + integration tests via `mcpp test` run: | - # Use freshly-built mcpp (has --mirror support) - MCPP=$(find target -path "*/bin/mcpp" | head -1) - MCPP=$(cd "$(dirname "$MCPP")" && pwd)/$(basename "$MCPP") - "$MCPP" self config --mirror GLOBAL - "$MCPP" test + "$MCPP_FRESH" test # WS8: this host's row of docs/01 and docs/20 is checked against the # one answer the resolver gives (`self env --format json`). - name: The documented default toolchain is this host's answer run: | - MCPP=$(find target -path "*/bin/mcpp" | head -1) - python3 .github/tools/check_default_toolchain_docs.py --mcpp "$MCPP" + python3 .github/tools/check_default_toolchain_docs.py --mcpp "$MCPP_FRESH" - name: Forensics — test-binary link + load state (on failure) if: failure() @@ -359,17 +354,13 @@ jobs: - name: "Toolchain: LLVM — build mcpp (self-host)" run: | - MCPP=$(find target -path "*/bin/mcpp" | head -1) - MCPP=$(cd "$(dirname "$MCPP")" && pwd)/$(basename "$MCPP") - test -x "$MCPP" - cp "$MCPP" /tmp/mcpp-fresh + cp "$MCPP_FRESH" /tmp/mcpp-fresh MCPP=/tmp/mcpp-fresh "$MCPP" toolchain default "llvm@${MCPP_LLVM_VER}" "$MCPP" clean "$MCPP" build "$MCPP" --version - # GRAPHICS ON THIS HOST, BUILD ONLY, AND THAT IS THE WHOLE CLAIM. # # This runner has no Vulkan device, so what is asserted is what this diff --git a/.github/workflows/ci-target-matrix.yml b/.github/workflows/ci-target-matrix.yml index 62f22b9b..ad10d2bb 100644 --- a/.github/workflows/ci-target-matrix.yml +++ b/.github/workflows/ci-target-matrix.yml @@ -6,14 +6,7 @@ name: target matrix # 装了某个载荷」而通过,或因为没装而跳过,而两者在退出码上与「全部正确」没有区别。 # 三个宿主各扫一遍,把结果与仓库里的期望表比对,差异即失败。 on: - pull_request: - push: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + workflow_call: env: XLINGS_NON_INTERACTIVE: '1' @@ -50,7 +43,22 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp + # This commit's mcpp: the one build.yml produced (use-built-mcpp, rule R1 + # of the 2026-10-02 CI record), except on the macOS leg. That leg runs on + # macos-14 and the macOS artifact is built on macos-15; whether it runs on + # macos-14 is not measured (the record's Part VII), so the leg builds its + # own until it is. + - if: matrix.host != 'macos-arm64' + uses: ./.github/actions/use-built-mcpp + with: + host: ${{ matrix.host }} + + - name: Name this commit's mcpp MCPP_UNDER_TEST + if: matrix.host != 'macos-arm64' + run: echo "MCPP_UNDER_TEST=$MCPP_FRESH" >> "$GITHUB_ENV" + - name: Build the mcpp in this pull request + if: matrix.host == 'macos-arm64' # ci-lint: allow-r1: the macOS leg builds its own mcpp until the artifact is measured on macos-14 run: | set -euo pipefail export MCPP_VENDORED_XLINGS="$XLINGS_BIN" @@ -258,7 +266,22 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp + # This commit's mcpp: the one build.yml produced (use-built-mcpp, rule R1 + # of the 2026-10-02 CI record), except on the macOS leg. That leg runs on + # macos-14 and the macOS artifact is built on macos-15; whether it runs on + # macos-14 is not measured (the record's Part VII), so the leg builds its + # own until it is. + - if: matrix.host != 'macos-arm64' + uses: ./.github/actions/use-built-mcpp + with: + host: ${{ matrix.host }} + + - name: Name this commit's mcpp MCPP_UNDER_TEST + if: matrix.host != 'macos-arm64' + run: echo "MCPP_UNDER_TEST=$MCPP_FRESH" >> "$GITHUB_ENV" + - name: Build the mcpp in this pull request + if: matrix.host == 'macos-arm64' # ci-lint: allow-r1: the macOS leg builds its own mcpp until the artifact is measured on macos-14 run: | set -euo pipefail export MCPP_VENDORED_XLINGS="$XLINGS_BIN" diff --git a/.github/workflows/ci-windows-e2e.yml b/.github/workflows/ci-windows-e2e.yml index da100042..85ad336c 100644 --- a/.github/workflows/ci-windows-e2e.yml +++ b/.github/workflows/ci-windows-e2e.yml @@ -1,37 +1,33 @@ name: ci-windows-e2e -# The e2e suite on Windows, split out of ci-windows.yml (where it was 9.7 of -# the job's 20.4 min) and sharded across two runners — same shape as -# ci-linux-e2e.yml. Shares the cache lineage in .github/actions/bootstrap-mcpp, -# so each shard restores a warm sandbox and pays one incremental `mcpp build`. +# The e2e suite on Windows x64, called by ci.yml beside ci-windows.yml after +# the Windows build. Three shards, assigned by measured duration +# (tests/e2e/timings/windows.tsv, rule R4 of the 2026-10-02 CI record): about +# fifteen tests of over thirty seconds took 58 to 64 percent of a round-robin +# shard, and the two shards differed by 3.6 minutes on average. Each shard runs +# the one binary build.yml produced (use-built-mcpp, rule R1) and writes the +# per-test report the e2e-coverage job of ci.yml reads. # -# Paired workflows: ci-windows.yml (build + unit + package, toolchains + -# regressions), ci-linux-e2e.yml, ci-macos-e2e.yml. +# Paired workflows: ci-windows.yml, ci-linux-e2e.yml, ci-macos-e2e.yml. on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + workflow_call: jobs: e2e: - name: e2e ${{ matrix.shard }}/2 (windows x64, self-host) + name: e2e ${{ matrix.shard }}/3 (windows x64, self-host) runs-on: windows-latest timeout-minutes: 45 strategy: fail-fast: false matrix: - shard: [1, 2] + shard: [1, 2, 3] env: MCPP_HOME: C:\Users\runneradmin\.mcpp # Round-robin slice of tests/e2e (see run_all.sh). - E2E_SHARD: ${{ matrix.shard }}/2 + E2E_SHARD: ${{ matrix.shard }}/3 + E2E_TIMINGS: tests/e2e/timings/windows.tsv + E2E_REPORT: ${{ github.workspace }}/e2e-report-windows-${{ matrix.shard }}.tsv # NOTE: do NOT force MCPP_VERBOSE here. The e2e suite includes tests that # assert mcpp's DEFAULT (quiet) output — e.g. 48_build_error_output and # 53_namespaced_cache_label — which forced verbose would break. @@ -46,23 +42,15 @@ jobs: submodules: recursive - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) + - uses: ./.github/actions/use-built-mcpp + with: + host: windows-x86_64 + + - name: Name this commit's mcpp MCPP_SELF shell: bash run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP" build - # Pick the NEWEST mcpp.exe, not an arbitrary one: `target/` is - # restored from cache and keeps a directory per build fingerprint, - # so after a version bump the freshly built binary sits alongside - # the previous release's. `find | head -1` returned whichever the - # directory walk hit first — which is how a 0.0.106 build ran the - # 0.0.105 binary and failed 01_help_and_version. - MCPP_SELF=$(find target -name "mcpp.exe" -path "*/bin/*" -printf "%T@ %p\n" \ - | sort -rn | head -1 | cut -d" " -f2-) - test -n "$MCPP_SELF" || { echo "FAIL: no mcpp.exe"; exit 1; } - MCPP_SELF=$(cd "$(dirname "$MCPP_SELF")" && pwd)/$(basename "$MCPP_SELF") - "$MCPP_SELF" --version - echo "MCPP_SELF=$MCPP_SELF" >> "$GITHUB_ENV" + "$MCPP_FRESH" --version + echo "MCPP_SELF=$MCPP_FRESH" >> "$GITHUB_ENV" # MinGW-w64 GCC (xim:mingw-gcc). Installed here so the `mingw` capability # is GRANTED rather than left to whatever the shared sandbox cache happens @@ -101,7 +89,8 @@ jobs: # claim that an older client can still build against a package the PR # produces is only worth making if something checks it against a real # old binary. Captured before $MCPP is repointed at the fresh build. - export MCPP_BOOT="${MCPP:-$MCPP_BOOT}" + # MCPP_BOOT is the released bootstrap (use-built-mcpp), for e2e 252. + export MCPP_BOOT export MCPP="$MCPP_SELF" export MCPP_VENDORED_XLINGS="$XLINGS_BIN" export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL @@ -112,6 +101,17 @@ jobs: # Measurement legs print READING lines (#646 F2 across PE images, #649 E10 # the llvm row's recorded CRT); collected where a reader finds them. + # One file, no glob: the report the e2e-coverage job of ci.yml reads. + - name: Upload the shard's report + if: always() + timeout-minutes: 5 + uses: actions/upload-artifact@v4 + with: + name: e2e-report-windows-${{ matrix.shard }} + path: e2e-report-windows-${{ matrix.shard }}.tsv + if-no-files-found: warn + retention-days: 7 + - name: Measurement readings if: always() shell: bash diff --git a/.github/workflows/ci-windows-msvc-xlings.yml b/.github/workflows/ci-windows-msvc-xlings.yml index 1ed1ff1f..f8b50da4 100644 --- a/.github/workflows/ci-windows-msvc-xlings.yml +++ b/.github/workflows/ci-windows-msvc-xlings.yml @@ -31,15 +31,7 @@ name: ci-windows-msvc-xlings # Paired workflows: ci-windows.yml, ci-windows-e2e.yml. on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + workflow_call: jobs: xlings-msvc: @@ -52,21 +44,15 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) + - uses: ./.github/actions/use-built-mcpp + with: + host: windows-x86_64 + + - name: Name this commit's mcpp MCPP_SELF shell: bash run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP" build - # Newest mcpp.exe, not an arbitrary one — `target/` is restored from - # cache and keeps a directory per build fingerprint. Same reasoning - # as ci-windows-e2e.yml, where picking wrong ran the previous - # release's binary. - MCPP_SELF=$(find target -name "mcpp.exe" -path "*/bin/*" -printf "%T@ %p\n" \ - | sort -rn | head -1 | cut -d" " -f2-) - test -n "$MCPP_SELF" || { echo "FAIL: no mcpp.exe"; exit 1; } - MCPP_SELF=$(cd "$(dirname "$MCPP_SELF")" && pwd)/$(basename "$MCPP_SELF") - "$MCPP_SELF" --version - echo "MCPP_SELF=$MCPP_SELF" >> "$GITHUB_ENV" + "$MCPP_FRESH" --version + echo "MCPP_SELF=$MCPP_FRESH" >> "$GITHUB_ENV" - name: xlings-managed msvc e2e shell: bash diff --git a/.github/workflows/ci-windows.yml b/.github/workflows/ci-windows.yml index e4fc311e..9122ebb3 100644 --- a/.github/workflows/ci-windows.yml +++ b/.github/workflows/ci-windows.yml @@ -3,55 +3,37 @@ name: ci-windows # Windows CI for mcpp — same flow as Linux (ci-linux.yml) and macOS (ci-macos.yml): # xlings install mcpp → self-host build → smoke → package # -# SHAPE: three INDEPENDENT jobs, no `needs:` between them; each restores the -# shared cache lineage (.github/actions/bootstrap-mcpp) and pays one warm -# `mcpp build` to get the PR's own binary. The e2e suite moved to -# ci-windows-e2e.yml (sharded ×2) — it was 9.7 of this job's 20.4 min. -# -# before: build → unit → xlings → stdin → e2e → toolchains → package ≈ 20 min -# after: max(build+unit+package, toolchains+integration) ≈ 8 min -# in parallel with ci-windows-e2e (≈ 8 min) +# SHAPE: called by ci.yml after the Windows build (build.yml). No job here +# builds mcpp to obtain the commit's binary: each takes the one binary that +# build produced (use-built-mcpp, rule R1 of the 2026-10-02 CI record) and names +# it MCPP_SELF. A job builds mcpp only when the build is what it tests (the LLVM +# rebuild of `toolchains`). The e2e suite is ci-windows-e2e.yml, called beside +# this one. on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-windows-${{ github.ref }} - cancel-in-progress: true + workflow_call: env: MCPP_HOME: C:\Users\runneradmin\.mcpp jobs: build-test: - name: build + test + package (windows x64, self-host) + name: test + package (windows x64, self-host) runs-on: windows-latest timeout-minutes: 45 steps: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) + - uses: ./.github/actions/use-built-mcpp + with: + host: windows-x86_64 + + - name: Name this commit's mcpp MCPP_SELF shell: bash run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP" build - # Pick the NEWEST mcpp.exe, not an arbitrary one: `target/` is - # restored from cache and keeps a directory per build fingerprint, - # so after a version bump the freshly built binary sits alongside - # the previous release's. `find | head -1` returned whichever the - # directory walk hit first — which is how a 0.0.106 build ran the - # 0.0.105 binary and failed 01_help_and_version. - MCPP_SELF=$(find target -name "mcpp.exe" -path "*/bin/*" -printf "%T@ %p\n" \ - | sort -rn | head -1 | cut -d" " -f2-) - test -n "$MCPP_SELF" || { echo "FAIL: no mcpp.exe"; exit 1; } - MCPP_SELF=$(cd "$(dirname "$MCPP_SELF")" && pwd)/$(basename "$MCPP_SELF") - "$MCPP_SELF" --version - echo "MCPP_SELF=$MCPP_SELF" >> "$GITHUB_ENV" + "$MCPP_FRESH" --version + echo "MCPP_SELF=$MCPP_FRESH" >> "$GITHUB_ENV" - name: Unit + integration tests via mcpp test shell: bash @@ -72,15 +54,11 @@ jobs: VERSION=$(awk -F '"' '/^version[[:space:]]*=/{print $2; exit}' mcpp.toml) WRAPPER="mcpp-${VERSION}-windows-x86_64" ZIPNAME="${WRAPPER}.zip" - # Pick the NEWEST mcpp.exe, not an arbitrary one: `target/` is - # restored from cache and keeps a directory per build fingerprint, - # so after a version bump the freshly built binary sits alongside - # the previous release's. `find | head -1` returned whichever the - # directory walk hit first — which is how a 0.0.106 build ran the - # 0.0.105 binary and failed 01_help_and_version. - MCPP_BIN=$(find target -name "mcpp.exe" -path "*/bin/*" -printf "%T@ %p\n" \ - | sort -rn | head -1 | cut -d" " -f2-) - test -n "$MCPP_BIN" || { echo "FAIL: no mcpp.exe in target/"; exit 1; } + # The binary of build.yml (use-built-mcpp), not one found under + # target/: nothing restores target/ any more, and this is the binary + # every other job of the commit tests. + MCPP_BIN="$MCPP_SELF" + test -f "$MCPP_BIN" || { echo "FAIL: no mcpp.exe at $MCPP_BIN"; exit 1; } STAGING=$(mktemp -d) mkdir -p "$STAGING/$WRAPPER/bin" "$STAGING/$WRAPPER/registry/bin" @@ -239,22 +217,15 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build mcpp from source (self-host) + - uses: ./.github/actions/use-built-mcpp + with: + host: windows-x86_64 + + - name: Name this commit's mcpp MCPP_SELF shell: bash run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP" build - # Pick the NEWEST mcpp.exe, not an arbitrary one: `target/` is - # restored from cache and keeps a directory per build fingerprint, - # so after a version bump the freshly built binary sits alongside - # the previous release's. `find | head -1` returned whichever the - # directory walk hit first — which is how a 0.0.106 build ran the - # 0.0.105 binary and failed 01_help_and_version. - MCPP_SELF=$(find target -name "mcpp.exe" -path "*/bin/*" -printf "%T@ %p\n" \ - | sort -rn | head -1 | cut -d" " -f2-) - test -n "$MCPP_SELF" || { echo "FAIL: no mcpp.exe"; exit 1; } - MCPP_SELF=$(cd "$(dirname "$MCPP_SELF")" && pwd)/$(basename "$MCPP_SELF") - echo "MCPP_SELF=$MCPP_SELF" >> "$GITHUB_ENV" + "$MCPP_FRESH" --version + echo "MCPP_SELF=$MCPP_FRESH" >> "$GITHUB_ENV" # Integration: the mcpp built from THIS PR's source ($MCPP_SELF, the # self-hosted binary) builds & runs a real external C++ project — xlings @@ -430,7 +401,6 @@ jobs: export MCPP_VENDORED_XLINGS="$XLINGS_BIN" MCPP="$MCPP_SELF" bash .github/tools/check_unicode_paths.sh - # GRAPHICS ON THIS HOST, BUILD ONLY, AND THAT IS THE WHOLE CLAIM. # # This runner has no Vulkan device, so what is asserted is what this diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 00000000..30272628 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,257 @@ +name: ci + +# THE CI OF ONE COMMIT, IN STAGES. The design and its measurements are in +# .agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md +# (Part II); the rules it cites are R1-R7 there. +# +# changes what the commit changed decides what runs (R2). A change of +# documentation that nothing reads runs `docs` and nothing else. +# docs the checks that need no binary, on every change. +# build-* this commit's mcpp, built ONCE per host (R1) by build.yml and +# uploaded as `mcpp-built-`. +# the rest the per-area workflows, called as reusable workflows. Each waits +# for its own host's build only, takes the binary with +# .github/actions/use-built-mcpp, and keeps the job names it had as +# a workflow of its own. +# e2e-coverage every e2e test ran somewhere or says why not (R5). +# +# Measured before this shape (2026-10-01, seven commits): 35 to 47 jobs started +# at once against the organisation's 20 slots, 30 to 37 of them built mcpp +# from source, and a commit waited 36 to 50 minutes, 9 to 13 of them in the +# queue. + +on: + push: + branches: [ main ] + pull_request: + branches: [ main ] + workflow_dispatch: + +# A superseded pull-request run is cancelled. A push to main runs to the end, +# because its build jobs are the only writers of the caches (R3), and a +# cancelled run saves nothing. +concurrency: + group: ci-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +permissions: + contents: read + # `changes` lists the files of the pull request. + pull-requests: read + +jobs: + changes: + name: what the change starts + runs-on: ubuntu-24.04 + timeout-minutes: 5 + outputs: + code: ${{ steps.classify.outputs.code }} + steps: + - uses: actions/checkout@v4 + + - name: Classify the changed paths + id: classify + env: + GH_TOKEN: ${{ github.token }} + EVENT: ${{ github.event_name }} + REPO: ${{ github.repository }} + PR: ${{ github.event.pull_request.number }} + BEFORE: ${{ github.event.before }} + SHA: ${{ github.sha }} + run: | + set -euo pipefail + full() { echo "code=true" >> "$GITHUB_OUTPUT"; echo "code=true ($1)"; } + case "$EVENT" in + pull_request) + gh api --paginate "repos/$REPO/pulls/$PR/files" \ + --jq '.[] | .filename, (.previous_filename // empty)' > changed.txt ;; + push) + if [ -z "$BEFORE" ] || [ "$BEFORE" = 0000000000000000000000000000000000000000 ]; then + full "a push with no previous commit"; exit 0 + fi + gh api "repos/$REPO/compare/$BEFORE...$SHA" > compare.json + # The compare API lists at most 300 files; a longer list is + # treated as a change of everything. + if [ "$(jq '.files | length' compare.json)" -ge 300 ]; then + full "300 or more changed files"; exit 0 + fi + jq -r '.files[] | .filename, (.previous_filename // empty)' compare.json > changed.txt ;; + *) + full "event $EVENT"; exit 0 ;; + esac + python3 .github/tools/classify_changes.py --github-output "$GITHUB_OUTPUT" < changed.txt + + docs: + name: documentation and repository checks + runs-on: ubuntu-24.04 + timeout-minutes: 15 + steps: + - uses: actions/checkout@v4 + + - name: Check version / xlings pin consistency + run: bash .github/tools/check_version_pins.sh + + - name: Check modules/ wiring + run: bash .github/tools/check_modules_wiring.sh + + - name: Check src/build/prepare* file lengths + run: bash .github/tools/check_file_lengths.sh + + - name: Check no walk-derived path is narrowed directly + run: bash .github/tools/check_narrow_conversions.sh + + - name: Where the CI assertions live + run: bash tools/lint-ci-assertions.sh + + - name: Steps assert what their names say + env: + GH_TOKEN: ${{ github.token }} + run: | + python3 tests/scripts/test_check_workflow_assertions.py + python3 .github/tools/check_workflow_assertions.py --check-open + + - name: The release canary runner runs each command under the named bash + run: python3 tests/scripts/test_release_canaries.py + + - name: The protocol table of SPEC-007 names the engine's protocol + run: python3 tests/scripts/test_protocol_table.py + + - name: The target matrix names every refusal + run: bash .github/tools/check_matrix_reasons.sh + + - name: Check documentation style and bilingual parity + run: bash .github/tools/check_docs_style.sh + + - name: Check documentation structure + run: bash .github/tools/check_docs_structure.sh + + - name: Documented target tiers agree with the table + run: python3 .github/tools/check_target_tiers.py + + - name: Reason tokens agree with the engine, and with the mirror + run: bash .github/tools/check_reason_tokens.sh + + - name: The tools of the CI stages pass their fixtures + run: | + python3 tests/scripts/test_classify_changes.py + python3 tests/scripts/test_check_e2e_coverage.py + python3 tests/scripts/test_check_default_toolchain_docs.py + + build-linux: + needs: changes + if: needs.changes.outputs.code == 'true' + uses: ./.github/workflows/build.yml + with: + host: linux-x86_64 + runs-on: ubuntu-24.04 + # What the Linux consumers install beyond the build's own toolchain: the + # e2e shards (musl, llvm, mingw-cross), the toolchain legs (musl, llvm). + prewarm: gcc 16.1.0-musl; llvm 22.1.8; mingw-cross 16.1.0 + + build-linux-arm: + needs: changes + if: needs.changes.outputs.code == 'true' + uses: ./.github/workflows/build.yml + with: + host: linux-aarch64 + runs-on: ubuntu-24.04-arm + + build-macos: + needs: changes + if: needs.changes.outputs.code == 'true' + uses: ./.github/workflows/build.yml + with: + host: macos-arm64 + runs-on: macos-15 + + build-windows: + needs: changes + if: needs.changes.outputs.code == 'true' + uses: ./.github/workflows/build.yml + with: + host: windows-x86_64 + runs-on: windows-latest + # The MinGW payload every Windows e2e shard installs. + prewarm: mingw 16.1.0 + + linux: + needs: build-linux + uses: ./.github/workflows/ci-linux.yml + + linux-e2e: + needs: build-linux + uses: ./.github/workflows/ci-linux-e2e.yml + + macos: + needs: build-macos + uses: ./.github/workflows/ci-macos.yml + with: + # R7: the known-red legs (#669) run where they can change a decision: on + # main, on dispatch, and on a pull request labelled `macos-27`. + known-red: ${{ github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'macos-27') }} + + macos-e2e: + needs: build-macos + uses: ./.github/workflows/ci-macos-e2e.yml + with: + known-red: ${{ github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'macos-27') }} + + macos-ios: + needs: build-macos + uses: ./.github/workflows/ci-macos-ios.yml + + windows: + needs: build-windows + uses: ./.github/workflows/ci-windows.yml + + windows-e2e: + needs: build-windows + uses: ./.github/workflows/ci-windows-e2e.yml + + windows-msvc-xlings: + needs: build-windows + uses: ./.github/workflows/ci-windows-msvc-xlings.yml + + cross: + needs: [build-linux, build-windows] + uses: ./.github/workflows/cross-build-test.yml + + target-matrix: + needs: [build-linux, build-linux-arm, build-macos, build-windows] + uses: ./.github/workflows/ci-target-matrix.yml + + # Pull requests and dispatch only, as when it was a workflow of its own. + openkal: + needs: [build-linux, build-macos, build-windows] + if: github.event_name != 'push' + uses: ./.github/workflows/openkal-cross.yml + + e2e-coverage: + name: every e2e test runs somewhere + needs: [changes, linux-e2e, windows-e2e, macos-e2e] + if: ${{ always() && needs.changes.outputs.code == 'true' && !cancelled() }} + runs-on: ubuntu-24.04 + timeout-minutes: 10 + steps: + - uses: actions/checkout@v4 + + - name: Fetch the shards' reports + uses: actions/download-artifact@v4 + with: + pattern: e2e-report-* + path: reports + merge-multiple: true + + - name: Every test ran, is run by a dedicated job, or is excused + run: python3 .github/tools/check_e2e_coverage.py --reports reports --timings-out timings + + # The measured durations, merged per host, in the format of + # tests/e2e/timings/; refreshing those tables is copying these files. + - name: Upload the measured durations + if: always() + uses: actions/upload-artifact@v4 + with: + name: e2e-timings + path: timings/ + if-no-files-found: ignore + retention-days: 14 diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 24e26bd2..8133f296 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -52,15 +52,7 @@ name: cross-build-test # xlings-res + xim-pkgindex. on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - workflow_dispatch: - -concurrency: - group: ci-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + workflow_call: jobs: cross-build: @@ -81,21 +73,25 @@ jobs: steps: - uses: actions/checkout@v4 - - name: Cache mcpp sandbox - uses: actions/cache@v4 + # Restored here and saved by this job alone, on main, after its last + # step (rule R3 of the 2026-10-02 CI record). + - name: Restore mcpp sandbox + id: sandbox + uses: actions/cache/restore@v4 with: path: ~/.mcpp key: mcpp-sandbox-${{ runner.os }}-cross-${{ matrix.target }}-${{ hashFiles('mcpp.toml', '.xlings.json') }} restore-keys: | mcpp-sandbox-${{ runner.os }}-cross-${{ matrix.target }}- - - name: Cache xlings - uses: actions/cache@v4 + - name: Restore xlings + id: xlings + uses: actions/cache/restore@v4 with: path: ~/.xlings - key: xlings-${{ runner.os }}-v2-${{ hashFiles('.xlings.json') }} + key: xlings-${{ runner.os }}-cross-v3-${{ hashFiles('.xlings.json') }} restore-keys: | - xlings-${{ runner.os }}-v2- + xlings-${{ runner.os }}-cross-v3- - name: Install qemu-user-static run: | @@ -161,16 +157,9 @@ jobs: echo "XLINGS_BIN=$HOME/.xlings/subos/default/bin/xlings" >> "$GITHUB_ENV" echo "MCPP_BOOT=$MCPP_BOOT" >> "$GITHUB_ENV" - - name: Self-host build (bootstrap mcpp -> fresh host mcpp) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP_BOOT" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP_BOOT" build - MCPP=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - test -x "$MCPP" - "$MCPP" self config --mirror GLOBAL - echo "MCPP=$MCPP" >> "$GITHUB_ENV" + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 - name: "Cross-build mcpp -> ${{ matrix.target }}" run: | @@ -216,6 +205,20 @@ jobs: xver=$($RUN "$XLINGS_XBIN" --version 2>&1 || true) echo "$xver" + - name: Save mcpp sandbox + if: ${{ github.event_name == 'push' && steps.sandbox.outputs.cache-hit != 'true' }} + uses: actions/cache/save@v4 + with: + path: ~/.mcpp + key: ${{ steps.sandbox.outputs.cache-primary-key }} + + - name: Save xlings + if: ${{ github.event_name == 'push' && steps.xlings.outputs.cache-hit != 'true' }} + uses: actions/cache/save@v4 + with: + path: ~/.xlings + key: ${{ steps.xlings.outputs.cache-primary-key }} + # ── Linux → Windows MinGW cross (OS-cross, same arch: ELF→PE) ───────────── # Builds a demo project for x86_64-w64-mingw32 with the from-source GCC-16 # MSVCRT cross toolchain, asserts the artefact is a fully-static PE, and runs @@ -232,32 +235,40 @@ jobs: steps: - uses: actions/checkout@v4 - - name: Cache mcpp sandbox - uses: actions/cache@v4 + # Restored here and saved by this job alone, on main, after its last + # step (rule R3). The Wine packages were evicted with everything else + # while the repository's caches exceeded their limit, and `apt` then + # took 26 to 50 minutes (2026-10-01); with one writer per key they stay. + - name: Restore mcpp sandbox + id: sandbox + uses: actions/cache/restore@v4 with: path: ~/.mcpp key: mcpp-sandbox-${{ runner.os }}-mingw-cross-${{ hashFiles('mcpp.toml', '.xlings.json') }} restore-keys: | mcpp-sandbox-${{ runner.os }}-mingw-cross- - - name: Cache xlings - uses: actions/cache@v4 + - name: Restore xlings + id: xlings + uses: actions/cache/restore@v4 with: path: ~/.xlings - key: xlings-${{ runner.os }}-v2-${{ hashFiles('.xlings.json') }} + key: xlings-${{ runner.os }}-mingw-cross-v3-${{ hashFiles('.xlings.json') }} restore-keys: | - xlings-${{ runner.os }}-v2- + xlings-${{ runner.os }}-mingw-cross-v3- # wine 的包集固定不变 —— 缓存整个 .deb 依赖闭包,命中时跳过 apt update # 与下载(每轮省 ~1-2min)。镜像月度更新可能改变依赖缺口,dpkg -i 失败时 # 由 apt-get -f 兜底并重新回填缓存。 - - name: Cache wine debs - uses: actions/cache@v4 + - name: Restore wine debs + id: wine + uses: actions/cache/restore@v4 with: path: ~/wine-debs key: wine-debs-${{ runner.os }}-ubuntu24.04-v1 - name: Install wine + timeout-minutes: 20 run: | sudo dpkg --add-architecture i386 || true if ls ~/wine-debs/*.deb >/dev/null 2>&1; then @@ -311,15 +322,9 @@ jobs: echo "XLINGS_BIN=$HOME/.xlings/subos/default/bin/xlings" >> "$GITHUB_ENV" echo "MCPP_BOOT=$MCPP_BOOT" >> "$GITHUB_ENV" - - name: Self-host build (fresh host mcpp) - run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP_BOOT" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP_BOOT" build - MCPP=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") - test -x "$MCPP" - "$MCPP" self config --mirror GLOBAL - echo "MCPP=$MCPP" >> "$GITHUB_ENV" + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 - name: Install mingw-cross toolchain run: | @@ -402,6 +407,40 @@ jobs: grep -qxF "$line" "$log" || { echo "::error::797 did not print: $line"; exit 1; } done + # 257 needs a Linux-hosted MinGW compiler AND wine, which only this job + # has; on the shards it skipped, and it had run on no runner (finding F9 + # of the 2026-10-02 CI record). + - name: "e2e: a shared library across the PE boundary (257)" + run: | + set -o pipefail + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + log="$RUNNER_TEMP/257.log" + bash tests/e2e/257_shared_library_pe.sh 2>&1 | tee "$log" + if grep -q '^SKIP' "$log"; then + echo "::error::257 skipped on the job that can run it"; exit 1 + fi + + - name: Save mcpp sandbox + if: ${{ github.event_name == 'push' && steps.sandbox.outputs.cache-hit != 'true' }} + uses: actions/cache/save@v4 + with: + path: ~/.mcpp + key: ${{ steps.sandbox.outputs.cache-primary-key }} + + - name: Save xlings + if: ${{ github.event_name == 'push' && steps.xlings.outputs.cache-hit != 'true' }} + uses: actions/cache/save@v4 + with: + path: ~/.xlings + key: ${{ steps.xlings.outputs.cache-primary-key }} + + - name: Save wine debs + if: ${{ github.event_name == 'push' && steps.wine.outputs.cache-hit != 'true' }} + uses: actions/cache/save@v4 + with: + path: ~/wine-debs + key: ${{ steps.wine.outputs.cache-primary-key }} + # ── windows → linux ─────────────────────────────────────────────────────── # The mirror of mingw-cross-wine. Two jobs because a Windows runner cannot # execute the ELF it produces; the artefact is handed to a Linux job and @@ -412,41 +451,20 @@ jobs: timeout-minutes: 60 steps: - uses: actions/checkout@v4 + # No job restores target/ any more. This one was the first to stop: a + # restored BMI tree beside a fresh one made GCC report + # `import 'std' has CRC mismatch` on the second of its two builds. - uses: ./.github/actions/bootstrap-mcpp + + - uses: ./.github/actions/use-built-mcpp with: - # Do NOT restore target/ in a cross job. This job builds twice — once - # for the host, then once for x86_64-linux-musl — and a restored BMI - # tree makes the second build read `std` BMIs that no longer match - # what the dependency BMIs were compiled against: - # - # mcpplibs.cmdline: error: import 'std' has CRC mismatch - # - # GCC bakes a CRC of each imported module's BMI into the importer, so - # the two have to come from the same build round; a cache that - # restores one without the other is not a partial speedup, it is an - # unbuildable tree. It reproduced on rerun, and only in this job — - # the other two cross jobs below cache ~/.mcpp and ~/.xlings but - # deliberately never target/, which is the convention this now follows. - # - # It stayed hidden until a PR touched neither mcpp.toml nor - # .xlings.json: those two files key the sandbox cache, so every - # earlier run had been a cold miss. - cache-target: 'false' + host: windows-x86_64 - - name: Build mcpp from source (self-host) + - name: Name this commit's mcpp MCPP_SELF shell: bash run: | - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP" build - # Newest, not first: target/ is cache-restored and keeps a directory - # per build fingerprint, so `find | head -1` can hand back the - # PREVIOUS release's binary (that is how a 0.0.106 build ran 0.0.105). - MCPP_SELF=$(find target -name "mcpp.exe" -path "*/bin/*" -printf "%T@ %p\n" \ - | sort -rn | head -1 | cut -d" " -f2-) - test -n "$MCPP_SELF" || { echo "FAIL: no mcpp.exe"; exit 1; } - MCPP_SELF=$(cd "$(dirname "$MCPP_SELF")" && pwd)/$(basename "$MCPP_SELF") - "$MCPP_SELF" --version - echo "MCPP_SELF=$MCPP_SELF" >> "$GITHUB_ENV" + "$MCPP_FRESH" --version + echo "MCPP_SELF=$MCPP_FRESH" >> "$GITHUB_ENV" - name: Install the linux-musl cross toolchain (windows-hosted canadian) shell: bash diff --git a/.github/workflows/measure-windows-tool-crt.yml b/.github/workflows/measure-windows-tool-crt.yml index f3a95551..c3a06c92 100644 --- a/.github/workflows/measure-windows-tool-crt.yml +++ b/.github/workflows/measure-windows-tool-crt.yml @@ -104,8 +104,6 @@ jobs: if ($left) { Write-Host "FAIL: VC tools still present after masking"; exit 1 } - uses: ./.github/actions/bootstrap-mcpp - with: - cache-target: 'false' - name: Install xim:qt-base 6.11.1 and remove its copy of the runtime shell: bash diff --git a/.github/workflows/openkal-cross.yml b/.github/workflows/openkal-cross.yml index dcd4923b..aed6e099 100644 --- a/.github/workflows/openkal-cross.yml +++ b/.github/workflows/openkal-cross.yml @@ -55,17 +55,7 @@ name: openkal cross-build (3 hosts × 3 targets) # means the unwinder found this image's own frame descriptions. on: - pull_request: - workflow_dispatch: - inputs: - openkal_ref: - description: "Branch of the openkal packages to build against (empty = main)" - required: false - default: "" - -concurrency: - group: ci-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + workflow_call: env: # No mcpp or xlings version here. `bootstrap-mcpp` owns both, and a second @@ -138,7 +128,22 @@ jobs: # THE mcpp UNDER TEST. Everything after this uses the binary this step # produces; the bootstrapped one above is only what compiles it. + # This commit's mcpp: the one build.yml produced (use-built-mcpp, rule R1 + # of the 2026-10-02 CI record), except on the macOS leg. That leg runs on + # macos-14 and the macOS artifact is built on macos-15; whether it runs on + # macos-14 is not measured (the record's Part VII), so the leg builds its + # own until it is. + - if: matrix.host != 'macos' + uses: ./.github/actions/use-built-mcpp + with: + host: ${{ matrix.host == 'linux' && 'linux-x86_64' || 'windows-x86_64' }} + + - name: Name this commit's mcpp MCPP_UNDER_TEST + if: matrix.host != 'macos' + run: echo "MCPP_UNDER_TEST=$MCPP_FRESH" >> "$GITHUB_ENV" + - name: Build the mcpp in this pull request + if: matrix.host == 'macos' # ci-lint: allow-r1: the macOS leg builds its own mcpp until the artifact is measured on macos-14 run: | set -euo pipefail export MCPP_VENDORED_XLINGS="$XLINGS_BIN" @@ -481,26 +486,12 @@ jobs: - uses: actions/checkout@v4 - uses: ./.github/actions/bootstrap-mcpp - - name: Build the mcpp in this pull request - run: | - set -euo pipefail - export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true - "$MCPP" self config --mirror GLOBAL 2>/dev/null || true - "$MCPP" build --dev - # NEWEST BY MTIME, NOT FIRST BY DIRECTORY ORDER. `target/` holds one - # directory per fingerprint and the runner restores a cache of it, so - # `find … | head -1` can return a binary an earlier run left behind. - # Measured: it reported the right VERSION STRING — the stale copy was - # built from an earlier push of this same release — while missing the - # last two commits, so one new test passed and two failed for reasons - # that were nowhere in the source. - BUILT=$(find target -type f -name 'mcpp' -printf '%T@ %p\n' \ - | sort -rn | head -1 | cut -d' ' -f2) - [ -n "$BUILT" ] || { echo "::error::mcpp did not build"; exit 1; } - BUILT=$(cd "$(dirname "$BUILT")" && pwd)/$(basename "$BUILT") - echo "MCPP_UNDER_TEST=$BUILT" >> "$GITHUB_ENV" - "$BUILT" --version + - uses: ./.github/actions/use-built-mcpp + with: + host: linux-x86_64 + + - name: Name this commit's mcpp MCPP_UNDER_TEST + run: echo "MCPP_UNDER_TEST=$MCPP_FRESH" >> "$GITHUB_ENV" - name: Install what the scripts declare run: | diff --git a/tests/e2e/coverage-exceptions.tsv b/tests/e2e/coverage-exceptions.tsv new file mode 100644 index 00000000..13a168b7 --- /dev/null +++ b/tests/e2e/coverage-exceptions.tsv @@ -0,0 +1,4 @@ +# Tests that no hosted runner can run, read by .github/tools/check_e2e_coverage.py. +# One `\t` line each. A test that runs on some shard, or that a +# workflow names, does not belong here, and the check fails if it is listed. +658_a_program_runs_on_an_attached_android_device.sh needs an Android device attached over adb; the android job of ci-linux-e2e.yml runs `mcpp test` on an emulator instead diff --git a/tests/e2e/run_all.sh b/tests/e2e/run_all.sh index a928404c..57c6f3dc 100755 --- a/tests/e2e/run_all.sh +++ b/tests/e2e/run_all.sh @@ -65,20 +65,34 @@ case "$OS" in CAPS+=(elf unix-shell fresh-sandbox) command -v g++ &>/dev/null && CAPS+=(gcc) command -v patchelf &>/dev/null && CAPS+=(patchelf) - # musl-gcc: check both system PATH and xlings-managed locations + # musl-gcc: check both system PATH and xlings-managed locations. + # + # ANY RELEASE OF THE FAMILY, NOT ONE VERSION. This probe named + # 15.1.0, and the mingw-cross probe below named 16.1.0: a capability + # that names a release turns every test that declares it into a silent + # skip the day the default moves (SPEC-009, check C3). if command -v x86_64-linux-musl-g++ &>/dev/null \ - || [[ -x "$HOME/.xlings/data/xpkgs/xim-x-musl-gcc/15.1.0/bin/x86_64-linux-musl-g++" ]] \ - || [[ -x "${MCPP_HOME}/registry/data/xpkgs/xim-x-musl-gcc/15.1.0/bin/x86_64-linux-musl-g++" ]]; then + || ls "$HOME"/.xlings/data/xpkgs/xim-x-musl-gcc/*/bin/x86_64-linux-musl-g++ 2>/dev/null | head -1 | grep -q . \ + || ls "${MCPP_HOME}"/registry/data/xpkgs/xim-x-musl-gcc/*/bin/x86_64-linux-musl-g++ 2>/dev/null | head -1 | grep -q .; then CAPS+=(musl) fi # mingw-cross: the Linux-hosted MinGW-w64 cross toolchain (xim # mingw-cross-gcc, GCC 16 MSVCRT). Must be the xim-managed GCC-16 build, # NOT the distro apt g++-mingw-w64 (GCC 13 — no `import std`). Probe the # xlings/mcpp payload location, mirroring the musl probe above. - if [[ -x "$HOME/.xlings/data/xpkgs/xim-x-mingw-cross-gcc/16.1.0/bin/x86_64-w64-mingw32-g++" ]] \ - || [[ -x "${MCPP_HOME}/registry/data/xpkgs/xim-x-mingw-cross-gcc/16.1.0/bin/x86_64-w64-mingw32-g++" ]]; then + if ls "$HOME"/.xlings/data/xpkgs/xim-x-mingw-cross-gcc/*/bin/x86_64-w64-mingw32-g++ 2>/dev/null | head -1 | grep -q . \ + || ls "${MCPP_HOME}"/registry/data/xpkgs/xim-x-mingw-cross-gcc/*/bin/x86_64-w64-mingw32-g++ 2>/dev/null | head -1 | grep -q .; then CAPS+=(mingw-cross) fi + # llvm: an LLVM payload in the home mcpp uses. It was a legal token that + # no line granted, so the seven tests that need nothing else (134-137, + # 741, 875, 876) ran on no runner while their skip lines read like + # legitimate ones (finding F9 of the 2026-10-02 CI record). Granted on + # Linux only: the macOS runner wires its LLVM into the sandbox for the + # build, and the tests that declare `llvm` were written for an ELF host. + if ls "${MCPP_HOME}"/registry/data/xpkgs/xim-x-llvm/*/bin/clang++ 2>/dev/null | head -1 | grep -q .; then + CAPS+=(llvm) + fi # mingw-host-headers: this Linux HOST's own mingw-w64 headers # (`apt install mingw-w64`, distro package). Distinct from both # `mingw-cross` above (an xim-managed cross GCC) and `mingw` below (a @@ -395,10 +409,80 @@ if [[ -n "${E2E_SHARD:-}" ]]; then echo "FATAL: bad E2E_SHARD='$E2E_SHARD' (want /, 1-based)" exit 1 fi - echo "Shard: ${SHARD_IDX}/${SHARD_TOTAL} (round-robin over the suite)" fi SHARD_POS=0 +# BY MEASURED DURATION WHEN A TIMING TABLE IS GIVEN (rule R4 of the 2026-10-02 +# CI record). Round-robin balances counts, not time: measured on 2026-10-01 the +# two Windows shards differed by 3.6 minutes on average, and about fifteen tests +# of over thirty seconds took 58-64 percent of a shard. E2E_TIMINGS names a +# table of `\t` lines (tests/e2e/timings/.tsv); the +# tests are assigned longest first, each to the shard with the least time so +# far, and a test the table does not know counts as the table's median. The +# assignment depends only on the file list and the table, both in the +# repository, so a shard's membership is reproducible from the commit, and it +# changes which shard runs a test, never whether one does. +SHARD_MEMBERS="" +if (( SHARD_TOTAL > 1 )) && [[ -n "${E2E_TIMINGS:-}" ]]; then + if [[ ! -r "$E2E_TIMINGS" ]]; then + echo "FATAL: E2E_TIMINGS='$E2E_TIMINGS' is not readable" + exit 1 + fi + SHARD_MEMBERS="$( + for t in "$HERE"/[0-9]*.sh; do basename "$t"; done \ + | awk -v tf="$E2E_TIMINGS" ' + BEGIN { FS = "\t"; n = 0 + while ((getline line < tf) > 0) { + split(line, f, "\t"); if (f[2] ~ /^[0-9]+$/) { ms[f[1]] = f[2] + 0; v[++n] = f[2] + 0 } } + # median of the known durations + for (i = 1; i <= n; i++) for (j = i + 1; j <= n; j++) if (v[j] < v[i]) { x = v[i]; v[i] = v[j]; v[j] = x } + med = n ? v[int((n + 1) / 2)] : 1000 } + { w = ($0 in ms) ? ms[$0] : med; printf "%d\t%s\n", w, $0 }' \ + | sort -t "$(printf '\t')" -k1,1nr -k2,2 \ + | awk -v total="$SHARD_TOTAL" -v me="$SHARD_IDX" ' + BEGIN { FS = "\t"; for (i = 1; i <= total; i++) load[i] = 0 } + { best = 1; for (i = 2; i <= total; i++) if (load[i] < load[best]) best = i + load[best] += $1; if (best == me) print $2 }')" + echo "Shard: ${SHARD_IDX}/${SHARD_TOTAL} (by measured duration, $E2E_TIMINGS)" +elif (( SHARD_TOTAL > 1 )); then + echo "Shard: ${SHARD_IDX}/${SHARD_TOTAL} (round-robin over the suite)" +fi +in_shard() { + local name="$1" + if [[ -n "$SHARD_MEMBERS" ]]; then + grep -qxF "$name" <<<"$SHARD_MEMBERS" + return + fi + local mine=$(( (SHARD_POS % SHARD_TOTAL) + 1 )) + SHARD_POS=$(( SHARD_POS + 1 )) + (( mine == SHARD_IDX )) +} + +# What each test of this run did, for the coverage check of ci.yml +# (.github/tools/check_e2e_coverage.py): one `\t\t\t` +# line per test of this shard, status pass, fail, timeout or skip. +report() { + [[ -n "${E2E_REPORT:-}" ]] || return 0 + printf '%s\t%s\t%s\t%s\n' "$1" "$2" "$3" "$4" >> "$E2E_REPORT" +} +if [[ -n "${E2E_REPORT:-}" ]]; then : > "$E2E_REPORT"; fi + +# E2E_LIST=1 prints the tests this invocation would run (after E2E_ONLY and the +# shard, before the capability check) and exits, so a shard's membership can be +# read without running it. +if [[ "${E2E_LIST:-}" == "1" ]]; then + for test in "$HERE"/[0-9]*.sh; do + name="$(basename "$test")" + if [[ -n "${E2E_ONLY:-}" ]]; then + # shellcheck disable=SC2053 -- glob match is the point + [[ "$name" == $E2E_ONLY ]] || continue + fi + if (( SHARD_TOTAL > 1 )); then in_shard "$name" || continue; fi + echo "$name" + done + exit 0 +fi + # Optional name filter: E2E_ONLY="" runs just the matching tests. # Used by the workflows that own ONE subject (see ci-windows-msvc-xlings.yml) # so the job name and the thing it runs cannot drift apart. @@ -407,17 +491,16 @@ for test in "$HERE"/[0-9]*.sh; do # shellcheck disable=SC2053 — glob match is the point [[ "$(basename "$test")" == $E2E_ONLY ]] || continue fi + name="$(basename "$test")" if (( SHARD_TOTAL > 1 )); then - _mine=$(( (SHARD_POS % SHARD_TOTAL) + 1 )) - SHARD_POS=$(( SHARD_POS + 1 )) - (( _mine == SHARD_IDX )) || continue + in_shard "$name" || continue fi - name="$(basename "$test")" echo missing_cap="$(check_requires "$test")" if [[ -n "$missing_cap" ]]; then echo "SKIP: $name (missing capability: $missing_cap)" ((SKIP++)) + report skip "$name" 0 "missing capability: $missing_cap" continue fi echo "=== $name ===" @@ -434,13 +517,16 @@ for test in "$HERE"/[0-9]*.sh; do if [[ $rc -eq 0 ]]; then echo "PASS: $name (${_dur})" ((PASS++)) + report pass "$name" "$_dur_ms" "" elif [[ $rc -eq 124 ]]; then + report timeout "$name" "$_dur_ms" "exceeded ${E2E_TEST_TIMEOUT}s" # GNU timeout: 124 = killed after deadline (TERM); 137 = SIGKILL after grace. echo "TIMEOUT: $name (exceeded ${E2E_TEST_TIMEOUT}s — likely network / xlings stall)" ((FAIL++)) FAILED_TESTS+=("$name (TIMEOUT)") TIMED_OUT_TESTS+=("$name") else + report fail "$name" "$_dur_ms" "exit $rc" echo "FAIL: $name (exit $rc, ${_dur})" ((FAIL++)) FAILED_TESTS+=("$name (exit $rc)") diff --git a/tests/e2e/timings/linux.tsv b/tests/e2e/timings/linux.tsv new file mode 100644 index 00000000..494f966b --- /dev/null +++ b/tests/e2e/timings/linux.tsv @@ -0,0 +1,466 @@ +00_fixture_path_hygiene.sh 2070 +01_help_and_version.sh 12 +02_new_build_run.sh 1520 +03_multi_module.sh 2160 +04_incremental.sh 3200 +05_errors.sh 1260 +06_emit_xpkg.sh 34 +07_static_library.sh 133 +08_shared_library.sh 547 +09_path_dependency.sh 2190 +100_cppfly_reflection.sh 13390 +100_feature_sources_test_mode.sh 3800 +101_cppfly_llvm_soft.sh 8 +103_target_vocabulary.sh 5680 +104_asm_sources_gas.sh 2680 +106_feature_gated_sources_toml.sh 6500 +107_generated_files_toml.sh 1310 +108_cfg_conditional_sources.sh 1240 +109_per_glob_flags.sh 2100 +10_env_command.sh 8810 +110_build_mcpp_env_contract.sh 7030 +111_dep_build_mcpp.sh 2290 +113_scanner_mcpp_dir_prune.sh 177 +114_run_scan_scope.sh 1580 +115_manifest_syntax_aot_brace.sh 1240 +116_include_family_and_define_spaces.sh 1020 +117_object_path_collision.sh 1240 +118_purview_include_rebuild.sh 3820 +119_dep_cfg_sources.sh 1420 +11_index_list.sh 9920 +120_ws_root_indices.sh 696 +121_default_ns_redirect.sh 678 +122_run_member.sh 2260 +123_same_named_main_across_dep.sh 1330 +124_gen_source_out_of_root_object_path.sh 1750 +125_build_mcpp_dep_dir.sh 1730 +126_default_features_opt_out.sh 392 +127_transitive_default_features_optout.sh 2050 +128_feature_forwarding.sh 274 +12_add_command.sh 868 +138_feature_sources_gate_vs_provider.sh 582 +139_build_program_advisory.sh 978 +13_toolchain_pin.sh 7 +141_include_dirs_after.sh 1170 +142_generated_files_incremental.sh 2430 +143_build_mcpp_source_directive.sh 2230 +144_build_mcpp_include_dir.sh 3640 +145_root_build_mcpp_dep_dirs.sh 1560 +146_feature_flags.sh 7850 +147_per_os_features_xpkg.sh 639 +148_wide_include_list.sh 1410 +149_conditional_glob_flags.sh 1110 +14_toolchain_fallback.sh 5760 +150_clang_module_operator_template.sh 6 +151_index_url_artifact_migration.sh 6040 +152_test_subdir_names.sh 242 +153_test_isolation.sh 226 +154_test_filter.sh 516 +155_test_json.sh 193 +156_nested_ld_library_path.sh 112 +157_test_glob_flags.sh 135 +158_test_signal_and_duration.sh 473 +159_test_list.sh 18 +15_test_passing.sh 2390 +160_test_timeout.sh 4640 +161_xpkg_name_form.sh 64 +162_bare_name_namespace_scope.sh 5830 +163_identity_first_resolution.sh 26190 +164_ninja_goal_command_length.sh 532 +165_bare_name_cross_namespace_wire_address.sh 5650 +166_run_env_no_private_glibc.sh 345 +167_build_defines_module_scan.sh 653 +168_build_mcpp_musl_host_static.sh 5320 +169_semver_project_index.sh 434 +170_bmi_staging_no_cascade.sh 1110 +171_bmi_staging_locked_dest.sh 2530 +172_build_cache_cross_project.sh 7280 +173_build_profile_isolation.sh 943 +173_index_refresh_policy.sh 598 +174_cache_modes_and_commands.sh 17720 +175_cpp20_import_std.sh 6130 +176_cpp20_std_bmi_isolation.sh 8370 +178_test_observability.sh 10780 +179_spaced_paths.sh 1070 +17_test_no_tests.sh 23 +181_build_mcpp_import_std.sh 6150 +183_cxx_runtime_contract.sh 3450 +184_build_cache_object_layout.sh 12980 +185_index_floor_degrades.sh 4330 +186_build_mcpp_protocol_and_bound.sh 7600 +187_dep_host_tool.sh 3310 +188_build_actions.sh 8710 +189_host_module_rules.sh 1700 +18_devdeps_isolation.sh 1220 +190_link_rspfile_newlines.sh 628 +191_link_scale.sh 18600 +192_install_path_namespace.sh 3630 +193_provision_reexport.sh 8180 +194_build_mcpp_glob_input.sh 2440 +195_target_cfg_feature_deps.sh 406 +196_version_identity_and_lock.sh 15370 +199_resources_validation.sh 207 +19_bmi_cache_reuse.sh 5800 +200_subos_env_reaches_program.sh 1780 +201_gcc_no_specs_pollution.sh 6720 +202_machine_output_contract.sh 490 +203_exact_selector_lock_migration.sh 387 +204_new_transactional_scaffold.sh 8990 +205_root_local_subos.sh 764 +206_runtime_binding_physics.sh 1150 +207_runtime_contract_provenance.sh 204 +208_private_libc_stays_in_the_binary.sh 299 +209_runtime_verdict_never_reruns_green.sh 750 +20_p1689_scanner.sh 4720 +210_local_index_addressing_on_every_host.sh 185 +211_configure_only_cdb.sh 470 +212_cached_dep_std_is_ordered.sh 4820 +213_build_after_test_is_not_the_test_graph.sh 388 +214_executable_carries_dt_rpath.sh 1880 +215_pack_has_no_build_machine_paths.sh 3430 +216_selfcontained_refuses_host_capability.sh 763 +217_module_extensions.sh 4750 +218_module_extensions_graph_shape.sh 5140 +219_runtime_search_farm_is_last.sh 370 +21_ninja_dyndep.sh 4050 +220_farm_only_library_runs.sh 432 +221_subos_without_info_still_builds.sh 140 +222_shared_library_cxx_runtime_contract.sh 2010 +22_doctor_cache_publish.sh 45720 +230_bench_harness.sh 54310 +231_jobs_option.sh 56340 +232_workflow_syntax.sh 323 +233_bench_matrix.sh 559 +234_bmi_schedule_on.sh 6460 +235_std_object_only_when_needed.sh 977 +236_module_extensions_default_toolchain.sh 3300 +237_default_subos_without_info.sh 256 +238_c_only_unit_links_with_c_driver.sh 3660 +23_remove_update.sh 16070 +242_pack_library_interface_and_headers.sh 609 +243_pack_library_interface_closure.sh 665 +244_pack_library_gate.sh 458 +246_explicit_empty_sources.sh 302 +247_bare_triple_conditional_native.sh 390 +249_pack_workspace_root_unchanged.sh 195 +24_git_dependency.sh 9290 +250_pack_names_the_target.sh 256 +251_pack_library_shared.sh 349 +252_pack_library_old_client.sh 309 +253_pack_library_undetermined_partition.sh 490 +254_pack_library_platform_coverage.sh 498 +25_convention_mode.sh 4460 +260_module_extension_is_configuration.sh 463 +261_pack_follows_the_module_extension.sh 463 +263_lib_root_follows_the_extension.sh 1230 +264_pack_library_is_relocatable.sh 397 +265_pack_strips_but_stays_usable.sh 916 +267_target_requested_is_empty_for_a_native_build.sh 746 +268_target_side_resolution.sh 218 +26_c_language_support.sh 1050 +26_toolchain_management.sh 40240 +27_namespace_dependencies.sh 6800 +27_self_contained_home.sh 11520 +280_target_side_layers.sh 2130 +281_target_side_rules.sh 212 +282_target_is_a_request.sh 64 +283_run_target_flag_owns_its_cache_slot.sh 305 +284_env_segment_is_optional_everywhere.sh 13840 +285_kernel_abi_from_graph_keeps_the_payload_c_library.sh 4500 +290_the_declaration_puts_an_environment_in_front.sh 1790 +293_the_requested_target_and_the_resolved_one_name_one_os.sh 7510 +295_naming_the_host_target_changes_nothing.sh 1440 +296_what_the_report_names_is_what_the_link_line_uses.sh 95750 +297_a_capability_pin_is_not_a_preference.sh 193 +299_a_request_that_named_no_c_library_resolves_to_a_row_that_exists.sh 92 +29_toolchain_partial_versions.sh 5700 +300_a_registered_family_is_not_reported_unknown.sh 36 +301_the_graphs_compiler_is_taken_and_nothing_is_written.sh 277 +302_a_stated_compiler_outranks_the_graph_and_two_requirements_do_not_stack.sh 95 +303_the_query_gives_one_answer_for_the_c_library.sh 103 +304_the_target_side_reaches_every_unit.sh 158 +305_built_from_is_not_published.sh 204 +306_dependency_linkage_axis.sh 346 +307_symbol_provision.sh 503 +308_dependency_required_features.sh 222 +309_host_module_identity.sh 950 +30_dev_binary_home.sh 19 +30_pack_modes.sh 5700 +310_build_dependencies.sh 1980 +311_rule_build_dependencies.sh 2770 +312_build_rules_example.sh 2960 +313_check_stamp_on_every_platform.sh 594 +314_dependency_action_generated_header.sh 2680 +315_blocking_check_gates_compilation.sh 4940 +316_link_unit_with_no_inputs_is_refused.sh 194 +317_project_build_hooks.sh 18670 +321_workspace_inheritance.sh 692 +322_dialect_flag_refusal.sh 9590 +323_post_link_record_durability.sh 425 +324_fast_path_sees_path_deps.sh 3630 +325_system_toolchain_refused.sh 176 +326_dependency_standard_floor.sh 834 +327_declared_xlings_deps_report_their_own_failure.sh 8800 +328_layer_predicates_apply_and_unknown_ones_say_so.sh 1540 +329_exit_codes_match_the_contract.sh 19 +32_semver_merge.sh 8710 +330_runner_hosted_targets.sh 1310 +331_xlings_workspace_one_table.sh 1810 +333_named_runners_locked_and_sbom.sh 4610 +334_dependency_declared_tool_is_found.sh 242 +335_xlings_tool_tiers.sh 32 +337_run_takes_features_and_profile.sh 1050 +339_run_reports_the_program_status.sh 1240 +33_multi_version_mangling.sh 13100 +340_no_orphan_survives_a_killed_mcpp.sh 11100 +35_workspace.sh 1860 +36_llvm_toolchain.sh 992 +38_self_config_mirror.sh 217 +39_xlings_index_migration.sh 209 +42_custom_local_index.sh 9340 +43_indices_lockfile.sh 7070 +44_indices_e2e_integration.sh 9270 +45_install_platform_mapping.sh 13 +46_self_config_mirror_no_bootstrap.sh 8070 +47_cdb_prebuilt_module_path_abs.sh 1210 +48_build_error_output.sh 284 +49_bmi_cache_nested_custom_index.sh 5420 +50_package_owned_build_flags.sh 182 +51_package_generated_files.sh 293 +52_local_path_namespaced_index.sh 477 +53_namespaced_cache_label.sh 398 +54_package_owned_ldflags.sh 268 +55_dependency_shared_artifact.sh 194 +56_transitive_shared_artifact.sh 232 +57_static_dep_shared_artifact.sh 205 +58_preinstall_mcpp_deps_for_hooks.sh 285 +59_cpp_standard_config.sh 5670 +600_accel_variant_selection.sh 415 +601_exclusive_capability.sh 211 +603_version_floor.sh 299 +604_offline_skips_sandbox_bootstrap.sh 366 +605_probe_channel_and_accel_reach_the_build_program.sh 1330 +606_constrained_source_globs_narrow_to_the_build.sh 1240 +607_chained_actions_form_the_device_link.sh 1100 +608_object_actions_reach_a_static_library.sh 785 +609_clean_stale.sh 4400 +609_shader_sources_are_device_sources.sh 1920 +60_stale_xpkg_cache_reinstall.sh 316 +610_feature_controlled_rule_collection.sh 4120 +611_variant_switch_is_not_replayed.sh 622 +612_build_program_file_input_fast_path.sh 1300 +613_sycl_sources_are_device_sources.sh 1720 +614_feature_xlings_reaches_xpkg_dir.sh 1160 +615_feature_deps_shared_library_is_linked.sh 470 +616_examples_and_their_index_agree.sh 242 +617_build_program_knows_the_cxx_stdlib.sh 550 +618_dependency_feature_tools_reach_its_build_program.sh 43940 +619_a_tool_may_not_be_conditioned_on_a_layer.sh 28 +61_dependency_visibility.sh 402 +620_link_flag_reaches_the_link_line.sh 548 +621_exports_narrows_the_symbol_surface.sh 591 +622_rule_declared_payload_reaches_the_consumers_build_program.sh 48530 +623_device_sources_must_reach_an_action.sh 263 +624_a_missing_host_module_names_the_key.sh 295 +625_a_payload_can_be_gated_on_the_accelerator.sh 43450 +626_naming_a_subset_of_backends_is_not_a_mismatch.sh 421 +627_one_package_one_version.sh 41240 +628_a_pin_below_a_stated_floor_is_refused.sh 43980 +629_a_version_range_is_answered_by_xpkg_dir.sh 49180 +62_dotted_dependency_selector_priority.sh 380 +62_runtime_library_dirs.sh 595 +630_a_rule_brings_its_own_environment.sh 44040 +631_action_depfile_tracks_a_runtime_discovered_include.sh 1110 +632_a_build_program_is_told_its_package_name.sh 546 +633_host_modules_are_ordered_by_what_they_import.sh 1660 +634_a_private_module_fragment_is_not_a_partition.sh 127 +635_a_byte_order_mark_does_not_hide_a_module.sh 326 +636_a_module_extension_file_need_not_provide_a_module.sh 163 +637_a_conditional_runtime_table_alone_is_applied.sh 730 +638_pack_format_dispatch.sh 5620 +639_the_scanner_does_not_read_inside_a_comment.sh 888 +63_bare_dependency_peer_root_priority.sh 356 +640_a_capability_pin_explains_its_own_row.sh 821 +641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh 90540 +644_windows_keys_are_inert_elsewhere.sh 847 +645_the_fast_path_compares_the_toolchain_request.sh 678 +646_runtime_deploy_places_files_in_a_directory.sh 1630 +647_abi_threads_is_one_switch_for_the_graph.sh 646 +648_an_install_hook_sees_the_build_target.sh 874 +649_pack_carries_runtime_deploy_files.sh 4650 +64_shared_soname_runtime_alias.sh 774 +650_wasm_row_names_its_launcher.sh 20 +651_a_build_program_deploys_what_it_generated.sh 1740 +652_an_application_is_a_kind.sh 937 +653_exceptions_is_the_second_abi_member.sh 6 +654_a_requirement_on_the_target_axis.sh 1000 +656_run_hands_the_distributable_to_the_runner.sh 2650 +657_a_build_program_under_the_web_row_uses_the_host_compiler.sh 3 +659_an_action_declaration_has_no_size_limit.sh 1290 +660_an_action_names_the_engine_through_mcpp_self.sh 1620 +661_one_identity_two_declarations.sh 744 +662_pack_stages_declared_files_before_the_closure.sh 5930 +665_a_host_tool_follows_its_source.sh 7930 +669_an_elf_host_pack_states_its_closure.sh 822 +66_runtime_provides.sh 1060 +670_an_rpath_loader_token_is_left_as_written.sh 164 +671_toolchain_is_an_option_of_run_test_and_pack.sh 5530 +672_a_runner_receives_the_files_the_artifact_carries.sh 9530 +673_test_discover_names_where_tests_are.sh 1080 +674_a_format_is_run_by_the_runner_named_after_it.sh 7840 +677_a_conditional_dependency_replaces_the_unconditional_one.sh 691 +678_a_row_states_a_library_form.sh 1280 +679_a_path_dependency_takes_its_manifests_identity.sh 619 +67_features_strict.sh 6560 +681_index_repos_reach_an_existing_home.sh 8070 +682_why_deps_prints_the_resolved_graph.sh 204 +683_a_build_program_reads_the_payload_pkg_config_view.sh 575 +684_a_runner_supplied_twice_is_refused.sh 517 +685_a_nested_test_loads_a_graph_shared_library.sh 397 +686_an_activated_xlings_subos_does_not_redirect_the_registry.sh 7720 +687_the_vendored_xlings_probe_is_an_argument_vector.sh 1220 +688_emit_build_database.sh 4810 +689_pack_takes_features.sh 1680 +68_profile_passthrough.sh 7460 +692_a_package_states_its_default_link_form.sh 1190 +693_the_root_build_program_reads_dependency_link_forms.sh 3420 +698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 3940 +69_package_templates.sh 11110 +701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh 1430 +702_a_static_package_under_a_shared_library_is_linked_into_it.sh 1260 +70_per_target_flags.sh 221 +710_a_forward_reaches_every_dependency_table.sh 422 +711_a_feature_deps_restatement_names_one_source.sh 4890 +712_a_tool_package_contributes_nothing_to_its_consumer.sh 5930 +713_a_git_dependency_selects_a_repository_member.sh 5390 +714_features_names_a_dependency_feature.sh 5740 +71_required_features_gate.sh 303 +720_the_root_build_program_reads_the_resolved_graph.sh 2680 +722_pack_strips_what_the_graph_built.sh 3670 +723_pack_reports_what_it_produced.sh 960 +72_test_profile_features.sh 336 +730_bare_compat_dependency_is_not_a_refresh_miss.sh 74 +731_planning_children_do_not_inherit_the_callers_pipe.sh 444 +732_xlings_refresh_is_bounded_and_owned.sh 4990 +733_envelope_reports_observed_network_access.sh 323 +734_auto_refresh_false_governs_first_index_sync.sh 107 +735_offline_missing_package_has_its_own_code.sh 101 +736_compile_flag_words_reach_the_compiler_and_the_databases.sh 346 +737_the_declared_runtime_payload_is_installed.sh 66950 +739_a_private_feature_dep_does_not_reach_the_consumer.sh 282 +73_issue131_per_target_cxxflag.sh 195 +740_gcc_over_a_graph_c_library_is_degraded_not_refused.sh 113 +743_kernel_abi_interfaces_are_resolved_not_preprocessed.sh 186 +744_a_declared_absence_explains_the_link_that_asked_for_it.sh 90 +745_no_run_builds_the_tests_and_says_so.sh 411 +746_macos_deployment_target_is_target_keyed_not_host_keyed.sh 233 +74_run_no_loader_env_leak.sh 1060 +75_index_status_offline.sh 7160 +765_a_consumer_include_directory_stays_in_the_consumer.sh 611 +76_compile_commands_generated.sh 1020 +76_qualified_custom_ns_noncanonical_filename.sh 457 +770_workspace_member_as_dependency.sh 996 +772_a_published_member_is_self_contained.sh 5520 +773_commands_outside_the_build_read_the_effective_manifest.sh 275 +774_an_index_member_inherits_its_archive_workspace.sh 319 +775_a_host_tool_merges_its_conditional_sections_once.sh 4270 +776_a_path_with_no_utf8_spelling_is_named.sh 308 +777_c_standard_applies_to_the_package_that_declares_it.sh 176 +779_a_build_program_declares_a_runtime_search_dir.sh 1810 +77_cdb_preserves_test_entries.sh 4460 +780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 1730 +781_deleted_root_cdb_is_restored.sh 2280 +782_foreign_root_cdb_entries_are_replaced.sh 2970 +787_emit_plans_every_member_on_its_own.sh 161 +788_emit_defers_an_unbuilt_host_tool.sh 996 +789_emit_describes_a_member_past_a_failed_build_program.sh 326 +78_test_main_combinations.sh 9330 +790_a_prepare_action_populates_an_unknown_directory.sh 1810 +791_a_prepare_action_that_creates_nothing_fails.sh 745 +792_a_rerun_input_inside_a_prepare_directory_is_warned.sh 1700 +793_an_unknown_action_role_is_refused.sh 2310 +795_a_link_flag_reaches_the_linker_as_written.sh 896 +798_a_rule_applies_through_a_source_it_claims.sh 13490 +799_an_action_runs_with_its_env_and_cwd.sh 828 +79_gtest_regular_dep_feature_main.sh 14480 +800_a_feature_provides_its_host_tools.sh 836 +801_a_dependency_program_is_shipped_with_the_consumer.sh 1800 +802_a_host_build_applies_its_host_row.sh 1330 +803_a_recorded_payload_that_is_gone_is_refused_offline.sh 102 +804_a_path_host_tool_builds_with_its_chosen_toolchain.sh 983 +805_rooted_workspace_reaches_its_own_path_dependency.sh 475 +806_dash_p_resolves_the_package_first.sh 799 +807_a_lib_root_imports_its_own_package.sh 506 +808_device_source_is_not_a_compile_unit.sh 11630 +809_failed_build_program_hides_no_device_source.sh 84 +80_feature_defines.sh 296 +810_two_packages_deploy_the_same_bytes_to_one_name.sh 1950 +812_an_index_floor_is_a_closing_tip.sh 206 +813_conditional_dialect_cxxflags_root_only.sh 18140 +815_the_database_describes_what_a_rule_generates.sh 780 +816_a_git_dependency_reports_its_fetch.sh 194 +817_emit_writes_no_xlings_json_under_private_work_dir.sh 211 +819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 316 +81_capability_binding.sh 559 +821_an_edge_states_its_advice_on_both_paths.sh 1590 +822_a_package_states_the_oldest_mcpp_it_supports.sh 267 +823_mcpp_core_and_mcpp_name_one_interface.sh 763 +824_a_build_program_reads_the_resolved_toolchain.sh 1030 +826_plugin_diagnostics_features_and_names.sh 666 +827_many_placements_run_as_one_process.sh 2290 +828_a_library_states_its_interface.sh 648 +829_pack_takes_a_member.sh 274 +82_feature_optional_deps.sh 259 +831_a_path_dependency_host_module_outside_src.sh 4380 +832_the_fast_path_resumes_after_an_edit.sh 3080 +833_a_workspace_is_one_graph_per_configuration.sh 1660 +834_a_workspace_plans_once_and_replays_in_one_check.sh 1290 +835_a_shared_library_is_placed_by_a_link.sh 297 +836_a_members_own_statements_reach_its_build.sh 1300 +838_a_program_shipped_through_artifacts_links_its_own_closure.sh 731 +839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 679 +83_feature_defines_propagate.sh 211 +840_a_workspace_database_describes_each_package_once.sh 551 +841_pack_summarizes_many_outputs.sh 3010 +842_the_build_reports_each_step_once.sh 7190 +843_a_terminal_names_the_action_that_runs.sh 4640 +844_a_planned_build_keeps_one_lock_entry_per_identity.sh 790 +845_play_game_restores_the_terminal.sh 12110 +846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 744 +847_a_module_name_is_unique_within_a_program.sh 2000 +849_a_staged_bmi_waits_for_a_module_compiled_here.sh 1470 +84_runtime_dll_deploy.sh 159 +850_a_selected_member_declares_as_the_root.sh 704 +851_a_shared_member_is_compiled_once_per_configuration.sh 521 +852_a_repeated_dash_p_selects_every_member_it_names.sh 334 +853_run_keeps_one_member_and_exclude_removes_members.sh 721 +854_a_test_over_several_members_plans_and_builds_once.sh 1370 +855_a_shared_test_build_is_reported_once_per_group.sh 345 +856_a_test_over_several_members_continues_past_a_failing_one.sh 1620 +857_a_host_module_several_members_import_is_compiled_once.sh 796 +858_the_engines_module_is_compiled_once_per_home.sh 4800 +859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 3170 +85_target_cfg_build_flags.sh 197 +860_a_workspaces_programs_compile_at_the_same_time.sh 16520 +861_concurrent_compiles_plan_what_serial_compiles_plan.sh 1080 +862_run_q_writes_exactly_the_programs_stdout.sh 196 +863_a_run_whose_build_failed_exits_101.sh 804 +864_a_builds_status_is_on_standard_error.sh 754 +865_mcpp_test_reports_its_steps_on_stderr_and_its_results_on_stdout.sh 557 +866_both_streams_on_one_pipe_keep_their_order_and_json_stays_clean.sh 3990 +867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh 8320 +868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh 5380 +869_a_pack_over_several_members_names_each_member_in_its_report.sh 1490 +86_target_cfg_dependencies.sh 145 +870_a_pack_over_several_members_resolves_and_fails_per_member.sh 6230 +871_a_pack_states_its_build_and_every_drive_takes_the_job_count.sh 10490 +872_a_member_compiles_once_whatever_the_selection.sh 1870 +87_build_default_profile.sh 634 +88_xlings_environment.sh 1620 +89_build_mcpp.sh 2910 +90_workspace_test.sh 1510 +91_target_bare_alias.sh 276 +92_build_mcpp_import.sh 269 +93_xpkg_parse.sh 67 +96_msvc_unavailable_nonwindows.sh 37 +98_reflection_import_std.sh 6570 diff --git a/tests/e2e/timings/macos.tsv b/tests/e2e/timings/macos.tsv new file mode 100644 index 00000000..3f9b56bc --- /dev/null +++ b/tests/e2e/timings/macos.tsv @@ -0,0 +1,242 @@ +00_fixture_path_hygiene.sh 7000 +01_help_and_version.sh 0 +02_new_build_run.sh 1000 +04_incremental.sh 4000 +05_errors.sh 1000 +06_emit_xpkg.sh 0 +100_feature_sources_test_mode.sh 6000 +101_cppfly_llvm_soft.sh 3000 +10_env_command.sh 8000 +117_object_path_collision.sh 1000 +118_purview_include_rebuild.sh 9000 +11_index_list.sh 8000 +122_run_member.sh 2000 +126_default_features_opt_out.sh 2000 +128_feature_forwarding.sh 2000 +12_add_command.sh 1000 +138_feature_sources_gate_vs_provider.sh 3000 +139_build_program_advisory.sh 3000 +13_toolchain_pin.sh 0 +148_wide_include_list.sh 2000 +149_conditional_glob_flags.sh 2000 +14_toolchain_fallback.sh 7000 +150_clang_module_operator_template.sh 1000 +151_index_url_artifact_migration.sh 8000 +152_test_subdir_names.sh 0 +153_test_isolation.sh 2000 +154_test_filter.sh 2000 +155_test_json.sh 1000 +156_nested_ld_library_path.sh 0 +157_test_glob_flags.sh 1000 +158_test_signal_and_duration.sh 1000 +159_test_list.sh 0 +15_test_passing.sh 2000 +160_test_timeout.sh 6000 +161_xpkg_name_form.sh 0 +162_bare_name_namespace_scope.sh 9000 +163_identity_first_resolution.sh 14000 +164_ninja_goal_command_length.sh 2000 +165_bare_name_cross_namespace_wire_address.sh 5000 +166_run_env_no_private_glibc.sh 1000 +167_build_defines_module_scan.sh 4000 +16_test_failing.sh 3000 +170_bmi_staging_no_cascade.sh 2000 +171_bmi_staging_locked_dest.sh 4000 +178_test_observability.sh 14000 +179_spaced_paths.sh 2000 +17_test_no_tests.sh 0 +181_build_mcpp_import_std.sh 7000 +183_cxx_runtime_contract.sh 8000 +18_devdeps_isolation.sh 1000 +190_link_rspfile_newlines.sh 2000 +191_link_scale.sh 32000 +195_target_cfg_feature_deps.sh 2000 +199_resources_validation.sh 1000 +19_bmi_cache_reuse.sh 7000 +202_machine_output_contract.sh 2000 +204_new_transactional_scaffold.sh 7000 +210_local_index_addressing_on_every_host.sh 0 +211_configure_only_cdb.sh 5000 +221_subos_without_info_still_builds.sh 1000 +230_bench_harness.sh 40000 +231_jobs_option.sh 8000 +232_workflow_syntax.sh 0 +233_bench_matrix.sh 0 +234_bmi_schedule_on.sh 2000 +236_module_extensions_default_toolchain.sh 1000 +23_remove_update.sh 12000 +242_pack_library_interface_and_headers.sh 4000 +243_pack_library_interface_closure.sh 4000 +244_pack_library_gate.sh 3000 +246_explicit_empty_sources.sh 1000 +247_bare_triple_conditional_native.sh 3000 +249_pack_workspace_root_unchanged.sh 1000 +24_git_dependency.sh 10000 +250_pack_names_the_target.sh 2000 +251_pack_library_shared.sh 0 +252_pack_library_old_client.sh 2000 +253_pack_library_undetermined_partition.sh 2000 +254_pack_library_platform_coverage.sh 2000 +259_shared_library_macho.sh 3000 +25_convention_mode.sh 6000 +260_module_extension_is_configuration.sh 1000 +261_pack_follows_the_module_extension.sh 3000 +263_lib_root_follows_the_extension.sh 3000 +266_pack_reads_a_macho_program_without_running_it.sh 1000 +267_target_requested_is_empty_for_a_native_build.sh 1000 +268_target_side_resolution.sh 4000 +269_openkal_llvm_spelling_still_resolves.sh 2000 +27_namespace_dependencies.sh 7000 +280_target_side_layers.sh 3000 +299_a_request_that_named_no_c_library_resolves_to_a_row_that_exists.sh 0 +300_a_registered_family_is_not_reported_unknown.sh 0 +301_the_graphs_compiler_is_taken_and_nothing_is_written.sh 1000 +302_a_stated_compiler_outranks_the_graph_and_two_requirements_do_not_stack.sh 0 +303_the_query_gives_one_answer_for_the_c_library.sh 2000 +304_the_target_side_reaches_every_unit.sh 0 +305_built_from_is_not_published.sh 1000 +313_check_stamp_on_every_platform.sh 2000 +317_project_build_hooks.sh 31000 +327_declared_xlings_deps_report_their_own_failure.sh 7000 +328_layer_predicates_apply_and_unknown_ones_say_so.sh 5000 +329_exit_codes_match_the_contract.sh 0 +32_semver_merge.sh 7000 +330_runner_hosted_targets.sh 5000 +331_xlings_workspace_one_table.sh 1000 +35_workspace.sh 2000 +36_llvm_toolchain.sh 1000 +37_llvm_import_std.sh 4000 +38_llvm_modules.sh 4000 +38_self_config_mirror.sh 0 +39_llvm_incremental.sh 6000 +39_xlings_index_migration.sh 0 +40_llvm_bmi_cache.sh 7000 +41_llvm_std_compat.sh 2000 +42_custom_local_index.sh 6000 +43_indices_lockfile.sh 6000 +44_indices_e2e_integration.sh 7000 +45_install_platform_mapping.sh 0 +46_self_config_mirror_no_bootstrap.sh 5000 +47_cdb_prebuilt_module_path_abs.sh 2000 +47_llvm_atomic_link.sh 0 +48_build_error_output.sh 1000 +601_exclusive_capability.sh 1000 +603_version_floor.sh 2000 +604_offline_skips_sandbox_bootstrap.sh 1000 +609_clean_stale.sh 4000 +616_examples_and_their_index_agree.sh 1000 +628_a_pin_below_a_stated_floor_is_refused.sh 26000 +645_the_fast_path_compares_the_toolchain_request.sh 5000 +646_runtime_deploy_places_files_in_a_directory.sh 3000 +659_an_action_declaration_has_no_size_limit.sh 2000 +660_an_action_names_the_engine_through_mcpp_self.sh 2000 +666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh 3000 +668_a_macho_program_is_staged_with_its_closure.sh 2000 +66_runtime_provides.sh 2000 +671_toolchain_is_an_option_of_run_test_and_pack.sh 3000 +672_a_runner_receives_the_files_the_artifact_carries.sh 11000 +673_test_discover_names_where_tests_are.sh 1000 +674_a_format_is_run_by_the_runner_named_after_it.sh 13000 +679_a_path_dependency_takes_its_manifests_identity.sh 2000 +67_features_strict.sh 6000 +681_index_repos_reach_an_existing_home.sh 7000 +682_why_deps_prints_the_resolved_graph.sh 1000 +683_a_build_program_reads_the_payload_pkg_config_view.sh 1000 +684_a_runner_supplied_twice_is_refused.sh 3000 +686_an_activated_xlings_subos_does_not_redirect_the_registry.sh 5000 +687_the_vendored_xlings_probe_is_an_argument_vector.sh 0 +688_emit_build_database.sh 6000 +68_profile_passthrough.sh 4000 +698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 6000 +69_package_templates.sh 8000 +702_a_static_package_under_a_shared_library_is_linked_into_it.sh 2000 +704_cxx_identity_across_macho_images.sh 3000 +720_the_root_build_program_reads_the_resolved_graph.sh 4000 +721_an_android_row_links_on_a_macos_host.sh 92000 +732_xlings_refresh_is_bounded_and_owned.sh 5000 +733_envelope_reports_observed_network_access.sh 1000 +734_auto_refresh_false_governs_first_index_sync.sh 0 +735_offline_missing_package_has_its_own_code.sh 0 +736_compile_flag_words_reach_the_compiler_and_the_databases.sh 2000 +739_a_private_feature_dep_does_not_reach_the_consumer.sh 2000 +746_macos_deployment_target_is_target_keyed_not_host_keyed.sh 1000 +75_index_status_offline.sh 7000 +765_a_consumer_include_directory_stays_in_the_consumer.sh 2000 +76_compile_commands_generated.sh 2000 +770_workspace_member_as_dependency.sh 4000 +774_an_index_member_inherits_its_archive_workspace.sh 0 +775_a_host_tool_merges_its_conditional_sections_once.sh 4000 +77_cdb_preserves_test_entries.sh 3000 +780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 3000 +781_deleted_root_cdb_is_restored.sh 3000 +782_foreign_root_cdb_entries_are_replaced.sh 3000 +787_emit_plans_every_member_on_its_own.sh 1000 +78_test_main_combinations.sh 6000 +799_an_action_runs_with_its_env_and_cwd.sh 2000 +79_gtest_regular_dep_feature_main.sh 5000 +809_failed_build_program_hides_no_device_source.sh 0 +80_feature_defines.sh 1000 +812_an_index_floor_is_a_closing_tip.sh 1000 +815_the_database_describes_what_a_rule_generates.sh 1000 +816_a_git_dependency_reports_its_fetch.sh 1000 +817_emit_writes_no_xlings_json_under_private_work_dir.sh 1000 +819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 0 +81_capability_binding.sh 3000 +821_an_edge_states_its_advice_on_both_paths.sh 3000 +822_a_package_states_the_oldest_mcpp_it_supports.sh 1000 +823_mcpp_core_and_mcpp_name_one_interface.sh 4000 +824_a_build_program_reads_the_resolved_toolchain.sh 1000 +826_plugin_diagnostics_features_and_names.sh 3000 +827_many_placements_run_as_one_process.sh 4000 +828_a_library_states_its_interface.sh 3000 +829_pack_takes_a_member.sh 2000 +82_feature_optional_deps.sh 1000 +831_a_path_dependency_host_module_outside_src.sh 6000 +832_the_fast_path_resumes_after_an_edit.sh 4000 +833_a_workspace_is_one_graph_per_configuration.sh 5000 +834_a_workspace_plans_once_and_replays_in_one_check.sh 6000 +836_a_members_own_statements_reach_its_build.sh 2000 +839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 4000 +83_feature_defines_propagate.sh 1000 +840_a_workspace_database_describes_each_package_once.sh 4000 +841_pack_summarizes_many_outputs.sh 4000 +842_the_build_reports_each_step_once.sh 11000 +843_a_terminal_names_the_action_that_runs.sh 5000 +845_play_game_restores_the_terminal.sh 13000 +846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 1000 +847_a_module_name_is_unique_within_a_program.sh 5000 +849_a_staged_bmi_waits_for_a_module_compiled_here.sh 3000 +84_runtime_dll_deploy.sh 0 +851_a_shared_member_is_compiled_once_per_configuration.sh 3000 +852_a_repeated_dash_p_selects_every_member_it_names.sh 2000 +853_run_keeps_one_member_and_exclude_removes_members.sh 5000 +854_a_test_over_several_members_plans_and_builds_once.sh 3000 +855_a_shared_test_build_is_reported_once_per_group.sh 3000 +856_a_test_over_several_members_continues_past_a_failing_one.sh 9000 +857_a_host_module_several_members_import_is_compiled_once.sh 2000 +858_the_engines_module_is_compiled_once_per_home.sh 7000 +859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 6000 +85_target_cfg_build_flags.sh 0 +860_a_workspaces_programs_compile_at_the_same_time.sh 8000 +861_concurrent_compiles_plan_what_serial_compiles_plan.sh 3000 +862_run_q_writes_exactly_the_programs_stdout.sh 1000 +863_a_run_whose_build_failed_exits_101.sh 3000 +864_a_builds_status_is_on_standard_error.sh 4000 +865_mcpp_test_reports_its_steps_on_stderr_and_its_results_on_stdout.sh 3000 +866_both_streams_on_one_pipe_keep_their_order_and_json_stays_clean.sh 6000 +867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh 15000 +868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh 12000 +869_a_pack_over_several_members_names_each_member_in_its_report.sh 9000 +86_llvm_hermetic_link.sh 0 +86_target_cfg_dependencies.sh 1000 +870_a_pack_over_several_members_resolves_and_fails_per_member.sh 13000 +872_a_member_compiles_once_whatever_the_selection.sh 10000 +87_build_default_profile.sh 3000 +88_xlings_environment.sh 2000 +89_build_mcpp.sh 4000 +90_workspace_test.sh 4000 +91_target_bare_alias.sh 0 +92_build_mcpp_import.sh 2000 +93_xpkg_parse.sh 0 +94_macos_test_stdlib.sh 1000 +96_msvc_unavailable_nonwindows.sh 0 diff --git a/tests/e2e/timings/windows.tsv b/tests/e2e/timings/windows.tsv new file mode 100644 index 00000000..8dadbccc --- /dev/null +++ b/tests/e2e/timings/windows.tsv @@ -0,0 +1,182 @@ +00_fixture_path_hygiene.sh 45090 +01_help_and_version.sh 427 +04_incremental.sh 15660 +05_errors.sh 7880 +06_emit_xpkg.sh 331 +100_feature_sources_test_mode.sh 5580 +101_cppfly_llvm_soft.sh 41760 +10_env_command.sh 7450 +117_object_path_collision.sh 1850 +118_purview_include_rebuild.sh 10630 +11_index_list.sh 8520 +122_run_member.sh 2060 +126_default_features_opt_out.sh 3900 +128_feature_forwarding.sh 4780 +12_add_command.sh 2890 +13_toolchain_pin.sh 121 +148_wide_include_list.sh 4370 +149_conditional_glob_flags.sh 2260 +14_toolchain_fallback.sh 7040 +150_clang_module_operator_template.sh 545 +151_index_url_artifact_migration.sh 7940 +152_test_subdir_names.sh 963 +153_test_isolation.sh 1680 +154_test_filter.sh 1360 +155_test_json.sh 1290 +156_nested_ld_library_path.sh 106 +157_test_glob_flags.sh 921 +159_test_list.sh 295 +15_test_passing.sh 2250 +161_xpkg_name_form.sh 1310 +162_bare_name_namespace_scope.sh 10220 +163_identity_first_resolution.sh 52510 +164_ninja_goal_command_length.sh 1780 +165_bare_name_cross_namespace_wire_address.sh 11580 +166_run_env_no_private_glibc.sh 1140 +167_build_defines_module_scan.sh 4070 +16_test_failing.sh 1770 +170_bmi_staging_no_cascade.sh 2210 +171_bmi_staging_locked_dest.sh 4250 +177_cpp20_msvc.sh 9700 +179_spaced_paths.sh 1900 +17_test_no_tests.sh 261 +180_msvc_build_mcpp.sh 17670 +183_cxx_runtime_contract.sh 9180 +18_devdeps_isolation.sh 1900 +190_link_rspfile_newlines.sh 3670 +191_link_scale.sh 37040 +195_target_cfg_feature_deps.sh 2640 +197_windows_resources.sh 8470 +199_resources_validation.sh 1690 +19_bmi_cache_reuse.sh 49270 +202_machine_output_contract.sh 2840 +204_new_transactional_scaffold.sh 9170 +210_local_index_addressing_on_every_host.sh 1360 +211_configure_only_cdb.sh 5100 +221_subos_without_info_still_builds.sh 1610 +230_bench_harness.sh 69640 +231_jobs_option.sh 8980 +232_workflow_syntax.sh 281 +233_bench_matrix.sh 926 +234_bmi_schedule_on.sh 2620 +236_module_extensions_default_toolchain.sh 2490 +23_remove_update.sh 21160 +241_windows_ucrt_runtime_identity.sh 14310 +242_pack_library_interface_and_headers.sh 4840 +243_pack_library_interface_closure.sh 4390 +244_pack_library_gate.sh 4710 +246_explicit_empty_sources.sh 2400 +247_bare_triple_conditional_native.sh 2700 +249_pack_workspace_root_unchanged.sh 1160 +250_pack_names_the_target.sh 1790 +251_pack_library_shared.sh 1270 +252_pack_library_old_client.sh 2180 +253_pack_library_undetermined_partition.sh 3460 +254_pack_library_platform_coverage.sh 3410 +255_pack_library_msvc_archiver.sh 1130 +256_pack_library_fat_windows.sh 5150 +258_shared_library_msvc_auto_def.sh 8210 +25_convention_mode.sh 5190 +260_module_extension_is_configuration.sh 1800 +261_pack_follows_the_module_extension.sh 2980 +262_pack_consumed_by_native_cl.sh 2630 +263_lib_root_follows_the_extension.sh 8150 +280_target_side_layers.sh 4220 +313_check_stamp_on_every_platform.sh 1800 +317_project_build_hooks.sh 23570 +327_declared_xlings_deps_report_their_own_failure.sh 10470 +328_layer_predicates_apply_and_unknown_ones_say_so.sh 7080 +329_exit_codes_match_the_contract.sh 324 +35_workspace.sh 2010 +36_llvm_toolchain.sh 42490 +38_self_config_mirror.sh 42110 +39_xlings_index_migration.sh 38520 +42_custom_local_index.sh 9200 +43_indices_lockfile.sh 9130 +44_indices_e2e_integration.sh 8780 +45_install_platform_mapping.sh 216 +46_self_config_mirror_no_bootstrap.sh 6700 +47_cdb_prebuilt_module_path_abs.sh 1840 +48_build_error_output.sh 40840 +601_exclusive_capability.sh 1960 +603_version_floor.sh 2840 +604_offline_skips_sandbox_bootstrap.sh 2130 +609_clean_stale.sh 5200 +628_a_pin_below_a_stated_floor_is_refused.sh 31990 +642_windows_subsystem.sh 5060 +645_the_fast_path_compares_the_toolchain_request.sh 42390 +646_runtime_deploy_places_files_in_a_directory.sh 5930 +659_an_action_declaration_has_no_size_limit.sh 10130 +660_an_action_names_the_engine_through_mcpp_self.sh 2850 +66_runtime_provides.sh 2550 +671_toolchain_is_an_option_of_run_test_and_pack.sh 3570 +673_test_discover_names_where_tests_are.sh 2360 +679_a_path_dependency_takes_its_manifests_identity.sh 3840 +67_features_strict.sh 10760 +681_index_repos_reach_an_existing_home.sh 11320 +682_why_deps_prints_the_resolved_graph.sh 1590 +683_a_build_program_reads_the_payload_pkg_config_view.sh 1900 +687_the_vendored_xlings_probe_is_an_argument_vector.sh 42730 +688_emit_build_database.sh 51110 +68_profile_passthrough.sh 6310 +698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 19360 +69_package_templates.sh 14860 +703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh 7020 +705_cxx_identity_across_pe_images.sh 5150 +720_the_root_build_program_reads_the_resolved_graph.sh 5720 +735_offline_missing_package_has_its_own_code.sh 609 +736_compile_flag_words_reach_the_compiler_and_the_databases.sh 3400 +739_a_private_feature_dep_does_not_reach_the_consumer.sh 2230 +75_index_status_offline.sh 7180 +760_msvc_toolset_is_chosen_once.sh 10320 +765_a_consumer_include_directory_stays_in_the_consumer.sh 45510 +76_compile_commands_generated.sh 1800 +770_workspace_member_as_dependency.sh 6610 +774_an_index_member_inherits_its_archive_workspace.sh 41050 +775_a_host_tool_merges_its_conditional_sections_once.sh 50850 +77_cdb_preserves_test_entries.sh 3980 +780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 3250 +781_deleted_root_cdb_is_restored.sh 2680 +782_foreign_root_cdb_entries_are_replaced.sh 4240 +787_emit_plans_every_member_on_its_own.sh 1390 +78_test_main_combinations.sh 11260 +794_a_windows_program_finds_a_dll_through_runtime_search_dir.sh 2830 +799_an_action_runs_with_its_env_and_cwd.sh 3460 +79_gtest_regular_dep_feature_main.sh 14340 +809_failed_build_program_hides_no_device_source.sh 909 +80_feature_defines.sh 1750 +811_place_dlls_never_overwrites_a_declared_deploy.sh 3470 +814_the_llvm_row_defaults_to_the_dynamic_crt.sh 6960 +819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 5370 +81_capability_binding.sh 4570 +820_a_windows_program_takes_its_runtime_from_one_resolver.sh 13960 +825_a_build_program_reads_the_msvc_toolset.sh 3430 +82_feature_optional_deps.sh 1970 +836_a_members_own_statements_reach_its_build.sh 2820 +839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 4870 +83_feature_defines_propagate.sh 1230 +840_a_workspace_database_describes_each_package_once.sh 7300 +841_pack_summarizes_many_outputs.sh 9800 +842_the_build_reports_each_step_once.sh 17680 +846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 38110 +847_a_module_name_is_unique_within_a_program.sh 8510 +848_msvc_binds_a_module_name_per_program.sh 1270 +849_a_staged_bmi_waits_for_a_module_compiled_here.sh 43610 +84_runtime_dll_deploy.sh 1090 +857_a_host_module_several_members_import_is_compiled_once.sh 41690 +858_the_engines_module_is_compiled_once_per_home.sh 153360 +859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 49500 +85_target_cfg_build_flags.sh 1090 +860_a_workspaces_programs_compile_at_the_same_time.sh 48910 +86_target_cfg_dependencies.sh 1170 +872_a_member_compiles_once_whatever_the_selection.sh 57560 +87_build_default_profile.sh 3370 +88_xlings_environment.sh 2220 +89_build_mcpp.sh 7200 +90_workspace_test.sh 3560 +91_target_bare_alias.sh 963 +92_build_mcpp_import.sh 1310 +93_xpkg_parse.sh 798 +95_msvc_system_toolchain.sh 3470 +97_mingw_toolchain.sh 11680 +99_msvc_native_build.sh 5080 diff --git a/tests/scripts/test_check_e2e_coverage.py b/tests/scripts/test_check_e2e_coverage.py new file mode 100644 index 00000000..7954ef75 --- /dev/null +++ b/tests/scripts/test_check_e2e_coverage.py @@ -0,0 +1,84 @@ +#!/usr/bin/env python3 +"""Fixture tests for .github/tools/check_e2e_coverage.py (rule R5). + +A test that ran, a test a workflow names, and a test the exceptions excuse are +covered; a test that is none of these fails the check, and so does an exception +for a test that ran or that does not exist. +""" + +from __future__ import annotations + +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[2] +SCRIPT = REPO_ROOT / ".github" / "tools" / "check_e2e_coverage.py" + + +class E2ECoverage(unittest.TestCase): + def setUp(self) -> None: + self._dir = tempfile.TemporaryDirectory() + self.root = Path(self._dir.name) + e2e = self.root / "tests" / "e2e" + e2e.mkdir(parents=True) + for name in ("10_runs.sh", "20_in_a_job.sh", "30_excused.sh", "40_by_pattern.sh"): + (e2e / name).write_text("#!/usr/bin/env bash\n# requires: unix-shell\n") + wf = self.root / ".github" / "workflows" + wf.mkdir(parents=True) + (wf / "ci.yml").write_text("run: bash tests/e2e/20_in_a_job.sh\nE2E_ONLY: '40_*.sh'\n") + (e2e / "coverage-exceptions.tsv").write_text("# test\treason\n30_excused.sh\tneeds a device\n") + self.reports = self.root / "reports" + self.reports.mkdir() + (self.reports / "e2e-report-linux-1.tsv").write_text( + "pass\t10_runs.sh\t1200\t\nskip\t20_in_a_job.sh\t0\tmissing capability: qemu-riscv\n") + + def tearDown(self) -> None: + self._dir.cleanup() + + def run_check(self, *extra: str) -> subprocess.CompletedProcess: + return subprocess.run([sys.executable, str(SCRIPT), "--root", str(self.root), + "--reports", str(self.reports), *extra], + capture_output=True, text=True, check=False) + + def test_ran_named_and_excused_tests_are_covered(self) -> None: + r = self.run_check() + self.assertEqual(r.returncode, 0, r.stdout) + self.assertIn("1 ran on a shard, 2 run by a dedicated job, 1 excused, 0 uncovered", r.stdout) + + def test_a_test_that_runs_nowhere_fails(self) -> None: + (self.root / "tests" / "e2e" / "50_nowhere.sh").write_text("# x\n# requires: llvm\n") + (self.reports / "e2e-report-linux-2.tsv").write_text( + "skip\t50_nowhere.sh\t0\tmissing capability: llvm\n") + r = self.run_check() + self.assertEqual(r.returncode, 1) + self.assertIn("UNCOVERED: 50_nowhere.sh: linux: missing capability: llvm", r.stdout) + + def test_an_exception_for_a_test_that_ran_fails(self) -> None: + (self.reports / "e2e-report-macos-1.tsv").write_text("pass\t30_excused.sh\t10\t\n") + r = self.run_check() + self.assertEqual(r.returncode, 1) + self.assertIn("STALE EXCEPTION: 30_excused.sh is excused but ran on macos", r.stdout) + + def test_an_exception_for_a_missing_test_fails(self) -> None: + with (self.root / "tests" / "e2e" / "coverage-exceptions.tsv").open("a") as f: + f.write("99_gone.sh\tremoved\n") + r = self.run_check() + self.assertEqual(r.returncode, 1) + self.assertIn("STALE EXCEPTION: 99_gone.sh is excused but does not exist", r.stdout) + + def test_no_report_says_nothing_and_fails(self) -> None: + for p in self.reports.iterdir(): + p.unlink() + self.assertEqual(self.run_check().returncode, 1) + + def test_merged_timings_are_written_per_host(self) -> None: + out = self.root / "timings" + self.assertEqual(self.run_check("--timings-out", str(out)).returncode, 0) + self.assertEqual((out / "linux.tsv").read_text(), "10_runs.sh\t1200\n") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/scripts/test_classify_changes.py b/tests/scripts/test_classify_changes.py new file mode 100644 index 00000000..e93df980 --- /dev/null +++ b/tests/scripts/test_classify_changes.py @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +"""Fixture tests for .github/tools/classify_changes.py (rule R2). + +A change of documentation that nothing names is `code=false`; a change of any +other path, of a document a test reads, or of nothing at all is `code=true`. +""" + +from __future__ import annotations + +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[2] +SCRIPT = REPO_ROOT / ".github" / "tools" / "classify_changes.py" + + +def classify(root: Path, paths: list[str]) -> str: + out = subprocess.run([sys.executable, str(SCRIPT), "--root", str(root)], + input="\n".join(paths), capture_output=True, text=True, check=True) + return out.stdout.splitlines()[0] + + +class ClassifyChanges(unittest.TestCase): + def setUp(self) -> None: + self._dir = tempfile.TemporaryDirectory() + self.root = Path(self._dir.name) + (self.root / "docs").mkdir() + (self.root / "docs" / "10-guide.md").write_text("guide\n") + (self.root / "docs" / "03-examples.md").write_text("examples\n") + (self.root / "tests" / "e2e").mkdir(parents=True) + (self.root / "tests" / "e2e" / "616_examples.sh").write_text( + 'grep -q x "$ROOT/docs/03-examples.md"\n') + (self.root / "src").mkdir() + (self.root / "src" / "main.cpp").write_text("int main() {}\n") + + def tearDown(self) -> None: + self._dir.cleanup() + + def test_documentation_that_nothing_names_is_not_code(self) -> None: + self.assertEqual(classify(self.root, ["docs/10-guide.md", ".agents/docs/x.md", + "README.md", "LICENSE"]), "code=false") + + def test_a_document_a_test_reads_is_code(self) -> None: + self.assertEqual(classify(self.root, ["docs/03-examples.md"]), "code=true") + + def test_the_translation_of_a_document_a_test_reads_is_code(self) -> None: + # The test names docs/03-examples.md; docs/zh/03-examples.md is its translation. + self.assertEqual(classify(self.root, ["docs/zh/03-examples.md"]), "code=true") + + def test_any_other_path_is_code(self) -> None: + self.assertEqual(classify(self.root, ["docs/10-guide.md", "src/main.cpp"]), "code=true") + + def test_an_empty_change_is_code(self) -> None: + self.assertEqual(classify(self.root, []), "code=true") + + def test_the_repository_classifies_its_own_documents(self) -> None: + # docs/20-toolchains.md is read by check_default_toolchain_docs.py. + self.assertEqual(classify(REPO_ROOT, ["docs/20-toolchains.md"]), "code=true") + # Release packaging copies the root README; the design-record index is + # read by nothing that a documentation check does not already run. + self.assertEqual(classify(REPO_ROOT, ["README.md"]), "code=true") + self.assertEqual(classify(REPO_ROOT, [".agents/docs/README.md", + ".agents/docs/2026-10-02-x.md"]), "code=false") + + def test_every_documentation_check_it_trusts_runs_in_the_docs_job(self) -> None: + sys.path.insert(0, str(SCRIPT.parent)) + import classify_changes # noqa: E402 + ci = (REPO_ROOT / ".github" / "workflows" / "ci.yml").read_text(encoding="utf-8") + docs_job = ci[ci.index("\n docs:"):ci.index("\n build-linux:")] + for check in classify_changes.DOCS_JOB_CHECKS: + with self.subTest(check=check): + name = Path(check).name + self.assertIn(name, docs_job if name != "gen_agents_index.py" + else (REPO_ROOT / ".github" / "tools" / "check_docs_structure.sh") + .read_text(encoding="utf-8")) + + +if __name__ == "__main__": + unittest.main() From 68c6a6db0ec08dbb1305ecab76684ffcf5b461b4 Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 07:25:43 +0800 Subject: [PATCH 04/14] tests: fixture paths that the documentation structure check does not read as chapters --- tests/scripts/test_classify_changes.py | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/tests/scripts/test_classify_changes.py b/tests/scripts/test_classify_changes.py index e93df980..961d0170 100644 --- a/tests/scripts/test_classify_changes.py +++ b/tests/scripts/test_classify_changes.py @@ -28,11 +28,11 @@ def setUp(self) -> None: self._dir = tempfile.TemporaryDirectory() self.root = Path(self._dir.name) (self.root / "docs").mkdir() - (self.root / "docs" / "10-guide.md").write_text("guide\n") - (self.root / "docs" / "03-examples.md").write_text("examples\n") + (self.root / "docs" / "guide.md").write_text("guide\n") + (self.root / "docs" / "examples.md").write_text("examples\n") (self.root / "tests" / "e2e").mkdir(parents=True) (self.root / "tests" / "e2e" / "616_examples.sh").write_text( - 'grep -q x "$ROOT/docs/03-examples.md"\n') + 'grep -q x "$ROOT/docs/examples.md"\n') (self.root / "src").mkdir() (self.root / "src" / "main.cpp").write_text("int main() {}\n") @@ -40,18 +40,18 @@ def tearDown(self) -> None: self._dir.cleanup() def test_documentation_that_nothing_names_is_not_code(self) -> None: - self.assertEqual(classify(self.root, ["docs/10-guide.md", ".agents/docs/x.md", + self.assertEqual(classify(self.root, ["docs/guide.md", ".agents/docs/x.md", "README.md", "LICENSE"]), "code=false") def test_a_document_a_test_reads_is_code(self) -> None: - self.assertEqual(classify(self.root, ["docs/03-examples.md"]), "code=true") + self.assertEqual(classify(self.root, ["docs/examples.md"]), "code=true") def test_the_translation_of_a_document_a_test_reads_is_code(self) -> None: - # The test names docs/03-examples.md; docs/zh/03-examples.md is its translation. - self.assertEqual(classify(self.root, ["docs/zh/03-examples.md"]), "code=true") + # The test names docs/examples.md; docs/zh/examples.md is its translation. + self.assertEqual(classify(self.root, ["docs/zh/examples.md"]), "code=true") def test_any_other_path_is_code(self) -> None: - self.assertEqual(classify(self.root, ["docs/10-guide.md", "src/main.cpp"]), "code=true") + self.assertEqual(classify(self.root, ["docs/guide.md", "src/main.cpp"]), "code=true") def test_an_empty_change_is_code(self) -> None: self.assertEqual(classify(self.root, []), "code=true") From 118aa9175869e30b5b420c4ad2c789f5bd601634 Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 07:35:40 +0800 Subject: [PATCH 05/14] ci: use-built-mcpp installs the toolchain the build used, after setting the mirror --- .github/actions/use-built-mcpp/action.yml | 3 +++ .github/actions/use-built-mcpp/use.sh | 25 ++++++++++++++++++----- 2 files changed, 23 insertions(+), 5 deletions(-) diff --git a/.github/actions/use-built-mcpp/action.yml b/.github/actions/use-built-mcpp/action.yml index 96ab65c6..7a87ff6f 100644 --- a/.github/actions/use-built-mcpp/action.yml +++ b/.github/actions/use-built-mcpp/action.yml @@ -16,6 +16,9 @@ description: > them; when it does not, the bootstrap installs that toolchain and the binary is run again. A binary that still does not run fails this step. + The toolchain mcpp.toml names for the host is then installed with the + binary, as a job that built mcpp used to install it as a side effect. + inputs: host: description: > diff --git a/.github/actions/use-built-mcpp/use.sh b/.github/actions/use-built-mcpp/use.sh index e94da450..99641571 100644 --- a/.github/actions/use-built-mcpp/use.sh +++ b/.github/actions/use-built-mcpp/use.sh @@ -54,14 +54,29 @@ if ! out=$("$bin" --version 2>&1); then fi echo "this commit's mcpp: $out ($bin)" +# The mirror first: the runners are outside CN, and the install below and +# every later download read it. +if [ -n "${XLINGS_BIN:-}" ]; then + "$XLINGS_BIN" config --mirror "$mirror" 2>/dev/null || true +fi +MCPP_VENDORED_XLINGS="${XLINGS_BIN:-}" "$bin" self config --mirror "$mirror" + +# THE TOOLCHAIN THE BUILD USED IS INSTALLED, AS IT WAS WHEN EVERY JOB BUILT. +# A job that built mcpp itself installed this toolchain as a side effect, and +# the steps after the build relied on it without saying so: measured on the +# first run of this action, the aarch64 leg of the target matrix restored no +# sandbox, its binary ran without any payload, and the invariants that list the +# host's toolchains found none ("gcc is not installed here"). Installing it here +# keeps every consumer's environment what it was. It is a lookup when the +# toolchain is present. +tc="$(manifest_toolchain)" +if [ -n "$tc" ]; then + MCPP_VENDORED_XLINGS="${XLINGS_BIN:-}" "$bin" toolchain install "${tc%@*}" "${tc#*@}" +fi + { echo "MCPP_BOOT=$boot" echo "MCPP=$bin" echo "MCPP_FRESH=$bin" if [ -n "${XLINGS_BIN:-}" ]; then echo "MCPP_VENDORED_XLINGS=$XLINGS_BIN"; fi } >> "$GITHUB_ENV" - -if [ -n "${XLINGS_BIN:-}" ]; then - "$XLINGS_BIN" config --mirror "$mirror" 2>/dev/null || true -fi -MCPP_VENDORED_XLINGS="${XLINGS_BIN:-}" "$bin" self config --mirror "$mirror" From 6238f8bcbb4a5faa8080bd3f70bc70ea09e1cb5e Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 08:00:15 +0800 Subject: [PATCH 06/14] ci: four Linux shards from the measured durations; e2e 741 compiles a library and links nothing --- ...-and-the-toolchain-specification-design.md | 68 ++ .github/workflows/ci-linux-e2e.yml | 12 +- .github/workflows/ci-windows-e2e.yml | 14 +- ...i_environment_declared_by_the_c_library.sh | 10 + tests/e2e/timings/linux.tsv | 959 +++++++++--------- tests/e2e/timings/macos.tsv | 266 ++--- tests/e2e/timings/windows.tsv | 364 +++---- 7 files changed, 909 insertions(+), 784 deletions(-) diff --git a/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md b/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md index 14f97da2..1fb5538e 100644 --- a/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md +++ b/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md @@ -564,6 +564,74 @@ measurement uses the same scripts as §2.1. - Cache usage below 8 GB. - A documentation-only pull request finishing under 5 minutes. +### 2.7 What was built, and where it departs from §2.4 + +Implemented in mcpp#759. Each departure below was decided by a measurement or a +reading taken while building it. + +- **The artifact is the binary itself, not a packed copy (§2.4.1).** The Linux + self-host binary has the interpreter + `~/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/lib64/ld-linux-x86-64.so.2`, needs + `libgcc_s.so.1` from the `xim-x-gcc/16.1.0` payload, and links libstdc++ + statically (measured locally, `readelf`). A consumer that restores the + sandbox runs it as it is. Packing it would test a different binary from the + one every self-host build produces. `use-built-mcpp` runs the binary first, + and only when it does not run does the bootstrap install the toolchain + `mcpp.toml` names for the host; a binary that still does not run fails the + step. The binary is 25 MB. +- **The Wine packages keep their cache (§2.4.4).** Their eviction was a + consequence of F3, not a property of the cache. With one writer per key, the + `wine-debs` cache is saved only by `mingw-cross-wine` on main and is no longer + displaced. A release asset would have added a second thing to publish and + keep current. +- **The timing tables are in the repository (§2.4.5).** They live under + `tests/e2e/timings/.tsv`, seeded from the per-test lines of the + 2026-10-01 logs. A shard's membership is then a function of the commit, + which makes it reproducible (`E2E_LIST=1` prints it). The `e2e-coverage` job + uploads the merged durations of each run as the artifact `e2e-timings`, and + refreshing a table is copying a file. The tables were seeded from the + 2026-10-01 logs, then replaced by the durations of the first run of this + change, in which 515 Linux tests ran instead of 466. Linux therefore has four + shards rather than three, each budgeted at 9.9 minutes. Windows has three at + 14.4 to 14.5, and macOS two at 9.1 and 9.3. The step limits are about twice + the budgets: 22, 30 and 20 minutes. +- **The classifier searches exact paths (§2.4.3).** It searches for the + changed path and for its translation (`docs/X` and `docs/zh/X`), and not for + a bare file name. A search on the name made `.agents/docs/README.md`, which + every new record regenerates, a code change, because release packaging names + `README.md`. The checks of the `docs` job do not count as readers, because + they run on every change. A document that any other script or source names + starts the whole CI rather than a subset of it, which is simpler and errs + towards running more. +- **The macOS legs of the target matrix and of openkal still build (§2.4.1).** + They run on `macos-14`, and the artifact is built on `macos-15`. release.yml + records that a bootstrap build linking the system libc++ with a minimum + version of 14 failed at launch on `macos-14`. Until the artifact is measured + there, those two legs build their own, and the gate carries + `ci-lint: allow-r1` with that reason. Their Linux and Windows legs consume the + artifact. +- **`invariants` and `scan` remain two jobs (§2.4.1).** Merging them is a + rewrite of the target-matrix workflow beside a change that already rewrites + twelve; it is left for a change of its own. +- **A test that never ran was broken.** 741, one of the seven `llvm` tests, + failed on its first run in CI. The root package of its fixture was a binary, + so the build linked `cabi-probe.exe` for `x86_64-windows-gnu`, and with + `allow_host_libs` the link found the host's mingw-w64 libraries. It passed on a + machine that has them and failed on every runner, and nothing noticed, because + no runner ran it. The fixture's root is now a library, which is what the test + says it is: "this test compiles only". +- **Coverage found more than F9.** Classifying the tests of the 2026-10-01 + logs found 24 that ran on no runner and were named by no workflow. The + seven `llvm` tests are among them, and so are three `musl` tests: the probe + named 15.1.0 while the runners installed 16.1.0. Seven `mingw-cross` tests + are in the list too, a toolchain no shard installed. The rest were 105 (nasm), + 65 (scan-deps), 239 (named by its `E2E_ONLY` pattern, which the check now + reads), 257 (needs wine and a Linux-hosted MinGW), 658 (an attached Android + device), and 873-877 (added after the logs). The capability probes now ask for + a family. The Linux shards install musl, llvm, mingw-cross and nasm. 257 runs + in `mingw-cross-wine`. 658 is the one entry of + `tests/e2e/coverage-exceptions.tsv`. + ## Part III. Defects repaired by the next release ### 3.1 #757: the engine's identity is part of a build record diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index b8195c3c..eade307d 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -1,7 +1,7 @@ name: ci-linux-e2e # The e2e suite (tests/e2e/run_all.sh) on Linux, called by ci.yml beside -# ci-linux.yml after the Linux build. Three shards, assigned by measured +# ci-linux.yml after the Linux build. Four shards, assigned by measured # duration (tests/e2e/timings/linux.tsv, rule R4 of the 2026-10-02 CI record), # each running the one binary build.yml produced (use-built-mcpp, rule R1). # Each shard writes a per-test report that the e2e-coverage job of ci.yml reads @@ -15,15 +15,15 @@ on: jobs: e2e: - name: e2e ${{ matrix.shard }}/3 (linux x86_64, self-host) + name: e2e ${{ matrix.shard }}/4 (linux x86_64, self-host) runs-on: ubuntu-24.04 timeout-minutes: 45 strategy: fail-fast: false matrix: - shard: [1, 2, 3] + shard: [1, 2, 3, 4] env: - E2E_SHARD: ${{ matrix.shard }}/3 + E2E_SHARD: ${{ matrix.shard }}/4 E2E_TIMINGS: tests/e2e/timings/linux.tsv E2E_REPORT: ${{ github.workspace }}/e2e-report-linux-${{ matrix.shard }}.tsv MCPP_HOME: /home/runner/.mcpp @@ -49,9 +49,9 @@ jobs: host: linux-x86_64 - name: E2E suite - # Twice the shard's budget of about ten minutes (R4): reached only by a + # About twice the shard's budget of ten minutes (R4): reached only by a # hang. The per-test 600 s limit in run_all.sh names the test that hung. - timeout-minutes: 25 + timeout-minutes: 22 run: | set -euo pipefail # MCPP is this commit's binary and MCPP_BOOT the released bootstrap diff --git a/.github/workflows/ci-windows-e2e.yml b/.github/workflows/ci-windows-e2e.yml index 85ad336c..c7cd5482 100644 --- a/.github/workflows/ci-windows-e2e.yml +++ b/.github/workflows/ci-windows-e2e.yml @@ -81,15 +81,13 @@ jobs: - name: E2E suite shell: bash - # Fail-fast on hung tests instead of burning the whole job budget. - # Per-test 600s timeout lives in run_all.sh. - timeout-minutes: 25 + # About twice the shard's budget of fifteen minutes (R4): reached only by + # a hang. The per-test 600 s limit in run_all.sh names the test that hung. + timeout-minutes: 30 run: | - # The RELEASED mcpp this job bootstrapped from, kept for e2e 252: the - # claim that an older client can still build against a package the PR - # produces is only worth making if something checks it against a real - # old binary. Captured before $MCPP is repointed at the fresh build. - # MCPP_BOOT is the released bootstrap (use-built-mcpp), for e2e 252. + # MCPP_BOOT is the released bootstrap (use-built-mcpp). e2e 252 checks + # an older client against a package this commit produces, so it needs + # a real old binary. export MCPP_BOOT export MCPP="$MCPP_SELF" export MCPP_VENDORED_XLINGS="$XLINGS_BIN" diff --git a/tests/e2e/741_c_abi_environment_declared_by_the_c_library.sh b/tests/e2e/741_c_abi_environment_declared_by_the_c_library.sh index e2abb14a..5514439a 100755 --- a/tests/e2e/741_c_abi_environment_declared_by_the_c_library.sh +++ b/tests/e2e/741_c_abi_environment_declared_by_the_c_library.sh @@ -63,6 +63,12 @@ cabi_probe_asm_marker: ret EOF +# THE ROOT IS A LIBRARY, SO NOTHING LINKS. With the inferred binary target the +# build linked `cabi-probe.exe` for x86_64-windows-gnu, which needs a MinGW +# runtime: it passed on a machine with the host's mingw-w64 installed and failed +# on every CI runner, and nothing noticed because no runner ever ran this test +# (it declared `llvm`, which no runner granted, until 2026-10-02). Its subject +# is the compile lines, which a library has as well. cat > mcpp.toml <<'EOF' [package] name = "cabi-probe" @@ -72,6 +78,10 @@ version = "0.1.0" fakemusl = { path = "fakemusl" } openkalwin = { path = "openkalwin" } +[targets.cabi-probe] +kind = "lib" +sources = ["src/*.cpp", "src/*.S"] + [build] allow_host_libs = true EOF diff --git a/tests/e2e/timings/linux.tsv b/tests/e2e/timings/linux.tsv index 494f966b..770bdb79 100644 --- a/tests/e2e/timings/linux.tsv +++ b/tests/e2e/timings/linux.tsv @@ -1,466 +1,515 @@ -00_fixture_path_hygiene.sh 2070 +00_fixture_path_hygiene.sh 2543 01_help_and_version.sh 12 -02_new_build_run.sh 1520 -03_multi_module.sh 2160 -04_incremental.sh 3200 -05_errors.sh 1260 -06_emit_xpkg.sh 34 +02_new_build_run.sh 5529 +03_multi_module.sh 5933 +04_incremental.sh 3342 +05_errors.sh 1223 +06_emit_xpkg.sh 15 07_static_library.sh 133 -08_shared_library.sh 547 -09_path_dependency.sh 2190 -100_cppfly_reflection.sh 13390 -100_feature_sources_test_mode.sh 3800 -101_cppfly_llvm_soft.sh 8 -103_target_vocabulary.sh 5680 -104_asm_sources_gas.sh 2680 -106_feature_gated_sources_toml.sh 6500 -107_generated_files_toml.sh 1310 -108_cfg_conditional_sources.sh 1240 -109_per_glob_flags.sh 2100 -10_env_command.sh 8810 -110_build_mcpp_env_contract.sh 7030 -111_dep_build_mcpp.sh 2290 -113_scanner_mcpp_dir_prune.sh 177 -114_run_scan_scope.sh 1580 -115_manifest_syntax_aot_brace.sh 1240 -116_include_family_and_define_spaces.sh 1020 -117_object_path_collision.sh 1240 -118_purview_include_rebuild.sh 3820 -119_dep_cfg_sources.sh 1420 -11_index_list.sh 9920 -120_ws_root_indices.sh 696 -121_default_ns_redirect.sh 678 -122_run_member.sh 2260 -123_same_named_main_across_dep.sh 1330 -124_gen_source_out_of_root_object_path.sh 1750 -125_build_mcpp_dep_dir.sh 1730 -126_default_features_opt_out.sh 392 -127_transitive_default_features_optout.sh 2050 -128_feature_forwarding.sh 274 -12_add_command.sh 868 -138_feature_sources_gate_vs_provider.sh 582 -139_build_program_advisory.sh 978 -13_toolchain_pin.sh 7 -141_include_dirs_after.sh 1170 -142_generated_files_incremental.sh 2430 -143_build_mcpp_source_directive.sh 2230 -144_build_mcpp_include_dir.sh 3640 -145_root_build_mcpp_dep_dirs.sh 1560 -146_feature_flags.sh 7850 -147_per_os_features_xpkg.sh 639 -148_wide_include_list.sh 1410 -149_conditional_glob_flags.sh 1110 -14_toolchain_fallback.sh 5760 -150_clang_module_operator_template.sh 6 -151_index_url_artifact_migration.sh 6040 -152_test_subdir_names.sh 242 -153_test_isolation.sh 226 -154_test_filter.sh 516 -155_test_json.sh 193 -156_nested_ld_library_path.sh 112 -157_test_glob_flags.sh 135 -158_test_signal_and_duration.sh 473 -159_test_list.sh 18 -15_test_passing.sh 2390 -160_test_timeout.sh 4640 -161_xpkg_name_form.sh 64 -162_bare_name_namespace_scope.sh 5830 -163_identity_first_resolution.sh 26190 -164_ninja_goal_command_length.sh 532 -165_bare_name_cross_namespace_wire_address.sh 5650 -166_run_env_no_private_glibc.sh 345 -167_build_defines_module_scan.sh 653 -168_build_mcpp_musl_host_static.sh 5320 -169_semver_project_index.sh 434 -170_bmi_staging_no_cascade.sh 1110 -171_bmi_staging_locked_dest.sh 2530 -172_build_cache_cross_project.sh 7280 -173_build_profile_isolation.sh 943 -173_index_refresh_policy.sh 598 -174_cache_modes_and_commands.sh 17720 -175_cpp20_import_std.sh 6130 -176_cpp20_std_bmi_isolation.sh 8370 -178_test_observability.sh 10780 -179_spaced_paths.sh 1070 -17_test_no_tests.sh 23 -181_build_mcpp_import_std.sh 6150 -183_cxx_runtime_contract.sh 3450 -184_build_cache_object_layout.sh 12980 -185_index_floor_degrades.sh 4330 -186_build_mcpp_protocol_and_bound.sh 7600 -187_dep_host_tool.sh 3310 -188_build_actions.sh 8710 -189_host_module_rules.sh 1700 -18_devdeps_isolation.sh 1220 -190_link_rspfile_newlines.sh 628 -191_link_scale.sh 18600 -192_install_path_namespace.sh 3630 -193_provision_reexport.sh 8180 -194_build_mcpp_glob_input.sh 2440 -195_target_cfg_feature_deps.sh 406 -196_version_identity_and_lock.sh 15370 -199_resources_validation.sh 207 -19_bmi_cache_reuse.sh 5800 -200_subos_env_reaches_program.sh 1780 -201_gcc_no_specs_pollution.sh 6720 -202_machine_output_contract.sh 490 -203_exact_selector_lock_migration.sh 387 -204_new_transactional_scaffold.sh 8990 -205_root_local_subos.sh 764 -206_runtime_binding_physics.sh 1150 -207_runtime_contract_provenance.sh 204 -208_private_libc_stays_in_the_binary.sh 299 -209_runtime_verdict_never_reruns_green.sh 750 -20_p1689_scanner.sh 4720 -210_local_index_addressing_on_every_host.sh 185 -211_configure_only_cdb.sh 470 -212_cached_dep_std_is_ordered.sh 4820 -213_build_after_test_is_not_the_test_graph.sh 388 -214_executable_carries_dt_rpath.sh 1880 -215_pack_has_no_build_machine_paths.sh 3430 -216_selfcontained_refuses_host_capability.sh 763 -217_module_extensions.sh 4750 -218_module_extensions_graph_shape.sh 5140 -219_runtime_search_farm_is_last.sh 370 -21_ninja_dyndep.sh 4050 -220_farm_only_library_runs.sh 432 -221_subos_without_info_still_builds.sh 140 -222_shared_library_cxx_runtime_contract.sh 2010 -22_doctor_cache_publish.sh 45720 -230_bench_harness.sh 54310 -231_jobs_option.sh 56340 -232_workflow_syntax.sh 323 -233_bench_matrix.sh 559 -234_bmi_schedule_on.sh 6460 -235_std_object_only_when_needed.sh 977 -236_module_extensions_default_toolchain.sh 3300 -237_default_subos_without_info.sh 256 -238_c_only_unit_links_with_c_driver.sh 3660 -23_remove_update.sh 16070 -242_pack_library_interface_and_headers.sh 609 -243_pack_library_interface_closure.sh 665 -244_pack_library_gate.sh 458 -246_explicit_empty_sources.sh 302 -247_bare_triple_conditional_native.sh 390 -249_pack_workspace_root_unchanged.sh 195 -24_git_dependency.sh 9290 -250_pack_names_the_target.sh 256 -251_pack_library_shared.sh 349 -252_pack_library_old_client.sh 309 -253_pack_library_undetermined_partition.sh 490 -254_pack_library_platform_coverage.sh 498 -25_convention_mode.sh 4460 -260_module_extension_is_configuration.sh 463 -261_pack_follows_the_module_extension.sh 463 -263_lib_root_follows_the_extension.sh 1230 -264_pack_library_is_relocatable.sh 397 +08_shared_library.sh 189 +09_path_dependency.sh 2437 +100_cppfly_reflection.sh 12254 +100_feature_sources_test_mode.sh 3887 +101_cppfly_llvm_soft.sh 3836 +103_target_vocabulary.sh 11073 +104_asm_sources_gas.sh 2741 +105_asm_sources_nasm.sh 8738 +106_feature_gated_sources_toml.sh 6781 +107_generated_files_toml.sh 1598 +108_cfg_conditional_sources.sh 1198 +109_per_glob_flags.sh 1218 +10_env_command.sh 5999 +110_build_mcpp_env_contract.sh 8839 +111_dep_build_mcpp.sh 2142 +112_build_mcpp_cross.sh 7194 +113_scanner_mcpp_dir_prune.sh 179 +114_run_scan_scope.sh 1563 +115_manifest_syntax_aot_brace.sh 1308 +116_include_family_and_define_spaces.sh 1322 +117_object_path_collision.sh 1311 +118_purview_include_rebuild.sh 8825 +119_dep_cfg_sources.sh 1351 +11_index_list.sh 7507 +120_ws_root_indices.sh 722 +121_default_ns_redirect.sh 491 +122_run_member.sh 2372 +123_same_named_main_across_dep.sh 1290 +124_gen_source_out_of_root_object_path.sh 1698 +125_build_mcpp_dep_dir.sh 1814 +126_default_features_opt_out.sh 391 +127_transitive_default_features_optout.sh 2577 +128_feature_forwarding.sh 279 +12_add_command.sh 450 +134_target_sysroot_and_queries.sh 4462 +135_diagnostic_promise_resolves.sh 7996 +136_aarch64_zero_libc_target.sh 577 +137_x86_64_zero_libc_target.sh 539 +138_feature_sources_gate_vs_provider.sh 601 +139_build_program_advisory.sh 637 +13_toolchain_pin.sh 8 +141_include_dirs_after.sh 1319 +142_generated_files_incremental.sh 2431 +143_build_mcpp_source_directive.sh 2716 +144_build_mcpp_include_dir.sh 3689 +145_root_build_mcpp_dep_dirs.sh 1621 +146_feature_flags.sh 8190 +147_per_os_features_xpkg.sh 504 +148_wide_include_list.sh 1408 +149_conditional_glob_flags.sh 1264 +14_toolchain_fallback.sh 7356 +150_clang_module_operator_template.sh 207 +151_index_url_artifact_migration.sh 5984 +152_test_subdir_names.sh 141 +153_test_isolation.sh 222 +154_test_filter.sh 150 +155_test_json.sh 188 +156_nested_ld_library_path.sh 124 +157_test_glob_flags.sh 140 +158_test_signal_and_duration.sh 308 +159_test_list.sh 24 +15_test_passing.sh 2176 +160_test_timeout.sh 4651 +161_xpkg_name_form.sh 63 +162_bare_name_namespace_scope.sh 5879 +163_identity_first_resolution.sh 25198 +164_ninja_goal_command_length.sh 529 +165_bare_name_cross_namespace_wire_address.sh 6126 +166_run_env_no_private_glibc.sh 353 +167_build_defines_module_scan.sh 813 +168_build_mcpp_musl_host_static.sh 5623 +169_semver_project_index.sh 511 +16_test_failing.sh 2361 +170_bmi_staging_no_cascade.sh 1320 +171_bmi_staging_locked_dest.sh 2585 +172_build_cache_cross_project.sh 9478 +173_build_profile_isolation.sh 1114 +173_index_refresh_policy.sh 791 +174_cache_modes_and_commands.sh 17259 +175_cpp20_import_std.sh 7801 +176_cpp20_std_bmi_isolation.sh 8248 +178_test_observability.sh 10866 +179_spaced_paths.sh 1435 +17_test_no_tests.sh 16 +181_build_mcpp_import_std.sh 5332 +183_cross_no_relink.sh 1167 +183_cxx_runtime_contract.sh 4291 +184_build_cache_object_layout.sh 12353 +185_index_floor_degrades.sh 5647 +186_build_mcpp_protocol_and_bound.sh 7594 +187_dep_host_tool.sh 3472 +188_build_actions.sh 8591 +189_host_module_rules.sh 1893 +18_devdeps_isolation.sh 1181 +190_link_rspfile_newlines.sh 1278 +191_link_scale.sh 20371 +192_install_path_namespace.sh 3686 +193_provision_reexport.sh 8603 +194_build_mcpp_glob_input.sh 2471 +195_target_cfg_feature_deps.sh 410 +196_version_identity_and_lock.sh 15118 +198_windows_resources_cross.sh 3729 +199_resources_validation.sh 222 +19_bmi_cache_reuse.sh 6699 +200_subos_env_reaches_program.sh 1778 +201_gcc_no_specs_pollution.sh 8564 +202_machine_output_contract.sh 2576 +203_exact_selector_lock_migration.sh 358 +204_new_transactional_scaffold.sh 9698 +205_root_local_subos.sh 732 +206_runtime_binding_physics.sh 1177 +207_runtime_contract_provenance.sh 200 +208_private_libc_stays_in_the_binary.sh 314 +209_runtime_verdict_never_reruns_green.sh 723 +20_p1689_scanner.sh 4818 +210_local_index_addressing_on_every_host.sh 248 +211_configure_only_cdb.sh 486 +212_cached_dep_std_is_ordered.sh 5431 +213_build_after_test_is_not_the_test_graph.sh 396 +214_executable_carries_dt_rpath.sh 1506 +215_pack_has_no_build_machine_paths.sh 3483 +216_selfcontained_refuses_host_capability.sh 545 +217_module_extensions.sh 9988 +218_module_extensions_graph_shape.sh 5117 +219_runtime_search_farm_is_last.sh 345 +21_ninja_dyndep.sh 4670 +220_farm_only_library_runs.sh 880 +221_subos_without_info_still_builds.sh 157 +222_shared_library_cxx_runtime_contract.sh 2362 +22_doctor_cache_publish.sh 47292 +230_bench_harness.sh 55578 +231_jobs_option.sh 2197 +232_workflow_syntax.sh 325 +233_bench_matrix.sh 224 +234_bmi_schedule_on.sh 6695 +235_std_object_only_when_needed.sh 1071 +236_module_extensions_default_toolchain.sh 3494 +237_default_subos_without_info.sh 287 +238_c_only_unit_links_with_c_driver.sh 3775 +23_remove_update.sh 12842 +240_pack_pe_zip_cross.sh 414 +242_pack_library_interface_and_headers.sh 779 +243_pack_library_interface_closure.sh 681 +244_pack_library_gate.sh 536 +245_pack_library_fat_target_selection.sh 763 +246_explicit_empty_sources.sh 261 +247_bare_triple_conditional_native.sh 389 +248_pack_library_fat_pe_leg.sh 513 +249_pack_workspace_root_unchanged.sh 217 +24_git_dependency.sh 10859 +250_pack_names_the_target.sh 261 +251_pack_library_shared.sh 375 +252_pack_library_old_client.sh 308 +253_pack_library_undetermined_partition.sh 532 +254_pack_library_platform_coverage.sh 503 +25_convention_mode.sh 4566 +260_module_extension_is_configuration.sh 198 +261_pack_follows_the_module_extension.sh 465 +263_lib_root_follows_the_extension.sh 1252 +264_pack_library_is_relocatable.sh 424 265_pack_strips_but_stays_usable.sh 916 -267_target_requested_is_empty_for_a_native_build.sh 746 -268_target_side_resolution.sh 218 -26_c_language_support.sh 1050 -26_toolchain_management.sh 40240 -27_namespace_dependencies.sh 6800 -27_self_contained_home.sh 11520 -280_target_side_layers.sh 2130 -281_target_side_rules.sh 212 -282_target_is_a_request.sh 64 -283_run_target_flag_owns_its_cache_slot.sh 305 -284_env_segment_is_optional_everywhere.sh 13840 -285_kernel_abi_from_graph_keeps_the_payload_c_library.sh 4500 -290_the_declaration_puts_an_environment_in_front.sh 1790 -293_the_requested_target_and_the_resolved_one_name_one_os.sh 7510 -295_naming_the_host_target_changes_nothing.sh 1440 -296_what_the_report_names_is_what_the_link_line_uses.sh 95750 -297_a_capability_pin_is_not_a_preference.sh 193 -299_a_request_that_named_no_c_library_resolves_to_a_row_that_exists.sh 92 -29_toolchain_partial_versions.sh 5700 -300_a_registered_family_is_not_reported_unknown.sh 36 -301_the_graphs_compiler_is_taken_and_nothing_is_written.sh 277 -302_a_stated_compiler_outranks_the_graph_and_two_requirements_do_not_stack.sh 95 -303_the_query_gives_one_answer_for_the_c_library.sh 103 -304_the_target_side_reaches_every_unit.sh 158 -305_built_from_is_not_published.sh 204 -306_dependency_linkage_axis.sh 346 -307_symbol_provision.sh 503 -308_dependency_required_features.sh 222 -309_host_module_identity.sh 950 -30_dev_binary_home.sh 19 -30_pack_modes.sh 5700 -310_build_dependencies.sh 1980 -311_rule_build_dependencies.sh 2770 -312_build_rules_example.sh 2960 -313_check_stamp_on_every_platform.sh 594 -314_dependency_action_generated_header.sh 2680 -315_blocking_check_gates_compilation.sh 4940 -316_link_unit_with_no_inputs_is_refused.sh 194 -317_project_build_hooks.sh 18670 -321_workspace_inheritance.sh 692 -322_dialect_flag_refusal.sh 9590 -323_post_link_record_durability.sh 425 -324_fast_path_sees_path_deps.sh 3630 -325_system_toolchain_refused.sh 176 -326_dependency_standard_floor.sh 834 -327_declared_xlings_deps_report_their_own_failure.sh 8800 -328_layer_predicates_apply_and_unknown_ones_say_so.sh 1540 -329_exit_codes_match_the_contract.sh 19 -32_semver_merge.sh 8710 -330_runner_hosted_targets.sh 1310 -331_xlings_workspace_one_table.sh 1810 -333_named_runners_locked_and_sbom.sh 4610 -334_dependency_declared_tool_is_found.sh 242 -335_xlings_tool_tiers.sh 32 -337_run_takes_features_and_profile.sh 1050 -339_run_reports_the_program_status.sh 1240 -33_multi_version_mangling.sh 13100 -340_no_orphan_survives_a_killed_mcpp.sh 11100 -35_workspace.sh 1860 -36_llvm_toolchain.sh 992 -38_self_config_mirror.sh 217 -39_xlings_index_migration.sh 209 -42_custom_local_index.sh 9340 -43_indices_lockfile.sh 7070 -44_indices_e2e_integration.sh 9270 -45_install_platform_mapping.sh 13 -46_self_config_mirror_no_bootstrap.sh 8070 -47_cdb_prebuilt_module_path_abs.sh 1210 -48_build_error_output.sh 284 -49_bmi_cache_nested_custom_index.sh 5420 +267_target_requested_is_empty_for_a_native_build.sh 744 +268_target_side_resolution.sh 242 +269_openkal_llvm_spelling_still_resolves.sh 485 +26_c_language_support.sh 1293 +26_toolchain_management.sh 40215 +27_namespace_dependencies.sh 8849 +27_self_contained_home.sh 16869 +280_target_side_layers.sh 2345 +281_target_side_rules.sh 223 +282_target_is_a_request.sh 67 +283_run_target_flag_owns_its_cache_slot.sh 323 +284_env_segment_is_optional_everywhere.sh 3132 +285_kernel_abi_from_graph_keeps_the_payload_c_library.sh 4622 +286_the_openkal_stack_still_builds.sh 40171 +287_the_openkal_stack_crosses_to_aarch64.sh 38940 +289_one_host_reaches_every_openkal_target.sh 77137 +28_target_static.sh 4 +290_the_declaration_puts_an_environment_in_front.sh 1998 +291_dynamic_linkage_is_refused_only_when_the_c_library_is_the_graphs.sh 30147 +292_a_package_that_names_a_layer_does_not_lose_the_targets_compiler.sh 355 +293_the_requested_target_and_the_resolved_one_name_one_os.sh 9515 +294_the_list_answers_what_can_be_built_not_what_has_a_payload.sh 23 +295_naming_the_host_target_changes_nothing.sh 1370 +296_what_the_report_names_is_what_the_link_line_uses.sh 26315 +297_a_capability_pin_is_not_a_preference.sh 251 +298_overriding_a_convention_requires_replacing_it.sh 28550 +299_a_request_that_named_no_c_library_resolves_to_a_row_that_exists.sh 9159 +29_toolchain_partial_versions.sh 5553 +300_a_registered_family_is_not_reported_unknown.sh 43 +301_the_graphs_compiler_is_taken_and_nothing_is_written.sh 298 +302_a_stated_compiler_outranks_the_graph_and_two_requirements_do_not_stack.sh 85 +303_the_query_gives_one_answer_for_the_c_library.sh 107 +304_the_target_side_reaches_every_unit.sh 188 +305_built_from_is_not_published.sh 228 +306_dependency_linkage_axis.sh 376 +307_symbol_provision.sh 551 +308_dependency_required_features.sh 213 +309_host_module_identity.sh 987 +30_dev_binary_home.sh 20 +30_pack_modes.sh 5699 +310_build_dependencies.sh 749 +311_rule_build_dependencies.sh 3019 +312_build_rules_example.sh 3353 +313_check_stamp_on_every_platform.sh 639 +314_dependency_action_generated_header.sh 2676 +315_blocking_check_gates_compilation.sh 4923 +316_link_unit_with_no_inputs_is_refused.sh 144 +317_project_build_hooks.sh 18666 +31_transitive_deps.sh 142 +321_workspace_inheritance.sh 687 +322_dialect_flag_refusal.sh 10582 +323_post_link_record_durability.sh 422 +324_fast_path_sees_path_deps.sh 3532 +325_system_toolchain_refused.sh 222 +326_dependency_standard_floor.sh 740 +327_declared_xlings_deps_report_their_own_failure.sh 7813 +328_layer_predicates_apply_and_unknown_ones_say_so.sh 808 +329_exit_codes_match_the_contract.sh 20 +32_semver_merge.sh 9628 +330_runner_hosted_targets.sh 1428 +331_xlings_workspace_one_table.sh 1855 +333_named_runners_locked_and_sbom.sh 7286 +334_dependency_declared_tool_is_found.sh 251 +335_xlings_tool_tiers.sh 37 +337_run_takes_features_and_profile.sh 961 +339_run_reports_the_program_status.sh 956 +33_multi_version_mangling.sh 15107 +340_no_orphan_survives_a_killed_mcpp.sh 11268 +35_workspace.sh 1807 +36_llvm_toolchain.sh 1246 +37_llvm_import_std.sh 3445 +38_llvm_modules.sh 4184 +38_self_config_mirror.sh 258 +39_llvm_incremental.sh 5127 +39_xlings_index_migration.sh 220 +40_llvm_bmi_cache.sh 5557 +41_llvm_std_compat.sh 2913 +42_custom_local_index.sh 8323 +43_indices_lockfile.sh 8452 +44_indices_e2e_integration.sh 7633 +45_install_platform_mapping.sh 12 +46_self_config_mirror_no_bootstrap.sh 6953 +47_cdb_prebuilt_module_path_abs.sh 1236 +47_llvm_atomic_link.sh 669 +48_build_error_output.sh 296 +49_bmi_cache_nested_custom_index.sh 6825 50_package_owned_build_flags.sh 182 -51_package_generated_files.sh 293 -52_local_path_namespaced_index.sh 477 -53_namespaced_cache_label.sh 398 -54_package_owned_ldflags.sh 268 -55_dependency_shared_artifact.sh 194 -56_transitive_shared_artifact.sh 232 -57_static_dep_shared_artifact.sh 205 -58_preinstall_mcpp_deps_for_hooks.sh 285 -59_cpp_standard_config.sh 5670 +51_package_generated_files.sh 351 +52_local_path_namespaced_index.sh 486 +53_namespaced_cache_label.sh 454 +54_package_owned_ldflags.sh 262 +55_dependency_shared_artifact.sh 206 +56_transitive_shared_artifact.sh 237 +57_static_dep_shared_artifact.sh 208 +58_preinstall_mcpp_deps_for_hooks.sh 295 +59_cpp_standard_config.sh 6668 600_accel_variant_selection.sh 415 -601_exclusive_capability.sh 211 -603_version_floor.sh 299 -604_offline_skips_sandbox_bootstrap.sh 366 -605_probe_channel_and_accel_reach_the_build_program.sh 1330 -606_constrained_source_globs_narrow_to_the_build.sh 1240 -607_chained_actions_form_the_device_link.sh 1100 -608_object_actions_reach_a_static_library.sh 785 -609_clean_stale.sh 4400 -609_shader_sources_are_device_sources.sh 1920 -60_stale_xpkg_cache_reinstall.sh 316 -610_feature_controlled_rule_collection.sh 4120 -611_variant_switch_is_not_replayed.sh 622 -612_build_program_file_input_fast_path.sh 1300 -613_sycl_sources_are_device_sources.sh 1720 -614_feature_xlings_reaches_xpkg_dir.sh 1160 -615_feature_deps_shared_library_is_linked.sh 470 -616_examples_and_their_index_agree.sh 242 -617_build_program_knows_the_cxx_stdlib.sh 550 -618_dependency_feature_tools_reach_its_build_program.sh 43940 -619_a_tool_may_not_be_conditioned_on_a_layer.sh 28 -61_dependency_visibility.sh 402 -620_link_flag_reaches_the_link_line.sh 548 -621_exports_narrows_the_symbol_surface.sh 591 -622_rule_declared_payload_reaches_the_consumers_build_program.sh 48530 -623_device_sources_must_reach_an_action.sh 263 -624_a_missing_host_module_names_the_key.sh 295 -625_a_payload_can_be_gated_on_the_accelerator.sh 43450 -626_naming_a_subset_of_backends_is_not_a_mismatch.sh 421 -627_one_package_one_version.sh 41240 -628_a_pin_below_a_stated_floor_is_refused.sh 43980 -629_a_version_range_is_answered_by_xpkg_dir.sh 49180 -62_dotted_dependency_selector_priority.sh 380 -62_runtime_library_dirs.sh 595 -630_a_rule_brings_its_own_environment.sh 44040 -631_action_depfile_tracks_a_runtime_discovered_include.sh 1110 -632_a_build_program_is_told_its_package_name.sh 546 -633_host_modules_are_ordered_by_what_they_import.sh 1660 -634_a_private_module_fragment_is_not_a_partition.sh 127 -635_a_byte_order_mark_does_not_hide_a_module.sh 326 -636_a_module_extension_file_need_not_provide_a_module.sh 163 -637_a_conditional_runtime_table_alone_is_applied.sh 730 -638_pack_format_dispatch.sh 5620 -639_the_scanner_does_not_read_inside_a_comment.sh 888 -63_bare_dependency_peer_root_priority.sh 356 -640_a_capability_pin_explains_its_own_row.sh 821 -641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh 90540 -644_windows_keys_are_inert_elsewhere.sh 847 -645_the_fast_path_compares_the_toolchain_request.sh 678 -646_runtime_deploy_places_files_in_a_directory.sh 1630 -647_abi_threads_is_one_switch_for_the_graph.sh 646 -648_an_install_hook_sees_the_build_target.sh 874 -649_pack_carries_runtime_deploy_files.sh 4650 -64_shared_soname_runtime_alias.sh 774 -650_wasm_row_names_its_launcher.sh 20 -651_a_build_program_deploys_what_it_generated.sh 1740 -652_an_application_is_a_kind.sh 937 +601_exclusive_capability.sh 172 +603_version_floor.sh 303 +604_offline_skips_sandbox_bootstrap.sh 434 +605_probe_channel_and_accel_reach_the_build_program.sh 1349 +606_constrained_source_globs_narrow_to_the_build.sh 1145 +607_chained_actions_form_the_device_link.sh 1132 +608_object_actions_reach_a_static_library.sh 532 +609_clean_stale.sh 4532 +609_shader_sources_are_device_sources.sh 1679 +60_stale_xpkg_cache_reinstall.sh 330 +610_feature_controlled_rule_collection.sh 4584 +611_variant_switch_is_not_replayed.sh 601 +612_build_program_file_input_fast_path.sh 1232 +613_sycl_sources_are_device_sources.sh 1808 +614_feature_xlings_reaches_xpkg_dir.sh 1043 +615_feature_deps_shared_library_is_linked.sh 481 +616_examples_and_their_index_agree.sh 292 +617_build_program_knows_the_cxx_stdlib.sh 566 +618_dependency_feature_tools_reach_its_build_program.sh 45430 +619_a_tool_may_not_be_conditioned_on_a_layer.sh 29 +61_dependency_visibility.sh 476 +620_link_flag_reaches_the_link_line.sh 579 +621_exports_narrows_the_symbol_surface.sh 570 +622_rule_declared_payload_reaches_the_consumers_build_program.sh 50647 +623_device_sources_must_reach_an_action.sh 292 +624_a_missing_host_module_names_the_key.sh 304 +625_a_payload_can_be_gated_on_the_accelerator.sh 44636 +626_naming_a_subset_of_backends_is_not_a_mismatch.sh 434 +627_one_package_one_version.sh 46552 +628_a_pin_below_a_stated_floor_is_refused.sh 47993 +629_a_version_range_is_answered_by_xpkg_dir.sh 47222 +62_dotted_dependency_selector_priority.sh 420 +62_runtime_library_dirs.sh 414 +630_a_rule_brings_its_own_environment.sh 44172 +631_action_depfile_tracks_a_runtime_discovered_include.sh 944 +632_a_build_program_is_told_its_package_name.sh 550 +633_host_modules_are_ordered_by_what_they_import.sh 1288 +634_a_private_module_fragment_is_not_a_partition.sh 130 +635_a_byte_order_mark_does_not_hide_a_module.sh 305 +636_a_module_extension_file_need_not_provide_a_module.sh 165 +637_a_conditional_runtime_table_alone_is_applied.sh 519 +638_pack_format_dispatch.sh 5742 +639_the_scanner_does_not_read_inside_a_comment.sh 790 +63_bare_dependency_peer_root_priority.sh 378 +640_a_capability_pin_explains_its_own_row.sh 9920 +641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh 81543 +643_windows_subsystem_cross.sh 1455 +644_windows_keys_are_inert_elsewhere.sh 598 +645_the_fast_path_compares_the_toolchain_request.sh 684 +646_runtime_deploy_places_files_in_a_directory.sh 1881 +647_abi_threads_is_one_switch_for_the_graph.sh 642 +648_an_install_hook_sees_the_build_target.sh 868 +649_pack_carries_runtime_deploy_files.sh 3179 +64_shared_soname_runtime_alias.sh 404 +650_wasm_row_names_its_launcher.sh 21 +651_a_build_program_deploys_what_it_generated.sh 1872 +652_an_application_is_a_kind.sh 514 653_exceptions_is_the_second_abi_member.sh 6 -654_a_requirement_on_the_target_axis.sh 1000 -656_run_hands_the_distributable_to_the_runner.sh 2650 +654_a_requirement_on_the_target_axis.sh 715 +656_run_hands_the_distributable_to_the_runner.sh 2616 657_a_build_program_under_the_web_row_uses_the_host_compiler.sh 3 -659_an_action_declaration_has_no_size_limit.sh 1290 -660_an_action_names_the_engine_through_mcpp_self.sh 1620 -661_one_identity_two_declarations.sh 744 -662_pack_stages_declared_files_before_the_closure.sh 5930 -665_a_host_tool_follows_its_source.sh 7930 -669_an_elf_host_pack_states_its_closure.sh 822 -66_runtime_provides.sh 1060 -670_an_rpath_loader_token_is_left_as_written.sh 164 -671_toolchain_is_an_option_of_run_test_and_pack.sh 5530 -672_a_runner_receives_the_files_the_artifact_carries.sh 9530 -673_test_discover_names_where_tests_are.sh 1080 -674_a_format_is_run_by_the_runner_named_after_it.sh 7840 -677_a_conditional_dependency_replaces_the_unconditional_one.sh 691 -678_a_row_states_a_library_form.sh 1280 -679_a_path_dependency_takes_its_manifests_identity.sh 619 -67_features_strict.sh 6560 -681_index_repos_reach_an_existing_home.sh 8070 -682_why_deps_prints_the_resolved_graph.sh 204 -683_a_build_program_reads_the_payload_pkg_config_view.sh 575 -684_a_runner_supplied_twice_is_refused.sh 517 -685_a_nested_test_loads_a_graph_shared_library.sh 397 -686_an_activated_xlings_subos_does_not_redirect_the_registry.sh 7720 -687_the_vendored_xlings_probe_is_an_argument_vector.sh 1220 -688_emit_build_database.sh 4810 -689_pack_takes_features.sh 1680 -68_profile_passthrough.sh 7460 -692_a_package_states_its_default_link_form.sh 1190 -693_the_root_build_program_reads_dependency_link_forms.sh 3420 -698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 3940 -69_package_templates.sh 11110 -701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh 1430 -702_a_static_package_under_a_shared_library_is_linked_into_it.sh 1260 -70_per_target_flags.sh 221 -710_a_forward_reaches_every_dependency_table.sh 422 -711_a_feature_deps_restatement_names_one_source.sh 4890 -712_a_tool_package_contributes_nothing_to_its_consumer.sh 5930 -713_a_git_dependency_selects_a_repository_member.sh 5390 -714_features_names_a_dependency_feature.sh 5740 -71_required_features_gate.sh 303 -720_the_root_build_program_reads_the_resolved_graph.sh 2680 -722_pack_strips_what_the_graph_built.sh 3670 -723_pack_reports_what_it_produced.sh 960 -72_test_profile_features.sh 336 -730_bare_compat_dependency_is_not_a_refresh_miss.sh 74 -731_planning_children_do_not_inherit_the_callers_pipe.sh 444 -732_xlings_refresh_is_bounded_and_owned.sh 4990 -733_envelope_reports_observed_network_access.sh 323 -734_auto_refresh_false_governs_first_index_sync.sh 107 -735_offline_missing_package_has_its_own_code.sh 101 -736_compile_flag_words_reach_the_compiler_and_the_databases.sh 346 -737_the_declared_runtime_payload_is_installed.sh 66950 -739_a_private_feature_dep_does_not_reach_the_consumer.sh 282 -73_issue131_per_target_cxxflag.sh 195 -740_gcc_over_a_graph_c_library_is_degraded_not_refused.sh 113 -743_kernel_abi_interfaces_are_resolved_not_preprocessed.sh 186 +659_an_action_declaration_has_no_size_limit.sh 1262 +65_toolchain_runtime_dirs_for_run.sh 661 +660_an_action_names_the_engine_through_mcpp_self.sh 1358 +661_one_identity_two_declarations.sh 754 +662_pack_stages_declared_files_before_the_closure.sh 7113 +663_a_graph_libcxx_over_the_payloads_c_library.sh 17317 +665_a_host_tool_follows_its_source.sh 9337 +669_an_elf_host_pack_states_its_closure.sh 1078 +66_runtime_provides.sh 1271 +670_an_rpath_loader_token_is_left_as_written.sh 189 +671_toolchain_is_an_option_of_run_test_and_pack.sh 4156 +672_a_runner_receives_the_files_the_artifact_carries.sh 9538 +673_test_discover_names_where_tests_are.sh 1177 +674_a_format_is_run_by_the_runner_named_after_it.sh 8008 +677_a_conditional_dependency_replaces_the_unconditional_one.sh 728 +678_a_row_states_a_library_form.sh 860 +679_a_path_dependency_takes_its_manifests_identity.sh 669 +67_features_strict.sh 7264 +681_index_repos_reach_an_existing_home.sh 9473 +682_why_deps_prints_the_resolved_graph.sh 213 +683_a_build_program_reads_the_payload_pkg_config_view.sh 514 +684_a_runner_supplied_twice_is_refused.sh 518 +685_a_nested_test_loads_a_graph_shared_library.sh 405 +686_an_activated_xlings_subos_does_not_redirect_the_registry.sh 7823 +687_the_vendored_xlings_probe_is_an_argument_vector.sh 629 +688_emit_build_database.sh 4886 +689_pack_takes_features.sh 2163 +68_profile_passthrough.sh 8118 +690_a_shared_library_over_a_graph_cxx_runtime.sh 29859 +692_a_package_states_its_default_link_form.sh 1247 +693_the_root_build_program_reads_dependency_link_forms.sh 4248 +696_a_cxx_layer_provider_keeps_its_own_standard.sh 33280 +698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 4322 +69_package_templates.sh 9948 +700_a_program_over_a_cxx_shared_library_has_one_cxx_runtime.sh 1123 +701_symbol_provision_does_not_report_what_the_build_shares_by_construction.sh 1437 +702_a_static_package_under_a_shared_library_is_linked_into_it.sh 747 +70_per_target_flags.sh 231 +710_a_forward_reaches_every_dependency_table.sh 454 +711_a_feature_deps_restatement_names_one_source.sh 5820 +712_a_tool_package_contributes_nothing_to_its_consumer.sh 6191 +713_a_git_dependency_selects_a_repository_member.sh 6173 +714_features_names_a_dependency_feature.sh 5951 +71_required_features_gate.sh 253 +720_the_root_build_program_reads_the_resolved_graph.sh 2778 +722_pack_strips_what_the_graph_built.sh 3741 +723_pack_reports_what_it_produced.sh 857 +72_test_profile_features.sh 337 +730_bare_compat_dependency_is_not_a_refresh_miss.sh 97 +731_planning_children_do_not_inherit_the_callers_pipe.sh 438 +732_xlings_refresh_is_bounded_and_owned.sh 4999 +733_envelope_reports_observed_network_access.sh 344 +734_auto_refresh_false_governs_first_index_sync.sh 159 +735_offline_missing_package_has_its_own_code.sh 118 +736_compile_flag_words_reach_the_compiler_and_the_databases.sh 414 +737_the_declared_runtime_payload_is_installed.sh 72189 +739_a_private_feature_dep_does_not_reach_the_consumer.sh 296 +73_issue131_per_target_cxxflag.sh 177 +740_gcc_over_a_graph_c_library_is_degraded_not_refused.sh 123 +741_c_abi_environment_declared_by_the_c_library.sh 177 +742_gcc_accepted_when_the_realisation_is_empty.sh 112 +743_kernel_abi_interfaces_are_resolved_not_preprocessed.sh 205 744_a_declared_absence_explains_the_link_that_asked_for_it.sh 90 -745_no_run_builds_the_tests_and_says_so.sh 411 +745_no_run_builds_the_tests_and_says_so.sh 367 746_macos_deployment_target_is_target_keyed_not_host_keyed.sh 233 -74_run_no_loader_env_leak.sh 1060 -75_index_status_offline.sh 7160 -765_a_consumer_include_directory_stays_in_the_consumer.sh 611 -76_compile_commands_generated.sh 1020 -76_qualified_custom_ns_noncanonical_filename.sh 457 -770_workspace_member_as_dependency.sh 996 -772_a_published_member_is_self_contained.sh 5520 -773_commands_outside_the_build_read_the_effective_manifest.sh 275 -774_an_index_member_inherits_its_archive_workspace.sh 319 -775_a_host_tool_merges_its_conditional_sections_once.sh 4270 -776_a_path_with_no_utf8_spelling_is_named.sh 308 -777_c_standard_applies_to_the_package_that_declares_it.sh 176 -779_a_build_program_declares_a_runtime_search_dir.sh 1810 +74_run_no_loader_env_leak.sh 1306 +75_index_status_offline.sh 8087 +765_a_consumer_include_directory_stays_in_the_consumer.sh 675 +76_compile_commands_generated.sh 1245 +76_qualified_custom_ns_noncanonical_filename.sh 458 +770_workspace_member_as_dependency.sh 1139 +772_a_published_member_is_self_contained.sh 5544 +773_commands_outside_the_build_read_the_effective_manifest.sh 349 +774_an_index_member_inherits_its_archive_workspace.sh 324 +775_a_host_tool_merges_its_conditional_sections_once.sh 5157 +776_a_path_with_no_utf8_spelling_is_named.sh 314 +777_c_standard_applies_to_the_package_that_declares_it.sh 187 +778_a_graph_link_searches_no_host_directory.sh 99076 +779_a_build_program_declares_a_runtime_search_dir.sh 1953 77_cdb_preserves_test_entries.sh 4460 -780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 1730 -781_deleted_root_cdb_is_restored.sh 2280 -782_foreign_root_cdb_entries_are_replaced.sh 2970 -787_emit_plans_every_member_on_its_own.sh 161 -788_emit_defers_an_unbuilt_host_tool.sh 996 -789_emit_describes_a_member_past_a_failed_build_program.sh 326 -78_test_main_combinations.sh 9330 -790_a_prepare_action_populates_an_unknown_directory.sh 1810 -791_a_prepare_action_that_creates_nothing_fails.sh 745 -792_a_rerun_input_inside_a_prepare_directory_is_warned.sh 1700 -793_an_unknown_action_role_is_refused.sh 2310 -795_a_link_flag_reaches_the_linker_as_written.sh 896 -798_a_rule_applies_through_a_source_it_claims.sh 13490 -799_an_action_runs_with_its_env_and_cwd.sh 828 -79_gtest_regular_dep_feature_main.sh 14480 -800_a_feature_provides_its_host_tools.sh 836 -801_a_dependency_program_is_shipped_with_the_consumer.sh 1800 -802_a_host_build_applies_its_host_row.sh 1330 -803_a_recorded_payload_that_is_gone_is_refused_offline.sh 102 -804_a_path_host_tool_builds_with_its_chosen_toolchain.sh 983 -805_rooted_workspace_reaches_its_own_path_dependency.sh 475 -806_dash_p_resolves_the_package_first.sh 799 -807_a_lib_root_imports_its_own_package.sh 506 -808_device_source_is_not_a_compile_unit.sh 11630 -809_failed_build_program_hides_no_device_source.sh 84 -80_feature_defines.sh 296 -810_two_packages_deploy_the_same_bytes_to_one_name.sh 1950 -812_an_index_floor_is_a_closing_tip.sh 206 -813_conditional_dialect_cxxflags_root_only.sh 18140 -815_the_database_describes_what_a_rule_generates.sh 780 -816_a_git_dependency_reports_its_fetch.sh 194 -817_emit_writes_no_xlings_json_under_private_work_dir.sh 211 -819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 316 -81_capability_binding.sh 559 -821_an_edge_states_its_advice_on_both_paths.sh 1590 -822_a_package_states_the_oldest_mcpp_it_supports.sh 267 -823_mcpp_core_and_mcpp_name_one_interface.sh 763 -824_a_build_program_reads_the_resolved_toolchain.sh 1030 -826_plugin_diagnostics_features_and_names.sh 666 -827_many_placements_run_as_one_process.sh 2290 -828_a_library_states_its_interface.sh 648 -829_pack_takes_a_member.sh 274 -82_feature_optional_deps.sh 259 -831_a_path_dependency_host_module_outside_src.sh 4380 -832_the_fast_path_resumes_after_an_edit.sh 3080 -833_a_workspace_is_one_graph_per_configuration.sh 1660 -834_a_workspace_plans_once_and_replays_in_one_check.sh 1290 -835_a_shared_library_is_placed_by_a_link.sh 297 -836_a_members_own_statements_reach_its_build.sh 1300 -838_a_program_shipped_through_artifacts_links_its_own_closure.sh 731 -839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 679 -83_feature_defines_propagate.sh 211 -840_a_workspace_database_describes_each_package_once.sh 551 -841_pack_summarizes_many_outputs.sh 3010 -842_the_build_reports_each_step_once.sh 7190 -843_a_terminal_names_the_action_that_runs.sh 4640 -844_a_planned_build_keeps_one_lock_entry_per_identity.sh 790 -845_play_game_restores_the_terminal.sh 12110 -846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 744 -847_a_module_name_is_unique_within_a_program.sh 2000 -849_a_staged_bmi_waits_for_a_module_compiled_here.sh 1470 -84_runtime_dll_deploy.sh 159 -850_a_selected_member_declares_as_the_root.sh 704 -851_a_shared_member_is_compiled_once_per_configuration.sh 521 -852_a_repeated_dash_p_selects_every_member_it_names.sh 334 -853_run_keeps_one_member_and_exclude_removes_members.sh 721 -854_a_test_over_several_members_plans_and_builds_once.sh 1370 -855_a_shared_test_build_is_reported_once_per_group.sh 345 -856_a_test_over_several_members_continues_past_a_failing_one.sh 1620 -857_a_host_module_several_members_import_is_compiled_once.sh 796 -858_the_engines_module_is_compiled_once_per_home.sh 4800 -859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 3170 -85_target_cfg_build_flags.sh 197 -860_a_workspaces_programs_compile_at_the_same_time.sh 16520 -861_concurrent_compiles_plan_what_serial_compiles_plan.sh 1080 -862_run_q_writes_exactly_the_programs_stdout.sh 196 -863_a_run_whose_build_failed_exits_101.sh 804 -864_a_builds_status_is_on_standard_error.sh 754 -865_mcpp_test_reports_its_steps_on_stderr_and_its_results_on_stdout.sh 557 -866_both_streams_on_one_pipe_keep_their_order_and_json_stays_clean.sh 3990 -867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh 8320 -868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh 5380 -869_a_pack_over_several_members_names_each_member_in_its_report.sh 1490 -86_target_cfg_dependencies.sh 145 -870_a_pack_over_several_members_resolves_and_fails_per_member.sh 6230 -871_a_pack_states_its_build_and_every_drive_takes_the_job_count.sh 10490 -872_a_member_compiles_once_whatever_the_selection.sh 1870 -87_build_default_profile.sh 634 -88_xlings_environment.sh 1620 -89_build_mcpp.sh 2910 -90_workspace_test.sh 1510 -91_target_bare_alias.sh 276 +780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 1689 +781_deleted_root_cdb_is_restored.sh 2308 +782_foreign_root_cdb_entries_are_replaced.sh 3587 +783_cdb_switches_whole_with_the_configuration.sh 3204 +784_cdb_replays_from_its_directory.sh 445 +785_cdb_interface_flag_module_extensions.sh 221 +786_std_unit_in_the_database_and_build_id.sh 1191 +787_emit_plans_every_member_on_its_own.sh 165 +788_emit_defers_an_unbuilt_host_tool.sh 1183 +789_emit_describes_a_member_past_a_failed_build_program.sh 327 +78_test_main_combinations.sh 11880 +790_a_prepare_action_populates_an_unknown_directory.sh 1823 +791_a_prepare_action_that_creates_nothing_fails.sh 955 +792_a_rerun_input_inside_a_prepare_directory_is_warned.sh 1743 +793_an_unknown_action_role_is_refused.sh 2595 +795_a_link_flag_reaches_the_linker_as_written.sh 960 +798_a_rule_applies_through_a_source_it_claims.sh 13286 +799_an_action_runs_with_its_env_and_cwd.sh 1024 +79_gtest_regular_dep_feature_main.sh 14959 +800_a_feature_provides_its_host_tools.sh 1031 +801_a_dependency_program_is_shipped_with_the_consumer.sh 1851 +802_a_host_build_applies_its_host_row.sh 1695 +803_a_recorded_payload_that_is_gone_is_refused_offline.sh 111 +804_a_path_host_tool_builds_with_its_chosen_toolchain.sh 1148 +805_rooted_workspace_reaches_its_own_path_dependency.sh 481 +806_dash_p_resolves_the_package_first.sh 755 +807_a_lib_root_imports_its_own_package.sh 508 +808_device_source_is_not_a_compile_unit.sh 15188 +809_failed_build_program_hides_no_device_source.sh 89 +80_feature_defines.sh 259 +810_two_packages_deploy_the_same_bytes_to_one_name.sh 1941 +812_an_index_floor_is_a_closing_tip.sh 252 +813_conditional_dialect_cxxflags_root_only.sh 23819 +815_the_database_describes_what_a_rule_generates.sh 970 +816_a_git_dependency_reports_its_fetch.sh 221 +817_emit_writes_no_xlings_json_under_private_work_dir.sh 283 +818_a_declared_deploy_outranks_a_search_dir_dll_cross.sh 949 +819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 377 +81_capability_binding.sh 573 +821_an_edge_states_its_advice_on_both_paths.sh 1614 +822_a_package_states_the_oldest_mcpp_it_supports.sh 288 +823_mcpp_core_and_mcpp_name_one_interface.sh 765 +824_a_build_program_reads_the_resolved_toolchain.sh 1076 +826_plugin_diagnostics_features_and_names.sh 726 +827_many_placements_run_as_one_process.sh 2327 +828_a_library_states_its_interface.sh 638 +829_pack_takes_a_member.sh 278 +82_feature_optional_deps.sh 264 +831_a_path_dependency_host_module_outside_src.sh 4402 +832_the_fast_path_resumes_after_an_edit.sh 2677 +833_a_workspace_is_one_graph_per_configuration.sh 1703 +834_a_workspace_plans_once_and_replays_in_one_check.sh 1180 +835_a_shared_library_is_placed_by_a_link.sh 319 +836_a_members_own_statements_reach_its_build.sh 1367 +837_a_members_resources_reach_its_image.sh 170 +838_a_program_shipped_through_artifacts_links_its_own_closure.sh 366 +839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 743 +83_feature_defines_propagate.sh 141 +840_a_workspace_database_describes_each_package_once.sh 602 +841_pack_summarizes_many_outputs.sh 3206 +842_the_build_reports_each_step_once.sh 7211 +843_a_terminal_names_the_action_that_runs.sh 4684 +844_a_planned_build_keeps_one_lock_entry_per_identity.sh 794 +845_play_game_restores_the_terminal.sh 12308 +846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 758 +847_a_module_name_is_unique_within_a_program.sh 1960 +849_a_staged_bmi_waits_for_a_module_compiled_here.sh 1607 +84_runtime_dll_deploy.sh 161 +850_a_selected_member_declares_as_the_root.sh 760 +851_a_shared_member_is_compiled_once_per_configuration.sh 536 +852_a_repeated_dash_p_selects_every_member_it_names.sh 273 +853_run_keeps_one_member_and_exclude_removes_members.sh 764 +854_a_test_over_several_members_plans_and_builds_once.sh 1333 +855_a_shared_test_build_is_reported_once_per_group.sh 346 +856_a_test_over_several_members_continues_past_a_failing_one.sh 1528 +857_a_host_module_several_members_import_is_compiled_once.sh 807 +858_the_engines_module_is_compiled_once_per_home.sh 4172 +859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 3220 +85_target_cfg_build_flags.sh 129 +860_a_workspaces_programs_compile_at_the_same_time.sh 16698 +861_concurrent_compiles_plan_what_serial_compiles_plan.sh 1228 +862_run_q_writes_exactly_the_programs_stdout.sh 199 +863_a_run_whose_build_failed_exits_101.sh 662 +864_a_builds_status_is_on_standard_error.sh 753 +865_mcpp_test_reports_its_steps_on_stderr_and_its_results_on_stdout.sh 575 +866_both_streams_on_one_pipe_keep_their_order_and_json_stays_clean.sh 3940 +867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh 10859 +868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh 5486 +869_a_pack_over_several_members_names_each_member_in_its_report.sh 1582 +86_llvm_hermetic_link.sh 3796 +86_target_cfg_dependencies.sh 156 +870_a_pack_over_several_members_resolves_and_fails_per_member.sh 6416 +871_a_pack_states_its_build_and_every_drive_takes_the_job_count.sh 11625 +872_a_member_compiles_once_whatever_the_selection.sh 1966 +873_payload_overrides.sh 1826 +874_payload_on_request.sh 2050 +875_toolchain_by_path.sh 665 +876_toolchain_phase.sh 1138 +877_why_sources.sh 83 +87_build_default_profile.sh 502 +88_xlings_environment.sh 9400 +89_build_mcpp.sh 3422 +90_workspace_test.sh 636 +91_target_bare_alias.sh 1021 92_build_mcpp_import.sh 269 -93_xpkg_parse.sh 67 -96_msvc_unavailable_nonwindows.sh 37 -98_reflection_import_std.sh 6570 +93_xpkg_parse.sh 84 +96_msvc_unavailable_nonwindows.sh 38 +98_reflection_import_std.sh 6868 diff --git a/tests/e2e/timings/macos.tsv b/tests/e2e/timings/macos.tsv index 3f9b56bc..c227203a 100644 --- a/tests/e2e/timings/macos.tsv +++ b/tests/e2e/timings/macos.tsv @@ -1,242 +1,242 @@ -00_fixture_path_hygiene.sh 7000 +00_fixture_path_hygiene.sh 4000 01_help_and_version.sh 0 -02_new_build_run.sh 1000 -04_incremental.sh 4000 -05_errors.sh 1000 +02_new_build_run.sh 5000 +04_incremental.sh 6000 +05_errors.sh 2000 06_emit_xpkg.sh 0 -100_feature_sources_test_mode.sh 6000 +100_feature_sources_test_mode.sh 5000 101_cppfly_llvm_soft.sh 3000 -10_env_command.sh 8000 -117_object_path_collision.sh 1000 +10_env_command.sh 6000 +117_object_path_collision.sh 2000 118_purview_include_rebuild.sh 9000 -11_index_list.sh 8000 -122_run_member.sh 2000 +11_index_list.sh 7000 +122_run_member.sh 3000 126_default_features_opt_out.sh 2000 -128_feature_forwarding.sh 2000 +128_feature_forwarding.sh 1000 12_add_command.sh 1000 138_feature_sources_gate_vs_provider.sh 3000 139_build_program_advisory.sh 3000 13_toolchain_pin.sh 0 148_wide_include_list.sh 2000 149_conditional_glob_flags.sh 2000 -14_toolchain_fallback.sh 7000 -150_clang_module_operator_template.sh 1000 -151_index_url_artifact_migration.sh 8000 -152_test_subdir_names.sh 0 -153_test_isolation.sh 2000 -154_test_filter.sh 2000 +14_toolchain_fallback.sh 5000 +150_clang_module_operator_template.sh 0 +151_index_url_artifact_migration.sh 6000 +152_test_subdir_names.sh 1000 +153_test_isolation.sh 1000 +154_test_filter.sh 1000 155_test_json.sh 1000 156_nested_ld_library_path.sh 0 157_test_glob_flags.sh 1000 -158_test_signal_and_duration.sh 1000 +158_test_signal_and_duration.sh 2000 159_test_list.sh 0 15_test_passing.sh 2000 -160_test_timeout.sh 6000 +160_test_timeout.sh 5000 161_xpkg_name_form.sh 0 -162_bare_name_namespace_scope.sh 9000 -163_identity_first_resolution.sh 14000 +162_bare_name_namespace_scope.sh 7000 +163_identity_first_resolution.sh 10000 164_ninja_goal_command_length.sh 2000 -165_bare_name_cross_namespace_wire_address.sh 5000 +165_bare_name_cross_namespace_wire_address.sh 6000 166_run_env_no_private_glibc.sh 1000 -167_build_defines_module_scan.sh 4000 -16_test_failing.sh 3000 +167_build_defines_module_scan.sh 3000 +16_test_failing.sh 2000 170_bmi_staging_no_cascade.sh 2000 171_bmi_staging_locked_dest.sh 4000 -178_test_observability.sh 14000 +178_test_observability.sh 13000 179_spaced_paths.sh 2000 17_test_no_tests.sh 0 -181_build_mcpp_import_std.sh 7000 -183_cxx_runtime_contract.sh 8000 -18_devdeps_isolation.sh 1000 +181_build_mcpp_import_std.sh 6000 +183_cxx_runtime_contract.sh 9000 +18_devdeps_isolation.sh 2000 190_link_rspfile_newlines.sh 2000 -191_link_scale.sh 32000 +191_link_scale.sh 38000 195_target_cfg_feature_deps.sh 2000 199_resources_validation.sh 1000 -19_bmi_cache_reuse.sh 7000 -202_machine_output_contract.sh 2000 +19_bmi_cache_reuse.sh 9000 +202_machine_output_contract.sh 3000 204_new_transactional_scaffold.sh 7000 210_local_index_addressing_on_every_host.sh 0 -211_configure_only_cdb.sh 5000 -221_subos_without_info_still_builds.sh 1000 -230_bench_harness.sh 40000 -231_jobs_option.sh 8000 +211_configure_only_cdb.sh 4000 +221_subos_without_info_still_builds.sh 2000 +230_bench_harness.sh 36000 +231_jobs_option.sh 7000 232_workflow_syntax.sh 0 -233_bench_matrix.sh 0 +233_bench_matrix.sh 1000 234_bmi_schedule_on.sh 2000 -236_module_extensions_default_toolchain.sh 1000 -23_remove_update.sh 12000 +236_module_extensions_default_toolchain.sh 2000 +23_remove_update.sh 14000 242_pack_library_interface_and_headers.sh 4000 243_pack_library_interface_closure.sh 4000 244_pack_library_gate.sh 3000 246_explicit_empty_sources.sh 1000 -247_bare_triple_conditional_native.sh 3000 +247_bare_triple_conditional_native.sh 2000 249_pack_workspace_root_unchanged.sh 1000 -24_git_dependency.sh 10000 -250_pack_names_the_target.sh 2000 -251_pack_library_shared.sh 0 +24_git_dependency.sh 9000 +250_pack_names_the_target.sh 1000 +251_pack_library_shared.sh 1000 252_pack_library_old_client.sh 2000 253_pack_library_undetermined_partition.sh 2000 254_pack_library_platform_coverage.sh 2000 259_shared_library_macho.sh 3000 -25_convention_mode.sh 6000 +25_convention_mode.sh 5000 260_module_extension_is_configuration.sh 1000 -261_pack_follows_the_module_extension.sh 3000 -263_lib_root_follows_the_extension.sh 3000 -266_pack_reads_a_macho_program_without_running_it.sh 1000 -267_target_requested_is_empty_for_a_native_build.sh 1000 +261_pack_follows_the_module_extension.sh 2000 +263_lib_root_follows_the_extension.sh 2000 +266_pack_reads_a_macho_program_without_running_it.sh 2000 +267_target_requested_is_empty_for_a_native_build.sh 2000 268_target_side_resolution.sh 4000 -269_openkal_llvm_spelling_still_resolves.sh 2000 -27_namespace_dependencies.sh 7000 -280_target_side_layers.sh 3000 +269_openkal_llvm_spelling_still_resolves.sh 3000 +27_namespace_dependencies.sh 8000 +280_target_side_layers.sh 4000 299_a_request_that_named_no_c_library_resolves_to_a_row_that_exists.sh 0 300_a_registered_family_is_not_reported_unknown.sh 0 -301_the_graphs_compiler_is_taken_and_nothing_is_written.sh 1000 +301_the_graphs_compiler_is_taken_and_nothing_is_written.sh 0 302_a_stated_compiler_outranks_the_graph_and_two_requirements_do_not_stack.sh 0 303_the_query_gives_one_answer_for_the_c_library.sh 2000 304_the_target_side_reaches_every_unit.sh 0 305_built_from_is_not_published.sh 1000 313_check_stamp_on_every_platform.sh 2000 -317_project_build_hooks.sh 31000 +317_project_build_hooks.sh 30000 327_declared_xlings_deps_report_their_own_failure.sh 7000 -328_layer_predicates_apply_and_unknown_ones_say_so.sh 5000 +328_layer_predicates_apply_and_unknown_ones_say_so.sh 6000 329_exit_codes_match_the_contract.sh 0 -32_semver_merge.sh 7000 +32_semver_merge.sh 6000 330_runner_hosted_targets.sh 5000 331_xlings_workspace_one_table.sh 1000 35_workspace.sh 2000 36_llvm_toolchain.sh 1000 -37_llvm_import_std.sh 4000 +37_llvm_import_std.sh 3000 38_llvm_modules.sh 4000 38_self_config_mirror.sh 0 39_llvm_incremental.sh 6000 -39_xlings_index_migration.sh 0 -40_llvm_bmi_cache.sh 7000 -41_llvm_std_compat.sh 2000 -42_custom_local_index.sh 6000 +39_xlings_index_migration.sh 1000 +40_llvm_bmi_cache.sh 6000 +41_llvm_std_compat.sh 4000 +42_custom_local_index.sh 8000 43_indices_lockfile.sh 6000 -44_indices_e2e_integration.sh 7000 +44_indices_e2e_integration.sh 8000 45_install_platform_mapping.sh 0 -46_self_config_mirror_no_bootstrap.sh 5000 -47_cdb_prebuilt_module_path_abs.sh 2000 +46_self_config_mirror_no_bootstrap.sh 6000 +47_cdb_prebuilt_module_path_abs.sh 1000 47_llvm_atomic_link.sh 0 -48_build_error_output.sh 1000 -601_exclusive_capability.sh 1000 +48_build_error_output.sh 2000 +601_exclusive_capability.sh 0 603_version_floor.sh 2000 -604_offline_skips_sandbox_bootstrap.sh 1000 +604_offline_skips_sandbox_bootstrap.sh 0 609_clean_stale.sh 4000 616_examples_and_their_index_agree.sh 1000 -628_a_pin_below_a_stated_floor_is_refused.sh 26000 -645_the_fast_path_compares_the_toolchain_request.sh 5000 -646_runtime_deploy_places_files_in_a_directory.sh 3000 -659_an_action_declaration_has_no_size_limit.sh 2000 -660_an_action_names_the_engine_through_mcpp_self.sh 2000 -666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh 3000 +628_a_pin_below_a_stated_floor_is_refused.sh 32000 +645_the_fast_path_compares_the_toolchain_request.sh 4000 +646_runtime_deploy_places_files_in_a_directory.sh 4000 +659_an_action_declaration_has_no_size_limit.sh 4000 +660_an_action_names_the_engine_through_mcpp_self.sh 3000 +666_a_macho_program_reaches_a_dispatched_format_with_its_tree.sh 4000 668_a_macho_program_is_staged_with_its_closure.sh 2000 -66_runtime_provides.sh 2000 +66_runtime_provides.sh 1000 671_toolchain_is_an_option_of_run_test_and_pack.sh 3000 -672_a_runner_receives_the_files_the_artifact_carries.sh 11000 -673_test_discover_names_where_tests_are.sh 1000 -674_a_format_is_run_by_the_runner_named_after_it.sh 13000 -679_a_path_dependency_takes_its_manifests_identity.sh 2000 -67_features_strict.sh 6000 -681_index_repos_reach_an_existing_home.sh 7000 +672_a_runner_receives_the_files_the_artifact_carries.sh 13000 +673_test_discover_names_where_tests_are.sh 2000 +674_a_format_is_run_by_the_runner_named_after_it.sh 15000 +679_a_path_dependency_takes_its_manifests_identity.sh 3000 +67_features_strict.sh 8000 +681_index_repos_reach_an_existing_home.sh 8000 682_why_deps_prints_the_resolved_graph.sh 1000 683_a_build_program_reads_the_payload_pkg_config_view.sh 1000 684_a_runner_supplied_twice_is_refused.sh 3000 -686_an_activated_xlings_subos_does_not_redirect_the_registry.sh 5000 -687_the_vendored_xlings_probe_is_an_argument_vector.sh 0 -688_emit_build_database.sh 6000 -68_profile_passthrough.sh 4000 -698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 6000 -69_package_templates.sh 8000 +686_an_activated_xlings_subos_does_not_redirect_the_registry.sh 4000 +687_the_vendored_xlings_probe_is_an_argument_vector.sh 1000 +688_emit_build_database.sh 7000 +68_profile_passthrough.sh 5000 +698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 8000 +69_package_templates.sh 11000 702_a_static_package_under_a_shared_library_is_linked_into_it.sh 2000 -704_cxx_identity_across_macho_images.sh 3000 +704_cxx_identity_across_macho_images.sh 4000 720_the_root_build_program_reads_the_resolved_graph.sh 4000 -721_an_android_row_links_on_a_macos_host.sh 92000 +721_an_android_row_links_on_a_macos_host.sh 107000 732_xlings_refresh_is_bounded_and_owned.sh 5000 733_envelope_reports_observed_network_access.sh 1000 734_auto_refresh_false_governs_first_index_sync.sh 0 735_offline_missing_package_has_its_own_code.sh 0 -736_compile_flag_words_reach_the_compiler_and_the_databases.sh 2000 -739_a_private_feature_dep_does_not_reach_the_consumer.sh 2000 +736_compile_flag_words_reach_the_compiler_and_the_databases.sh 3000 +739_a_private_feature_dep_does_not_reach_the_consumer.sh 3000 746_macos_deployment_target_is_target_keyed_not_host_keyed.sh 1000 -75_index_status_offline.sh 7000 +75_index_status_offline.sh 6000 765_a_consumer_include_directory_stays_in_the_consumer.sh 2000 -76_compile_commands_generated.sh 2000 -770_workspace_member_as_dependency.sh 4000 -774_an_index_member_inherits_its_archive_workspace.sh 0 +76_compile_commands_generated.sh 1000 +770_workspace_member_as_dependency.sh 5000 +774_an_index_member_inherits_its_archive_workspace.sh 1000 775_a_host_tool_merges_its_conditional_sections_once.sh 4000 -77_cdb_preserves_test_entries.sh 3000 +77_cdb_preserves_test_entries.sh 5000 780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 3000 781_deleted_root_cdb_is_restored.sh 3000 -782_foreign_root_cdb_entries_are_replaced.sh 3000 +782_foreign_root_cdb_entries_are_replaced.sh 4000 787_emit_plans_every_member_on_its_own.sh 1000 -78_test_main_combinations.sh 6000 +78_test_main_combinations.sh 9000 799_an_action_runs_with_its_env_and_cwd.sh 2000 -79_gtest_regular_dep_feature_main.sh 5000 +79_gtest_regular_dep_feature_main.sh 9000 809_failed_build_program_hides_no_device_source.sh 0 -80_feature_defines.sh 1000 +80_feature_defines.sh 2000 812_an_index_floor_is_a_closing_tip.sh 1000 -815_the_database_describes_what_a_rule_generates.sh 1000 +815_the_database_describes_what_a_rule_generates.sh 2000 816_a_git_dependency_reports_its_fetch.sh 1000 -817_emit_writes_no_xlings_json_under_private_work_dir.sh 1000 +817_emit_writes_no_xlings_json_under_private_work_dir.sh 0 819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 0 81_capability_binding.sh 3000 821_an_edge_states_its_advice_on_both_paths.sh 3000 -822_a_package_states_the_oldest_mcpp_it_supports.sh 1000 +822_a_package_states_the_oldest_mcpp_it_supports.sh 2000 823_mcpp_core_and_mcpp_name_one_interface.sh 4000 -824_a_build_program_reads_the_resolved_toolchain.sh 1000 +824_a_build_program_reads_the_resolved_toolchain.sh 2000 826_plugin_diagnostics_features_and_names.sh 3000 827_many_placements_run_as_one_process.sh 4000 -828_a_library_states_its_interface.sh 3000 +828_a_library_states_its_interface.sh 4000 829_pack_takes_a_member.sh 2000 82_feature_optional_deps.sh 1000 -831_a_path_dependency_host_module_outside_src.sh 6000 -832_the_fast_path_resumes_after_an_edit.sh 4000 +831_a_path_dependency_host_module_outside_src.sh 5000 +832_the_fast_path_resumes_after_an_edit.sh 5000 833_a_workspace_is_one_graph_per_configuration.sh 5000 -834_a_workspace_plans_once_and_replays_in_one_check.sh 6000 -836_a_members_own_statements_reach_its_build.sh 2000 -839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 4000 +834_a_workspace_plans_once_and_replays_in_one_check.sh 10000 +836_a_members_own_statements_reach_its_build.sh 4000 +839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 5000 83_feature_defines_propagate.sh 1000 -840_a_workspace_database_describes_each_package_once.sh 4000 -841_pack_summarizes_many_outputs.sh 4000 -842_the_build_reports_each_step_once.sh 11000 +840_a_workspace_database_describes_each_package_once.sh 8000 +841_pack_summarizes_many_outputs.sh 5000 +842_the_build_reports_each_step_once.sh 13000 843_a_terminal_names_the_action_that_runs.sh 5000 845_play_game_restores_the_terminal.sh 13000 -846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 1000 +846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 2000 847_a_module_name_is_unique_within_a_program.sh 5000 849_a_staged_bmi_waits_for_a_module_compiled_here.sh 3000 -84_runtime_dll_deploy.sh 0 +84_runtime_dll_deploy.sh 1000 851_a_shared_member_is_compiled_once_per_configuration.sh 3000 -852_a_repeated_dash_p_selects_every_member_it_names.sh 2000 -853_run_keeps_one_member_and_exclude_removes_members.sh 5000 -854_a_test_over_several_members_plans_and_builds_once.sh 3000 -855_a_shared_test_build_is_reported_once_per_group.sh 3000 +852_a_repeated_dash_p_selects_every_member_it_names.sh 3000 +853_run_keeps_one_member_and_exclude_removes_members.sh 6000 +854_a_test_over_several_members_plans_and_builds_once.sh 4000 +855_a_shared_test_build_is_reported_once_per_group.sh 2000 856_a_test_over_several_members_continues_past_a_failing_one.sh 9000 -857_a_host_module_several_members_import_is_compiled_once.sh 2000 -858_the_engines_module_is_compiled_once_per_home.sh 7000 -859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 6000 -85_target_cfg_build_flags.sh 0 -860_a_workspaces_programs_compile_at_the_same_time.sh 8000 -861_concurrent_compiles_plan_what_serial_compiles_plan.sh 3000 +857_a_host_module_several_members_import_is_compiled_once.sh 3000 +858_the_engines_module_is_compiled_once_per_home.sh 8000 +859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 7000 +85_target_cfg_build_flags.sh 1000 +860_a_workspaces_programs_compile_at_the_same_time.sh 10000 +861_concurrent_compiles_plan_what_serial_compiles_plan.sh 4000 862_run_q_writes_exactly_the_programs_stdout.sh 1000 -863_a_run_whose_build_failed_exits_101.sh 3000 +863_a_run_whose_build_failed_exits_101.sh 4000 864_a_builds_status_is_on_standard_error.sh 4000 -865_mcpp_test_reports_its_steps_on_stderr_and_its_results_on_stdout.sh 3000 +865_mcpp_test_reports_its_steps_on_stderr_and_its_results_on_stdout.sh 4000 866_both_streams_on_one_pipe_keep_their_order_and_json_stays_clean.sh 6000 -867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh 15000 -868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh 12000 -869_a_pack_over_several_members_names_each_member_in_its_report.sh 9000 +867_a_pack_over_several_members_plans_builds_and_runs_programs_once.sh 16000 +868_a_pack_over_several_members_is_refused_before_anything_is_compiled.sh 14000 +869_a_pack_over_several_members_names_each_member_in_its_report.sh 10000 86_llvm_hermetic_link.sh 0 -86_target_cfg_dependencies.sh 1000 -870_a_pack_over_several_members_resolves_and_fails_per_member.sh 13000 -872_a_member_compiles_once_whatever_the_selection.sh 10000 +86_target_cfg_dependencies.sh 0 +870_a_pack_over_several_members_resolves_and_fails_per_member.sh 16000 +872_a_member_compiles_once_whatever_the_selection.sh 12000 87_build_default_profile.sh 3000 88_xlings_environment.sh 2000 89_build_mcpp.sh 4000 -90_workspace_test.sh 4000 +90_workspace_test.sh 5000 91_target_bare_alias.sh 0 -92_build_mcpp_import.sh 2000 -93_xpkg_parse.sh 0 -94_macos_test_stdlib.sh 1000 +92_build_mcpp_import.sh 1000 +93_xpkg_parse.sh 1000 +94_macos_test_stdlib.sh 3000 96_msvc_unavailable_nonwindows.sh 0 diff --git a/tests/e2e/timings/windows.tsv b/tests/e2e/timings/windows.tsv index 8dadbccc..8eb6380c 100644 --- a/tests/e2e/timings/windows.tsv +++ b/tests/e2e/timings/windows.tsv @@ -1,182 +1,182 @@ -00_fixture_path_hygiene.sh 45090 -01_help_and_version.sh 427 -04_incremental.sh 15660 -05_errors.sh 7880 -06_emit_xpkg.sh 331 -100_feature_sources_test_mode.sh 5580 -101_cppfly_llvm_soft.sh 41760 -10_env_command.sh 7450 -117_object_path_collision.sh 1850 -118_purview_include_rebuild.sh 10630 -11_index_list.sh 8520 -122_run_member.sh 2060 -126_default_features_opt_out.sh 3900 -128_feature_forwarding.sh 4780 -12_add_command.sh 2890 -13_toolchain_pin.sh 121 -148_wide_include_list.sh 4370 -149_conditional_glob_flags.sh 2260 -14_toolchain_fallback.sh 7040 -150_clang_module_operator_template.sh 545 -151_index_url_artifact_migration.sh 7940 -152_test_subdir_names.sh 963 -153_test_isolation.sh 1680 -154_test_filter.sh 1360 -155_test_json.sh 1290 -156_nested_ld_library_path.sh 106 -157_test_glob_flags.sh 921 -159_test_list.sh 295 -15_test_passing.sh 2250 -161_xpkg_name_form.sh 1310 -162_bare_name_namespace_scope.sh 10220 -163_identity_first_resolution.sh 52510 -164_ninja_goal_command_length.sh 1780 -165_bare_name_cross_namespace_wire_address.sh 11580 -166_run_env_no_private_glibc.sh 1140 -167_build_defines_module_scan.sh 4070 -16_test_failing.sh 1770 -170_bmi_staging_no_cascade.sh 2210 -171_bmi_staging_locked_dest.sh 4250 -177_cpp20_msvc.sh 9700 -179_spaced_paths.sh 1900 -17_test_no_tests.sh 261 -180_msvc_build_mcpp.sh 17670 -183_cxx_runtime_contract.sh 9180 -18_devdeps_isolation.sh 1900 -190_link_rspfile_newlines.sh 3670 -191_link_scale.sh 37040 -195_target_cfg_feature_deps.sh 2640 -197_windows_resources.sh 8470 -199_resources_validation.sh 1690 -19_bmi_cache_reuse.sh 49270 -202_machine_output_contract.sh 2840 -204_new_transactional_scaffold.sh 9170 -210_local_index_addressing_on_every_host.sh 1360 -211_configure_only_cdb.sh 5100 -221_subos_without_info_still_builds.sh 1610 -230_bench_harness.sh 69640 -231_jobs_option.sh 8980 -232_workflow_syntax.sh 281 -233_bench_matrix.sh 926 -234_bmi_schedule_on.sh 2620 -236_module_extensions_default_toolchain.sh 2490 -23_remove_update.sh 21160 -241_windows_ucrt_runtime_identity.sh 14310 -242_pack_library_interface_and_headers.sh 4840 -243_pack_library_interface_closure.sh 4390 -244_pack_library_gate.sh 4710 -246_explicit_empty_sources.sh 2400 -247_bare_triple_conditional_native.sh 2700 -249_pack_workspace_root_unchanged.sh 1160 -250_pack_names_the_target.sh 1790 -251_pack_library_shared.sh 1270 -252_pack_library_old_client.sh 2180 -253_pack_library_undetermined_partition.sh 3460 -254_pack_library_platform_coverage.sh 3410 -255_pack_library_msvc_archiver.sh 1130 -256_pack_library_fat_windows.sh 5150 -258_shared_library_msvc_auto_def.sh 8210 -25_convention_mode.sh 5190 -260_module_extension_is_configuration.sh 1800 -261_pack_follows_the_module_extension.sh 2980 -262_pack_consumed_by_native_cl.sh 2630 -263_lib_root_follows_the_extension.sh 8150 -280_target_side_layers.sh 4220 -313_check_stamp_on_every_platform.sh 1800 -317_project_build_hooks.sh 23570 -327_declared_xlings_deps_report_their_own_failure.sh 10470 -328_layer_predicates_apply_and_unknown_ones_say_so.sh 7080 -329_exit_codes_match_the_contract.sh 324 -35_workspace.sh 2010 -36_llvm_toolchain.sh 42490 -38_self_config_mirror.sh 42110 -39_xlings_index_migration.sh 38520 -42_custom_local_index.sh 9200 -43_indices_lockfile.sh 9130 -44_indices_e2e_integration.sh 8780 -45_install_platform_mapping.sh 216 -46_self_config_mirror_no_bootstrap.sh 6700 -47_cdb_prebuilt_module_path_abs.sh 1840 -48_build_error_output.sh 40840 -601_exclusive_capability.sh 1960 -603_version_floor.sh 2840 -604_offline_skips_sandbox_bootstrap.sh 2130 -609_clean_stale.sh 5200 -628_a_pin_below_a_stated_floor_is_refused.sh 31990 -642_windows_subsystem.sh 5060 -645_the_fast_path_compares_the_toolchain_request.sh 42390 -646_runtime_deploy_places_files_in_a_directory.sh 5930 -659_an_action_declaration_has_no_size_limit.sh 10130 -660_an_action_names_the_engine_through_mcpp_self.sh 2850 -66_runtime_provides.sh 2550 -671_toolchain_is_an_option_of_run_test_and_pack.sh 3570 -673_test_discover_names_where_tests_are.sh 2360 -679_a_path_dependency_takes_its_manifests_identity.sh 3840 -67_features_strict.sh 10760 -681_index_repos_reach_an_existing_home.sh 11320 -682_why_deps_prints_the_resolved_graph.sh 1590 -683_a_build_program_reads_the_payload_pkg_config_view.sh 1900 -687_the_vendored_xlings_probe_is_an_argument_vector.sh 42730 -688_emit_build_database.sh 51110 -68_profile_passthrough.sh 6310 -698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 19360 -69_package_templates.sh 14860 -703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh 7020 -705_cxx_identity_across_pe_images.sh 5150 -720_the_root_build_program_reads_the_resolved_graph.sh 5720 -735_offline_missing_package_has_its_own_code.sh 609 -736_compile_flag_words_reach_the_compiler_and_the_databases.sh 3400 -739_a_private_feature_dep_does_not_reach_the_consumer.sh 2230 -75_index_status_offline.sh 7180 -760_msvc_toolset_is_chosen_once.sh 10320 -765_a_consumer_include_directory_stays_in_the_consumer.sh 45510 -76_compile_commands_generated.sh 1800 -770_workspace_member_as_dependency.sh 6610 -774_an_index_member_inherits_its_archive_workspace.sh 41050 -775_a_host_tool_merges_its_conditional_sections_once.sh 50850 -77_cdb_preserves_test_entries.sh 3980 -780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 3250 -781_deleted_root_cdb_is_restored.sh 2680 -782_foreign_root_cdb_entries_are_replaced.sh 4240 -787_emit_plans_every_member_on_its_own.sh 1390 -78_test_main_combinations.sh 11260 -794_a_windows_program_finds_a_dll_through_runtime_search_dir.sh 2830 -799_an_action_runs_with_its_env_and_cwd.sh 3460 -79_gtest_regular_dep_feature_main.sh 14340 -809_failed_build_program_hides_no_device_source.sh 909 -80_feature_defines.sh 1750 -811_place_dlls_never_overwrites_a_declared_deploy.sh 3470 -814_the_llvm_row_defaults_to_the_dynamic_crt.sh 6960 -819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 5370 -81_capability_binding.sh 4570 -820_a_windows_program_takes_its_runtime_from_one_resolver.sh 13960 -825_a_build_program_reads_the_msvc_toolset.sh 3430 -82_feature_optional_deps.sh 1970 -836_a_members_own_statements_reach_its_build.sh 2820 -839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 4870 -83_feature_defines_propagate.sh 1230 -840_a_workspace_database_describes_each_package_once.sh 7300 -841_pack_summarizes_many_outputs.sh 9800 -842_the_build_reports_each_step_once.sh 17680 -846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 38110 -847_a_module_name_is_unique_within_a_program.sh 8510 -848_msvc_binds_a_module_name_per_program.sh 1270 -849_a_staged_bmi_waits_for_a_module_compiled_here.sh 43610 -84_runtime_dll_deploy.sh 1090 -857_a_host_module_several_members_import_is_compiled_once.sh 41690 -858_the_engines_module_is_compiled_once_per_home.sh 153360 -859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 49500 -85_target_cfg_build_flags.sh 1090 -860_a_workspaces_programs_compile_at_the_same_time.sh 48910 -86_target_cfg_dependencies.sh 1170 -872_a_member_compiles_once_whatever_the_selection.sh 57560 -87_build_default_profile.sh 3370 -88_xlings_environment.sh 2220 -89_build_mcpp.sh 7200 -90_workspace_test.sh 3560 -91_target_bare_alias.sh 963 -92_build_mcpp_import.sh 1310 -93_xpkg_parse.sh 798 -95_msvc_system_toolchain.sh 3470 -97_mingw_toolchain.sh 11680 -99_msvc_native_build.sh 5080 +00_fixture_path_hygiene.sh 33892 +01_help_and_version.sh 184 +04_incremental.sh 9490 +05_errors.sh 2616 +06_emit_xpkg.sh 211 +100_feature_sources_test_mode.sh 10409 +101_cppfly_llvm_soft.sh 63285 +10_env_command.sh 7244 +117_object_path_collision.sh 11002 +118_purview_include_rebuild.sh 14229 +11_index_list.sh 5582 +122_run_member.sh 2332 +126_default_features_opt_out.sh 2747 +128_feature_forwarding.sh 2165 +12_add_command.sh 2435 +13_toolchain_pin.sh 112 +148_wide_include_list.sh 4911 +149_conditional_glob_flags.sh 2369 +14_toolchain_fallback.sh 8046 +150_clang_module_operator_template.sh 497 +151_index_url_artifact_migration.sh 10016 +152_test_subdir_names.sh 874 +153_test_isolation.sh 1594 +154_test_filter.sh 1385 +155_test_json.sh 1233 +156_nested_ld_library_path.sh 86 +157_test_glob_flags.sh 887 +159_test_list.sh 282 +15_test_passing.sh 1669 +161_xpkg_name_form.sh 632 +162_bare_name_namespace_scope.sh 10735 +163_identity_first_resolution.sh 38467 +164_ninja_goal_command_length.sh 1749 +165_bare_name_cross_namespace_wire_address.sh 10356 +166_run_env_no_private_glibc.sh 978 +167_build_defines_module_scan.sh 3560 +16_test_failing.sh 1861 +170_bmi_staging_no_cascade.sh 1813 +171_bmi_staging_locked_dest.sh 4057 +177_cpp20_msvc.sh 9276 +179_spaced_paths.sh 3756 +17_test_no_tests.sh 235 +180_msvc_build_mcpp.sh 22389 +183_cxx_runtime_contract.sh 9771 +18_devdeps_isolation.sh 1459 +190_link_rspfile_newlines.sh 3424 +191_link_scale.sh 27144 +195_target_cfg_feature_deps.sh 3223 +197_windows_resources.sh 8404 +199_resources_validation.sh 1740 +19_bmi_cache_reuse.sh 68995 +202_machine_output_contract.sh 2952 +204_new_transactional_scaffold.sh 9113 +210_local_index_addressing_on_every_host.sh 1074 +211_configure_only_cdb.sh 7706 +221_subos_without_info_still_builds.sh 2254 +230_bench_harness.sh 71394 +231_jobs_option.sh 9198 +232_workflow_syntax.sh 304 +233_bench_matrix.sh 653 +234_bmi_schedule_on.sh 4292 +236_module_extensions_default_toolchain.sh 3768 +23_remove_update.sh 20063 +241_windows_ucrt_runtime_identity.sh 18807 +242_pack_library_interface_and_headers.sh 4069 +243_pack_library_interface_closure.sh 8029 +244_pack_library_gate.sh 8759 +246_explicit_empty_sources.sh 1734 +247_bare_triple_conditional_native.sh 2950 +249_pack_workspace_root_unchanged.sh 1248 +250_pack_names_the_target.sh 1736 +251_pack_library_shared.sh 1204 +252_pack_library_old_client.sh 2007 +253_pack_library_undetermined_partition.sh 3015 +254_pack_library_platform_coverage.sh 3324 +255_pack_library_msvc_archiver.sh 2483 +256_pack_library_fat_windows.sh 5554 +258_shared_library_msvc_auto_def.sh 7734 +25_convention_mode.sh 5300 +260_module_extension_is_configuration.sh 1544 +261_pack_follows_the_module_extension.sh 2876 +262_pack_consumed_by_native_cl.sh 2052 +263_lib_root_follows_the_extension.sh 7435 +280_target_side_layers.sh 3141 +313_check_stamp_on_every_platform.sh 5557 +317_project_build_hooks.sh 27463 +327_declared_xlings_deps_report_their_own_failure.sh 10378 +328_layer_predicates_apply_and_unknown_ones_say_so.sh 4863 +329_exit_codes_match_the_contract.sh 305 +35_workspace.sh 2263 +36_llvm_toolchain.sh 114148 +38_self_config_mirror.sh 64649 +39_xlings_index_migration.sh 66344 +42_custom_local_index.sh 8209 +43_indices_lockfile.sh 11686 +44_indices_e2e_integration.sh 12795 +45_install_platform_mapping.sh 206 +46_self_config_mirror_no_bootstrap.sh 7828 +47_cdb_prebuilt_module_path_abs.sh 1915 +48_build_error_output.sh 63119 +601_exclusive_capability.sh 1806 +603_version_floor.sh 2585 +604_offline_skips_sandbox_bootstrap.sh 1823 +609_clean_stale.sh 6691 +628_a_pin_below_a_stated_floor_is_refused.sh 49452 +642_windows_subsystem.sh 5247 +645_the_fast_path_compares_the_toolchain_request.sh 110810 +646_runtime_deploy_places_files_in_a_directory.sh 5219 +659_an_action_declaration_has_no_size_limit.sh 11091 +660_an_action_names_the_engine_through_mcpp_self.sh 2681 +66_runtime_provides.sh 2221 +671_toolchain_is_an_option_of_run_test_and_pack.sh 3428 +673_test_discover_names_where_tests_are.sh 2238 +679_a_path_dependency_takes_its_manifests_identity.sh 3719 +67_features_strict.sh 8619 +681_index_repos_reach_an_existing_home.sh 10633 +682_why_deps_prints_the_resolved_graph.sh 1655 +683_a_build_program_reads_the_payload_pkg_config_view.sh 1509 +687_the_vendored_xlings_probe_is_an_argument_vector.sh 64285 +688_emit_build_database.sh 74035 +68_profile_passthrough.sh 5954 +698_a_tool_source_under_its_dependency_has_a_bounded_address.sh 13584 +69_package_templates.sh 18548 +703_the_llvm_row_on_the_msvc_abi_records_its_static_crt.sh 6359 +705_cxx_identity_across_pe_images.sh 5435 +720_the_root_build_program_reads_the_resolved_graph.sh 5906 +735_offline_missing_package_has_its_own_code.sh 640 +736_compile_flag_words_reach_the_compiler_and_the_databases.sh 2679 +739_a_private_feature_dep_does_not_reach_the_consumer.sh 2267 +75_index_status_offline.sh 6268 +760_msvc_toolset_is_chosen_once.sh 8601 +765_a_consumer_include_directory_stays_in_the_consumer.sh 68276 +76_compile_commands_generated.sh 1575 +770_workspace_member_as_dependency.sh 4873 +774_an_index_member_inherits_its_archive_workspace.sh 65596 +775_a_host_tool_merges_its_conditional_sections_once.sh 75532 +77_cdb_preserves_test_entries.sh 4293 +780_a_passing_check_moves_its_stamp_past_a_changed_input.sh 2885 +781_deleted_root_cdb_is_restored.sh 2778 +782_foreign_root_cdb_entries_are_replaced.sh 4060 +787_emit_plans_every_member_on_its_own.sh 1293 +78_test_main_combinations.sh 10826 +794_a_windows_program_finds_a_dll_through_runtime_search_dir.sh 2965 +799_an_action_runs_with_its_env_and_cwd.sh 2956 +79_gtest_regular_dep_feature_main.sh 11982 +809_failed_build_program_hides_no_device_source.sh 919 +80_feature_defines.sh 1649 +811_place_dlls_never_overwrites_a_declared_deploy.sh 1912 +814_the_llvm_row_defaults_to_the_dynamic_crt.sh 12482 +819_the_crt_is_placed_by_its_rule_not_by_search_order.sh 2294 +81_capability_binding.sh 4393 +820_a_windows_program_takes_its_runtime_from_one_resolver.sh 11452 +825_a_build_program_reads_the_msvc_toolset.sh 3484 +82_feature_optional_deps.sh 5090 +836_a_members_own_statements_reach_its_build.sh 3069 +839_a_workspaces_build_programs_are_named_ordered_and_cached.sh 4317 +83_feature_defines_propagate.sh 1073 +840_a_workspace_database_describes_each_package_once.sh 7573 +841_pack_summarizes_many_outputs.sh 8552 +842_the_build_reports_each_step_once.sh 15936 +846_the_vendored_xlings_is_replaced_from_the_newest_source.sh 73849 +847_a_module_name_is_unique_within_a_program.sh 8333 +848_msvc_binds_a_module_name_per_program.sh 1399 +849_a_staged_bmi_waits_for_a_module_compiled_here.sh 71571 +84_runtime_dll_deploy.sh 961 +857_a_host_module_several_members_import_is_compiled_once.sh 72667 +858_the_engines_module_is_compiled_once_per_home.sh 235616 +859_a_host_module_entry_is_reused_while_its_inputs_agree.sh 74384 +85_target_cfg_build_flags.sh 942 +860_a_workspaces_programs_compile_at_the_same_time.sh 85295 +86_target_cfg_dependencies.sh 1033 +872_a_member_compiles_once_whatever_the_selection.sh 76821 +87_build_default_profile.sh 3359 +88_xlings_environment.sh 2315 +89_build_mcpp.sh 6883 +90_workspace_test.sh 3360 +91_target_bare_alias.sh 915 +92_build_mcpp_import.sh 1371 +93_xpkg_parse.sh 685 +95_msvc_system_toolchain.sh 3086 +97_mingw_toolchain.sh 11239 +99_msvc_native_build.sh 8995 From 45e61078d3f9dc6996bc9d8f65192ea7c447430a Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 08:23:37 +0800 Subject: [PATCH 07/14] A build record names its engine, one predicate admits a recorded build, and each path dependency is classified by its own package (#757, #756) #757. build.ninja names the engine that wrote it by absolute path, and the emitter assumed that a new engine plans the graph again because the version is a fingerprint input. No fast path computes a fingerprint, so after an upgrade that removed the previous install the recorded graph was replayed and every action started a program that no longer existed. - BuildCacheEntry records the engine that wrote the graph: its version and mcpp_exe_path(), the function the emitter writes into the graph (now exported from mcpp.build.ninja so the two cannot be spelled apart). An entry that predates the field declines once ("the recorded build predates the engine identity"). - The gates the three fast paths share are one function, admit_recorded_build: the engine, the runtime binding and environment key, the fingerprint directory, the graph's existence, mode and request tag, the freshness of the runtime, the manifest and the project's sources, the recorded path-dependency roots, the xlings payloads, the toolchain named by path, and the artifact snapshot. try_fast_build, try_fast_workspace_build and try_fast_run call it and keep only their own gates (the selection, a program to run, a runner, the run tier). A field added to the record is checked in one place. #756. The freshness sweep classified every path-dependency tree with the consumer's extension table, so a provider's `.ixx` (declared only by the provider) was a file of no interest and an edit to its host module was replayed as "no work". - The plan records each root with its own package's module_extensions and device_extensions (DepSourceRoot), from the effective manifest. - The record carries them in a count-prefixed `depSources=` block; the block that listed the paths alone is read past and left unrecorded, so such an entry declines once. - dep_sources_newer_than classifies the files below each root with that root's table. Tests: e2e 878 (an engine moved to another path does not replay the old graph; the record written before the field declines once; `mcpp run`), e2e 879 (a provider's `.ixx` reaches the project, run and workspace fast paths with a consumer that declares nothing; an aged record declines once), and unit tests of the engine decision, the admission order, the record's round trip and the per-root sweep. --- docs/04-mcpp-toml.md | 6 +- docs/09-commands-by-scenario.md | 7 + src/build/execute.cppm | 535 +++++++++++------- src/build/ninja_backend.cppm | 24 +- src/build/prepare.cppm | 20 +- src/build/prepare/plan.cpp | 22 +- ...ed_engine_does_not_replay_the_old_graph.sh | 110 ++++ ..._module_extension_reaches_the_fast_path.sh | 186 ++++++ tests/unit/test_build_cache_record.cpp | 277 +++++++++ 9 files changed, 969 insertions(+), 218 deletions(-) create mode 100755 tests/e2e/878_an_upgraded_engine_does_not_replay_the_old_graph.sh create mode 100755 tests/e2e/879_a_providers_module_extension_reaches_the_fast_path.sh create mode 100644 tests/unit/test_build_cache_record.cpp diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index b79af2dc..48be22ce 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -906,7 +906,11 @@ one key rather than several: 2. those units compile with the **module** rule — they emit a BMI and their objects are linked unconditionally; 3. the freshness fast path watches them, so adding an `import` to one - invalidates the build graph instead of silently reusing a stale one. + invalidates the build graph instead of silently reusing a stale one. A + package's files are classified by that package's own table: a `path` + dependency that declares `.ixx` for its sources is watched for `.ixx` + whatever its consumer declares, and the consumer needs no entry of its own + for it. Any extension is accepted **except** ones that already name a non-module role (`.cpp` `.cc` `.cxx` `.c` `.m` `.mm` `.h` `.hpp` `.hh` `.hxx` `.S` `.s` diff --git a/docs/09-commands-by-scenario.md b/docs/09-commands-by-scenario.md index 86c19fce..6b64d59f 100644 --- a/docs/09-commands-by-scenario.md +++ b/docs/09-commands-by-scenario.md @@ -123,6 +123,13 @@ For that invocation the option takes the place of `[toolchain] default` in `MCPP_TOOLCHAIN`. Each toolchain builds into its own output directory, and a recorded build is replayed only for the toolchain request that recorded it. +A recorded build is replayed only by the engine that recorded it. The record +names the engine by version and by the path of its executable, because the +build graph names that executable and starts it. After an upgrade, or once the +executable has moved or been removed, the next `mcpp build` plans the graph +again and writes the record anew; nothing under `target/` needs to be deleted +by hand. + ## Explaining a resolution `mcpp why` reports what a build would resolve, and builds nothing: diff --git a/src/build/execute.cppm b/src/build/execute.cppm index ea1e65ef..639b28d8 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -39,6 +39,7 @@ import mcpp.source_kind; import mcpp.modgraph.scanner; import mcpp.toolchain.post_install; import mcpp.toolchain.stdmod; +import mcpp.version; // MCPP_VERSION — the engine a build record names import mcpp.xlings; import mcpp.xlings.subos_info; import mcpp.runtime.binding; @@ -147,7 +148,55 @@ std::string toolchain_request_identity() { return std::format("cli={};default={}", cli, machineDefault); } -struct BuildCacheEntry { +// THE ENGINE THAT WROTE A BUILD RECORD. +// +// build.ninja names the engine that wrote it by absolute path: the `$mcpp` +// rules (dyndep, stage, the BMI schedule), and the `__action` wrapper of an +// action that declares an environment or a stamp, all start that executable. +// The emitter said a version change regenerates the file, because the version +// is a fingerprint input, and the fast paths never computed a fingerprint: they +// matched an entry by target, profile, cache mode, features and toolchain +// request, and compared its recorded fingerprint with its own directory's name. +// After an upgrade that removed the previous install the entry was replayed, and +// every action started a program that no longer existed (#757). When the old +// executable still exists the failure is silent instead: a newer front end +// drives the actions, and the graph, of an older engine. +// +// So the record carries the engine itself, and the one admission predicate +// below compares it. The version and the executable's path are both kept: the +// same version at another path (a moved or reinstalled binary) leaves the same +// stale text in the graph that a new version leaves, and only the path says so. +export struct EngineIdentity { + std::string version; // MCPP_VERSION + std::string exe; // mcpp_exe_path(), the spelling the graph uses + bool operator==(const EngineIdentity&) const = default; +}; + +// The engine this process is: the one that would write a record now, and the +// one that would replay it. +export EngineIdentity running_engine() { + return {std::string(mcpp::MCPP_VERSION), mcpp_exe_path().string()}; +} + +// Why a record written by `recorded` is not replayed by `running`; nullopt when +// they are the same engine. `recorded` is empty for an entry written before the +// field existed, which is not the same answer as an engine that happens to +// match -- the entry then declines once and the next write records it, the +// discipline of every field of the entry below. +export std::optional +engine_declined_because(const std::optional& recorded, + const EngineIdentity& running) { + if (!recorded) return "the recorded build predates the engine identity"; + if (recorded->version != running.version) + return std::format("the recorded build was written by mcpp {}, and this is mcpp {}", + recorded->version, running.version); + if (recorded->exe != running.exe) + return std::format("the recorded build was written by the engine at {}, and this one is at {}", + recorded->exe, running.exe); + return std::nullopt; +} + +export struct BuildCacheEntry { std::string targetTriple; // "" for default target std::string outputDir; std::string ninjaProgram; @@ -204,7 +253,14 @@ struct BuildCacheEntry { // invisible: ninja has no edge for a file that did not exist when // build.ninja was written, so `mcpp build` replays the stale graph and // reports success. See BuildContext::depSourceRoots. - std::vector depSourceRoots; + // + // Each root carries the extension tables of its own package, which is what + // the sweep classifies the root's files with (#756): the consumer's table + // says nothing about a provider's `.ixx`. Written as `depSources=`; the + // block of an engine that recorded the paths alone (`depSourceRoots=`) is + // read past and counts as absent, since a root without its tables cannot + // be swept. + std::vector depSourceRoots; // Was the block present at all? An EMPTY list is a legitimate answer — a // project with no path dependencies has none — so it cannot stand in for // "this cache predates the field", and the two need opposite treatment: @@ -271,9 +327,41 @@ struct BuildCacheEntry { // fast path matches every entry of its selection. std::string selection; std::string group; + // The engine that wrote the graph this entry records, absent for an entry + // written before the field (see EngineIdentity). Optional rather than an + // empty identity, because an engine whose version is the empty string does + // not exist and "no record" must not be representable as a value. + std::optional engine; }; -std::vector read_build_cache(const std::filesystem::path& projectRoot) { +// One list of extensions as a line of the record carries it: joined by a unit +// separator. An extension is a dot and a name, so it holds neither that +// character nor a tab or a newline, which the line's own structure relies on. +// The two directions are kept together so that the reader and the writer cannot +// disagree about the separator. +constexpr char kExtensionSeparator = '\x1f'; + +std::vector split_extensions(std::string_view joined) { + std::vector out; + while (!joined.empty()) { + const auto at = joined.find(kExtensionSeparator); + out.emplace_back(joined.substr(0, at)); + if (at == std::string_view::npos) break; + joined.remove_prefix(at + 1); + } + return out; +} + +std::string join_extensions(const std::vector& extensions) { + std::string out; + for (auto const& x : extensions) { + if (!out.empty()) out += kExtensionSeparator; + out += x; + } + return out; +} + +export std::vector read_build_cache(const std::filesystem::path& projectRoot) { auto path = projectRoot / kBuildCacheFile; std::ifstream f(path); if (!f) return {}; @@ -368,12 +456,38 @@ std::vector read_build_cache(const std::filesystem::path& proje // zero-length list and an absent block must not read the same. Absent // means the cache predates the field, and the fast path then declines // once so the next write records it. + // + // The block that listed the paths alone is read past and left + // unrecorded: its roots have no tables, so the entry declines once and + // the next write records both. if (haveNextLine && line.starts_with("depSourceRoots=")) { std::size_t n = 0; try { n = std::stoul(line.substr(15)); } catch (...) { n = 0; } - for (std::size_t i = 0; i < n && std::getline(f, line); ++i) - e.depSourceRoots.push_back(line); - e.depSourceRootsRecorded = true; + for (std::size_t i = 0; i < n && std::getline(f, line); ++i) {} + haveNextLine = static_cast(std::getline(f, line)); + } + // `depSources=`, then one line per root: the path, the module + // extensions and the device extensions of the root's package, separated + // by a tab, the extensions of one list by a unit separator. A line that + // is not of that shape makes the whole block unrecorded rather than + // read with a table nobody wrote. + if (haveNextLine && line.starts_with("depSources=")) { + std::size_t n = 0; + try { n = std::stoul(line.substr(11)); } catch (...) { n = 0; } + bool wellFormed = true; + std::vector roots; + for (std::size_t i = 0; i < n && std::getline(f, line); ++i) { + const auto t1 = line.find('\t'); + const auto t2 = t1 == std::string::npos ? t1 : line.find('\t', t1 + 1); + if (t2 == std::string::npos) { wellFormed = false; continue; } + roots.push_back({std::filesystem::path(line.substr(0, t1)), + split_extensions(std::string_view(line).substr(t1 + 1, t2 - t1 - 1)), + split_extensions(std::string_view(line).substr(t2 + 1))}); + } + if (wellFormed && roots.size() == n) { + e.depSourceRoots = std::move(roots); + e.depSourceRootsRecorded = true; + } haveNextLine = static_cast(std::getline(f, line)); } // Optional `runner=0|1` (#544). Absent ⇒ false; see the field. @@ -417,6 +531,15 @@ std::vector read_build_cache(const std::filesystem::path& proje e.group = line.substr(6); haveNextLine = static_cast(std::getline(f, line)); } + // `engine=`, a tab, and the executable's path; see + // EngineIdentity. The version holds no tab, so the path is everything + // after the first one. Absent, or without the tab, the entry declines + // once. + if (haveNextLine && line.starts_with("engine=")) { + if (const auto tab = line.find('\t'); tab != std::string::npos) + e.engine = EngineIdentity{line.substr(7, tab - 7), line.substr(tab + 1)}; + haveNextLine = static_cast(std::getline(f, line)); + } entries.push_back(std::move(e)); if (!haveNextLine || line.empty()) break; } @@ -425,8 +548,8 @@ std::vector read_build_cache(const std::filesystem::path& proje // Serialize the P3 format. Declared ahead of its single caller so the reader // and the writer of this file sit next to each other. -void write_build_cache_entries(const std::filesystem::path& path, - const std::vector& entries); +export void write_build_cache_entries(const std::filesystem::path& path, + const std::vector& entries); // `a, b` and `b a` are one request. Normalised on both sides of the comparison // — the entry stores this form and the fast path computes it — so a cache hit @@ -460,7 +583,7 @@ void write_build_cache(const std::filesystem::path& projectRoot, const std::string& profile = "", const std::string& cacheMode = "", const mcpp::platform::runtime::RuntimeBinding& runtimeBinding = {}, - std::vector depSourceRoots = {}, + std::vector depSourceRoots = {}, bool runnerDeclared = false, bool runTierPending = false, const std::string& features = {}, @@ -503,6 +626,9 @@ void write_build_cache(const std::filesystem::path& projectRoot, newEntry.xlingsPayloadsRecorded = true; newEntry.selection = selection; newEntry.group = group; + // The engine that wrote the graph is the one writing the record: both + // happen in this process, after the graph was emitted. + newEntry.engine = running_engine(); entries.insert(entries.begin(), std::move(newEntry)); // Trim to LRU capacity. @@ -541,8 +667,10 @@ void write_build_cache_entries(const std::filesystem::path& path, << '\n'; f << "profile=" << e.profile << '\n'; f << "cacheMode=" << e.cacheMode << '\n'; - f << "depSourceRoots=" << e.depSourceRoots.size() << '\n'; - for (auto& r : e.depSourceRoots) f << r << '\n'; + f << "depSources=" << e.depSourceRoots.size() << '\n'; + for (auto& r : e.depSourceRoots) + f << r.root.generic_string() << '\t' << join_extensions(r.moduleExtensions) + << '\t' << join_extensions(r.deviceExtensions) << '\n'; f << "runner=" << (e.runnerDeclared ? 1 : 0) << '\n'; f << "runtier=" << (e.runTierPending ? 1 : 0) << '\n'; f << "features=" << e.features << '\n'; @@ -553,6 +681,8 @@ void write_build_cache_entries(const std::filesystem::path& path, f << "selection=" << e.selection << '\n'; f << "group=" << e.group << '\n'; } + if (e.engine) + f << "engine=" << e.engine->version << '\t' << e.engine->exe << '\n'; } } @@ -1012,13 +1142,7 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, std::move(runTargets), runEnvKey, runEnvValue, ctx.profile, std::string(cache_mode_name(ctx.cacheMode)), ctx.plan.runtimeBinding, - [&] { - std::vector v; - v.reserve(ctx.depSourceRoots.size()); - for (auto const& r : ctx.depSourceRoots) - v.push_back(r.generic_string()); - return v; - }(), + ctx.depSourceRoots, // #544: the run fast path declines an entry whose // target has a runner declared — see the field. !choose_runner(ctx).tmpl.empty(), @@ -1169,12 +1293,21 @@ bool sources_newer_than(const std::filesystem::path& projectRoot, // The dependency's own manifest is swept too. A member that gains a target, a // `[modules] sources` glob or a `[build]` flag changes what the graph SHOULD // be, and none of that is visible from its source files' timestamps. -bool dep_sources_newer_than(const std::vector& depSourceRoots, - std::filesystem::file_time_type ninjaTime, - const mcpp::ExtensionTable& extTable) { +// +// EACH ROOT IS CLASSIFIED BY ITS OWN PACKAGE (#756). Which of a tree's files can +// change the graph depends on the extensions the package that owns the tree +// declares: a provider's `.ixx` is a module interface because the provider says +// so, whatever the consumer declares, and a consumer with only `.cpp` sources +// has no reason to. The sweep used the consumer's table for every root, so an +// edit to the provider's host module was classified as a file of no interest and +// replayed as "no work". The tables are the ones prepare recorded with the root. +export bool dep_sources_newer_than(const std::vector& depSourceRoots, + std::filesystem::file_time_type ninjaTime) { std::error_code ec; - for (auto const& rootStr : depSourceRoots) { - std::filesystem::path depRoot(rootStr); + for (auto const& dep : depSourceRoots) { + const std::filesystem::path& depRoot = dep.root; + const auto extTable = mcpp::extension_table_for(dep.moduleExtensions, + dep.deviceExtensions); // A dependency directory that has gone away is a resolution question, // not a staleness one: fall through to prepare_build, which reports it // with the dependency's name instead of a bare missing path. @@ -1583,14 +1716,139 @@ bool local_toolchain_unchanged(const std::filesystem::path& outputDir) { return true; } -// Why the project fast path declined, under `-v` (#734 E5). Each refusal names -// its condition, so a platform on which the fast path never serves shows which +// Why a fast path declined, under `-v` (#734 E5). Each refusal names its +// condition, so a platform on which the fast path never serves shows which // precondition it fails instead of only the slower build. std::optional fast_path_declined(std::string_view path, std::string_view why) { mcpp::log::verbose("fast-path", std::format("{} declined: {}", path, why)); return std::nullopt; } +// WHAT A RECORDED BUILD MUST SATISFY BEFORE ANY FAST PATH REPLAYS IT. +// +// `try_fast_build`, `try_fast_workspace_build` and `try_fast_run` each carried +// a list of declines of their own, 23, 30 and 30 of them, and a property added +// to one had to be added to the others by hand. #757 is the case in which no +// path received the property at all: the engine that wrote a graph was recorded +// nowhere and compared nowhere. The gates the three share are this one +// function, and a path keeps only what is its own: the selection a workspace +// command matches, a program to run, a runner, the run tier, and the project's +// own sources. A field added to the record is then checked in one place. +// +// The order is the cost of the question: the record itself first, then the +// files it names, then the sweeps of the trees it names, and last the snapshot +// of the artifacts, which reads the build directory. +export struct ReplayAsk { + // What the graph must have been planned for (`request_tag`): a project's + // features, or a workspace group's members. + std::string request; + // The manifest whose edit re-plans, and how a decline names it: the + // project's, or the workspace's. + std::filesystem::path manifest; + std::string_view manifestName = "mcpp.toml"; + // How a decline names the trees the recorded roots stand for: a project + // has path dependencies, a workspace has members besides. + std::string_view treesName = "a path dependency's"; + // The project's own tree, for the paths that sweep one: asked with the time + // of build.ninja, true when a source of it is newer. Empty for a path whose + // trees are all recorded roots, which is what a workspace's members are. + std::function projectSourcesNewer; +}; + +// What a path needs of an admitted record to go on: where the graph is, the +// ninja that drives it, and the artifacts' state before ninja runs, which is +// what tells afterwards whether an artifact was relinked. +export struct ReplayableBuild { + std::filesystem::path outputDir; + std::filesystem::path ninjaPath; + std::string ninjaProgram; + mcpp::build::runtime_validation::ArtifactSnapshot validated; +}; + +export std::expected +admit_recorded_build(const BuildCacheEntry& e, const ReplayAsk& ask) { + // The engine that wrote the graph is the engine that replays it (#757). + // First, because every later question is about a graph this engine did not + // write when this one fails. + if (auto why = engine_declined_because(e.engine, running_engine())) return std::unexpected(*why); + + // The runtime the recorded build ran under. An entry written before the + // immutable snapshot cannot say which environment the program needs, and + // running it with another is worse than not using the cache: it works once + // and then silently stops finding its runtime data (mcpp#352). + if (!e.runtimeBinding) return std::unexpected("the recorded build predates the runtime binding"); + if (e.runtimeEnvKey.empty()) + return std::unexpected("the recorded build predates the runtime environment key"); // regenerate build.ninja once + + // P1: verify fingerprint matches the outputDir basename. + const std::filesystem::path outputDir(e.outputDir); + if (!e.fingerprint.empty() && outputDir.filename().string() != e.fingerprint) + return std::unexpected("the recorded build directory is not the one for this fingerprint"); + + std::error_code ec; + const auto ninjaPath = outputDir / "build.ninja"; + if (!std::filesystem::exists(ninjaPath, ec)) return std::unexpected("build.ninja does not exist"); + + // #407. Freshness is measured against the SOURCES, which says nothing + // about what kind of graph this is. `mcpp test` and + // `mcpp build --configure-only` write their plan -- dev-deps, test targets, + // `default` naming the test binaries and NOT the package's target -- into + // this same file, because the fingerprint covers neither input. Replaying + // that for a plain build linked the tests, never linked the target, and + // printed `Finished`; and a broken file under tests/ (never scanned here) + // failed a plain `mcpp build` outright. + if (!mcpp::build::is_plain_build_graph(ninjaPath)) + return std::unexpected("build.ninja was written by another mode (test, pack or a named target)"); + // What the graph was planned for (workspace design 2026-09-29 §3): the + // features no longer name the directory, so the graph says which it has. + if (mcpp::build::read_request(ninjaPath) != ask.request) + return std::unexpected("build.ninja was written for another request (features or workspace members)"); + + const auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); + if (ec) return std::unexpected("the time of build.ninja cannot be read"); + + const auto runtimeTime = std::filesystem::last_write_time( + e.runtimeBinding->subosDir / ".xlings.json", ec); + if (ec || runtimeTime > ninjaTime) + return std::unexpected("the runtime's .xlings.json is newer than build.ninja"); + + const auto manifestTime = std::filesystem::last_write_time(ask.manifest, ec); + if (ec || manifestTime > ninjaTime) + return std::unexpected(std::format("{} is newer than build.ninja", ask.manifestName)); + + // mcpp#225: bounded + vcs/build-dir-excluded walk (see sources_newer_than) + // instead of a hand-rolled recursive_directory_iterator over src/. + if (ask.projectSourcesNewer && ask.projectSourcesNewer(ninjaTime)) + return std::unexpected("a project source, build.mcpp, a build-program input or a resource script is newer than build.ninja"); + + // A cache written before this field existed cannot say whether the build + // had `path` dependencies, and answering "assume none" is the wrong half of + // that guess: it would keep replaying a stale graph for exactly the projects + // the field was added for. Decline once; the write below records the list + // and every later invocation is fast again. It has to be every path: `mcpp + // run` reaches its binary through the same check, and a `run` that skipped + // it would execute an artifact built from a source set that no longer + // exists. + if (!e.depSourceRootsRecorded) + return std::unexpected("the recorded build predates the list of path-dependency roots and their extension tables"); + if (dep_sources_newer_than(e.depSourceRoots, ninjaTime)) + return std::unexpected(std::format("{} manifest or source is newer than build.ninja", ask.treesName)); + if (!xlings_payloads_present(e)) return std::unexpected("a recorded xlings payload is missing"); + if (!local_toolchain_unchanged(e.outputDir)) + return std::unexpected("a program of the toolchain named by path changed"); + + auto validated = mcpp::build::runtime_validation::validated_artifact_snapshot( + outputDir, *e.runtimeBinding); + if (!validated) return std::unexpected("no validated artifact snapshot is recorded for this build"); + + auto ninjaProgram = e.ninjaProgram; + // Legacy caches stored a shell-quoted path; execvp needs the raw path. + if (ninjaProgram.size() >= 2 && ninjaProgram.front() == '\'' + && ninjaProgram.back() == '\'') + ninjaProgram = ninjaProgram.substr(1, ninjaProgram.size() - 2); + return ReplayableBuild{outputDir, ninjaPath, std::move(ninjaProgram), std::move(*validated)}; +} + export std::optional try_fast_build(const std::filesystem::path& projectRoot, bool verbose, bool no_cache, std::string_view currentTarget = "") { @@ -1641,80 +1899,23 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo } } if (!match) return fast_path_declined("build", "no recorded build matches this request"); - if (!match->runtimeBinding) return fast_path_declined("build", "the recorded build predates the runtime binding"); - auto outputDirStr = match->outputDir; - auto ninjaProgram = match->ninjaProgram; - // Legacy caches stored a shell-quoted path; execvp needs the raw path. - if (ninjaProgram.size() >= 2 && ninjaProgram.front() == '\'' - && ninjaProgram.back() == '\'') - ninjaProgram = ninjaProgram.substr(1, ninjaProgram.size() - 2); - auto cachedFingerprint = match->fingerprint; + // The gates every fast path shares (see admit_recorded_build); what is left + // here is the project's own tree. + auto admitted = admit_recorded_build(*match, { + .request = mcpp::build::request_tag({}, want->features), + .manifest = projectRoot / "mcpp.toml", + .projectSourcesNewer = [&](std::filesystem::file_time_type ninjaTime) { + return sources_newer_than(projectRoot, ninjaTime, want->resourceScripts, + want->extTable); + }, + }); + if (!admitted) return fast_path_declined("build", admitted.error()); + const auto& outputDir = admitted->outputDir; + const auto& ninjaPath = admitted->ninjaPath; + const auto& validatedBefore = admitted->validated; auto runtimeEnvKey = match->runtimeEnvKey; auto runtimeEnvValue = match->runtimeEnvValue; - if (runtimeEnvKey.empty()) - return fast_path_declined("build", "the recorded build predates the runtime environment key"); // old cache entry; regenerate build.ninja once - - // P1: verify fingerprint matches the outputDir basename. - if (!cachedFingerprint.empty()) { - auto dirBasename = std::filesystem::path(outputDirStr).filename().string(); - if (dirBasename != cachedFingerprint) { - return fast_path_declined("build", "the recorded build directory is not the one for this fingerprint"); - } - } - - std::error_code ec; - std::filesystem::path outputDir(outputDirStr); - - auto ninjaPath = outputDir / "build.ninja"; - if (!std::filesystem::exists(ninjaPath, ec)) return fast_path_declined("build", "build.ninja does not exist"); - - // #407. Freshness is measured against the SOURCES, which says nothing - // about what kind of graph this is. `mcpp test` and - // `mcpp build --configure-only` write their plan — dev-deps, test targets, - // `default` naming the test binaries and NOT the package's target — into - // this same file, because the fingerprint covers neither input. Replaying - // that for a plain build linked the tests, never linked the target, and - // printed `Finished`; and a broken file under tests/ (never scanned here) - // failed a plain `mcpp build` outright. - if (!mcpp::build::is_plain_build_graph(ninjaPath)) return fast_path_declined("build", "build.ninja was written by another mode (test, pack or a named target)"); - // What the graph was planned for (workspace design 2026-09-29 §3): the - // features no longer name the directory, so the graph says which it has. - if (mcpp::build::read_request(ninjaPath) != mcpp::build::request_tag({}, want->features)) - return fast_path_declined("build", "build.ninja was written for another request (features or workspace members)"); - - auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); - if (ec) return fast_path_declined("build", "the time of build.ninja cannot be read"); - - auto runtimeManifest = match->runtimeBinding->subosDir / ".xlings.json"; - auto runtimeTime = std::filesystem::last_write_time(runtimeManifest, ec); - if (ec || runtimeTime > ninjaTime) return fast_path_declined("build", "the runtime's .xlings.json is newer than build.ninja"); - - // Check mcpp.toml - auto tomlPath = projectRoot / "mcpp.toml"; - auto tomlTime = std::filesystem::last_write_time(tomlPath, ec); - if (ec || tomlTime > ninjaTime) return fast_path_declined("build", "mcpp.toml is newer than build.ninja"); - - // mcpp#225: bounded + vcs/build-dir-excluded walk (see sources_newer_than) - // instead of a hand-rolled recursive_directory_iterator over src/. - if (sources_newer_than(projectRoot, ninjaTime, want->resourceScripts, - want->extTable)) return fast_path_declined("build", "a project source, build.mcpp, a build-program input or a resource script is newer than build.ninja"); - // A cache written before this field existed cannot say whether the build - // had `path` dependencies, and answering "assume none" is the wrong - // half of that guess: it would keep replaying a stale graph for exactly - // the projects the field was added for. Decline once; the write below - // records the list and every later invocation is fast again. - if (!match->depSourceRootsRecorded) return fast_path_declined("build", "the recorded build predates the list of path-dependency roots"); - if (dep_sources_newer_than(match->depSourceRoots, ninjaTime, want->extTable)) - return fast_path_declined("build", "a path dependency's manifest or source is newer than build.ninja"); - if (!xlings_payloads_present(*match)) return fast_path_declined("build", "a recorded xlings payload is missing"); - if (!local_toolchain_unchanged(match->outputDir)) - return fast_path_declined("build", "a program of the toolchain named by path changed"); - - auto validatedBefore = - mcpp::build::runtime_validation::validated_artifact_snapshot( - outputDir, *match->runtimeBinding); - if (!validatedBefore) return fast_path_declined("build", "no validated artifact snapshot is recorded for this build"); // All inputs are older than build.ninja → fast-path: just run ninja. // C1: this configuration is confirmed current, so the root database is @@ -1723,12 +1924,12 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo // see it back. restore_root_compile_commands(projectRoot, outputDir); std::chrono::milliseconds elapsed{}; - auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, verbose, + auto rc = run_ninja_fast(admitted->ninjaProgram, outputDir, ninjaPath, verbose, runtimeEnvKey, runtimeEnvValue, &elapsed); if (!rc) return fast_path_declined("build", "ninja reported a stale graph"); if (*rc != 0) return rc; if (!mcpp::build::runtime_validation::artifact_snapshot_unchanged( - *validatedBefore)) + validatedBefore)) return fast_path_declined("build", "ninja relinked an artifact, whose closure the full path validates"); // relinked: full path reconstructs + validates closure // The descriptor the plan recorded (revision 3, §7.3); empty for a record @@ -1794,43 +1995,18 @@ export std::optional try_fast_workspace_build( && e.toolchainRecorded && e.toolchainRequest == want->toolchainRequest && e.selection == selection && e.group == group) { match = &e; break; } if (!match) return fast_path_declined("workspace", "no recorded build matches this selection"); - if (!match->runtimeBinding || match->runtimeEnvKey.empty()) - return fast_path_declined("workspace", "the recorded build predates the runtime binding"); - const std::filesystem::path outputDir(match->outputDir); - if (!match->fingerprint.empty() && outputDir.filename().string() != match->fingerprint) - return fast_path_declined("workspace", "the recorded build directory is not the one for this fingerprint"); - const auto ninjaPath = outputDir / "build.ninja"; - if (!std::filesystem::exists(ninjaPath, ec)) - return fast_path_declined("workspace", "build.ninja does not exist"); - if (!mcpp::build::is_plain_build_graph(ninjaPath)) - return fast_path_declined("workspace", "build.ninja was written by another mode (test, pack or a named target)"); - if (mcpp::build::read_request(ninjaPath) != mcpp::build::request_tag(group, {})) - return fast_path_declined("workspace", "build.ninja was written for another request (features or workspace members)"); - const auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); - if (ec) return fast_path_declined("workspace", "the time of build.ninja cannot be read"); - const auto runtimeTime = std::filesystem::last_write_time( - match->runtimeBinding->subosDir / ".xlings.json", ec); - if (ec || runtimeTime > ninjaTime) - return fast_path_declined("workspace", "the runtime's .xlings.json is newer than build.ninja"); - if (wsToml > ninjaTime) - return fast_path_declined("workspace", "the workspace's mcpp.toml is newer than build.ninja"); - if (!match->depSourceRootsRecorded - || dep_sources_newer_than(match->depSourceRoots, ninjaTime, want->extTable)) - return fast_path_declined("workspace", "a member's or a path dependency's manifest or source is newer than build.ninja"); - if (!xlings_payloads_present(*match)) - return fast_path_declined("workspace", "a recorded xlings payload is missing"); - if (!local_toolchain_unchanged(match->outputDir)) - return fast_path_declined("workspace", "a program of the toolchain named by path changed"); - auto validated = mcpp::build::runtime_validation::validated_artifact_snapshot( - outputDir, *match->runtimeBinding); - if (!validated) - return fast_path_declined("workspace", "no validated artifact snapshot is recorded for this build"); - auto ninjaProgram = match->ninjaProgram; - if (ninjaProgram.size() >= 2 && ninjaProgram.front() == '\'' - && ninjaProgram.back() == '\'') - ninjaProgram = ninjaProgram.substr(1, ninjaProgram.size() - 2); - ready.push_back({outputDir, ninjaProgram, match->runtimeEnvKey, - match->runtimeEnvValue, std::move(*validated)}); + // The gates every fast path shares (see admit_recorded_build). A + // workspace has no project tree of its own to sweep: its members are + // recorded roots, each classified by its own package. + auto admitted = admit_recorded_build(*match, { + .request = mcpp::build::request_tag(group, {}), + .manifest = wsRoot / "mcpp.toml", + .manifestName = "the workspace's mcpp.toml", + .treesName = "a member's or a path dependency's", + }); + if (!admitted) return fast_path_declined("workspace", admitted.error()); + ready.push_back({admitted->outputDir, admitted->ninjaProgram, match->runtimeEnvKey, + match->runtimeEnvValue, std::move(admitted->validated)}); profile = want->profile; } @@ -1941,33 +2117,6 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, // declared tool absent. prepare_build provisions the difference. if (match->runTierPending) return fast_path_declined("run", "the run tier is not yet decided"); - auto outputDirStr = match->outputDir; - auto ninjaProgram = match->ninjaProgram; - // Legacy caches stored a shell-quoted path; execvp needs the raw path. - if (ninjaProgram.size() >= 2 && ninjaProgram.front() == '\'' - && ninjaProgram.back() == '\'') - ninjaProgram = ninjaProgram.substr(1, ninjaProgram.size() - 2); - if (match->runtimeEnvKey.empty()) - return fast_path_declined("run", "the recorded build predates the runtime environment key"); // old cache entry; go through prepare_build once - // Written before this mcpp knew about subos environments (mcpp#352). Taking - // the fast path here would run the program without them -- which is the - // defect this field exists to fix, surviving an upgrade. - // - // It survives it for a long time, too: the fast path's identity is the - // profile, the cache mode and the resource list, and its fingerprint check - // compares a cached entry against ITSELF. Neither notices that a different - // mcpp wrote the entry, so without this line an upgraded mcpp would reuse a - // pre-upgrade build until something else happened to invalidate it. Measured - // on a real upgrade from 2026.8.7.1, not reasoned about. - if (!match->runtimeBinding) - return fast_path_declined("run", "the recorded build predates the runtime binding"); // predates the immutable snapshot; rebuild once - - // P1: verify fingerprint matches the outputDir basename. - if (!match->fingerprint.empty()) { - auto dirBasename = std::filesystem::path(outputDirStr).filename().string(); - if (dirBasename != match->fingerprint) return fast_path_declined("run", "the recorded build directory is not the one for this fingerprint"); - } - // Locate the requested run-target before doing any filesystem freshness // work — an unrecognized name falls back to prepare_build, which gives // a proper "no binary target 'x' found" error instead of a silent miss. @@ -1979,55 +2128,35 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, } if (!chosen) return fast_path_declined("run", "the requested program is not among the recorded ones"); - std::error_code ec; - std::filesystem::path outputDir(outputDirStr); - auto ninjaPath = outputDir / "build.ninja"; - if (!std::filesystem::exists(ninjaPath, ec)) return fast_path_declined("run", "build.ninja does not exist"); - // #407, same reason as try_fast_build: a test-shaped graph does not build - // the run target at all, so running ninja against it would report success - // and then exec a stale (or absent) binary. - if (!mcpp::build::is_plain_build_graph(ninjaPath)) return fast_path_declined("run", "build.ninja was written by another mode (test, pack or a named target)"); - // What the graph was planned for (workspace design 2026-09-29 §3): the - // features no longer name the directory, so the graph says which it has. - if (mcpp::build::read_request(ninjaPath) != mcpp::build::request_tag({}, want->features)) - return fast_path_declined("run", "build.ninja was written for another request (features or workspace members)"); - auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); - if (ec) return fast_path_declined("run", "the time of build.ninja cannot be read"); - - auto runtimeManifest = match->runtimeBinding->subosDir / ".xlings.json"; - auto runtimeTime = std::filesystem::last_write_time(runtimeManifest, ec); - if (ec || runtimeTime > ninjaTime) return fast_path_declined("run", "the runtime's .xlings.json is newer than build.ninja"); - - auto tomlPath = projectRoot / "mcpp.toml"; - auto tomlTime = std::filesystem::last_write_time(tomlPath, ec); - if (ec || tomlTime > ninjaTime) return fast_path_declined("run", "mcpp.toml is newer than build.ninja"); - - if (sources_newer_than(projectRoot, ninjaTime, want->resourceScripts, - want->extTable)) return fast_path_declined("run", "a project source, build.mcpp, a build-program input or a resource script is newer than build.ninja"); - // Same gate as try_fast_build's, and it has to be BOTH places: `mcpp run` - // reaches its binary through this path, so a `run` that skipped the check - // would execute an artifact built from a source set that no longer exists. - if (!match->depSourceRootsRecorded) return fast_path_declined("run", "the recorded build predates the list of path-dependency roots"); - if (dep_sources_newer_than(match->depSourceRoots, ninjaTime, want->extTable)) - return fast_path_declined("run", "a path dependency's manifest or source is newer than build.ninja"); - if (!xlings_payloads_present(*match)) return fast_path_declined("run", "a recorded xlings payload is missing"); - if (!local_toolchain_unchanged(match->outputDir)) - return fast_path_declined("run", "a program of the toolchain named by path changed"); - - auto validatedBefore = - mcpp::build::runtime_validation::validated_artifact_snapshot( - outputDir, *match->runtimeBinding); - if (!validatedBefore) return fast_path_declined("run", "no validated artifact snapshot is recorded for this build"); + // The gates every fast path shares (see admit_recorded_build), for the + // reason `mcpp run` is among the paths that need them: it execs the + // artifact itself, so an entry that cannot say which engine wrote it, or + // which runtime the program needs, must not be taken. What is left here is + // the project's own tree, as for try_fast_build; a graph that is not the + // plain build's does not build the run target at all (#407), so ninja + // against it would report success and then exec a stale or absent binary. + auto admitted = admit_recorded_build(*match, { + .request = mcpp::build::request_tag({}, want->features), + .manifest = projectRoot / "mcpp.toml", + .projectSourcesNewer = [&](std::filesystem::file_time_type ninjaTime) { + return sources_newer_than(projectRoot, ninjaTime, want->resourceScripts, + want->extTable); + }, + }); + if (!admitted) return fast_path_declined("run", admitted.error()); + const auto& outputDir = admitted->outputDir; + const auto& ninjaPath = admitted->ninjaPath; + const auto& validatedBefore = admitted->validated; // Fresh → run ninja (picks up any incremental object/link work) then // exec the cached exe path directly. C1, same reason as try_fast_build's. restore_root_compile_commands(projectRoot, outputDir); - auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, /*verbose=*/false, + auto rc = run_ninja_fast(admitted->ninjaProgram, outputDir, ninjaPath, /*verbose=*/false, match->runtimeEnvKey, match->runtimeEnvValue); if (!rc) return fast_path_declined("run", "ninja reported a stale graph"); if (*rc != 0) return kRunBuildFailed; if (!mcpp::build::runtime_validation::artifact_snapshot_unchanged( - *validatedBefore)) + validatedBefore)) return fast_path_declined("run", "ninja relinked an artifact, whose closure the full path validates"); // never execute an artifact not validated for this binding auto exe = outputDir / chosen->second; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index fbf3822e..bc15b36b 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -283,10 +283,20 @@ std::optional ninja_encoding_mismatch(std::string_view reported, unsigned processCodePage, std::string_view ninjaProgram); +// Where this engine is: the absolute path the emitter writes into build.ninja +// for the `$mcpp` rules and the `__action` wrapper. It is exported so that the +// record of a build (`.build_cache`) states the engine by the same function the +// graph does; two spellings of "where am I" would be one decision derived twice. +std::filesystem::path mcpp_exe_path(); + } // namespace mcpp::build namespace mcpp::build { +std::filesystem::path mcpp_exe_path() { + return mcpp::platform::fs::self_exe_path(); +} + namespace { // U+FEFF in UTF-8. The response files of the MSVC tools begin with it (#693). @@ -655,10 +665,6 @@ bool dyndep_mode_enabled() { return !(sv == "0" || sv == "off" || sv == "false"); } -std::filesystem::path mcpp_exe_path() { - return mcpp::platform::fs::self_exe_path(); -} - bool is_c_source(const mcpp::build::CompileUnit& cu) { return cu.kind == mcpp::SourceKind::C; } @@ -3504,10 +3510,12 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, // the same decision derived twice. // // The absolute path is baked into build.ninja, as every other tool - // path in it is. A version change regenerates the file (the version - // is in the fingerprint); moving the binary without changing its - // version would leave a stale path here, exactly as it would for - // the compiler. + // path in it is. A version change moves the build directory (the + // version is in the fingerprint), but that is not what keeps a + // stale path out of a replayed graph: the fast paths compute no + // fingerprint. They compare the engine the build record names (its + // version and `mcpp_exe_path()`, see `admit_recorded_build`), and a + // different engine plans the graph again (#757). const auto q = [](const std::string& v) { return shell_quote_arg(escape_ninja_chars(v)); }; diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 4342a38d..21578941 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -412,6 +412,24 @@ export struct PlanNote { std::string path; }; +// A source tree outside the project that a build read, with the classification +// of its own package. +// +// WHICH FILES OF A TREE CAN CHANGE THE GRAPH IS A PROPERTY OF THE PACKAGE THAT +// OWNS THEM. `.ixx` is a module interface only where a package declares it in +// `[build] module_extensions` (the built-in table is `.cppm`), and a provider +// declares it for its own sources without the consumer having a `.ixx` file at +// all. The freshness sweep used the consumer's table for every tree and so +// classified a provider's `.ixx` as a file of no interest (#756). The tables +// are recorded with the root, from the package's effective manifest, so the +// sweep asks each tree the question its owner answers. +export struct DepSourceRoot { + std::filesystem::path root; + std::vector moduleExtensions; + std::vector deviceExtensions; + bool operator==(const DepSourceRoot&) const = default; +}; + export struct BuildContext { // THE PER-MACHINE JOB DEFAULT, carried so it is read once. // @@ -451,7 +469,7 @@ export struct BuildContext { // cannot resolve dependencies without becoming prepare_build, and a second // derivation would drift from the first exactly when a resolution rule // changes. Written into `.build_cache`; see BuildCacheEntry::depSourceRoots. - std::vector depSourceRoots; + std::vector depSourceRoots; // `/bin` and then `` of every installed `[xlings] deps` // payload of the runtime-owner manifest, in declaration order (#544); the // pair comes from runner_lookup::payload_search_dirs. Read by diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 52306850..e1717ddd 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -138,7 +138,19 @@ static std::expected step13_source_packages(PrepareState& sta { std::vector owned = state.storeRoots; owned.push_back(mcpp::home::root()); - std::vector roots; + std::vector roots; + // A root is recorded with its own package's extension tables, from the + // package's effective manifest (see DepSourceRoot). + auto rootOf = [&](std::size_t i, std::filesystem::path normalized) { + const auto& bc = state.packages[i].manifest.buildConfig; + return DepSourceRoot{std::move(normalized), bc.moduleExtensions, + bc.deviceExtensions}; + }; + auto recorded = [&](const std::filesystem::path& normalized) { + return std::ranges::any_of(roots, [&](const DepSourceRoot& r) { + return r.root == normalized; + }); + }; // The same enumeration answers a second reader: which packages were // read from an editable tree, with their source globs (the build // database lists them as the inputs that change the plan). @@ -161,8 +173,8 @@ static std::expected step13_source_packages(PrepareState& sta normalized, state.packages[i].manifest.modules.sources}); if (i == 0 || normalized == state.root->lexically_normal()) continue; - if (std::find(roots.begin(), roots.end(), normalized) == roots.end()) - roots.push_back(std::move(normalized)); + if (!recorded(normalized)) + roots.push_back(rootOf(i, std::move(normalized))); } // A workspace plan's members are its projects: their trees are what // the fast path sweeps, the workspace's own package included. @@ -170,8 +182,8 @@ static std::expected step13_source_packages(PrepareState& sta for (std::size_t i = 1; i < state.packages.size(); ++i) { if (!state.packages[i].selectedMember) continue; auto normalized = state.packages[i].root.lexically_normal(); - if (std::find(roots.begin(), roots.end(), normalized) == roots.end()) - roots.push_back(normalized); + if (!recorded(normalized)) + roots.push_back(rootOf(i, std::move(normalized))); } ctx.depSourceRoots = std::move(roots); } diff --git a/tests/e2e/878_an_upgraded_engine_does_not_replay_the_old_graph.sh b/tests/e2e/878_an_upgraded_engine_does_not_replay_the_old_graph.sh new file mode 100755 index 00000000..f4b5bbc2 --- /dev/null +++ b/tests/e2e/878_an_upgraded_engine_does_not_replay_the_old_graph.sh @@ -0,0 +1,110 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 878_an_upgraded_engine_does_not_replay_the_old_graph.sh -- #757. +# +# build.ninja names the engine that wrote it by absolute path: the `$mcpp` rules +# (dyndep, stage, the BMI schedule) and the `__action` wrapper start that +# executable. The emitter assumed that a new engine plans the graph again, "the +# version is in the fingerprint", and no fast path computes a fingerprint: each +# matched the recorded entry by target, profile, cache mode, features and +# toolchain request, and replayed it. After an upgrade that removed the previous +# install, `mcpp build` ran the old graph and every `stage` action failed +# because the program it names was gone; with the old install still present the +# new front end drove the actions of an older engine without saying so. +# +# The record of a build now carries the engine that wrote it, its version and the +# path of its executable (the path the graph uses), and one predicate +# (`admit_recorded_build`) compares it for every fast path. +# +# One program is built with an engine copied to one path, and the same binary is +# then run from another path, which is what a reinstall into another directory +# looks like to the record (the version is unchanged, so the path is what +# differs, and it is the part that breaks the graph). +# +# A the engine is moved and nothing else changes: the build succeeds, plans +# the graph again, and no command of the new graph names the old path; +# B after that one build the next is replayed by the fast path again, and a +# second move together with an edit of a deployed file (an action that +# starts the engine has to run) succeeds with the new content in place; +# C a record written before the engine was recorded declines once (under -v +# the reason is printed), and the build after it is replayed; +# D `mcpp run` is asked the same question, and runs the program. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +MCPP="${MCPP:-mcpp}" +export NO_COLOR=1 +# The engine is copied out of the directory a home would be derived from, so the +# home the build uses is stated rather than found from the copy's location. +export MCPP_HOME="${MCPP_HOME:-$HOME/.mcpp}" + +EXE="" +case "$(uname -s)" in MINGW*|MSYS*|CYGWIN*) EXE=".exe" ;; esac + +replayed() { # replayed : the build was served by the fast path + ! grep -q "declined" "$1" && ! grep -q "Resolving toolchain" "$1" +} +names() { # names : does a graph under target/ name the directory + grep -rl --include=build.ninja -F -- "$1" target 2>/dev/null | grep -q . +} + +mkdir -p "$TMP/engine-one" "$TMP/engine-two" "$TMP/engine-three" "$TMP/app/src" "$TMP/app/assets" +cp "$MCPP" "$TMP/engine-one/mcpp$EXE" +cd "$TMP/app" +cat > mcpp.toml <<'TOML' +[package] +name = "upgrade878" +version = "0.1.0" + +[runtime] +deploy = [ { from = "assets/data.txt", to = "." } ] +TOML +printf '#include \nint main() { std::puts("hello"); return 0; }\n' > src/main.cpp +printf 'one\n' > assets/data.txt +deployed() { find target -name data.txt -path '*/bin/*' | head -1; } + +ONE="$TMP/engine-one/mcpp$EXE" +"$ONE" build > b0.log 2>&1 || fail "the first build failed" b0.log +"$ONE" build -v > b1.log 2>&1 || fail "the warm build failed" b1.log +replayed b1.log || fail "the warm build was not replayed, so nothing below is measured against the fast path" b1.log +names "engine-one" || fail "the graph does not name the engine that wrote it, so the criterion has nothing to detect" b0.log +grep -q '^engine=' target/.build_cache || fail "the record does not name the engine that wrote it" target/.build_cache + +# A. The engine moves; no source, manifest or deployed file changes. +mv "$TMP/engine-one/mcpp$EXE" "$TMP/engine-two/mcpp$EXE" +TWO="$TMP/engine-two/mcpp$EXE" +"$TWO" build -v > a1.log 2>&1 || fail "A: the build with the engine at its new path failed" a1.log +names "engine-one" && fail "A: a graph still names the path the engine was moved from" a1.log +names "engine-two" || fail "A: the regenerated graph does not name the engine that built it" a1.log +grep -q "fast-path: build declined" a1.log || fail "A: the build did not say that the fast path declined" a1.log + +# B. Replayed again, and an action that starts the engine runs. +"$TWO" build -v > b2.log 2>&1 || fail "B: the build after the regeneration failed" b2.log +replayed b2.log || fail "B: the build after the regeneration was not replayed by the fast path" b2.log +mv "$TMP/engine-two/mcpp$EXE" "$TMP/engine-three/mcpp$EXE" +THREE="$TMP/engine-three/mcpp$EXE" +sleep 1.1 +printf 'two\n' > assets/data.txt +"$THREE" build > b3.log 2>&1 || fail "B: the build after the second move and an edit of a deployed file failed" b3.log +[ "$(cat "$(deployed)")" = "two" ] || fail "B: the edit of the deployed file did not reach the output directory" b3.log +names "engine-two" && fail "B: a graph still names the path the engine was moved from" + +# C. A record written before the engine was recorded declines once. +grep -v '^engine=' target/.build_cache > target/.build_cache.aged +mv target/.build_cache.aged target/.build_cache +"$THREE" build -v > c1.log 2>&1 || fail "C: the build with an aged record failed" c1.log +grep -q "predates the engine identity" c1.log || fail "C: the decline did not name the missing engine identity" c1.log +grep -q '^engine=' target/.build_cache || fail "C: the build did not record the engine" target/.build_cache +"$THREE" build -v > c2.log 2>&1 || fail "C: the build after the aged record failed" c2.log +replayed c2.log || fail "C: the record written after the decline was not replayed" c2.log + +# D. `mcpp run`, whose fast path is a different function, asks the same question. +mv "$TMP/engine-three/mcpp$EXE" "$TMP/engine-one/mcpp$EXE" +"$ONE" run -v > d1.log 2>&1 || fail "D: the run with the engine at another path failed" d1.log +grep -q '^hello$' d1.log || fail "D: the program did not run" d1.log +grep -q "fast-path: run declined" d1.log || fail "D: the run did not say that the fast path declined" d1.log +names "engine-three" && fail "D: a graph still names the path the engine was moved from" + +echo "OK" diff --git a/tests/e2e/879_a_providers_module_extension_reaches_the_fast_path.sh b/tests/e2e/879_a_providers_module_extension_reaches_the_fast_path.sh new file mode 100755 index 00000000..b97fbc8d --- /dev/null +++ b/tests/e2e/879_a_providers_module_extension_reaches_the_fast_path.sh @@ -0,0 +1,186 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 879_a_providers_module_extension_reaches_the_fast_path.sh -- #756. +# +# A host module that a `path` build-dependency provides is compiled into the +# consumer's build program, which no edge of the consumer's build.ninja names, +# so the fast path has to see an edit to it itself. It sweeps the dependency's +# tree and decides which files can change the graph with an extension table. It +# used the CONSUMER's table, so a `.ixx` that only the provider declares was +# classified as a file of no interest, the edit was replayed as "no work", and +# the build program kept the old module. The consumer cannot be asked to declare +# `.ixx` as a workaround: it has no `.ixx` source, and the dead-entry warning it +# would then receive is correct. +# +# Classification belongs to the package that owns the file. Each root the build +# cache records now carries its own package's `module_extensions` and +# `device_extensions`, and the sweep classifies the files below a root with that +# root's table. +# +# The same provider and consumer are built as a project (the project fast path +# and `mcpp run`'s) and as a workspace member (the workspace fast path). +# +# A after a build and a warm build, an edit to `rules.ixx` re-runs the +# consumer's build program: the program prints the new value, in all three +# forms; +# B after one build that confirmed the edit the next build is replayed again +# (under -v: no "declined", no "Resolving toolchain"), and the next edit is +# seen as well; +# C the consumer declares nothing about `.ixx`, and no build prints the +# dead-entry warning; +# D a record that lists the roots without their tables declines once, and +# the build after it is replayed. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +MCPP="${MCPP:-mcpp}" +export NO_COLOR=1 + +EXE="" +case "$(uname -s)" in MINGW*|MSYS*|CYGWIN*) EXE=".exe" ;; esac +bin_of() { find "$1" -path "*/bin/*" -name "$2$EXE" -type f | head -1; } + +# A provider that declares `.ixx` and has one source; a consumer with only a +# `.cpp` source whose build program imports the provider's module. +provider() { + mkdir -p "$1" + cat > "$1/mcpp.toml" <<'TOML' +[package] +namespace = "repro" +name = "rules" +version = "0.1.0" +standard = "c++23" + +[lib] +path = "rules.ixx" + +[build] +sources = ["rules.ixx"] +module_extensions = [".ixx"] +TOML + value "$1" 1 +} +value() { + printf 'export module repro.rules;\n\nexport const char* repro_definition() {\n return "REPRO_VALUE=%s";\n}\n' "$2" > "$1/rules.ixx" +} +consumer() { + local dir="$1" + mkdir -p "$dir" + cat > "$dir/mcpp.toml" <<'TOML' +[package] +namespace = "repro" +name = "app" +version = "0.1.0" +standard = "c++23" + +[build] +sources = ["main.cpp"] + +[build-dependencies] +"repro.rules" = { path = "../rules", host-module = true } + +[targets.app] +kind = "bin" +main = "main.cpp" +TOML + cat > "$dir/build.mcpp" <<'CPP' +import mcpp; +import repro.rules; + +int main() { + mcpp::define(repro_definition()); + return 0; +} +CPP + cat > "$dir/main.cpp" <<'CPP' +#include + +int main() { + std::printf("REPRO_VALUE=%d\n", REPRO_VALUE); +} +CPP +} + +replayed() { # replayed : the build was served by the fast path + ! grep -q "declined" "$1" && ! grep -q "Resolving toolchain" "$1" +} + +cd "$TMP" + +# ── A project: the consumer builds alone, `mcpp run` runs its program ─────── +provider proj/rules +consumer proj/app +cd proj/app +run_value() { "$MCPP" run 2>/dev/null | grep '^REPRO_VALUE=' | tail -1; } + +"$MCPP" build > p0.log 2>&1 || fail "project: the first build failed" p0.log +[ "$(run_value)" = "REPRO_VALUE=1" ] || fail "project: the program did not print the provider's value" p0.log +"$MCPP" build -v > p1.log 2>&1 || fail "project: the warm build failed" p1.log +replayed p1.log || fail "project: the warm build was not replayed, so the next assertion measures nothing" p1.log + +# A, project build +sleep 1.1 +value ../rules 2 +"$MCPP" build > p2.log 2>&1 || fail "project: the build after the edit failed" p2.log +bin="$(bin_of target app)" +[ -n "$bin" ] || fail "project: the program was not found" p2.log +[ "$("$bin")" = "REPRO_VALUE=2" ] || fail "A: the edit of the provider's rules.ixx did not reach the program (build)" p2.log + +# B +"$MCPP" build -v > p3.log 2>&1 || fail "project: the build after the confirmed edit failed" p3.log +replayed p3.log || fail "B: the build after a confirmed edit was not replayed by the fast path" p3.log + +# A, project run: the fast path of `mcpp run` +sleep 1.1 +value ../rules 3 +out="$("$MCPP" run 2>&1)" || fail "project: the run after the second edit failed"$'\n'"$out" +grep -q '^REPRO_VALUE=3$' <<<"$out" || fail "A: the edit of the provider's rules.ixx did not reach the program (run)"$'\n'"$out" + +# D. A record that listed the roots without their tables (an engine before this +# one wrote that) declines once, and the record written after it is replayed. +"$MCPP" build > p4.log 2>&1 || fail "project: the build before ageing the record failed" p4.log +grep -q '^depSources=1$' target/.build_cache || fail "D: the record does not carry the root with its tables" target/.build_cache +awk '/^depSources=/ { n = substr($0, 12) + 0; print "depSourceRoots=" n + while (n-- > 0) { getline; split($0, f, "\t"); print f[1] } + next } + { print }' target/.build_cache > target/.build_cache.aged +mv target/.build_cache.aged target/.build_cache +grep -q '^depSourceRoots=1$' target/.build_cache || fail "D: the record was not aged" target/.build_cache +"$MCPP" build -v > d1.log 2>&1 || fail "D: the build with an aged record failed" d1.log +grep -q "predates the list of path-dependency roots" d1.log || fail "D: the decline did not name the missing tables" d1.log +"$MCPP" build -v > d2.log 2>&1 || fail "D: the build after the aged record failed" d2.log +replayed d2.log || fail "D: the record written after the decline was not replayed" d2.log +cd "$TMP" + +# ── A workspace member: the workspace fast path ───────────────────────────── +mkdir ws +printf '[workspace]\nmembers = ["app"]\n' > ws/mcpp.toml +provider ws/rules +consumer ws/app +cd ws + +"$MCPP" build -p app > w0.log 2>&1 || fail "workspace: the first build failed" w0.log +"$MCPP" build -p app -v > w1.log 2>&1 || fail "workspace: the warm build failed" w1.log +replayed w1.log || fail "workspace: the warm build was not replayed, so the next assertion measures nothing" w1.log +bin="$(bin_of target app)" +[ -n "$bin" ] && [ "$("$bin")" = "REPRO_VALUE=1" ] || fail "workspace: the program did not print the provider's value" w0.log + +# A, workspace +sleep 1.1 +value rules 2 +"$MCPP" build -p app > w2.log 2>&1 || fail "workspace: the build after the edit failed" w2.log +[ "$("$bin")" = "REPRO_VALUE=2" ] || fail "A: the edit of the provider's rules.ixx did not reach the program (workspace)" w2.log + +# B +"$MCPP" build -p app -v > w3.log 2>&1 || fail "workspace: the build after the confirmed edit failed" w3.log +replayed w3.log || fail "B: the workspace build after a confirmed edit was not replayed by the fast path" w3.log + +# C +for f in "$TMP"/proj/app/*.log "$TMP"/ws/*.log; do + if grep -q "dead entry" "$f"; then fail "C: a build printed the dead-entry warning" "$f"; fi +done +grep -q "module_extensions" "$TMP/proj/app/mcpp.toml" && fail "C: the consumer declares module_extensions, which the criterion forbids" + +echo "OK" diff --git a/tests/unit/test_build_cache_record.cpp b/tests/unit/test_build_cache_record.cpp new file mode 100644 index 00000000..4a4ddfd7 --- /dev/null +++ b/tests/unit/test_build_cache_record.cpp @@ -0,0 +1,277 @@ +#include + +import std; +import mcpp.build.execute; +import mcpp.build.prepare; + +using namespace mcpp::build; + +// The record of a build (`target/.build_cache`) and the one predicate every fast +// path asks of it. Three properties are fixed here: +// +// - the engine that wrote a record is part of it, and a record whose engine is +// not the running one is not replayed (#757); +// - each path-dependency root is recorded with, and swept by, the extension +// table of its own package (#756); +// - an entry written before either field declines, once, and does not +// misparse what follows it. + +namespace { + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_cache_record_test_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { + std::error_code ec; + std::filesystem::remove_all(path, ec); + } +}; + +void write_file(const std::filesystem::path& p, std::string_view body) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream os(p, std::ios::binary); + os << body; +} + +BuildCacheEntry minimal_entry() { + BuildCacheEntry e; + e.targetTriple = ""; + e.outputDir = "/work/target/x86_64-linux-gnu/abc"; + e.ninjaProgram = "/usr/bin/ninja"; + e.fingerprint = "abc"; + e.runtimeEnvKey = "-"; + return e; +} + +} // namespace + +// ─── the engine ───────────────────────────────────────────────────────────── + +TEST(EngineIdentity, TheRunningEngineIsNotDeclined) { + const auto engine = running_engine(); + EXPECT_FALSE(engine.version.empty()); + EXPECT_FALSE(engine.exe.empty()); + EXPECT_FALSE(engine_declined_because(engine, engine).has_value()); +} + +TEST(EngineIdentity, AnEntryWithoutTheFieldDeclinesAndSaysWhy) { + const auto why = engine_declined_because(std::nullopt, running_engine()); + ASSERT_TRUE(why.has_value()); + EXPECT_NE(why->find("predates the engine identity"), std::string::npos) << *why; +} + +TEST(EngineIdentity, AnotherVersionDeclinesAndNamesBoth) { + const auto running = running_engine(); + auto recorded = running; + recorded.version = "2026.1.1.1"; + const auto why = engine_declined_because(recorded, running); + ASSERT_TRUE(why.has_value()); + EXPECT_NE(why->find("2026.1.1.1"), std::string::npos) << *why; + EXPECT_NE(why->find(running.version), std::string::npos) << *why; +} + +// The path alone is enough. A binary moved to another directory, or reinstalled +// there, is the same version and leaves the same stale text in the graph that a +// new version does: the graph names the executable by absolute path. +TEST(EngineIdentity, AnotherPathAtTheSameVersionDeclinesAndNamesBoth) { + const auto running = running_engine(); + auto recorded = running; + recorded.exe = "/removed/install/bin/mcpp"; + const auto why = engine_declined_because(recorded, running); + ASSERT_TRUE(why.has_value()); + EXPECT_NE(why->find("/removed/install/bin/mcpp"), std::string::npos) << *why; + EXPECT_NE(why->find(running.exe), std::string::npos) << *why; +} + +// ─── the admission predicate ──────────────────────────────────────────────── + +// The engine is the first thing asked: when it is not the same engine, nothing +// else about the record is read, so the reason is the engine's whatever else is +// wrong with the entry. This is what makes the one predicate cover an upgrade +// for the three fast paths that call it. +TEST(AdmitRecordedBuild, AnotherEngineIsDeclinedBeforeAnythingElseIsRead) { + auto e = minimal_entry(); + e.engine = running_engine(); + e.engine->version = "2026.1.1.1"; + const auto admitted = admit_recorded_build(e, ReplayAsk{.manifest = "mcpp.toml"}); + ASSERT_FALSE(admitted.has_value()); + EXPECT_NE(admitted.error().find("2026.1.1.1"), std::string::npos) << admitted.error(); +} + +TEST(AdmitRecordedBuild, AnEntryWithoutTheEngineIsDeclined) { + auto e = minimal_entry(); + const auto admitted = admit_recorded_build(e, ReplayAsk{.manifest = "mcpp.toml"}); + ASSERT_FALSE(admitted.has_value()); + EXPECT_NE(admitted.error().find("predates the engine identity"), std::string::npos) + << admitted.error(); +} + +// With the engine matching, the question moves on to the entry's other fields: +// the decline is no longer the engine's. +TEST(AdmitRecordedBuild, TheSameEngineIsAskedTheNextQuestion) { + auto e = minimal_entry(); + e.engine = running_engine(); + const auto admitted = admit_recorded_build(e, ReplayAsk{.manifest = "mcpp.toml"}); + ASSERT_FALSE(admitted.has_value()); + EXPECT_EQ(admitted.error().find("engine"), std::string::npos) << admitted.error(); + EXPECT_NE(admitted.error().find("runtime binding"), std::string::npos) << admitted.error(); +} + +// ─── the record, written and read ─────────────────────────────────────────── + +TEST(BuildCacheRecord, TheEngineAndEachRootsTablesSurviveAWriteAndARead) { + Tmp tmp; + auto e = minimal_entry(); + e.engine = EngineIdentity{"2026.10.2.1", "/opt/with space/bin/mcpp"}; + e.depSourceRoots = { + {"/work/rules", {".ixx", ".ccm"}, {}}, + {"/work/plain", {}, {}}, + {"/work/device", {}, {".cu"}}, + }; + e.depSourceRootsRecorded = true; + e.toolchainRecorded = true; + e.toolchainRequest = "cli=;default=llvm@22.1.8"; + write_build_cache_entries(tmp.path / "target" / ".build_cache", {e}); + + const auto read = read_build_cache(tmp.path); + ASSERT_EQ(read.size(), 1u); + ASSERT_TRUE(read[0].engine.has_value()); + EXPECT_EQ(*read[0].engine, *e.engine); + EXPECT_TRUE(read[0].depSourceRootsRecorded); + EXPECT_EQ(read[0].depSourceRoots, e.depSourceRoots); + // The fields written after the two above were read too: the blocks do not + // swallow one another. + EXPECT_TRUE(read[0].toolchainRecorded); + EXPECT_EQ(read[0].toolchainRequest, e.toolchainRequest); +} + +TEST(BuildCacheRecord, AnEmptyListOfRootsIsRecordedNotAbsent) { + Tmp tmp; + auto e = minimal_entry(); + e.depSourceRootsRecorded = true; + write_build_cache_entries(tmp.path / "target" / ".build_cache", {e}); + const auto read = read_build_cache(tmp.path); + ASSERT_EQ(read.size(), 1u); + EXPECT_TRUE(read[0].depSourceRootsRecorded); + EXPECT_TRUE(read[0].depSourceRoots.empty()); + EXPECT_FALSE(read[0].engine.has_value()); +} + +// An engine that recorded the paths alone wrote `depSourceRoots=`. Its roots +// have no tables, so the block is read past and left unrecorded, and the fields +// after it still parse. +TEST(BuildCacheRecord, TheOlderBlockOfPathsIsReadPastAndUnrecorded) { + Tmp tmp; + write_file(tmp.path / "target" / ".build_cache", + "[target=]\n" + "/work/target/x86_64-linux-gnu/abc\n" + "/usr/bin/ninja\n" + "abc\n" + "-\n" + "\n" + "runTargets=0\n" + "runEnv=\n" + "\n" + "subos=\n" + "profile=dev\n" + "cacheMode=global\n" + "depSourceRoots=2\n" + "/work/rules\n" + "/work/other\n" + "runner=0\n" + "runtier=0\n" + "features=\n" + "toolchain=cli=;default=gcc@16.1.0\n" + "xlingsPayloads=0\n"); + const auto read = read_build_cache(tmp.path); + ASSERT_EQ(read.size(), 1u); + EXPECT_FALSE(read[0].depSourceRootsRecorded); + EXPECT_TRUE(read[0].depSourceRoots.empty()); + EXPECT_FALSE(read[0].engine.has_value()); + EXPECT_EQ(read[0].profile, "dev"); + EXPECT_TRUE(read[0].toolchainRecorded); + EXPECT_EQ(read[0].toolchainRequest, "cli=;default=gcc@16.1.0"); + EXPECT_TRUE(read[0].xlingsPayloadsRecorded); +} + +// A line of the block that is not a path and two lists leaves the block +// unrecorded rather than read with a table nobody wrote. +TEST(BuildCacheRecord, AMalformedRootLineLeavesTheBlockUnrecorded) { + Tmp tmp; + write_file(tmp.path / "target" / ".build_cache", + "[target=]\n" + "/work/target/x86_64-linux-gnu/abc\n" + "/usr/bin/ninja\n" + "abc\n" + "-\n" + "\n" + "depSources=1\n" + "/work/rules\n" + "runner=0\n"); + const auto read = read_build_cache(tmp.path); + ASSERT_EQ(read.size(), 1u); + EXPECT_FALSE(read[0].depSourceRootsRecorded); +} + +// ─── the sweep ────────────────────────────────────────────────────────────── + +namespace { + +// A dependency tree whose manifest is older than `ninjaTime` and whose one +// source, `rules.ixx`, is newer. +struct EditedProvider { + Tmp tmp; + std::filesystem::file_time_type ninjaTime; + EditedProvider() { + ninjaTime = std::filesystem::file_time_type::clock::now(); + write_file(tmp.path / "mcpp.toml", "[package]\nname = \"rules\"\n"); + write_file(tmp.path / "rules.ixx", "export module rules;\n"); + std::filesystem::last_write_time(tmp.path / "mcpp.toml", ninjaTime - std::chrono::seconds(20)); + std::filesystem::last_write_time(tmp.path / "rules.ixx", ninjaTime + std::chrono::seconds(20)); + } +}; + +} // namespace + +// The defect of #756: the provider declares `.ixx` and the consumer does not. +// The root is swept with its own package's table, so the edit is seen. +TEST(DepSourcesNewerThan, AProvidersOwnExtensionMakesItsEditCount) { + EditedProvider p; + EXPECT_TRUE(dep_sources_newer_than({{p.tmp.path, {".ixx"}, {}}}, p.ninjaTime)); +} + +// A root whose package declares nothing about `.ixx` does not have an `.ixx` of +// interest, whatever any other package declares: the table is the root's own. +TEST(DepSourcesNewerThan, APackageThatDeclaresNothingHasNoInterestInTheFile) { + EditedProvider p; + EXPECT_FALSE(dep_sources_newer_than({{p.tmp.path, {}, {}}}, p.ninjaTime)); +} + +// One root's table does not leak into the next: of two roots holding the same +// file, the one whose package declares the extension is the one that answers. +TEST(DepSourcesNewerThan, EachRootIsClassifiedByItsOwnTable) { + EditedProvider declares; + EditedProvider silent; + const auto ninjaTime = declares.ninjaTime; + EXPECT_FALSE(dep_sources_newer_than({{silent.tmp.path, {}, {}}}, ninjaTime)); + EXPECT_TRUE(dep_sources_newer_than( + {{silent.tmp.path, {}, {}}, {declares.tmp.path, {".ixx"}, {}}}, ninjaTime)); +} + +// What was always swept still is, with any table. +TEST(DepSourcesNewerThan, TheBuiltInInterfaceExtensionIsSweptWithoutADeclaration) { + EditedProvider p; + write_file(p.tmp.path / "m.cppm", "export module m;\n"); + std::filesystem::last_write_time(p.tmp.path / "m.cppm", p.ninjaTime + std::chrono::seconds(20)); + EXPECT_TRUE(dep_sources_newer_than({{p.tmp.path, {}, {}}}, p.ninjaTime)); +} + +TEST(DepSourcesNewerThan, ANewerManifestCounts) { + EditedProvider p; + std::filesystem::last_write_time(p.tmp.path / "mcpp.toml", p.ninjaTime + std::chrono::seconds(20)); + EXPECT_TRUE(dep_sources_newer_than({{p.tmp.path, {}, {}}}, p.ninjaTime)); +} From 6f42afbaac57f08857e3f4b3d20d451d08e9e8db Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 08:34:16 +0800 Subject: [PATCH 08/14] The graph's own engine binding is asked by the fast path; 2026.10.2.1 admit_recorded_build reads the `$mcpp` binding of build.ninja (read_engine_binding) and declines a graph that runs another engine, which the record alone cannot see when another install's --configure-only rewrote the graph. e2e 878 gains section E; three unit tests cover the reader. The version is 2026.10.2.1, with its CHANGELOG entry. --- CHANGELOG.md | 64 +++++++++++++++++++ mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- src/build/execute.cppm | 7 ++ src/build/graph_shape.cppm | 30 +++++++++ ...ed_engine_does_not_replay_the_old_graph.sh | 17 ++++- tests/unit/test_build_cache_record.cpp | 23 +++++++ 7 files changed, 142 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 75f2b13c..d8460bf6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,70 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. +## [2026.10.2.1] - 2026-10-02 + +This release repairs two fast-path defects (mcpp#756, mcpp#757), publishes the +specification by which toolchains are supported and maintained (SPEC-009), and +restructures the repository's CI so that a commit builds mcpp once per host +(`.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md`). +No default toolchain changes. + +### Fixed + +- **An upgraded mcpp no longer replays a graph that names the previous one** + (mcpp#757). The record of a build names the engine that wrote it, its version + and the path of its executable, and the fast paths of `mcpp build`, of a + workspace build and of `mcpp run` decline a record that another engine wrote. + The graph's own `$mcpp` binding is asked as well, since `--configure-only` + from another install rewrites the graph without the record. Before, the fast + path compared the recorded fingerprint with the recorded directory, and after + an upgrade that removed the old install every action that starts mcpp failed + with `CreateProcess failed` or `not found`. A record written by an earlier + release declines once. +- **One predicate admits a recorded build for all three fast paths**, so a + property of the record is checked in one place rather than three. +- **A path dependency's sources are classified by its own package** (mcpp#756). + A provider that declares `.ixx` in `[build] module_extensions` has its edits + seen by the fast path whatever the consumer declares; before, the sweep used + the consumer's table, and the edit of a `.ixx` host module was replayed as + "no work to do". + +### Added + +- **SPEC-009, the support and maintenance of toolchains** + (`docs/specs/toolchain-maintenance.md`). It states the support tiers, the one + table every default is read from, how a toolchain line moves as a whole, the + provenance of a payload and when a patched payload may be published, what it + means that mcpp supports a line, the compiler-defect register, the handling + of a new host platform release, the mirror rule, and the order in which a + default moves. A default moves only after the candidate passes a measured + gate: the module experience is not worse. SPEC-006 §7 now refers to it. Most + rules are not implemented yet, and each says so. + +### Changed (repository CI) + +- **One build of mcpp per host.** `ci.yml` runs a commit's CI in stages. + `changes` classifies the changed paths, and a change of documentation that + nothing reads runs the documentation checks only. `docs` runs the checks that + need no binary. `build.yml` builds mcpp once per host, and the per-area + workflows, now reusable, take that binary through + `.github/actions/use-built-mcpp`. Measured before this change, 30 to 37 jobs + of a commit built mcpp from source, which was 44 to 49 percent of its runner + minutes. +- **One writer per cache.** Every job restores. One job per key saves, on a + push to `main` only, and `target/` is no longer cached, because a restored + `target/` made no build incremental. +- **E2E shards by measured duration**, three on Linux, three on Windows and two + on macOS, from `tests/e2e/timings/`. `run_all.sh` takes `E2E_TIMINGS`, + `E2E_REPORT` and `E2E_LIST`. +- **Every e2e test runs somewhere.** The `e2e-coverage` job fails when a test + ran on no host, is named by no workflow, and is not excused in + `tests/e2e/coverage-exceptions.tsv`. Before it, 24 tests ran on no runner. + `run_all.sh` grants `llvm` on Linux, and it probes `musl` and `mingw-cross` + by family rather than by one release. +- The legs that are known red (#669) run on `main`, on dispatch, and on a pull + request labelled `macos-27`. + ## [2026.10.1.3] - 2026-10-01 This release gives every tool a build uses a source that can be declared, diff --git a/mcpp.toml b/mcpp.toml index 7bd71af1..9d0ebf02 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.10.1.3" +version = "2026.10.2.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 0c24be54..079b2dbb 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.10.1.3"; +inline constexpr std::string_view MCPP_VERSION = "2026.10.2.1"; } // namespace mcpp diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 639b28d8..443a47e7 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -1789,6 +1789,13 @@ admit_recorded_build(const BuildCacheEntry& e, const ReplayAsk& ask) { const auto ninjaPath = outputDir / "build.ninja"; if (!std::filesystem::exists(ninjaPath, ec)) return std::unexpected("build.ninja does not exist"); + // The graph names the engine it runs, and that is asked of the graph, not + // only of the record: another engine's `--configure-only` rewrites the + // graph and leaves the record that names this one. + if (const auto named = mcpp::build::read_engine_binding(ninjaPath); + !named.empty() && named != mcpp_exe_path().generic_string()) + return std::unexpected(std::format("build.ninja runs another engine ({})", named)); + // #407. Freshness is measured against the SOURCES, which says nothing // about what kind of graph this is. `mcpp test` and // `mcpp build --configure-only` write their plan -- dev-deps, test targets, diff --git a/src/build/graph_shape.cppm b/src/build/graph_shape.cppm index 16a0be93..24a2c353 100644 --- a/src/build/graph_shape.cppm +++ b/src/build/graph_shape.cppm @@ -133,6 +133,36 @@ std::string read_request(const std::filesystem::path& ninjaPath) { return {}; } +// The engine this graph runs as `$mcpp`, the binding every graph carries (see +// the emitter in ninja_backend.cppm), unescaped and in generic form; empty when +// the file cannot be read or binds no engine. The record of a build names the +// engine that wrote the record, and that is not always the one that wrote the +// graph: another engine's `--configure-only` rewrites build.ninja and leaves the +// record as it was. The graph is the authority on what it will run. +std::string read_engine_binding(const std::filesystem::path& ninjaPath) { + std::ifstream input(ninjaPath); + if (!input) return {}; + std::string line; + while (std::getline(input, line)) { + if (!line.starts_with("mcpp ") && !line.starts_with("mcpp=")) continue; + const auto eq = line.find('='); + if (eq == std::string::npos) continue; + std::string_view value = std::string_view(line).substr(eq + 1); + while (!value.empty() && value.front() == ' ') value.remove_prefix(1); + while (!value.empty() && (value.back() == '\r' || value.back() == ' ')) + value.remove_suffix(1); + // The inverse of escape_ninja_path: `$$`, `$:` and `$ ` stand for the + // character after the dollar. + std::string out; + for (std::size_t i = 0; i < value.size(); ++i) { + if (value[i] == '$' && i + 1 < value.size()) { out += value[++i]; continue; } + out += value[i]; + } + return out; + } + return {}; +} + // Read the shape back. `nullopt` means "this file does not say" — a build.ninja // written before this line existed, an unreadable file, or something that is // not a mcpp graph at all. Callers must treat that as a MISS, never as diff --git a/tests/e2e/878_an_upgraded_engine_does_not_replay_the_old_graph.sh b/tests/e2e/878_an_upgraded_engine_does_not_replay_the_old_graph.sh index f4b5bbc2..b4bbc39c 100755 --- a/tests/e2e/878_an_upgraded_engine_does_not_replay_the_old_graph.sh +++ b/tests/e2e/878_an_upgraded_engine_does_not_replay_the_old_graph.sh @@ -28,7 +28,10 @@ # starts the engine has to run) succeeds with the new content in place; # C a record written before the engine was recorded declines once (under -v # the reason is printed), and the build after it is replayed; -# D `mcpp run` is asked the same question, and runs the program. +# D `mcpp run` is asked the same question, and runs the program; +# E a graph that another engine rewrote while the record still names this +# one (what `--configure-only` from another install does) is not replayed: +# the graph's own `$mcpp` binding is asked as well. set -e TMP=$(mktemp -d) @@ -107,4 +110,16 @@ grep -q '^hello$' d1.log || fail "D: the program did not run" d1.log grep -q "fast-path: run declined" d1.log || fail "D: the run did not say that the fast path declined" d1.log names "engine-three" && fail "D: a graph still names the path the engine was moved from" +# E. The graph names its engine itself. Another engine rewrote the graph and +# left the record: the fast path asks the graph, declines, and plans again. +"$ONE" build > e0.log 2>&1 || fail "E: the build before the rewrite failed" e0.log +for g in $(find target -name build.ninja); do + awk -v p="$TMP/engine-gone/mcpp$EXE" '/^mcpp *=/ { print "mcpp = " p; next } { print }' \ + "$g" > "$g.rewritten" && mv "$g.rewritten" "$g" +done +names "engine-gone" || fail "E: the rewrite did not take" +"$ONE" build -v > e1.log 2>&1 || fail "E: the build of a graph another engine wrote failed" e1.log +grep -q "runs another engine" e1.log || fail "E: the decline did not name the graph's engine" e1.log +names "engine-gone" && fail "E: a graph still names the engine that rewrote it" + echo "OK" diff --git a/tests/unit/test_build_cache_record.cpp b/tests/unit/test_build_cache_record.cpp index 4a4ddfd7..2ae7a392 100644 --- a/tests/unit/test_build_cache_record.cpp +++ b/tests/unit/test_build_cache_record.cpp @@ -3,6 +3,7 @@ import std; import mcpp.build.execute; import mcpp.build.prepare; +import mcpp.build.graph_shape; using namespace mcpp::build; @@ -275,3 +276,25 @@ TEST(DepSourcesNewerThan, ANewerManifestCounts) { std::filesystem::last_write_time(p.tmp.path / "mcpp.toml", p.ninjaTime + std::chrono::seconds(20)); EXPECT_TRUE(dep_sources_newer_than({{p.tmp.path, {}, {}}}, p.ninjaTime)); } + +// The graph names the engine it runs as `$mcpp`, escaped for ninja; the reader +// returns the path the emitter escaped, so that it compares with +// `mcpp_exe_path()` in generic form. +TEST(ReadEngineBinding, TheEscapedPathIsReadBack) { + Tmp tmp; + write_file(tmp.path / "build.ninja", + "# mcpp:graph=normal\ncxx = /opt/llvm/bin/clang++\n" + "mcpp = C$:/Program$ Files/mcpp$$1/mcpp.exe\n\nrule cxx\n"); + EXPECT_EQ(read_engine_binding(tmp.path / "build.ninja"), "C:/Program Files/mcpp$1/mcpp.exe"); +} + +TEST(ReadEngineBinding, AGraphWithoutTheBindingSaysNothing) { + Tmp tmp; + write_file(tmp.path / "build.ninja", "cxx = clang++\nmcpp_extra = x\nrule cxx\n"); + EXPECT_EQ(read_engine_binding(tmp.path / "build.ninja"), ""); +} + +TEST(ReadEngineBinding, AMissingFileSaysNothing) { + Tmp tmp; + EXPECT_EQ(read_engine_binding(tmp.path / "absent.ninja"), ""); +} From 7e88d01aa721b5328fa821f66b18d862f7e44cf1 Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 08:35:26 +0800 Subject: [PATCH 09/14] docs: the contributing and release skills and the README badge describe the staged CI --- .agents/skills/mcpp-contributing/SKILL.md | 18 ++++++++++-------- .agents/skills/mcpp-release/SKILL.md | 6 +++--- README.md | 2 +- README.zh-CN.md | 2 +- 4 files changed, 15 insertions(+), 13 deletions(-) diff --git a/.agents/skills/mcpp-contributing/SKILL.md b/.agents/skills/mcpp-contributing/SKILL.md index e747335a..d6ebec57 100644 --- a/.agents/skills/mcpp-contributing/SKILL.md +++ b/.agents/skills/mcpp-contributing/SKILL.md @@ -182,14 +182,16 @@ gh pr checks # 查看状态 gh run view --log-failed # 查看失败日志 ``` -CI 由分平台的基础构建/单元集成检查与独立 E2E 检查组成： -| Workflow | 平台 | 内容 | -|----------|------|------| -| `ci-linux` / `ci-linux-e2e` | Linux x86_64 | 自举构建、unit/integration / 分片 E2E | -| `ci-macos` / `ci-macos-e2e` | macOS ARM64 | 自举构建、unit/integration / E2E | -| `ci-windows` / `ci-windows-e2e` | Windows x86_64 | 自举构建、toolchain 回归 / E2E | -| `cross-build-test` | Linux/Windows cross targets | 交叉构建、产物运行与 MinGW/Wine 检查 | -| `ci-aarch64-fresh-install` | Linux ARM64 native | path-filtered fresh install、原生自举与 musl `build.mcpp` host-helper 回归 | +一次提交的 CI 是 `ci.yml` 的一次运行，分段执行（设计见 `.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md` 第二部分）： +| 阶段 | 内容 | +|------|------| +| `changes` | 按改动路径分类；只改了没有任何脚本、测试或源码读取的文档时，只跑 `docs` | +| `docs` | 不需要二进制的检查（版本钉、文档风格与结构、工作流断言等） | +| `build-*` | 每个宿主构建一次 mcpp（`build.yml`），上传为 `mcpp-built-` | +| `linux` / `linux-e2e`、`macos` / `macos-e2e` / `macos-ios`、`windows` / `windows-e2e` / `windows-msvc-xlings`、`cross`、`target-matrix`、`openkal` | 各领域的可复用工作流（原来的 `ci-*.yml`），通过 `.github/actions/use-built-mcpp` 使用上面那次构建，不再各自构建 | +| `e2e-coverage` | 每个 e2e 测试都在某个宿主上运行、由专门 job 运行，或在 `tests/e2e/coverage-exceptions.tsv` 中写明原因 | + +`ci-aarch64-fresh-install`、`measure-windows-tool-crt` 与 `pypi-publish` 仍是按路径触发的独立工作流。缓存只在 main 上由一个 job 保存；PR 只恢复。 **以 PR 实际 required checks 为准，所有未跳过的 required checks 必须通过。** 如果某个平台失败： 1. 下载日志分析原因 diff --git a/.agents/skills/mcpp-release/SKILL.md b/.agents/skills/mcpp-release/SKILL.md index e28e37d0..abe28c57 100644 --- a/.agents/skills/mcpp-release/SKILL.md +++ b/.agents/skills/mcpp-release/SKILL.md @@ -73,9 +73,9 @@ gh run list --branch main --limit 3 ``` 以分支保护和 `gh pr checks ` 显示的 actual required checks 为准。 -在 main 上监控当前运行时，检查 `ci-linux`、`ci-linux-e2e`、`ci-macos`、 -`ci-macos-e2e`、`ci-windows`、`ci-windows-e2e` 与 `cross-build-test` 的结果； -跳过或非 required 的 workflow 不是合入 gate。不要在 required CI 红的时候发版。 +在 main 上监控当前运行时，检查 `ci` 这一个工作流的运行（它包含各平台、e2e 分片、 +交叉构建与 `e2e-coverage`）；known-red 的腿（名字里带 issue 号）允许失败。跳过或 +非 required 的 workflow 不是合入 gate。不要在 `ci` 红的时候发版。 ### 2. bump 版本号（第一组两处，单个 commit，走 PR） diff --git a/README.md b/README.md index 14a62860..06306cd0 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ | [Documentation](docs/) · [Getting Started](docs/01-getting-started.md) · [mcpp.toml Guide](docs/04-mcpp-toml.md) · [Examples](docs/03-examples.md) · [Toolchains](docs/20-toolchains.md) | |:---:| | [Package index mcpp-index](https://mcpplibs.github.io/mcpp-index/) · [Module libraries mcpplibs](https://github.com/mcpplibs) · [Community Forum](https://forum.d2learn.org/category/20) · [Issues](https://github.com/mcpp-community/mcpp/issues) · [Releases](https://github.com/mcpp-community/mcpp/releases) | -| [![ci-linux](https://github.com/mcpp-community/mcpp/actions/workflows/ci-linux.yml/badge.svg?branch=main)](https://github.com/mcpp-community/mcpp/actions/workflows/ci-linux.yml) [![ci-macos](https://github.com/mcpp-community/mcpp/actions/workflows/ci-macos.yml/badge.svg?branch=main)](https://github.com/mcpp-community/mcpp/actions/workflows/ci-macos.yml) [![ci-windows](https://github.com/mcpp-community/mcpp/actions/workflows/ci-windows.yml/badge.svg?branch=main)](https://github.com/mcpp-community/mcpp/actions/workflows/ci-windows.yml) | +| [![ci](https://github.com/mcpp-community/mcpp/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/mcpp-community/mcpp/actions/workflows/ci.yml) | | Plugins · [mcpp-language-server (mcppls)](https://github.com/Sunrisepeak/mcpp-language-server) — a C++20/23 modules language server for VS Code, Zed, CLion, Neovim, AI agents (MCP) and CI |

diff --git a/README.zh-CN.md b/README.zh-CN.md index 521dfcb9..5c0fdb1f 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -12,7 +12,7 @@ | [文档](docs/zh/) · [快速开始](docs/zh/01-getting-started.md) · [mcpp.toml 指南](docs/zh/04-mcpp-toml.md) · [示例项目](docs/zh/03-examples.md) · [工具链管理](docs/zh/20-toolchains.md) | |:---:| | [包索引 mcpp-index](https://mcpplibs.github.io/mcpp-index/) · [模块化库 mcpplibs](https://github.com/mcpplibs) · [社区论坛](https://forum.d2learn.org/category/20) · [Issues](https://github.com/mcpp-community/mcpp/issues) · [Releases](https://github.com/mcpp-community/mcpp/releases) | -| [![ci-linux](https://github.com/mcpp-community/mcpp/actions/workflows/ci-linux.yml/badge.svg?branch=main)](https://github.com/mcpp-community/mcpp/actions/workflows/ci-linux.yml) [![ci-macos](https://github.com/mcpp-community/mcpp/actions/workflows/ci-macos.yml/badge.svg?branch=main)](https://github.com/mcpp-community/mcpp/actions/workflows/ci-macos.yml) [![ci-windows](https://github.com/mcpp-community/mcpp/actions/workflows/ci-windows.yml/badge.svg?branch=main)](https://github.com/mcpp-community/mcpp/actions/workflows/ci-windows.yml) | +| [![ci](https://github.com/mcpp-community/mcpp/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/mcpp-community/mcpp/actions/workflows/ci.yml) | | 支持的插件 · [mcpp-language-server（mcppls）](https://github.com/Sunrisepeak/mcpp-language-server) —— C++20/23 模块语言服务器，面向 VS Code、Zed、CLion、Neovim、AI Agent（MCP）与 CI |

From cd8dad92c2e5c502b2eec6b082308693b43cf59d Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 08:42:20 +0800 Subject: [PATCH 10/14] Repairs from the review of #759: whole-token coverage, a classifier that falls back to the whole CI, a concurrency group per push to main, cache keys that name the install list, a step for the shard's installs, and a record that leaves a tab-holding root unrecorded --- ...-and-the-toolchain-specification-design.md | 21 +++++++++- .github/actions/bootstrap-mcpp/action.yml | 5 ++- .github/tools/check_e2e_coverage.py | 34 ++++++++++++---- .github/workflows/ci-linux-e2e.yml | 39 +++++++++++-------- .github/workflows/ci.yml | 23 +++++++---- .github/workflows/cross-build-test.yml | 4 +- CHANGELOG.md | 4 +- src/build/execute.cppm | 19 +++++++-- tests/scripts/test_check_e2e_coverage.py | 15 +++++++ tests/unit/test_build_cache_record.cpp | 17 ++++++++ 10 files changed, 139 insertions(+), 42 deletions(-) diff --git a/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md b/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md index 1fb5538e..2f2647d6 100644 --- a/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md +++ b/.agents/docs/2026-10-02-pr-ci-acceleration-and-the-toolchain-specification-design.md @@ -421,8 +421,13 @@ repository has paid for that before. are the main cause of F3. They return only if the build becomes incremental across checkouts. That needs a measured cause (Part VII) and a fix that keeps the stale-object hazards already recorded in `.agents/docs/2026-05-15-stdcompat-restat-e2e.md` and `cross-build-test.yml:416-434` out of the result. -- **Concurrency.** `cancel-in-progress` becomes `${{ github.event_name == 'pull_request' }}`. A push to - `main` runs to completion and writes its caches; a superseded pull-request run is still cancelled. +- **Concurrency.** `cancel-in-progress` becomes `${{ github.event_name == 'pull_request' }}`, and a push + to `main` has a group of its own commit. A push to `main` runs to completion and writes its caches; a + superseded pull-request run is still cancelled. One group for all pushes to `main` would not be enough, + because GitHub keeps one pending run per group and cancels the older pending run when another arrives. +- **Keys name what fills the cache.** The sandbox key hashes `ci.yml` as well as `mcpp.toml` and + `.xlings.json`, because `ci.yml` lists what the build job installs before it saves. A cache saved under + an unchanged key is never saved again, so a toolchain added to that list would otherwise never reach it. - **Wine.** The Wine packages come from one pinned archive, published once as a release asset of this repository, instead of from the `apt` mirrors with a cache that eviction removes (F7). The archive's sha256 is checked before installation. @@ -620,6 +625,18 @@ reading taken while building it. machine that has them and failed on every runner, and nothing noticed, because no runner ran it. The fixture's root is now a library, which is what the test says it is: "this test compiles only". +- **The review of the change** (a code review of mcpp#759) found eight defects. Each was repaired before + merging: + - the coverage check matched test names inside comments and inside longer names, and now matches whole + tokens outside comments; + - `changes` failed, rather than running the whole CI, when the GitHub API did not answer; + - pushes to `main` shared a concurrency group; + - the cache keys did not name the install list; + - the Linux shards' toolchain installs shared the suite's time limit, and now have a step of their own; + - the `macos-27` label's effect was undocumented (it is read on the next push); + - the CHANGELOG gave three Linux shards; + - a path-dependency root with a tab in its path could be cut short in the build record, and is now left + unrecorded, which declines the fast path. - **Coverage found more than F9.** Classifying the tests of the 2026-10-01 logs found 24 that ran on no runner and were named by no workflow. The seven `llvm` tests are among them, and so are three `musl` tests: the probe diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 5ca4e06a..b549ccae 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -85,7 +85,10 @@ runs: # sandbox — which is what actually resolves dependencies — would # silently stay behind (observed: a 0.4.30 sandbox surviving under a # 0.4.69 bootstrap for weeks). - key: mcpp-sandbox-${{ runner.os }}-${{ runner.arch }}-ci-xl${{ inputs.xlings-version }}-${{ hashFiles('mcpp.toml', '.xlings.json') }} + # ci.yml is part of the key because it names the toolchains the build + # job installs before it saves (`prewarm`): a sandbox saved under an + # unchanged key would never gain one added there. + key: mcpp-sandbox-${{ runner.os }}-${{ runner.arch }}-ci-xl${{ inputs.xlings-version }}-${{ hashFiles('mcpp.toml', '.xlings.json', '.github/workflows/ci.yml') }} restore-keys: | mcpp-sandbox-${{ runner.os }}-${{ runner.arch }}-ci-xl${{ inputs.xlings-version }}- diff --git a/.github/tools/check_e2e_coverage.py b/.github/tools/check_e2e_coverage.py index 54cca913..c98c9869 100644 --- a/.github/tools/check_e2e_coverage.py +++ b/.github/tools/check_e2e_coverage.py @@ -18,9 +18,11 @@ under tests/e2e: ran some report says pass, fail or timeout; - job no report ran it, but a workflow names it (by file name, or by its - number as an `E2E_ONLY` pattern such as `239_*.sh`): a dedicated job - runs it and asserts its result itself; + job no report ran it, but a workflow names it outside a comment, as a + whole token: by file name, by name without `.sh`, or by its number + as an `E2E_ONLY` pattern such as `239_*.sh`. A dedicated job runs it + and asserts its result itself. A name in a comment does not count, + and neither does a longer name that contains it; excused tests/e2e/coverage-exceptions.tsv lists it with the reason no hosted runner can run it; uncovered none of these. The check fails. @@ -74,9 +76,25 @@ def read_exceptions(path: Path) -> dict[str, str]: return out -def named_by_a_workflow(test: str, workflows: str) -> bool: - number = test.split("_", 1)[0] - return test in workflows or test[:-3] in workflows or f"{number}_*" in workflows +TOKEN = re.compile(r"(? set[str]: + """Every test name or `_*` pattern named outside a comment.""" + tokens: set[str] = set() + for text in texts: + for line in text.splitlines(): + code = line.split("#", 1)[0] if line.lstrip().startswith("#") else line + if not code.strip(): + continue + tokens.update(m.group(1) for m in TOKEN.finditer(code)) + return tokens + + +def named_by_a_workflow(test: str, tokens: set[str]) -> bool: + stem = test[:-3] + number = stem.split("_", 1)[0] + return stem in tokens or f"{number}_*" in tokens def main() -> int: @@ -88,8 +106,8 @@ def main() -> int: root = Path(args.root).resolve() tests = sorted(p.name for p in (root / "tests" / "e2e").glob("[0-9]*.sh")) - workflows = "\n".join(p.read_text(encoding="utf-8") - for p in sorted((root / ".github" / "workflows").glob("*.yml"))) + workflows = workflow_tokens([p.read_text(encoding="utf-8") + for p in sorted((root / ".github" / "workflows").glob("*.yml"))]) exceptions = read_exceptions(root / "tests" / "e2e" / "coverage-exceptions.tsv") ran: dict[str, set[str]] = defaultdict(set) diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index eade307d..8c5b3966 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -48,23 +48,12 @@ jobs: with: host: linux-x86_64 - - name: E2E suite - # About twice the shard's budget of ten minutes (R4): reached only by a - # hang. The per-test 600 s limit in run_all.sh names the test that hung. - timeout-minutes: 22 + # Its own step and its own limit, so that downloads on a cold sandbox are + # not taken from the suite's budget. + - name: The toolchains the suite probes + timeout-minutes: 15 run: | set -euo pipefail - # MCPP is this commit's binary and MCPP_BOOT the released bootstrap - # (use-built-mcpp). e2e 252 needs the latter: the claim that an older - # client can build against a package this commit produces is only - # worth making against a real old binary. - export MCPP MCPP_BOOT - test -x "$MCPP_VENDORED_XLINGS" - # GitHub-hosted runners are outside CN; keep CI toolchain downloads on - # the global mirror while mcpp's default remains CN for fresh local - # sandboxes. E2E tests with their own MCPP_HOME read this variable. - export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL - "$MCPP" self config # Pin the global default so test 28 (default-toolchain path) gets a # deterministic GNU answer instead of an auto-install pick. Installed # explicitly, not assumed: the restored sandbox is the build job's, @@ -81,6 +70,24 @@ jobs: "$MCPP" toolchain install llvm 22.1.8 "$MCPP" toolchain install mingw-cross 16.1.0 XLINGS_HOME="$MCPP_HOME/registry" "$MCPP_VENDORED_XLINGS" install xim:nasm -y + + - name: E2E suite + # About twice the shard's budget of ten minutes (R4): reached only by a + # hang. The per-test 600 s limit in run_all.sh names the test that hung. + timeout-minutes: 22 + run: | + set -euo pipefail + # MCPP is this commit's binary and MCPP_BOOT the released bootstrap + # (use-built-mcpp). e2e 252 needs the latter: the claim that an older + # client can build against a package this commit produces is only + # worth making against a real old binary. + export MCPP MCPP_BOOT + test -x "$MCPP_VENDORED_XLINGS" + # GitHub-hosted runners are outside CN; keep CI toolchain downloads on + # the global mirror while mcpp's default remains CN for fresh local + # sandboxes. E2E tests with their own MCPP_HOME read this variable. + export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL + "$MCPP" self config bash tests/e2e/run_all.sh # One file, no glob: the report the e2e-coverage job of ci.yml reads. @@ -346,7 +353,7 @@ jobs: uses: actions/cache/restore@v4 with: path: ~/.mcpp - key: mcpp-hermetic-${{ hashFiles('mcpp.toml') }} + key: mcpp-hermetic-${{ hashFiles('mcpp.toml', '.github/workflows/ci-linux-e2e.yml') }} restore-keys: | mcpp-hermetic- diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 30272628..77f622f2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -27,11 +27,13 @@ on: branches: [ main ] workflow_dispatch: -# A superseded pull-request run is cancelled. A push to main runs to the end, -# because its build jobs are the only writers of the caches (R3), and a -# cancelled run saves nothing. +# A superseded pull-request run is cancelled. Every push to main runs to the +# end, in a group of its own commit: its build jobs are the only writers of the +# caches (R3), and a cancelled run saves nothing. A group shared by all pushes +# to main would still lose runs, because GitHub keeps one pending run per group +# and cancels the older pending one when a third arrives. concurrency: - group: ci-${{ github.ref }} + group: ci-${{ github.event_name == 'push' && github.sha || github.ref }} cancel-in-progress: ${{ github.event_name == 'pull_request' }} permissions: @@ -61,15 +63,19 @@ jobs: run: | set -euo pipefail full() { echo "code=true" >> "$GITHUB_OUTPUT"; echo "code=true ($1)"; } + # Anything this step cannot list runs the whole CI: an API error, a + # force-push whose previous commit is gone, an event without a diff. case "$EVENT" in pull_request) gh api --paginate "repos/$REPO/pulls/$PR/files" \ - --jq '.[] | .filename, (.previous_filename // empty)' > changed.txt ;; + --jq '.[] | .filename, (.previous_filename // empty)' > changed.txt \ + || { full "the files of the pull request could not be listed"; exit 0; } ;; push) if [ -z "$BEFORE" ] || [ "$BEFORE" = 0000000000000000000000000000000000000000 ]; then full "a push with no previous commit"; exit 0 fi - gh api "repos/$REPO/compare/$BEFORE...$SHA" > compare.json + gh api "repos/$REPO/compare/$BEFORE...$SHA" > compare.json \ + || { full "the push could not be compared with its previous commit"; exit 0; } # The compare API lists at most 300 files; a longer list is # treated as a change of everything. if [ "$(jq '.files | length' compare.json)" -ge 300 ]; then @@ -187,7 +193,10 @@ jobs: uses: ./.github/workflows/ci-macos.yml with: # R7: the known-red legs (#669) run where they can change a decision: on - # main, on dispatch, and on a pull request labelled `macos-27`. + # main, on dispatch, and on a pull request labelled `macos-27`. The label + # is read when the pull request is pushed to; adding it starts no run, + # because a run on every label of every pull request would cost a whole + # CI each time. known-red: ${{ github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'macos-27') }} macos-e2e: diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 8133f296..ca61fc70 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -80,7 +80,7 @@ jobs: uses: actions/cache/restore@v4 with: path: ~/.mcpp - key: mcpp-sandbox-${{ runner.os }}-cross-${{ matrix.target }}-${{ hashFiles('mcpp.toml', '.xlings.json') }} + key: mcpp-sandbox-${{ runner.os }}-cross-${{ matrix.target }}-${{ hashFiles('mcpp.toml', '.xlings.json', '.github/workflows/cross-build-test.yml') }} restore-keys: | mcpp-sandbox-${{ runner.os }}-cross-${{ matrix.target }}- @@ -244,7 +244,7 @@ jobs: uses: actions/cache/restore@v4 with: path: ~/.mcpp - key: mcpp-sandbox-${{ runner.os }}-mingw-cross-${{ hashFiles('mcpp.toml', '.xlings.json') }} + key: mcpp-sandbox-${{ runner.os }}-mingw-cross-${{ hashFiles('mcpp.toml', '.xlings.json', '.github/workflows/cross-build-test.yml') }} restore-keys: | mcpp-sandbox-${{ runner.os }}-mingw-cross- diff --git a/CHANGELOG.md b/CHANGELOG.md index d8460bf6..461a702c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,7 +57,7 @@ No default toolchain changes. - **One writer per cache.** Every job restores. One job per key saves, on a push to `main` only, and `target/` is no longer cached, because a restored `target/` made no build incremental. -- **E2E shards by measured duration**, three on Linux, three on Windows and two +- **E2E shards by measured duration**, four on Linux, three on Windows and two on macOS, from `tests/e2e/timings/`. `run_all.sh` takes `E2E_TIMINGS`, `E2E_REPORT` and `E2E_LIST`. - **Every e2e test runs somewhere.** The `e2e-coverage` job fails when a test @@ -66,7 +66,7 @@ No default toolchain changes. `run_all.sh` grants `llvm` on Linux, and it probes `musl` and `mingw-cross` by family rather than by one release. - The legs that are known red (#669) run on `main`, on dispatch, and on a pull - request labelled `macos-27`. + request labelled `macos-27` (the label is read on the next push). ## [2026.10.1.3] - 2026-10-01 diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 443a47e7..85acf380 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -667,10 +667,21 @@ void write_build_cache_entries(const std::filesystem::path& path, << '\n'; f << "profile=" << e.profile << '\n'; f << "cacheMode=" << e.cacheMode << '\n'; - f << "depSources=" << e.depSourceRoots.size() << '\n'; - for (auto& r : e.depSourceRoots) - f << r.root.generic_string() << '\t' << join_extensions(r.moduleExtensions) - << '\t' << join_extensions(r.deviceExtensions) << '\n'; + // A root whose path holds a tab or a line break cannot be written in + // this line format: read back, the path would be cut at the tab, and a + // directory that happened to exist under the shorter name would be + // swept with the wrong table. Such an entry records no roots, which + // reads as "predates the list" and declines the fast path, the safe + // direction. + const bool writable = std::ranges::none_of(e.depSourceRoots, [](auto const& r) { + return r.root.generic_string().find_first_of("\t\n\r") != std::string::npos; + }); + if (writable) { + f << "depSources=" << e.depSourceRoots.size() << '\n'; + for (auto& r : e.depSourceRoots) + f << r.root.generic_string() << '\t' << join_extensions(r.moduleExtensions) + << '\t' << join_extensions(r.deviceExtensions) << '\n'; + } f << "runner=" << (e.runnerDeclared ? 1 : 0) << '\n'; f << "runtier=" << (e.runTierPending ? 1 : 0) << '\n'; f << "features=" << e.features << '\n'; diff --git a/tests/scripts/test_check_e2e_coverage.py b/tests/scripts/test_check_e2e_coverage.py index 7954ef75..e27bd644 100644 --- a/tests/scripts/test_check_e2e_coverage.py +++ b/tests/scripts/test_check_e2e_coverage.py @@ -48,6 +48,21 @@ def test_ran_named_and_excused_tests_are_covered(self) -> None: self.assertEqual(r.returncode, 0, r.stdout) self.assertIn("1 ran on a shard, 2 run by a dedicated job, 1 excused, 0 uncovered", r.stdout) + def test_a_name_in_a_comment_or_inside_a_longer_name_does_not_count(self) -> None: + e2e = self.root / "tests" / "e2e" + (e2e / "60_commented.sh").write_text("# x\n# requires: llvm\n") + (e2e / "70_short.sh").write_text("# x\n# requires: llvm\n") + with (self.root / ".github" / "workflows" / "ci.yml").open("a") as f: + f.write(" # 60_commented.sh is mentioned in a comment only\n" + "run: bash tests/e2e/170_short_but_longer.sh\n") + (self.reports / "e2e-report-linux-2.tsv").write_text( + "skip\t60_commented.sh\t0\tmissing capability: llvm\n" + "skip\t70_short.sh\t0\tmissing capability: llvm\n") + r = self.run_check() + self.assertEqual(r.returncode, 1) + self.assertIn("UNCOVERED: 60_commented.sh", r.stdout) + self.assertIn("UNCOVERED: 70_short.sh", r.stdout) + def test_a_test_that_runs_nowhere_fails(self) -> None: (self.root / "tests" / "e2e" / "50_nowhere.sh").write_text("# x\n# requires: llvm\n") (self.reports / "e2e-report-linux-2.tsv").write_text( diff --git a/tests/unit/test_build_cache_record.cpp b/tests/unit/test_build_cache_record.cpp index 2ae7a392..60d80e66 100644 --- a/tests/unit/test_build_cache_record.cpp +++ b/tests/unit/test_build_cache_record.cpp @@ -150,6 +150,23 @@ TEST(BuildCacheRecord, TheEngineAndEachRootsTablesSurviveAWriteAndARead) { EXPECT_EQ(read[0].toolchainRequest, e.toolchainRequest); } +// A path the line format cannot hold is not written at all, so the entry reads +// as one that predates the list and declines, rather than as a shorter path. +TEST(BuildCacheRecord, ARootWithATabInItsPathIsLeftUnrecorded) { + Tmp tmp; + auto e = minimal_entry(); + e.depSourceRoots = {{"/work/a\tb", {".ixx"}, {}}}; + e.depSourceRootsRecorded = true; + e.toolchainRecorded = true; + e.toolchainRequest = "cli=;default=gcc@16.1.0"; + write_build_cache_entries(tmp.path / "target" / ".build_cache", {e}); + const auto read = read_build_cache(tmp.path); + ASSERT_EQ(read.size(), 1u); + EXPECT_FALSE(read[0].depSourceRootsRecorded); + EXPECT_TRUE(read[0].depSourceRoots.empty()); + EXPECT_TRUE(read[0].toolchainRecorded); +} + TEST(BuildCacheRecord, AnEmptyListOfRootsIsRecordedNotAbsent) { Tmp tmp; auto e = minimal_entry(); From 3ea3fb11d7b1657af6ccadbbfae923cf8b7699fb Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 17:59:35 +0800 Subject: [PATCH 11/14] PR #759, round 4: the Windows literal-to-optional form, and a 5-line tighten so src/build/prepare/plan.cpp is back under the 2,500-line gate The third CI round (run 36946677551) had two failures from the #756/#757 commit's +12 lines: - Windows clang+MSVC STL would not compile a literal returned directly to std::optional in engine_declined_because (execute.cppm:189). std::string(...) makes the conversion explicit and identical on every STL. - src/build/prepare/plan.cpp went from 2,491 to 2,503 lines, three over the check_file_lengths.sh cap. The new rootOf/recorded lambdas in step13_source_packages carry DepSourceRoot (a comment is redundant with the struct's own), and the two are tight enough to put the file back at 2,498. No semantic change. Local unit tests: 20/20 BuildCacheRecord tests pass (BuildCacheRecord, DepSourcesNewerThan, ReadEngineBinding). --- src/build/execute.cppm | 2 +- src/build/prepare/plan.cpp | 13 ++++--------- 2 files changed, 5 insertions(+), 10 deletions(-) diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 85acf380..d58b9ca2 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -186,7 +186,7 @@ export EngineIdentity running_engine() { export std::optional engine_declined_because(const std::optional& recorded, const EngineIdentity& running) { - if (!recorded) return "the recorded build predates the engine identity"; + if (!recorded) return std::string("the recorded build predates the engine identity"); if (recorded->version != running.version) return std::format("the recorded build was written by mcpp {}, and this is mcpp {}", recorded->version, running.version); diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index e1717ddd..ad2f93a1 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -139,17 +139,12 @@ static std::expected step13_source_packages(PrepareState& sta std::vector owned = state.storeRoots; owned.push_back(mcpp::home::root()); std::vector roots; - // A root is recorded with its own package's extension tables, from the - // package's effective manifest (see DepSourceRoot). - auto rootOf = [&](std::size_t i, std::filesystem::path normalized) { + auto rootOf = [&](std::size_t i, std::filesystem::path n) { const auto& bc = state.packages[i].manifest.buildConfig; - return DepSourceRoot{std::move(normalized), bc.moduleExtensions, - bc.deviceExtensions}; + return DepSourceRoot{std::move(n), bc.moduleExtensions, bc.deviceExtensions}; }; - auto recorded = [&](const std::filesystem::path& normalized) { - return std::ranges::any_of(roots, [&](const DepSourceRoot& r) { - return r.root == normalized; - }); + auto recorded = [&](const std::filesystem::path& n) { + return std::ranges::any_of(roots, [&](const DepSourceRoot& r) { return r.root == n; }); }; // The same enumeration answers a second reader: which packages were // read from an editable tree, with their source globs (the build From 59a4f016a6186384e0ad3d1df14bd1df431e6ce9 Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 23:12:36 +0800 Subject: [PATCH 12/14] ci, round 5: a fast path that skips the xlings tarball when the cache is at the pin, and a Windows e2e step that installs mingw only when the sandbox does not already hold it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both guards keep the existing fallback, so a no-cache restore still runs the full bootstrap and a sandbox without mingw still installs it. The guards are covering measured cost, not invented one: - bootstrap-mcpp pays the xlings tarball fetch + extract on every job even when the restored cache already holds the pinned binary. Measured 5 to 30 s per job on Linux and ~30 s per job on Windows, repeated across ~30 jobs per run. The cache key is already xl$VER; the check is the one case the guard would otherwise miss — a stale xlings cache from before the pin was bumped, or a binary that no longer runs because its dynamic loader is gone. - ci-windows-e2e's three shards each call `toolchain install mingw 16.1.0`. The build job prewarms mingw on main and saves it into the sandbox cache, so every PR restores a sandbox that already holds it; the call is a no-op but still pays mcpp-toolchain startup. Measured 68 s per shard on three shards per PR. The check looks for the g++ in the two homes run_all.sh probes, and the install is gated on the missing-payload case only. The fallback paths are the same code in C++ or in \ux\. The guards do not change what a cold cache installs or what a warm cache restores; they only save the cost of the redundant calls on warm caches. --- .github/actions/bootstrap-mcpp/action.yml | 54 +++++++++++++++++++++++ .github/workflows/ci-windows-e2e.yml | 22 +++++++-- 2 files changed, 72 insertions(+), 4 deletions(-) diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index b549ccae..0ed1a875 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -132,6 +132,37 @@ runs: esac tarball="xlings-${XLINGS_VERSION}-linux-${xa}.tar.gz" ;; esac + # FAST PATH: the xlings cache already holds the pinned version, so the + # tarball fetch + extract + `self install` are skipped. Without this + # guard every job of every CI run paid the download and the extract, + # measured at 5 to 30 seconds per job on Linux and ~30 seconds on + # Windows, across thirty jobs per run — about half the bootstrap-mcpp + # step on Windows, more the 7 seconds the unix leg pays. The cache key + # is `xl$VER` already; the check is the one case this guard would + # otherwise miss: a stale `xlings` cache from BEFORE the pin was + # bumped (a partial restore-key match hands the same OS/ARCH cache + # back, but with the previous release), or a binary that no longer runs + # because its dynamic loader is gone. + XL_BIN_PATH="$HOME/.xlings/subos/default/bin/xlings" + if [ -x "$XL_BIN_PATH" ]; then + xl_ver="$("$XL_BIN_PATH" --version 2>/dev/null | head -1 || true)" + if [ -n "$xl_ver" ] && echo "$xl_ver" | grep -qF "$XLINGS_VERSION"; then + export PATH="$HOME/.xlings/subos/default/bin:$PATH" + echo "$HOME/.xlings/subos/default/bin" >> "$GITHUB_PATH" + xlings --version + MCPP=$(bash "$REPO_DIR/.github/tools/install_pinned_mcpp.sh" "$REPO_DIR") + echo "system xlings: $(xlings --version 2>/dev/null | head -1)" + if [ -x "$HOME/.mcpp/registry/bin/xlings" ]; then + echo "sandbox xlings: $("$HOME/.mcpp/registry/bin/xlings" --version 2>/dev/null | head -1)" + else + echo "sandbox xlings: (not initialised yet)" + fi + echo "MCPP=$MCPP" >> "$GITHUB_ENV" + echo "XLINGS_BIN=$XL_BIN_PATH" >> "$GITHUB_ENV" + exit 0 + fi + fi + WORK=$(mktemp -d) # Retried and verified — see .github/tools/fetch_release.sh. A bare curl # here was the single largest source of unexplained CI red on this repo @@ -202,6 +233,29 @@ runs: XLINGS_VERSION: ${{ inputs.xlings-version }} run: | REPO_DIR="$(pwd)" + # FAST PATH: see the unix leg for the reasoning. The cost on Windows is + # larger (the zip is bigger and the runner's network path to + # github.com is slower), measured at ~30 s per job. + XL_BIN_PATH="$USERPROFILE/.xlings/subos/default/bin/xlings.exe" + if [ -x "$XL_BIN_PATH" ]; then + xl_ver="$("$XL_BIN_PATH" --version 2>/dev/null | head -1 || true)" + if [ -n "$xl_ver" ] && echo "$xl_ver" | grep -qF "$XLINGS_VERSION"; then + export PATH="$USERPROFILE/.xlings/subos/default/bin:$PATH" + echo "$USERPROFILE/.xlings/subos/default/bin" >> "$GITHUB_PATH" + xlings.exe --version + MCPP=$(bash "$REPO_DIR/.github/tools/install_pinned_mcpp.sh" "$REPO_DIR") + echo "system xlings: $(xlings.exe --version 2>/dev/null | head -1)" + if [ -x "$USERPROFILE/.mcpp/registry/bin/xlings.exe" ]; then + echo "sandbox xlings: $("$USERPROFILE/.mcpp/registry/bin/xlings.exe" --version 2>/dev/null | head -1)" + else + echo "sandbox xlings: (not initialised yet)" + fi + echo "MCPP=$MCPP" >> "$GITHUB_ENV" + echo "XLINGS_BIN=$(cygpath -w "$XL_BIN_PATH")" >> "$GITHUB_ENV" + exit 0 + fi + fi + WORK=$(mktemp -d) zipfile="xlings-${XLINGS_VERSION}-windows-x86_64.zip" # Same helper as the unix leg. This is the leg that kept failing, and a diff --git a/.github/workflows/ci-windows-e2e.yml b/.github/workflows/ci-windows-e2e.yml index c7cd5482..592e5543 100644 --- a/.github/workflows/ci-windows-e2e.yml +++ b/.github/workflows/ci-windows-e2e.yml @@ -66,16 +66,30 @@ jobs: - name: "Toolchain: MinGW payload for the fat-package e2e" shell: bash run: | + # The build job's prewarm installs `mingw 16.1.0` on `main` and saves + # it into the sandbox cache, so every PR restores a sandbox that + # already holds it. The install call is then a no-op but still pays + # the mcpp-toolchain startup cost; measured 68 s on three shards per + # PR. Skip the call when the payload is already reachable, and keep + # the verification that names the missing-capability case (the whole + # reason this step exists is so a skipped shard does not read like a + # passing one). export MCPP_VENDORED_XLINGS="$XLINGS_BIN" - "$MCPP_SELF" toolchain install mingw 16.1.0 - # Verified through the SAME two locations run_all.sh probes — checking - # only one of them would let the step pass while the capability stays - # ungranted, which is the shape of a green run that tested nothing. found="" for c in "${MCPP_HOME:-$HOME/.mcpp}"/registry/data/xpkgs/xim-x-mingw-gcc/*/bin/g++.exe \ "$HOME"/.xlings/data/xpkgs/xim-x-mingw-gcc/*/bin/g++.exe; do [[ -x "$c" ]] && { found="$c"; break; } done + if [ -z "$found" ]; then + "$MCPP_SELF" toolchain install mingw 16.1.0 + for c in "${MCPP_HOME:-$HOME/.mcpp}"/registry/data/xpkgs/xim-x-mingw-gcc/*/bin/g++.exe \ + "$HOME"/.xlings/data/xpkgs/xim-x-mingw-gcc/*/bin/g++.exe; do + [[ -x "$c" ]] && { found="$c"; break; } + done + fi + # Verified through the SAME two locations run_all.sh probes — checking + # only one of them would let the step pass while the capability stays + # ungranted, which is the shape of a green run that tested nothing. test -n "$found" || { echo "FAIL: mingw installed but not where run_all.sh looks"; exit 1; } echo "mingw payload: $found" From a16e6def644c54b4030ce5a77c181f32f22bdf6e Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Fri, 2 Oct 2026 23:21:24 +0800 Subject: [PATCH 13/14] ci, round 5 follow-up: address xlings by absolute path in the bootstrap fast-path; bare `xlings.exe` failed on Git Bash for Windows because MSYS re-derives PATH from the Windows environment on every child shell and drops the mixed-separator entry the `export PATH` writes. `xlings self install` would write it back via `[Environment]::SetEnvironmentVariable`, but the fast path skips that step. `$XL_BIN_PATH` is the same path the cold path uses; install_pinned_mcpp.sh carries the same note. Found by round 5's first build-windows, failed with `xlings.exe: command not found` at line 11 (the bare call). --- .github/actions/bootstrap-mcpp/action.yml | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 0ed1a875..0b7887ab 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -149,9 +149,9 @@ runs: if [ -n "$xl_ver" ] && echo "$xl_ver" | grep -qF "$XLINGS_VERSION"; then export PATH="$HOME/.xlings/subos/default/bin:$PATH" echo "$HOME/.xlings/subos/default/bin" >> "$GITHUB_PATH" - xlings --version + "$XL_BIN_PATH" --version MCPP=$(bash "$REPO_DIR/.github/tools/install_pinned_mcpp.sh" "$REPO_DIR") - echo "system xlings: $(xlings --version 2>/dev/null | head -1)" + echo "system xlings: $("$XL_BIN_PATH" --version 2>/dev/null | head -1)" if [ -x "$HOME/.mcpp/registry/bin/xlings" ]; then echo "sandbox xlings: $("$HOME/.mcpp/registry/bin/xlings" --version 2>/dev/null | head -1)" else @@ -236,15 +236,23 @@ runs: # FAST PATH: see the unix leg for the reasoning. The cost on Windows is # larger (the zip is bigger and the runner's network path to # github.com is slower), measured at ~30 s per job. + # + # The fast path addresses xlings by ABSOLUTE PATH, not by bare + # `xlings.exe`. The cold path runs `xlings self install` first, which + # writes the dir into Windows PATH via `[Environment]::SetEnvironmentVariable`; + # without that step a bare `xlings.exe` call depends on the bash + # export PATH, which Git Bash re-derives from Windows on every child + # shell and drops the mixed-separator entry. install_pinned_mcpp.sh + # carries the same note. XL_BIN_PATH="$USERPROFILE/.xlings/subos/default/bin/xlings.exe" if [ -x "$XL_BIN_PATH" ]; then xl_ver="$("$XL_BIN_PATH" --version 2>/dev/null | head -1 || true)" if [ -n "$xl_ver" ] && echo "$xl_ver" | grep -qF "$XLINGS_VERSION"; then export PATH="$USERPROFILE/.xlings/subos/default/bin:$PATH" echo "$USERPROFILE/.xlings/subos/default/bin" >> "$GITHUB_PATH" - xlings.exe --version + "$XL_BIN_PATH" --version MCPP=$(bash "$REPO_DIR/.github/tools/install_pinned_mcpp.sh" "$REPO_DIR") - echo "system xlings: $(xlings.exe --version 2>/dev/null | head -1)" + echo "system xlings: $("$XL_BIN_PATH" --version 2>/dev/null | head -1)" if [ -x "$USERPROFILE/.mcpp/registry/bin/xlings.exe" ]; then echo "sandbox xlings: $("$USERPROFILE/.mcpp/registry/bin/xlings.exe" --version 2>/dev/null | head -1)" else From 142afee97473089b358d2ec9d9c84af0f25bf632 Mon Sep 17 00:00:00 2001 From: SPeak Agent Date: Sat, 3 Oct 2026 02:22:25 +0800 Subject: [PATCH 14/14] chore: bump version to 2026.10.3.1 (one past the engine-binding release at 2026.10.2.1) The release at 2026.10.2.1 carries the build record's engine field and the package-rooted dep-source classification. Bumping to 2026.10.3.1 so the next publish-ecosystem run writes a fresh tag and the index PR is the single canonical bump record. Not touched: - .xlings.json (still 2026.9.24.1): bumped AFTER release/ecosystem publish. - ci-fresh-install.yml MCPP_PIN: derived from wait-index, never hand-edited. - src/xlings/xlings.cppm::kXlingsVersion: 2026.9.30.1 (unchanged this iteration). - check_version_pins.sh: OK; building 2026.10.3.1, bootstrapping 2026.9.24.1. --- mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/mcpp.toml b/mcpp.toml index 9d0ebf02..687186da 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.10.2.1" +version = "2026.10.3.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 079b2dbb..0ed6c554 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.10.2.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.10.3.1"; } // namespace mcpp