2026.10.1.1: member selection, build programs prepared once, a pack over several members, and output streams (#748, #749, #750) - #752
Merged
Conversation
A child inherits every descriptor (every inheritable handle on Windows) its parent holds, so a child started by one thread while another thread's capture pipe is open kept that pipe's write end open, and the other thread's reader waited for end of file until an unrelated child exited. Linux creates the pipes with pipe2(O_CLOEXEC). Where the platform has no such call (macOS) and on Windows (handle inheritance, _popen), the creation of the pipe, the start of the child and the parent's close of the write end form one critical section under a process-wide mutex. The registry of children that a signal terminates is serialized for its writers, so two threads no longer claim one slot, and holds 256 entries instead of 8, since one is held per running child. The Windows capture path sets a process environment variable only when its value differs, so concurrent callers passing one value stop mutating the environment block after the first. A test starts a quick and a slow child from two threads at one instant, 12 rounds, and requires the quick reader to return before the slow child exits.
…un; a repeated -p selects every member it names, --exclude removes members, and test plans a selection once per configuration group Replace workspace_fanout_members and workspace_selection with mcpp.cli.selection: a set of members in [workspace] members order, refusals before planning (unknown, ambiguous, --exclude with -p, unknown --exclude, everything excluded). -p is repeatable on build, test, run and emit build-database; run refuses a second -p naming every member. mcpp test over several members plans once per configuration group with each member's tests in member_targets, builds Phase A and the test goals once, then runs each member's tests in member order, continuing past a member whose tests, package or plan fail. A member's tests run with that member's runtime directories. The stream gains a group_build record and a build_group field; build_ms is the group's build wall time.
… once per group, the group_build record and a failing member that fails alone
… every test binary of the plan
…nce, the group_build record, and the --workspace-timeout change (workspace, testing, machine output; English and Chinese)
…an read as --workspace
…led mcpp run build exits 101 mcpp.ui decides in one place (narration_stream) which stream narrates: the status, info, finished and line functions, the progress bars, the live region, its heartbeat, the colour decision and the terminal size follow it, and it is standard error. A command's result stays on standard output: plain lines, and the two summary verbs of mcpp test (test result, workspace result). Ordering between the two streams on one terminal is kept by flushing standard output before an unframed line is written. R1: the blank line after the Running line is narration as well, written through mcpp.ui, so it is on the stream of the Running line and absent under -q; mcpp run -q 2>/dev/null is the program's standard output, byte for byte. R2: a mcpp run whose planning or build failed exits 101, Cargo's status. A program's own status and a refused spawn (125 to 127) are unchanged, and so are the statuses of build, test and pack. Narration that was printed to standard output directly (bootstrap status lines, xlings install output, verbose ninja output, toolchain install tips, mcpp new and clean confirmations, add, update and publish guidance) moves to the narration stream. The platform terminal functions take the stream they measure: cols, rows, ambiguous_wide and KeyInput. The unit tests that read narration from standard output read standard error, and a new unit test states which stream each mcpp.ui function writes to.
…am output
862 mcpp run -q 2>/dev/null is the program's standard output, byte for byte
863 a mcpp run whose build failed exits 101, a program's own status passes
through, and build and test keep theirs
864 a build's status is on standard error, and standard output is empty
865 mcpp test reports on standard output and narrates on standard error
866 both streams on one pipe keep their order, JSON standard output is clean,
and the live status row follows standard error
No existing script asserted a status line on a capture of standard output
alone: every script that names one captures both streams. 122 captured
standard output alone and asserted only the program's marker; it now asserts
that the capture is exactly the program's output.
… 2>&1 migration docs/50 and docs/09 state that narration is on standard error and results on standard output, that a failed mcpp run build exits 101 with the runner caveat, and that mcpp build | tee log becomes mcpp build 2>&1 | tee log. docs/08 states where the test report goes; SPEC-003 lists 101. The Chinese mirrors follow.
…e's programs are compiled at the same time (#748, B1, B2) B1. The bundled mcpp module and the host modules a program imports were compiled into each program's own directory, before the program's own compile, and again for every program and every invocation. They are now entries of a store, addressed by the host compiler's identity, the standard flag, the flags of the compile, the BMIs it imports, the SHA-256 of the interface file, the mcpp version (the bundled module) and the providing package (a host module). The inputs are recorded in entry.json and a hit compares them field by field. An entry lives where its text comes from: the bundled module and the host modules of index packages whose sources are in the immutable store go to the global cache, in the layout of a dependency's entry, so mcpp cache list, verify and gc treat them as they treat one; host modules of path and git dependencies and of workspace members go to <workspace>/target/.build-mcpp/host-modules/, keyed also by a digest of the package's tree, and never to the global cache. The cache mode applies. An entry is compiled in a staging directory and renamed into place, entry.json last. GCC is staged a copy of each BMI it imports in its own gcm.cache; clang is given -fmodule-file and MSVC /reference. B2. step9_member_build_programs compiles every program whose result is stale at the same time, up to the job count, and then runs the programs in the order they always ran in, each taking its compile. The compile phase applies no directive, reports nothing, records no refusal and writes no cache, so the plan is the one a serial build writes; a failed compile is reported by the program's turn, so the failure reported is the first in the order the programs run in. Threads that want one key take one lock, so a key is compiled once. build_mcpp_module and build_host_module moved out of hostprogram.cppm into mcpp.build.host_module_compile; the store is mcpp.build.host_module_store, and bmi_cache gains directDir, stage_entry and publish_staged. Existing e2e scripts that read the old location of the bundled module and of a host module (92, 180, 309, 312) read the new one. New e2e scripts 857 to 861 and a unit test of the store cover the criteria of #748; the user documentation and its Chinese mirrors describe where compiled host modules live and that programs compile concurrently.
…d a failed mcpp run build exits 101
… once per key, and compile concurrently (#748)
…ilds once, and stages each member in a tree of its own (#749) `mcpp pack` takes a repeated -p, --workspace and --exclude, through the selection `mcpp build` plans (mcpp.cli.selection). For pack, every member means every member with a program target: --workspace skips one that has none, and -p names one in a refusal. One member keeps the path it had: `mcpp pack -p X` enters X and plans X's closure alone. The pipeline is one sequence over groups of members, of which a single member is the one-element case: the plan of each configuration group, the refusals that need only the plans, one build per group, each member staged from the build of its group through its own view of the plan, and one dispatch pass per group. The refusals before any compile are a target name, more than one --target, an --output that is a file, a dispatched format no package acts for a member, and two members that would write one archive or tree. The view of one member of a plan of several is `with_member`, promoted from the static helper `mcpp test` kept in execute.cppm to an exported function of mcpp.build.prepare next to `focus_on_member`. It also exchanges the context's manifest and root, and restores everything when the call returns or throws; `run_workspace_tests` and the pack pipeline read a member through it. BuildOverrides::pack_stage_dir, pack_stage_reason, pack_strip and pack_debug_symbols_dir become one map from member to PackStage. A program is told the stage of the member it acts for: its own when it is a selected member, the one selected member whose dependency closure reaches it, and nothing when several do, so that one run of a shared package's program serves every member. The attribution (`pack_owner`, over the closures recorded while the graph was walked) is written once: prepare reads it to fill each program's environment and to expand ${mcpp.stage_dir} against the stage of the package that declared the action, and the pipeline reads it to attribute each action a provider submitted to a member. A member whose provider submitted nothing fails alone, by name, and the others continue; the status is that of the first member that failed. `--message-format json` keeps its envelope: for several members data.artifacts lists every member's artifacts, each naming its member, data.stage stays null and data.stages names one tree per member. member_request and workspace_groups move from cmd_build to mcpp.cli.selection so that pack reads them without importing cmd_build: the new import edge from cmd_publish to cmd_build made GCC 16.1 fail with an internal compiler error in `import mcpp.cli;`, the failure the prepare module's header records.
867 states #749's criteria A to D on two providers that share a member with a build program: each provider's program runs once per pass and the shared member's once in all, each member's tree and distributable are those of `mcpp pack -p <member>`, a pack after `mcpp build --workspace` compiles no object, and each program reads its own member's staged tree. 868 states criterion E and the refusals before any compile: a member without the format, a target name, two --target, an --output that is a file, a member with no program, two members that write one archive, and a package that several members reach providing the format; and that a member whose provider submitted nothing fails alone while the others are packed. 869 states what the built-in formats write and report: --workspace and a repeated -p in member order, --output <dir>, --exclude, members of two configurations, the envelope of several members and the unchanged envelope of one, and that a bare `mcpp pack` at a virtual root packs its first member with a program.
10 gains the section on a pack over several members: the selection, what each member receives, the attribution of a package to a member, and the refusals before anything is compiled. 50 gains data.stages and the member of each artifact, which are added beside the fields of one member. Both have their Chinese mirrors.
…pace root, and -p outside a workspace keeps its diagnostics The `Packed` lines of several members name each path below the directory the command was typed in (the workspace's root) instead of below the member's root, where two members' `target/dist/...` would read alike. A `-p` outside a workspace reports an unreadable manifest as that, and not as a directory that is not a workspace, as it did before it was routed through the selection.
…e a member that provides the format itself (868)
- A test's duration_ms keeps its meaning (the run of a test that ran, the build of a compile_fail); the test binary's own build time is the added field build_ms. - A member whose package fails in a group prints its own diagnostics when they differ from the group's first failure. - A GCC host-module entry no longer keeps the copies of the imported BMIs, the std module's among them, that were staged for its compile. - The concurrent compile phase of a workspace's programs is counted once, as its wall time, in the Finished breakdown, and states no warning of its own. - A test over several members names the members it compiles instead of the virtual root. - Comments and documentation that no longer described the code.
… the prepare state, and the prepare files stay within their limits plan.cpp had grown past the 2,500-line limit and step13_build_graph_actions to within a few lines of the 400-line limit of the prepare decomposition. The reason a package has no stage when several packed members reach it is now a PrepareState method beside the rest of the attribution, and the context receives the reach map whole. The refusal table of docs/10 names its column with a noun.
…one, and the pipeline header states what it does (P3) A pack over several configurations plans every group before it compiles anything. A group whose plan fails (a build program that does not compile, say) used to end the command; it now fails for its own members, named as the command named them, and the other groups are built and packed. A pack of one group still ends at once, with the status it always had. The status of the command is that of the first member, in member order, that failed.
…, builds once, and stages each member in a tree of its own (#749)
…n record, and the sandbox verification script
…861 runs where -rpath is a linker flag
…an action is for, fails a member alone in the distribution step, and reports the members in [workspace] members order
- ${mcpp.target_file:<name>} is resolved by mcpp::build::resolve_target_file:
the unit of the member the referring package acts for, and a refusal when
several members define the name and the package acts for none of them.
- With several members the distribution drive keeps going; each member's
declared products are removed before it, so a file present after it is this
pack's; a member without a staged tree whose provider reads it fails alone.
- The members are reported, and the exit status chosen, in [workspace]
members order across configurations.
- e2e 870.
…uild::refuse_unresolved_references, and step13_build_graph_actions is back under 400 lines
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements
.agents/docs/2026-09-30-member-selection-and-build-program-cost-plan.mdas 2026.10.1.1. Section 15 of the record states what was built, measured and changed during implementation.mcpp test -p a -p btests b alone, and says nothing #750). One selection function (mcpp.cli.selection) servesbuild,test,packandmcpp emit build-database.-pis repeatable and the selection is a set in[workspace] membersorder.--excluderemoves members from a whole-workspace selection. An unknown member,--excludewith-p,--workspacewith-p, and a second-ponrunare refused before planning.group_buildrecord, abuild_groupfield, and a per-testbuild_ms; no field changes meaning.mcppmodule and each host module are compiled once per key. The key covers the compiler, the flags, the imported BMIs and the interface content.pipe2(O_CLOEXEC)on Linux, and a launch section on macOS and Windows.--workspace, a repeated-pand--excludepack several members.mcpp run -qwrites exactly the program's output.mcpp runwhose build failed exits 101.2>&1migration.Closes #748
Closes #749
Closes #750
#751 stays open. The record's section 8 states why it is outside this change.
Test plan
mcpp test)..agents/docs/2026-10-01-member-selection-verify.shpasses on the host against this build. Against 2026.9.30.2 every CHANGE section fails.