Skip to content

Commit c3cbd19

Browse files
committed
0.25.0: tools-steps, a member describes a step and a backend runs it
The previous revision of this change made the mechanism CMake's: members wrote CMake script text. A step of more than one command is now a program of portable operations (run, touch, mkdir, remove, copy, rename, symlink, glob into a list, append, require) that a member builds, and mcpp.tools.steps runs it through a backend. The backend is cmake -P until the engine runs step programs itself (mcpp-community/mcpp#955); runner_spec and command are the only functions that change then. Values are taken verbatim, and a list built when the step runs follows a command's own arguments. mcpp.deps is unchanged: its tool is CMake itself. dist-apk and dist-apple describe their steps as programs; dist-apple's plan check asserts the runner, the program as an input, and the program's copy and signature. Loops in the new code index rather than iterate a string, which GCC 16.1 cannot inline in an importer.
1 parent aae8822 commit c3cbd19

17 files changed

Lines changed: 479 additions & 268 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2604,9 +2604,9 @@ jobs:
26042604
26052605
# dist-apk ON A HOST THAT IS NOT LINUX (0.24.0). Every tool the member
26062606
# runs is a native program (`aapt2`, `zipalign`) or a jar on the JDK's
2607-
# `java`, and each step of more than one command is `cmake -P` over a
2608-
# script it writes (0.25.0, `tools-script`), so no step needs a shell and
2609-
# a Windows host packs what a Linux host does. On Windows the NDK's
2607+
# `java`, and each step of more than one command is a step program
2608+
# (0.25.0, `tools-steps`), so no step needs a shell and a Windows host
2609+
# packs what a Linux host does. On Windows the NDK's
26102610
# libc++ module sources come from `xim:android-ndk-libcxx-modules`,
26112611
# which `xim:android-ndk`'s windows entry depends on. The criteria are the
26122612
# level-0 step's on the Linux job, plus a level-1 package whose dex holds

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ mcpp 2026.11.7.1 or later; cross configuration requires CMake 3.19 or later.
7575
| `rules-sycl` | `mcpp.rules.sycl` | 2026.10.1.3 | Compiles SYCL (`*.sycl`) with DPC++. | [rules](docs/rules.md#rules-sycl) |
7676
| `tools-embed` | `mcpp.tools.embed` | 2026.9.5.4 | Writes a data file into a header the program compiles in. | [tools-embed](docs/tools-embed.md) |
7777
| `tools-island` | `mcpp.tools.island` | 2026.9.7.1 | Generates the `extern "C"` boundary and the C++ module of a code island. | [tools-island](docs/tools-island.md) |
78-
| `tools-script` | `mcpp.tools.script` | 2026.10.1.3 | Runs an action whose work is more than one command as `cmake -P` over a script the member writes; implied by `deps`, `dist-apk` and `dist-apple`. | [tools-script](docs/tools-script.md) |
78+
| `tools-steps` | `mcpp.tools.steps` | 2026.10.1.3 | Describes an action whose work is more than one command as portable operations a backend runs; implied by `dist-apk` and `dist-apple`. | [tools-steps](docs/tools-steps.md) |
7979
| `dist-appimage` | `mcpp.dist.appimage` | 2026.10.1.3 | Turns the tree `mcpp pack` stages into an AppImage (Linux). | [dist](docs/dist.md#dist-appimage) |
8080
| `dist-deb` | `mcpp.dist.deb` | 2026.10.30.1 | Installs the staged tree under `/opt/<name>` with a launcher, a desktop entry and an icon, as a Debian package (Linux targets). | [dist](docs/dist.md#dist-deb-and-dist-rpm) |
8181
| `dist-rpm` | `mcpp.dist.rpm` | 2026.10.30.1 | The same installation as an RPM package (Linux targets). | [dist](docs/dist.md#dist-deb-and-dist-rpm) |

‎deps/deps.cppm‎

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -39,18 +39,16 @@ import mcpp.plugins;
3939
export import mcpp.deps.compat;
4040
// The one resolver of a member's tools (0.19.0, mcpp#755).
4141
export import mcpp.plugins.tool;
42-
// An action whose work is more than one command: `cmake -P` over a script the
43-
// member writes (0.25.0, from this unit).
44-
export import mcpp.tools.script;
4542

4643
export namespace mcpp::deps {
4744

48-
// THE `cmake` deps-cmake AND deps-archive RUN (0.19.0, mcpp#755), and the
49-
// bracket argument their scripts quote every value with: since 0.25.0 the
50-
// mechanism the members share, `mcpp.tools.script`, under the names these
51-
// members have always used.
52-
using mcpp::tools::script::cmake_spec;
53-
using mcpp::tools::script::bracket;
45+
// THE `cmake` deps-cmake AND deps-archive RUN (0.19.0, mcpp#755): the payload
46+
// both features declare, in its two layouts (`bin/` and, on macOS,
47+
// `CMake.app/Contents/bin`), found by the one resolver every member uses.
48+
inline mcpp::plugins::tool::spec cmake_spec(std::string who) {
49+
return { .who = std::move(who), .package = "cmake", .programs = {"cmake"},
50+
.bin_dirs = {"bin", "CMake.app/Contents/bin"}, .option = "options::cmake" };
51+
}
5452

5553
// Prints `message` and records it as a `mcpp::warning`, folded onto one line.
5654
// The engine discards a build program's output when it exits 0, so a note that
@@ -72,6 +70,16 @@ inline void warn(const std::string& message) {
7270
mcpp::warning(folded.c_str());
7371
}
7472

73+
// A CMake script this build program writes, for an action whose work is more
74+
// than one command: `cmake -P <script>` is one argument vector, and the script
75+
// runs the steps in order and fails on the first that fails. Every value is a
76+
// bracket argument, so a path or a `-D` value keeps its spaces and semicolons.
77+
// The script is written only when its text changes, so its time stamp -- an
78+
// input of the action -- moves only when the work does.
79+
inline std::string bracket(std::string_view text) {
80+
return "[==[" + std::string(text) + "]==]";
81+
}
82+
7583
inline bool is_windows() { return std::string_view(mcpp::target_os()) == "windows"; }
7684
inline bool is_macos() { return std::string_view(mcpp::target_os()) == "macos"; }
7785

‎dist/apk.cppm‎

Lines changed: 50 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ import std;
102102
import mcpp;
103103
import mcpp.plugins;
104104
import mcpp.plugins.tool;
105-
import mcpp.tools.script;
105+
import mcpp.tools.steps;
106106

107107
// `std::format` is header-only and used throughout; `std::println` is not --
108108
// see `rules/spirv.cppm` for the libc++-on-macOS-14 measurement that this
@@ -205,9 +205,9 @@ struct options {
205205
// `platform` is a root carrying `android.jar`; `keystore_package` above
206206
// stays the way a keystore package is named.
207207
mcpp::plugins::tool::choice build_tools, platform, jdk, bundletool_dir, kotlin, coursier;
208-
// The `cmake` that runs the steps of more than one command (0.25.0,
209-
// `mcpp.tools.script`): the `xim:cmake` payload, asked for when a package
210-
// is planned, or a program named here.
208+
// The runner of the steps of more than one command (0.25.0,
209+
// `mcpp.tools.steps`; `cmake` until mcpp#955): the `xim:cmake` payload,
210+
// asked for when a package is planned, or a program named here.
211211
mcpp::plugins::tool::choice cmake;
212212
std::string keystore_alias;
213213
// The NAME of an environment variable `apksigner` itself reads
@@ -2270,38 +2270,31 @@ inline plan plan_for(options opt = {}) {
22702270
// stands for them; d8 takes no directory, so the classes are listed when it
22712271
// runs; the linked archive gains the native libraries, the assets and every
22722272
// `classes*.dex` d8 wrote; an App Bundle's base module is the linked archive
2273-
// laid out as bundletool reads it. Each is `cmake -P` over a script this
2274-
// program writes (`mcpp.tools.script`), so no step needs a shell on any
2275-
// host, and the script and `cmake` are inputs of the step.
2276-
namespace script = mcpp::tools::script;
2273+
// laid out as bundletool reads it. Each is a step program
2274+
// (`mcpp.tools.steps`): this member describes it, and its runner runs it,
2275+
// so no step needs a shell on any host and the runner and the written
2276+
// program are inputs of the step.
2277+
namespace steps = mcpp::tools::steps;
22772278
// Resolved here, once the project's own inputs have been checked, so a
22782279
// refusal about them is not hidden behind a tool that is still to come.
2279-
const auto cmakeSpec = script::cmake_spec("mcpp.dist.apk");
2280-
const auto cmakeTool = mcpp::plugins::tool::resolve(cmakeSpec, opt.cmake);
2281-
if (cmakeTool.pending()) return refuse(p, "cmake requested", {});
2282-
if (!cmakeTool) {
2280+
const auto runnerSpec = steps::runner_spec("mcpp.dist.apk");
2281+
const auto runnerTool = mcpp::plugins::tool::resolve(runnerSpec, opt.cmake);
2282+
if (runnerTool.pending()) return refuse(p, "cmake requested", {});
2283+
if (!runnerTool) {
22832284
return refuse(p, "cmake not found",
2284-
mcpp::plugins::tool::describe_missing(cmakeSpec, cmakeTool)
2285+
mcpp::plugins::tool::describe_missing(runnerSpec, runnerTool)
22852286
+ "\n So this plan packs nothing.");
22862287
}
2287-
const std::string cmake = cmakeTool.program;
2288-
const auto scripted = [&](step& s, std::string_view name, const std::string& text) {
2289-
const auto how = script::command(cmake, outDir / "steps" / (std::string(name) + ".cmake"),
2290-
"mcpp.dist.apk", text);
2288+
const std::string runner = runnerTool.program;
2289+
const auto attach = [&](step& s, std::string_view name, const steps::program& program) {
2290+
const auto how = steps::command(runner, outDir / "steps" / std::string(name), "mcpp.dist.apk", program);
22912291
s.argv = how.argv;
22922292
s.inputs.insert(s.inputs.end(), how.inputs.begin(), how.inputs.end());
22932293
};
22942294
const auto joined = [](std::vector<std::string> head, const std::vector<std::string>& tail) {
22952295
head.insert(head.end(), tail.begin(), tail.end());
22962296
return head;
22972297
};
2298-
// A compiler whose outputs no plan can list, then the stamp that stands
2299-
// for them.
2300-
const auto run_and_stamp = [](const std::vector<std::string>& argv, const std::string& stamp) {
2301-
return script::run(argv)
2302-
+ "file(MAKE_DIRECTORY " + script::bracket(fs::path(stamp).parent_path().generic_string()) + ")\n"
2303-
+ "file(TOUCH " + script::bracket(stamp) + ")\n";
2304-
};
23052298

23062299
// ── the pipeline ────────────────────────────────────────────────────
23072300
std::vector<std::string> assembled; // every step's own output, for later inputs
@@ -2497,7 +2490,7 @@ inline plan plan_for(options opt = {}) {
24972490
kt.inputs = kotlinFiles;
24982491
for (auto const& f : javaFiles) kt.inputs.push_back(f);
24992492
for (auto const& c : classpath) kt.inputs.push_back(c);
2500-
scripted(kt, id("apk-kotlinc", "aab-kotlinc"), run_and_stamp(command, kt.output));
2493+
attach(kt, id("apk-kotlinc", "aab-kotlinc"), steps::program{}.run(command).touch(kt.output));
25012494
p.steps.push_back(kt);
25022495
kotlinStamp = kt.output;
25032496
}
@@ -2515,7 +2508,8 @@ inline plan plan_for(options opt = {}) {
25152508
javacStep.inputs = javaFiles;
25162509
for (auto const& c : classpath) javacStep.inputs.push_back(c);
25172510
if (!kotlinStamp.empty()) javacStep.inputs.push_back(kotlinStamp);
2518-
scripted(javacStep, id("apk-javac", "aab-javac"), run_and_stamp(javacCommand, javacStep.output));
2511+
attach(javacStep, id("apk-javac", "aab-javac"),
2512+
steps::program{}.run(javacCommand).touch(javacStep.output));
25192513
p.steps.push_back(javacStep);
25202514

25212515
const std::string dexDir = (outDir / "dex").string();
@@ -2537,25 +2531,17 @@ inline plan plan_for(options opt = {}) {
25372531
if (!kotlinFiles.empty()) classDirs.push_back(kotlinClassesDir);
25382532
std::vector<std::string> d8Archives = archivesToDex;
25392533
if (!kotlinFiles.empty()) d8Archives.push_back(kotlinStdlib);
2540-
std::string d8Script = "set(inputs)\n";
2541-
for (auto const& d : classDirs) {
2542-
d8Script += "file(GLOB_RECURSE found LIST_DIRECTORIES false " + script::bracket(d + "/*.class") + ")\n"
2543-
"list(APPEND inputs ${found})\n";
2544-
}
2545-
if (!d8Archives.empty()) d8Script += "list(APPEND inputs " + script::brackets(d8Archives) + ")\n";
2546-
d8Script += "if(NOT inputs)\n"
2547-
" message(FATAL_ERROR \"mcpp.dist.apk: no .class file and no archive to dex\")\n"
2548-
"endif()\n"
2549-
"file(GLOB stale " + script::bracket(dexDir + "/classes*.dex") + ")\n"
2550-
"if(stale)\n"
2551-
" file(REMOVE ${stale})\n"
2552-
"endif()\n"
2553-
+ script::run(joined(d8, { "--min-api", minSdk, "--lib", androidJar, "--output", dexDir }),
2554-
{ "inputs" });
2534+
steps::program d8Program;
2535+
for (auto const& d : classDirs) d8Program.glob("inputs", d + "/*.class", { .recursive = true });
2536+
d8Program.append("inputs", d8Archives)
2537+
.require("inputs", "mcpp.dist.apk: no .class file and no archive to dex")
2538+
.remove_glob(dexDir + "/classes*.dex")
2539+
.run(joined(d8, { "--min-api", minSdk, "--lib", androidJar, "--output", dexDir }),
2540+
{ .append_list = "inputs" });
25552541
d8Step.inputs = { javacStep.output, androidJar };
25562542
if (!kotlinStamp.empty()) { d8Step.inputs.push_back(kotlinStamp); d8Step.inputs.push_back(kotlinStdlib); }
25572543
for (auto const& a : archivesToDex) d8Step.inputs.push_back(a);
2558-
scripted(d8Step, id("apk-d8", "aab-d8"), d8Script);
2544+
attach(d8Step, id("apk-d8", "aab-d8"), d8Program);
25592545
p.steps.push_back(d8Step);
25602546
javaOutputs.push_back(d8Step.output);
25612547
}
@@ -2572,32 +2558,26 @@ inline plan plan_for(options opt = {}) {
25722558
// bundletool reads a module whose manifest is under `manifest/`, whose
25732559
// dex is under `dex/` and whose native libraries and assets are under
25742560
// `lib/` and `assets/`. This lays the linked archive out that way.
2575-
const std::string module = (outDir / "module").generic_string();
2561+
const std::string moduleDir = (outDir / "module").generic_string();
25762562
const std::string workDir = work.generic_string();
2577-
std::string moduleScript =
2578-
"file(REMOVE_RECURSE " + script::bracket(module) + ")\n"
2579-
"file(REMOVE " + script::bracket(baseModule.output) + ")\n"
2580-
"file(MAKE_DIRECTORY " + script::bracket(module + "/manifest") + ")\n"
2581-
+ script::run({ jar, "xf", link.output }, {}, module)
2582-
+ "file(RENAME " + script::bracket(module + "/AndroidManifest.xml") + " "
2583-
+ script::bracket(module + "/manifest/AndroidManifest.xml") + ")\n";
2584-
for (auto const* sub : { "lib", "assets" }) {
2585-
moduleScript += std::string("if(IS_DIRECTORY ") + script::bracket(workDir + "/" + sub) + ")\n"
2586-
" file(COPY " + script::bracket(workDir + "/" + sub) + " DESTINATION "
2587-
+ script::bracket(module) + ")\n"
2588-
"endif()\n";
2589-
}
2563+
steps::program moduleProgram;
2564+
moduleProgram.remove(moduleDir)
2565+
.remove(baseModule.output)
2566+
.mkdir(moduleDir + "/manifest")
2567+
.run({ jar, "xf", link.output }, { .working_directory = moduleDir })
2568+
.rename(moduleDir + "/AndroidManifest.xml", moduleDir + "/manifest/AndroidManifest.xml")
2569+
.copy_tree(workDir + "/lib", moduleDir + "/lib", true)
2570+
.copy_tree(workDir + "/assets", moduleDir + "/assets", true);
25902571
if (!javaOutputs.empty()) {
2591-
moduleScript += "file(GLOB dex " + script::bracket(fs::path(javaOutputs.front()).parent_path()
2592-
.generic_string() + "/classes*.dex") + ")\n"
2593-
"file(COPY ${dex} DESTINATION " + script::bracket(module + "/dex") + ")\n";
2572+
moduleProgram.copy_glob(fs::path(javaOutputs.front()).parent_path().generic_string() + "/classes*.dex",
2573+
moduleDir + "/dex");
25942574
}
2595-
moduleScript += script::run({ jar, "cfM", baseModule.output, "-C", module, "." });
2575+
moduleProgram.run({ jar, "cfM", baseModule.output, "-C", moduleDir, "." });
25962576
baseModule.inputs = { link.output };
25972577
for (auto const& f : libInputs) baseModule.inputs.push_back(f);
25982578
for (auto const& f : assetInputs) baseModule.inputs.push_back(f);
25992579
for (auto const& f : javaOutputs) baseModule.inputs.push_back(f);
2600-
scripted(baseModule, "aab-module", moduleScript);
2580+
attach(baseModule, "aab-module", moduleProgram);
26012581
p.steps.push_back(baseModule);
26022582

26032583
p.output = !opt.output.empty() ? opt.output
@@ -2646,30 +2626,25 @@ inline plan plan_for(options opt = {}) {
26462626
// `classes*.dex` d8 wrote is added: a package whose classes pass the
26472627
// 64K-method limit of one dex gets `classes2.dex` and on, and which of
26482628
// them exist is known only when d8 has run.
2649-
std::string libsScript = "file(COPY_FILE " + script::bracket(link.output) + " "
2650-
+ script::bracket(libs.output) + ")\n";
2629+
steps::program libsProgram;
2630+
libsProgram.copy_file(link.output, libs.output);
26512631
if (inPlaceLibraries) {
2652-
libsScript += script::run({ jar, "--update", "--no-compress", "--file", libs.output,
2653-
"-C", work.string(), "lib" });
2654-
libsScript += script::run({ jar, "uf", libs.output, "-C", work.string(), "assets" });
2632+
libsProgram.run({ jar, "--update", "--no-compress", "--file", libs.output, "-C", work.string(), "lib" })
2633+
.run({ jar, "uf", libs.output, "-C", work.string(), "assets" });
26552634
} else {
2656-
libsScript += script::run({ jar, "uf", libs.output, "-C", work.string(), "lib",
2657-
"-C", work.string(), "assets" });
2635+
libsProgram.run({ jar, "uf", libs.output, "-C", work.string(), "lib", "-C", work.string(), "assets" });
26582636
}
26592637
if (!javaOutputs.empty()) {
26602638
const std::string dexDir = (outDir / "dex").generic_string();
2661-
libsScript += "file(GLOB dex RELATIVE " + script::bracket(dexDir) + " "
2662-
+ script::bracket(dexDir + "/classes*.dex") + ")\n"
2663-
"if(NOT dex)\n"
2664-
" message(FATAL_ERROR \"mcpp.dist.apk: d8 wrote no classes*.dex in " + dexDir + "\")\n"
2665-
"endif()\n"
2666-
+ script::run({ jar, "uf", libs.output }, { "dex" }, dexDir);
2639+
libsProgram.glob("dex", dexDir + "/classes*.dex", { .relative_to = dexDir })
2640+
.require("dex", "mcpp.dist.apk: d8 wrote no classes*.dex in " + dexDir)
2641+
.run({ jar, "uf", libs.output }, { .working_directory = dexDir, .append_list = "dex" });
26672642
}
26682643
libs.inputs = { link.output };
26692644
for (auto const& f : libInputs) libs.inputs.push_back(f);
26702645
for (auto const& f : assetInputs) libs.inputs.push_back(f);
26712646
for (auto const& f : javaOutputs) libs.inputs.push_back(f);
2672-
scripted(libs, "apk-libs", libsScript);
2647+
attach(libs, "apk-libs", libsProgram);
26732648
p.steps.push_back(libs);
26742649

26752650
step align;

0 commit comments

Comments
 (0)