From 638eaf6a22a810e68a7616c01f2205a98eac4bdf Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 16:23:34 +0900 Subject: [PATCH 01/36] design record: N43-N47 (UEFI Shell arguments, the DOS first stack, N9 and dist-web 0.20.0, a run's input files, the UEFI payload disk) and the layers of further target rows --- ...6-11-15-ecosystem-gap-supplement-design.md | 74 +++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md b/.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md index 946856d5e..3fc3f6fbb 100644 --- a/.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md +++ b/.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md @@ -57,6 +57,11 @@ mcpp、openkal、xlings/xim 的包、dirun 与 mcpp:plugins 是通用基础设 | N34 | UEFI 程序读不到任何文件:`cd` 为 `/`,`dir` 为 File Not Found | openkal-uefi 未实现 `fs`(其 README 写明 `fs` 待 SIMPLE_FILE_SYSTEM) | 实现 · openkal-uefi | | N35 | UEFI 参数含引号时切分错误 | openkal-uefi 按空格切分 load options;由 Shell 启动时,Shell 已给出 `EFI_SHELL_PARAMETERS_PROTOCOL` 的 argv | 实现 · openkal-uefi | | N42 | UEFI 参数中的空白在启动文件中未引用 | dirun 0.4.0 以空格连接 `{args}` 写入 `startup.nsh`,Shell 按空白切分(13.9) | 实现 · dirun 0.5.0 `[payload] quote`;数据 · ovmf r3 | +| N43 | UEFI 参数含连续空白时被截短(`"two words"` 到达为 `two wor`) | OVMF(edk2 9.2.4)内置 Shell 交给程序的 argv 已截短;dirun 写出的行与 openkal-uefi 的转换均已核对(13.12) | 外部 · edk2 Shell;待评审 | +| N44 | DOS 行的程序在栈超过 64 KiB 后静默结束(状态 0),或挂起 | openkal-dos 的 stub 把首个栈固定为 64 KiB,栈越界不产生故障(mcpplibs/openkal-dos#3;13.13) | 实现 · openkal-dos;待评审 | +| N45 | 工程自行命名的 emscripten 程序在 dist-web 0.20.0 下打包失败,报“no action claimed” | N9 使行的 `.js` 让位于工程命名,旧的 dist-web 按 `.js` 查找;报告不含原因(13.14) | 插件 · mcpp.plugins;平台 · plugins 版本 | +| N46 | 单次运行无法附带输入文件(脚本、数据) | 运行器协议有 `MCPP_RUNTIME_FILES`,但其条目只来自随产物发布的 `[runtime]`(13.15) | 引擎;待评审 | +| N47 | UEFI 行的程序不能写文件 | ovmf 配置以只读的 `fat:{dir}` 挂载载荷目录(13.16) | 运行器 · dirun、ovmf 包;待评审 | | N15 | UEFI 上 mimalloc 警告无安全随机源 | 未接 EFI_RNG_PROTOCOL | 实现 · openkal-uefi | | N19 | okrun 打不开含 `//` 的路径 | 打开映像前未规范化 | 实现 · openkal-image | | N12 | dirun 0.3.2 携带的 okrun 落后,仍有 `../` 错误 | dirun 自带一份 okrun 副本 | 运行器 · dirun | @@ -721,3 +726,72 @@ profile 给出一次提示。需要评审后实现,不在本轮。 - 在 openkal-platform 仓库中以 mcpp 2026.11.16.1 执行 `mcpp why deps --target x86_64-openkal`,内置平台 0.28.0 的 openkal-tools 0.6.3 与仓库所述的 0.6.4 合并失败(`'=0.6.4,=0.6.3' matches none`)。`mcpp test` 把仓库本身当作平台,不受影响。 - 待定:在平台包自身的仓库中,`why`、`build` 等命令是否也应以该包为平台(与 `mcpp test` 一致)。本轮不改。 + +### 13.12 N43:UEFI Shell 截短含连续空白的参数(外部验证工程中定位,待评审) + +- 现象:`mcpp run --target x86_64-uefi -- "two words" "x y z" "ab cdefg" "aaaaaaaaaaaa b"` 中,程序收到 `["two wor", "x y z", "ab cdef", "aaaaaaaaaaaa "]`。截短只出现在含三个以上连续空白、且长度在 10 以上的参数中,长度更短的参数完整到达。 +- 已排除: + - dirun:`startup.nsh` 中的行为 `APP.EFI "two words" …`,Shell 的 `echo` 对同一写法输出完整文本。 + - 引用写法:`"…"`、`^ ` 转义和两者混用,结果相同。 + - openkal-uefi:0.11.0 逐个转换 `EFI_SHELL_PARAMETERS_PROTOCOL` 的 `Argv`,不做切分。 +- 结论:截短发生在 edk2 Shell 为应用程序生成 `Argv` 的过程中,属于 xim 的 ovmf 9.2.4 所带的 Shell。 +- 候选: + 1. openkal-uefi 在 Shell 启动时改读映像的 `LoadOptions`,按 Shell 的规则自行切分。前提是 `LoadOptions` 保留完整的行,尚未实测。 + 2. 换用修复了该问题的 edk2 版本(需先确认上游状态)。 + 3. 在 openkal-platform 的文档中写明此限制。 + +### 13.13 N44:DOS 行的首个栈(openkal-dos#3 的第二种表现,待评审) + +- 现象:一个批处理解释器在 `i686-dos-musl` 上执行脚本,到第一个 `IF` 时以状态 0 结束,没有输出,也没有故障报告。 +- 定位: + - 同一份源码在 Linux 上,栈限制为 96 KiB 时正常运行,64 KiB 时段错误。该程序一层嵌套命令执行持有约 80 KiB 的栈缓冲区。 + - openkal-dos 0.1.0 的 `stub/stub.S` 把首个栈设为 `0x10000`,且栈与可加载段位于同一块内存中,越界写入相邻内存,而不是产生故障。 + - #3 记录的是同一原因下的挂起。 +- 候选设计: + 1. 首个栈默认 1 MiB,与 Windows 主线程一致。 + 2. 可执行文件可以声明栈大小(`PT_GNU_STACK` 的 `p_memsz`,即 `ld -z stack-size`),由 stub 读取。 + 3. 栈置于内存块低端,越界时离开描述符的界限,产生 `#SS` 或 `#PF`,经现有的故障报告输出。 +- 发布路径:openkal-dos 0.1.1,收入索引,再由 openkal-platform 0.29.1 引用。 + +### 13.14 N45:N9 与 dist-web 0.20.0 + +- 现象:工程以 `artifact_name("exe", "{name}")` 去掉宿主追加的后缀;在 mcpp 2026.11.16.1 下,emscripten 行的启动文件从 `.js` 变为 ``,wasm 文件从 `.wasm` 变为去掉扩展名后的 `.wasm`。dist-web 0.20.0 按原名查找,未提交任何产物,引擎报告“no action claimed --pkg-format 'web'”。 +- 判断:产物名的变化是 N9 的既定行为,已写入 CHANGELOG。不足有两处: + 1. 引擎的报告没有给出提供者未提交的原因(N30 的机制要求提供者陈述原因,dist-web 0.20.0 没有陈述)。 + 2. openkal-platform 0.29.0 以 plugins 0.20.0 构建,一次构建只容纳一个 plugins 版本,选用该平台行的工程因此无法单独升级 dist-web。 +- 工程侧的处理:只在非 emscripten 行上调用 `artifact_name`。 +- 候选:dist-web 在找不到启动文件时以警告陈述原因;openkal-platform 的下一个补丁版本跟进 plugins 的最新版本。 + +### 13.15 N46:单次运行的输入文件(待评审) + +- 现象:一个程序在 UEFI 与 DOS 行上运行脚本时,脚本需要放在程序旁边。现有的做法只能绕过 mcpp 直接调用 dirun,在包目录中查找 dirun、ovmf 与 qemu 的路径,并在脚本里写死它们的版本。这样做依赖存储布局,重复了平台的 runner 数据,也绕过了按需启用与运行器协议。 +- 现有机制:运行器协议的 `MCPP_RUNTIME_FILES` 已由 dirun(0.4.0 起,经 `[payload] place` 旁或文件盘)与 Android runner(adb)投递;但它的条目只来自 `[runtime] deploy` 与 `deploy_files`,二者随产物发布,不适合测试输入。 +- 候选设计: + 1. `mcpp run --file HOST[=DEST]`(可重复),只为本次运行向 `MCPP_RUNTIME_FILES` 追加条目,`DEST` 相对于程序所在目录。 + 2. 测试作用域的同类声明,供 `mcpp test` 的用例使用。 + 3. 在此之上,可由 mcpp.plugins 提供脚本测试(发现 `tests/*.` 与对应的预期输出,经目标的 runner 运行,归一化行尾与颜色转义后比对),使 `mcpp test --target ` 覆盖解释器类工程。 +- 待定:在本机直接运行时(不经过 runner),`DEST` 的落点有三种候选:只校验、复制到临时工作目录、或要求 `HOST` 与 `DEST` 是同一路径。 +- 不采用插件包装 runner 的做法:运行输入属于运行期,而插件在构建期执行;包装 runner 需要复制平台 runner 的命令行,而各 runner 的命令行不同。 + +### 13.16 N47:UEFI 载荷目录只读(待评审) + +- ovmf 配置以 `-drive …,readonly=on,file=fat:{dir}` 挂载载荷目录。QEMU 的 vvfat 写入不可靠,因此只读挂载是有意为之;代价是程序写文件失败(openkal-uefi 把 `EFI_WRITE_PROTECTED` 报为 `kal_err_permission`,这一侧正确)。 +- 候选:dirun 为载荷生成一个可写的 FAT 映像,运行后读回其中的改动,由配置文件选择(例如 `[payload] disk = "fat-image"`)。 + +### 13.17 更多目标行的分层(待评审) + +- 一个新系统的支持分三层,各层都只用数据: + - xim:sysroot 包,以及“磁盘映像加其 dirun 配置”的包。 + - 平台包(mcpp-index):行、工具链,以及 `enabled = false` 的 runner,命令形如 `dirun --profile {pkg:}/dirun.toml`。 + - 工程:只写 `[target] .platform`。 +- FreeBSD 行(`x86_64-freebsd-sysroot@14.5`、`freebsd-disk-image`)是现成的模板。 +- 候选与可行性: + +| 档 | 系统 | 依据 | +|---|---|---| +| A | OpenBSD、NetBSD | clang 支持其三元组;sysroot 取自官方安装集;可在 QEMU 中运行。待实测:lld 对 OpenBSD 版本化共享库(`libc.so.N.M`)的查找;NetBSD 下 lld 能否链接 | +| A | illumos(OmniOS、OpenIndiana) | `x86_64-pc-solaris2.11`;作为 System V R4 系统的现代后代,映像可公开分发 | +| B | Haiku、GNU/Hurd | clang 支持其三元组;Haiku 的官方工具链为 GCC,需要专用的 dirun 配置;Hurd 在 QEMU 中运行慢且不稳定 | +| C | PureDarwin、原版 System V R4 | 没有可用的 SDK 与可启动的映像;原版 SVR4 没有对应的 LLVM 目标,头文件、库与映像有版权,只能做成“自带 sysroot 与映像”的本地平台 | + +- OpenBSD 自 7.4 起只允许经 libc 发出系统调用(`pinsyscalls`),因此 OpenBSD 上的 okrun 须以原生行构建;openkal 镜像在 OpenBSD 上运行,以原生行为前提。 From 232123e528fc8fa826eaa0c4c45f133f32b01705 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 16:40:45 +0900 Subject: [PATCH 02/36] design record: ecosystem gap supplement, part 2 -- the roles of a file, a run's inputs, the manifest's writing and part 1's remaining items (for review) --- ...6-ecosystem-gap-supplement-part2-design.md | 409 ++++++++++++++++++ 1 file changed, 409 insertions(+) create mode 100644 .agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md new file mode 100644 index 000000000..ba4e9f77d --- /dev/null +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -0,0 +1,409 @@ +--- +subject: cross-platform +status: draft +--- + +# 生态缺口补充方案(第二部分):文件的角色、运行输入、清单书写与第一部分的遗留项 + +**日期:** 2026-11-16 · **版本:** v1(评审稿) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 +openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平台以 0.20.0 构建)、xlings v2026.10.26.1 · +**前文:** [第一部分](2026-11-15-ecosystem-gap-supplement-design.md)(N1–N47,13.12–13.17 为本文的来源) + +## 1. 来源与范围 + +第一部分发布后,同一外部验证工程在 mcpp 2026.11.16.1 与 openkal-platform 0.29.0 上运行,暴露了两类问题。 + +- **第一类:已有设计的缝隙。** 第一部分 13.12–13.16 记录了这一类。 +- **第二类:文件放置的语义不清。** 一个程序运行时读取的文件、随包发布的文件、只在某一次运行中使用的文件,目前没有 + 清晰的区分,书写也冗长。 + +本文给出一个统一的方案,并把每项改动划归到具体的仓库。 + +已由评审确定、本文只展开细节的项: + +| 编号 | 决定 | +|---|---| +| D1 | N44:DOS 行的首个栈由可执行文件声明大小(缺省 1 MiB),越界产生故障 | +| D2 | N46:单次运行的输入文件;不经 runner 时在临时工作目录中运行 | +| D3 | 新增 OpenBSD、NetBSD、illumos 三行,先以平台包提供 | +| D4 | N43 与 N47 都实现:openkal-uefi 读取 `LoadOptions`;dirun 提供可写的 FAT 映像 | + +本文新提出、待评审的项: + +- 文件角色的模型(第 3 节); +- `deploy` 的条目语法与 `phase`(第 4 节); +- 运行输入的书写与契约(第 5 节); +- 清单书写的简化(第 6 节); +- 运行器规范的收口(第 7 节)。 + +## 2. 评审依据 + +- **第一部分 §2 的分层**:引擎、数据、实现、运行器、插件、包、工程。本文沿用,不重复。 +- **mcpp AGENTS.md 的规则作为缺省遵守。** 偏离须写明理由,并作为评审项单列: + - 引擎只提供机制,事实写在数据里; + - 每个缺省值是数据,并有四层覆盖; + - 一个概念一种拼写,已发布的键不改名; + - 未知键必须报告; + - 做得少时不沉默。 +- **openkal SPEC 0.25.3 的相关条款:** + - 6.5:由产物的生成方式决定的可用性,在依赖解析时确定,不在运行时报告; + - 7.6:参数向量原样传递; + - 7.13:程序把第一个授予的目录视为它的启动目录。 + +## 3. 文件的角色:运行时与发布的语义区分 + +### 3.1 四种角色 + +一个文件按“谁读它、何时需要它”分为四种角色: + +| 角色 | 读取者 | 需要的时机 | 构建目录 | 运行时投递 | 打包时暂存 | 例 | +|---|---|---|---|---|---|---| +| 构建输入 | 编译器、链接器、构建程序 | 构建 | — | 否 | 否 | 资源脚本、链接脚本、`link.inputs` | +| **运行时文件** | 程序本身,相对于自身目录 | 每次运行 | 放在程序旁 | 是 | 是 | 插件、ICD 清单、数据文件、程序旁的 DLL | +| **发布文件** | 人与分发系统,程序不读 | 安装与审计 | 不放 | 否 | 是 | 许可证、声明、README、图标、手册页 | +| **运行输入** | 程序本身,相对于工作目录 | 某一次运行 | 不放 | 只在这次运行 | 否 | 测试脚本、测试数据、一次性的配置 | + +判别方法有两条: + +- 删去一个文件后,程序的行为若不变,它就不是运行时文件; +- 一个文件若不属于产物,它就不随包发布。 + +### 3.2 现状与角色的对应 + +| 机制 | 实际承担的角色 | 问题 | +|---|---|---| +| `[runtime] deploy_files` | 运行时文件(放在程序旁) | 与 `deploy` 是同一概念的两种写法 | +| `[runtime] deploy`(`from`、`to`) | 运行时文件(放在 `to` 子目录) | 每条一个文件,没有通配符和目录;`to` 只能是目录,不能改名 | +| `[pack] include`、`exclude` | 发布文件,**只限本工程** | 依赖包没有办法声明“随消费者的包发布”的文件 | +| `share/licenses//` | 发布文件,自动生成 | 只在 `self-contained` 方式下,只覆盖共享库 | +| 无 | 运行输入 | 外部工程只能绕过 mcpp 直接调用 dirun(第一部分 N46) | + +实例:mcpp-index 的 `compat.onnxruntime` 用 `[runtime] deploy` 放置 `LICENSE` 与 `ThirdPartyNotices.txt`。 +- 这两份文件是**发布文件**,被当作运行时文件处理,因而在每次构建中都被复制到 `bin/licenses/`。 +- 三个系统各写两条,`to` 都相同,一共六条。 + +### 3.3 结论 + +1. 运行时文件只需要一种拼写,并补上通配符、整目录和改名。 +2. 依赖包需要有办法声明发布文件:只在打包时暂存,不进入构建目录,也不投递给 runner。 +3. 运行输入是一个独立的概念。它的锚点是**工作目录**,而运行时文件的锚点是**程序所在目录**;锚点不同,所以 + 不能与 `deploy` 合并(第 5 节)。 + +## 4. 方案 F:运行时文件与发布文件 + +### 4.1 条目语法 + +列表中的每一项是一个字符串,或者一张表。字符串是表的简写: + +```toml +deploy = [ + "share/data.bin", # = { from = "share/data.bin" }:放在程序旁,名字不变 + { from = "share/vulkan/icd.d/", to = "vulkan/icd.d" }, # 整个目录 + { from = "res/*.json", to = "res" }, # 通配符 + { from = "tools/x.conf", name = "app.conf" }, # 改名:from 必须恰好匹配一个文件 + { from = "lib-*/LICENSE", to = "licenses/lib", phase = "pack" }, # 发布文件 +] +``` + +| 字段 | 语义 | +|---|---| +| `from` | 相对于声明它的包的根,用 `/` 分隔。可以含 `*`、`**`、`?`;以 `/` 结尾时表示整个目录,保留相对路径。匹配为空时报错,并指出该条目的序号 | +| `to` | 目的地目录,相对于程序所在目录,缺省为 `"."`。语义与现有的 `deploy.to` 相同 | +| `name` | 目的地的文件名。只在 `from` 恰好匹配一个文件时可用,否则报错 | +| `phase` | `"run"`(缺省):运行时文件,进入构建目录,投递给 runner,打包时暂存。`"pack"`:发布文件,只在打包时暂存,放在 `to` 下,不进入构建目录,也不投递 | + +**沿用的规则:** +- 绝对路径、盘符、空段、`.`、`..` 一律拒绝; +- 同一目的地有多个来源时,内容相同则合并为一份,不同则拒绝(mcpp#723); +- 构建程序的 `mcpp::deploy(from, to)` 等价于一条表项。协议增加可选的第三个参数 `name` 与 `phase`。 + +`phase` 沿用 `[runtime] requirements` 中已有的键名,含义相同,都表示“在哪个阶段生效”;只是取值增加了 `pack`。 + +### 4.2 键名(评审项) + +| 方案 | 内容 | 理由 | 代价 | +|---|---|---|---| +| **A(推荐)扩展 `deploy`** | `deploy` 是这个概念的唯一拼写,接受第 4.1 节的全部写法;`deploy_files` 作为它的字符串形式继续有效,文档写成一条规则(“`deploy_files = [x]` 即 `deploy = [x]`”),新文档只展示 `deploy` | 符合“一个概念一种拼写、已发布的键不改名”;不引入第三个名字;`deploy` 在业界(windeployqt、macdeployqt)恰好就是“把运行所需的文件放在程序旁” | “deploy”字面上不涵盖 `phase = "pack"` 的发布文件;需要文档说明 | +| B 新键 `files` | `[runtime] files` 取代两者,旧键作为别名 | 名字更中性,`phase = "pack"` 读起来也通顺 | 偏离已发布键不改名的规则;过渡期内同一概念有三种拼写;旧引擎会**忽略**新键,文件不放置也不报错,这种静默失效比方案 A 的风险更大 | + +推荐 A。这一节需要偏离规则才能成立的论据只有“名字更中性”这一条,不足以抵消三种拼写和静默失效的代价。 + +### 4.3 发布文件放在 `phase`,还是 `[pack]`(评审项) + +| 方案 | 内容 | 判断 | +|---|---|---| +| **P1(推荐)`deploy` 条目带 `phase = "pack"`** | 依赖包和工程都能写;暂存位置相对于程序目录,与运行时文件同一套规则 | 一种语法覆盖两种角色;依赖包的发布文件随它的二进制一起进入消费者的包 | +| P2 扩展 `[pack] include`,让依赖包也能贡献 | `[pack]` 现在只作用于本工程的包;改成“为消费者贡献”会改变已发布键的含义 | 违反键义稳定 | + +`[pack] include` 保持现有含义:本工程的文件,按相同的相对路径暂存,只放一份。它与 P1 互补: +- `[pack] include` 的锚点是**包的根**(例如 `README`、`share/**`); +- `phase = "pack"` 的锚点是**程序所在目录**(例如程序旁的 `licenses/`)。 + +### 4.4 改写示例 + +`compat.onnxruntime`(xim 的 Lua 数据;多行内联表要等 mcpp 支持 #929 的版本才可用): + +```lua +-- 现状:三个系统,每个两条,共六条 +deploy = { + { from = "onnxruntime-linux-x64-1.25.1/LICENSE", to = "licenses/onnxruntime" }, + { from = "onnxruntime-linux-x64-1.25.1/ThirdPartyNotices.txt", to = "licenses/onnxruntime" }, +}, +-- 改写:一条,写在三个系统共用的块中 +deploy = { { from = "onnxruntime-*/{LICENSE,ThirdPartyNotices.txt}", to = "licenses/onnxruntime", phase = "pack" } }, +``` + +通配符是否支持 `{a,b}` 形式需要决定(评审项)。不支持时写成两条,仍然比现在少四条。 + +### 4.5 兼容 + +- 现有清单的行为不变: + - `to` 的含义不变; + - 字符串形式与 `deploy_files` 相同; + - 不写 `phase` 时为 `run`,与现在相同。 +- 发布到索引的包只有声明了 `mcpp = ">=<本版本>"` 时,才能使用通配符、`name`、`phase`。原因是旧引擎会把一个含 + `*` 的 `from` 当作字面路径,然后报“文件不存在”,并不会静默。 + +## 5. 方案 I:运行输入 + +### 5.1 语义 + +运行输入是“这一次运行开始时,工作目录中应当有的文件”。它与运行时文件的区别只在锚点: + +| | 锚点 | 生命周期 | +|---|---|---| +| 运行时文件(`deploy`) | 程序所在目录 | 与产物相同 | +| 运行输入(`inputs`) | 本次运行的工作目录 | 与本次运行相同 | + +### 5.2 书写 + +```sh +mcpp run app --target x86_64-uefi --input tests/t.bat=TEST.BAT -- /c TEST.BAT +``` + +```toml +[test] +inputs = ["tests/data/", { from = "tests/t.bat", name = "TEST.BAT" }] + +[target.'cfg(os = "uefi")'.test] +inputs = ["tests/uefi/"] +``` + +- 条目语法与第 4.1 节相同,但没有 `phase`。`to` 和 `name` 相对于工作目录。 +- 命令行写作 `FROM[=TO]`,`TO` 以 `/` 结尾时表示目录,否则表示文件名。 +- 名字取 `inputs`,沿用平台 `link.inputs` 中“交给一个步骤的文件”的含义(一个概念一种拼写)。 + +### 5.3 工作目录 + +| 情形 | 工作目录 | +|---|---| +| 没有运行输入 | 不变:本机运行时为调用时的当前目录(无感升级) | +| 有运行输入,宿主的文件系统对程序可见(本机运行、qemu-user、wine、okrun) | `target///run//`;运行前复制输入,运行后删除。`mcpp test` 在测试失败时保留,并在报告中给出路径 | +| 有运行输入,程序有自己的文件系统(虚拟机、固件、设备) | runner 把输入与运行时文件一起投递到程序旁边;在这些环境中,程序所在目录就是它的工作目录 | + +引擎对两种情形**做同一件事**:准备好临时工作目录,再把输入写进运行器契约(7.1)。投递文件的 runner 照常投递, +共享宿主文件系统的 runner 从工作目录读取。因此不需要为 runner 新增“能否投递”的键(被否定的方案见第 13 节)。 + +openkal 的一致性:okrun 把启动目录作为第一个授予的目录(SPEC 7.13)。所以一个 ok 映像在临时工作目录中看到的输入, +与它在原生程序下看到的相同。 + +## 6. 清单书写的简化 + +### 6.1 多行内联表(#929) + +下一个 mcpp 版本接受跨多行、带注释、末尾带逗号的内联表(TOML 1.1)。随该版本: + +- 文档中 runner、`deploy`、依赖的示例改为多行写法; +- openkal-platform 的 runner 改为多行,在下一个要求该 mcpp 版本的平台版本中进行。现在每个 runner 都是一行超过 + 200 个字符的内联表。 + +```toml +runner = { + cmd = ["dirun", "--payload", "{}", "--profile", "{pkg:ovmf}/dirun.toml", + "--emulator", "{pkg:qemu-x86}/bin/qemu-system-x86_64", "--memory", "2048", "--"], + hosts = ["linux-x86_64", "linux-aarch64", "macos-aarch64", "windows-x86_64"], + requires = ["xim:dirun@0.6.0", "xim:qemu-x86@9.2.4-1", "xim:ovmf@9.2.4-4"], + enabled = false, +} +``` + +### 6.2 同一平台的版本重复(评审项) + +外部工程为四行写了同一个平台的同一个版本: + +```toml +[target] +i686-linux-musl.platform = "openkal-platform@0.29.0" +i686-openkal.platform = "openkal-platform@0.29.0" +x86_64-uefi.platform = "openkal-platform@0.29.0" +i686-dos-musl.platform = "openkal-platform@0.29.0" +``` + +这四行的实际意图是:“openkal-platform 用 0.29.0,而不是 mcpp 内置的 0.28.0”。 + +| 方案 | 内容 | 判断 | +|---|---|---| +| **V1(推荐)平台系列的版本** | `[platforms]` 表,`openkal-platform = "0.29.0"`:目标表或工程选中这个平台包的每一行,都用这个版本。四行变为一行;该平台以后新增的行自动沿用 | 新表需要一个新名字;它的含义是“一个系列的版本选择”(SPEC-013 §14),与 `[target] platform` 的“为某行选平台”正交,不算同一概念的第二种拼写 | +| V2 `cfg` 选择器 | `[target.'cfg(any(os = "uefi", os = "dos", arch = "x86"))'] platform = ...` | 选择器表达的是“哪些行”,而这里要表达的是“哪个版本”;选错行时也难以发现 | +| V3 保持现状 | — | 冗长,平台升级时要改多处 | + +### 6.3 索引数据的清理(不改引擎) + +mcpp-index 中仍有一些包同时写着 `library_dirs`(旧拼写,只映射到运行时搜索)和 `link_library_dirs`。按 SPEC 的 +现行规则,这些改为 `link_library_dirs` 加 `runtime_search_dirs`,在 mcpp-index 的 PR 中完成。 + +## 7. 运行器规范的收口 + +runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符、状态、启用与合并;SPEC-002 §6 规定组合方式与契约。 +本节只补三处。 + +### 7.1 文件通道(SPEC-002 §6.2) + +- `MCPP_RUN_CLOSURE` 是唯一的规范通道。它的 JSON 在 `files`(运行时文件)之外增加 `inputs`:每项有目的地 + (相对于程序所在目录;在有自己文件系统的环境中,这也就是工作目录)和来源的绝对路径。`schema` 版本加一。 +- `MCPP_RUNTIME_FILES`(TSV)改为从 `MCPP_RUN_CLOSURE` 派生的镜像,内容是 `files` 与 `inputs` 的并集。dirun 0.4.0 + 及以后的版本、Android runner 不需要修改。 +- 规范写明:`phase = "pack"` 的文件不进入这两个通道。 + +### 7.2 工作目录(SPEC-002 §6,新增一条) + +第 5.3 节的规则作为条款写入,并给出 `mcpp run` 与 `mcpp test` 各自的行为。 + +### 7.3 不改的部分 + +SPEC-013 §10 的键集合不变。runner 不新增键(见第 13 节)。 + +## 8. 已决定的实现项 + +### 8.1 N43:openkal-uefi 0.11.1 + +- **实测**:在 OVMF(edk2 9.2.4)中,由 Shell 启动的映像,其 `LoadOptions` 保留完整的命令行 + (`APP.EFI "two words" …`)。按 Shell 的规则切分(引号外的空白分隔,`"` 开关引用,`^` 使下一个字符按字面 + 读取)后,`two words`、`ab cdefg`、`q"x`、`a b` 都完整到达。Shell 交出的 `Argv` 则会截短前两个。 +- **规则**:由 Shell 启动时,以 `LoadOptions` 切分的结果作为参数;`LoadOptions` 不可用(不是文本,或为空)时, + 退回 Shell 的 `Argv`;不由 Shell 启动时,行为不变。这一规则对应 openkal SPEC 7.6:参数应当原样到达程序。 +- **测试**:openkal-uefi 的 CI 中,`startup.nsh` 增加含连续空白的参数;openkal-platform 的 UEFI 检查同样增加。 + +### 8.2 N44:openkal-dos 0.1.1 + +- **大小**:stub 读取 `PT_GNU_STACK` 的 `p_memsz`(lld 的 `-z stack-size=N`);为 0 或缺失时取 1 MiB。 + 这是由产物的生成方式决定的属性,在链接时确定(openkal SPEC 6.5)。 +- **布局**:内存块改为 `[可加载段][保护页][栈]`。保护页通过 DPMI 1.0 的 `0507h`(设置页属性)设为未提交,栈向下 + 越界时产生 `#PF`,经现有的故障报告输出。 + - 第一部分 13.13 第 3 条提出“把栈放到块的低端,越界离开描述符界限”。本节否定这一做法:程序的虚拟地址相对于 + 块基址是固定的,平坦模型下 DS 与 SS 共用同一个描述符,栈放到低端就要移动全部段的地址。 +- **退路**:DPMI 宿主不支持 `0507h` 时,在栈底写入哨兵值,程序结束时检查;被改写时以专用状态报告“栈越界”,并 + 在启动时以一行诊断说明保护页不可用(做得少时不沉默)。 +- **待实测**:CWSDPMI r7 与 DOSBox Staging 内置的 DPMI 对 `0507h` 的支持。 +- **测试**:#3 的最小复现程序在 8 KiB 栈帧下递归 64 层后正常返回;声明 64 KiB 栈时,以 `#PF` 报告。 + +### 8.3 N45:dist-web 与平台的插件版本 + +- mcpp.plugins:dist-web 找不到它要包装的启动文件时,以警告写明查找过的路径(N30 的机制)。 +- openkal-platform 0.29.1 改为以 mcpp.plugins 的最新版本构建,使选用平台行的工程可以使用新的 dist-web。 + +### 8.4 N47:dirun 0.6.0,可写的 FAT 映像 + +- 配置文件中新增 `[payload] disk`: + - `"directory"`(缺省,即现状 `fat:{dir}` 只读); + - `"fat-image"`:dirun 生成一个 FAT32 映像,写入载荷、`--file` 与运行时文件;运行结束后,把程序写下的文件读回到 + 本次运行的工作目录。 +- 这与第 5 节一致:工作目录被镜像进虚拟机的盘,再镜像回来。 +- **实现方式(评审项)**: + - **推荐**:dirun 自带最小的 FAT32 读写代码,不依赖外部工具;Windows 与 macOS 宿主上同样可用。 + - 另一种:调用 mtools,需要为三个宿主平台各提供一个包。 +- ovmf 9.2.4-4:配置文件写明 `disk = "fat-image"`。r1–r3 不变。 + +### 8.5 新行:OpenBSD、NetBSD、illumos + +| 部分 | 内容 | +|---|---| +| xim 包 | `x86_64-openbsd-sysroot`(官方安装集 base 与 comp 中的头文件、crt 与库)、`openbsd-disk-image`(带 dirun 配置);NetBSD 与 illumos(OmniOS)各一组 | +| 平台包 | 一个新仓库,三行:三元组、clang 三元组、执行系统、`family = "unix"`;工具链 llvm;runner 为 `dirun --profile {pkg:}/dirun.toml`,`enabled = false`。名字与仓库归属是评审项:推荐 `mcpplibs/unix-platform` | +| 待实测 | lld 能否按 OpenBSD 的规则查找带版本号的共享库(`libc.so.N.M`);NetBSD 下 lld 能否链接(否则用 GNU ld);illumos 的 libc 与 crt 的组成 | +| 约束 | OpenBSD 7.4 起只允许经 libc 发出系统调用(`pinsyscalls`),所以 OpenBSD 上的 okrun 以原生行构建;openkal 映像在 OpenBSD 上运行,以这一行为前提 | + +## 9. 归属 + +| 仓库 | 改动 | 版本 | +|---|---|---| +| mcpp | `deploy` 的条目语法与 `phase`(4.1–4.3);`inputs` 与 `--input`(5.2);工作目录(5.3);`MCPP_RUN_CLOSURE` 的 `inputs` 与 TSV 镜像(7.1);`[platforms]`(6.2,如采纳);SPEC-002、SPEC-013、SPEC-017 的条款;中英文文档;把多行内联表写进示例 | 下一版 | +| mcpp.plugins | dist-web 写明未提交的原因(8.3) | 补丁版本 | +| dirun | `[payload] disk = "fat-image"` 与读回(8.4) | 0.6.0 | +| openkal-uefi | 读取 `LoadOptions`(8.1) | 0.11.1 | +| openkal-dos | 栈的大小、保护页与退路(8.2) | 0.1.1 | +| 新平台包 | OpenBSD、NetBSD、illumos 三行(8.5) | 0.1.0 | +| xim-pkgindex | dirun 0.6.0;ovmf 9.2.4-4;三个系统的 sysroot 与磁盘映像 | — | +| mcpp-index | openkal-uefi 0.11.1、openkal-dos 0.1.1、openkal-platform 0.29.1、新平台包、mcpp.plugins;`compat.onnxruntime` 的改写与 `library_dirs` 的清理(等新版 mcpp 发布后进行) | — | +| openkal-platform | 0.29.1:新成员;ovmf r4 与 dirun 0.6.0;以新版 plugins 构建;UEFI 检查增加含连续空白的参数;runner 改为多行(在要求新 mcpp 的版本中) | 0.29.1 | +| 外部验证工程(fork) | 改用 `--input`,去掉对 dirun 的直接调用和对包目录的 `find`;DOS 与 UEFI 运行完整的脚本;三个新行;`[platforms]`(如采纳) | — | + +## 10. PR 划分与依赖 + +``` +openkal-uefi ─┐ +openkal-dos ──┼─> mcpp-index(成员)─> openkal-platform 0.29.1 ─> mcpp-index(平台)─┐ +dirun ─> xim-pkgindex(dirun、ovmf r4)───────────────────────┘ │ +mcpp.plugins ─> mcpp-index(plugins)─> openkal-platform 0.29.1 ├─> 外部验证工程 +mcpp(F、I、V1)─> 发布 ───────────────────────────────────────────────────────┤ +xim-pkgindex(sysroot、映像)─> 新平台包 ─> mcpp-index(新平台包)──────────────────┘ +``` + +- 每个仓库一个 PR;xim-pkgindex 与 mcpp-index 是发布渠道,按前置的就绪时间各分两批(第一部分 §7 的规则)。 +- 第一层的五个仓库(openkal-uefi、openkal-dos、dirun、mcpp.plugins、mcpp)互不依赖,可以并行;新行的 sysroot + 与磁盘映像需要先实测(8.5),也可以与第一层并行。 + +## 11. 验证 + +| 项 | 方法 | +|---|---| +| F | e2e:通配符、目录、改名、`phase = "pack"` 不进入构建目录但进入包、碰撞规则、`deploy_files` 等价;`mcpp::deploy` 的新参数 | +| I | e2e:本机运行在临时工作目录中,且不运行时目录不变;qemu-user 与 okrun 看到输入;dirun 收到输入(TSV 镜像);`mcpp test` 失败时保留目录 | +| 7.1 | 单元测试:`MCPP_RUN_CLOSURE` 的 `inputs` 与 TSV 镜像一致 | +| 8.1 | openkal-uefi 与 openkal-platform 的 CI 在 OVMF 中检查含连续空白的参数 | +| 8.2 | openkal-dos 的 CI 中运行 #3 的复现程序,覆盖缺省栈、声明的栈与越界报告 | +| 8.4 | dirun 的单元测试覆盖 FAT 的读写;在 OVMF 中写文件并读回 | +| 8.5 | 新平台包的 CI:三个系统各构建一个 C 程序与一个 C++ 程序,并在虚拟机中运行 | +| 端到端 | 外部验证工程:DOS 与 UEFI 运行完整的脚本(含写文件的部分);三个新行运行脚本;CI 中不出现包目录的路径 | + +## 12. 自我评审 + +| 角度 | 检查 | 结论 | +|---|---|---| +| 架构 | 引擎是否出现系统名或工具名 | 没有:投递由 runner 完成,工作目录规则不区分 runner;FAT 的知识只在 dirun 中 | +| 规范 | 一个概念一种拼写 | `deploy` 保持唯一(4.2 方案 A);`inputs` 与 `phase` 沿用已有的名字;`[platforms]` 是新概念,单列待评审 | +| 规范 | `inputs` 与平台 `link.inputs` 的条目字段 | 名字相同、含义相同(交给一个步骤的文件);字段不同:`link.inputs` 只需 `file`,运行输入需要 `from`、`to`、`name`,沿用 `deploy` 的条目。写成一条规则:`link.inputs` 的 `{ file = x }` 即 `{ from = x }` | +| 规范 | 已发布的键不改名、键义稳定 | `deploy_files` 与 `[pack] include` 的含义不变(4.2、4.3) | +| 规范 | 做得少时不沉默 | 通配符匹配为空时报错;`name` 用于多个匹配时报错;DOS 的保护页不可用时给出诊断 | +| openkal | 7.6 参数原样到达 | 8.1 恢复了这一点 | +| openkal | 7.13 启动目录是第一个授予的目录 | 5.3 的工作目录经 okrun 授予,映像与原生程序看到的相同 | +| openkal | 6.5 由生成方式决定的属性在链接时确定 | 栈大小由 `PT_GNU_STACK` 声明,不在运行时询问 | +| 自我修正 | 第一部分 13.13 的“栈放在低端” | 平坦模型下不可行,改为保护页(8.2) | +| 自我修正 | 第一部分 13.15 的候选 `--file` | 改名为 `--input`,与 `inputs` 一致,并与运行时文件区分锚点 | +| 兼容 | 升级后现有工程的行为 | 不变:缺省 `phase` 为 `run`;不写输入时工作目录不变;旧的 ovmf 修订与旧平台不受影响 | +| 跨平台 | Windows 宿主 | 临时工作目录与复制在三个宿主上相同;FAT 映像由 dirun 自行生成,不依赖 mtools | +| 简洁 | 新增的键 | `deploy` 增加 `name`、`phase` 两个字段;`[test] inputs` 与 `--input`;`[platforms]`(待评审)。runner 没有新增键 | +| 测试 | 每项都有自动化验证 | 见第 11 节;8.2 的 `0507h` 与 8.5 的 lld 先实测,再写成测试 | + +## 13. 被否定的方案 + +| 方案 | 否定的原因 | +|---|---| +| `[runtime] files` 取代 `deploy` 与 `deploy_files`(4.2 方案 B) | 过渡期内同一概念有三种拼写;旧引擎忽略新键,会静默失效 | +| 让依赖包通过 `[pack] include` 贡献发布文件(4.3 方案 P2) | 改变已发布键的含义 | +| 运行输入并入 `deploy`(加上 `phase = "run-input"`) | 锚点不同(工作目录与程序所在目录),合并之后,本机运行时的落点无法说清 | +| runner 新增 `files = "delivers" \| "host"` 之类的键 | 引擎对两种 runner 做同一件事即可(5.3),这个键不带来新信息 | +| 用插件包装 runner 来投递输入 | 输入属于运行期,插件在构建期执行;还需要复制平台 runner 的命令行 | +| 把栈放在块的低端,越界离开描述符界限(第一部分 13.13 的第 3 条) | 平坦模型下需要移动全部段的地址 | +| ovmf 直接改用 QEMU 的可写 vvfat(`fat:rw:`) | QEMU 文档说明它不可靠,写入可能损坏宿主目录 | +| 以 `cfg` 选择器为多行指定同一平台(6.2 方案 V2) | 选择器表达的是“哪些行”,不是“哪个版本” | + +## 14. 待评审的决定点 + +1. 文件的四种角色(3.1),以及“运行时文件”与“发布文件”的判别方法。 +2. 键名用方案 A(扩展 `deploy`)还是方案 B(新键 `files`)(4.2)。 +3. 发布文件用 `phase = "pack"`(P1)表达(4.3),以及通配符是否支持 `{a,b}`(4.4)。 +4. 运行输入的名字 `inputs` 与 `--input`,以及工作目录的三条规则(5.2、5.3)。 +5. `[platforms]` 平台系列版本(6.2 方案 V1)及其名字。 +6. dirun 的 FAT 读写自带实现,还是调用 mtools(8.4)。 +7. 新平台包的名字与仓库(8.5,推荐 `mcpplibs/unix-platform`)。 From 638470c2d9db6adbf61938ca4bcc79f13eb9d26b Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 16:54:19 +0900 Subject: [PATCH 03/36] Design record: part 2 revised after a multi-angle review (v2) --- ...6-ecosystem-gap-supplement-part2-design.md | 380 +++++++++++------- .agents/docs/README.md | 4 +- 2 files changed, 239 insertions(+), 145 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index ba4e9f77d..10444d8f1 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -3,9 +3,9 @@ subject: cross-platform status: draft --- -# 生态缺口补充方案(第二部分):文件的角色、运行输入、清单书写与第一部分的遗留项 +# 生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项 -**日期:** 2026-11-16 · **版本:** v1(评审稿) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 +**日期:** 2026-11-16 · **版本:** v2(评审稿,经第 12 节的多角度评审修订) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平台以 0.20.0 构建)、xlings v2026.10.26.1 · **前文:** [第一部分](2026-11-15-ecosystem-gap-supplement-design.md)(N1–N47,13.12–13.17 为本文的来源) @@ -31,8 +31,8 @@ openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平 本文新提出、待评审的项: - 文件角色的模型(第 3 节); -- `deploy` 的条目语法与 `phase`(第 4 节); -- 运行输入的书写与契约(第 5 节); +- `deploy` 的条目语法与许可证文件(第 4 节); +- 运行文件的书写与契约(第 5 节); - 清单书写的简化(第 6 节); - 运行器规范的收口(第 7 节)。 @@ -61,7 +61,7 @@ openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平 | 构建输入 | 编译器、链接器、构建程序 | 构建 | — | 否 | 否 | 资源脚本、链接脚本、`link.inputs` | | **运行时文件** | 程序本身,相对于自身目录 | 每次运行 | 放在程序旁 | 是 | 是 | 插件、ICD 清单、数据文件、程序旁的 DLL | | **发布文件** | 人与分发系统,程序不读 | 安装与审计 | 不放 | 否 | 是 | 许可证、声明、README、图标、手册页 | -| **运行输入** | 程序本身,相对于工作目录 | 某一次运行 | 不放 | 只在这次运行 | 否 | 测试脚本、测试数据、一次性的配置 | +| **运行文件** | 程序本身,相对于工作目录 | 某一次运行 | 不放 | 只在这次运行 | 否 | 测试脚本、测试数据、一次性的配置 | 判别方法有两条: @@ -76,7 +76,7 @@ openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平 | `[runtime] deploy`(`from`、`to`) | 运行时文件(放在 `to` 子目录) | 每条一个文件,没有通配符和目录;`to` 只能是目录,不能改名 | | `[pack] include`、`exclude` | 发布文件,**只限本工程** | 依赖包没有办法声明“随消费者的包发布”的文件 | | `share/licenses//` | 发布文件,自动生成 | 只在 `self-contained` 方式下,只覆盖共享库 | -| 无 | 运行输入 | 外部工程只能绕过 mcpp 直接调用 dirun(第一部分 N46) | +| 无 | 运行文件 | 外部工程只能绕过 mcpp 直接调用 dirun(第一部分 N46) | 实例:mcpp-index 的 `compat.onnxruntime` 用 `[runtime] deploy` 放置 `LICENSE` 与 `ThirdPartyNotices.txt`。 - 这两份文件是**发布文件**,被当作运行时文件处理,因而在每次构建中都被复制到 `bin/licenses/`。 @@ -86,126 +86,172 @@ openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平 1. 运行时文件只需要一种拼写,并补上通配符、整目录和改名。 2. 依赖包需要有办法声明发布文件:只在打包时暂存,不进入构建目录,也不投递给 runner。 -3. 运行输入是一个独立的概念。它的锚点是**工作目录**,而运行时文件的锚点是**程序所在目录**;锚点不同,所以 +3. 运行文件是一个独立的概念。它的锚点是**工作目录**,而运行时文件的锚点是**程序所在目录**;锚点不同,所以 不能与 `deploy` 合并(第 5 节)。 ## 4. 方案 F:运行时文件与发布文件 -### 4.1 条目语法 +### 4.1 `deploy` 的条目语法(运行时文件) 列表中的每一项是一个字符串,或者一张表。字符串是表的简写: ```toml +[runtime] deploy = [ - "share/data.bin", # = { from = "share/data.bin" }:放在程序旁,名字不变 - { from = "share/vulkan/icd.d/", to = "vulkan/icd.d" }, # 整个目录 - { from = "res/*.json", to = "res" }, # 通配符 - { from = "tools/x.conf", name = "app.conf" }, # 改名:from 必须恰好匹配一个文件 - { from = "lib-*/LICENSE", to = "licenses/lib", phase = "pack" }, # 发布文件 + "share/data.bin", # = { from = "share/data.bin" }:放在程序旁 + { from = "share/vulkan/icd.d/**", to = "vulkan/icd.d" }, # 一棵目录 + { from = "res/*.{json,bin}", to = "res" }, # 通配符 + { from = "tools/x.conf", name = "app.conf" }, # 改名:from 必须恰好匹配一个文件 ] ``` | 字段 | 语义 | |---|---| -| `from` | 相对于声明它的包的根,用 `/` 分隔。可以含 `*`、`**`、`?`;以 `/` 结尾时表示整个目录,保留相对路径。匹配为空时报错,并指出该条目的序号 | +| `from` | 相对于声明它的包的根,用 `/` 分隔。通配符与 `sources`、`[pack] include`、`[test] discover` 是同一种(引擎已有的 `expand_glob`:`*`、`**`、`?`、`{a,b}`),不引入第二种方言。匹配为空时报错,并指出该条目的序号 | | `to` | 目的地目录,相对于程序所在目录,缺省为 `"."`。语义与现有的 `deploy.to` 相同 | | `name` | 目的地的文件名。只在 `from` 恰好匹配一个文件时可用,否则报错 | -| `phase` | `"run"`(缺省):运行时文件,进入构建目录,投递给 runner,打包时暂存。`"pack"`:发布文件,只在打包时暂存,放在 `to` 下,不进入构建目录,也不投递 | -**沿用的规则:** -- 绝对路径、盘符、空段、`.`、`..` 一律拒绝; -- 同一目的地有多个来源时,内容相同则合并为一份,不同则拒绝(mcpp#723); -- 构建程序的 `mcpp::deploy(from, to)` 等价于一条表项。协议增加可选的第三个参数 `name` 与 `phase`。 +**规则:** +- **目的地的相对路径**:一个匹配的文件放在 `to` 下,路径取它相对于通配符的固定目录的部分。这是 SPEC-017 B1 第 3 条 + 为测试程序命名时已有的规则。字面路径的固定目录是它的上级目录,所以现有的每一条 `deploy` 的落点不变。 + - `share/vulkan/icd.d/**` 的固定目录是 `share/vulkan/icd.d`,整棵目录保持结构; + - v1 中“以 `/` 结尾表示目录”的写法因此去掉:它与 `dir/**` 是同一件事的两种拼写。 +- **沿用**:绝对路径、盘符、空段、`.`、`..` 一律拒绝;同一目的地有多个来源时,内容相同则合并为一份,不同则拒绝 + (mcpp#723)。 +- **展开的时机**:清单中的通配符在准备阶段对包内已有的文件展开。构建程序生成的文件由 `mcpp::deploy(from, to)` + 以字面路径声明;路径由构建程序自己计算。**插件协议不变**(v1 中拟加的 `name`、`phase` 参数取消)。 +- **增量与清理**:展开的结果进入构建图。一个文件不再匹配时,它在程序旁的副本被移除;否则旧文件会留在运行环境中。 + 通配符的遍历以固定目录为界(mcpp#225 已有的规则)。 -`phase` 沿用 `[runtime] requirements` 中已有的键名,含义相同,都表示“在哪个阶段生效”;只是取值增加了 `pack`。 +### 4.2 键名:扩展 `deploy`(评审项) -### 4.2 键名(评审项) +| 方案 | 内容 | 判断 | +|---|---|---| +| **A(推荐)扩展 `deploy`** | `deploy` 是这个概念的唯一拼写;`deploy_files` 作为历史上的字符串形式继续有效,文档写成一条规则(“`deploy_files = [x]` 即 `deploy = [x]`”),新文档只展示 `deploy` | 不引入第三个名字。旧引擎遇到新写法时**报错**,不静默:读源码确认,`deploy` 中的字符串报“must be a table”,条目中的未知键报“unsupported key”(`modules/manifest/src/toml.cppm`、`xpkg.cppm`) | +| B 新键 `files` | `[runtime] files` 取代两者,旧键作为别名 | 偏离“已发布的键不改名”;过渡期内同一概念有三种拼写;旧引擎对未知的 `[runtime]` 键只写一条提示后忽略,文件不会放置 | -| 方案 | 内容 | 理由 | 代价 | -|---|---|---|---| -| **A(推荐)扩展 `deploy`** | `deploy` 是这个概念的唯一拼写,接受第 4.1 节的全部写法;`deploy_files` 作为它的字符串形式继续有效,文档写成一条规则(“`deploy_files = [x]` 即 `deploy = [x]`”),新文档只展示 `deploy` | 符合“一个概念一种拼写、已发布的键不改名”;不引入第三个名字;`deploy` 在业界(windeployqt、macdeployqt)恰好就是“把运行所需的文件放在程序旁” | “deploy”字面上不涵盖 `phase = "pack"` 的发布文件;需要文档说明 | -| B 新键 `files` | `[runtime] files` 取代两者,旧键作为别名 | 名字更中性,`phase = "pack"` 读起来也通顺 | 偏离已发布键不改名的规则;过渡期内同一概念有三种拼写;旧引擎会**忽略**新键,文件不放置也不报错,这种静默失效比方案 A 的风险更大 | +规则可以在理由充分时偏离。v1 中支持 B 的唯一论据是“`deploy` 字面上不涵盖发布文件”;第 4.3 节把发布文件移出 +`deploy` 之后,这条论据不再成立,所以推荐 A。 -推荐 A。这一节需要偏离规则才能成立的论据只有“名字更中性”这一条,不足以抵消三种拼写和静默失效的代价。 +### 4.3 发布文件:`license_files`(评审项,取代 v1 的 `phase = "pack"`) -### 4.3 发布文件放在 `phase`,还是 `[pack]`(评审项) +v1 提出在 `deploy` 条目上加 `phase = "pack"`。评审否定了它: -| 方案 | 内容 | 判断 | -|---|---|---| -| **P1(推荐)`deploy` 条目带 `phase = "pack"`** | 依赖包和工程都能写;暂存位置相对于程序目录,与运行时文件同一套规则 | 一种语法覆盖两种角色;依赖包的发布文件随它的二进制一起进入消费者的包 | -| P2 扩展 `[pack] include`,让依赖包也能贡献 | `[pack]` 现在只作用于本工程的包;改成“为消费者贡献”会改变已发布键的含义 | 违反键义稳定 | +- **语义矛盾**:`[runtime]` 段陈述“产物在运行时需要什么”。`[runtime] deploy = [{ phase = "pack" }]` 的字面意思是 + “运行时放置一个运行时不需要的文件”。 +- **`phase` 不是划分**:运行时文件同样进入包,所以 `run` 与 `pack` 是包含关系,不是并列的两个阶段;它与 + `requirements` 中的 `phase`(`run`、`link`,表示在哪个阶段检查)含义不同,却用同一个名字。 +- **事实与机制**:依赖包需要随消费者的包发布的文件,实际上只有许可证与声明。“这是许可证”是一个事实,应当写在数据里; + 放在哪里、何时放,是机制,应当由引擎决定。`phase = "pack"` 加 `to = "licenses/onnxruntime"` 让每个包自己决定 + 落点,结果不会一致。 +- **现有实现的问题**(`src/pack/pack.cppm` 的 `collect_licenses`):引擎内写死七个文件名,从共享库所在目录向上找 + 三层,找不到时不报告;只在 `self-contained` 方式下进行,只覆盖共享库。这同时违反“事实写在数据里”与“做得少时 + 不沉默”。静态链接进程序的依赖同样有许可证义务,目前完全没有覆盖。 + +**方案:** + +```toml +[package] +license = "MIT" +license_files = ["LICENSE", "ThirdPartyNotices.txt"] # 通配符同 4.1;相对于包的根 +``` + +- **机制**:打包时,对程序的链接闭包与运行闭包中的每一个包(静态库、共享库、带 `deploy` 文件的包、工程自身), + 把它的 `license_files` 放到 `share/licenses/<包名>/`,角色为 `metadata`(SPEC-014 §4.5 已有的角色与目录)。 + 所有打包方式都进行,不只是 `self-contained`。 +- **描述符**:xim 的 Lua 描述符在 `mcpp` 段写同名的键,可以写在各系统共用的块中。 +- **没有声明的包**:保留现有的按文件名查找作为退路,并在打包报告中列出“未声明许可证文件”的包(做得少时不沉默)。 +- **`[pack] include` 不变**:本工程的其他发布文件(README、图标、手册页)仍由它表达,锚点是包的根。 -`[pack] include` 保持现有含义:本工程的文件,按相同的相对路径暂存,只放一份。它与 P1 互补: -- `[pack] include` 的锚点是**包的根**(例如 `README`、`share/**`); -- `phase = "pack"` 的锚点是**程序所在目录**(例如程序旁的 `licenses/`)。 +四种角色因此各有一个归属,互不重叠: + +| 角色 | 写在哪里 | 锚点 | +|---|---|---| +| 构建输入 | `[build]`、`[resources]`、平台的 `link.inputs` | — | +| 运行时文件 | `[runtime] deploy`(任何包) | 程序所在目录 | +| 发布文件 | `[package] license_files`(任何包);`[pack] include`(本工程) | 包的根 | +| 运行文件 | `mcpp run --file`、`[test] files`(第 5 节) | 本次运行的工作目录 | ### 4.4 改写示例 -`compat.onnxruntime`(xim 的 Lua 数据;多行内联表要等 mcpp 支持 #929 的版本才可用): +`compat.onnxruntime`(xim 的 Lua 数据): ```lua --- 现状:三个系统,每个两条,共六条 +-- 现状:三个系统,每个两条 deploy,共六条;每次构建都把许可证复制到 bin/licenses/ deploy = { { from = "onnxruntime-linux-x64-1.25.1/LICENSE", to = "licenses/onnxruntime" }, { from = "onnxruntime-linux-x64-1.25.1/ThirdPartyNotices.txt", to = "licenses/onnxruntime" }, }, --- 改写:一条,写在三个系统共用的块中 -deploy = { { from = "onnxruntime-*/{LICENSE,ThirdPartyNotices.txt}", to = "licenses/onnxruntime", phase = "pack" } }, +-- 改写:一行,写在三个系统共用的块中;构建目录中不再出现许可证 +license_files = { "onnxruntime-*/{LICENSE,ThirdPartyNotices.txt}" }, ``` -通配符是否支持 `{a,b}` 形式需要决定(评审项)。不支持时写成两条,仍然比现在少四条。 - ### 4.5 兼容 -- 现有清单的行为不变: - - `to` 的含义不变; - - 字符串形式与 `deploy_files` 相同; - - 不写 `phase` 时为 `run`,与现在相同。 -- 发布到索引的包只有声明了 `mcpp = ">=<本版本>"` 时,才能使用通配符、`name`、`phase`。原因是旧引擎会把一个含 - `*` 的 `from` 当作字面路径,然后报“文件不存在”,并不会静默。 +- 现有清单的行为不变:`to` 的含义不变;字符串形式与 `deploy_files` 相同;字面路径的落点不变。 +- 发布到索引的包只有声明了 `mcpp = ">=<本版本>"` 时,才能使用通配符、`name`、字符串形式与 `license_files`。 + 旧引擎对前三者报错(4.2);对 `[package]` 的未知键报告后忽略,后果只是许可证没有进入包,与现状相同。 +- 包中许可证的位置从 `bin/licenses/<名字>/` 变为 `share/licenses/<包名>/`,只发生在改写了描述符的包上。 -## 5. 方案 I:运行输入 +### 4.6 已知的边界(不在本轮) + +`to` 拒绝 `..`,所以“程序在 `bin/`,数据在 `../share//`”的布局不能用 `deploy` 表达;`[pack] include` +能把工程的文件放进包的 `share/`,但构建目录中没有对应的位置。这是既有的限制,本文只记录,不改变。 + +## 5. 方案 I:运行文件 ### 5.1 语义 -运行输入是“这一次运行开始时,工作目录中应当有的文件”。它与运行时文件的区别只在锚点: +运行文件是“这一次运行开始时,工作目录中应当有的文件”。它与运行时文件的区别只在锚点和生命周期: | | 锚点 | 生命周期 | |---|---|---| | 运行时文件(`deploy`) | 程序所在目录 | 与产物相同 | -| 运行输入(`inputs`) | 本次运行的工作目录 | 与本次运行相同 | +| 运行文件(`--file`) | 本次运行的工作目录 | 与本次运行相同 | -### 5.2 书写 +### 5.2 书写与名字(评审项) ```sh -mcpp run app --target x86_64-uefi --input tests/t.bat=TEST.BAT -- /c TEST.BAT +mcpp run app --target x86_64-uefi --file tests/t.bat=TEST.BAT -- /c TEST.BAT ``` ```toml [test] -inputs = ["tests/data/", { from = "tests/t.bat", name = "TEST.BAT" }] +files = ["tests/data/**", { from = "tests/t.bat", name = "TEST.BAT" }] [target.'cfg(os = "uefi")'.test] -inputs = ["tests/uefi/"] +files = ["tests/uefi/**"] ``` -- 条目语法与第 4.1 节相同,但没有 `phase`。`to` 和 `name` 相对于工作目录。 -- 命令行写作 `FROM[=TO]`,`TO` 以 `/` 结尾时表示目录,否则表示文件名。 -- 名字取 `inputs`,沿用平台 `link.inputs` 中“交给一个步骤的文件”的含义(一个概念一种拼写)。 +- 条目语法与第 4.1 节相同;`to` 和 `name` 相对于工作目录。 +- 命令行写作 `FROM[=TO]`,与 dirun 已发布的 `--file HOST=GUEST` 相同。 +- **名字**:v1 把评审时选定的 `--file` 改成了 `--input`、`inputs`。评审认为这一改动不成立,改回: + - 生态中这个概念已有一个发布过的拼写:dirun 的 `--file`。mcpp 的命令行用另一个词,就是同一概念的两种拼写; + - `inputs` 与平台的 `link.inputs` 同名,但后者的 `from` 是一条命令的 argv(SPEC-013 §9.2),条目语法不同。 + v1 第 12 节写的换算规则“`{ file = x }` 即 `{ from = x }`”是错的,已删除。 + - 剩余的不足:`[test] files` 可能被读成“测试的源文件”(那是 `discover`)。另一个候选是保留 `inputs`。 + 推荐 `--file` 与 `files`。 ### 5.3 工作目录 | 情形 | 工作目录 | |---|---| -| 没有运行输入 | 不变:本机运行时为调用时的当前目录(无感升级) | -| 有运行输入,宿主的文件系统对程序可见(本机运行、qemu-user、wine、okrun) | `target///run//`;运行前复制输入,运行后删除。`mcpp test` 在测试失败时保留,并在报告中给出路径 | -| 有运行输入,程序有自己的文件系统(虚拟机、固件、设备) | runner 把输入与运行时文件一起投递到程序旁边;在这些环境中,程序所在目录就是它的工作目录 | - -引擎对两种情形**做同一件事**:准备好临时工作目录,再把输入写进运行器契约(7.1)。投递文件的 runner 照常投递, -共享宿主文件系统的 runner 从工作目录读取。因此不需要为 runner 新增“能否投递”的键(被否定的方案见第 13 节)。 - -openkal 的一致性:okrun 把启动目录作为第一个授予的目录(SPEC 7.13)。所以一个 ok 映像在临时工作目录中看到的输入, -与它在原生程序下看到的相同。 +| 没有运行文件 | 不变(无感升级) | +| 有运行文件,宿主的文件系统对程序可见(本机运行、qemu-user、wine、okrun) | `target///run//`:运行前复制运行文件,运行后删除;`mcpp test` 在失败时保留,并在报告中给出路径 | +| 有运行文件,程序有自己的文件系统(虚拟机、固件、设备) | runner 把运行文件与运行时文件一起投递到程序旁边;在这些环境中,程序所在目录就是它的工作目录 | + +- **引擎对两种情形做同一件事**:备好工作目录,在其中启动 runner,并把运行文件写进运行器契约(7.1)。投递文件的 + runner 照常投递,共享宿主文件系统的 runner 从工作目录读取。runner 不需要新增键。 +- **粒度**:`mcpp run` 一次运行一个目录。`mcpp test` 每个测试程序一个目录,该程序按 `split` 分出的各个进程共用它; + 只有声明了 `files` 的工程付出复制的代价。文件系统支持时用写时复制的克隆,否则复制;不用硬链接和符号链接, + 因为程序写入时会改到源文件。 +- **与测试协议一致**:SPEC-017 B5 的 `launch.cwd` 已经是规范的一部分。工作目录在写出 `launch` 记录时备好, + 所以 `--no-run` 之后由外部启动者(包括 O9 的会话)启动的程序,看到的目录相同。 +- **用户会注意到的一点**:带 `--file` 的本机运行,工作目录不再是调用时的目录,`--` 之后的相对路径不再指向原处。 + 文档写明这一点;`-v` 输出工作目录的路径。 +- **openkal 的一致性**:okrun 把启动目录作为第一个授予的目录(SPEC 7.13)。所以一个 ok 映像在工作目录中看到的 + 文件,与原生程序看到的相同。 +- **范围**:v1 的运行文件是工程中已有的文件。构建时生成的数据经 `deploy` 到达程序旁。 ## 6. 清单书写的简化 @@ -227,25 +273,19 @@ runner = { } ``` -### 6.2 同一平台的版本重复(评审项) - -外部工程为四行写了同一个平台的同一个版本: +### 6.2 同一平台写多行(评审项:本轮不做) -```toml -[target] -i686-linux-musl.platform = "openkal-platform@0.29.0" -i686-openkal.platform = "openkal-platform@0.29.0" -x86_64-uefi.platform = "openkal-platform@0.29.0" -i686-dos-musl.platform = "openkal-platform@0.29.0" -``` +外部工程为四行写了同一个平台的同一个版本(`i686-linux-musl`、`i686-openkal`、`x86_64-uefi`、`i686-dos-musl` +各一行 `openkal-platform@0.29.0`)。v1 推荐新增 `[platforms] openkal-platform = "0.29.0"`。评审后改为不做: -这四行的实际意图是:“openkal-platform 用 0.29.0,而不是 mcpp 内置的 0.28.0”。 +- **这四行不只是选版本,也是选行。** openkal-platform 的 `targets` 还列有 `x86_64-linux-gnu`、 + `x86_64-linux-musl` 等宿主行(它的 hosted 部分)。“这个平台列出的每一行都由它提供”会把本机构建也交给平台, + 这不是四行显式写法的含义。要避免,就得在 `[platforms]` 中再列出行,于是又回到了逐行书写。 +- **它是一个新的选择机制。** SPEC-013 §3.1 的六步顺序中要插入一步;解析时要先取得所列的每一个平台包,才能知道 + 哪些行属于它。 +- **没有功能缺口。** 四行对齐后是清楚的;升级时改四处,替换一次即可。 -| 方案 | 内容 | 判断 | -|---|---|---| -| **V1(推荐)平台系列的版本** | `[platforms]` 表,`openkal-platform = "0.29.0"`:目标表或工程选中这个平台包的每一行,都用这个版本。四行变为一行;该平台以后新增的行自动沿用 | 新表需要一个新名字;它的含义是“一个系列的版本选择”(SPEC-013 §14),与 `[target] platform` 的“为某行选平台”正交,不算同一概念的第二种拼写 | -| V2 `cfg` 选择器 | `[target.'cfg(any(os = "uefi", os = "dos", arch = "x86"))'] platform = ...` | 选择器表达的是“哪些行”,而这里要表达的是“哪个版本”;选错行时也难以发现 | -| V3 保持现状 | — | 冗长,平台升级时要改多处 | +记录为独立的设计项:若以后做,前提是平台包的行在索引的元数据中可见,并且选择是按行集合而不是按整个平台。 ### 6.3 索引数据的清理(不改引擎) @@ -255,23 +295,31 @@ mcpp-index 中仍有一些包同时写着 `library_dirs`(旧拼写,只映射 ## 7. 运行器规范的收口 runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符、状态、启用与合并;SPEC-002 §6 规定组合方式与契约。 -本节只补三处。 +本节只补两处,并列出各条款的落点。 ### 7.1 文件通道(SPEC-002 §6.2) -- `MCPP_RUN_CLOSURE` 是唯一的规范通道。它的 JSON 在 `files`(运行时文件)之外增加 `inputs`:每项有目的地 - (相对于程序所在目录;在有自己文件系统的环境中,这也就是工作目录)和来源的绝对路径。`schema` 版本加一。 -- `MCPP_RUNTIME_FILES`(TSV)改为从 `MCPP_RUN_CLOSURE` 派生的镜像,内容是 `files` 与 `inputs` 的并集。dirun 0.4.0 - 及以后的版本、Android runner 不需要修改。 -- 规范写明:`phase = "pack"` 的文件不进入这两个通道。 +- `MCPP_RUN_CLOSURE` 是唯一的规范通道。它的 JSON 在 `files`(运行时文件)之外增加 `run_files`:每项有目的地 + (相对于工作目录)和来源的绝对路径。`schema` 版本加一。 +- `MCPP_RUNTIME_FILES`(TSV)改为从 `MCPP_RUN_CLOSURE` 派生的镜像,内容是两者的并集。在有自己文件系统的环境中, + 程序所在目录就是工作目录,所以并集对投递文件的 runner 是正确的;dirun 0.4.0 及以后的版本、Android runner + 不需要修改。 +- 许可证文件不进入这两个通道。 ### 7.2 工作目录(SPEC-002 §6,新增一条) -第 5.3 节的规则作为条款写入,并给出 `mcpp run` 与 `mcpp test` 各自的行为。 +第 5.3 节的规则作为条款写入:runner 在工作目录中启动;`mcpp run` 与 `mcpp test` 各自的粒度;保留与删除。 -### 7.3 不改的部分 +### 7.3 条款的落点 -SPEC-013 §10 的键集合不变。runner 不新增键(见第 13 节)。 +| 内容 | 规范 | +|---|---| +| `deploy` 的条目语法、`license_files` | SPEC-004(清单语义) | +| 许可证的暂存位置与报告 | SPEC-014(打包)§4 | +| 运行文件、工作目录、`MCPP_RUN_CLOSURE` | SPEC-002 §6 | +| `[test] files`、`launch.cwd` 的备好时机 | SPEC-017 B1、B5 | +| runner 的键 | SPEC-013 §10:不变 | +| 插件协议 | SPEC-007:不变 | ## 8. 已决定的实现项 @@ -286,8 +334,11 @@ SPEC-013 §10 的键集合不变。runner 不新增键(见第 13 节)。 ### 8.2 N44:openkal-dos 0.1.1 -- **大小**:stub 读取 `PT_GNU_STACK` 的 `p_memsz`(lld 的 `-z stack-size=N`);为 0 或缺失时取 1 MiB。 - 这是由产物的生成方式决定的属性,在链接时确定(openkal SPEC 6.5)。 +- **大小**:stub 读取 `PT_GNU_STACK` 的 `p_memsz`(lld 的 `-z stack-size=N`)。这是由产物的生成方式决定的属性, + 在链接时确定(openkal SPEC 6.5)。 +- **缺省值是数据**:1 MiB 写在 openkal-platform 的 DOS 行的链接参数中(`-z stack-size=1048576`),工程以自己的 + 链接参数覆盖。stub 只在程序头没有声明时(不经平台构建的映像)才用内置的 1 MiB。缺省值因此在行的数据中可见, + 并且有显式的覆盖方式。 - **布局**:内存块改为 `[可加载段][保护页][栈]`。保护页通过 DPMI 1.0 的 `0507h`(设置页属性)设为未提交,栈向下 越界时产生 `#PF`,经现有的故障报告输出。 - 第一部分 13.13 第 3 条提出“把栈放到块的低端,越界离开描述符界限”。本节否定这一做法:程序的虚拟地址相对于 @@ -306,37 +357,57 @@ SPEC-013 §10 的键集合不变。runner 不新增键(见第 13 节)。 - 配置文件中新增 `[payload] disk`: - `"directory"`(缺省,即现状 `fat:{dir}` 只读); - - `"fat-image"`:dirun 生成一个 FAT32 映像,写入载荷、`--file` 与运行时文件;运行结束后,把程序写下的文件读回到 - 本次运行的工作目录。 -- 这与第 5 节一致:工作目录被镜像进虚拟机的盘,再镜像回来。 -- **实现方式(评审项)**: - - **推荐**:dirun 自带最小的 FAT32 读写代码,不依赖外部工具;Windows 与 macOS 宿主上同样可用。 + - `"fat-image"`:dirun 生成一个 FAT32 映像,写入载荷、`--file` 与运行时文件,以可写的盘交给虚拟机。 +- **不做读回**(修订 v1):v1 让 dirun 在运行后把程序写下的文件读回工作目录。但工作目录在运行后即删除(5.3), + 没有读者;测试在虚拟机内自行检查结果,结论经标准输出与退出状态返回。去掉读回之后,dirun 只需要**生成**映像, + 不需要读 FAT。 +- **实现(评审项)**: + - **推荐**:dirun 自带 FAT32 映像的生成代码,含长文件名(`name` 不限于 8.3);不依赖外部工具,Windows 与 + macOS 宿主上同样可用。只写不读,规模小。 - 另一种:调用 mtools,需要为三个宿主平台各提供一个包。 +- **复用**:生成代码写成 dirun 仓库中一个独立的模块。UEFI 程序以 ESP 映像分发是一种真实的打包格式,以后的打包 + 格式提供者可以直接使用这个模块;本轮不新建包。 - ovmf 9.2.4-4:配置文件写明 `disk = "fat-image"`。r1–r3 不变。 ### 8.5 新行:OpenBSD、NetBSD、illumos -| 部分 | 内容 | +**模板(读 `builtin.toml` 与外部工程的 CI 确认)**:FreeBSD 的行只有构建,没有 runner;运行验证在 CI 的虚拟机 +动作中进行。新行按同一模板分两步,第一步不依赖磁盘映像。 + +| 步骤 | 内容 | |---|---| -| xim 包 | `x86_64-openbsd-sysroot`(官方安装集 base 与 comp 中的头文件、crt 与库)、`openbsd-disk-image`(带 dirun 配置);NetBSD 与 illumos(OmniOS)各一组 | -| 平台包 | 一个新仓库,三行:三元组、clang 三元组、执行系统、`family = "unix"`;工具链 llvm;runner 为 `dirun --profile {pkg:}/dirun.toml`,`enabled = false`。名字与仓库归属是评审项:推荐 `mcpplibs/unix-platform` | -| 待实测 | lld 能否按 OpenBSD 的规则查找带版本号的共享库(`libc.so.N.M`);NetBSD 下 lld 能否链接(否则用 GNU ld);illumos 的 libc 与 crt 的组成 | -| 约束 | OpenBSD 7.4 起只允许经 libc 发出系统调用(`pinsyscalls`),所以 OpenBSD 上的 okrun 以原生行构建;openkal 映像在 OpenBSD 上运行,以这一行为前提 | +| 第一步:构建 | xim 的 sysroot 包(官方安装集中的头文件、crt 与库);平台包声明行(三元组、clang 三元组、`family = "unix"`、加载器)、工具链 llvm 与 sysroot;工程的 CI 在对应系统的虚拟机动作中运行产物,与 FreeBSD 相同 | +| 第二步:`mcpp run` | 带 dirun 配置的磁盘映像包与 runner(`enabled = false`)。映像的体积、再分发条款与 GitCode 镜像的大小限制都还没有实测,单独立项 | + +- **包的划分与名字(评审项)**:推荐每个系统一个平台包:`openbsd-platform`、`netbsd-platform`、 + `illumos-platform`。 + - 已有的平台包按系统命名(`openkal-platform`、`djgpp-platform`); + - 平台的版本跟随 sysroot 的系统版本(OpenBSD 7.x、NetBSD 10.x),三个系统共用一个版本号没有含义; + - 一个系统的问题(见下方的待实测项)不阻塞另外两个的发布; + - v1 推荐的 `unix-platform` 名不副实:Linux、macOS、FreeBSD 同属 unix,却不在其中。 + - 代价:三个仓库各有一次发布流程。三个包都只有数据,没有源码。 +- **待实测**:lld 能否按 OpenBSD 的规则查找带版本号的共享库(`libc.so.N.M`);NetBSD 下 lld 能否链接(否则用 + GNU ld);illumos 的 libc 与 crt 的组成;三个系统的虚拟机动作是否可用。 +- **约束**:OpenBSD 7.4 起只允许经 libc 发出系统调用(`pinsyscalls`),所以 OpenBSD 上的 okrun 以原生行构建; + openkal 映像在 OpenBSD 上运行,以这一行为前提。 ## 9. 归属 | 仓库 | 改动 | 版本 | |---|---|---| -| mcpp | `deploy` 的条目语法与 `phase`(4.1–4.3);`inputs` 与 `--input`(5.2);工作目录(5.3);`MCPP_RUN_CLOSURE` 的 `inputs` 与 TSV 镜像(7.1);`[platforms]`(6.2,如采纳);SPEC-002、SPEC-013、SPEC-017 的条款;中英文文档;把多行内联表写进示例 | 下一版 | +| mcpp | `deploy` 的条目语法(4.1);`license_files` 与打包时的暂存、报告(4.3);`--file` 与 `[test] files`(5.2);工作目录(5.3);`MCPP_RUN_CLOSURE` 的 `run_files` 与 TSV 镜像(7.1);SPEC-002、SPEC-004、SPEC-014、SPEC-017 的条款;中英文文档;把多行内联表写进示例 | 下一版 | | mcpp.plugins | dist-web 写明未提交的原因(8.3) | 补丁版本 | -| dirun | `[payload] disk = "fat-image"` 与读回(8.4) | 0.6.0 | +| dirun | `[payload] disk = "fat-image"`(8.4) | 0.6.0 | | openkal-uefi | 读取 `LoadOptions`(8.1) | 0.11.1 | | openkal-dos | 栈的大小、保护页与退路(8.2) | 0.1.1 | -| 新平台包 | OpenBSD、NetBSD、illumos 三行(8.5) | 0.1.0 | -| xim-pkgindex | dirun 0.6.0;ovmf 9.2.4-4;三个系统的 sysroot 与磁盘映像 | — | -| mcpp-index | openkal-uefi 0.11.1、openkal-dos 0.1.1、openkal-platform 0.29.1、新平台包、mcpp.plugins;`compat.onnxruntime` 的改写与 `library_dirs` 的清理(等新版 mcpp 发布后进行) | — | -| openkal-platform | 0.29.1:新成员;ovmf r4 与 dirun 0.6.0;以新版 plugins 构建;UEFI 检查增加含连续空白的参数;runner 改为多行(在要求新 mcpp 的版本中) | 0.29.1 | -| 外部验证工程(fork) | 改用 `--input`,去掉对 dirun 的直接调用和对包目录的 `find`;DOS 与 UEFI 运行完整的脚本;三个新行;`[platforms]`(如采纳) | — | +| `openbsd-platform`、`netbsd-platform`、`illumos-platform` | 各自的行、工具链与 sysroot(8.5 第一步) | 0.1.0 | +| xim-pkgindex | dirun 0.6.0;ovmf 9.2.4-4;三个系统的 sysroot | — | +| mcpp-index | openkal-uefi 0.11.1、openkal-dos 0.1.1、openkal-platform 0.29.1、三个新平台包、mcpp.plugins;`compat.onnxruntime` 改用 `license_files` 与 `library_dirs` 的清理(等新版 mcpp 发布后进行) | — | +| openkal-platform | 0.29.1:新成员;ovmf r4 与 dirun 0.6.0;DOS 行的缺省栈大小(8.2);以新版 plugins 构建;UEFI 检查增加含连续空白的参数;runner 改为多行(在要求新 mcpp 的版本中) | 0.29.1 | +| 外部验证工程(fork) | 改用 `--file`,去掉对 dirun 的直接调用和对包目录的 `find`;DOS 与 UEFI 运行完整的脚本;三个新行 | — | + +通用的能力在引擎与生态包中(文件的放置、许可证的收集、工作目录、FAT 映像、sysroot、行);工程中只留下工程自己的 +事实(哪些文件是它的测试数据、它选用哪些行)。 ## 10. PR 划分与依赖 @@ -345,65 +416,86 @@ openkal-uefi ─┐ openkal-dos ──┼─> mcpp-index(成员)─> openkal-platform 0.29.1 ─> mcpp-index(平台)─┐ dirun ─> xim-pkgindex(dirun、ovmf r4)───────────────────────┘ │ mcpp.plugins ─> mcpp-index(plugins)─> openkal-platform 0.29.1 ├─> 外部验证工程 -mcpp(F、I、V1)─> 发布 ───────────────────────────────────────────────────────┤ -xim-pkgindex(sysroot、映像)─> 新平台包 ─> mcpp-index(新平台包)──────────────────┘ +mcpp(F、I)─> 发布 ────────────────────────────────────────────────────────────┤ +xim-pkgindex(sysroot)─> 三个新平台包 ─> mcpp-index(新平台包)─────────────────────┘ ``` - 每个仓库一个 PR;xim-pkgindex 与 mcpp-index 是发布渠道,按前置的就绪时间各分两批(第一部分 §7 的规则)。 - 第一层的五个仓库(openkal-uefi、openkal-dos、dirun、mcpp.plugins、mcpp)互不依赖,可以并行;新行的 sysroot - 与磁盘映像需要先实测(8.5),也可以与第一层并行。 + 需要先实测(8.5),也可以与第一层并行。 ## 11. 验证 | 项 | 方法 | |---|---| -| F | e2e:通配符、目录、改名、`phase = "pack"` 不进入构建目录但进入包、碰撞规则、`deploy_files` 等价;`mcpp::deploy` 的新参数 | -| I | e2e:本机运行在临时工作目录中,且不运行时目录不变;qemu-user 与 okrun 看到输入;dirun 收到输入(TSV 镜像);`mcpp test` 失败时保留目录 | -| 7.1 | 单元测试:`MCPP_RUN_CLOSURE` 的 `inputs` 与 TSV 镜像一致 | +| F | e2e:通配符与 `{a,b}`、目录树保持结构、改名、碰撞规则、`deploy_files` 等价、不再匹配的文件被移除;字面路径的落点与旧版相同 | +| 许可证 | e2e:静态与共享的依赖各一个,三种打包方式下都出现在 `share/licenses/<包名>/`;未声明的包出现在报告中 | +| I | e2e:本机运行在工作目录中,且没有运行文件时目录不变;qemu-user 与 okrun 看到文件;dirun 收到文件(TSV 镜像);`mcpp test` 失败时保留目录;`--no-run` 的 `launch.cwd` 指向备好的目录 | +| 7.1 | 单元测试:`MCPP_RUN_CLOSURE` 的 `run_files` 与 TSV 镜像一致 | | 8.1 | openkal-uefi 与 openkal-platform 的 CI 在 OVMF 中检查含连续空白的参数 | | 8.2 | openkal-dos 的 CI 中运行 #3 的复现程序,覆盖缺省栈、声明的栈与越界报告 | -| 8.4 | dirun 的单元测试覆盖 FAT 的读写;在 OVMF 中写文件并读回 | -| 8.5 | 新平台包的 CI:三个系统各构建一个 C 程序与一个 C++ 程序,并在虚拟机中运行 | +| 8.4 | dirun 的单元测试覆盖 FAT 映像的生成(含长文件名);在 OVMF 中写文件并在同一次运行中读出 | +| 8.5 | 新平台包的 CI:各构建一个 C 程序与一个 C++ 程序,并在虚拟机动作中运行 | | 端到端 | 外部验证工程:DOS 与 UEFI 运行完整的脚本(含写文件的部分);三个新行运行脚本;CI 中不出现包目录的路径 | -## 12. 自我评审 +## 12. 多角度评审(v2) + +本节记录对 v1 的评审。依据是实际读到的规范条款、源码与数据,不是推断。每一条给出发现与处理。 + +| # | 角度 | 发现 | 依据 | 处理 | +|---|---|---|---|---| +| R1 | 语义清晰 | `[runtime] deploy` 上的 `phase = "pack"` 与段的含义矛盾;`run` 与 `pack` 是包含关系,不是划分 | SPEC-004 §1(`[runtime]` 陈述产物);`requirements.phase` 的取值是 `run`、`link` | 去掉 `phase`;发布文件改为 `license_files`(4.3) | +| R2 | 数据驱动 | 引擎内写死许可证的文件名,按目录猜测,找不到时不报告;静态依赖没有覆盖 | `src/pack/pack.cppm` `collect_licenses` | 事实写进包的数据;引擎只负责暂存位置;未声明的包进入报告(4.3) | +| R3 | 一致、复用 | v1 为 `deploy` 另设通配符规则,并把是否支持 `{a,b}` 列为待定 | `expand_glob`(`src/modgraph/scanner.cppm`)已支持分支,`sources` 的缺省值就用它 | 复用同一实现;“以 `/` 结尾表示目录”去掉;落点规则沿用 SPEC-017 B1 第 3 条(4.1) | +| R4 | 兼容 | v1 只断言旧引擎不会静默 | 读两处读取代码:`deploy` 中的字符串与未知键都报错;`[runtime]` 的未知键报告后忽略 | 方案 A 的依据由推断变为事实(4.2) | +| R5 | 插件架构 | v1 拟为 `mcpp::deploy` 增加 `name`、`phase` 参数 | 构建程序可以自己计算路径与生成文件 | 插件协议不变(4.1) | +| R6 | 一个概念一种拼写 | v1 把 `--file` 改为 `--input`、`inputs` | dirun 已发布 `--file HOST=GUEST`;`link.inputs` 的 `from` 是 argv(SPEC-013 §9.2) | 改回 `--file`、`files`;删除错误的换算规则(5.2) | +| R7 | 规范化 | v1 没有考虑已有的 `launch.cwd` 与外部启动者 | SPEC-017 B5 | 工作目录在写出 `launch` 时备好;写明粒度(5.3) | +| R8 | 性能 | 每个测试进程一个目录时,`split = "case"` 的复制次数不受控;目录形式的 `deploy` 每次构建都复制 | SPEC-017 O1 | 每个测试程序一个目录;写时复制的克隆;`deploy` 的展开进入构建图并清理(4.1、5.3) | +| R9 | 简洁 | dirun 读回文件没有读者 | 5.3:工作目录运行后删除 | 去掉读回;dirun 只生成映像(8.4) | +| R10 | 通用架构 | `[platforms]` 会把平台列出的宿主行一并接管;它是新的选择机制 | openkal-platform `mcpp.toml` 的 `targets`;SPEC-013 §3.1 | 本轮不做,单独立项(6.2) | +| R11 | 缺省值是数据 | v1 把 DOS 的缺省栈大小只放在 stub 中 | AGENTS.md §2 | 写进平台行的链接参数,stub 的内置值只作退路(8.2) | +| R12 | 生态一致 | v1 让新行同时带磁盘映像与 runner;`unix-platform` 名不副实 | FreeBSD 的行没有 runner(`builtin.toml`),运行在 CI 的虚拟机动作中;已有平台包按系统命名 | 分两步;每个系统一个平台包(8.5) | +| R13 | 用户体验 | 带 `--file` 的本机运行会改变工作目录 | 5.3 | 文档写明;`-v` 输出路径 | +| R14 | 跨平台 | FAT 的 8.3 文件名会限制 `name` | — | 映像生成含长文件名(8.4) | +| R15 | 边界 | `to` 拒绝 `..`,`../share` 的布局不能表达 | 4.1 的沿用规则 | 记录为已知边界,不在本轮(4.6) | + +**评审后保持不变的部分:** | 角度 | 检查 | 结论 | |---|---|---| -| 架构 | 引擎是否出现系统名或工具名 | 没有:投递由 runner 完成,工作目录规则不区分 runner;FAT 的知识只在 dirun 中 | -| 规范 | 一个概念一种拼写 | `deploy` 保持唯一(4.2 方案 A);`inputs` 与 `phase` 沿用已有的名字;`[platforms]` 是新概念,单列待评审 | -| 规范 | `inputs` 与平台 `link.inputs` 的条目字段 | 名字相同、含义相同(交给一个步骤的文件);字段不同:`link.inputs` 只需 `file`,运行输入需要 `from`、`to`、`name`,沿用 `deploy` 的条目。写成一条规则:`link.inputs` 的 `{ file = x }` 即 `{ from = x }` | -| 规范 | 已发布的键不改名、键义稳定 | `deploy_files` 与 `[pack] include` 的含义不变(4.2、4.3) | -| 规范 | 做得少时不沉默 | 通配符匹配为空时报错;`name` 用于多个匹配时报错;DOS 的保护页不可用时给出诊断 | -| openkal | 7.6 参数原样到达 | 8.1 恢复了这一点 | -| openkal | 7.13 启动目录是第一个授予的目录 | 5.3 的工作目录经 okrun 授予,映像与原生程序看到的相同 | -| openkal | 6.5 由生成方式决定的属性在链接时确定 | 栈大小由 `PT_GNU_STACK` 声明,不在运行时询问 | -| 自我修正 | 第一部分 13.13 的“栈放在低端” | 平坦模型下不可行,改为保护页(8.2) | -| 自我修正 | 第一部分 13.15 的候选 `--file` | 改名为 `--input`,与 `inputs` 一致,并与运行时文件区分锚点 | -| 兼容 | 升级后现有工程的行为 | 不变:缺省 `phase` 为 `run`;不写输入时工作目录不变;旧的 ovmf 修订与旧平台不受影响 | -| 跨平台 | Windows 宿主 | 临时工作目录与复制在三个宿主上相同;FAT 映像由 dirun 自行生成,不依赖 mtools | -| 简洁 | 新增的键 | `deploy` 增加 `name`、`phase` 两个字段;`[test] inputs` 与 `--input`;`[platforms]`(待评审)。runner 没有新增键 | -| 测试 | 每项都有自动化验证 | 见第 11 节;8.2 的 `0507h` 与 8.5 的 lld 先实测,再写成测试 | +| 架构 | 引擎是否出现系统名或工具名 | 没有:投递由 runner 完成;工作目录规则不区分 runner;FAT 的知识只在 dirun 中;许可证的文件名由包声明 | +| 规范 | 已发布的键不改名、键义稳定 | `deploy`、`deploy_files`、`[pack] include` 的含义都不变 | +| 规范 | 做得少时不沉默 | 通配符匹配为空报错;`name` 用于多个匹配报错;未声明许可证的包进入报告;DOS 的保护页不可用时给出诊断 | +| openkal | 7.6、7.13、6.5 | 8.1 恢复参数原样到达;工作目录经 okrun 授予;栈大小在链接时声明 | +| 兼容 | 升级后现有工程的行为 | 不变:字面路径的落点不变;没有运行文件时工作目录不变;旧的 ovmf 修订与旧平台不受影响 | +| 简洁 | 新增的键 | `deploy` 增加 `name` 一个字段与两种写法;`license_files`;`[test] files` 与 `--file`。runner、插件协议没有新增 | ## 13. 被否定的方案 | 方案 | 否定的原因 | |---|---| -| `[runtime] files` 取代 `deploy` 与 `deploy_files`(4.2 方案 B) | 过渡期内同一概念有三种拼写;旧引擎忽略新键,会静默失效 | -| 让依赖包通过 `[pack] include` 贡献发布文件(4.3 方案 P2) | 改变已发布键的含义 | -| 运行输入并入 `deploy`(加上 `phase = "run-input"`) | 锚点不同(工作目录与程序所在目录),合并之后,本机运行时的落点无法说清 | +| `[runtime] files` 取代 `deploy` 与 `deploy_files`(4.2 方案 B) | 过渡期内同一概念有三种拼写;旧引擎忽略新键,文件不会放置 | +| `deploy` 条目的 `phase = "pack"`(v1 的 P1) | 与 `[runtime]` 的含义矛盾;落点由各包自定;不覆盖静态依赖(R1、R2) | +| 让依赖包通过 `[pack] include` 贡献发布文件(v1 的 P2) | 改变已发布键的含义 | +| 运行文件并入 `deploy` | 锚点不同(工作目录与程序所在目录),合并之后,本机运行时的落点无法说清 | +| `--input`、`inputs`(v1) | 与 dirun 的 `--file` 是同一概念的两种拼写;与 `link.inputs` 同名而语法不同(R6) | | runner 新增 `files = "delivers" \| "host"` 之类的键 | 引擎对两种 runner 做同一件事即可(5.3),这个键不带来新信息 | -| 用插件包装 runner 来投递输入 | 输入属于运行期,插件在构建期执行;还需要复制平台 runner 的命令行 | +| 用插件包装 runner 来投递运行文件 | 运行文件属于运行期,插件在构建期执行;还需要复制平台 runner 的命令行 | +| 为 `mcpp::deploy` 增加参数(v1) | 构建程序可以自己计算(R5) | +| dirun 读回程序写下的文件(v1) | 没有读者(R9) | +| `[platforms]`(v1 的 V1) | 会接管平台列出的宿主行;是新的选择机制;没有功能缺口(R10) | | 把栈放在块的低端,越界离开描述符界限(第一部分 13.13 的第 3 条) | 平坦模型下需要移动全部段的地址 | | ovmf 直接改用 QEMU 的可写 vvfat(`fat:rw:`) | QEMU 文档说明它不可靠,写入可能损坏宿主目录 | -| 以 `cfg` 选择器为多行指定同一平台(6.2 方案 V2) | 选择器表达的是“哪些行”,不是“哪个版本” | +| 一个 `unix-platform` 包含三个系统(v1) | 名不副实;版本无法跟随各自的系统版本(R12) | ## 14. 待评审的决定点 -1. 文件的四种角色(3.1),以及“运行时文件”与“发布文件”的判别方法。 -2. 键名用方案 A(扩展 `deploy`)还是方案 B(新键 `files`)(4.2)。 -3. 发布文件用 `phase = "pack"`(P1)表达(4.3),以及通配符是否支持 `{a,b}`(4.4)。 -4. 运行输入的名字 `inputs` 与 `--input`,以及工作目录的三条规则(5.2、5.3)。 -5. `[platforms]` 平台系列版本(6.2 方案 V1)及其名字。 -6. dirun 的 FAT 读写自带实现,还是调用 mtools(8.4)。 -7. 新平台包的名字与仓库(8.5,推荐 `mcpplibs/unix-platform`)。 +1. 四种角色与各自的归属(3.1、4.3 末尾的表)。 +2. `deploy` 的条目语法:字符串形式、通配符、`name`,落点规则;键名用方案 A(4.1、4.2)。 +3. 发布文件用 `[package] license_files`,引擎统一放到 `share/licenses/<包名>/`,覆盖静态依赖(4.3)。 +4. 运行文件的名字:`--file` 与 `[test] files`(推荐),或 `--input` 与 `inputs`(5.2);工作目录的规则与粒度(5.3)。 +5. `[platforms]` 本轮不做(6.2)。 +6. dirun 自带 FAT32 映像的生成,不做读回(8.4)。 +7. 新行分两步;每个系统一个平台包(8.5)。 +8. DOS 的缺省栈大小写在平台行的数据中(8.2)。 diff --git a/.agents/docs/README.md b/.agents/docs/README.md index dda05bc7b..eee2d2ec2 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -363 records. +364 records. ## By subject @@ -52,6 +52,7 @@ Records that declare one. Everything else is listed by date below. ### cross-platform +- [生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项](2026-11-16-ecosystem-gap-supplement-part2-design.md) — draft - [生态缺口补充方案:一个 C 程序走遍全部目标所暴露的通用问题](2026-11-15-ecosystem-gap-supplement-design.md) — active - [生态方案:跨平台通用基础设施(mcpp + openkal + xim),以一个 C89 工程为验证](2026-11-10-cross-platform-ecosystem-design.md) — active @@ -202,6 +203,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-11 +- [生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项](2026-11-16-ecosystem-gap-supplement-part2-design.md) — draft - [生态缺口补充方案:一个 C 程序走遍全部目标所暴露的通用问题](2026-11-15-ecosystem-gap-supplement-design.md) — active - [发布的发布面与 macOS 链接的 SDK 契约(2026-11-14 补充)](2026-11-14-release-publication-and-macos-sdk-design.md) — active - [CI 与发布的时间成本:按测量调度的测试、分片、并行发布](2026-11-14-ci-and-release-time-cost-design.md) — active From 3f432ecdf2ad054a3f08670a4e5cc9ffd7809084 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 17:25:56 +0900 Subject: [PATCH 04/36] Design record: part 2 v3, the decided points and a two-way review against the verification project --- ...6-ecosystem-gap-supplement-part2-design.md | 197 +++++++++++++----- 1 file changed, 144 insertions(+), 53 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index 10444d8f1..ba35d0cd5 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -5,7 +5,7 @@ status: draft # 生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项 -**日期:** 2026-11-16 · **版本:** v2(评审稿,经第 12 节的多角度评审修订) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 +**日期:** 2026-11-16 · **版本:** v3(第 15 节的八项已定;第 13 节为与外部验证工程的双向交叉评审) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平台以 0.20.0 构建)、xlings v2026.10.26.1 · **前文:** [第一部分](2026-11-15-ecosystem-gap-supplement-design.md)(N1–N47,13.12–13.17 为本文的来源) @@ -159,8 +159,11 @@ license_files = ["LICENSE", "ThirdPartyNotices.txt"] # 通配符同 4.1; - **机制**:打包时,对程序的链接闭包与运行闭包中的每一个包(静态库、共享库、带 `deploy` 文件的包、工程自身), 把它的 `license_files` 放到 `share/licenses/<包名>/`,角色为 `metadata`(SPEC-014 §4.5 已有的角色与目录)。 所有打包方式都进行,不只是 `self-contained`。 +- **缺省值**:不写时取包的根下的 `{LICENSE,LICENCE,COPYING,NOTICE}*`。缺省值是引擎的数据,包以 `license_files` + 覆盖;多数包因此不需要写这个键。Python 的 PEP 639(`license-files`)是同样的做法。 - **描述符**:xim 的 Lua 描述符在 `mcpp` 段写同名的键,可以写在各系统共用的块中。 -- **没有声明的包**:保留现有的按文件名查找作为退路,并在打包报告中列出“未声明许可证文件”的包(做得少时不沉默)。 +- **没有匹配的包**:缺省值与声明都没有匹配到文件时,打包报告列出这个包(做得少时不沉默)。现有的 + `collect_licenses` 由这一机制取代。 - **`[pack] include` 不变**:本工程的其他发布文件(README、图标、手册页)仍由它表达,锚点是包的根。 四种角色因此各有一个归属,互不重叠: @@ -209,7 +212,7 @@ license_files = { "onnxruntime-*/{LICENSE,ThirdPartyNotices.txt}" }, | 运行时文件(`deploy`) | 程序所在目录 | 与产物相同 | | 运行文件(`--file`) | 本次运行的工作目录 | 与本次运行相同 | -### 5.2 书写与名字(评审项) +### 5.2 书写与名字(已定) ```sh mcpp run app --target x86_64-uefi --file tests/t.bat=TEST.BAT -- /c TEST.BAT @@ -229,8 +232,7 @@ files = ["tests/uefi/**"] - 生态中这个概念已有一个发布过的拼写:dirun 的 `--file`。mcpp 的命令行用另一个词,就是同一概念的两种拼写; - `inputs` 与平台的 `link.inputs` 同名,但后者的 `from` 是一条命令的 argv(SPEC-013 §9.2),条目语法不同。 v1 第 12 节写的换算规则“`{ file = x }` 即 `{ from = x }`”是错的,已删除。 - - 剩余的不足:`[test] files` 可能被读成“测试的源文件”(那是 `discover`)。另一个候选是保留 `inputs`。 - 推荐 `--file` 与 `files`。 + - 剩余的不足:`[test] files` 可能被读成“测试的源文件”(那是 `discover`);文档在该键处写明。 ### 5.3 工作目录 @@ -273,19 +275,34 @@ runner = { } ``` -### 6.2 同一平台写多行(评审项:本轮不做) +### 6.2 平台列由索引补充(已定,取代 v1 的 `[platforms]`) -外部工程为四行写了同一个平台的同一个版本(`i686-linux-musl`、`i686-openkal`、`x86_64-uefi`、`i686-dos-musl` -各一行 `openkal-platform@0.29.0`)。v1 推荐新增 `[platforms] openkal-platform = "0.29.0"`。评审后改为不做: +外部工程为四行写了同一个平台(`i686-linux-musl`、`i686-openkal`、`x86_64-uefi`、`i686-dos-musl` 各一行 +`openkal-platform@0.29.0`)。原因不在版本,而在于引擎不认识这些行:平台选择顺序的第 5 步“目标表的平台列” +(SPEC-013 §3.1)只有引擎内置的数据。 -- **这四行不只是选版本,也是选行。** openkal-platform 的 `targets` 还列有 `x86_64-linux-gnu`、 - `x86_64-linux-musl` 等宿主行(它的 hosted 部分)。“这个平台列出的每一行都由它提供”会把本机构建也交给平台, - 这不是四行显式写法的含义。要避免,就得在 `[platforms]` 中再列出行,于是又回到了逐行书写。 -- **它是一个新的选择机制。** SPEC-013 §3.1 的六步顺序中要插入一步;解析时要先取得所列的每一个平台包,才能知道 - 哪些行属于它。 -- **没有功能缺口。** 四行对齐后是清楚的;升级时改四处,替换一次即可。 +**方案:这一列增加一个数据来源,即索引。** 清单不新增键。 -记录为独立的设计项:若以后做,前提是平台包的行在索引的元数据中可见,并且选择是按行集合而不是按整个平台。 +| 需要的东西 | 复用的语义 | +|---|---| +| 哪一行由哪个平台包提供 | 目标表的平台列(SPEC-013 §3.1 第 5 步) | +| 取哪个版本 | 平台系列的规则:与当前引擎相容的最新版本,写入 `mcpp.lock`(SPEC-013 §14) | +| 升级 | `mcpp update <平台包>` | +| 工程固定或更换版本 | 既有的 `[target.<选择器>] platform`,顺序在前,不变 | +| 索引中包一级的信息 | 条目中已有的目录字段(`licenses`、`repo`),`mcpp search` 已在读取 | + +**规则:** +- 平台包的索引条目列出它提供的行。这份列表由索引的工具从平台包自己的清单生成,不手写;索引的 CI 检查同一行 + 只有一个平台包认领,并与包的清单一致。 +- 第 5 步先查内置表,内置表没有答案时查索引。内置表有答案的行不变,所以现有工程的行为不变。 +- 选择来源报告为 `index`。 +- 一处行为变化:内置表中“计划中”的行(如 `i686-linux-musl`)在索引中有平台包认领后可用。这是从拒绝变为可用。 +- 效果:`mcpp build --target x86_64-uefi` 不需要工程写任何一行;外部工程的四行变为零行。 + +**v1 的 `[platforms]` 表被否定**:openkal-platform 的 `targets` 还列有 `x86_64-linux-gnu` 等宿主行,按平台整体 +选择会把本机构建也交给平台;它又是一个新的清单概念。按行查找的平台列没有这两个问题。 + +**待实测**:早于本版本的引擎读到条目中的新字段时是否忽略;索引同步之后查找的开销。 ### 6.3 索引数据的清理(不改引擎) @@ -336,9 +353,10 @@ runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符 - **大小**:stub 读取 `PT_GNU_STACK` 的 `p_memsz`(lld 的 `-z stack-size=N`)。这是由产物的生成方式决定的属性, 在链接时确定(openkal SPEC 6.5)。 -- **缺省值是数据**:1 MiB 写在 openkal-platform 的 DOS 行的链接参数中(`-z stack-size=1048576`),工程以自己的 +- **缺省值是数据(已定)**:1 MiB 写在 openkal-platform 的 DOS 行的链接参数中(`-z stack-size=1048576`),工程以自己的 链接参数覆盖。stub 只在程序头没有声明时(不经平台构建的映像)才用内置的 1 MiB。缺省值因此在行的数据中可见, - 并且有显式的覆盖方式。 + 并且有显式的覆盖方式。可执行文件声明自己的栈大小、由工具链给出缺省值,是没有操作系统代为决定的环境中的通行 + 做法(PE 的 `/STACK`,缺省 1 MiB;WebAssembly 的 `-z stack-size`)。通用键 `stack_size` 本轮不加(13.1 X6)。 - **布局**:内存块改为 `[可加载段][保护页][栈]`。保护页通过 DPMI 1.0 的 `0507h`(设置页属性)设为未提交,栈向下 越界时产生 `#PF`,经现有的故障报告输出。 - 第一部分 13.13 第 3 条提出“把栈放到块的低端,越界离开描述符界限”。本节否定这一做法:程序的虚拟地址相对于 @@ -369,25 +387,36 @@ runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符 格式提供者可以直接使用这个模块;本轮不新建包。 - ovmf 9.2.4-4:配置文件写明 `disk = "fat-image"`。r1–r3 不变。 -### 8.5 新行:OpenBSD、NetBSD、illumos +### 8.5 新行与平台扩展包 + +**分层(已定)**:常用的系统内置在 mcpp 中;小众与早期的系统放在一个官方的扩展包中。两处使用同一种数据格式 +(`builtin.toml` 的文件头写明它使用平台包的格式),所以一行成熟之后进入内置表只是数据的移动。 + +**包(已定)**:仓库 `mcpp-community/mcpp-platforms`,包 `mcpp.platforms`,与 `mcpp-community/mcpp-plugins` +(包 `mcpp.plugins`)对称。一个包,多行:每个系统各有一组 `[[platform.define]]` 与 `[[platform.provider]]`,带各自的 +工具链、sysroot、链接约定与 runner。openkal-platform 已经以这种方式提供 UEFI、DOS 与 i686 的行。 -**模板(读 `builtin.toml` 与外部工程的 CI 确认)**:FreeBSD 的行只有构建,没有 runner;运行验证在 CI 的虚拟机 -动作中进行。新行按同一模板分两步,第一步不依赖磁盘映像。 +| 角度 | 判断 | +|---|---| +| 使用 | `mcpp build --target x86_64-openbsd`;配合 6.2,工程不写平台 | +| 架构 | 引擎中不出现系统名;SPEC-013 不变 | +| 开发 | 加入一个系统 = 一段行的数据、一个 sysroot 包、一个磁盘映像包、一项 CI 检查 | +| 发布 | 一条发布链;有问题的行标为 `preview` 或暂不收录,不阻塞其余的行 | +| 限度 | 所有行在一个清单中;行数到十几个时再考虑分包 | +| 待确认 | 打包目录的命名规则读取以 `-platform` 结尾的包名(SPEC-014 §4.2);该规则只用于带 runner 的映像平台,实现时确认新包不受影响 | + +**本轮的三个系统**:OpenBSD、NetBSD、illumos。构建与运行在同一个 PR 中完成,以提交作为阶段点,不拆到以后: -| 步骤 | 内容 | +| 阶段 | 内容 | |---|---| -| 第一步:构建 | xim 的 sysroot 包(官方安装集中的头文件、crt 与库);平台包声明行(三元组、clang 三元组、`family = "unix"`、加载器)、工具链 llvm 与 sysroot;工程的 CI 在对应系统的虚拟机动作中运行产物,与 FreeBSD 相同 | -| 第二步:`mcpp run` | 带 dirun 配置的磁盘映像包与 runner(`enabled = false`)。映像的体积、再分发条款与 GitCode 镜像的大小限制都还没有实测,单独立项 | - -- **包的划分与名字(评审项)**:推荐每个系统一个平台包:`openbsd-platform`、`netbsd-platform`、 - `illumos-platform`。 - - 已有的平台包按系统命名(`openkal-platform`、`djgpp-platform`); - - 平台的版本跟随 sysroot 的系统版本(OpenBSD 7.x、NetBSD 10.x),三个系统共用一个版本号没有含义; - - 一个系统的问题(见下方的待实测项)不阻塞另外两个的发布; - - v1 推荐的 `unix-platform` 名不副实:Linux、macOS、FreeBSD 同属 unix,却不在其中。 - - 代价:三个仓库各有一次发布流程。三个包都只有数据,没有源码。 +| 构建 | xim 的 sysroot 包(官方安装集中的头文件、crt 与库);行的数据(三元组、clang 三元组、`family = "unix"`、加载器)、工具链 llvm | +| 运行 | xim 的磁盘映像包,带 dirun 的配置;行的 runner(`enabled = false`)。这与 Android 行已有的机制相同:`android-system-image` 带 `dirun.toml`,runner 为 `dirun --profile {pkg:…}/dirun.toml` | + +- **收录规范**:仓库中有一份“加入一个平台”的文档,规定行的必填数据、sysroot 包与映像包的要求、CI 必须验证的 + 内容、`preview` 到稳定的条件、进入内置表的条件。语义以 mcpp 的 SPEC-013、SPEC-016 为准,这份文档只规定流程。 - **待实测**:lld 能否按 OpenBSD 的规则查找带版本号的共享库(`libc.so.N.M`);NetBSD 下 lld 能否链接(否则用 - GNU ld);illumos 的 libc 与 crt 的组成;三个系统的虚拟机动作是否可用。 + GNU ld);illumos 的 libc 与 crt 的组成;映像的体积、再分发条款与 GitCode 镜像的大小限制。实测出不可逾越的 + 限制时如实记录,不静默降级。 - **约束**:OpenBSD 7.4 起只允许经 libc 发出系统调用(`pinsyscalls`),所以 OpenBSD 上的 okrun 以原生行构建; openkal 映像在 OpenBSD 上运行,以这一行为前提。 @@ -395,16 +424,16 @@ runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符 | 仓库 | 改动 | 版本 | |---|---|---| -| mcpp | `deploy` 的条目语法(4.1);`license_files` 与打包时的暂存、报告(4.3);`--file` 与 `[test] files`(5.2);工作目录(5.3);`MCPP_RUN_CLOSURE` 的 `run_files` 与 TSV 镜像(7.1);SPEC-002、SPEC-004、SPEC-014、SPEC-017 的条款;中英文文档;把多行内联表写进示例 | 下一版 | +| mcpp | `deploy` 的条目语法(4.1);`license_files` 与打包时的暂存、报告(4.3);`--file` 与 `[test] files`(5.2);工作目录(5.3);`MCPP_RUN_CLOSURE` 的 `run_files` 与 TSV 镜像(7.1);平台列查索引(6.2);SPEC-002、SPEC-004、SPEC-014、SPEC-017 的条款;中英文文档;把多行内联表写进示例 | 下一版 | | mcpp.plugins | dist-web 写明未提交的原因(8.3) | 补丁版本 | | dirun | `[payload] disk = "fat-image"`(8.4) | 0.6.0 | | openkal-uefi | 读取 `LoadOptions`(8.1) | 0.11.1 | | openkal-dos | 栈的大小、保护页与退路(8.2) | 0.1.1 | -| `openbsd-platform`、`netbsd-platform`、`illumos-platform` | 各自的行、工具链与 sysroot(8.5 第一步) | 0.1.0 | -| xim-pkgindex | dirun 0.6.0;ovmf 9.2.4-4;三个系统的 sysroot | — | -| mcpp-index | openkal-uefi 0.11.1、openkal-dos 0.1.1、openkal-platform 0.29.1、三个新平台包、mcpp.plugins;`compat.onnxruntime` 改用 `license_files` 与 `library_dirs` 的清理(等新版 mcpp 发布后进行) | — | +| mcpp-platforms(新,`mcpp-community`) | OpenBSD、NetBSD、illumos 的行、工具链、sysroot 与 runner;收录规范(8.5) | 0.1.0 | +| xim-pkgindex | dirun 0.6.0;ovmf 9.2.4-4;三个系统的 sysroot 与磁盘映像 | — | +| mcpp-index | openkal-uefi 0.11.1、openkal-dos 0.1.1、openkal-platform 0.29.1、`mcpp.platforms`、mcpp.plugins;平台条目中行的列表及其生成与校验工具(6.2);`compat.onnxruntime` 改用 `license_files` 与 `library_dirs` 的清理(等新版 mcpp 发布后进行) | — | | openkal-platform | 0.29.1:新成员;ovmf r4 与 dirun 0.6.0;DOS 行的缺省栈大小(8.2);以新版 plugins 构建;UEFI 检查增加含连续空白的参数;runner 改为多行(在要求新 mcpp 的版本中) | 0.29.1 | -| 外部验证工程(fork) | 改用 `--file`,去掉对 dirun 的直接调用和对包目录的 `find`;DOS 与 UEFI 运行完整的脚本;三个新行 | — | +| 外部验证工程(fork) | 改用 `--file`,去掉对 dirun 的直接调用和对包目录的 `find`;DOS、UEFI 与 Android 经 `mcpp run` 运行完整的脚本;三个新行;去掉平台的四行与 13.2 的两处过时绕行 | — | 通用的能力在引擎与生态包中(文件的放置、许可证的收集、工作目录、FAT 映像、sysroot、行);工程中只留下工程自己的 事实(哪些文件是它的测试数据、它选用哪些行)。 @@ -416,8 +445,8 @@ openkal-uefi ─┐ openkal-dos ──┼─> mcpp-index(成员)─> openkal-platform 0.29.1 ─> mcpp-index(平台)─┐ dirun ─> xim-pkgindex(dirun、ovmf r4)───────────────────────┘ │ mcpp.plugins ─> mcpp-index(plugins)─> openkal-platform 0.29.1 ├─> 外部验证工程 -mcpp(F、I)─> 发布 ────────────────────────────────────────────────────────────┤ -xim-pkgindex(sysroot)─> 三个新平台包 ─> mcpp-index(新平台包)─────────────────────┘ +mcpp(F、I、T)─> 发布 ────────────────────────────────────────────────────────────┤ +xim-pkgindex(sysroot、映像)─> mcpp-platforms ─> mcpp-index(mcpp.platforms)────────┘ ``` - 每个仓库一个 PR;xim-pkgindex 与 mcpp-index 是发布渠道,按前置的就绪时间各分两批(第一部分 §7 的规则)。 @@ -435,7 +464,8 @@ xim-pkgindex(sysroot)─> 三个新平台包 ─> mcpp-index(新平台包 | 8.1 | openkal-uefi 与 openkal-platform 的 CI 在 OVMF 中检查含连续空白的参数 | | 8.2 | openkal-dos 的 CI 中运行 #3 的复现程序,覆盖缺省栈、声明的栈与越界报告 | | 8.4 | dirun 的单元测试覆盖 FAT 映像的生成(含长文件名);在 OVMF 中写文件并在同一次运行中读出 | -| 8.5 | 新平台包的 CI:各构建一个 C 程序与一个 C++ 程序,并在虚拟机动作中运行 | +| 6.2 | e2e:索引中有平台认领的行不写平台即可构建,来源报告为 `index`;内置的行不变;工程的 `platform` 优先;索引 CI 拒绝重复认领 | +| 8.5 | mcpp-platforms 的 CI:每个系统构建一个 C 程序与一个 C++ 程序,并经 `mcpp run` 在磁盘映像中运行 | | 端到端 | 外部验证工程:DOS 与 UEFI 运行完整的脚本(含写文件的部分);三个新行运行脚本;CI 中不出现包目录的路径 | ## 12. 多角度评审(v2) @@ -453,9 +483,9 @@ xim-pkgindex(sysroot)─> 三个新平台包 ─> mcpp-index(新平台包 | R7 | 规范化 | v1 没有考虑已有的 `launch.cwd` 与外部启动者 | SPEC-017 B5 | 工作目录在写出 `launch` 时备好;写明粒度(5.3) | | R8 | 性能 | 每个测试进程一个目录时,`split = "case"` 的复制次数不受控;目录形式的 `deploy` 每次构建都复制 | SPEC-017 O1 | 每个测试程序一个目录;写时复制的克隆;`deploy` 的展开进入构建图并清理(4.1、5.3) | | R9 | 简洁 | dirun 读回文件没有读者 | 5.3:工作目录运行后删除 | 去掉读回;dirun 只生成映像(8.4) | -| R10 | 通用架构 | `[platforms]` 会把平台列出的宿主行一并接管;它是新的选择机制 | openkal-platform `mcpp.toml` 的 `targets`;SPEC-013 §3.1 | 本轮不做,单独立项(6.2) | +| R10 | 通用架构 | `[platforms]` 会把平台列出的宿主行一并接管;它是新的选择机制 | openkal-platform `mcpp.toml` 的 `targets`;SPEC-013 §3.1 | 改为平台列由索引补充(6.2) | | R11 | 缺省值是数据 | v1 把 DOS 的缺省栈大小只放在 stub 中 | AGENTS.md §2 | 写进平台行的链接参数,stub 的内置值只作退路(8.2) | -| R12 | 生态一致 | v1 让新行同时带磁盘映像与 runner;`unix-platform` 名不副实 | FreeBSD 的行没有 runner(`builtin.toml`),运行在 CI 的虚拟机动作中;已有平台包按系统命名 | 分两步;每个系统一个平台包(8.5) | +| R12 | 生态一致 | v1 让新行同时带磁盘映像与 runner;`unix-platform` 名不副实 | FreeBSD 的行没有 runner(`builtin.toml`),运行在 CI 的虚拟机动作中;已有平台包按系统命名 | 一个扩展包多行,构建与运行同在本轮(8.5) | | R13 | 用户体验 | 带 `--file` 的本机运行会改变工作目录 | 5.3 | 文档写明;`-v` 输出路径 | | R14 | 跨平台 | FAT 的 8.3 文件名会限制 `name` | — | 映像生成含长文件名(8.4) | | R15 | 边界 | `to` 拒绝 `..`,`../share` 的布局不能表达 | 4.1 的沿用规则 | 记录为已知边界,不在本轮(4.6) | @@ -471,7 +501,58 @@ xim-pkgindex(sysroot)─> 三个新平台包 ─> mcpp-index(新平台包 | 兼容 | 升级后现有工程的行为 | 不变:字面路径的落点不变;没有运行文件时工作目录不变;旧的 ovmf 修订与旧平台不受影响 | | 简洁 | 新增的键 | `deploy` 增加 `name` 一个字段与两种写法;`license_files`;`[test] files` 与 `--file`。runner、插件协议没有新增 | -## 13. 被否定的方案 +## 13. 双向交叉评审:通用设施与外部验证工程 + +评审的问题有两个方向:工程中的哪些做法其实是通用设施的缺口(应当上移);通用设施中有没有为这一个工程写死的内容 +(应当下移)。依据是工程的清单、构建程序、CI 与源码改动,以及对各仓库的检索和实测。 + +### 13.1 工程中的绕行,对应通用设施的缺口 + +| # | 工程中的做法 | 缺口 | 归属与处理 | +|---|---|---|---| +| X1 | UEFI 与 DOS:CI 直接调用 dirun,并在包目录中查找 dirun、固件与模拟器 | 运行文件不能经 mcpp 交付;固件的盘只读 | 本轮:第 5 节、8.4 | +| X2 | Android:CI 以 adb 手工推送程序与脚本并运行,而该行已有 runner | 同 X1:runner 收不到脚本 | 本轮:`--file` 之后改为 `mcpp run`,作为验证项 | +| X3 | FreeBSD:在 CI 的虚拟机动作中手工运行 | 该行没有 runner | 8.5 的磁盘映像机制同样适用;**评审项**:本轮是否为 FreeBSD 的行加上 runner | +| X4 | 四行重复写平台 | 平台列只有内置数据 | 本轮:6.2 | +| X5 | 构建程序按系统判断是否改产物名(`emscripten` 除外) | 工程给出的名字与行的后缀之间的先后不一致:emscripten 上工程的名字去掉了加载所必需的 `.js`,UEFI 上平台的 `{name}.efi` 却保留 | 新发现 N48。建议:行的数据陈述后缀是否为加载所必需,必需的后缀不被工程的模板去掉;工程无条件写一行即可。**评审项** | +| X6 | wasm 的 `-sSTACK_SIZE=1048576` 写在构建程序中,DOS 依赖平台的缺省 | 同一个事实(需要 1 MiB 的栈)按链接器各写一次 | 通用键 `stack_size` 的证据;已定本轮不加,记录 | +| X7 | 每个作业约 15 行安装 xlings 与 mcpp,带重试 | 没有公开的安装动作(mcpp 自己用仓库内的 `bootstrap-mcpp`) | 候选:`mcpp-community/setup-mcpp`。**评审项** | +| X8 | 插件版本须与平台构建时的版本一致,工程写死 `0.20.0` | N45 | 本轮:8.3;之后工程写范围 | +| X9 | CI 以 `find` 在 `target/` 下查找产物与打包目录 | 无:机器输出已有(docs/50) | 工程侧改用机器输出 | + +### 13.2 工程中已经过时的绕行 + +| # | 做法 | 实测 | 处理 | +|---|---|---|---| +| Y1 | 为第二个程序另写一个入口文件,只为带上一个不同的宏 | mcpp 2026.11.16.1:两个目标共用同一个 `main`、`defines` 不同时,各自得到自己的目标文件,运行结果分别正确 | 删除包装文件,两个目标指向同一个入口 | +| Y2 | FreeBSD 上直接指名包内的 okrun,绕过启动器 | N20 已在第一部分处理 | 以启动器验证后去掉 | + +### 13.3 通用设施中有没有为这个工程写死的内容 + +| 检查 | 结果 | +|---|---| +| 检索 mcpp 的源码与模块、mcpp.plugins、dirun、openkal-platform、openkal-uefi、openkal-dos、openkal-musl 中的工程名与脚本名 | 没有。引擎中出现的命令解释器名字是 Windows 宿主的解释器(钩子的执行),与工程无关 | +| ovmf 配置中的引用规则与结束标记 | 是 UEFI Shell 的通用规则,不含工程的内容 | +| DOS 的 1 MiB 缺省栈 | 由这个工程暴露(每层约 80 KiB),取值是 PE 的行业缺省,写在数据中并可覆盖 | +| N43 的参数截短 | edk2 Shell 的通用缺陷,任何带连续空白的参数都会遇到 | +| N48 的命名规则 | 尚未实现;建议的规则以行的数据表达,不识别任何系统名 | + +### 13.4 正确地留在工程中的内容 + +- 工程对 C 库的假设:`cfg(c-abi = "musl")` 下的宏、DJGPP 的替代实现。选择器是通用设施,事实是工程的。 +- 网页模板、版本资源、测试脚本与期望输出。 +- Emscripten 的链接选项中属于程序自身行为的部分(`SUPPORT_LONGJMP`、文件系统的选择)。 + +### 13.5 覆盖度 + +| 上游声明支持的系统 | 状态 | +|---|---| +| Linux、macOS、Windows、Android、FreeBSD、DOS、Web,以及 openkal 映像与 UEFI | 已有行(22 个目标) | +| OpenBSD、NetBSD、illumos | 本轮(8.5) | +| Haiku、Hurd、PureDarwin | 之后经 `mcpp.platforms` 的收录流程加入 | +| System V R4 | 以系统自带的 C 编译器构建,没有 LLVM 的目标;不能以行的方式到达,记录为范围之外 | + +## 14. 被否定的方案 | 方案 | 否定的原因 | |---|---| @@ -484,18 +565,28 @@ xim-pkgindex(sysroot)─> 三个新平台包 ─> mcpp-index(新平台包 | 用插件包装 runner 来投递运行文件 | 运行文件属于运行期,插件在构建期执行;还需要复制平台 runner 的命令行 | | 为 `mcpp::deploy` 增加参数(v1) | 构建程序可以自己计算(R5) | | dirun 读回程序写下的文件(v1) | 没有读者(R9) | -| `[platforms]`(v1 的 V1) | 会接管平台列出的宿主行;是新的选择机制;没有功能缺口(R10) | +| `[platforms]`(v1 的 V1) | 会接管平台列出的宿主行;是新的清单概念。由平台列查索引取代(6.2) | +| 每个系统一个平台包、各一个仓库(v2) | 分散,三条发布链;系统的版本本来就在行的数据中 | +| 新行只做构建,运行另行立项(v2) | 已定的内容不拆到以后 | | 把栈放在块的低端,越界离开描述符界限(第一部分 13.13 的第 3 条) | 平坦模型下需要移动全部段的地址 | | ovmf 直接改用 QEMU 的可写 vvfat(`fat:rw:`) | QEMU 文档说明它不可靠,写入可能损坏宿主目录 | -| 一个 `unix-platform` 包含三个系统(v1) | 名不副实;版本无法跟随各自的系统版本(R12) | +| 名字 `unix-platform`、`sysroot-platform` | 限定了收录的范围;`mcpp.platforms` 与 `mcpp.plugins` 对称 | -## 14. 待评审的决定点 +## 15. 决定与待评审的点 -1. 四种角色与各自的归属(3.1、4.3 末尾的表)。 -2. `deploy` 的条目语法:字符串形式、通配符、`name`,落点规则;键名用方案 A(4.1、4.2)。 -3. 发布文件用 `[package] license_files`,引擎统一放到 `share/licenses/<包名>/`,覆盖静态依赖(4.3)。 -4. 运行文件的名字:`--file` 与 `[test] files`(推荐),或 `--input` 与 `inputs`(5.2);工作目录的规则与粒度(5.3)。 -5. `[platforms]` 本轮不做(6.2)。 +**已定(2026-11-16 的评审):** + +1. 四种角色与各自的归属(3.1、4.3)。 +2. `deploy` 的条目语法与方案 A(4.1、4.2)。 +3. `[package] license_files`,带缺省值,引擎统一放到 `share/licenses/<包名>/`(4.3)。 +4. 运行文件:`--file` 与 `[test] files`;工作目录的规则,每个测试程序一个目录(5.2、5.3)。 +5. 平台列由索引补充,取代 `[platforms]`(6.2)。 6. dirun 自带 FAT32 映像的生成,不做读回(8.4)。 -7. 新行分两步;每个系统一个平台包(8.5)。 -8. DOS 的缺省栈大小写在平台行的数据中(8.2)。 +7. 新行在一个 PR 中完成构建与运行;`mcpp-community/mcpp-platforms`,包 `mcpp.platforms`,一个包多行(8.5)。 +8. DOS 的缺省栈大小写在平台行的数据中;通用键 `stack_size` 本轮不加(8.2)。 + +**待评审(第 13 节新提出):** + +1. N48:行的数据陈述后缀是否为加载所必需,必需的后缀不被工程的名字模板去掉(13.1 X5)。 +2. 本轮是否为 FreeBSD 的行加上 runner,与三个新系统使用同一机制(13.1 X3)。 +3. 是否提供公开的安装动作 `mcpp-community/setup-mcpp`(13.1 X7)。 From 1bb4809d582f15ee2c43fa4a8e672557aae75af0 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 17:35:49 +0900 Subject: [PATCH 05/36] Design record: part 2 v4, N48, the FreeBSD runner, setup-mcpp, and the implementation plan with its review --- ...6-ecosystem-gap-supplement-part2-design.md | 168 ++++++++++++++++-- 1 file changed, 157 insertions(+), 11 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index ba35d0cd5..9a454ca26 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -5,7 +5,7 @@ status: draft # 生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项 -**日期:** 2026-11-16 · **版本:** v3(第 15 节的八项已定;第 13 节为与外部验证工程的双向交叉评审) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 +**日期:** 2026-11-16 · **版本:** v4(第 15 节的十一项已定;第 13 节为双向交叉评审;第 16、17 节为实施计划及其评审) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平台以 0.20.0 构建)、xlings v2026.10.26.1 · **前文:** [第一部分](2026-11-15-ecosystem-gap-supplement-design.md)(N1–N47,13.12–13.17 为本文的来源) @@ -420,17 +420,73 @@ runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符 - **约束**:OpenBSD 7.4 起只允许经 libc 发出系统调用(`pinsyscalls`),所以 OpenBSD 上的 okrun 以原生行构建; openkal 映像在 OpenBSD 上运行,以这一行为前提。 +### 8.6 N48:产物名与行的后缀(已定) + +- **现状**:工程以 `mcpp::artifact_name("exe", "{name}")` 给出名字时,行的后缀被去掉。对 Windows 与 DOS 这是工程 + 想要的(名字本身已带扩展名)。对 Emscripten 则不是:`.js` 是启动文件与 `.wasm` 成对的依据,去掉之后 node、 + 网页与 dist-web 都找不到启动文件。平台以 `link.exe = "{name}.efi"` 给出的名字又不受工程的模板影响。工程 + 因此要在构建程序中按系统判断。 +- **规则**:行的数据陈述它的后缀是否为加载所必需(行键 `exe-suffix-fixed`,缺省为假)。必需的后缀总是出现在 + 产物名的末尾,工程的模板只决定它之前的部分;模板的结果已经以该后缀结尾时不重复添加。 +- **数据**:内置表中 Emscripten 的行写 `exe-suffix-fixed = true`;平台包为自己的行陈述。引擎不识别系统名。 +- **效果**:工程无条件写一行 `mcpp::artifact_name("exe", "{name}")`,不再判断系统。 +- **兼容**:没有给出模板的工程不变。给出模板而又面向 Emscripten 的工程,产物名恢复为带 `.js`;这正是它们现在 + 以条件判断得到的结果。 + +### 8.7 FreeBSD 的行加上 runner(已定) + +- FreeBSD 的四行在内置表中,目前没有 runner;运行验证只能在 CI 的虚拟机动作中手工进行。 +- 与 8.5 的三个系统使用同一机制:xim 的 `freebsd-disk-image` 包带 dirun 的配置,行的 runner 为 + `dirun --profile {pkg:freebsd-disk-image}/dirun.toml`,`enabled = false`。Android 的行已是这一写法。 +- 数据写在 mcpp 的内置表中(这些行本来就在那里);映像包在 xim-pkgindex。 +- 映像的要求对四个系统相同,写进 `mcpp.platforms` 的收录规范:串口控制台、自动登录、不需要网络、dirun 能把 + 文件送入并取得退出状态。 + +### 8.8 安装动作 `setup-mcpp`(已定) + +- **缺口**:使用 mcpp 的每个 CI 作业都要重复约 15 行:按系统安装 xlings、加入 PATH、带重试地安装指定版本的 + mcpp、设置镜像。mcpp 自己的 CI 用仓库内的 `bootstrap-mcpp`,它与 mcpp 的缓存策略绑定,不能给外部工程用。 +- **形式**:仓库 `mcpp-community/setup-mcpp`,根目录是一个 composite 动作(不需要构建产物,三个宿主系统通用), + 以 `v1` 这样的主版本标签发布。用法: + + ```yaml + - uses: mcpp-community/setup-mcpp@v1 + with: + version: 2026.11.16.1 # 缺省 latest + toolchains: llvm 23.1.3 # 可选,每行一个 + ``` + +| 输入 | 缺省 | 含义 | +|---|---|---| +| `version` | `latest` | mcpp 的版本 | +| `xlings-version` | 动作发布时验证过的版本 | 用来安装 mcpp 的 xlings | +| `mirror` | `GLOBAL` | `mcpp self config --mirror` 的值 | +| `toolchains` | 空 | 预先安装的工具链,每行一个 `mcpp toolchain install` 的参数 | +| `cache` | `false` | `true` 时缓存 mcpp 的包目录;键含系统、架构、xlings 与 mcpp 的版本、`mcpp.lock` 的散列 | + +| 输出 | 含义 | +|---|---| +| `version` | 实际安装的 mcpp 版本 | +| `cache-hit` | 缓存是否命中 | + +- **通用性**:动作中没有任何工程的名字;全部行为由输入决定。 +- **沿用已有的教训**(`bootstrap-mcpp` 的注释记录了它们):缓存键必须含架构(否则 arm 的机器取回 x86 的二进制); + xlings 的版本是缓存谱系的一部分;缓存缺省关闭,由使用者显式打开。 +- **验证**:动作自己的 CI 在 Linux x86_64、Linux aarch64、macOS、Windows 上安装并构建一个最小工程。 +- **采用**:本轮新建的仓库与外部验证工程直接使用;已有的生态仓库在下一次修改 CI 时改用,不为此单开 PR。 + ## 9. 归属 | 仓库 | 改动 | 版本 | |---|---|---| -| mcpp | `deploy` 的条目语法(4.1);`license_files` 与打包时的暂存、报告(4.3);`--file` 与 `[test] files`(5.2);工作目录(5.3);`MCPP_RUN_CLOSURE` 的 `run_files` 与 TSV 镜像(7.1);平台列查索引(6.2);SPEC-002、SPEC-004、SPEC-014、SPEC-017 的条款;中英文文档;把多行内联表写进示例 | 下一版 | +| mcpp | `deploy` 的条目语法(4.1);`license_files` 与打包时的暂存、报告(4.3);`--file` 与 `[test] files`(5.2);工作目录(5.3);`MCPP_RUN_CLOSURE` 的 `run_files` 与 TSV 镜像(7.1);平台列查索引(6.2);N48 的行键(8.6);FreeBSD 行的 runner 数据(8.7);SPEC-002、SPEC-004、SPEC-014、SPEC-017 的条款;中英文文档;把多行内联表写进示例 | 下一版 | | mcpp.plugins | dist-web 写明未提交的原因(8.3) | 补丁版本 | | dirun | `[payload] disk = "fat-image"`(8.4) | 0.6.0 | | openkal-uefi | 读取 `LoadOptions`(8.1) | 0.11.1 | | openkal-dos | 栈的大小、保护页与退路(8.2) | 0.1.1 | +| setup-mcpp(新,`mcpp-community`) | 安装动作(8.8) | v1 | | mcpp-platforms(新,`mcpp-community`) | OpenBSD、NetBSD、illumos 的行、工具链、sysroot 与 runner;收录规范(8.5) | 0.1.0 | -| xim-pkgindex | dirun 0.6.0;ovmf 9.2.4-4;三个系统的 sysroot 与磁盘映像 | — | +| xim-pkgindex | dirun 0.6.0;ovmf 9.2.4-4;三个系统的 sysroot;四个系统(含 FreeBSD)的磁盘映像 | — | | mcpp-index | openkal-uefi 0.11.1、openkal-dos 0.1.1、openkal-platform 0.29.1、`mcpp.platforms`、mcpp.plugins;平台条目中行的列表及其生成与校验工具(6.2);`compat.onnxruntime` 改用 `license_files` 与 `library_dirs` 的清理(等新版 mcpp 发布后进行) | — | | openkal-platform | 0.29.1:新成员;ovmf r4 与 dirun 0.6.0;DOS 行的缺省栈大小(8.2);以新版 plugins 构建;UEFI 检查增加含连续空白的参数;runner 改为多行(在要求新 mcpp 的版本中) | 0.29.1 | | 外部验证工程(fork) | 改用 `--file`,去掉对 dirun 的直接调用和对包目录的 `find`;DOS、UEFI 与 Android 经 `mcpp run` 运行完整的脚本;三个新行;去掉平台的四行与 13.2 的两处过时绕行 | — | @@ -440,6 +496,8 @@ runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符 ## 10. PR 划分与依赖 +本节是概览;逐个 PR 的阶段点与分层的依赖见第 16 节。 + ``` openkal-uefi ─┐ openkal-dos ──┼─> mcpp-index(成员)─> openkal-platform 0.29.1 ─> mcpp-index(平台)─┐ @@ -512,11 +570,11 @@ xim-pkgindex(sysroot、映像)─> mcpp-platforms ─> mcpp-index(mcpp.pla |---|---|---|---| | X1 | UEFI 与 DOS:CI 直接调用 dirun,并在包目录中查找 dirun、固件与模拟器 | 运行文件不能经 mcpp 交付;固件的盘只读 | 本轮:第 5 节、8.4 | | X2 | Android:CI 以 adb 手工推送程序与脚本并运行,而该行已有 runner | 同 X1:runner 收不到脚本 | 本轮:`--file` 之后改为 `mcpp run`,作为验证项 | -| X3 | FreeBSD:在 CI 的虚拟机动作中手工运行 | 该行没有 runner | 8.5 的磁盘映像机制同样适用;**评审项**:本轮是否为 FreeBSD 的行加上 runner | +| X3 | FreeBSD:在 CI 的虚拟机动作中手工运行 | 该行没有 runner | 本轮:8.7 | | X4 | 四行重复写平台 | 平台列只有内置数据 | 本轮:6.2 | -| X5 | 构建程序按系统判断是否改产物名(`emscripten` 除外) | 工程给出的名字与行的后缀之间的先后不一致:emscripten 上工程的名字去掉了加载所必需的 `.js`,UEFI 上平台的 `{name}.efi` 却保留 | 新发现 N48。建议:行的数据陈述后缀是否为加载所必需,必需的后缀不被工程的模板去掉;工程无条件写一行即可。**评审项** | +| X5 | 构建程序按系统判断是否改产物名(`emscripten` 除外) | 工程给出的名字与行的后缀之间的先后不一致:emscripten 上工程的名字去掉了加载所必需的 `.js`,UEFI 上平台的 `{name}.efi` 却保留 | 新发现 N48。本轮:8.6 | | X6 | wasm 的 `-sSTACK_SIZE=1048576` 写在构建程序中,DOS 依赖平台的缺省 | 同一个事实(需要 1 MiB 的栈)按链接器各写一次 | 通用键 `stack_size` 的证据;已定本轮不加,记录 | -| X7 | 每个作业约 15 行安装 xlings 与 mcpp,带重试 | 没有公开的安装动作(mcpp 自己用仓库内的 `bootstrap-mcpp`) | 候选:`mcpp-community/setup-mcpp`。**评审项** | +| X7 | 每个作业约 15 行安装 xlings 与 mcpp,带重试 | 没有公开的安装动作(mcpp 自己用仓库内的 `bootstrap-mcpp`) | 本轮:8.8 | | X8 | 插件版本须与平台构建时的版本一致,工程写死 `0.20.0` | N45 | 本轮:8.3;之后工程写范围 | | X9 | CI 以 `find` 在 `target/` 下查找产物与打包目录 | 无:机器输出已有(docs/50) | 工程侧改用机器输出 | @@ -535,7 +593,7 @@ xim-pkgindex(sysroot、映像)─> mcpp-platforms ─> mcpp-index(mcpp.pla | ovmf 配置中的引用规则与结束标记 | 是 UEFI Shell 的通用规则,不含工程的内容 | | DOS 的 1 MiB 缺省栈 | 由这个工程暴露(每层约 80 KiB),取值是 PE 的行业缺省,写在数据中并可覆盖 | | N43 的参数截短 | edk2 Shell 的通用缺陷,任何带连续空白的参数都会遇到 | -| N48 的命名规则 | 尚未实现;建议的规则以行的数据表达,不识别任何系统名 | +| N48 的命名规则 | 以行的数据表达,不识别任何系统名(8.6) | ### 13.4 正确地留在工程中的内容 @@ -585,8 +643,96 @@ xim-pkgindex(sysroot、映像)─> mcpp-platforms ─> mcpp-index(mcpp.pla 7. 新行在一个 PR 中完成构建与运行;`mcpp-community/mcpp-platforms`,包 `mcpp.platforms`,一个包多行(8.5)。 8. DOS 的缺省栈大小写在平台行的数据中;通用键 `stack_size` 本轮不加(8.2)。 -**待评审(第 13 节新提出):** +9. N48:行的数据陈述后缀是否为加载所必需(8.6)。 +10. FreeBSD 的行加上 runner,与三个新系统使用同一机制(8.7)。 +11. 公开的安装动作 `mcpp-community/setup-mcpp`(8.8)。 + +**待确认:** 两个新仓库由谁创建(16.4)。 + +## 16. 实施计划 + +### 16.1 仓库与 PR + +每个仓库一个 PR,以提交作为阶段点;每个阶段点带它的测试与规范、文档的更新。两个发布渠道(xim-pkgindex、 +mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表中列出。 + +| PR | 仓库 | 内容(阶段点) | 版本 | +|---|---|---|---| +| P1 | openkal-uefi(#19,已开) | 读取 `LoadOptions`(8.1) | 0.11.1 | +| P2 | openkal-dos | ① 读 `PT_GNU_STACK` ② 保护页 ③ 哨兵值与诊断 ④ 复现程序进入 CI(8.2) | 0.1.1 | +| P3 | dirun | ① FAT32 映像生成模块与单元测试 ② `[payload] disk` ③ OVMF 中写入并读出的检查(8.4) | 0.6.0 | +| P4 | mcpp.plugins | dist-web 写明原因(8.3) | 补丁 | +| P5 | mcpp(复用 #944,改标题) | ① `deploy` 的条目语法 ② `license_files` ③ 运行文件、工作目录与契约 ④ 平台列查索引 ⑤ N48 ⑥ FreeBSD 的 runner 数据 ⑦ 文档改用多行内联表 | 下一版 | +| P6 | setup-mcpp(新) | 动作与它的 CI(8.8) | v1 | +| P7a | xim-pkgindex | dirun 0.6.0、ovmf 9.2.4-4 | — | +| P7b | xim-pkgindex | 三个系统的 sysroot;四个系统的磁盘映像 | — | +| P8a | mcpp-index | openkal-uefi 0.11.1、openkal-dos 0.1.1、mcpp.plugins | — | +| P8b | mcpp-index | openkal-platform 0.29.1;平台条目中行的列表及其生成、校验工具 | — | +| P8c | mcpp-index | `mcpp.platforms` 0.1.0;`compat.onnxruntime` 改用 `license_files`;`library_dirs` 的清理 | — | +| P9 | openkal-platform | 新成员、DOS 的缺省栈、ovmf r4 与 dirun 0.6.0、新版 plugins、UEFI 检查 | 0.29.1 | +| P10 | mcpp-platforms(新) | ① 收录规范 ② OpenBSD ③ NetBSD ④ illumos,每个系统先构建后运行(8.5) | 0.1.0 | +| P11 | 外部验证工程(fork 的 PR) | 改用 `setup-mcpp`、`--file`;去掉平台的四行与两处过时绕行;三个新行;FreeBSD 与 Android 经 `mcpp run` | — | + +### 16.2 依赖 + +``` +第 0 层(实测,互不依赖) + M1 CWSDPMI 对 0507h 的支持 ─> P2 + M2 lld 与三个系统的链接;sysroot 的组成 ─> P7b、P10 + M3 四个系统的磁盘映像:体积、条款、镜像 ─> P7b + M4 旧引擎读到索引条目的新字段时的行为 ─> P5④、P8b + M5 打包目录命名规则对新包名的影响 ─> P10 + +第 1 层(互不依赖,并行) + P1 P2 P3 P4 P5 P6 + +第 2 层 + P3 ─> P7a + P1、P2、P4 ─> P8a + P5 ─> mcpp 发布 + +第 3 层 + P7a、P8a ─> P9 ─> P8b + M2、M3 ─> P7b ─> P10 ─> P8c(并需要 mcpp 发布) + +第 4 层 + mcpp 发布、P8b、P8c、P6 ─> P11 ─> 真实环境验证 ─> 生态级评审 ─> 关闭 issue ─> 报告 +``` + +- **关键路径有两条**:P5(引擎的五项功能,工作量最大)与 M3 → P7b → P10(磁盘映像的制作)。两条互不依赖, + 同时开始。 +- **等待 CI 时做什么**:第 1 层的六个 PR 轮流推进;CI 运行期间做第 0 层的实测与 P7b 的映像制作。 +- **发布的顺序**:先发布被依赖的包,再发布依赖它的包;每次发布之后在本地以 gtc 补 GitCode 的资源,再重新触发 + 主干的 CI。 + +### 16.3 每个 PR 的完成条件 + +1. 实现、测试、规范条款、中英文文档在同一个 PR 中; +2. 该仓库的全部 CI 通过;失败的作业先读日志再重跑; +3. 合入前按第 12 节的角度自查一次; +4. 合入、发布、镜像、索引更新之后,在隔离的 xlings 环境中从索引取用并验证。 -1. N48:行的数据陈述后缀是否为加载所必需,必需的后缀不被工程的名字模板去掉(13.1 X5)。 -2. 本轮是否为 FreeBSD 的行加上 runner,与三个新系统使用同一机制(13.1 X3)。 -3. 是否提供公开的安装动作 `mcpp-community/setup-mcpp`(13.1 X7)。 +### 16.4 需要用户完成或授权的事 + +| 事项 | 影响 | +|---|---| +| 创建 `mcpp-community/mcpp-platforms` 与 `mcpp-community/setup-mcpp`(或授权创建) | P6、P10 及其后续;其余不受影响 | + +## 17. 计划的自我评审 + +| 角度 | 检查 | 结论 | +|---|---|---| +| 架构 | 每项改动是否落在它所属的层 | 是:机制在引擎(P5),事实在数据(内置表、平台包、索引、xim 包),实现在各自的包(P1–P3),工程只留自己的事实(P11) | +| 通用性 | 是否有只为一个工程存在的改动 | 没有:第 13 节逐项检查;`setup-mcpp`、平台列查索引、`--file` 对任何工程成立 | +| 简洁 | 新增的清单键与命令行 | 清单两个键(`license_files`、`[test] files`),`deploy` 一个字段,命令行一个选项,行的数据一个键(`exe-suffix-fixed`);runner、插件协议、平台选择的清单语法没有新增 | +| 单 PR | P5 含七个阶段点,是否过大 | 是最大的一个。七项各自独立、各带测试,以提交分隔,可以逐个评审与回退;按“每个仓库一个 PR”保留为一个 | +| 单 PR | 发布渠道分批是否必要 | 必要:P8a 是 P9 的前置,P9 是 P8b 的前置,P10 是 P8c 的前置,不能合成一个 | +| 稳定性 | 实测项失败时的出路 | M1 失败:哨兵值作为唯一手段,启动时给出诊断。M2 中某个系统不能链接:该行暂不收录,记录原因,另外两个照常。M3 中映像不能再分发:该行只构建,在收录规范中标明,并报告 | +| 无感升级 | 发布的中间状态是否会破坏现有用户 | 不会:新语法只出现在声明了新下限的包中;平台列先查内置表;ovmf 的旧修订与平台的旧版本保留;`setup-mcpp` 是新增 | +| 无感升级 | P5④ 与 P8b 的先后 | 索引先带上行的列表而引擎尚未读取时没有影响,前提是 M4 确认旧引擎忽略该字段;否则 P8b 提高索引的引擎下限 | +| 一致性 | 四个 BSD 类系统与 Android 的运行方式 | 相同:磁盘映像包带 dirun 的配置 | +| 一致性 | 生态仓库的 CI 安装方式 | 新仓库与外部工程用 `setup-mcpp`;已有仓库在下一次修改 CI 时改用,本轮不为此单开 PR,这是唯一留到以后的事项,且不改变任何行为 | +| 跨平台 | 每项在三个宿主系统上的验证 | P3 的映像生成、P5 的工作目录与复制、P6 的动作都在 Linux、macOS、Windows 上有 CI | +| 测试 | 每个阶段点是否有自动化验证 | 有:第 11 节;P11 是端到端的检查 | +| 资源 | 磁盘映像的体积与本地存储 | 制作前清理过期的临时文件;体积在 M3 中实测并记录 | +| 风险 | 最不确定的部分 | M3(映像的制作与分发)与 P5④(解析顺序中插入索引查找)。两者都排在最早开始 | From a2af69ae105e5f402ac4084b5501a8be8dec77a9 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 17:35:59 +0900 Subject: [PATCH 06/36] Design record: part 2 v4, mark the settled sections --- .../2026-11-16-ecosystem-gap-supplement-part2-design.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index 9a454ca26..392b38c80 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -123,7 +123,7 @@ deploy = [ - **增量与清理**:展开的结果进入构建图。一个文件不再匹配时,它在程序旁的副本被移除;否则旧文件会留在运行环境中。 通配符的遍历以固定目录为界(mcpp#225 已有的规则)。 -### 4.2 键名:扩展 `deploy`(评审项) +### 4.2 键名:扩展 `deploy`(已定) | 方案 | 内容 | 判断 | |---|---|---| @@ -133,7 +133,7 @@ deploy = [ 规则可以在理由充分时偏离。v1 中支持 B 的唯一论据是“`deploy` 字面上不涵盖发布文件”;第 4.3 节把发布文件移出 `deploy` 之后,这条论据不再成立,所以推荐 A。 -### 4.3 发布文件:`license_files`(评审项,取代 v1 的 `phase = "pack"`) +### 4.3 发布文件:`license_files`(已定,取代 v1 的 `phase = "pack"`) v1 提出在 `deploy` 条目上加 `phase = "pack"`。评审否定了它: @@ -379,7 +379,7 @@ runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符 - **不做读回**(修订 v1):v1 让 dirun 在运行后把程序写下的文件读回工作目录。但工作目录在运行后即删除(5.3), 没有读者;测试在虚拟机内自行检查结果,结论经标准输出与退出状态返回。去掉读回之后,dirun 只需要**生成**映像, 不需要读 FAT。 -- **实现(评审项)**: +- **实现(已定为第一种)**: - **推荐**:dirun 自带 FAT32 映像的生成代码,含长文件名(`name` 不限于 8.3);不依赖外部工具,Windows 与 macOS 宿主上同样可用。只写不读,规模小。 - 另一种:调用 mtools,需要为三个宿主平台各提供一个包。 From b4633caabe9b82bf3c079b6d9c5d7aaacf337171 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 17:41:05 +0900 Subject: [PATCH 07/36] Design record: part 2 v5, the plan is settled and new repositories go through their first pull request --- ...1-16-ecosystem-gap-supplement-part2-design.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index 392b38c80..ca019e00c 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -5,7 +5,7 @@ status: draft # 生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项 -**日期:** 2026-11-16 · **版本:** v4(第 15 节的十一项已定;第 13 节为双向交叉评审;第 16、17 节为实施计划及其评审) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 +**日期:** 2026-11-16 · **版本:** v5(第 15 节的十二项已定;第 13 节为双向交叉评审;第 16、17 节为实施计划及其评审) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平台以 0.20.0 构建)、xlings v2026.10.26.1 · **前文:** [第一部分](2026-11-15-ecosystem-gap-supplement-design.md)(N1–N47,13.12–13.17 为本文的来源) @@ -647,7 +647,7 @@ xim-pkgindex(sysroot、映像)─> mcpp-platforms ─> mcpp-index(mcpp.pla 10. FreeBSD 的行加上 runner,与三个新系统使用同一机制(8.7)。 11. 公开的安装动作 `mcpp-community/setup-mcpp`(8.8)。 -**待确认:** 两个新仓库由谁创建(16.4)。 +12. 实施计划(第 16 节);两个新仓库由实施者创建,内容经 PR #1、CI 通过之后合入(16.4)。 ## 16. 实施计划 @@ -712,11 +712,15 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 3. 合入前按第 12 节的角度自查一次; 4. 合入、发布、镜像、索引更新之后,在隔离的 xlings 环境中从索引取用并验证。 -### 16.4 需要用户完成或授权的事 +### 16.4 新仓库的建立流程(已定) -| 事项 | 影响 | -|---|---| -| 创建 `mcpp-community/mcpp-platforms` 与 `mcpp-community/setup-mcpp`(或授权创建) | P6、P10 及其后续;其余不受影响 | +`mcpp-community/mcpp-platforms` 与 `mcpp-community/setup-mcpp` 由实施者创建,按常规流程进行: + +1. 建立仓库时只有许可证与一份简短的 README;主干不直接接受内容。 +2. 全部内容经第一个 PR(PR #1)进入:实现、CI、文档在同一个 PR 中。 +3. PR #1 的 CI 全部通过之后才合入;合入之后发布第一个版本。 +4. README 简洁:一句话说明它是什么,一个最小的用法,输入或行的一张表,指向规范的链接。细节放在文档中, + 不放在 README 中。 ## 17. 计划的自我评审 From 25e18381b4ffc4c7ff05e426e75404b9f8bf849e Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 18:39:44 +0900 Subject: [PATCH 08/36] N48: a row's fixed executable suffix ends a templated name A row states exe-suffix-fixed = true when its executable suffix is part of loading its programs. A name template (a build program's artifact_name, link.exe) then decides what precedes the suffix, and the name ends in it once: artifact_name("exe", "{name}") gives app.js beside app.wasm on wasm32-emscripten, where it gave a bare app, and {name}.js gives app.js, not app.js.js. The built-in wasm32-emscripten row states it; a platform states it for its own rows. The key is read by the built-in table's generator and by the platform reader alike. Spec: targets-and-providers v1.4 (2.1). Test: e2e web 1101. --- CHANGELOG.md | 10 +++++++++ docs/35-platforms.md | 7 ++++++- docs/specs/targets-and-providers.md | 6 +++++- docs/zh/35-platforms.md | 5 ++++- modules/manifest/src/platform.cppm | 3 ++- modules/manifest/src/platform_toml.cppm | 2 +- modules/toolchain-model/build.mcpp | 11 +++++++--- modules/toolchain-model/data/builtin.toml | 7 +++++++ .../toolchain-model/src/target_schema.cppm | 5 +++++ src/build/plan.cppm | 21 ++++++++++++++++++- src/build/prepare/platform.cpp | 1 + tests/e2e/web/emscripten.cpp | 20 ++++++++++++++++++ 12 files changed, 89 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 52a080d33..cfb28dbd1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,16 @@ ## [Unreleased] +### Changed + +- A row whose executable suffix is part of loading its programs states + `exe-suffix-fixed = true`, and a name template keeps that suffix: a build + program's `artifact_name("exe", "{name}")` names the Web row's launcher + `app.js` beside `app.wasm`, where it was a bare `app` that neither node nor a + browser loads, and `{name}.js` gives `app.js`, not `app.js.js`. The built-in + `wasm32-emscripten` row states it; a platform states it for its own rows. + The engine reads no system name for this (design 2026-11-16 N48). + ## [2026.11.16.1] - 2026-11-16 ### Changed diff --git a/docs/35-platforms.md b/docs/35-platforms.md index 24de54710..bd9bf594c 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -322,7 +322,12 @@ names the files the link may write beside a program, named after it: they are re before each link, staged when the link wrote one, and not an error when it did not. `exceptions` is the model of the row's builds when the project states no `[target..abi] exceptions`; under `"wasm"` the long-jump model is `"wasm"` too -(the built-in `wasm32-emscripten` row states `"wasm"`). A machine +(the built-in `wasm32-emscripten` row states `"wasm"`). `exe-suffix-fixed = true` +(2026.11.17.1+) states that the row's executable suffix is part of loading its +programs: a name template (`artifact_name("exe", "{name}")`, `link.exe`) decides +what precedes the suffix, and the name ends in it once. The built-in +`wasm32-emscripten` row states it, so a program named `{name}` is still `app.js` +beside `app.wasm`. A machine runs another architecture's programs only where the execution table states it (`[[platform.executes]]`, SPEC-016 §2.4); there a `same-arch` runner is built for the artifact's architecture and started on this machine. diff --git a/docs/specs/targets-and-providers.md b/docs/specs/targets-and-providers.md index b98620708..a3d863165 100644 --- a/docs/specs/targets-and-providers.md +++ b/docs/specs/targets-and-providers.md @@ -37,7 +37,7 @@ 行的字段:`triple`(`-[-]`)、`tier`(verified、preview、planned)、`standard`、`exact`、`march`、`mabi`、 `code-model`、`multilib`、`flags`、`link-emulation`、`clang-triple`、`object-format`、`family`、`data-model`、`exe-suffix`、 -`shared-suffix`、`loader`、`execution`、`descriptor-keys`、`companions`、`exceptions`。 +`shared-suffix`、`loader`、`execution`、`descriptor-keys`、`companions`、`exceptions`、`exe-suffix-fixed`。 - 一个系统可能用到两种以上 C 库时,身份**必须**含 env;env 即 C 库。业界原生标识无 env 段的系统,其原生身份为两段 (`aarch64-macos`),其余 C 库以 env 区分(`aarch64-macos-musl`)。 @@ -55,6 +55,9 @@ 或 `false` 陈述其他模型或不启用(v1.3)。 - `companions` 是链接可能在程序旁写出、以程序命名的文件,后缀替换程序的后缀(`app.js` 旁的 `app.wasm`)。每次链接前 删除;链接写出的被暂存;缺席不是错误(`-sSINGLE_FILE=1` 不写)(v1.3)。 +- `exe-suffix-fixed` 陈述行的可执行文件后缀是加载其程序的一部分(缺省为假)。为真时,名字模板(构建程序的 + `artifact_name`、`link.exe`)只决定后缀之前的部分,后缀总在名字末尾出现一次:模板的结果已以该后缀结尾时不重复。 + 内置的 `wasm32-emscripten` 行为真(`.js` 使启动文件与模块成对)。引擎不按系统名判断(v1.4)。 ### 2.2 内置行 @@ -234,4 +237,5 @@ | 1.0 | 2026-11-09 | 初版:行、提供者、选择、平台定义的行、C 环境、执行、报告与锁 | | 1.1 | 2026-11-10 | `clang-triple`;内置行的系统词;FreeBSD 两种身份与 32 位 planned 行;提供者 sysroot 为基座陈述的最后一级;`mingw-llvm` 与 `aarch64-windows-gnu`;C 库由平台描述的行与内置行同一读法;打包读文件闭包;`link.flags` 的警告;构建程序的 `MCPP_KERNEL_ABI_INTERFACES` | | 1.2 | 2026-11-12 | 行的 `loader` 与 `execution` 为内置行所读;系统表(`uname`);提供者的 `tier`、`system-headers`、`runner`;一条条目一种安排;请求的回答是契约;`glibc-sysroot`;一个执行谓词,另一系统的程序不被启动;程序解释器与基座包的单一读法;运行器的占位、`requires` 与 `built-as` | +| 1.4 | 2026-11-17 | mcpp 2026.11.17.1(设计 2026-11-16 N48):行的 `exe-suffix-fixed` | | 1.3 | 2026-11-16 | mcpp 2026.11.16.1(设计 2026-11-15 G7、N16、N23、N26、N33、R1、R2、X1):执行关系表(§2.4);系统表的 `resources` 与 `command-scripts`(§2.3);提供者运行器的 `host`、`enabled` 与 `exclusive`(§6);Android 行的运行器(G7);行的 `exceptions`(工程未陈述时的异常模型);行的 `companions`;平台以工具链列出的 planned 行可作载体的行 | diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index d59b3868f..7a873adc6 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -263,7 +263,10 @@ hosted = true # C 环境有 `main` **行的附带文件与异常模型(2026.11.16.1+)。** `companions = [".wasm"]` 指链接可能在程序旁写出、以程序命名的文件:每次链接前 删除,链接写出时被暂存,未写出不是错误。`exceptions` 是工程未陈述 `[target..abi] exceptions` 时该行构建的异常模型, -为 `"wasm"` 时 long-jump 模型亦为 `"wasm"`(内置的 `wasm32-emscripten` 行为 `"wasm"`)。机器只在执行关系表陈述时运行另一架构的程序(`[[platform.executes]]`, +为 `"wasm"` 时 long-jump 模型亦为 `"wasm"`(内置的 `wasm32-emscripten` 行为 `"wasm"`)。`exe-suffix-fixed = true` +(2026.11.17.1+)陈述行的可执行文件后缀是加载其程序的一部分:名字模板(`artifact_name("exe", "{name}")`、`link.exe`) +只决定后缀之前的部分,名字以该后缀结尾且只出现一次。内置的 `wasm32-emscripten` 行陈述它,所以命名为 `{name}` 的程序 +仍是 `app.wasm` 旁的 `app.js`。机器只在执行关系表陈述时运行另一架构的程序(`[[platform.executes]]`, SPEC-016 §2.4);此时 `same-arch` 的 runner 为产物的架构构建,在本机启动。 **定义的行与内置行同一读法(2026.11.10.1+)。** 它的 `march` 与 `link-emulation` 进入编译与 `ld.lld` 链接(`mcpp why flags` diff --git a/modules/manifest/src/platform.cppm b/modules/manifest/src/platform.cppm index 3fbdc33e5..5e7394d8b 100644 --- a/modules/manifest/src/platform.cppm +++ b/modules/manifest/src/platform.cppm @@ -264,7 +264,7 @@ struct RowSpec { exeSuffix, sharedSuffix, loader, execution, exceptions; std::string march, mabi, codeModel, multilib, linkEmulation; std::vector flags, descriptorKeys, companions; - bool standard = true, exact = false; + bool standard = true, exact = false, exeSuffixFixed = false; Origin origin; }; @@ -393,6 +393,7 @@ inline constexpr std::string_view kDefineKeys[] = { "triple", "tier", "note", "standard", "exact", "clang-triple", "gcc-triple", "object-format", "family", "data-model", "exe-suffix", "shared-suffix", "loader", "execution", "march", "mabi", "code-model", "multilib", "flags", "link-emulation", "descriptor-keys", "companions", "exceptions", + "exe-suffix-fixed", }; inline constexpr std::string_view kProviderKeys[] = { "name", "serves", "priority", "hosts", "pairs", "toolchain", "sysroot", "linkage", "base", diff --git a/modules/manifest/src/platform_toml.cppm b/modules/manifest/src/platform_toml.cppm index 35abaf27b..80d5f70b2 100644 --- a/modules/manifest/src/platform_toml.cppm +++ b/modules/manifest/src/platform_toml.cppm @@ -617,7 +617,7 @@ std::expected Reader::define(const toml::Table& t, std::st *to = std::move(*w); } for (auto [key, to] : std::initializer_list>{ - {"standard", &r.standard}, {"exact", &r.exact}}) + {"standard", &r.standard}, {"exact", &r.exact}, {"exe-suffix-fixed", &r.exeSuffixFixed}}) if (auto it = t.find(key); it != t.end()) { if (!it->second.is_bool()) return std::unexpected(std::format("{}.{} must be true or false", where, key)); *to = it->second.as_bool(); diff --git a/modules/toolchain-model/build.mcpp b/modules/toolchain-model/build.mcpp index 93fd89e1c..47fbc69cc 100644 --- a/modules/toolchain-model/build.mcpp +++ b/modules/toolchain-model/build.mcpp @@ -17,7 +17,7 @@ namespace { constexpr std::string_view kRowKeys[] = { "triple", "tier", "note", "standard", "exact", "march", "mabi", "code-model", "multilib", "flags", "link-emulation", "clang-triple", "gcc-triple", "family", "object-format", "data-model", "exe-suffix", - "shared-suffix", "loader", "execution", "companions", "exceptions", + "shared-suffix", "loader", "execution", "companions", "exceptions", "exe-suffix-fixed", }; constexpr std::string_view kProviderKeys[] = { "name", "serves", "priority", "hosts", "probe", "toolchain", "pairs", "sysroot", "linkage", @@ -119,12 +119,17 @@ std::string row_entry(const toml::Table& t, Arrays& arrays, std::setsecond.is_bool()) throw Failure{std::format("{}: `exact` is not a boolean", where)}; exact = it->second.as_bool(); } + bool suffixFixed = false; + if (auto it = t.find("exe-suffix-fixed"); it != t.end()) { + if (!it->second.is_bool()) throw Failure{std::format("{}: `exe-suffix-fixed` is not a boolean", where)}; + suffixFixed = it->second.as_bool(); + } return std::format( " {{ .triple = {}, .arch = {}, .os = {}, .env = {}, .tier = {}, .note = {}, .standard = {}, " ".exact = {}, .march = {}, .mabi = {}, " ".codeModel = {}, .multilib = {}, .flags = {}, .linkEmulation = {}, .clangTriple = {}, " ".gccTriple = {}, .objectFormat = {}, .family = {}, .dataModel = {}, .exeSuffix = {}, .sharedSuffix = {}, " - ".loader = {}, .execution = {}, .companions = {}, .exceptions = {} }},\n", + ".loader = {}, .execution = {}, .companions = {}, .exceptions = {}, .exeSuffixFixed = {} }},\n", lit(triple), lit(seg[0]), lit(seg[1]), lit(seg.size() == 3 ? seg[2] : std::string{}), lit(tier), lit(text(t, "note", where)), standard ? "true" : "false", exact ? "true" : "false", lit(text(t, "march", where)), lit(text(t, "mabi", where)), @@ -134,7 +139,7 @@ std::string row_entry(const toml::Table& t, Arrays& arrays, std::set& rows) { diff --git a/modules/toolchain-model/data/builtin.toml b/modules/toolchain-model/data/builtin.toml index bd3ce5784..9ba6f905c 100644 --- a/modules/toolchain-model/data/builtin.toml +++ b/modules/toolchain-model/data/builtin.toml @@ -49,6 +49,10 @@ # companions the files the link may write beside a program, named after # it (`.wasm` beside `app.js`): removed before each link, # staged when written, not an error when absent +# exe-suffix-fixed +# the row's executable suffix is part of loading its +# programs: a name template (`artifact_name`, `link.exe`) +# decides what precedes it, and the name ends in it once # # Provider keys # name the provider's name; `builtin:` selects it @@ -465,6 +469,9 @@ exceptions = "wasm" # The module Emscripten writes beside the launcher (`app.wasm` beside `app.js`), # none under `-sSINGLE_FILE=1` (N33). companions = [".wasm"] +# `.js` pairs the launcher with its module: a name template decides what +# precedes it, and the name ends in `.js` (N48). +exe-suffix-fixed = true # OK IMAGES (openkal SPEC clause 12): one artifact per architecture, run by # `okrun` on Linux, macOS and Windows of that architecture. diff --git a/modules/toolchain-model/src/target_schema.cppm b/modules/toolchain-model/src/target_schema.cppm index 7cd59dd50..9bd5f7fed 100644 --- a/modules/toolchain-model/src/target_schema.cppm +++ b/modules/toolchain-model/src/target_schema.cppm @@ -57,6 +57,11 @@ struct Row { // (`[target..abi] exceptions`; design 2026-11-15 N26): `wasm` on // wasm32-emscripten. Empty: the toolchain's default. std::string_view exceptions; + // THE ROW'S EXECUTABLE SUFFIX IS PART OF HOW ITS PROGRAMS ARE LOADED (design + // 2026-11-16 N48): `.js` of wasm32-emscripten, which pairs the launcher with + // its module. A name template (`artifact_name("exe", "{name}")`) decides what + // precedes it; the suffix ends the name, once. + bool exeSuffixFixed = false; bool defined = false; // stated by a platform package }; diff --git a/src/build/plan.cppm b/src/build/plan.cppm index a348606bd..ebd5b98ff 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -20,6 +20,7 @@ import mcpp.toolchain.dialect; import mcpp.toolchain.fingerprint; import mcpp.toolchain.linkmodel; import mcpp.toolchain.triple; +import mcpp.toolchain.target_rows; // a row's fixed executable suffix (N48) import mcpp.pack.prebuilt; // is_distribution_package — a shipped shared lib is not rebuilt import mcpplibs.os.platform; import mcpp.toolchain.platform; @@ -1993,6 +1994,16 @@ make_plan(const mcpp::manifest::Manifest& manifest, return t ? *t : mcpp::toolchain::triple::Triple{}; }(); auto naming = naming_for(tc); + // A ROW WHOSE SUFFIX IS PART OF LOADING (N48): the template decides what + // precedes the row's suffix, and the suffix ends the name once. Owned + // here, since `naming` holds views and the joined suffix is new text. + const std::string_view rowExeSuffix = naming.exeSuffix; + const bool exeSuffixFixed = [&] { + auto const* row = targetTriple.empty() ? nullptr + : mcpp::toolchain::target::find_row(targetTriple.str()); + return row && row->exeSuffixFixed; + }(); + std::string joinedExeSuffix; // A PLATFORM NAMES ITS ARTIFACTS (`[platform.link] exe`, `shared`): an // ok image is `.oke`, an image another loads `lib.okso`. // The patterns were checked where they were read. @@ -2012,7 +2023,15 @@ make_plan(const mcpp::manifest::Manifest& manifest, auto const& l = platform->fields.link; if (l.exe.set && mcpp::platform::split_name_pattern(l.exe.value)) { const std::string_view v = l.exe.value; - if (v.starts_with(kName)) naming.exeSuffix = v.substr(kName.size()); + if (v.starts_with(kName)) { + const std::string_view tail = v.substr(kName.size()); + if (exeSuffixFixed && !rowExeSuffix.empty() && !tail.ends_with(rowExeSuffix)) { + joinedExeSuffix = std::string(tail) + std::string(rowExeSuffix); + naming.exeSuffix = joinedExeSuffix; + } else { + naming.exeSuffix = tail; + } + } } if (l.shared.set && mcpp::platform::split_name_pattern(l.shared.value)) { const std::string_view v = l.shared.value; diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index e7c603789..b15eeaaaa 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -214,6 +214,7 @@ void define_platform(const mp::Statement& s, std::string_view shortName, std::st r.descriptorKeys = set.keep(d.descriptorKeys); r.companions = set.keep(d.companions); r.exceptions = set.keep(d.exceptions); + r.exeSuffixFixed = d.exeSuffixFixed; r.execution = set.keep(d.execution.empty() ? seg[1] : d.execution); r.defined = true; set.rows.push_back(r); diff --git a/tests/e2e/web/emscripten.cpp b/tests/e2e/web/emscripten.cpp index ec00ad63c..4dd6670c1 100644 --- a/tests/e2e/web/emscripten.cpp +++ b/tests/e2e/web/emscripten.cpp @@ -248,6 +248,26 @@ E2E emscripten{"emscripten", [] { expect(fatal(!s::find_file(app.at("target/dist"), "app.wasm").has_value())) << "a module was packed"; }; + "1101 a stated name keeps the web row's launcher suffix"_test[s::elf] = [] { + // The row's `.js` is part of loading (`exe-suffix-fixed`, design 2026-11-16 N48): a build + // program that names the program `{name}` gets `app.js` beside `app.wasm`, and one that + // names it `{name}.js` gets the same file, not `app.js.js`. + if (!have_emsdk() || !s::on_path("node")) skip("1101 -- no emsdk payload or node on this machine"); + for (std::string_view pattern : {"{name}", "{name}.js"}) { + s::project app(s::case_dir() / (pattern == "{name}" ? "bare" : "js"), + {{"mcpp.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[targets.app]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n[target.wasm32-emscripten]\nrunner = [\"node\"]\n"}, + {"src/main.cpp", "import std;\nint main() { std::println(\"named\"); }\n"}, + {"build.mcpp", std::format("import mcpp;\nint main() {{ mcpp::artifact_name(\"exe\", \"{}\"); return 0; }}\n", pattern)}}); + auto r = app.mcpp({"run", "--target", "wasm32-emscripten"}); + expect(fatal(e2e::succeeded(r) && s::has_line(s::text(r), "^named$"))) << pattern << ": " << s::text(r); + const auto js = s::find_file(app.at("target/wasm32-emscripten"), "app.js"); + expect(fatal(js.has_value())) << pattern << ": no app.js"; + expect(fatal(fs::is_regular_file(js->parent_path() / "app.wasm"))) << pattern << ": no app.wasm beside app.js"; + expect(fatal(!fs::exists(js->parent_path() / "app"))) << pattern << ": a launcher without its suffix"; + expect(fatal(!fs::exists(js->parent_path() / "app.js.js"))) << pattern << ": the suffix was doubled"; + } + }; + "1079 an emscripten rows longjmp model round trips"_test[s::elf] = [] { // `longjmp = "wasm"` makes a non-local exit over two call levels round-trip; on a Linux // row it is refused naming that row's os. From ed15faa5dd713a22c5ef20d6dd89d747c6567457 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 18:55:55 +0900 Subject: [PATCH 09/36] deploy: an entry is a path or a table; globs, name, and stale copies removed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit [runtime] deploy takes a string, which is { from = }, or a { from, to, name } table, in mcpp.toml and in descriptors. from may be a glob of the engine's one dialect; a match lands under to at its path below the glob's fixed directory (a literal path's is its parent, so every existing entry lands where it did). name renames the one match. A glob matching nothing and a name over several files are refused, naming the package and the entry; a literal path may still be generated by the build. deploy_files globs expand by the same rule. The expansion runs when the build is planned and enters the graph; placed.list beside build.ninja remembers the destinations, and a copy the new graph no longer places is removed from beside the program. SPEC-004 §15 (v1.20); docs/04 in both languages; unit tests for the readers and expand_deploy_entry; e2e runtime/deploy 1102. --- CHANGELOG.md | 13 ++++ docs/04-mcpp-toml.md | 51 ++++++++++----- docs/specs/manifest-semantics.md | 19 ++++++ docs/zh/04-mcpp-toml.md | 36 +++++++---- modules/manifest/src/toml.cppm | 44 ++++++++----- modules/manifest/src/types.cppm | 21 ++++++- modules/manifest/src/xpkg.cppm | 28 +++++++-- src/build/ninja_backend.cppm | 51 ++++++++++++++- src/build/plan.cppm | 92 +++++++++++++++++++++++++--- src/build/prepare/records.cpp | 8 ++- tests/e2e/runtime/deploy.cpp | 42 ++++++++++++- tests/unit/test_manifest.cpp | 35 ++++++++++- tests/unit/test_runtime_contract.cpp | 66 ++++++++++++++++++++ 13 files changed, 438 insertions(+), 68 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cfb28dbd1..086c96321 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,19 @@ ## [Unreleased] +### Added + +- A `[runtime] deploy` entry is a path or a table, and its `from` is a glob of + the engine's one dialect (`*`, `**`, `?`, `{a,b}`). `"share/data.bin"` is + `{ from = "share/data.bin" }`, placed beside the program; a match is placed + under `to` at its path below the glob's fixed directory, so + `{ from = "share/vulkan/icd.d/**", to = "vulkan/icd.d" }` keeps the tree; and + `name` renames the one file `from` matches. A glob that matches nothing, or a + `name` over several files, is refused naming the package and the entry. The + globs are expanded each time the build is planned, and a copy that is no + longer placed is removed from beside the program. `deploy_files = [x]` is + `deploy = [x]`. Descriptors read the same entries (design 2026-11-16 §4.1). + ### Changed - A row whose executable suffix is part of loading its programs states diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 9e74a6828..28e585e00 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1857,8 +1857,11 @@ link_library_dirs = ["lib"] transitive_needed_dirs = ["runtime/closure"] runtime_search_dirs = ["runtime"] frameworks = ["WindowKit"] -deploy_files = ["bin/widget.dll"] -deploy = [ { from = "share/vulkan/icd.d/widget_icd.json", to = "vulkan/icd.d" } ] +deploy = [ + "bin/widget.dll", # beside the program + { from = "share/vulkan/icd.d/**", to = "vulkan/icd.d" }, # a tree, its structure kept + { from = "tools/widget.conf", name = "app.conf" }, # renamed +] # Use an exact canonical identity when multiple providers exist. [runtime."display.present"] @@ -1930,23 +1933,39 @@ Link intent keeps discovery stages separate: | `deploy_files` | copy edge | copy edge | copy beside the output; never a linker flag | | `deploy` *(2026.9.12.2+)* | copy edge into `bin//` | copy edge into `bin//` | copy edge into `bin//`; never a linker flag | -`deploy` places a file in a directory relative to the executable, which -`deploy_files` cannot express because it places every entry beside the -executable. A loader that reads a fixed subdirectory needs it: the Vulkan loader -on macOS reads driver manifests from `/vulkan/icd.d`. Each entry -is a table of exactly two strings. `from` is relative to the declaring package's -root, and `to` is relative to the executable's directory, where `"."` means that -directory itself. Both are separated by `/` on every host, and neither may be -absolute, name a drive, or contain an empty, `.` or `..` component; an entry -that does is refused, and the refusal names its index. Two or more sources for +`deploy` lists the runtime files of a package: the files a program reads from +its own directory when it runs. A loader that reads a fixed subdirectory needs +them there: the Vulkan loader on macOS reads driver manifests from +`/vulkan/icd.d`. An entry is a string or a table +*(the string form, globs and `name`: 2026.11.17.1+)*: + +| Field | Meaning | +|---|---| +| `from` | relative to the declaring package's root; a glob of the engine's one dialect (`*`, `**`, `?`, `{a,b}`), the same as `sources` and `[pack] include` | +| `to` | the destination directory, relative to the executable's directory; `"."`, the default, is that directory itself | +| `name` | the destination's file name; `from` must then match exactly one file | + +A string is `{ from = }`. A file `from` matches is placed under `to` at +its path relative to the glob's fixed directory, the directory before the first +wildcard: `share/vulkan/icd.d/**` keeps the tree below `share/vulkan/icd.d`, and +a literal path, whose fixed directory is its parent, lands in `to` itself. A +glob that matches no file, or a `name` whose `from` matches more than one, is +an error naming the package and the entry. A literal path need not exist when +the build is planned, since a build program may generate it. The globs are +expanded over the package's files each time the build is planned; a file that +no longer matches is removed from beside the program. `from`, `to` and `name` +are separated by `/` on every host, and none may be absolute, name a drive, or +contain an empty, `.` or `..` component; an entry that does is refused, and the +refusal names its index. `deploy_files = [x]` is `deploy = [x]`, the earlier +spelling of the string form; new manifests write `deploy`. Two or more sources for one destination merge into a single copy, placed once every source is byte-identical (mcpp#723); they are refused at build time, naming every source and the destination, when they are not. One file name in two directories is -not a collision. `deploy` is a key of its own rather than a table -form of `deploy_files`, because a descriptor reader that predates it meets `{` -inside `deploy_files` and does not terminate, whereas it skips a `runtime` key it -does not know. `mcpp pack` stages the files of both keys at the same relative -path beside the packed executable. +not a collision. A package published to an index uses the string form, a glob +or `name` only with `mcpp = ">=2026.11.17.1"`: an earlier mcpp refuses a +string in `deploy` (`must be a table`) and an unknown key in an entry. `mcpp +pack` stages the files of both keys at the same relative path beside the packed +executable. For one compatibility train, `library_dirs` maps only to runtime search, `dlopen_libs` maps to required run-phase soname requirements, and diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 4a9307365..44b18d5e4 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -768,6 +768,24 @@ mcpp 2026.10.10.1,#785、#786;第 14 至 16 条与第 13 条的依赖位置为 m **状态:已实现**(mcpp 2026.10.26.2,mcpp#845)。 +## 15. 运行时文件与发布文件 + +文件按锚点分为三种(mcpp 设计 2026-11-16 §3):**运行时文件**是程序运行时从自身所在目录读取的文件,锚点是程序所在目录; +**发布文件**随包发布而运行时不读取,锚点是包的根;**运行文件**是一次运行开始时工作目录中应有的文件,锚点是该次运行的 +工作目录(SPEC-002 §6、SPEC-017 B1)。 + +1. `[runtime] deploy` 的条目是字符串或表。字符串即 `{ from = <字符串> }`。表的键为 `from`(必需)、`to`(缺省 `"."`) + 与 `name`;其他键**必须**被拒绝。`deploy_files = [x]` 即 `deploy = [x]`。 +2. `from` 相对声明它的包的根,可为引擎唯一的通配符方言(`*`、`**`、`?`、`{a,b}`),与 `sources` 相同。一个匹配放在 + `to` 下,路径取它相对于通配符**固定目录**(第一个通配符之前的目录)的部分;字面路径的固定目录是它的上级目录。 +3. `name` 是目的地的文件名,只在 `from` 恰好匹配一个文件时成立。通配符没有匹配,或 `name` 的 `from` 匹配到多个文件, + **必须**被拒绝并指出包与条目的序号。字面路径在规划时可以不存在(构建程序可能生成它)。 +4. 通配符在每次规划时对包内的文件展开,展开的结果进入构建图;此前放置而现在不再放置的文件**必须**从程序旁移除。 +5. 路径规则与同一目的地多个来源的规则不变:绝对路径、盘符、空段、`.`、`..` **必须**被拒绝;内容相同的来源合并为一份, + 不同则在构建时拒绝(SPEC-007 R4.2)。构建程序的 `mcpp::deploy(from, to)` 以字面路径声明,协议不变。 + +**状态:已实现**(mcpp 2026.11.17.1)。 + ## 变更记录 | 版本 | 日期 | 变更 | @@ -792,3 +810,4 @@ mcpp 2026.10.10.1,#785、#786;第 14 至 16 条与第 13 条的依赖位置为 m | 1.17 | 2026-10-23 | mcpp 2026.10.24.1:§2 陈述值的四档层次,`sysroot` 与 `run_under` 可出现在工程与机器默认中;§4.2 增加 `run_under`,陈述 `sysroot` 的表写法。 | | 1.18 | 2026-10-27 | mcpp 2026.10.26.2(mcpp#845):新增 §14 程序起点与镜像入口(`kind = "entry"`、`main = { from }`、`start`、`[test] main` 与 `start`、`mcpp:link-start`);§3.1.1 的按行形式可陈述起点。 | | 1.19 | 2026-10-28 | mcpp 2026.10.27.1(mcpp#847):§2 增加平台平面(`[platform]`、`[host]`),平台键可出现在工程与机器默认中;§4.2 增加 `[target.]` 的平台键与 `[host] platform`(SPEC-013)。 | +| 1.20 | 2026-11-17 | mcpp 2026.11.17.1(设计 2026-11-16 §4):新增 §15 运行时文件与发布文件:`deploy` 的条目语法(字符串、通配符、`name`)、固定目录规则与过期副本的移除。 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 733d31821..c3e8d257f 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1684,8 +1684,11 @@ link_library_dirs = ["lib"] transitive_needed_dirs = ["runtime/closure"] runtime_search_dirs = ["runtime"] frameworks = ["WindowKit"] -deploy_files = ["bin/widget.dll"] -deploy = [ { from = "share/vulkan/icd.d/widget_icd.json", to = "vulkan/icd.d" } ] +deploy = [ + "bin/widget.dll", # 放在程序旁 + { from = "share/vulkan/icd.d/**", to = "vulkan/icd.d" }, # 一棵目录,保持结构 + { from = "tools/widget.conf", name = "app.conf" }, # 改名 +] # Use an exact canonical identity when multiple providers exist. [runtime."display.present"] @@ -1752,19 +1755,26 @@ Link intent 把各个发现阶段分开处理: | `deploy_files` | 拷贝边 | 拷贝边 | 拷贝到输出旁边;从不是链接器旗标 | | `deploy` *(2026.9.12.2+)* | 拷贝边,进 `bin//` | 拷贝边,进 `bin//` | 拷贝边,进 `bin//`;从不是链接器旗标 | -`deploy` 把一个文件放进相对可执行文件的某个目录,而 `deploy_files` -表达不了这一点,因为它把每一条都放在可执行文件旁边。一个读取固定 -子目录的加载器需要它:macOS 上的 Vulkan loader 从 -`<可执行文件目录>/vulkan/icd.d` 读取驱动 manifest。每一条都是恰好 -两个字符串组成的表。`from` 相对声明它的包根目录,`to` 相对可执行 -文件所在目录,`"."` 意味着那个目录本身。两者在每个宿主上都以 `/` -分隔,都不能是绝对路径、不能命名一个驱动器、不能含有空、`.` 或 `..` -组成部分;不满足的条目会被拒绝,拒绝信息点名它的索引。两个或更多 +`deploy` 列出包的运行时文件:程序运行时从自己所在目录读取的文件。读取固定子目录的加载器需要它们在那里: +macOS 上的 Vulkan loader 从 `<可执行文件目录>/vulkan/icd.d` 读取驱动 manifest。条目是一个字符串或一张表 +*(字符串形式、通配符与 `name`:2026.11.17.1+)*: + +| 字段 | 含义 | +|---|---| +| `from` | 相对声明它的包的根;可以是引擎唯一的通配符方言(`*`、`**`、`?`、`{a,b}`),与 `sources`、`[pack] include` 相同 | +| `to` | 目的地目录,相对程序所在目录;缺省的 `"."` 即该目录 | +| `name` | 目的地的文件名;此时 `from` 必须恰好匹配一个文件 | + +字符串即 `{ from = <字符串> }`。`from` 匹配到的文件放在 `to` 下,路径取它相对于通配符固定目录(第一个通配符之前的 +目录)的部分:`share/vulkan/icd.d/**` 保持 `share/vulkan/icd.d` 之下的结构;字面路径的固定目录是它的上级目录,所以落在 +`to` 本身。通配符没有匹配到文件,或 `name` 的 `from` 匹配到多个文件,是错误,并指出包与条目。字面路径在规划时可以不存在, +因为构建程序可能生成它。通配符在每次规划构建时对包内的文件展开;不再匹配的文件从程序旁移除。`from`、`to` 与 `name` +在每个宿主上都以 `/` 分隔,都不能是绝对路径、不能命名驱动器、不能含有空、`.` 或 `..` 组成部分;不满足的条目被拒绝, +拒绝信息指出它的序号。`deploy_files = [x]` 即 `deploy = [x]`,是字符串形式较早的拼写;新清单写 `deploy`。两个或更多 来源指向同一个目的地会合并为一次拷贝,字节相同则放置(mcpp#723); 不同则在施工时失败,点名每一个来源与该目的地。一个文件名出现在两个 -不同目录下不算冲突。`deploy` 是一个独立的键,而不是 `deploy_files` 的 -表格形式,因为一个早于它出现的描述符读取器,遇到 `deploy_files` 里的 -`{` 会无法终止,而它会跳过一个不认识的 `runtime` 键。`mcpp pack` 把 +不同目录下不算冲突。发布到索引的包只有声明 `mcpp = ">=2026.11.17.1"` 时才能使用字符串形式、通配符或 `name`: +更早的 mcpp 对 `deploy` 中的字符串报 `must be a table`,对条目中的未知键报错。`mcpp pack` 把 这两个键指向的文件,以打包出的可执行文件为参照,拷贝到同样的相对 路径。 diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 63a202f83..db13c0d11 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -3445,36 +3445,48 @@ std::expected parse_document(const t::Document& documen out = it->second.as_string(); return true; }; - // `runtime.deploy` (#615): `{ from, to }` tables. A key of its own rather - // than a table form of `deploy_files`; see manifest::DeployEntry and the - // matching branch of the descriptor reader. + // `runtime.deploy` (#615; the entry grammar of design 2026-11-16 §4.1): a + // string, which is `{ from = }`, or a `{ from, to, name }` table. + // See manifest::DeployEntry and the matching branch of the descriptor reader. if (auto* deploy = doc->get("runtime.deploy")) { if (!deploy->is_array()) return std::unexpected(error(origin, - "runtime.deploy must be an array of `{ from = \"...\", to = \"...\" }` tables")); + "runtime.deploy must be an array of paths or `{ from = \"...\", to = \"...\" }` tables")); std::size_t index = 0; for (auto const& value : deploy->as_array()) { ++index; - if (!value.is_table()) - return std::unexpected(error(origin, std::format( - "runtime.deploy[{}] must be a table with `from` and `to`", index))); - auto const& table = value.as_table(); - for (auto const& [key, _] : table) - if (key != "from" && key != "to") + std::string from, to = ".", name; + if (value.is_string()) { + from = value.as_string(); + } else if (value.is_table()) { + auto const& table = value.as_table(); + for (auto const& [key, _] : table) + if (key != "from" && key != "to" && key != "name") + return std::unexpected(error(origin, std::format( + "runtime.deploy[{}] has unsupported key '{}'; the keys are " + "`from`, `to` and `name`", index, key))); + if (!table.contains("from")) + return std::unexpected(error(origin, std::format( + "runtime.deploy[{}] has no `from`", index))); + if (!table_string(table, "from", from) || !table_string(table, "to", to) + || !table_string(table, "name", name)) return std::unexpected(error(origin, std::format( - "runtime.deploy[{}] has unsupported key '{}'; the keys are " - "`from` and `to`", index, key))); - std::string from, to; - if (!table_string(table, "from", from) || !table_string(table, "to", to)) + "runtime.deploy[{}]: `from`, `to` and `name` must be strings", index))); + if (table.contains("name")) + if (auto p = deploy_name_problem(name); !p.empty()) + return std::unexpected(error(origin, + std::format("runtime.deploy[{}]: {}", index, p))); + } else { return std::unexpected(error(origin, std::format( - "runtime.deploy[{}]: `from` and `to` must be strings", index))); + "runtime.deploy[{}] must be a path or a table with `from`", index))); + } if (auto p = deploy_path_problem("from", from, false); !p.empty()) return std::unexpected(error(origin, std::format("runtime.deploy[{}]: {}", index, p))); if (auto p = deploy_path_problem("to", to, true); !p.empty()) return std::unexpected(error(origin, std::format("runtime.deploy[{}]: {}", index, p))); - m.runtimeConfig.linkIntent.deploy.push_back({from, to}); + m.runtimeConfig.linkIntent.deploy.push_back({from, to, name}); } } if (auto* requirements = doc->get("runtime.requirements")) { diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index e237e170c..7b05e60da 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1394,11 +1394,30 @@ struct RuntimeArtifact { // entry into `bin/`. The Vulkan loader on macOS reads its driver // manifest from `/vulkan/icd.d`, and a flattened copy is never // found there. +// +// THE ENTRY GRAMMAR (design 2026-11-16 §4.1). An entry is a string, which is +// `{ from = }`, or a table. `from` is relative to the package root and +// may be a glob of the engine's one dialect (`*`, `**`, `?`, `{a,b}`); a match +// is placed under `to` at its path relative to the glob's fixed directory (a +// literal path's fixed directory is its parent, so a literal file lands in +// `to` itself). `name` renames the one file `from` must then match. struct DeployEntry { - std::filesystem::path from; // relative to the package root + std::filesystem::path from; // relative to the package root; a glob before planning std::string to; // relative to the executable's directory; "." is that directory + std::string name; // the destination's file name; empty: the source's }; +// The problem with a deploy entry's `name`, or an empty string: one path +// component, the same rule as a component of `to`. +inline std::string deploy_name_problem(std::string_view v) { + if (v.empty()) return "`name` is empty"; + if (v.find('/') != std::string_view::npos || v.find('\\') != std::string_view::npos) + return "`name` is a file name, not a path"; + if (v == "." || v == "..") return "`name` is `.` or `..`"; + if (v.find(':') != std::string_view::npos) return "`name` names a drive or a scheme"; + return {}; +} + // One rule for a path a deploy entry names, checked on the string so that the // answer is the same on every host; see the payload descriptor's `frontend` // check for the measurement behind that choice. `dotAllowed` admits exactly diff --git a/modules/manifest/src/xpkg.cppm b/modules/manifest/src/xpkg.cppm index 70bae6560..efa8b55d2 100644 --- a/modules/manifest/src/xpkg.cppm +++ b/modules/manifest/src/xpkg.cppm @@ -2355,15 +2355,27 @@ synthesize_from_xpkg_lua(std::string_view luaContent, std::size_t index = 0; while (!rc.eof() && rc.peek() != '}') { ++index; + // A string is `{ from = }` (design 2026-11-16 §4.1). + if (rc.at_string_start()) { + std::string from = rc.read_string(); + if (auto p = deploy_path_problem("from", from, false); !p.empty()) + return std::unexpected(ManifestError{ + std::format("runtime.deploy[{}]: {}", index, p), + m.sourcePath, 0, 0}); + m.runtimeConfig.linkIntent.deploy.push_back({from, ".", {}}); + rc.skip_ws_and_comments(); + continue; + } if (rc.peek() != '{') { return std::unexpected(ManifestError{ - std::format("runtime.deploy[{}] must be a table with " - "`from` and `to`", index), + std::format("runtime.deploy[{}] must be a path or a table " + "with `from`", index), m.sourcePath, 0, 0}); } auto entryBody = rc.read_table_body(); LuaCursor entry{entryBody}; - std::string from, to; + std::string from, to = ".", name; + bool named = false; entry.skip_ws_and_comments(); while (!entry.eof()) { auto field = entry.read_key(); @@ -2380,10 +2392,11 @@ synthesize_from_xpkg_lua(std::string_view luaContent, } if (field == "from") from = entry.read_string(); else if (field == "to") to = entry.read_string(); + else if (field == "name") { name = entry.read_string(); named = true; } else { return std::unexpected(ManifestError{ std::format("runtime.deploy[{}] has unsupported key " - "'{}'; the keys are `from` and `to`", + "'{}'; the keys are `from`, `to` and `name`", index, field), m.sourcePath, 0, 0}); } @@ -2397,7 +2410,12 @@ synthesize_from_xpkg_lua(std::string_view luaContent, return std::unexpected(ManifestError{ std::format("runtime.deploy[{}]: {}", index, p), m.sourcePath, 0, 0}); - m.runtimeConfig.linkIntent.deploy.push_back({from, to}); + if (named) + if (auto p = deploy_name_problem(name); !p.empty()) + return std::unexpected(ManifestError{ + std::format("runtime.deploy[{}]: {}", index, p), + m.sourcePath, 0, 0}); + m.runtimeConfig.linkIntent.deploy.push_back({from, to, name}); rc.skip_ws_and_comments(); } rc.consume('}'); diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 6940ae62d..b78d77909 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -94,11 +94,22 @@ std::string ninja_not_found_advice(const mcpp::toolchain::Toolchain& tc); // Helper exposed for testing / debugging std::string emit_ninja_string(const BuildPlan& plan); +// THE COPIES A BUILD NO LONGER PLACES ARE REMOVED (design 2026-11-16 §4.1). A +// file that stops matching a deploy glob leaves its edge, and ninja does not +// remove the output of an edge that is gone, so the copy would stay beside the +// program and in its runtime environment. `placed.list`, beside build.ninja, +// holds the destinations of the previous graph; one that is not among +// `placed` now is removed when it is a file inside `outputDir`. +void sweep_stale_placements(const std::filesystem::path& outputDir, + const std::vector& placed); // The same, and when the plan places two or more files beside its programs, // the placement list the one `stage_list` edge reads (#734 E4), for the caller // to write as `/placements.list`. Empty when there is no such edge. +// `placed`, when given, receives every destination placed beside a program, +// relative to the output directory (the stale-copy sweep reads it). std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, - mcpp::build::progress::Attribution* attribution = nullptr); + mcpp::build::progress::Attribution* attribution = nullptr, + std::vector* placed = nullptr); std::string filter_ninja_output(std::string_view output, std::span commandPrefixes); @@ -1475,8 +1486,38 @@ std::string emit_ninja_string(const BuildPlan& plan) { return emit_ninja_string(plan, nullptr); } +void sweep_stale_placements(const std::filesystem::path& outputDir, + const std::vector& placed) { + namespace fs = std::filesystem; + std::set now; + std::string text; + for (auto const& d : placed) { + const auto rel = d.is_absolute() ? d.lexically_normal().lexically_relative(outputDir) : d.lexically_normal(); + const auto line = rel.generic_string(); + if (line.empty() || line.starts_with("..")) continue; + if (now.insert(line).second) text += line + "\n"; + } + const auto listPath = outputDir / "placed.list"; + std::error_code ec; + std::string old; + if (fs::exists(listPath, ec)) { + std::ifstream in(listPath, std::ios::binary); + old.assign(std::istreambuf_iterator(in), {}); + } + if (old == text) return; + std::istringstream lines(old); + for (std::string line; std::getline(lines, line);) { + if (line.empty() || now.contains(line) || line.starts_with("..")) continue; + const auto stale = outputDir / fs::path(line); + if (fs::is_regular_file(stale, ec)) fs::remove(stale, ec); + } + fs::create_directories(outputDir, ec); + std::ofstream(listPath, std::ios::binary | std::ios::trunc) << text; +} + std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, - mcpp::build::progress::Attribution* attribution) { + mcpp::build::progress::Attribution* attribution, + std::vector* placed) { // dyndep requires P1689 scanning capability: // GCC: built-in -fdeps-format=p1689r5 // Clang: external clang-scan-deps tool (same P1689 output format) @@ -1584,6 +1625,8 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, for (auto const& pl : plan.linkGroups[g].placements) placedFiles.push_back({{pl.source}, pl.dest}); } + if (placed) + for (auto const& d : placedFiles) placed->push_back(d.dest); // ── A platform's link inputs and seal (SPEC-013 §4, §4.6) ────────────── // @@ -4605,7 +4648,9 @@ std::expected NinjaBackend::build(const BuildPlan& plan write_graph_c_library_sidecar(plan.outputDir, plan.targetSide.cAbi); std::string placements; mcpp::build::progress::Attribution attribution; - auto manifest = emit_ninja_string(plan, &placements, &attribution); + std::vector placed; + auto manifest = emit_ninja_string(plan, &placements, &attribution, &placed); + if (!opts.dryRun) sweep_stale_placements(plan.outputDir, placed); stage("emit-ninja"); // The step record, beside build.ninja like the other files the fast path // reads (build progress design 2026-09-29, §6.3). diff --git a/src/build/plan.cppm b/src/build/plan.cppm index ebd5b98ff..c1f753f7c 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -14,6 +14,7 @@ import mcpp.manifest; import mcpp.source_kind; import mcpp.modgraph.graph; import mcpp.modgraph.scanner; +import mcpp.modgraph.glob; // a deploy glob's fixed directory (design 2026-11-16 §4.1) import mcpp.toolchain.cppfly; import mcpp.toolchain.detect; import mcpp.toolchain.dialect; @@ -273,8 +274,19 @@ struct ResolvedRuntimeContract { std::vector artifacts; mcpp::manifest::LinkIntent linkIntent; std::vector providers; + // A deploy glob that matched no file, or a `name` whose `from` did not + // match exactly one: each a sentence naming the package and the entry. + std::vector errors; }; +// The files a deploy entry of `root`'s package places (design 2026-11-16 +// §4.1): each match of `from` under `to`, at its path relative to the glob's +// fixed directory, renamed by `name`. A literal `from` (no wildcard) is +// returned as written, existing or not: a build may generate it. Exported as a +// pure seam for the tests. +std::expected, std::string> +expand_deploy_entry(const std::filesystem::path& root, const mcpp::manifest::DeployEntry& entry); + // Normalize structured and legacy runtime metadata and stamp every fact with // the exact package identity that supplied it. Exported as a pure seam for // contract tests and machine-readable tooling; it performs no host probing. @@ -1286,6 +1298,12 @@ local_include_dirs_after_for_manifest(const std::filesystem::path& root, namespace { +// A deploy path with a wildcard of the engine's one glob dialect, which +// `glob_literal_prefix` reads the same way. +bool has_wildcard(std::string_view path) { + return path.find_first_of("*?{[") != std::string_view::npos; +} + void append_unique_path(std::vector& out, std::filesystem::path path) { @@ -1296,6 +1314,35 @@ void append_unique_path(std::vector& out, } // namespace +std::expected, std::string> +expand_deploy_entry(const std::filesystem::path& root, const mcpp::manifest::DeployEntry& entry) { + const std::string pattern = entry.from.generic_string(); + const std::string to = entry.to.empty() ? std::string(".") : entry.to; + if (entry.from.is_absolute() || !has_wildcard(pattern)) + return std::vector{{entry.from, to, entry.name}}; + std::vector out; + std::set seen; + for (auto const& branch : mcpp::modgraph::expand_braces(pattern)) { + const auto base = (root / mcpp::modgraph::glob_literal_prefix(branch)).lexically_normal(); + for (auto const& file : mcpp::modgraph::expand_glob(root, branch)) { + if (!seen.insert(file.lexically_normal()).second) continue; + const auto rel = file.lexically_normal().lexically_relative(base); + auto dir = std::filesystem::path(to); + if (rel.has_parent_path()) dir /= rel.parent_path(); + auto dirText = dir.lexically_normal().generic_string(); + if (dirText.empty()) dirText = "."; + if (dirText.size() > 1 && dirText.ends_with('/')) dirText.pop_back(); + out.push_back({file.lexically_relative(root), dirText, entry.name}); + } + } + if (out.empty()) + return std::unexpected(std::format("`{}` matches no file under {}", pattern, root.generic_string())); + if (!entry.name.empty() && out.size() != 1) + return std::unexpected(std::format("`name` renames one file, and `{}` matches {}", pattern, out.size())); + std::ranges::sort(out, {}, [](auto const& e) { return e.from; }); + return out; +} + ResolvedRuntimeContract resolve_runtime_contract( const std::vector& packages) { @@ -1390,8 +1437,22 @@ ResolvedRuntimeContract resolve_runtime_contract( out.linkIntent.transitiveNeededDirs); append_paths(runtime.linkIntent.runtimeSearchDirs, out.linkIntent.runtimeSearchDirs); - append_paths(runtime.linkIntent.deployFiles, - out.linkIntent.deployFiles); + // A `deploy_files` entry is the string form of a `deploy` entry: a glob + // is expanded by the same rule; a literal path keeps its own list. + std::vector entries; + for (std::size_t i = 0; i < runtime.linkIntent.deployFiles.size(); ++i) { + auto const& f = runtime.linkIntent.deployFiles[i]; + if (f.is_absolute() || !has_wildcard(f.generic_string())) { + append_unique_path(out.linkIntent.deployFiles, absolute_from(package.root, f)); + continue; + } + auto expanded = expand_deploy_entry(package.root, {f, ".", {}}); + if (!expanded) { + out.errors.push_back(std::format("{}: runtime.deploy_files[{}]: {}", package.manifest.package.name, i + 1, expanded.error())); + continue; + } + entries.insert(entries.end(), expanded->begin(), expanded->end()); + } // `runtime.deploy` (#615): the source resolves against the package that // declared it; the destination stays relative, because it is relative // to an executable this package has not seen. @@ -1405,12 +1466,20 @@ ResolvedRuntimeContract resolve_runtime_contract( // manifest, because it may be an action's own declared output. Both // shapes pass through this one line correctly without it knowing // which one it was handed. - for (auto const& entry : runtime.linkIntent.deploy) { + for (std::size_t i = 0; i < runtime.linkIntent.deploy.size(); ++i) { + auto expanded = expand_deploy_entry(package.root, runtime.linkIntent.deploy[i]); + if (!expanded) { + out.errors.push_back(std::format("{}: runtime.deploy[{}]: {}", package.manifest.package.name, i + 1, expanded.error())); + continue; + } + entries.insert(entries.end(), expanded->begin(), expanded->end()); + } + for (auto& entry : entries) { mcpp::manifest::DeployEntry resolved{ - absolute_from(package.root, entry.from), entry.to}; + absolute_from(package.root, entry.from), entry.to, entry.name}; const bool seen = std::ranges::any_of(out.linkIntent.deploy, [&](auto const& e) { - return e.from == resolved.from && e.to == resolved.to; + return e.from == resolved.from && e.to == resolved.to && e.name == resolved.name; }); if (!seen) out.linkIntent.deploy.push_back(std::move(resolved)); } @@ -2150,9 +2219,11 @@ make_plan(const mcpp::manifest::Manifest& manifest, // over the member's closure, placed in its product directory (workspace // design 2026-09-29 §15). `out` is the plan or a link group; both carry // the same fields. + std::vector runtimeErrors; auto derive_runtime = [&](const std::vector& pkgs, const std::filesystem::path& binDir, auto& out) { auto runtimeContract = resolve_runtime_contract(pkgs); + if (runtimeErrors.empty()) runtimeErrors = runtimeContract.errors; out.runtimeRequirements = std::move(runtimeContract.requirements); out.runtimeArtifacts = std::move(runtimeContract.artifacts); out.linkIntent = std::move(runtimeContract.linkIntent); @@ -2196,11 +2267,11 @@ make_plan(const mcpp::manifest::Manifest& manifest, const bool peTarget = targetTriple.empty() ? bool(mcpplibs::os::is_windows) : targetTriple.is_pe(); auto add_deploy = [&](const std::filesystem::path& source, - std::string_view toDir = {}) { + std::string_view toDir = {}, std::string_view name = {}) { const auto normalized = source.lexically_normal(); auto destDir = binDir; if (!toDir.empty() && toDir != ".") destDir /= std::filesystem::path(toDir); - const auto dest = destDir / source.filename(); + const auto dest = destDir / (name.empty() ? source.filename() : std::filesystem::path(name)); auto existing = std::ranges::find_if(out.runtimeDeployFiles, [&](auto const& value) { return value.is_destination(dest, peTarget); }); if (existing != out.runtimeDeployFiles.end()) { @@ -2217,7 +2288,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, add_deploy(source); } for (auto const& entry : out.linkIntent.deploy) { - add_deploy(entry.from, entry.to); + add_deploy(entry.from, entry.to, entry.name); } // A DLL found in a runtime search directory is a DERIVED candidate // (SPEC-007 R4.3). The plan lists it and does not decide: which file sits @@ -2295,6 +2366,11 @@ make_plan(const mcpp::manifest::Manifest& manifest, } }; derive_runtime(packages, std::filesystem::path("bin"), plan); + if (!runtimeErrors.empty()) { + std::string text = "a runtime file cannot be placed:"; + for (auto const& e : runtimeErrors) text += "\n " + e; + return std::unexpected(text); + } // 1a. Object addressing. // diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index 58ba0a141..735d2b234 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -734,9 +734,11 @@ void step13_resolution_json(PrepareState& state, BuildContext& ctx) { {"deploy_files", path_array(ctx.plan.linkIntent.deployFiles)}, {"deploy", [&] { auto a = nlohmann::json::array(); - for (auto const& d : ctx.plan.linkIntent.deploy) - a.push_back({{"from", d.from.generic_string()}, - {"to", d.to}}); + for (auto const& d : ctx.plan.linkIntent.deploy) { + nlohmann::json e{{"from", d.from.generic_string()}, {"to", d.to}}; + if (!d.name.empty()) e["name"] = d.name; + a.push_back(std::move(e)); + } return a; }()}, }}, diff --git a/tests/e2e/runtime/deploy.cpp b/tests/e2e/runtime/deploy.cpp index e62cc206a..d294d2a15 100644 --- a/tests/e2e/runtime/deploy.cpp +++ b/tests/e2e/runtime/deploy.cpp @@ -1,7 +1,8 @@ // `runtime.deploy` and a build program's deploy: a file placed in a directory relative to the // executable (the root's and a dependency's), seen by a test program too; two sources for one // destination placed once when their bytes agree and refused at build time when they do not; a -// destination outside the executable's directory refused (formerly tests/e2e/646, 810). +// destination outside the executable's directory refused (formerly tests/e2e/646, 810); the +// entry grammar of design 2026-11-16 §4.1 and the removal of copies no longer placed (1102). import std; import mcpplibs.mtest; import mcpp.test.support; @@ -111,6 +112,45 @@ int main(int, char** argv) { expect(fatal(e2e::contains(s::text(r), "runtime.deploy[1]: `to` has a `.` or `..` component"))) << "FAIL: the refusal does not name the entry and the component: " << s::text(r); }; + "1102 deploy entries are strings, globs and renames, and a copy no longer placed goes"_test = [] { + // Design 2026-11-16 §4.1: a string is `{ from }`; a glob keeps the path below its fixed + // directory; braces are one glob; `name` renames the one match. A file that stops matching + // leaves the program's directory on the next build. A glob that matches nothing, and a + // `name` over several files, are refused naming the entry. + s::project app(s::case_dir() / "app", {{"src/main.cpp", "int main() { return 0; }\n"}, + {"share/data.bin", "data\n"}, + {"share/icd.d/x.json", "x\n"}, + {"share/icd.d/sub/y.json", "y\n"}, + {"res/a.json", "a\n"}, {"res/b.bin", "b\n"}, {"res/c.txt", "c\n"}, + {"tools/x.conf", "conf\n"}}); + const std::string head = "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[runtime]\ndeploy = [\n"; + const std::string entries = " \"share/data.bin\",\n { from = \"share/icd.d/**\", to = \"vulkan/icd.d\" },\n" + " { from = \"res/*.{json,bin}\", to = \"res\" },\n { from = \"tools/x.conf\", name = \"app.conf\" },\n"; + app.write("mcpp.toml", head + entries + "]\n"); + auto r = app.mcpp({"build"}); + expect(fatal(e2e::succeeded(r))) << s::text(r); + std::optional bin; + for (const auto& f : s::find_files(app.at("target"), R"(app(\.exe)?)")) + if (f.generic_string().contains("/bin/") && !f.generic_string().contains("/.build-mcpp/")) { bin = f.parent_path(); break; } + expect(fatal(bin.has_value())) << "no program under target/"; + for (auto rel : {"data.bin", "vulkan/icd.d/x.json", "vulkan/icd.d/sub/y.json", "res/a.json", "res/b.bin", "app.conf"}) + expect(fatal(fs::is_regular_file(*bin / rel))) << rel << " is not beside the program"; + expect(fatal(!fs::exists(*bin / "res/c.txt"))) << "a file the braces do not name was placed"; + fs::remove(app.at("share/icd.d/sub/y.json")); + r = app.mcpp({"build"}); + expect(fatal(e2e::succeeded(r))) << s::text(r); + expect(fatal(!fs::exists(*bin / "vulkan/icd.d/sub/y.json"))) << "the copy of a file that no longer matches stayed"; + expect(fatal(fs::is_regular_file(*bin / "vulkan/icd.d/x.json"))) << "a file that still matches was removed"; + app.write("mcpp.toml", head + entries + " \"none/*.dat\",\n]\n"); + r = app.mcpp({"build"}); + expect(fatal(!e2e::succeeded(r))) << "a glob that matches nothing was accepted"; + expect(fatal(e2e::contains(s::text(r), "runtime.deploy[5]: `none/*.dat` matches no file"))) << s::text(r); + app.write("mcpp.toml", head + " { from = \"res/*\", name = \"one\" },\n]\n"); + r = app.mcpp({"build"}); + expect(fatal(!e2e::succeeded(r))) << "a name over three files was accepted"; + expect(fatal(e2e::contains(s::text(r), "`name` renames one file, and `res/*` matches 3"))) << s::text(r); + }; + "810 two packages deploy the same bytes to one name"_test[s::gcc][s::elf] = [] { // Two build programs deploy a generated file to one name: identical bytes give one // stage_file edge with both sources and `emit build-database` succeeds; different bytes diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index afac4b859..86e6827aa 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -5855,8 +5855,12 @@ UTest runtime_deploy_refuses_each_malformed_entry_naming_it{"Manifest/RuntimeDep {R"({ from = ".", to = "x" })", "`from` has a `.` or `..` component"}, {R"({ from = "a//b.json", to = "x" })", "`from` has an empty path component"}, {R"({ from = "a.json", dest = "x" })", "has unsupported key 'dest'"}, - {R"({ from = "a.json", to = 3 })", "`from` and `to` must be strings"}, - {R"("a.json")", "must be a table with `from` and `to`"}, + {R"({ from = "a.json", to = 3 })", "`from`, `to` and `name` must be strings"}, + {R"({ to = "x" })", "has no `from`"}, + {R"({ from = "a.json", name = "x/y" })", "`name` is a file name, not a path"}, + {R"({ from = "a.json", name = ".." })", "`name` is `.` or `..`"}, + {R"(3)", "must be a path or a table with `from`"}, + {R"("../a.json")", "`from` has a `.` or `..` component"}, }; for (auto [entry, expected] : cases) { const auto src = std::format(R"( @@ -5873,6 +5877,33 @@ deploy = [ {} ] } }}; +// Design 2026-11-16 §4.1: a string is `{ from = }`; `to` defaults to +// "."; `name` renames; a glob is kept as written for planning to expand. +UTest runtime_deploy_reads_strings_globs_and_names{"Manifest/RuntimeDeployReadsStringsGlobsAndNames", [] { + constexpr auto src = R"( +[package] +name = "icd" +version = "0.1.0" +[runtime] +deploy = [ + "share/data.bin", + { from = "share/vulkan/icd.d/**", to = "vulkan/icd.d" }, + { from = "tools/x.conf", name = "app.conf" }, +] +)"; + auto m = mcpp::manifest::parse_string(src); + if (auto&& failed = expect(fatal(m.has_value())); !failed) failed << m.error().format(); + auto const& d = m->runtimeConfig.linkIntent.deploy; + expect(fatal(eq(d.size(), 3u))); + expect(eq(d[0].from, std::filesystem::path("share/data.bin"))); + expect(eq(d[0].to, ".")); + expect(eq(d[0].name, "")); + expect(eq(d[1].from, std::filesystem::path("share/vulkan/icd.d/**"))); + expect(eq(d[2].to, ".")); + expect(eq(d[2].name, "app.conf")); + expect(m->schemaWarnings.empty()); +}}; + // ── `requires_abi` (design 2026-09-12, section 5.2) ──────────────────────── UTest requires_abi_threads_parses_on_the_package_and_on_a_feature{"Manifest/RequiresAbiThreadsParsesOnThePackageAndOnAFeature", [] { diff --git a/tests/unit/test_runtime_contract.cpp b/tests/unit/test_runtime_contract.cpp index f1b8b7f2f..550a0e09e 100644 --- a/tests/unit/test_runtime_contract.cpp +++ b/tests/unit/test_runtime_contract.cpp @@ -473,6 +473,72 @@ package = { expect(eq(parsed->runtimeConfig.linkIntent.deployFiles, std::vector{"bin/backend.dll"})); }}; +// Design 2026-11-16 §4.1 in a descriptor: a string entry and `name`. +UTest xpkg_reads_runtime_deploy_strings_and_names{"RuntimeContract/XpkgReadsRuntimeDeployStringsAndNames", [] { + constexpr auto lua = R"( +package = { + spec = "1", namespace = "acme", name = "icd", + xpm = { linux = { ["1.0.0"] = { url = "u", sha256 = "h" } } }, + mcpp = { + sources = { "src/icd.cpp" }, + runtime = { + deploy = { + "share/data.bin", + { from = "onnxruntime-*/lib/*.so*", to = "lib" }, + { from = "tools/x.conf", name = "app.conf" }, + }, + deploy_files = { "bin/backend.dll" }, + }, + }, +} +)"; + auto parsed = mf::synthesize_from_xpkg_lua( + lua, "icd", "1.0.0", mcpp::toolchain::platform::HostPlatform::current()); + if (auto&& failed = expect(fatal(bool(parsed))); !failed) failed << parsed.error().format(); + auto const& d = parsed->runtimeConfig.linkIntent.deploy; + expect(fatal(eq(d.size(), 3u))); + expect(eq(d[0].from, std::filesystem::path("share/data.bin"))); + expect(eq(d[0].to, ".")); + expect(eq(d[1].to, "lib")); + expect(eq(d[2].name, "app.conf")); + expect(eq(parsed->runtimeConfig.linkIntent.deployFiles, std::vector{"bin/backend.dll"})); +}}; + +// The expansion of one entry: a match keeps its path below the glob's fixed +// directory, a literal path lands in `to`, braces are one glob, `name` renames +// the one match, and a glob that matches nothing or a `name` over several is +// an error naming the glob. +UTest expand_deploy_entry_places_each_match{"RuntimeContract/ExpandDeployEntryPlacesEachMatch", [] { + const auto root = std::filesystem::temp_directory_path() / "mcpp-deploy-expand"; + std::filesystem::remove_all(root); + for (auto rel : {"share/icd.d/a.json", "share/icd.d/sub/b.json", "res/x.json", "res/y.bin", "res/z.txt", "t/one.conf"}) { + std::filesystem::create_directories((root / rel).parent_path()); + std::ofstream(root / rel) << rel; + } + auto tree = build::expand_deploy_entry(root, {"share/icd.d/**", "vulkan/icd.d", {}}); + expect(fatal(tree.has_value())); + expect(fatal(eq(tree->size(), 2u))); + expect(eq((*tree)[0].to, "vulkan/icd.d")); + expect(eq((*tree)[1].from, std::filesystem::path("share/icd.d/sub/b.json"))); + expect(eq((*tree)[1].to, "vulkan/icd.d/sub")); + auto braces = build::expand_deploy_entry(root, {"res/*.{json,bin}", "res", {}}); + expect(fatal(braces.has_value())); + expect(eq(braces->size(), 2u)); + auto literal = build::expand_deploy_entry(root, {"res/missing.dat", ".", {}}); + expect(fatal(literal.has_value())) << "a literal path may be generated by the build"; + expect(eq((*literal)[0].from, std::filesystem::path("res/missing.dat"))); + auto renamed = build::expand_deploy_entry(root, {"t/*.conf", ".", "app.conf"}); + expect(fatal(renamed.has_value())); + expect(eq((*renamed)[0].name, "app.conf")); + auto none = build::expand_deploy_entry(root, {"nothing/*.dat", ".", {}}); + expect(fatal(!none.has_value())); + expect(ne(none.error().find("`nothing/*.dat` matches no file"), std::string::npos)) << none.error(); + auto several = build::expand_deploy_entry(root, {"res/*", ".", "one.bin"}); + expect(fatal(!several.has_value())); + expect(ne(several.error().find("matches 3"), std::string::npos)) << several.error(); + std::filesystem::remove_all(root); +}}; + UTest xpkg_runtime_deploy_refuses_an_escaping_destination{"RuntimeContract/XpkgRuntimeDeployRefusesAnEscapingDestination", [] { constexpr auto lua = R"( package = { From 59bdfbee924544968800925fa7b8506849124d11 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 19:07:59 +0900 Subject: [PATCH 10/36] license_files: every pack stages each package's license files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit [package] license_files names a package's license and notice files as globs relative to its root; a descriptor writes the same key in its mcpp segment. Unset, the engine's default {LICENSE,LICENCE,COPYING, NOTICE}* applies. The build plan records each package of the graph with its patterns, and every pack, in every mode, stages them under share/licenses// with the role metadata: static libraries are covered as shared ones are. A package for which nothing is found is named in the report; a declared glob that selects nothing is warned about. This replaces collect_licenses, which ran only in self-contained packs, under seven fixed names, and said nothing when it found none. That mode still looks for the default files beside the shared objects it bundles that no package declares, and names those without. SPEC-004 §15.6-7, SPEC-014 §4.5.1 (v2.6); docs/04 and docs/10 in both languages; unit tests for both readers; e2e pack/modes 1103. --- CHANGELOG.md | 10 +++ docs/04-mcpp-toml.md | 10 +++ docs/10-pack-and-release.md | 16 ++++ docs/specs/manifest-semantics.md | 7 +- docs/specs/packaging.md | 10 +++ docs/zh/04-mcpp-toml.md | 7 ++ docs/zh/10-pack-and-release.md | 9 ++ modules/manifest/src/toml.cppm | 3 +- modules/manifest/src/types.cppm | 12 +++ modules/manifest/src/xpkg.cppm | 18 ++++ src/build/plan.cppm | 21 +++++ src/pack/pack.cppm | 125 ++++++++++++++++++++------- src/pack/pipeline.cppm | 2 + tests/e2e/pack/modes.cpp | 28 ++++++ tests/unit/test_manifest.cpp | 19 ++++ tests/unit/test_runtime_contract.cpp | 19 ++++ 16 files changed, 285 insertions(+), 31 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 086c96321..8c10e9573 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,16 @@ globs are expanded each time the build is planned, and a copy that is no longer placed is removed from beside the program. `deploy_files = [x]` is `deploy = [x]`. Descriptors read the same entries (design 2026-11-16 §4.1). +- `[package] license_files` names a package's license and notice files, and + every `mcpp pack`, in every mode, stages them for each package of the + program's graph under `share/licenses//` with the role `metadata`: + static libraries included, which carry the obligation as shared ones do. A + package that states none uses `{LICENSE,LICENCE,COPYING,NOTICE}*` at its + root; a package for which nothing is found is named in the report. This + replaces the search that ran only in `self-contained` packs, under seven + fixed names, and said nothing when it found none; that mode still looks for + the default files beside the shared objects it bundles that no package + declares, and names those without (design 2026-11-16 §4.3). ### Changed diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 28e585e00..9e0657315 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -68,6 +68,7 @@ version = "0.1.0" # Semantic version (required) standard = "c++23" # C++ standard (default c++23; can be set to c++20 / c++26) description = "My awesome app" # Description (optional) license = "MIT" # License (optional) +license_files = ["LICENSE", "ThirdPartyNotices.txt"] # License and notice files (optional) authors = ["Alice", "Bob"] # Author list (optional) repo = "https://github.com/user/myapp" # Repository URL (optional) ``` @@ -78,6 +79,15 @@ repo = "https://github.com/user/myapp" # Repository URL (optional) - `c++20`: the lowest level mcpp accepts — named modules are a C++20 feature, so nothing below it exists for this build model. Use it when an external constraint (an older in-house rule, a third-party API that stops at C++20) forces the level down. `import std;` still works there: it is a C++23 *library* feature, but GCC (≥ 15), Clang + libc++ (≥ 17) and the MSVC STL (from VS 2022 17.8) all provide the `std` module in C++20 mode as well. Note that C++23 library facilities (`std::print`, `std::expected`, …) are not available, including in the code generated by `mcpp new`. - `c++26`: for C++26 language features. - `c++2a` / `c++2c`: compatibility aliases, normalized to `c++20` / `c++26` after parsing. + +`license_files` *(2026.11.17.1+)* names the package's license and notice files, as globs relative to +its root (the dialect of `sources`). Every `mcpp pack`, in every mode, stages the license files of +each package of the program's graph -- the project, its static and shared libraries, and the packages +whose runtime files it carries -- under `share/licenses//`, with the role `metadata`. A +package that states none uses the engine's default, `{LICENSE,LICENCE,COPYING,NOTICE}*` at its root, +so most packages need not write the key; a package whose license files lie elsewhere states them. +A package for which neither finds a file is named in the pack's report, and a declared glob that +selects nothing is warned about. A descriptor writes the same key in its `mcpp` segment. - `gnu++20` / `gnu++23` / `gnu++26`: GNU dialects; the choice enters the fingerprint and the std BMI cache key. - `c++latest`: resolves to the newest standard level the resolved toolchain supports. Good for local experimentation, but not recommended for release packages that require reproducibility. - `c++fly`: `c++latest` **plus every experimental standard feature the resolved toolchain can enable** (language + standard library). On GCC ≥ 16 this turns on C++26 reflection (`-freflection`) and contracts; on Clang/libc++ it adds `-fexperimental-library`; unsupported gates are skipped with a printed summary. Deliberately toolchain-dependent — the bleeding-edge playground mode, never for published packages. diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index aa007be5a..1b0aa9f09 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -443,6 +443,22 @@ a library, with no `--pkg-format`, is refused naming the flag and that same format set. `mcpp test` is unaffected: a test binary is a program on every row. +### The license files of the packages a program carries (2026.11.17.1+) + +Every pack stages the license and notice files of each package of the +program's graph under `share/licenses//`: the project's own, its +static libraries' (linked into the program, they carry the same obligation as +a shared one) and its shared libraries'. A package's `[package] license_files` +names them; where it states none, the engine's default +`{LICENSE,LICENCE,COPYING,NOTICE}*` at the package root applies. The files have +the role `metadata` in `mcpp-package.json`. A package for which no file is +found is named in the report (`Licenses none found for ...`). A +`self-contained` pack also looks for the default files beside each shared +object it bundles that no package declares (a toolchain's C library), under +`share/licenses//`, and names the objects for which it finds none. +`[pack] include` keeps its role: the project's other release files (a README, +an icon, a manual page). + ### The build inputs of a packed artifact, and its contents Two things differ from `mcpp build`, and both exist because a package leaves diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 44b18d5e4..ede13d92d 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -784,6 +784,11 @@ mcpp 2026.10.10.1,#785、#786;第 14 至 16 条与第 13 条的依赖位置为 m 5. 路径规则与同一目的地多个来源的规则不变:绝对路径、盘符、空段、`.`、`..` **必须**被拒绝;内容相同的来源合并为一份, 不同则在构建时拒绝(SPEC-007 R4.2)。构建程序的 `mcpp::deploy(from, to)` 以字面路径声明,协议不变。 +6. `[package] license_files` 列出包的许可证与声明文件,相对包的根,通配符方言同第 2 条;未写时取引擎的缺省值 + `{LICENSE,LICENCE,COPYING,NOTICE}*`,空列表陈述没有。打包时的放置与报告见 SPEC-014 §4.5.1。描述符在 `mcpp` 段写 + 同名的键。 +7. `[pack] include` 只表达本工程其余的发布文件,锚点是工程的根。 + **状态:已实现**(mcpp 2026.11.17.1)。 ## 变更记录 @@ -810,4 +815,4 @@ mcpp 2026.10.10.1,#785、#786;第 14 至 16 条与第 13 条的依赖位置为 m | 1.17 | 2026-10-23 | mcpp 2026.10.24.1:§2 陈述值的四档层次,`sysroot` 与 `run_under` 可出现在工程与机器默认中;§4.2 增加 `run_under`,陈述 `sysroot` 的表写法。 | | 1.18 | 2026-10-27 | mcpp 2026.10.26.2(mcpp#845):新增 §14 程序起点与镜像入口(`kind = "entry"`、`main = { from }`、`start`、`[test] main` 与 `start`、`mcpp:link-start`);§3.1.1 的按行形式可陈述起点。 | | 1.19 | 2026-10-28 | mcpp 2026.10.27.1(mcpp#847):§2 增加平台平面(`[platform]`、`[host]`),平台键可出现在工程与机器默认中;§4.2 增加 `[target.]` 的平台键与 `[host] platform`(SPEC-013)。 | -| 1.20 | 2026-11-17 | mcpp 2026.11.17.1(设计 2026-11-16 §4):新增 §15 运行时文件与发布文件:`deploy` 的条目语法(字符串、通配符、`name`)、固定目录规则与过期副本的移除。 | +| 1.20 | 2026-11-17 | mcpp 2026.11.17.1(设计 2026-11-16 §4):新增 §15 运行时文件与发布文件:`deploy` 的条目语法(字符串、通配符、`name`)、固定目录规则与过期副本的移除;`license_files` 与其缺省值。 | diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 1cd38a4af..a66f11b40 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -285,6 +285,15 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` `library`;`.oke` 为 `image`;`bin/` 下的可执行文件为 `program`;`.cmd`、`.sh` 与树根的可执行文件为 `launcher`; 其他为 `resource`。 +### 4.5.1 许可证文件 + +每次打包(全部方式,**mcpp >= 2026.11.17.1**)把程序依赖图中每个包(工程自身、静态库、共享库、带运行时文件的包) +的许可证文件放到 `share/licenses/<包>/`,角色为 `metadata`。文件由包的 `license_files`(SPEC-004 §15)给出,未写时取 +引擎的缺省值 `{LICENSE,LICENCE,COPYING,NOTICE}*`;文件名相同的两个文件保留各自相对包根的路径。没有找到文件的包**必须** +在打包报告中列出;声明的通配符没有匹配**必须**给出警告;二者都不使打包失败。`self-contained` 方式另为不属于图中任何 +包的捆绑共享对象,在其所在目录及向上三层查找缺省值的文件,放到 `share/licenses//`,找不到的**必须**列出。 +许可证文件不进入运行器的文件通道(SPEC-002 §6.2)。 + ### 4.6 合并 `mcpp pack --merge --merge ...`(也可 `--merge ,`)读入至少两个本命令写出的包(目录,或 `.zip`、 @@ -506,4 +515,5 @@ openkal 目标的单文件:载体携带一条腿的映像本身(映像不引用 | 2.2 | 2026-11-10 | mcpp 2026.11.10.1(设计 2026-11-10 E7,mcpp#911 N8):§1.2 另一系统的产物从文件读取闭包,不运行本机加载器;`--mode static` 链接全静态程序并在打包前核对。 | | 2.3 | 2026-11-12 | mcpp 2026.11.12.1(设计 2026-11-12 C6、C7):§4.4 `--systems all` 取系统表,无程序可构建的对写为未覆盖;三个系统之外的程序按 `runner.built-as` 构建;单文件按系统表的 `uname` 选择程序(SPEC-016 §2.3)。 | | 2.4 | 2026-11-14 | mcpp 2026.11.13.1:§4.4 平台定义的行即该系统的程序之行(行名可带环境段),`built-as` 指向定义的行时在平台之上构建;strip 不读的封装容器按封装结果打包。 | +| 2.6 | 2026-11-17 | mcpp 2026.11.17.1(设计 2026-11-16 §4.3):新增 §4.5.1 许可证文件,全部打包方式按包放置并报告缺失,取代只在 `self-contained` 中按固定文件名查找且不报告的做法。 | | 2.5 | 2026-11-16 | mcpp 2026.11.16.1(设计 2026-11-15 N11、N20):§4.3 携带运行器的包不回落到 `PATH`,本机没有时以 126 退出并写出所含各项;`` 取系统表的 `uname`;`--all-programs` 的树中每个程序有自己的启动器;§6 `mcpp::pack_program` 与提供者不提交的原因(N30)。 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index c3e8d257f..98b238b56 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -68,6 +68,7 @@ version = "0.1.0" # Semantic version (required) standard = "c++23" # C++ standard (default c++23; can be set to c++20 / c++26) description = "My awesome app" # Description (optional) license = "MIT" # License (optional) +license_files = ["LICENSE", "ThirdPartyNotices.txt"] # 许可证与声明文件(可选) authors = ["Alice", "Bob"] # Author list (optional) repo = "https://github.com/user/myapp" # Repository URL (optional) ``` @@ -88,6 +89,12 @@ repo = "https://github.com/user/myapp" # Repository URL (optional) BMI 的缓存键。 - `c++latest`:解析为所用工具链支持的最新标准档位。适合本地实验,但不 推荐用在要求可复现的发布包上。 + +`license_files` *(2026.11.17.1+)* 列出包的许可证与声明文件,是相对包根的通配符(方言同 `sources`)。每次 `mcpp pack` +在每种方式下,都把程序依赖图中每个包——工程自身、它的静态库与共享库、它携带其运行时文件的包——的许可证文件放到 +`share/licenses/<包>/`,角色为 `metadata`。未写这个键的包取引擎的缺省值:包根下的 `{LICENSE,LICENCE,COPYING,NOTICE}*`, +所以多数包不必写;许可证文件在别处的包写明它们。两者都找不到文件的包在打包报告中列出,声明的通配符没有匹配时给出警告。 +描述符在 `mcpp` 段写同名的键。 - `c++fly`:`c++latest` **加上所用工具链能启用的每一项实验性标准特性** (语言特性加标准库)。在 GCC ≥ 16 上会打开 C++26 反射(`-freflection`) 与 contracts;在 Clang/libc++ 上会加上 `-fexperimental-library`;工具链 diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index b5aa45b1e..01e5889d7 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -379,6 +379,15 @@ mcpp run --target aarch64-ios-sim --pkg-format app 拒绝,点名该 flag 与同一个格式集合。`mcpp test` 不受影响:测试二进制在每一行 上都是一个程序。 +### 程序所携带各包的许可证文件(2026.11.17.1+) + +每次打包都把程序依赖图中每个包的许可证与声明文件放到 `share/licenses/<包>/`:工程自身的、静态库的(链入程序后与共享库 +有同样的义务)与共享库的。包的 `[package] license_files` 给出这些文件;未写时取包根下引擎的缺省值 +`{LICENSE,LICENCE,COPYING,NOTICE}*`。这些文件在 `mcpp-package.json` 中的角色为 `metadata`。找不到文件的包在报告中列出 +(`Licenses none found for ...`)。`self-contained` 打包还为它捆绑的、不属于任何包的共享对象(工具链的 C 库)在旁边查找 +缺省的文件,放到 `share/licenses//`,并列出找不到的对象。`[pack] include` 的角色不变:工程其余的发布文件 +(README、图标、手册页)。 + ### 打包产物的构建输入,以及随包内容 与 `mcpp build` 相比有两点不同,都源于同一个原因:「这个产物要离开本机」。 diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index db13c0d11..29f2baa4d 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -1104,6 +1104,7 @@ std::expected parse_document(const t::Document& documen if (auto v = doc->get_string("package.description")) m.package.description = *v; if (auto v = doc->get_string("package.license")) m.package.license = *v; + if (auto v = doc->get_string_array("package.license_files")) m.package.licenseFiles = *v; if (auto v = doc->get_string("package.repo")) m.package.repo = *v; if (auto v = doc->get_string_array("package.authors")) m.package.authors = *v; if (auto v = doc->get_string_array("package.platforms")) m.package.platforms = *v; @@ -1846,7 +1847,7 @@ std::expected parse_document(const t::Document& documen // MUST stay in sync with the `doc->get_*("package.")` reads above. static constexpr std::string_view kKnownPackageKeys[] = { "accelerators", "authors", "c-environment", "description", "exclusive", - "license", "mcpp", "metadata", "name", "namespace", "platforms", "provides", + "license", "license_files", "mcpp", "metadata", "name", "namespace", "platforms", "provides", "repo", "requires", "requires_abi", "standard", "std-compat-module", "std-module", "std-module-flags", "version", }; diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 7b05e60da..8ae441dae 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -74,6 +74,10 @@ struct Package { bool virtualRoot = false; std::string description; std::string license; + // `license_files` (design 2026-11-16 §4.3): the package's license and + // notice files, globs relative to its root, staged by every pack under + // `share/licenses//`. Unset: `default_license_files()`. + std::optional> licenseFiles; std::vector authors; std::string repo; // `mcpp = ">=V"`: the oldest mcpp release this package supports, stored as @@ -1389,6 +1393,14 @@ struct RuntimeArtifact { std::string hostFingerprint; }; +// THE LICENSE FILES OF A PACKAGE THAT STATES NONE: engine data, the files a +// package root conventionally holds (design 2026-11-16 §4.3, after PEP 639's +// default). A package states `license_files` to replace it. +inline const std::vector& default_license_files() { + static const std::vector patterns{"{LICENSE,LICENCE,COPYING,NOTICE}*"}; + return patterns; +} + // `runtime.deploy` (#615): a file placed in a DIRECTORY RELATIVE TO THE // EXECUTABLE, which `deploy_files` cannot express, because it flattens every // entry into `bin/`. The Vulkan loader on macOS reads its driver diff --git a/modules/manifest/src/xpkg.cppm b/modules/manifest/src/xpkg.cppm index efa8b55d2..02681bc63 100644 --- a/modules/manifest/src/xpkg.cppm +++ b/modules/manifest/src/xpkg.cppm @@ -273,6 +273,7 @@ inline constexpr std::string_view kKnownXpkgKeys[] = { "linux", "macosx", "modules", "provides", "requires", "runtime", "scan_overrides", "schema", "sources", "target_cfg", "targets", "windows", + "license_files", // The per-platform blocks of the rows that read their own (design // 2026-10-29 §5.1): the Web, Apple's phones, ok images, bare metal. "android", "emscripten", "ios", "openkal", "none", @@ -1470,6 +1471,23 @@ synthesize_from_xpkg_lua(std::string_view luaContent, } cur.consume('}'); } + else if (key == "license_files") { + // design 2026-11-16 §4.3: the package's license files, globs + // relative to its root. + if (!cur.consume('{')) { + return std::unexpected(ManifestError{ + "expected '{' after `license_files =`", m.sourcePath, 0, 0}); + } + std::vector files; + cur.skip_ws_and_comments(); + while (!cur.eof() && cur.peek() != '}') { + auto s = cur.read_string(); + if (!s.empty()) files.push_back(std::move(s)); + cur.skip_ws_and_comments(); + } + cur.consume('}'); + m.package.licenseFiles = std::move(files); + } else if (key == "sources") { if (!cur.consume('{')) { return std::unexpected(ManifestError{ diff --git a/src/build/plan.cppm b/src/build/plan.cppm index c1f753f7c..bf3f61bb1 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -441,6 +441,16 @@ struct TestFramework { std::vector session; }; +// A package's license files, as a pack stages them (design 2026-11-16 §4.3): +// the package as its directory under `share/licenses/` names it, its root, and +// its `license_files` or the engine's default. +struct LicenseSource { + std::string package; + std::filesystem::path root; + std::vector patterns; + bool declared = false; +}; + struct BuildPlan { mcpp::manifest::Manifest manifest; // Every package of the graph, as the report names it; the backend writes @@ -456,6 +466,9 @@ struct BuildPlan { // `-L` (openkal-musl's empty archives), and a path dependency's root can // be anywhere on the machine, not only under the store. std::vector packageRoots; + // Every package of the graph with its license files (§4.3), the root's + // first: the program's link and run closure, whatever its linkage. + std::vector licenseSources; // THE PATHS AN ARTIFACT CARRIES, MAPPED (`trim_paths`; design 2026-10-23 // §15.3, mcpp#832): each package's root to `@`, and the // store, the home, the build directory, the toolchain and the target base @@ -2044,6 +2057,14 @@ make_plan(const mcpp::manifest::Manifest& manifest, plan.toolchain = tc; plan.fingerprint = fp; for (auto const& p : packages) plan.packageRoots.push_back(p.root); + for (auto const& p : packages) { + auto const& pkg = p.manifest.package; + if (pkg.virtualRoot) continue; + plan.licenseSources.push_back({ + pkg.namespace_.empty() ? pkg.name : pkg.namespace_ + "." + pkg.name, p.root, + pkg.licenseFiles.value_or(mcpp::manifest::default_license_files()), + pkg.licenseFiles.has_value()}); + } if (manifest.buildConfig.trimPaths) plan.trimMap = trim_map(packages, tc, outputDir, storeRoots); diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 3fc913e59..c7d3bd359 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -51,6 +51,7 @@ import mcpp.pack.polyglot; import mcpp.pack.compress; import mcpp.pack.distribution; import mcpp.modgraph.glob; // [pack] include / exclude +import mcpp.modgraph.scanner; // license_files globs (design 2026-11-16 §4.3) import mcpplibs.os.platform; import mcpp.toolchain.platform; import mcpp.invocation; @@ -314,6 +315,17 @@ struct Plan { // From manifest [pack] std::vector includeGlobs; std::vector excludeGlobs; + // THE LICENSE FILES OF EVERY PACKAGE OF THE PROGRAM'S CLOSURE (design + // 2026-11-16 §4.3), set by the caller from the build plan: each package's + // `license_files`, or the engine's default, staged under + // `share/licenses//` by every mode. + struct LicenseSource { + std::string package; + std::filesystem::path root; + std::vector patterns; + bool declared = false; + }; + std::vector licenseSources; std::vector alsoSkipLibs; std::vector hostProvidedLibs; // Options::hostProvidedLibs std::vector forceBundleLibs; @@ -1360,40 +1372,52 @@ std::string find_loader_soname(const std::vector& deps) { return {}; } -// Walk each bundled .so's parent directory looking for a license file -// (COPYING / LICENSE / LICENSE.txt etc.) and copy it under -// /share/licenses//. Best-effort: silently skips -// libs without a discoverable license file (the user will see those -// gaps when they audit the bundle). -void collect_licenses(const std::vector& deps, - const std::filesystem::path& stagingRoot) +// The files `patterns` (license_files globs) select under `root`, each with +// the name it takes in its package's license directory: its own file name, or +// its path below `root` where two selected files share a name. +std::vector> +license_files_of(const std::filesystem::path& root, const std::vector& patterns) { + std::vector files; + std::set seen; + for (auto const& g : patterns) + for (auto const& f : mcpp::modgraph::expand_glob(root, g)) + if (seen.insert(f.lexically_normal()).second) files.push_back(f.lexically_normal()); + std::map names; + for (auto const& f : files) ++names[f.filename().string()]; + std::vector> out; + for (auto const& f : files) + out.push_back({f, names[f.filename().string()] > 1 ? f.lexically_relative(root) : f.filename()}); + return out; +} + +// THE LICENSES OF THE SHARED OBJECTS A SELF-CONTAINED PACK BUNDLES THAT NO +// PACKAGE OF THE GRAPH DECLARES (a toolchain's C library, its C++ runtime): +// the engine's default license files in the object's directory or up to three +// above it, under `share/licenses//`. Returns the objects for which +// none was found, which the pack reports. +std::vector collect_bundled_licenses(const std::vector& deps, + const std::filesystem::path& stagingRoot) { - static constexpr std::array kLicenseNames = { - "LICENSE", "LICENSE.txt", "LICENCE", "COPYING", - "COPYING.LIB", "COPYRIGHT", "NOTICE", - }; + std::vector without; std::error_code ec; - auto outRoot = stagingRoot / "share" / "licenses"; + const auto outRoot = stagingRoot / "share" / "licenses"; for (auto& d : deps) { - auto libDir = std::filesystem::path(d.path).parent_path(); - // Walk up at most 3 levels to catch e.g. /usr/lib/x86_64-linux-gnu - // → /usr/share/doc/libfoo/copyright (Debian convention). - for (int up = 0; up < 3; ++up) { - for (auto& name : kLicenseNames) { - auto cand = libDir / name; - if (std::filesystem::exists(cand, ec)) { - auto dst = outRoot / d.soname / name; - std::filesystem::create_directories(dst.parent_path(), ec); - std::filesystem::copy_file(cand, dst, - std::filesystem::copy_options::overwrite_existing, ec); - goto next; - } + auto dir = std::filesystem::path(d.path).parent_path(); + bool found = false; + for (int up = 0; up < 4 && !found && !dir.empty(); ++up) { + for (auto const& [src, name] : license_files_of(dir, mcpp::manifest::default_license_files())) { + if (!std::filesystem::is_regular_file(src, ec)) continue; + const auto dst = outRoot / d.soname / name; + std::filesystem::create_directories(dst.parent_path(), ec); + std::filesystem::copy_file(src, dst, std::filesystem::copy_options::overwrite_existing, ec); + found = true; } - libDir = libDir.parent_path(); - if (libDir.empty() || libDir == "/") break; + if (dir == dir.root_path()) break; + dir = dir.parent_path(); } - next: continue; + if (!found) without.push_back(d.soname); } + return without; } void copy_if_exists(const std::filesystem::path& src, @@ -1655,6 +1679,43 @@ std::expected, Error> stage_included(const Plan& plan) return staged; } +// `license_files` of every package of the closure (design 2026-11-16 §4.3), +// under `share/licenses//`, in every mode. A package whose declared +// globs select nothing is warned about; packages with no declaration whose +// default finds nothing are listed in one line. Neither stops the pack. +std::expected stage_licenses(const Plan& plan) { + namespace fs = std::filesystem; + std::vector silent; + std::set done; + for (auto const& src : plan.licenseSources) { + if (!done.insert(src.package).second) continue; + const auto files = license_files_of(src.root, src.patterns); + if (files.empty()) { + if (src.declared) + mcpp::ui::warning(std::format("{}: license_files selects no file under {}", src.package, + src.root.generic_string())); + else + silent.push_back(src.package); + continue; + } + std::error_code ec; + for (auto const& [from, name] : files) { + const auto to = plan.stagingRoot / "share" / "licenses" / src.package / name; + fs::create_directories(to.parent_path(), ec); + fs::copy_file(from, to, fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format("cannot stage the license file {}: {}", + from.string(), ec.message())}); + } + } + if (!silent.empty()) { + std::string names; + for (auto const& n : silent) names += (names.empty() ? "" : ", ") + n; + mcpp::ui::info("Licenses", std::format("none found for {} (no license_files, and no {} at the root)", + names, mcpp::manifest::default_license_files().front())); + } + return {}; +} + std::expected finalize_tree(const Plan& plan) { namespace fs = std::filesystem; namespace d = mcpp::pack::distribution; @@ -1662,6 +1723,7 @@ std::expected finalize_tree(const Plan& plan) { if (!fs::is_directory(plan.stagingRoot, ec)) return {}; auto included = stage_included(plan); if (!included) return std::unexpected(included.error()); + if (auto r = stage_licenses(plan); !r) return std::unexpected(r.error()); const auto canonical = [](std::string const& t) { if (t.empty()) return mcpp::toolchain::triple::host_triple().str(); auto parsed = mcpp::toolchain::triple::parse(t); @@ -3489,7 +3551,12 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) if (auto r = write_bundle_all_wrappers(plan.stagingRoot, plan.binaryName, loader, localeData, gconvData); !r) return std::unexpected(Error{r.error()}); - collect_licenses(toBundle, plan.stagingRoot); + if (auto without = collect_bundled_licenses(toBundle, plan.stagingRoot); !without.empty()) { + std::string names; + for (auto const& n : without) names += (names.empty() ? "" : ", ") + n; + mcpp::ui::info("Licenses", std::format( + "no license file found beside the bundled {}", names)); + } } } else { // Static / BundleProject: drop a thin shell wrapper at the diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 854be48b0..cdb7cac05 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -1143,6 +1143,8 @@ std::optional stage_member(PackRun& run, GroupJob& g, MemberJob& m) { // parameter comment on `extraLegs` above. A no-op (default-constructed, // empty) for every caller before this item. plan->extraSharedLegs = std::move(run.extraLegs); + for (auto const& l : ctx.plan.licenseSources) + plan->licenseSources.push_back({l.package, l.root, l.patterns, l.declared}); // #649 E5: the shared libraries this graph built, primary leg. From the // plan's link units, never from a directory listing: a vendor library // deployed beside the program is not one of them and stays as shipped. diff --git a/tests/e2e/pack/modes.cpp b/tests/e2e/pack/modes.cpp index d7efcee02..857b72377 100644 --- a/tests/e2e/pack/modes.cpp +++ b/tests/e2e/pack/modes.cpp @@ -312,6 +312,34 @@ deploy = [ { from = "share/vulkan/icd.d/lvp_icd.json", to = "vulkan/icd.d" } ] const auto notes = *root / "bin/notes.txt"; expect(fs::exists(notes) && e2e::contains(s::read_file(notes), "notes")) << "runtime.deploy_files did not reach bin/ in the bundle\n" << s::text(r); }; + "1103 every package's license files reach share/licenses"_test = [] { + // Design 2026-11-16 §4.3: each package of the closure stages its `license_files`, or the + // engine's default {LICENSE,LICENCE,COPYING,NOTICE}*, under share/licenses// with + // the role `metadata`, whatever the linkage; a package with none is named in the report. + const auto tmp = s::case_dir(); + s::project a(tmp / "a", {{"mcpp.toml", "[package]\nname = \"a\"\nversion = \"0.1.0\"\nlicense_files = [\"LICENSE.md\", \"third/*-NOTICE.txt\"]\n\n[targets.a]\nkind = \"lib\"\n"}, + {"src/a.cppm", "export module a;\nexport int av() { return 1; }\n"}, + {"LICENSE.md", "A\n"}, {"third/x-NOTICE.txt", "x\n"}}); + s::project b(tmp / "b", {{"mcpp.toml", "[package]\nname = \"b\"\nversion = \"0.1.0\"\n\n[targets.b]\nkind = \"lib\"\n"}, + {"src/b.cppm", "export module b;\nexport int bv() { return 2; }\n"}, {"COPYING", "B\n"}}); + s::project c(tmp / "c", {{"mcpp.toml", "[package]\nname = \"c\"\nversion = \"0.1.0\"\n\n[targets.c]\nkind = \"lib\"\n"}, + {"src/c.cppm", "export module c;\nexport int cv() { return 3; }\n"}}); + s::project app(tmp / "app", {{"mcpp.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[dependencies]\na = { path = \"../a\" }\nb = { path = \"../b\" }\nc = { path = \"../c\" }\n"}, + {"src/main.cpp", "import a;\nimport b;\nimport c;\nint main() { return av() + bv() + cv() - 6; }\n"}, + {"LICENSE", "app\n"}}); + auto r = app.mcpp({"pack", "--pkg-format", "dir"}); + expect(fatal(e2e::succeeded(r))) << s::text(r); + std::optional root; + for (const auto& e : entries(app.at("target/dist"))) + if (!root && fs::is_directory(e) && e.filename().string().starts_with("app-0.1.0-")) root = e; + expect(fatal(root.has_value())) << "no staged tree\n" << s::text(r); + for (auto rel : {"share/licenses/app/LICENSE", "share/licenses/mcpplibs.a/LICENSE.md", + "share/licenses/mcpplibs.a/x-NOTICE.txt", "share/licenses/mcpplibs.b/COPYING"}) + expect(fatal(fs::is_regular_file(*root / rel))) << rel << " is not in the tree\n" << s::text(r); + expect(fatal(!fs::exists(*root / "share/licenses/mcpplibs.c"))) << "a package with no license has a directory"; + expect(fatal(e2e::contains(s::text(r), "none found for mcpplibs.c"))) << "the package with none is not named\n" << s::text(r); + expect(fatal(e2e::contains(s::read_file(*root / "mcpp-package.json"), "\"share/licenses/mcpplibs.b/COPYING\""))) << "the license is not described"; + }; "652 an application is a kind"_test[s::elf][s::gcc] = [] { // `kind = "app"` (#622 A3): an unrecognized kind is refused listing every kind; on an // ELF host an app links as an executable beside a bin; `mcpp run` runs either and still diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 86e6827aa..a150c4b7c 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -5904,6 +5904,25 @@ deploy = [ expect(m->schemaWarnings.empty()); }}; +// Design 2026-11-16 §4.3: `license_files` is the package's; unset means the +// engine's default, an empty list states that there is none. +UTest package_license_files_are_read{"Manifest/PackageLicenseFilesAreRead", [] { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "ort" +version = "1.0.0" +license_files = ["LICENSE", "ThirdPartyNotices.txt"] +)"); + if (auto&& failed = expect(fatal(m.has_value())); !failed) failed << m.error().format(); + expect(fatal(m->package.licenseFiles.has_value())); + expect(eq(m->package.licenseFiles->size(), 2u)); + expect(m->schemaWarnings.empty()); + auto none = mcpp::manifest::parse_string("[package]\nname = \"x\"\nversion = \"1.0.0\"\n"); + expect(fatal(none.has_value())); + expect(!none->package.licenseFiles.has_value()); + expect(eq(mcpp::manifest::default_license_files().front(), std::string("{LICENSE,LICENCE,COPYING,NOTICE}*"))); +}}; + // ── `requires_abi` (design 2026-09-12, section 5.2) ──────────────────────── UTest requires_abi_threads_parses_on_the_package_and_on_a_feature{"Manifest/RequiresAbiThreadsParsesOnThePackageAndOnAFeature", [] { diff --git a/tests/unit/test_runtime_contract.cpp b/tests/unit/test_runtime_contract.cpp index 550a0e09e..745211891 100644 --- a/tests/unit/test_runtime_contract.cpp +++ b/tests/unit/test_runtime_contract.cpp @@ -504,6 +504,25 @@ package = { expect(eq(parsed->runtimeConfig.linkIntent.deployFiles, std::vector{"bin/backend.dll"})); }}; +// Design 2026-11-16 §4.3 in a descriptor, in the segment every system shares. +UTest xpkg_reads_license_files{"RuntimeContract/XpkgReadsLicenseFiles", [] { + constexpr auto lua = R"( +package = { + spec = "1", namespace = "compat", name = "ort", + xpm = { linux = { ["1.0.0"] = { url = "u", sha256 = "h" } } }, + mcpp = { + license_files = { "onnxruntime-*/{LICENSE,ThirdPartyNotices.txt}" }, + sources = { "src/x.cpp" }, + }, +} +)"; + auto parsed = mf::synthesize_from_xpkg_lua( + lua, "ort", "1.0.0", mcpp::toolchain::platform::HostPlatform::current()); + if (auto&& failed = expect(fatal(bool(parsed))); !failed) failed << parsed.error().format(); + expect(fatal(parsed->package.licenseFiles.has_value())); + expect(eq(parsed->package.licenseFiles->front(), std::string("onnxruntime-*/{LICENSE,ThirdPartyNotices.txt}"))); +}}; + // The expansion of one entry: a match keeps its path below the glob's fixed // directory, a literal path lands in `to`, braces are one glob, `name` renames // the one match, and a glob that matches nothing or a `name` over several is From 4c439c2575299fedc3ef546c82db49a4a11eb1af Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 19:29:40 +0900 Subject: [PATCH 11/36] run files: mcpp run --file, [test] files, and the run's working directory MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit mcpp run --file FROM[=TO] (repeatable, dirun's spelling) and [test] files (the deploy entry grammar, with [target.''.test] files) name the files a run finds in its working directory. With them, the program runs in run// of the build tree, into which they are copied (never linked): mcpp run waits for the program and removes the directory; a test program's directory is shared by its split parts, removed when it passes, kept and named when it fails. The directory is prepared before the launch is composed, so --print-launch and a test's launch name a directory that holds the files. Without run files nothing changes. A runner receives them as run_files in MCPP_RUN_CLOSURE (schema 2) and in MCPP_RUNTIME_FILES beside the runtime files, which is now derived as the union: where the program has a file system of its own, its directory is its working directory. Both fast paths decline when run files are named. SPEC-002 §6.2 and §6.2.1 (v1.9), SPEC-017 B1.6 and B5 (2.4); docs/04, docs/08 and docs/09 in both languages; unit test for [test] files; e2e run/contract 1104 and test/selection 1105; 927 reads schema 2. --- CHANGELOG.md | 10 ++ docs/04-mcpp-toml.md | 3 +- docs/08-testing.md | 25 ++++ docs/09-commands-by-scenario.md | 19 +++ docs/specs/target-side.md | 19 ++- docs/specs/test.md | 8 +- docs/zh/04-mcpp-toml.md | 3 +- docs/zh/08-testing.md | 18 +++ docs/zh/09-commands-by-scenario.md | 14 ++ modules/manifest/src/toml.cppm | 112 +++++++++------ modules/manifest/src/types.cppm | 9 +- src/build/execute.cppm | 222 +++++++++++++++++++++++++++-- src/build/test_targets.cpp | 17 +++ src/build/test_targets.cppm | 6 + src/cli.cppm | 5 + src/cli/cmd_build.cppm | 14 ++ tests/e2e/run/contract.cpp | 39 ++++- tests/e2e/test/selection.cpp | 29 +++- tests/unit/test_manifest.cpp | 27 +++- 19 files changed, 530 insertions(+), 69 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c10e9573..42d73d654 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,6 +28,16 @@ fixed names, and said nothing when it found none; that mode still looks for the default files beside the shared objects it bundles that no package declares, and names those without (design 2026-11-16 §4.3). +- `mcpp run --file FROM[=TO]` (repeatable; dirun's spelling) and `[test] + files` (in the `deploy` entry grammar, with `[target.''.test] files`) + name the files a run finds in its working directory. With them, a program + runs in a directory of the build tree, `run//`, into which they are + copied: removed after `mcpp run`, removed after a test program that passed + and kept, and named, after one that failed. Without them, a program runs + where mcpp was started, as before. A runner receives them as `run_files` in + `MCPP_RUN_CLOSURE` (schema 2) and in `MCPP_RUNTIME_FILES` beside the runtime + files, so a runner whose program has a file system of its own delivers them + beside the program (design 2026-11-16 §5, §7). ### Changed diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 9e0657315..5e52f7fd2 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1552,7 +1552,8 @@ and `run_under`. A runner reached this way receives `MCPP_RUN_TRIPLE`, `MCPP_RUN_KIND` (`exe` / `bundle`), `MCPP_RUN_MODE` (`run` / `test`), `MCPP_RUN_ID`, `MCPP_RUN_CLOSURE` -(a versioned JSON naming the artifact, the files it carries and the base) and, +(a versioned JSON naming the artifact, the files it carries, the run's files +(`run_files`, schema 2, 2026.11.17.1+) and the base) and, when a base is stated, `MCPP_SYSROOT`. Its exit status is the program's. An Apple artifact on a host that is not macOS, with no `run_under`, is refused before anything is spawned, naming this key. diff --git a/docs/08-testing.md b/docs/08-testing.md index 90a872813..d39f100a0 100644 --- a/docs/08-testing.md +++ b/docs/08-testing.md @@ -186,6 +186,31 @@ not one of its tests: `mcpp test --list --target ` names the set, and a case that is built everywhere and holds on some machines only is the test protocol's (a skip with its reason), not this table's. +### A test's run files (2026.11.17.1+) + +`[test] files` names the files a test program finds in its working directory, +in the entry grammar of `[runtime] deploy`: a path, or `{ from, to, name }`, +with `to` and `name` relative to that directory. A +`[target.''.test] files` table adds files for the targets it selects. +Not the test's sources, which `discover` names: + +```toml +[test] +files = ["tests/data/**", { from = "tests/t.bat", name = "TEST.BAT" }] + +[target.'cfg(os = "uefi")'.test] +files = ["tests/uefi/**"] +``` + +Each test program then runs in a directory of the build tree, +`run/test-/`, into which the files are copied before it starts; the +processes a split divides it into share it. The directory of a program that +passed is removed, and that of one that failed is kept and named in the report +(`Kept the working directory of ...`). A runner whose program has a file system +of its own (a virtual machine, firmware) delivers the files beside the program, +as it delivers the runtime files. A package without `[test] files` runs its +tests where `mcpp test` was started, as before. + The tests are a consumer of their package, and `[test] features` is that consumer's request: diff --git a/docs/09-commands-by-scenario.md b/docs/09-commands-by-scenario.md index 220e1e3aa..90908d3d4 100644 --- a/docs/09-commands-by-scenario.md +++ b/docs/09-commands-by-scenario.md @@ -489,6 +489,25 @@ test` and `mcpp pack` keep their statuses. A program, or a runner failed build; [50 — Machine-Readable Output](50-machine-output.md) §6 gives the bands. +### The files of one run (`--file`, 2026.11.17.1+) + +`--file FROM[=TO]` (repeatable) copies FROM, a file or a directory relative to +where `mcpp run` is started, into the run's working directory as TO, which is +FROM's name when omitted; it is dirun's `--file HOST=GUEST`. With it, the +program runs in a directory of the build tree, `run//`, removed after the +program ends, so a relative path after `--` names a file there, not in the +directory `mcpp run` was started from; `-v` writes the directory's path. A +runner receives the files with the program's runtime files +(`MCPP_RUN_CLOSURE`'s `run_files`, `MCPP_RUNTIME_FILES`), and one whose program +has a file system of its own delivers them beside the program: + +```sh +mcpp run --target x86_64-uefi --file tests/t.bat=TEST.BAT -- /c TEST.BAT +``` + +With `--file`, `mcpp` waits for the program in order to remove the directory, +rather than being replaced by it. + ### The program and the terminal *(2026.10.5.1+)* The program `mcpp run` starts owns the terminal as it would if the shell had diff --git a/docs/specs/target-side.md b/docs/specs/target-side.md index a6db8ce28..3da8d0c9d 100644 --- a/docs/specs/target-side.md +++ b/docs/specs/target-side.md @@ -310,12 +310,26 @@ hosted 与否是 C 环境的一个声明:提供 `hosted-standard-library` 的 sy | `MCPP_RUN_KIND` | `exe`;产物是目录时为 `bundle` | | `MCPP_RUN_MODE` | `run` 或 `test` | | `MCPP_RUN_ID` | 每次执行唯一;运行器可能被并发调用 | -| `MCPP_RUN_CLOSURE` | 运行闭包清单的路径:带 `schema` 版本的 JSON,含产物、它从自身目录加载或读取的文件、基座的根 | +| `MCPP_RUN_CLOSURE` | 运行闭包清单的路径:带 `schema` 版本的 JSON,含产物、它从自身目录加载或读取的文件(`files`)、本次运行的工作目录中的运行文件(`run_files`,schema 2)、基座的根 | | `MCPP_SYSROOT` | 被陈述的目标基座的根,没有时不设置 | -| `MCPP_RUNTIME_FILES` | 既有的文件清单(TSV),保留 | +| `MCPP_RUNTIME_FILES` | 由闭包派生的文件清单(TSV):`files` 与 `run_files` 的并集,保留 | 运行器**可以**包装运行器,argv **必须**原样传递;运行器的退出码即程序的退出码。工具专属的变量 (例如 `QEMU_LD_PREFIX`)**禁止**由引擎设置,由对应运行器的包从 `MCPP_SYSROOT` 转换(SPEC-006 §3.11)。 +`MCPP_RUN_CLOSURE` 是规范通道;`MCPP_RUNTIME_FILES` 是它的派生,条目为 `<目的地>\t<来源的绝对路径>`。程序有自己文件系统 +的执行环境(虚拟机、固件、设备)中,程序所在目录即其工作目录,所以投递文件的运行器把两者一起放到程序旁。许可证文件 +(SPEC-014 §4.5.1)不进入这两个通道(mcpp >= 2026.11.17.1)。 + +### 6.2.1 工作目录 + +运行文件是一次运行开始时工作目录中应有的文件:`mcpp run --file FROM[=TO]`,与 `[test] files`(SPEC-017 B1)。 + +1. 没有运行文件时,程序的工作目录是 mcpp 被启动时的目录,与此前相同。 +2. 有运行文件时,工作目录是构建树中的 `run/<标识>/`:运行前清空并复制运行文件(复制,**禁止**硬链接与符号链接, + 程序写入时不得改到来源),运行器与原生程序都在其中启动。`mcpp run` 一次运行一个目录,运行结束后删除。`mcpp test` + 每个测试程序一个目录,`split` 分出的进程共用;程序通过后删除,失败时保留并在报告中写出路径。 +3. 工作目录在写出 `launch` 记录(SPEC-017 B5)之前备好,所以只描述而不启动的运行,其 `launch.cwd` 指向的目录已备好。 +4. 运行器的声明不因此新增键(SPEC-013 §10)(mcpp >= 2026.11.17.1)。 ### 6.3 物理边界 @@ -395,4 +409,5 @@ hosted 与否是 C 环境的一个声明:提供 `hosted-standard-library` 的 sy | v1.6 | 2026-11-10 | 随 mcpp 2026.11.10.1:新增 §2.5 基座包(按普通包读取:`provides` 回答层,`[build]` 陈述层,包根即 sysroot;新键 `compiler_runtime_dir`)与 §3.8 层供给者的需求不成环(`layer-cycle`)。 | | v1.7 | 2026-11-10 | 随 mcpp 2026.11.10.2:§3.4 写明段与 C 库名的对应,MSVC 载荷的 `c-abi` 回答 `ucrt`。 | | v1.2 | 2026-10-23 | 随 mcpp 2026.10.24.1:§1.3 `prebuilt` 放宽为被点名、按内容识别的预制目录;§2.4 改为由测量回答、`provides` 声明或核对,标为已实现;规则一增加基座是一个供给者与 c-abi 独占律;规则二增加基座上的编译器运行库与可测量的下限;规则三增加可叠加层;新增 §6 目标侧执行(组合、运行器契约、物理边界)。 | +| v1.9 | 2026-11-17 | 随 mcpp 2026.11.17.1(设计 2026-11-16 §5、§7):§6.2 闭包的 `run_files`(schema 2)与 `MCPP_RUNTIME_FILES` 为派生的并集;新增 §6.2.1 工作目录。 | | v1.8 | 2026-11-12 | 随 mcpp 2026.11.12.1:§2.5 基座包只有一种读法(带清单即包根 sysroot,布局只对无清单的目录探测)。 | diff --git a/docs/specs/test.md b/docs/specs/test.md index 7412a801e..cc7d7ae17 100644 --- a/docs/specs/test.md +++ b/docs/specs/test.md @@ -52,7 +52,10 @@ 3. 程序名是其路径相对于首个匹配 glob 的固定目录、去掉扩展名的部分。同名的两个文件被拒绝,并写出两者。 4. `[test] main = { from = "<依赖>/<入口>" }` 为每个程序提供起点;`[test] start` 为每个测试镜像陈述入口符号;`windows_code_page`、`windows_long_paths` 与同名目标键含义相同。 5. `[test] split` 与 `[test] store` 分别属于 O1 与 B4;`[test] link` 规定测试链接其所达(`"used"`,缺省)或全部目标文件(`"objects"`)。 -6. `[test]` 中的其他键产生警告,在 `--strict` 下为错误。 +6. `[test] files` 列出每个测试程序的工作目录中应有的文件,条目语法与 `[runtime] deploy` 相同(SPEC-004 §15),`to` 与 + `name` 相对于工作目录;`[target.<选择器>.test] files` 为该选择器选中的目标追加文件。工作目录的规则见 SPEC-002 §6.2.1 + (mcpp >= 2026.11.17.1)。 +7. `[test]` 中的其他键产生警告,在 `--strict` 下为错误。 ### B2. 协议核心 @@ -179,7 +182,7 @@ 1. `mcpp test --no-run --message-format json` 判断所选测试是否最新,只在需要时构建,并为每个程序写出状态为 `built` 的程序记录。记录**必须**带 `launch`: - `argv`:经由执行宿主或运行器的完整参数向量,含 `--` 之后的参数; - `env`:mcpp 交给程序的环境(运行时库目录、运行时环境的变量、`MCPP_BIN_`、工具目录); - - `cwd`:程序的工作目录; + - `cwd`:程序的工作目录;声明了 `[test] files` 时为已备好的目录(SPEC-002 §6.2.1); - `via`:`run_under`、`runner` 或空。 2. 协议的变量(P2.1)与所选用例属于启动者自己,`launch` 中不含。 3. 构建状态与构建目录的锁只在 mcpp 一处。启动者(包括 O9 的会话)**禁止**自行构建测试程序;它请求 B5,并按 `launch` 启动。 @@ -354,4 +357,5 @@ mcpp 以 feature(docs/50 §4)宣告它实现的条款;客户端按需门 | 2.0 | 2026-11-10 | 改为 mcpp 测试规范:合并 SPEC-017 v1.0、mtest 协议版本 1 的规范正文与 SPEC-004 的 `[test]` 键;分为基础能力 B1–B6 与可选能力 O1–O9;新增 B5 启动描述、B6 的 `test-capabilities` 与 `test-session`、O3 检查点细节、O8 输出归属、O9 测试会话(mcpp#914、mcpp#915) | | 2.1 | 2026-11-14 | P3.3 的位置与能力(`only`、`except`、`capabilities`、列为跳过时的 `reason`;mtest 0.8.0),及 mcpp 的实现 | | 2.2 | 2026-11-14 | O10 按测量调度(顺序、`split = "auto"`、`--shard`、`--failed`);B4 的 `shard` 与 `part_of` | +| 2.4 | 2026-11-17 | B1 第 6 条 `[test] files`;B5 的 `cwd` 为已备好的工作目录(mcpp 2026.11.17.1,设计 2026-11-16 §5) | | 2.3 | 2026-11-15 | P4.3 排除与按声明选择(mtest 0.9.0);O10 按声明的用例划分并以测量判定是否保留(`divides`),用例的用时按程序与标识记取 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 98b238b56..0f1d96d5b 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1421,7 +1421,8 @@ runner 之前:` <产物> <参数>`(Bazel 的 `--run_un runner 与 `run_under`。 经此到达的运行器收到 `MCPP_RUN_TRIPLE`、`MCPP_RUN_KIND`(`exe` / `bundle`)、`MCPP_RUN_MODE` -(`run` / `test`)、`MCPP_RUN_ID`、`MCPP_RUN_CLOSURE`(一份带版本的 JSON,列出产物、它随带的文件与基座), +(`run` / `test`)、`MCPP_RUN_ID`、`MCPP_RUN_CLOSURE`(一份带版本的 JSON,列出产物、它随带的文件、本次运行的文件 +(`run_files`,schema 2,2026.11.17.1+)与基座), 以及陈述了基座时的 `MCPP_SYSROOT`;其退出码即程序的退出码。非 macOS 宿主上的 Apple 产物在没有 `run_under` 时于执行前被拒绝,拒绝信息点名本键。 diff --git a/docs/zh/08-testing.md b/docs/zh/08-testing.md index 634f827c4..c7ec90295 100644 --- a/docs/zh/08-testing.md +++ b/docs/zh/08-testing.md @@ -136,6 +136,24 @@ discover = ["!tests/native/**"] # 不为 Web 构建 `[build] target`,否则是本机。在某个目标上被排除的测试不是该目标的测试:`mcpp test --list --target ` 给出这个集合。处处都能构建、只在部分机器上成立的用例属于测试协议(带原因的跳过),不属于这张表。 +### 测试的运行文件(2026.11.17.1+) + +`[test] files` 列出测试程序在其工作目录中找到的文件,条目语法与 `[runtime] deploy` 相同:一个路径,或 +`{ from, to, name }`,`to` 与 `name` 相对于该目录。`[target.''.test] files` 为它选中的目标追加文件。它不是测试的源文件, +源文件由 `discover` 给出: + +```toml +[test] +files = ["tests/data/**", { from = "tests/t.bat", name = "TEST.BAT" }] + +[target.'cfg(os = "uefi")'.test] +files = ["tests/uefi/**"] +``` + +此时每个测试程序在构建树中的 `run/test-<名字>/` 中运行,启动前文件被复制进去;`split` 分出的进程共用它。通过的程序的目录 +被删除,失败的程序的目录被保留并在报告中写出(`Kept the working directory of ...`)。程序有自己文件系统的 runner(虚拟机、 +固件)把文件放到程序旁,与运行时文件相同。没有 `[test] files` 的包,测试仍在 `mcpp test` 启动处运行。 + 测试是其包的消费者,`[test] features` 就是这个消费者的请求: ```toml diff --git a/docs/zh/09-commands-by-scenario.md b/docs/zh/09-commands-by-scenario.md index 2d6c50371..b8bf4850a 100644 --- a/docs/zh/09-commands-by-scenario.md +++ b/docs/zh/09-commands-by-scenario.md @@ -384,6 +384,20 @@ $ echo $? 无法启动的程序保留拒绝时的状态(`127` 找不到,`126` 不可执行,`125` 其他),`mcpp build`、`mcpp test`、`mcpp pack` 的状态不变。程序本身,或者 runner(`--runner`、`[target.].runner`),自己返回 101 时,读起来与构建失败相同;各区间见 [50 —— 机器可读输出](50-machine-output.md) §6。 +### 一次运行的文件(`--file`,2026.11.17.1+) + +`--file FROM[=TO]`(可重复)把 FROM(相对 `mcpp run` 启动处的文件或目录)复制到本次运行的工作目录中,名为 TO,省略时取 +FROM 的名字;写法与 dirun 的 `--file HOST=GUEST` 相同。给出它时,程序在构建树中的目录 `run/<标识>/` 中运行,程序结束后 +该目录被删除,所以 `--` 之后的相对路径指向那里,而不是 `mcpp run` 启动处;`-v` 写出该目录的路径。runner 与程序的运行时 +文件一起收到这些文件(`MCPP_RUN_CLOSURE` 的 `run_files`、`MCPP_RUNTIME_FILES`),程序有自己文件系统的 runner 把它们放到 +程序旁: + +```sh +mcpp run --target x86_64-uefi --file tests/t.bat=TEST.BAT -- /c TEST.BAT +``` + +给出 `--file` 时,`mcpp` 等待程序结束以删除该目录,而不是被程序替换。 + ### 程序与终端 *(2026.10.5.1+)* `mcpp run` 启动的程序拥有终端,与由 shell 直接启动时相同:它读取终端的输入,Ctrl-C、Ctrl-\ 与 Ctrl-Z 送达程序,由程序决定其含义。runner 与具名 runner(`--runner`)同样如此。 diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 29f2baa4d..af407dfcb 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -662,6 +662,52 @@ void locate_named_key(ManifestError& e, const t::Document& doc) { // One `requires` array, of `[package]` or of a `[target.]` (design // 2026-10-26 §6.3, D20): each entry a string, or a table +// A list of file entries (design 2026-11-16 §4.1): a string, which is +// `{ from = }`, or a `{ from, to, name }` table; `to` defaults to ".". +// `where` names the key for the messages (`runtime.deploy`, `[test] files`). +std::expected, std::string> +read_file_entries(const t::Value& v, std::string_view where) { + if (!v.is_array()) + return std::unexpected(std::format( + "{} must be an array of paths or `{{ from = \"...\", to = \"...\" }}` tables", where)); + std::vector out; + std::size_t index = 0; + for (auto const& value : v.as_array()) { + ++index; + std::string from, to = ".", name; + bool named = false; + if (value.is_string()) { + from = value.as_string(); + } else if (value.is_table()) { + auto const& table = value.as_table(); + for (auto const& [key, kv] : table) { + if (key != "from" && key != "to" && key != "name") + return std::unexpected(std::format( + "{}[{}] has unsupported key '{}'; the keys are `from`, `to` and `name`", + where, index, key)); + if (!kv.is_string()) + return std::unexpected(std::format( + "{}[{}]: `from`, `to` and `name` must be strings", where, index)); + (key == "from" ? from : key == "to" ? to : name) = kv.as_string(); + named = named || key == "name"; + } + if (!table.contains("from")) + return std::unexpected(std::format("{}[{}] has no `from`", where, index)); + } else { + return std::unexpected(std::format("{}[{}] must be a path or a table with `from`", where, index)); + } + if (auto p = deploy_path_problem("from", from, false); !p.empty()) + return std::unexpected(std::format("{}[{}]: {}", where, index, p)); + if (auto p = deploy_path_problem("to", to, true); !p.empty()) + return std::unexpected(std::format("{}[{}]: {}", where, index, p)); + if (named) + if (auto p = deploy_name_problem(name); !p.empty()) + return std::unexpected(std::format("{}[{}]: {}", where, index, p)); + out.push_back({from, to, name}); + } + return out; +} + // `{ layer = "", visibility = "public" | "private" }`. A string // is a public requirement, as it always was. `private` is accepted for the // C++ layer alone -- a second C library in one image is not an island, it is @@ -1040,9 +1086,12 @@ std::expected parse_document(const t::Document& documen "target.*.build.flags", // #258 — middle segment is the cfg predicate "runtime.requirements", "runtime.artifacts", - // #615: `deploy = [{ from = "...", to = "..." }]`. The reader refuses - // every entry that is not a table of exactly those two strings. + // #615: `deploy = [{ from = "...", to = "..." }]`; `[test] files` + // (design 2026-11-16 §5.2) in the same entry grammar. + // `read_file_entries` type-checks every entry. "runtime.deploy", + "test.files", + "target.*.test.files", // #544: `deps = [{ linux = "..." }]` — every entry a per-platform // table — is the same Value shape as `[[xlings.deps]]`, and the guard // cannot tell the inline form from the doubled-bracket typo. The @@ -3450,45 +3499,9 @@ std::expected parse_document(const t::Document& documen // string, which is `{ from = }`, or a `{ from, to, name }` table. // See manifest::DeployEntry and the matching branch of the descriptor reader. if (auto* deploy = doc->get("runtime.deploy")) { - if (!deploy->is_array()) - return std::unexpected(error(origin, - "runtime.deploy must be an array of paths or `{ from = \"...\", to = \"...\" }` tables")); - std::size_t index = 0; - for (auto const& value : deploy->as_array()) { - ++index; - std::string from, to = ".", name; - if (value.is_string()) { - from = value.as_string(); - } else if (value.is_table()) { - auto const& table = value.as_table(); - for (auto const& [key, _] : table) - if (key != "from" && key != "to" && key != "name") - return std::unexpected(error(origin, std::format( - "runtime.deploy[{}] has unsupported key '{}'; the keys are " - "`from`, `to` and `name`", index, key))); - if (!table.contains("from")) - return std::unexpected(error(origin, std::format( - "runtime.deploy[{}] has no `from`", index))); - if (!table_string(table, "from", from) || !table_string(table, "to", to) - || !table_string(table, "name", name)) - return std::unexpected(error(origin, std::format( - "runtime.deploy[{}]: `from`, `to` and `name` must be strings", index))); - if (table.contains("name")) - if (auto p = deploy_name_problem(name); !p.empty()) - return std::unexpected(error(origin, - std::format("runtime.deploy[{}]: {}", index, p))); - } else { - return std::unexpected(error(origin, std::format( - "runtime.deploy[{}] must be a path or a table with `from`", index))); - } - if (auto p = deploy_path_problem("from", from, false); !p.empty()) - return std::unexpected(error(origin, - std::format("runtime.deploy[{}]: {}", index, p))); - if (auto p = deploy_path_problem("to", to, true); !p.empty()) - return std::unexpected(error(origin, - std::format("runtime.deploy[{}]: {}", index, p))); - m.runtimeConfig.linkIntent.deploy.push_back({from, to, name}); - } + auto entries = read_file_entries(*deploy, "runtime.deploy"); + if (!entries) return std::unexpected(error(origin, entries.error())); + for (auto& e : *entries) m.runtimeConfig.linkIntent.deploy.push_back(std::move(e)); } if (auto* requirements = doc->get("runtime.requirements")) { if (!requirements->is_array()) { @@ -3822,10 +3835,16 @@ std::expected parse_document(const t::Document& documen } continue; } + if (key == "files") { + auto entries = read_file_entries(value, "[test] files"); + if (!entries) return std::unexpected(error(origin, entries.error())); + m.testFiles = std::move(*entries); + continue; + } if (key != "discover") { m.schemaWarnings.push_back(std::format( "[test] has unsupported key '{}' (ignored). Supported keys: " - "discover, features, link, main, programs, split, start, store, windows_code_page, " + "discover, features, files, link, main, programs, split, start, store, windows_code_page, " "windows_long_paths.", key)); continue; } @@ -4947,10 +4966,17 @@ std::expected parse_document(const t::Document& documen return std::unexpected(error(origin, std::format( "[target.{}.test] must be a table", triple))); for (auto& [tk, tv] : t->second.as_table()) { + if (tk == "files") { + auto entries = read_file_entries(tv, std::format("[target.{}.test] files", triple)); + if (!entries) return std::unexpected(error(origin, entries.error())); + for (auto& e : *entries) cc.testFiles.push_back(std::move(e)); + continue; + } if (tk != "discover") { m.schemaWarnings.push_back(std::format( "[target.{}.test] has unsupported key '{}' (ignored). A conditional " - "section contributes test programs only: discover.", triple, tk)); + "section contributes test programs and their files only: discover, files.", + triple, tk)); continue; } bool listOk = tv.is_array(); diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 8ae441dae..a408ae9cb 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1774,6 +1774,9 @@ struct ConditionalConfig { // tests are discovered, before dependencies resolve: a predicate on a // target-side layer cannot select them. std::vector testDiscover; + // `[target..test] files` (design 2026-11-16 §5.2): run files of the + // tests of the targets the predicate selects, after `[test] files`. + std::vector testFiles; // `[target..runtime]` — the DIALECT-NEUTRAL half of a link line. // // `inputs.ldflags` above is spelled the GNU way and native `cl.exe` rejects @@ -1926,7 +1929,7 @@ inline bool is_empty(const ConditionalConfig& c) { // before it is ever evaluated, the same failure #296 was filed for. && c.dialectCxxflags.empty() // mcpp#942: a section that only selects test programs. - && c.testDiscover.empty(); + && c.testDiscover.empty() && c.testFiles.empty(); } // `[lib]` — library "root" interface convention. @@ -2321,6 +2324,10 @@ struct Manifest { // optional vector, for the reason `TargetEntry::sysrootDeclared` records. std::vector testDiscover; bool testDiscoverDeclared = false; + // `[test] files` (design 2026-11-16 §5.2): the files the working directory + // of each test program holds, in the entry grammar of `[runtime] deploy`; + // `to` and `name` are relative to that directory. + std::vector testFiles; // `[test] features` (mcpp#941): the features the package's own test build // turns on, beside its defaults and `--features`. The tests are a consumer // of the package, and this is that consumer's request: it is resolved in diff --git a/src/build/execute.cppm b/src/build/execute.cppm index bae801fc9..5f7aa33c5 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -1234,17 +1234,97 @@ write_runtime_files_list( // MCPP_SYSROOT the stated target base, when there is one // // The closure manifest is versioned (`schema`), names the artifact, every -// file it loads or reads from its own directory (the entries of -// MCPP_RUNTIME_FILES, which stays for runners written against it), and the -// base's root -- what a runner that moves the artifact to another machine has -// to move with it. -constexpr int kRunClosureSchema = 1; +// file it loads or reads from its own directory (`files`), the files the run's +// working directory holds (`run_files`, schema 2, design 2026-11-16 §7.1), and +// the base's root -- what a runner that moves the artifact to another machine +// has to move with it. MCPP_RUNTIME_FILES, which stays for runners written +// against it, lists both: where the program has a file system of its own, its +// directory is its working directory. +constexpr int kRunClosureSchema = 2; + +// ─── RUN FILES (design 2026-11-16 §5) ───────────────────────────────────── +// +// The files the working directory of one run holds: `mcpp run --file +// FROM[=TO]`, `[test] files`. With none, a program runs where mcpp was +// started, as before. With some, it runs in a directory of the build tree, +// `run//`, prepared before it starts and removed after it ends (`mcpp +// test` keeps a failed test's); a runner whose program has a file system of +// its own delivers them beside the program, through the same lists as the +// runtime files. +export struct RunFile { + std::filesystem::path from; // absolute + std::string to; // relative to the working directory +}; + +std::vector& run_files_setting() { + static std::vector files; + return files; +} +export void set_run_files(std::vector files) { run_files_setting() = std::move(files); } + +// `FROM[=TO]` as dirun's `--file HOST=GUEST` writes it: FROM relative to +// `base`, TO relative to the working directory and by default FROM's name. +export std::expected parse_run_file(std::string_view spec, + const std::filesystem::path& base) { + const auto eq = spec.find('='); + const std::string from{spec.substr(0, eq)}; + std::string to = eq == std::string_view::npos ? std::string() : std::string(spec.substr(eq + 1)); + if (from.empty()) return std::unexpected(std::format("--file {}: no file before `=`", spec)); + auto path = std::filesystem::path(from); + if (path.is_relative()) path = base / path; + std::error_code ec; + if (!std::filesystem::exists(path, ec)) + return std::unexpected(std::format("--file {}: {} does not exist", spec, path.generic_string())); + if (to.empty()) to = path.lexically_normal().filename().generic_string(); + if (auto p = mcpp::manifest::deploy_path_problem("TO", to, false); !p.empty()) + return std::unexpected(std::format("--file {}: {}", spec, p)); + return RunFile{path.lexically_normal(), to}; +} + +// The working directory `/run//`, emptied, then each file (a +// directory: its tree) copied to its destination. Copies, never links: a +// program that writes a run file must not change its source. Returns the +// placed files as (destination, copy). +struct PreparedRunDir { + std::filesystem::path dir; + std::vector> files; +}; +std::expected +prepare_run_dir(const std::filesystem::path& outputDir, std::string_view id, const std::vector& files) { + namespace fs = std::filesystem; + PreparedRunDir out; + out.dir = outputDir / "run" / std::string(id); + std::error_code ec; + fs::remove_all(out.dir, ec); + fs::create_directories(out.dir, ec); + if (ec) return std::unexpected(std::format("cannot create the run's directory {}: {}", out.dir.string(), ec.message())); + for (auto const& f : files) { + const auto to = out.dir / fs::path(f.to); + fs::create_directories(to.parent_path(), ec); + if (fs::is_directory(f.from, ec)) { + for (auto const& e : fs::recursive_directory_iterator(f.from, ec)) { + if (!e.is_regular_file(ec)) continue; + const auto rel = e.path().lexically_relative(f.from); + fs::create_directories((to / rel).parent_path(), ec); + fs::copy_file(e.path(), to / rel, fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(std::format("cannot copy {}: {}", e.path().string(), ec.message())); + out.files.emplace_back((fs::path(f.to) / rel).generic_string(), to / rel); + } + } else { + fs::copy_file(f.from, to, fs::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(std::format("cannot copy {}: {}", f.from.string(), ec.message())); + out.files.emplace_back(f.to, to); + } + } + return out; +} std::expected write_run_closure(const mcpp::build::BuildContext& ctx, const std::filesystem::path& artifact, const std::vector>& carried, - std::string_view mode, std::string_view triple, std::filesystem::path tree = {}) { + std::string_view mode, std::string_view triple, std::filesystem::path tree = {}, + const std::vector>& runFiles = {}) { namespace fs = std::filesystem; std::error_code ec; if (tree.empty()) tree = ctx.outputDir; @@ -1270,7 +1350,11 @@ write_run_closure(const mcpp::build::BuildContext& ctx, for (std::size_t i = 0; i < carried.size(); ++i) j += std::format("{}\n {{ \"dest\": {}, \"source\": {} }}", i ? "," : "", quote(carried[i].first), quote(carried[i].second.generic_string())); - j += std::format("{}],\n \"sysroot\": {}\n}}\n", carried.empty() ? "" : "\n ", + j += std::format("{}],\n \"run_files\": [", carried.empty() ? "" : "\n "); + for (std::size_t i = 0; i < runFiles.size(); ++i) + j += std::format("{}\n {{ \"dest\": {}, \"source\": {} }}", i ? "," : "", + quote(runFiles[i].first), quote(runFiles[i].second.generic_string())); + j += std::format("{}],\n \"sysroot\": {}\n}}\n", runFiles.empty() ? "" : "\n ", quote(ctx.tc.base.stated() ? ctx.tc.base.root.generic_string() : std::string{})); std::ofstream os(path, std::ios::binary | std::ios::trunc); os << j; @@ -1313,12 +1397,15 @@ run_contract_env(const mcpp::build::BuildContext& ctx, const std::filesystem::pa std::expected>, std::string> runner_handoff_env(const mcpp::build::BuildContext& ctx, const std::filesystem::path& artifact, const std::vector>& carried, - std::string_view mode, std::string_view triple, const std::filesystem::path& tree = {}) { + std::string_view mode, std::string_view triple, const std::filesystem::path& tree = {}, + const std::vector>& runFiles = {}) { std::vector> env; - auto listed = write_runtime_files_list(ctx, artifact, carried, tree); + auto both = carried; + both.insert(both.end(), runFiles.begin(), runFiles.end()); + auto listed = write_runtime_files_list(ctx, artifact, both, tree); if (!listed) return std::unexpected(listed.error()); env.emplace_back(std::string(kRuntimeFilesEnv), listed->string()); - auto closure = write_run_closure(ctx, artifact, carried, mode, triple, tree); + auto closure = write_run_closure(ctx, artifact, carried, mode, triple, tree, runFiles); if (!closure) return std::unexpected(closure.error()); for (auto& kv : run_contract_env(ctx, artifact, mode, triple, *closure)) env.push_back(std::move(kv)); return env; @@ -2636,6 +2723,9 @@ struct FastPathIdentity { // `[test] features` (mcpp#941), normalised: what a test plan of this // package adds to `features`, and so what its recorded graph names. std::string testFeatures; + // `[test] files`, unconditional or conditional (design 2026-11-16 §5.3): + // the tests' working directories are prepared on the full path. + bool testFiles = false; }; std::optional @@ -2663,6 +2753,8 @@ fast_path_identity(const std::filesystem::path& projectRoot, for (auto const& f : m->testFeatures) joined += (joined.empty() ? "" : ",") + f; return joined; }()), + !m->testFiles.empty() || std::ranges::any_of(m->conditionalConfigs, + [](auto const& cc) { return !cc.testFiles.empty(); }), }; } @@ -3325,6 +3417,8 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, // `--locked` is an assertion about resolution. if (mcpp::invocation::get("MCPP_LOCKED").value_or("") == "1") return fast_path_declined("run", "MCPP_LOCKED=1 asks for a locked resolution"); + if (!run_files_setting().empty()) + return fast_path_declined("run", "the run names files (--file): its working directory is prepared on the full path"); // `--managed-only` is checked against the sources prepare decides, which // a fast path does not run (mcpp#755). if (auto v = mcpp::invocation::get("MCPP_MANAGED_ONLY"); v && *v != "0") @@ -3641,6 +3735,25 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, } std::optional runtimeFilesList; std::vector> carriedFiles; + // THE RUN'S WORKING DIRECTORY (design 2026-11-16 §5.3): prepared when the + // run names files, before the launch is composed, so that a launch + // described instead of started (`--print-launch`) names a directory that + // holds them. + std::optional runDir; + if (!run_files_setting().empty()) { + static std::atomic serial{0}; + const auto self = mcpplibs::os::process::self(); + auto prepared = prepare_run_dir(ctx.outputDir, std::format("run-{}-{}", self ? self->id : 0, serial.fetch_add(1)), + run_files_setting()); + if (!prepared) { + std::println(libc::stderr(), "error: {}", prepared.error()); + return 2; + } + runDir = std::move(*prepared); + mcpp::log::verbose("run", std::format("the working directory is {}", runDir->dir.generic_string())); + } + const std::vector> noRunFiles; + auto const& runFiles = runDir ? runDir->files : noRunFiles; if (!choice.tmpl.empty()) { // The program is located by mcpp, not by posix_spawnp: a declared // payload's bin/ and then its root, then PATH — see runner_lookup for the shim @@ -3671,7 +3784,9 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, // holds its own files, so its list is empty and still exists. carriedFiles = format_name.empty() ? runtime_files_for(ctx, exe) : std::vector>{}; - auto listed = write_runtime_files_list(ctx, exe, carriedFiles); + auto listedFiles = carriedFiles; + listedFiles.insert(listedFiles.end(), runFiles.begin(), runFiles.end()); + auto listed = write_runtime_files_list(ctx, exe, listedFiles); if (!listed) { std::println(libc::stderr(), "error: {}", listed.error()); return 1; @@ -3705,7 +3820,7 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, mcpp::ui::status("Via", std::format("{} [{}]", shown, ctx.runUnderOrigin)); } if (!host.prefix.empty() || !choice.tmpl.empty()) { - auto closure = write_run_closure(ctx, exe, carriedFiles, "run", choice.tripleKey); + auto closure = write_run_closure(ctx, exe, carriedFiles, "run", choice.tripleKey, {}, runFiles); if (!closure) { std::println(libc::stderr(), "error: {}", closure.error()); return 1; @@ -3741,7 +3856,8 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, ctx.buildLock.reset(); if (print_launch_flag()) { mcpp::build::result::Launch launch{argv, childEnv, - std::filesystem::current_path().generic_string(), + runDir ? runDir->dir.generic_string() + : std::filesystem::current_path().generic_string(), !host.prefix.empty() ? "run_under" : !choice.tmpl.empty() ? "runner" : ""}; mcpp::wire::Envelope env{.kind = "mcpp.launch", @@ -3755,7 +3871,24 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, return 0; } int spawnErr = 0; - const int rc = mcpp::run::run_foreground(argv, childEnv, &spawnErr); + int rc = 0; + if (runDir) { + // mcpp waits for the program, which runs in the prepared directory, + // and removes the directory after it: the program is not exec'd. + mcpp::run::Options o; + o.env = childEnv; + o.cwd = runDir->dir.string(); + o.input = mcpp::run::Input::inherit; + o.streams = mcpp::run::Streams::inherit; + o.report_spawn_failure = false; + auto r = mcpp::run::execute(argv, o); + rc = r.exit_code; + spawnErr = r.spawn_error; + std::error_code rmEc; + std::filesystem::remove_all(runDir->dir, rmEc); + } else { + rc = mcpp::run::run_foreground(argv, childEnv, &spawnErr); + } if (spawnErr != 0) { using namespace mcpp::build::runner_lookup; if (!choice.tmpl.empty() || !host.prefix.empty()) @@ -5688,6 +5821,35 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, }; std::map objectMarkers; + // THE TESTS' RUN FILES (design 2026-11-16 §5.3): a working directory per + // test program, `run/test-/`, which the processes its split divides + // it into share; prepared before the program's launch is composed, so a + // launch described (`--no-run`) names a directory that holds them; + // removed after the program passes, kept and named when it fails. Only a + // package that states `[test] files` pays for the copies. + std::vector testRunFiles; + { + auto entries = mcpp::build::test_run_files(ctx->manifest, ctx->tc.targetTriple); + if (!entries) { + mcpp::ui::error(entries.error()); + return 2; + } + for (auto const& e : *entries) { + auto expanded = mcpp::build::expand_deploy_entry(ctx->projectRoot, e); + if (!expanded) { + mcpp::ui::error(std::format("[test] files: {}", expanded.error())); + return 2; + } + for (auto const& x : *expanded) { + const auto from = (ctx->projectRoot / x.from).lexically_normal(); + auto to = (x.to.empty() || x.to == ".") ? std::filesystem::path() : std::filesystem::path(x.to); + to /= x.name.empty() ? from.filename() : std::filesystem::path(x.name); + testRunFiles.push_back({from, to.generic_string()}); + } + } + } + std::map testRunDirs; + // An archive's tests are composed by its reader; they run as the plan's do. if (tb.archived) { runnable = *tb.archived; @@ -5776,6 +5938,20 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, for (auto& kv : tb.programEnv) childEnv.push_back(kv); if (auto mp = tb.memberProgramEnv.find(lu.memberOf); mp != tb.memberProgramEnv.end()) for (auto& kv : mp->second) childEnv.push_back(kv); + // Its working directory, when the package states run files. + std::optional testDir; + if (!testRunFiles.empty()) { + std::string leaf(lu.targetName); + for (auto& ch : leaf) + if (!std::isalnum(static_cast(ch)) && ch != '-' && ch != '.') ch = '_'; + auto prepared = prepare_run_dir(ctx->outputDir, "test-" + leaf, testRunFiles); + if (!prepared) { + if (invocationNotRunReason.empty()) invocationNotRunReason = prepared.error(); + } else { + testDir = std::move(*prepared); + testRunDirs[lu.targetName] = testDir->dir; + } + } // ...and, through a runner, the files the test carries with it, as // `mcpp run` hands them over (see `runtime_files_for`). Written here, // in the single-threaded pass, one list per test program. @@ -5786,7 +5962,9 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, // own product directory with that group's deploy set. const std::string_view of = owner.empty() ? std::string_view(lu.memberOf) : owner; with_member(*ctx, of, [&] { carried = runtime_files_for(*ctx, exe, of); }); - if (auto handoff = runner_handoff_env(*ctx, exe, carried, "test", runnerChoice.tripleKey)) + const std::vector> none; + if (auto handoff = runner_handoff_env(*ctx, exe, carried, "test", runnerChoice.tripleKey, {}, + testDir ? testDir->files : none)) for (auto& kv : *handoff) childEnv.push_back(std::move(kv)); else if (invocationNotRunReason.empty()) invocationNotRunReason = handoff.error(); @@ -5812,6 +5990,7 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, // its file carries (P2.3). A version this mcpp does not speak runs as // a program that implements none. auto& added = runnable.back(); + if (testDir) added.cwd = testDir->dir; const int version = test_protocol_of(*ctx, lu, objectMarkers); if (version == mcpp::build::test_protocol::kVersion) { added.protocol = version; @@ -6224,6 +6403,18 @@ static int test_run_member(TestBuild& tb, const TestOptions& testOpts, } else { ++failed; failures.push_back(r.name); } } + // A passed program's working directory goes; a failed one's stays, for the + // reader to look into, and is named (design 2026-11-16 §5.3). A launch only + // described (`--no-run`) keeps every directory for the program it names. + if (!testOpts.noRun) + for (auto const& [name, dir] : testRunDirs) { + const bool failedHere = std::ranges::any_of(failures, [&](std::string const& f) { + return f == name || f.starts_with(name + "/"); + }); + std::error_code rmEc; + if (!failedHere) std::filesystem::remove_all(dir, rmEc); + else if (!json) mcpp::ui::info("Kept", std::format("the working directory of {}: {}", name, dir.generic_string())); + } summary.passed = passed; summary.failed = failed; @@ -6747,6 +6938,7 @@ fast_test_plan(const std::filesystem::path& projectRoot, const TestOptions& test auto want = fast_path_identity(projectRoot); if (!want) return declined("the manifest or the request could not be read"); if (want->hooksActive) return declined("the project declares [hooks]"); + if (want->testFiles) return declined("the tests have run files ([test] files)"); if (!want->defaultTarget.empty()) return declined("the project names a default target"); // A test plan names the package's `[test] features` beside the request. const auto testRequest = normalize_features(want->features + "," + want->testFeatures); diff --git a/src/build/test_targets.cpp b/src/build/test_targets.cpp index ce61241e8..667726f69 100644 --- a/src/build/test_targets.cpp +++ b/src/build/test_targets.cpp @@ -30,4 +30,21 @@ conditional_test_globs(const mcpp::manifest::Manifest& m, std::string_view targe return out; } +std::expected, std::string> +test_run_files(const mcpp::manifest::Manifest& m, std::string_view targetTriple) { + std::vector out = m.testFiles; + const auto ctx = cfgpred::context_for(targetTriple.empty() ? std::string_view(m.buildConfig.target) + : targetTriple); + for (auto const& cc : m.conditionalConfigs) { + if (cc.testFiles.empty()) continue; + if (cfgpred::uses_layer(cc.predicate)) + return std::unexpected(std::format( + "[target.'{}'.test] files names a target-side layer; select the files by the target " + "(os, arch, env, family, object-format)", cc.predicate)); + if (!cfgpred::matches(cc.predicate, ctx)) continue; + out.insert(out.end(), cc.testFiles.begin(), cc.testFiles.end()); + } + return out; +} + } // namespace mcpp::build diff --git a/src/build/test_targets.cppm b/src/build/test_targets.cppm index 7e91e7e53..38d19094c 100644 --- a/src/build/test_targets.cppm +++ b/src/build/test_targets.cppm @@ -32,6 +32,12 @@ inline constexpr std::string_view kDefaultTestDiscover = "tests/**/*.cpp"; std::expected, std::string> conditional_test_globs(const mcpp::manifest::Manifest& m, std::string_view targetTriple); +// The run files of the package's tests (design 2026-11-16 §5.2): `[test] +// files`, then the `[target..test] files` whose predicate `targetTriple` +// matches, read as `conditional_test_globs` reads its tables. +std::expected, std::string> +test_run_files(const mcpp::manifest::Manifest& m, std::string_view targetTriple); + // 损坏 manifest 时仅提供 best-effort 文件清单;严格调用方仍须走 prepare/validate。 // // `targetTriple` is the target the tests are built for; empty: the package's diff --git a/src/cli.cppm b/src/cli.cppm index 3d98b47b6..954fe2476 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -812,6 +812,11 @@ int run(int argc, char** argv) { .help("Play a game in the status row while it builds: --play-game=snake|stack|runner, or one at random")) .option(cl::Option("print-launch") .help("Build, then print how the program would be started (argv, the environment mcpp adds, cwd, runner) as the mcpp.launch envelope, instead of starting it")) + // RUN FILES (design 2026-11-16 §5): the spelling dirun's `--file + // HOST=GUEST` already has, for the same concept. + .option(cl::Option("file").takes_value().multiple().value_name("FROM[=TO]") + .help("Copy FROM into the run's working directory as TO (default: its name); " + "the program then runs in a directory of the build tree, removed after it ends; repeatable")) .option(cl::Option("message-format").takes_value().value_name("FMT") .help("With --print-launch: human (default) | json (the build's events as NDJSON, then the envelope as the last line)")) .action([&passthrough](const cl::ParsedArgs& p) { diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index f584b23f5..37b533cf2 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -1160,6 +1160,20 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, return mcpp::build::list_runners(package_filter, cache_mode, no_cache, target_triple, features, profile, accel); mcpp::build::set_print_launch(parsed.is_flag_set("print-launch")); + if (auto o = parsed.option("file"); o && !o->get().values.empty()) { + std::vector files; + std::error_code cwdEc; + const auto base = std::filesystem::current_path(cwdEc); + for (auto const& spec : o->get().values) { + auto f = mcpp::build::parse_run_file(spec, base); + if (!f) { + mcpp::ui::error(f.error()); + return 2; + } + files.push_back(std::move(*f)); + } + mcpp::build::set_run_files(std::move(files)); + } // A launch asked for as JSON streams the build's events, and its envelope is // the last line (design 2026-11-07 §4.9): an editor shows the build as it goes. if (auto mf = parsed.value("message-format"); mf && *mf != "human" && *mf != "json") { diff --git a/tests/e2e/run/contract.cpp b/tests/e2e/run/contract.cpp index 5f59f10bf..03d0e1dbc 100644 --- a/tests/e2e/run/contract.cpp +++ b/tests/e2e/run/contract.cpp @@ -347,6 +347,42 @@ E2E contract_machine{"contract-machine", resource{"machine"}, [] { expect(fatal(e2e::contains(s::text(r), R"("launch-stream")"))) << "E: launch-stream is not stated: " << s::text(r); }; + "1104 a run's files are in its working directory"_test[s::unix_shell] = [] { + // Design 2026-11-16 §5: `--file FROM[=TO]` copies FROM into a directory of the build tree, + // where the program runs; the directory is removed after it; without --file the program + // runs where mcpp was started. Through a runner, the files are listed in + // MCPP_RUNTIME_FILES with the runtime files and as `run_files` in the closure (schema 2), + // and the runner starts in the directory. + const auto work = s::case_dir(); + s::project app(work / "app", {{"mcpp.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n"}, + {"data/in.txt", "payload\n"}, {"data/sub/d.txt", "deep\n"}, + {"src/main.cpp", "import std;\nint main(int argc, char** argv) {\n" + " std::println(\"cwd={}\", std::filesystem::current_path().filename().string());\n" + " for (int i = 1; i < argc; ++i) { std::ifstream in(argv[i]); std::string l; std::getline(in, l); std::println(\"{}=[{}]\", argv[i], l); }\n" + " std::ofstream(\"written.txt\") << 1;\n return 3;\n}\n"}}); + auto r = app.mcpp({"run", "--file", "data/in.txt=IN.TXT", "--file", "data", "--", "IN.TXT", "data/sub/d.txt"}); + expect(fatal(r.started && r.status == 3)) << "the program's status did not pass through: " << s::text(r); + expect(fatal(s::has_line(s::text(r), "^cwd=run-"))) << "the program did not run in the run's directory: " << s::text(r); + expect(fatal(s::has_line(s::text(r), "^IN.TXT=\\[payload\\]$"))) << s::text(r); + expect(fatal(s::has_line(s::text(r), "^data/sub/d.txt=\\[deep\\]$"))) << s::text(r); + expect(fatal(!fs::exists(app.at("written.txt")))) << "the program wrote into the project"; + for (const auto& f : s::find_files(app.at("target"), R"(written\.txt)")) + expect(fatal(false)) << "the run's directory was left: " << f; + r = app.mcpp({"run", "--file", "missing.txt"}); + expect(fatal(r.started && r.status == 2 && e2e::contains(s::text(r), "missing.txt does not exist"))) << s::text(r); + s::write_executable(work / "runner.sh", "#!/bin/sh\necho \"runner-cwd=$(basename \"$PWD\" | cut -c1-4)\"\n" + "cat \"$MCPP_RUNTIME_FILES\"\ngrep -c '\"dest\": \"IN.TXT\"' \"$MCPP_RUN_CLOSURE\"\nexec \"$@\"\n"); + auto machine = s::run(*s::on_path("uname"), {"-m"}).out; + machine.erase(machine.find_last_not_of("\r\n") + 1); + app.write("mcpp.toml", std::format("[package]\nname = \"app\"\nversion = \"0.1.0\"\n[target.{}-linux-gnu]\nrunner = [\"{}\"]\n", + machine, (work / "runner.sh").generic_string())); + r = app.mcpp({"run", "--target", machine + "-linux-gnu", "--file", "data/in.txt=IN.TXT", "--", "IN.TXT"}); + expect(fatal(r.started && r.status == 3)) << s::text(r); + expect(fatal(s::has_line(s::text(r), "^runner-cwd=run-"))) << "the runner did not start in the run's directory: " << s::text(r); + expect(fatal(s::has_line(s::text(r), "^IN.TXT\t.*/run/run-[^/]*/IN.TXT$"))) << "MCPP_RUNTIME_FILES does not list the run file: " << s::text(r); + expect(fatal(s::has_line(s::text(r), "^1$"))) << "the closure does not list the run file: " << s::text(r); + }; + "927 the runner contract"_test[s::gcc][s::unix_shell] = [] { // run_under composes in front of the artifact with the contract variables and a closure // manifest, returns the program's status and states where it was declared; the @@ -386,7 +422,8 @@ exec "$@" const auto closure = s::last_starting(rec, "closure=").substr(8); expect(fatal(fs::is_regular_file(closure))) << "MCPP_RUN_CLOSURE names no file"; const auto cl = s::read_file(closure); - expect(fatal(e2e::contains(cl, R"("schema": 1)"))) << "the closure manifest carries no schema: " << cl; + expect(fatal(e2e::contains(cl, R"("schema": 2)"))) << "the closure manifest carries no schema: " << cl; + expect(fatal(e2e::contains(cl, R"("run_files": [])"))) << "the closure manifest lists no run files: " << cl; expect(fatal(s::has_line(cl, R"("artifact": ".*/bin/app")"))) << "the closure manifest does not name the artifact: " << cl; fs::remove(record); std::string var = "MCPP_TARGET_" + triple + "_RUN_UNDER"; diff --git a/tests/e2e/test/selection.cpp b/tests/e2e/test/selection.cpp index 9df113f89..3429723da 100644 --- a/tests/e2e/test/selection.cpp +++ b/tests/e2e/test/selection.cpp @@ -1,6 +1,7 @@ // Which tests `mcpp test` finds and runs: names from paths, a pattern and several, the listing, // `[test] discover` (formerly tests/e2e/152, 154, 159 and 673; 1084), the tests of a target -// (`[target..test]`, 1098) and the features they are built with (`[test] features`, 1099, 1100). +// (`[target..test]`, 1098), the features they are built with (`[test] features`, 1099, 1100) +// and their run files (`[test] files`, 1105). import std; import mcpplibs.mtest; import mcpp.test.support; @@ -214,7 +215,7 @@ E2E selection{"selection", [] { "programs = \"directory\"\n"); r = p.mcpp({"test"}); expect(fatal(e2e::succeeded(r))) << s::text(r); - expect(e2e::contains(s::text(r), "contributes test programs only: discover")) << s::text(r); + expect(e2e::contains(s::text(r), "contributes test programs and their files only: discover, files")) << s::text(r); }; "1099 a package's tests are built with the features [test] features names"_test = [] { @@ -270,6 +271,30 @@ E2E selection{"selection", [] { expect(fatal(e2e::succeeded(r))) << s::text(r); expect(s::lacks(s::text(r), "b's extra was requested")) << "another member's feature of the same name was requested"; }; + + "1105 a test's run files are in its working directory"_test = [] { + // Design 2026-11-16 §5: `[test] files` (and `[target..test] files` for the target the + // tests are built for) are copied into a directory per test program, where it runs; the + // directory of a test that passed is removed, that of one that failed is kept and named. + s::project p(s::case_dir() / "p", {{"src/main.cpp", "int main() { return 0; }\n"}, + {"tests/data/in.txt", "payload\n"}, {"tests/data/sub/d.txt", "deep\n"}, + {"tests/host/h.txt", "host\n"}, + {"tests/reads.cpp", "import std;\nint main() {\n" + " std::ifstream a(\"IN.TXT\"), b(\"in.txt\"), c(\"sub/d.txt\"), d(\"h.txt\");\n" + " std::string w, x, y, z; std::getline(a, w); std::getline(b, x); std::getline(c, y); std::getline(d, z);\n" + " std::ofstream(\"written\") << 1;\n" + " return w == \"payload\" && x == \"payload\" && y == \"deep\" && z == \"host\" ? 0 : 1;\n}\n"}, + {"tests/fails.cpp", "import std;\nint main() { std::ofstream(\"evidence.txt\") << 1; return 4; }\n"}}); + p.write("mcpp.toml", manifest("p") + "\n[test]\nfiles = [\"tests/data/**\", { from = \"tests/data/in.txt\", name = \"IN.TXT\" }]\n" + "\n[target.'cfg(unix)'.test]\nfiles = [\"tests/host/h.txt\"]\n"); + auto r = p.mcpp({"test"}); + expect(fatal(r.started && r.status != 0)) << "the failing test passed: " << s::text(r); + expect(fatal(s::has_line(s::text(r), "^reads \\.\\.\\. ok"))) << "the test that reads its files failed: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), "the working directory of fails:"))) << "the failed test's directory is not named: " << s::text(r); + expect(fatal(s::find_file(p.at("target"), "evidence.txt").has_value())) << "the failed test's directory was removed"; + expect(fatal(!s::find_file(p.at("target"), "written").has_value())) << "the passed test's directory was left"; + expect(fatal(!fs::exists(p.at("written")))) << "a test wrote into the project"; + }; }}; } // namespace diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index a150c4b7c..fa9390cd5 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -5923,6 +5923,31 @@ license_files = ["LICENSE", "ThirdPartyNotices.txt"] expect(eq(mcpp::manifest::default_license_files().front(), std::string("{LICENSE,LICENCE,COPYING,NOTICE}*"))); }}; +// Design 2026-11-16 §5.2: `[test] files` in the deploy entry grammar, and the +// conditional form for the tests of a target. +UTest test_files_are_read{"Manifest/TestFilesAreRead", [] { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "t" +version = "0.1.0" +[test] +files = ["tests/data/**", { from = "tests/t.bat", name = "TEST.BAT" }] +[target.'cfg(os = "uefi")'.test] +files = ["tests/uefi/**"] +)"); + if (auto&& failed = expect(fatal(m.has_value())); !failed) failed << m.error().format(); + expect(fatal(eq(m->testFiles.size(), 2u))); + expect(eq(m->testFiles[1].name, "TEST.BAT")); + bool conditional = false; + for (auto const& cc : m->conditionalConfigs) + if (cc.testFiles.size() == 1 && cc.testFiles.front().from == std::filesystem::path("tests/uefi/**")) conditional = true; + expect(conditional); + expect(m->schemaWarnings.empty()); + auto bad = mcpp::manifest::parse_string("[package]\nname = \"t\"\nversion = \"0.1.0\"\n[test]\nfiles = [{ to = \"x\" }]\n"); + expect(fatal(!bad.has_value())); + expect(ne(bad.error().message.find("[test] files[1] has no `from`"), std::string::npos)) << bad.error().message; +}}; + // ── `requires_abi` (design 2026-09-12, section 5.2) ──────────────────────── UTest requires_abi_threads_parses_on_the_package_and_on_a_feature{"Manifest/RequiresAbiThreadsParsesOnThePackageAndOnAFeature", [] { @@ -7270,7 +7295,7 @@ UTest a_conditional_test_table_carries_discover_alone{"Manifest/AConditionalTest expect(eq(m->conditionalConfigs.front().testDiscover, std::vector{"!tests/native/**"})); expect(!mcpp::manifest::is_empty(m->conditionalConfigs.front())); expect(fatal(eq(m->schemaWarnings.size(), 1u))); - expect(ne(m->schemaWarnings.front().find("contributes test programs only: discover"), std::string::npos)); + expect(ne(m->schemaWarnings.front().find("contributes test programs and their files only: discover, files"), std::string::npos)); auto bad = mcpp::manifest::parse_string( "[package]\nname = \"bt\"\nversion = \"0.1.0\"\n" "[target.'cfg(os = \"emscripten\")'.test]\ndiscover = \"tests/x.cpp\"\n"); From 37129dc734362b371fa12e2a43df16eba65f1847 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 20:40:09 +0900 Subject: [PATCH 12/36] platform: a row the built-in table has no platform for takes the one the index lists MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A platform package's index entry lists the rows it serves (`platform_rows = { "x86_64-uefi", ... }`). Step 5 of the platform choice, the target table's platform column, consults the index when the built-in table has no answer: a row the engine does not know, a row a platform defines, or a `planned` row. The package is selected as a series (`.@*`), so its release is chosen, locked and updated as any platform series is, and the report names the source `index`. - The project's own indices ([indices], .mcpp/) rank before the shared ones; within one rank two claims are named and the first is taken. - A row the table answers is unchanged, and every statement of the project, the machine, the environment and the command line still comes first. - The entries are scanned once per command, and only for such a row (about 0.05 s for the whole refusal over ~400 entries). - mcpp 2026.11.16.1 reads an entry carrying the field and ignores it (measured with openkal-platform's entry). Tests: e2e 1106 (unknown row, planned row, the project first, two claims, no column), PlatformSelect unit. SPEC-013 v1.11 (§3.1, §14, criterion 16), docs/35 en/zh; the openkal-platform example uses multi-line inline tables. --- CHANGELOG.md | 9 ++ docs/35-platforms.md | 37 ++++++- docs/specs/platforms.md | 35 ++++++- docs/zh/35-platforms.md | 33 +++++- modules/manifest/src/platform.cppm | 2 + src/build/prepare/platform.cpp | 158 ++++++++++++++++++++++++++++- src/build/prepare/state.cppm | 5 + src/platform/platform.cppm | 5 +- tests/e2e/openkal/platforms.cpp | 104 ++++++++++++++++++- tests/unit/test_platform.cpp | 11 ++ 10 files changed, 382 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 42d73d654..aa707e41c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,15 @@ `MCPP_RUN_CLOSURE` (schema 2) and in `MCPP_RUNTIME_FILES` beside the runtime files, so a runner whose program has a file system of its own delivers them beside the program (design 2026-11-16 §5, §7). +- A platform package's index entry lists the rows it serves, + `platform_rows = { "x86_64-uefi", ... }`, and a row the built-in table has no + platform for takes that package as a series (`.@*`): `mcpp build + --target x86_64-uefi` needs no `[target.x86_64-uefi] platform`, and a + `planned` row such as `i686-linux-musl` builds once a platform claims it. A + row the table answers is unchanged, a project's own statement still comes + first, the project's `[indices]` rank before the shared ones, and the report + names the source `index`. The entries are read once per command, and only + for such a row (design 2026-11-16 §6.2, SPEC-013 §14). ### Changed diff --git a/docs/35-platforms.md b/docs/35-platforms.md index bd9bf594c..e365c1a50 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -119,7 +119,7 @@ The first of these that answers selects the target's platform: | the environment | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | | the project | `[target.] platform = "openkal-platform@0.24.0"`, or `{ path = "…" }` | | this machine | the same key in `~/.mcpp/config.toml` | -| the target table | the platform column (`mcpp toolchain list`) | +| the target table | the platform column (`mcpp toolchain list`), then the package index (below) | `platform` is not accepted under a `cfg(platform = …)` selector. `none` builds with the engine's conventions for the target and nothing else. @@ -215,6 +215,28 @@ next build takes the recorded release while the series admits it. `mcpp update` (or `mcpp update openkal-platform`) drops the record, and the build after it takes the newest release again. +## A row the index names (2026.11.17.1+) + +A row the built-in table has no platform for — a row the engine does not know, +such as `x86_64-uefi`, or a `planned` one, such as `i686-linux-musl` — takes the +platform whose index entry lists it: + +```lua +package = { + namespace = "mcpplibs", + name = "openkal-platform", + platform_rows = { "x86_64-uefi", "i686-dos-musl", "i686-linux-musl", "i686-openkal" }, + -- ... +} +``` + +`mcpp build --target x86_64-uefi` then needs no line in `mcpp.toml`. The row +selects the package as a series (`openkal-platform@*`), so the release is chosen, +locked and updated as above, and the build reports the source as `index`. A +project's own `[target.] platform` still comes first, and so do the +project's own `[indices]` before the shared ones. The list is generated from the +platform's manifest by the index's tools; engines before 2026.11.17.1 ignore it. + ## Writing a platform package A platform package is a package with a `[platform]` section. openkal-platform, @@ -230,14 +252,23 @@ mcpp = ">=2026.11.3.1" [platform] targets = ["x86_64-openkal", "aarch64-openkal"] toolchain = "llvm@23.1.3" -runner = { cmd = ["okrun", "{}"], host = "same-arch", hosts = ["linux-x86_64", "linux-aarch64", "macos-aarch64", "windows-x86_64"] } +runner = { + cmd = ["okrun", "{}"], + host = "same-arch", + hosts = ["linux-x86_64", "linux-aarch64", "macos-aarch64", "windows-x86_64"], +} carrier = { package = "openkal-run@0.5.0", section = ".oke", main = "carrier/main.cpp" } plugin = "build.mcpp" [platform.layers] compiler-runtime = "openkal-llvm-runtime@0.19.0" "c++-abi" = "openkal-llvm-runtime@0.19.0" -c-abi = { name = "openkal-musl", version = "0.28.0", default-features = false, features = ["posix", "malloc-mimalloc"] } +c-abi = { + name = "openkal-musl", + version = "0.28.0", + default-features = false, + features = ["posix", "malloc-mimalloc"], +} [platform.tools] okld = "openkal-tools@0.5.0" diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index d68f90c85..e55bd9dd1 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -4,11 +4,11 @@ |---|---| | 规范编号 | SPEC-013 | | 标题 | 平台:一类目标的可复用配置,其声明、选择、合并、对依赖图的补全、链接与封装、运行器与载体 | -| 状态 | 评审中 v1.10 | -| 版本 | 1.10 | +| 状态 | 评审中 v1.11 | +| 版本 | 1.11 | | 最后修改 | 2026-11-16 | -| 对应实现 | mcpp >= 2026.10.27.1;§10 的执行宿主与 §11 的载体架构 mcpp >= 2026.10.29.1;`runner.hosts`、载体列表、`descriptor_keys` 与 `universal` 的弃用 mcpp >= 2026.10.30.1;§7.5 版本组参与解析、§14 平台系列与 §15.1 系列的锁 mcpp >= 2026.11.7.1;§14 内置平台为确切版本与提示 mcpp >= 2026.11.7.2;v1.7 条款 mcpp >= 2026.11.10.1;v1.10 条款 mcpp >= 2026.11.16.1;逐条标注 | -| 相关设计文档 | `.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md`、`.agents/docs/2026-10-28-platforms-design.md`、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md` | +| 对应实现 | mcpp >= 2026.10.27.1;§10 的执行宿主与 §11 的载体架构 mcpp >= 2026.10.29.1;`runner.hosts`、载体列表、`descriptor_keys` 与 `universal` 的弃用 mcpp >= 2026.10.30.1;§7.5 版本组参与解析、§14 平台系列与 §15.1 系列的锁 mcpp >= 2026.11.7.1;§14 内置平台为确切版本与提示 mcpp >= 2026.11.7.2;v1.7 条款 mcpp >= 2026.11.10.1;v1.10 条款 mcpp >= 2026.11.16.1;v1.11 条款(§14 索引的平台列) mcpp >= 2026.11.17.1;逐条标注 | +| 相关设计文档 | `.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md`、`.agents/docs/2026-11-15-ecosystem-gap-supplement-design.md`、`.agents/docs/2026-10-28-platforms-design.md`、`.agents/docs/2026-10-28-platforms-part2-packaging-and-distribution-design.md`、`.agents/docs/2026-10-29-openkal-run-test-pack-design.md` | | 相关 issue | mcpp#846、mcpp#847、mcpp#848 | | 相关规范 | [SPEC-002](target-side.md)(目标侧的层)、[SPEC-004](manifest-semantics.md)(条件化形状)、[SPEC-006](toolchain-management.md)(工具链选择)、[SPEC-007](build-plugins.md)(构建程序与协议)、[SPEC-011](concurrency.md)(原子发布)、[SPEC-014](packaging.md)(打包) | | 使用文档 | [35 —— 平台](../zh/35-platforms.md) | @@ -135,7 +135,7 @@ plugin = "build.mcpp" 2. `MCPP_TARGET__PLATFORM`; 3. 工程的 `[target.] platform`(工作空间层先于根包,§13); 4. `config.toml` 的 `[target.] platform`; -5. 目标表的平台列(§14); +5. 目标表的平台列(§14):引擎的内置表,内置表没有答案时为索引(v1.11); 6. 无:宿主目标的内置约定,或无平台。 第 3、4 步在命中的表中取具体程度最高者(SPEC-004 §3.1.1)。**已实现** @@ -445,6 +445,28 @@ run_under = ["qemu-aarch64"] 平台发布补丁版本因此不需要发布 mcpp。精确版本(`@0.24.1`、`@=0.24.1`)按写出的取。**已实现** +**索引的平台列(mcpp >= 2026.11.17.1)。** 平台包的索引条目可以列出它提供的行: + +```lua +package = { + namespace = "mcpplibs", + name = "openkal-platform", + platform_rows = { "x86_64-uefi", "i686-dos-musl", "i686-linux-musl", "i686-openkal" }, + -- ... +} +``` + +1. 第 5 步先查内置表。内置表没有答案的行,即引擎不认识的行、平台定义的行(SPEC-016 §2)与 `planned` 行,查索引; + 内置表有答案的行不查索引,其选择不变。 +2. 认领一行的平台包以系列 `<命名空间>.<名字>@*` 选择,按本节的平台系列取版本并写入锁;选择来源报告为 `index`。 +3. 工程自己的索引(`[indices]`)先于共享的索引;同一层有多个平台包认领一行时取名字最小者,并给出警告列出它们。 + 索引的检查应保证一行只有一个平台包认领。 +4. `planned` 行由认领它的平台服务时可以构建,与工程选择平台相同(§3.1 第 3 步)。 +5. 条目是平台包清单的派生数据:列表由索引的工具从平台包的 `[platform] targets`、`[[platform.define]]` 与 + `[[platform.provider]] serves` 生成,索引的 CI 检查二者一致。早于本版本的引擎不读这个字段。 + +每条命令至多扫描一次索引的条目,且只在内置表没有答案时扫描(约 400 个条目时整个拒绝过程 0.05 s)。**已实现** + **描述符键(mcpp >= 2026.10.30.1)。** 每个目标行声明它读取依赖描述符的哪些按平台块及其顺序,第一个存在的块被读取; `xpm` 下的版本与 `mcpp` 段的按平台表都按此读取,构建机不参与: @@ -562,6 +584,8 @@ openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscri 取更新的版本(e2e 1021)。 15. 内置平台为确切版本:不写平台的工程构建该版本并锁定;锁中记录的其他版本不被采用并写出;工程写出的平台照常采用 (e2e 1025;提示的条件与内容见单元测试 PlatformDefault)。 +16. 工程的索引在 `platform_rows` 中列出的行不需要工程的声明即选择该平台,报告来源 `index`;`planned` 行由认领的平台服务; + 工程的声明优先;两个认领被写出并取第一个;没有这一列时行未知、`planned` 行被拒绝(e2e 1106)。 ## 变更记录 @@ -578,3 +602,4 @@ openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscri | 1.8 | 2026-11-12 | mcpp 2026.11.12.1(设计 2026-11-12 §5.2、§5.5):§10 运行器的 `requires` 与 `built-as`,模板占位 `{sysroot}`、`{pkg:}`,提供者的 `runner` 为陈述的最低一级;`hosts` 的系统取自系统表。 | | 1.9 | 2026-11-14 | mcpp 2026.11.13.1:§7.3 一行所带的工具,`base = "none"` 的提供者只带其自身约定指名的工具,其余身份不带只由这类提供者指名的工具。 | | 1.10 | 2026-11-16 | mcpp 2026.11.16.1(设计 2026-11-15 R1、R2):§10 运行器的 `enabled` 与 `longlived`;具名运行器与 `runner` 同一种表;§6 覆盖层不含 `cmd` 的表、`enabled` 随 `cmd` 的陈述、`MCPP_TARGET__RUNNER_ENABLED`。;运行器的程序:查找、Windows 的后缀、命令解释器启动的脚本的参数、占位符与状态(R2)。 | +| 1.11 | 2026-11-16 | mcpp 2026.11.17.1(设计 2026-11-16 §6.2):§3.1 第 5 步与 §14 索引的平台列 `platform_rows`;判据 16(e2e 1106)。 | diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index 7a873adc6..70b4d65be 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -103,7 +103,7 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 | 环境变量 | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | | 工程 | `[target.] platform = "openkal-platform@0.24.0"`,或 `{ path = "…" }` | | 本机 | `~/.mcpp/config.toml` 中的同一个键 | -| 目标表 | 平台列(`mcpp toolchain list`) | +| 目标表 | 平台列(`mcpp toolchain list`),其次索引(见下文) | `platform` 不能写在 `cfg(platform = …)` 选择器下。`none` 只以引擎对该目标的约定构建。 @@ -177,6 +177,24 @@ platform = "openkal-platform@0.28.0" # 一个版本,或一个系列:"openkal 之后的构建在系列容许时采用记录的版本。`mcpp update`(或 `mcpp update openkal-platform`)删除这条记录,下一次构建重新 取最新版本。 +## 索引列出的行(2026.11.17.1+) + +内置表中没有平台的行,即引擎不认识的行(如 `x86_64-uefi`)与 `planned` 行(如 `i686-linux-musl`),采用索引条目中 +列出它的平台: + +```lua +package = { + namespace = "mcpplibs", + name = "openkal-platform", + platform_rows = { "x86_64-uefi", "i686-dos-musl", "i686-linux-musl", "i686-openkal" }, + -- ... +} +``` + +`mcpp build --target x86_64-uefi` 因此不需要 `mcpp.toml` 中的任何一行。行以系列(`openkal-platform@*`)选择平台包, +版本的选择、锁定与更新同上,报告的来源为 `index`。工程自己的 `[target.] platform` 仍然在前,工程自己的 +`[indices]` 先于共享的索引。这份列表由索引的工具从平台的清单生成;2026.11.17.1 之前的引擎忽略它。 + ## 编写平台包 平台包是带 `[platform]` 段的包。openkal-platform 的节选: @@ -191,14 +209,23 @@ mcpp = ">=2026.11.3.1" [platform] targets = ["x86_64-openkal", "aarch64-openkal"] toolchain = "llvm@23.1.3" -runner = { cmd = ["okrun", "{}"], host = "same-arch", hosts = ["linux-x86_64", "linux-aarch64", "macos-aarch64", "windows-x86_64"] } +runner = { + cmd = ["okrun", "{}"], + host = "same-arch", + hosts = ["linux-x86_64", "linux-aarch64", "macos-aarch64", "windows-x86_64"], +} carrier = { package = "openkal-run@0.5.0", section = ".oke", main = "carrier/main.cpp" } plugin = "build.mcpp" [platform.layers] compiler-runtime = "openkal-llvm-runtime@0.19.0" "c++-abi" = "openkal-llvm-runtime@0.19.0" -c-abi = { name = "openkal-musl", version = "0.28.0", default-features = false, features = ["posix", "malloc-mimalloc"] } +c-abi = { + name = "openkal-musl", + version = "0.28.0", + default-features = false, + features = ["posix", "malloc-mimalloc"], +} [platform.tools] okld = "openkal-tools@0.5.0" diff --git a/modules/manifest/src/platform.cppm b/modules/manifest/src/platform.cppm index 5e7394d8b..5acdda3e8 100644 --- a/modules/manifest/src/platform.cppm +++ b/modules/manifest/src/platform.cppm @@ -27,6 +27,7 @@ export namespace mcpp::manifest::platform { // a later tier replaces an earlier one field by field (§4.5). enum class Tier : std::uint8_t { BuiltIn, // the engine's own conventions for a host target + Index, // the target table's platform column, answered by the index (§3.1 step 5) Platform, // the platform package's `[platform]` section Program, // the platform's plugin or a build program (directives) Config, // config.toml: `[target.]`, `[host]` @@ -39,6 +40,7 @@ enum class Tier : std::uint8_t { constexpr std::string_view tier_name(Tier t) { switch (t) { case Tier::BuiltIn: return "built-in"; + case Tier::Index: return "index"; case Tier::Platform: return "platform"; case Tier::Program: return "build-program"; case Tier::Config: return "config.toml"; diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index b15eeaaaa..b563043de 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -25,6 +25,7 @@ import mcpp.config; import mcpp.invocation; import mcpp.manifest; import mcpp.platform; +import mcpp.provider; // the synced index's roots (the platform column, design 2026-11-16 §6.2) import mcpp.pm.compat; import mcpp.project; import mcpp.targetside; @@ -286,7 +287,19 @@ void define_early_platform(PrepareState& state) { } // A request no definition knows, with a platform chosen for it that is not on // this machine yet: a provisional row lets the first pass fetch the platform. - if (raw.empty() || mcpp::toolchain::triple::parse(raw) || !chosen) return; + // With no choice, the index's platform column may answer (design 2026-11-16 + // §6.2). + // A request that parses but names a row nothing defines (`x86_64-toyos`) + // is answered so only by the column. + if (raw.empty()) return; + if (const auto parsed = mcpp::toolchain::triple::parse(raw); + parsed && (chosen || !chosenText.empty() || parsed->defined() + || mcpp::toolchain::triple::find_known_target(*parsed))) + return; + if (!chosen && chosenText.empty()) + if (auto column = index_platform_for(state, raw); !column.empty()) + if (auto ref = mp::parse_ref_text(column)) chosen = *ref; + if (!chosen) return; std::vector seg; for (auto part : std::views::split(std::string_view(raw), '-')) seg.emplace_back(part.begin(), part.end()); if (seg.size() < 2 || seg.size() > 3 || std::ranges::any_of(seg, [](auto const& x) { return x.empty(); })) return; @@ -309,6 +322,111 @@ void define_early_platform(PrepareState& state) { state.provisionalTarget = true; } +std::string index_platform_for(const PrepareState& state, std::string_view triple) { + // One scan of the indices per process: the column is consulted only for a + // row the built-in table does not answer, and the answer does not change + // while one command runs. The project's own indices ([indices] of + // mcpp.toml) rank before the shared ones; one rank's claims are compared. + struct Claim { + int rank; + std::string package; + }; + static std::mutex lock; + static std::optional>> claims; + std::scoped_lock guard(lock); + if (!claims) { + claims.emplace(); + auto cfg = state.get_cfg(false); + if (!cfg) return {}; + // ` = ` at the start of a line or after `{` or `,`: the first such + // key is the package table's own (its keys precede its nested tables). + auto field = [](std::string_view lua, std::string_view key) -> std::size_t { + for (std::size_t pos = 0; (pos = lua.find(key, pos)) != std::string_view::npos; pos += key.size()) { + std::size_t b = pos; + while (b > 0 && (lua[b - 1] == ' ' || lua[b - 1] == '\t')) --b; + std::size_t e = pos + key.size(); + while (e < lua.size() && (lua[e] == ' ' || lua[e] == '\t')) ++e; + if ((b == 0 || lua[b - 1] == '\n' || lua[b - 1] == ',' || lua[b - 1] == '{') && e < lua.size() + && lua[e] == '=' && (e + 1 >= lua.size() || lua[e + 1] != '=')) + return e + 1; + } + return std::string_view::npos; + }; + auto quoted = [](std::string_view lua, std::size_t at) { + std::string out; + const auto q1 = lua.find('"', at), nl = lua.find('\n', at); + if (q1 == std::string_view::npos || (nl != std::string_view::npos && q1 > nl)) return out; + const auto q2 = lua.find('"', q1 + 1); + if (q2 != std::string_view::npos) out = std::string(lua.substr(q1 + 1, q2 - q1 - 1)); + return out; + }; + // (root, rank, the namespace of a descriptor that names none) + std::vector> roots; + const auto add = [&](fs::path dir, int rank, std::string ns) { + if (std::ranges::none_of(roots, [&](auto const& r) { return std::get<0>(r) == dir; })) + roots.emplace_back(std::move(dir), rank, std::move(ns)); + }; + if (state.root && state.m) { + const auto project = state.projectIndexRoot(); + for (auto const& [name, spec] : state.m->indices) + if (spec.is_local()) add(mcpp::config::resolve_project_index_path(*state.root, spec), 0, name); + for (auto const& dir : mcpp::provider::layout().index_roots_of(project / ".mcpp")) + add(dir, 0, dir.filename().string()); + for (auto const& data : mcpp::config::project_xlings_data_roots(project)) + for (auto const& dir : mcpp::provider::layout().index_roots_of(data.parent_path())) + add(dir, 0, dir.filename().string()); + } + for (auto const& dir : mcpp::provider::open(mcpp::config::make_xlings_env(**cfg))->index_roots()) + add(dir, 1, std::string(mcpp::pm::kDefaultNamespace)); + for (auto const& [root, rank, defaultNs] : roots) { + std::error_code ec; + const auto pkgs = root / "pkgs"; + if (!fs::is_directory(pkgs, ec)) continue; + for (auto it = fs::recursive_directory_iterator(pkgs, ec); + !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { + if (it->path().extension() != ".lua") continue; + std::ifstream is(it->path(), std::ios::binary); + const std::string lua{std::istreambuf_iterator(is), {}}; + const auto rows = field(lua, "platform_rows"); + if (rows == std::string_view::npos) continue; + const auto open = lua.find('{', rows), close = lua.find('}', rows); + if (open == std::string::npos || close == std::string::npos || close < open) continue; + const auto nameAt = field(lua, "name"), nsAt = field(lua, "namespace"); + if (nameAt == std::string_view::npos) continue; + const auto ns = nsAt == std::string_view::npos ? defaultNs : quoted(lua, nsAt); + auto name = quoted(lua, nameAt); + // A descriptor may spell its name qualified (`acme.plat`). + if (name.starts_with(ns + ".")) name.erase(0, ns.size() + 1); + if (ns.empty() || name.empty()) continue; + const auto pkg = ns + "." + name; + for (std::size_t p = open; p < close; ++p) { + if (lua[p] != '"') continue; + const auto q = lua.find('"', p + 1); + if (q == std::string::npos || q > close) break; + auto& who = (*claims)[lua.substr(p + 1, q - p - 1)]; + if (std::ranges::none_of(who, [&](auto const& c) { return c.package == pkg; })) + who.push_back({rank, pkg}); + p = q; + } + } + } + } + auto it = claims->find(std::string(triple)); + if (it == claims->end() || it->second.empty()) return {}; + const int first = std::ranges::min(it->second, {}, &Claim::rank).rank; + std::vector who; + for (auto const& c : it->second) + if (c.rank == first) who.push_back(c.package); + std::ranges::sort(who); + if (who.size() > 1) { + std::string names; + for (auto const& w : who) names += (names.empty() ? "" : ", ") + w; + mcpp::ui::warning(std::format("the package index lists {} for {}; {} is taken. The index's own check " + "allows one platform for a row", names, triple, who.front())); + } + return who.front() + "@*"; +} + std::string platform_short_name(std::string_view ref) { auto r = mp::parse_ref_text(ref); return r ? known_short_name(*r) : std::string{}; @@ -324,10 +442,27 @@ std::string named_provider(const PrepareState& state, const mcpp::toolchain::tri const auto project = project_tables(*state.m); mcpp::platform::ChoiceInputs in{ flag, env, envKey, project, config, {} }; auto sel = mcpp::platform::select_target(in, matcher(cfgpred::context_for(t.str()))); - if (!sel || sel->origin.tier == mp::Tier::BuiltIn) return {}; + if (!sel) return {}; + // Nothing chose: the selection's answer is "none" from the (empty) column. + const bool unchosen = sel->none && sel->origin.tier == mp::Tier::BuiltIn; + if (!unchosen && sel->origin.tier == mp::Tier::BuiltIn) return {}; if (!sel->builtin.empty()) return sel->builtin; - if (sel->none) return "none"; - const auto chosen = known_short_name(sel->ref); + if (sel->none && !unchosen) return "none"; + std::string chosen; + if (!unchosen) { + chosen = known_short_name(sel->ref); + } else { + // Nothing chose, and the table has no platform for a `planned` row: the + // index's column names the platform that serves it (design 2026-11-16 + // §6.2), as a choice of the project would. + const auto* known = mcpp::toolchain::triple::find_known_target(t); + if (!known || known->tier != "planned" || t.defined()) return {}; + const auto column = index_platform_for(state, t.str()); + if (column.empty()) return {}; + const auto ref = mp::parse_ref_text(column); + if (!ref) return {}; + chosen = known_short_name(*ref); + } if (chosenOut) *chosenOut = chosen; for (auto const* p : mcpp::toolchain::target::providers()) { if (p->platform.empty()) continue; @@ -390,7 +525,22 @@ std::expected step1_select_platform(PrepareState& state) { std::string column; if (state.selectedTarget) column = std::string(state.selectedTarget->platform); else if (auto* row = mcpp::toolchain::triple::find_known_target(t)) column = std::string(row->platform); + // THE INDEX ANSWERS WHERE THE BUILT-IN TABLE DOES NOT (design 2026-11-16 + // §6.2): a row the table does not hold (a platform defines it), or holds + // as planned. A row the table answers -- a host row included, which a + // platform's `targets` may also list -- is never taken from the index. + bool fromIndex = false; + { + const auto* known = mcpp::toolchain::triple::find_known_target(t); + const bool tableAnswers = known && known->tier != "planned" && !t.defined(); + if (!tableAnswers) + if (auto indexed = index_platform_for(state, t.str()); !indexed.empty()) { + column = std::move(indexed); + fromIndex = true; + } + } mcpp::platform::ChoiceInputs in{ flag, env, envKey, project, config, column }; + in.columnFromIndex = fromIndex; if (state.overrides.platform_choice.empty() && !stated_on_command_line("MCPP_PLATFORM")) in.flagKey = "MCPP_PLATFORM"; auto sel = mcpp::platform::select_target(in, matcher(cfgpred::context_for(t.str()))); diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index 607610c84..92b2782bd 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -1347,6 +1347,11 @@ std::string named_provider(const PrepareState& state, const mcpp::toolchain::tri std::string* platform = nullptr, std::string* chosen = nullptr); // The short name of the package a reference names (`path:` reads its manifest). std::string platform_short_name(std::string_view ref); +// THE PLATFORM COLUMN FROM THE INDEX (design 2026-11-16 §6.2; SPEC-013 §3.1 +// step 5): the platform package whose index entry lists `triple` among its +// `platform_rows`, as `.@*` (the series of §14), when the +// built-in table has no answer for it. Empty when no entry claims the row. +std::string index_platform_for(const PrepareState& state, std::string_view triple); void step4b_seed_platform(PrepareState& state); std::expected step4b_apply_platform(PrepareState& state); diff --git a/src/platform/platform.cppm b/src/platform/platform.cppm index 183018e1c..71b4afadf 100644 --- a/src/platform/platform.cppm +++ b/src/platform/platform.cppm @@ -69,6 +69,7 @@ struct ChoiceInputs { std::span project; // the plan root's tables, the workspace layer's first std::span config; // config.toml's std::string_view column; // the target table's platform column + bool columnFromIndex = false; // the column's answer is the index's std::string flagKey = "--platform"; // or MCPP_PLATFORM, from the environment }; @@ -325,7 +326,9 @@ std::expected select_target(const ChoiceInputs& in, cons return s; } if (!in.column.empty()) { - auto s = parse_choice(in.column, mp::Origin{mp::Tier::BuiltIn, {}, 0, "the target table"}); + auto s = parse_choice(in.column, in.columnFromIndex + ? mp::Origin{mp::Tier::Index, {}, 0, "the package index"} + : mp::Origin{mp::Tier::BuiltIn, {}, 0, "the target table"}); if (!s) return s; return s; } diff --git a/tests/e2e/openkal/platforms.cpp b/tests/e2e/openkal/platforms.cpp index 6f646021e..7ffa0b052 100644 --- a/tests/e2e/openkal/platforms.cpp +++ b/tests/e2e/openkal/platforms.cpp @@ -1,7 +1,7 @@ // Platform packages: a class of targets a platform states, its version group and its series, // the default platform release, a target a platform defines, the platform skeleton, and a row a // platform defines taking nothing of the payload's host (formerly tests/e2e/952, 1020, 1021, -// 1025, 1040, 1049, 1057). +// 1025, 1040, 1049, 1057), and the package index naming a row's platform (1106). import std; import mcpplibs.mtest; import mcpp.test.support; @@ -482,6 +482,108 @@ flags = ["-e", "_start"] const auto h = s::read_header(*exe); expect(fatal(h.kind == s::format::elf && !h.wide)) << "the program is not ELF32"; }; + "1106 the package index names the platform of a row"_test[s::elf][s::llvm][s::fresh_sandbox] = [] { + // A row no table knows, `x86_64-toyos`, and a `planned` row, `i686-linux-musl`, are listed + // in the `platform_rows` of a descriptor in the project's index: A `--target x86_64-toyos` + // selects that platform with no statement of the project, and the report says the index + // chose; E the planned row takes the platform that serves it; B a statement of the + // project stands over the index; C two descriptors of one index listing the row are + // named, and the first is taken; D without the column the row is unknown and the planned + // row is refused. + const auto platform = [](std::string_view name) { + return std::format(R"([package] +namespace = "acme" +name = "{0}" +version = "0.1.0" + +[platform] +toolchain = "llvm@{1}" + +[[platform.define]] +triple = "x86_64-toyos" +clang-triple = "x86_64-unknown-linux-gnu" +object-format = "elf" +family = "unix" +execution = "linux" + +[[platform.provider]] +serves = ["x86_64-toyos", "i686-linux-musl"] + +[platform.build] +cflags = ["-ffreestanding", "-fno-stack-protector", "-fno-pic", "-DTOY_NAME=\"{0}\""] + +[platform.link] +driver = "compiler" + +[platform.layers] +"c-abi" = false +"c++-abi" = false +"compiler-runtime" = false +"kernel-abi" = false +)", name, s::llvm_version()); + }; + const auto descriptor = [](std::string_view name, std::string_view rows) { + return std::format(R"(package = {{ + spec = "1", namespace = "acme", name = "{0}", + description = "platform column fixture", licenses = {{"MIT"}}, type = "package", + mcpp = "mcpp.toml", +{1} xpm = {{ linux = {{ ["0.1.0"] = {{ url = "u", sha256 = "0" }} }} }}, +}} +)", name, rows); + }; + const std::string rows = " platform_rows = { \"x86_64-toyos\", \"i686-linux-musl\" },\n"; + const auto t = s::case_dir(); + s::project app(t / "app", {{"mcpp.toml", R"([package] +name = "toyapp" +version = "0.1.0" + +[indices] +acme = { path = "local-index" } + +[build] +sources = ["src/*.c"] +ldflags = ["-nostdlib", "-static", "-no-pie"] + +[targets.toyapp] +kind = "bin" +main = "src/main.c" +)"}, + {"local-index/pkgs/t/toy.lua", descriptor("toy", rows)}, + {".mcpp/.xlings/data/xpkgs/acme.toy/0.1.0/mcpp.toml", platform("toy")}, + {".mcpp/.xlings/data/xpkgs/acme.toy2/0.1.0/mcpp.toml", platform("toy2")}, + {"src/main.c", "void _start(void) {\n#ifdef __x86_64__\n __asm__ volatile(\"mov $60, %eax\\n mov $7, %edi\\n syscall\");\n#else\n for (;;) {}\n#endif\n}\n"}}); + // A + auto r = app.mcpp({"build", "--target", "x86_64-toyos"}, inherited); + expect(fatal(e2e::succeeded(r))) << "A: the row the index names did not build: " << s::text(r); + expect(fatal(s::has_line(s::text(r), R"(Platform +toy 0\.1\.0 \(index\))"))) << "A: the report does not say the index chose the platform: " << s::text(r); + // E + // (selected, not linked: the fixture's platform supplies no i686 run-time) + r = app.mcpp({"build", "--target", "i686-linux-musl"}, inherited); + expect(fatal(s::lacks(s::text(r), "not yet supported"))) << "E: the planned row the index names was refused: " << s::text(r); + expect(fatal(s::has_line(s::text(r), R"(Platform +toy 0\.1\.0 \(index)"))) << "E: the planned row did not take the index's platform: " << s::text(r); + // B + const auto plain = app.read("mcpp.toml"); + app.write("mcpp.toml", plain + "\n[target.x86_64-toyos]\nplatform = \"acme.toy2@0.1.0\"\n"); + app.write("local-index/pkgs/t/toy2.lua", descriptor("toy2", "")); + r = app.mcpp({"build", "--target", "x86_64-toyos"}, inherited); + expect(fatal(e2e::succeeded(r))) << "B: the stated platform did not build: " << s::text(r); + expect(fatal(s::has_line(s::text(r), R"(Platform +toy2 0\.1\.0)"))) << "B: the index stood over the project's statement: " << s::text(r); + app.write("mcpp.toml", plain); + // C + app.write("local-index/pkgs/t/toy2.lua", descriptor("toy2", rows)); + fs::remove_all(app.at("target")); + r = app.mcpp({"build", "--target", "x86_64-toyos"}, inherited); + expect(fatal(e2e::succeeded(r))) << "C: the build with two claims failed: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), "lists acme.toy, acme.toy2 for x86_64-toyos"))) << "C: the two claims are not named: " << s::text(r); + expect(fatal(s::has_line(s::text(r), R"(Platform +toy 0\.1\.0)"))) << "C: the first claim was not taken: " << s::text(r); + // D + app.write("local-index/pkgs/t/toy.lua", descriptor("toy", "")); + app.write("local-index/pkgs/t/toy2.lua", descriptor("toy2", "")); + r = app.mcpp({"build", "--target", "x86_64-toyos"}, inherited); + expect(fatal(e2e::contains(s::text(r), "unknown target 'x86_64-toyos'"))) << "D: without the column the row was known: " << s::text(r); + r = app.mcpp({"build", "--target", "i686-linux-musl"}, inherited); + expect(fatal(e2e::contains(s::text(r), "registered but not yet supported (planned)"))) << "D: without the column the planned row was built: " << s::text(r); + }; }}; } // namespace diff --git a/tests/unit/test_platform.cpp b/tests/unit/test_platform.cpp index 863bce5ae..6ca761442 100644 --- a/tests/unit/test_platform.cpp +++ b/tests/unit/test_platform.cpp @@ -286,6 +286,17 @@ platform = "machine-platform@1.0.0" expect(eq(pick("", "", false, false, "row-platform@1.0.0"), "row-platform|built-in")); expect(eq(pick("", "", false, false, ""), "none|built-in")); expect(eq(pick("none", "", true, true, "row-platform@1.0.0"), "none|flag")); + // The column answered by the index (design 2026-11-16 §6.2): the same place, its own source. + pf::ChoiceInputs indexed{ "", "", "", {}, {}, "acme.row-platform@*" }; + indexed.columnFromIndex = true; + auto s = pf::select_target(indexed, t); + expect(fatal(bool(s))); + expect(eq(s->ref.spec.shortName, "row-platform")); + expect(eq(std::string(mp::tier_name(s->origin.tier)), "index")); + indexed.project = m.platformOverrides; + s = pf::select_target(indexed, t); + expect(fatal(bool(s))); + expect(eq(s->ref.spec.shortName, "project-platform")) << "the project's statement comes before the index"; }}; UTest the_most_specific_selector_decides{"PlatformSelect/TheMostSpecificSelectorDecides", [] { From 422439c6102882da6f4141f93e2689906d511249 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 20:40:09 +0900 Subject: [PATCH 13/36] docs: multi-line inline tables are 2026.11.17.1+, and the changelog says so (#929) --- CHANGELOG.md | 4 ++++ docs/04-mcpp-toml.md | 4 ++-- docs/zh/04-mcpp-toml.md | 2 +- 3 files changed, 7 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aa707e41c..e770c4761 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,10 @@ first, the project's `[indices]` rank before the shared ones, and the report names the source `index`. The entries are read once per command, and only for such a row (design 2026-11-16 §6.2, SPEC-013 §14). +- An inline table may span several lines, carry comments and end with a comma + after its last pair, as TOML 1.1 allows, in every TOML file mcpp reads; a + pair stays on one line. `mcpp add` and `mcpp remove` replace and erase such a + value whole, and `mcpp index pin` refuses one, saying why (#928, #929). ### Changed diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 5e52f7fd2..630fac5f2 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -14,8 +14,8 @@ own — dependencies are [05](05-dependencies.md), features are `mcpp.toml` is the project configuration file for the mcpp build tool, analogous to Cargo's `Cargo.toml` or Node's `package.json`. Place it in the project root; `mcpp build` discovers and reads it automatically. The file is TOML. An inline table may span several lines, carry comments and -end with a comma after its last pair, as TOML 1.1 allows; each `key = value` -pair stays on one line (#928). The same holds for every TOML file mcpp reads +end with a comma after its last pair, as TOML 1.1 allows (mcpp 2026.11.17.1+); +each `key = value` pair stays on one line (#928). The same holds for every TOML file mcpp reads (`mcpp.lock`, `config.toml`, template metadata). ```toml diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 0f1d96d5b..96a6f511d 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -14,7 +14,7 @@ Node 的 `package.json`。把它放在工程根目录;`mcpp build` 会自动发现并读取它。 该文件是 TOML。内联表可以跨多行书写、包含注释,并在最后一对之后带一个逗号, -这与 TOML 1.1 的规定一致;每一对 `key = value` 仍写在一行之内(#928)。mcpp +这与 TOML 1.1 的规定一致(mcpp 2026.11.17.1+);每一对 `key = value` 仍写在一行之内(#928)。mcpp 读取的每个 TOML 文件(`mcpp.lock`、`config.toml`、模板元数据)都遵循同一规则。 ```toml From 0f6c252f49dfa702a91352924b4ad0aaf77f7737 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 20:41:46 +0900 Subject: [PATCH 14/36] Design record: part 2 v6, measurements and the decisions taken while implementing --- ...6-ecosystem-gap-supplement-part2-design.md | 54 ++++++++++++++++--- 1 file changed, 46 insertions(+), 8 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index ca019e00c..a67ba0400 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -5,7 +5,7 @@ status: draft # 生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项 -**日期:** 2026-11-16 · **版本:** v5(第 15 节的十二项已定;第 13 节为双向交叉评审;第 16、17 节为实施计划及其评审) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 +**日期:** 2026-11-16 · **版本:** v6(实施中:第 18 节记录实测结果与实施中的决定;v5 为第 15 节的十二项与第 16、17 节的计划) · **基线:** mcpp 2026.11.16.1(main 另含 #929:多行内联表)、 openkal-platform 0.29.0、dirun 0.5.0、ovmf 9.2.4-3、mcpp:plugins 0.22.0(平台以 0.20.0 构建)、xlings v2026.10.26.1 · **前文:** [第一部分](2026-11-15-ecosystem-gap-supplement-design.md)(N1–N47,13.12–13.17 为本文的来源) @@ -302,7 +302,7 @@ runner = { **v1 的 `[platforms]` 表被否定**:openkal-platform 的 `targets` 还列有 `x86_64-linux-gnu` 等宿主行,按平台整体 选择会把本机构建也交给平台;它又是一个新的清单概念。按行查找的平台列没有这两个问题。 -**待实测**:早于本版本的引擎读到条目中的新字段时是否忽略;索引同步之后查找的开销。 +**实测(M4,第 18 节)**:字段名为 `platform_rows`;mcpp 2026.11.16.1 读到带该字段的条目时忽略它;查找的开销见第 18 节。 ### 6.3 索引数据的清理(不改引擎) @@ -363,13 +363,16 @@ runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符 块基址是固定的,平坦模型下 DS 与 SS 共用同一个描述符,栈放到低端就要移动全部段的地址。 - **退路**:DPMI 宿主不支持 `0507h` 时,在栈底写入哨兵值,程序结束时检查;被改写时以专用状态报告“栈越界”,并 在启动时以一行诊断说明保护页不可用(做得少时不沉默)。 -- **待实测**:CWSDPMI r7 与 DOSBox Staging 内置的 DPMI 对 `0507h` 的支持。 +- **实测(M1)**:DOSBox Staging 中的 CWSDPMI r7 支持 `0507h`:保护页之下的记录可读,保护页本身产生故障。哨兵值 + 只作为不支持时的退路保留。 - **测试**:#3 的最小复现程序在 8 KiB 栈帧下递归 64 层后正常返回;声明 64 KiB 栈时,以 `#PF` 报告。 ### 8.3 N45:dist-web 与平台的插件版本 - mcpp.plugins:dist-web 找不到它要包装的启动文件时,以警告写明查找过的路径(N30 的机制)。 - openkal-platform 0.29.1 改为以 mcpp.plugins 的最新版本构建,使选用平台行的工程可以使用新的 dist-web。 +- **实施结论(P4)**:mcpp.plugins 0.22.0 的 dist-web 已经以警告写出它查找过的路径,引擎已把提供者的警告并入 + 错误信息,不需要新版本。N45 余下的部分是 P9。 ### 8.4 N47:dirun 0.6.0,可写的 FAT 映像 @@ -433,12 +436,16 @@ runner 已有规范:SPEC-013 §10 规定声明、查找、执行、占位符 - **兼容**:没有给出模板的工程不变。给出模板而又面向 Emscripten 的工程,产物名恢复为带 `.js`;这正是它们现在 以条件判断得到的结果。 -### 8.7 FreeBSD 的行加上 runner(已定) +### 8.7 FreeBSD 的行加上 runner(已定;v6 修订归属) - FreeBSD 的四行在内置表中,目前没有 runner;运行验证只能在 CI 的虚拟机动作中手工进行。 - 与 8.5 的三个系统使用同一机制:xim 的 `freebsd-disk-image` 包带 dirun 的配置,行的 runner 为 `dirun --profile {pkg:freebsd-disk-image}/dirun.toml`,`enabled = false`。Android 的行已是这一写法。 -- 数据写在 mcpp 的内置表中(这些行本来就在那里);映像包在 xim-pkgindex。 +- **归属(v6,2026-11-16 决定)**:runner 只能写在提供者上。内置表中原生的两行(`-freebsd`)按 2026-11-10 + E4 没有提供者:系统 ABI 由声明 sysroot 的平台提供,未选平台时拒绝。原生行的提供者(`xim:freebsd-sysroot`、 + llvm、上述 runner)因此放在 `mcpp.platforms`,与 OpenBSD、NetBSD、illumos 相同;配合 6.2,`--target + x86_64-freebsd` 的工程不写平台。musl 的两行由 openkal-platform 的 `freebsd-vm` 提供者服务,runner 在 0.29.0 + 已经写出。引擎本轮不为此修改;v5 的“数据写在 mcpp 的内置表中”作废。 - 映像的要求对四个系统相同,写进 `mcpp.platforms` 的收录规范:串口控制台、自动登录、不需要网络、dirun 能把 文件送入并取得退出状态。 @@ -644,7 +651,7 @@ xim-pkgindex(sysroot、映像)─> mcpp-platforms ─> mcpp-index(mcpp.pla 8. DOS 的缺省栈大小写在平台行的数据中;通用键 `stack_size` 本轮不加(8.2)。 9. N48:行的数据陈述后缀是否为加载所必需(8.6)。 -10. FreeBSD 的行加上 runner,与三个新系统使用同一机制(8.7)。 +10. FreeBSD 的行加上 runner,与三个新系统使用同一机制(8.7);原生行的提供者在 `mcpp.platforms`(v6)。 11. 公开的安装动作 `mcpp-community/setup-mcpp`(8.8)。 12. 实施计划(第 16 节);两个新仓库由实施者创建,内容经 PR #1、CI 通过之后合入(16.4)。 @@ -662,7 +669,7 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 | P2 | openkal-dos | ① 读 `PT_GNU_STACK` ② 保护页 ③ 哨兵值与诊断 ④ 复现程序进入 CI(8.2) | 0.1.1 | | P3 | dirun | ① FAT32 映像生成模块与单元测试 ② `[payload] disk` ③ OVMF 中写入并读出的检查(8.4) | 0.6.0 | | P4 | mcpp.plugins | dist-web 写明原因(8.3) | 补丁 | -| P5 | mcpp(复用 #944,改标题) | ① `deploy` 的条目语法 ② `license_files` ③ 运行文件、工作目录与契约 ④ 平台列查索引 ⑤ N48 ⑥ FreeBSD 的 runner 数据 ⑦ 文档改用多行内联表 | 下一版 | +| P5 | mcpp(复用 #944,改标题) | ① `deploy` 的条目语法 ② `license_files` ③ 运行文件、工作目录与契约 ④ 平台列查索引 ⑤ N48 ⑥ (v6:移至 P10)⑦ 文档改用多行内联表 | 下一版 | | P6 | setup-mcpp(新) | 动作与它的 CI(8.8) | v1 | | P7a | xim-pkgindex | dirun 0.6.0、ovmf 9.2.4-4 | — | | P7b | xim-pkgindex | 三个系统的 sysroot;四个系统的磁盘映像 | — | @@ -670,7 +677,7 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 | P8b | mcpp-index | openkal-platform 0.29.1;平台条目中行的列表及其生成、校验工具 | — | | P8c | mcpp-index | `mcpp.platforms` 0.1.0;`compat.onnxruntime` 改用 `license_files`;`library_dirs` 的清理 | — | | P9 | openkal-platform | 新成员、DOS 的缺省栈、ovmf r4 与 dirun 0.6.0、新版 plugins、UEFI 检查 | 0.29.1 | -| P10 | mcpp-platforms(新) | ① 收录规范 ② OpenBSD ③ NetBSD ④ illumos,每个系统先构建后运行(8.5) | 0.1.0 | +| P10 | mcpp-platforms(新) | ① 收录规范 ② OpenBSD ③ NetBSD ④ illumos,每个系统先构建后运行(8.5)⑤ 原生 FreeBSD 的提供者(8.7,v6) | 0.1.0 | | P11 | 外部验证工程(fork 的 PR) | 改用 `setup-mcpp`、`--file`;去掉平台的四行与两处过时绕行;三个新行;FreeBSD 与 Android 经 `mcpp run` | — | ### 16.2 依赖 @@ -740,3 +747,34 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 | 测试 | 每个阶段点是否有自动化验证 | 有:第 11 节;P11 是端到端的检查 | | 资源 | 磁盘映像的体积与本地存储 | 制作前清理过期的临时文件;体积在 M3 中实测并记录 | | 风险 | 最不确定的部分 | M3(映像的制作与分发)与 P5④(解析顺序中插入索引查找)。两者都排在最早开始 | + +## 18. 实施记录(v6) + +### 18.1 实测结果 + +| 项 | 结果 | +|---|---| +| M1 | DOSBox Staging 中的 CWSDPMI r7 支持 `0507h`(8.2)。 | +| M4 | mcpp 2026.11.16.1 读到带 `platform_rows` 的 openkal-platform 条目时照常构建 `x86_64-openkal`,字段被忽略。索引先于引擎带上这一列没有影响,P8b 不需要提高索引的引擎下限。 | +| 查找开销 | 只在内置表没有答案的行上扫描,每条命令至多一次。约 400 个条目时,对一个未知行的完整拒绝过程用时 0.05 s;有答案的行不扫描。 | +| N47 | OVMF 中的 UEFI 程序在 `disk = "fat-image"` 的盘上写入文件并读回;只读的 `directory` 盘上写入失败(状态 3)。 | + +### 18.2 平台列查索引的实现要点(6.2) + +- 条目的键为 `platform_rows`,值为行的列表;平台包以系列 `<命名空间>.<名字>@*` 选择,版本按 SPEC-013 §14 的 + 系列规则取得并锁定。 +- 内置表没有答案的三种行查索引:引擎不认识的行、平台定义的行、`planned` 行。`planned` 行与工程选择平台时相同, + 由认领它的平台服务后可以构建。 +- 工程自己的索引(`[indices]`)先于共享的索引;同一层两个认领时取名字最小者并给出警告。 +- 规范:SPEC-013 v1.11(§3.1 第 5 步、§14、判据 16),e2e 1106。 + +### 18.3 实施中的发现 + +- **mimalloc 的随机数(待定)**:openkal-uefi 的程序在 OVMF 中启动时输出 + `mimalloc: warning: thread …: unable to use secure randomness`。UEFI 下 mimalloc 没有取得安全随机数的途径; + 可行的做法是由 openkal-uefi 经 `EFI_RNG_PROTOCOL` 提供,不可用时静默退回。不在本轮范围内,记为后续项。 +- **dirun 在 macOS 上的文件大小**:FAT 映像的写入在 macOS 上把一个三级目录中的文件记为 0 字节;改为向文件 + 本身询问大小(`std::filesystem::file_size(path)`),并在测试中先检查源文件的大小,结果待 CI 确认。 +- **磁盘空间**:本机 `/home` 的可用空间一度降到 1 GB 以下,e2e 因 `No space left on device` 失败;清理过期的临时 + 目录与构建缓存之后重跑通过。磁盘映像的制作(M3、P7b)需要先确认空间。 + From f130967a5bbbfb66822f1654690bb88a04c37fd1 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 20:48:14 +0900 Subject: [PATCH 15/36] 2026.11.17.1: the version and the changelog's section --- CHANGELOG.md | 2 +- mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e770c4761..9af127072 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. -## [Unreleased] +## [2026.11.17.1] - 2026-11-17 ### Added diff --git a/mcpp.toml b/mcpp.toml index 0934aeb2c..fcdf59be1 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.11.16.1" +version = "2026.11.17.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index e3c2cc7ec..ab13bfea1 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.11.16.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.11.17.1"; } // namespace mcpp From c9996fd7b6b856368a5ba9fdc3f85e33287639bd Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 20:59:42 +0900 Subject: [PATCH 16/36] Design record: part 2 is active; the .agents/docs index; the dirun test's cause --- .../2026-11-16-ecosystem-gap-supplement-part2-design.md | 7 ++++--- .agents/docs/README.md | 4 ++-- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index a67ba0400..04e9b08b5 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -1,6 +1,6 @@ --- subject: cross-platform -status: draft +status: active --- # 生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项 @@ -773,8 +773,9 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 - **mimalloc 的随机数(待定)**:openkal-uefi 的程序在 OVMF 中启动时输出 `mimalloc: warning: thread …: unable to use secure randomness`。UEFI 下 mimalloc 没有取得安全随机数的途径; 可行的做法是由 openkal-uefi 经 `EFI_RNG_PROTOCOL` 提供,不可用时静默退回。不在本轮范围内,记为后续项。 -- **dirun 在 macOS 上的文件大小**:FAT 映像的写入在 macOS 上把一个三级目录中的文件记为 0 字节;改为向文件 - 本身询问大小(`std::filesystem::file_size(path)`),并在测试中先检查源文件的大小,结果待 CI 确认。 +- **dirun 的测试在 macOS 上失败**:FAT 的测试在 macOS 上读到一个文件为 0 字节。原因在测试:同一测试程序的用例可以 + 同时运行,四个用例共用一个源目录,一个用例重建目录时另一个正在写映像。改为每个用例一个源目录;写入方以 + `std::filesystem::file_size(path)` 询问大小并报告错误。 - **磁盘空间**:本机 `/home` 的可用空间一度降到 1 GB 以下,e2e 因 `No space left on device` 失败;清理过期的临时 目录与构建缓存之后重跑通过。磁盘映像的制作(M3、P7b)需要先确认空间。 diff --git a/.agents/docs/README.md b/.agents/docs/README.md index eee2d2ec2..878890923 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -52,7 +52,7 @@ Records that declare one. Everything else is listed by date below. ### cross-platform -- [生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项](2026-11-16-ecosystem-gap-supplement-part2-design.md) — draft +- [生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项](2026-11-16-ecosystem-gap-supplement-part2-design.md) — active - [生态缺口补充方案:一个 C 程序走遍全部目标所暴露的通用问题](2026-11-15-ecosystem-gap-supplement-design.md) — active - [生态方案:跨平台通用基础设施(mcpp + openkal + xim),以一个 C89 工程为验证](2026-11-10-cross-platform-ecosystem-design.md) — active @@ -203,7 +203,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-11 -- [生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项](2026-11-16-ecosystem-gap-supplement-part2-design.md) — draft +- [生态缺口补充方案(第二部分):文件的角色、运行文件、清单书写与第一部分的遗留项](2026-11-16-ecosystem-gap-supplement-part2-design.md) — active - [生态缺口补充方案:一个 C 程序走遍全部目标所暴露的通用问题](2026-11-15-ecosystem-gap-supplement-design.md) — active - [发布的发布面与 macOS 链接的 SDK 契约(2026-11-14 补充)](2026-11-14-release-publication-and-macos-sdk-design.md) — active - [CI 与发布的时间成本:按测量调度的测试、分片、并行发布](2026-11-14-ci-and-release-time-cost-design.md) — active From 385a3251cea8e01624c90b1dc45e56890f1d120c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Mon, 16 Nov 2026 21:28:21 +0900 Subject: [PATCH 17/36] platform: a manifest's [[platform.system]] reaches the platform reader --- CHANGELOG.md | 7 +++++++ modules/manifest/src/toml.cppm | 8 +++++--- tests/unit/test_platform.cpp | 25 +++++++++++++++++++++++++ 3 files changed, 37 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9af127072..87cb6add2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,6 +62,13 @@ `wasm32-emscripten` row states it; a platform states it for its own rows. The engine reads no system name for this (design 2026-11-16 N48). +### Fixed + +- A platform package's `[[platform.system]]` is read: the manifest refused the + array of tables before the platform reader saw it, so a platform could define + rows for a system but not state the system (`uname`, `resources`, + `command-scripts`; SPEC-016 §2.3). + ## [2026.11.16.1] - 2026-11-16 ### Changed diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index af407dfcb..6b90d94c2 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -1112,10 +1112,11 @@ std::expected parse_document(const t::Document& documen // 2026-10-29 §4.5): the platform reader type-checks every entry. "platform.carrier", "target.*.carrier", - // `[[platform.define]]` and `[[platform.provider]]` (SPEC-016 §2, §3): - // the platform reader type-checks every entry. + // `[[platform.define]]`, `[[platform.provider]]` and `[[platform.system]]` + // (SPEC-016 §2, §2.3, §3): the platform reader type-checks every entry. "platform.define", "platform.provider", + "platform.system", }; if (auto badPath = find_disallowed_array_of_tables(doc->root(), "", kAllowedArraysOfTables)) { return std::unexpected(error(origin, std::format( @@ -1123,7 +1124,8 @@ std::expected parse_document(const t::Document& documen "array-of-tables syntax is only supported for [[build.flags]], " "[[features..flags]], [[runtime.requirements]], " "[[runtime.artifacts]], [[runtime.deploy]], [[platform.link.inputs]], " - "[[platform.carrier]], [[platform.define]], [[platform.provider]] and [xlings] deps entries", + "[[platform.carrier]], [[platform.define]], [[platform.provider]], [[platform.system]], " + "[[test.files]] and [xlings] deps entries", *badPath, *badPath))); } diff --git a/tests/unit/test_platform.cpp b/tests/unit/test_platform.cpp index 6ca761442..3b34a16a9 100644 --- a/tests/unit/test_platform.cpp +++ b/tests/unit/test_platform.cpp @@ -119,6 +119,31 @@ UTest every_group_is_read{"PlatformManifest/EveryGroupIsRead", [] { expect(gt(p.runner.origin.line, 0)); }}; +// A platform names the systems of the rows it defines (SPEC-016 §2.3): the manifest +// accepts the array of tables the platform reader reads (refused before 2026.11.17.1). +UTest a_system_is_read{"PlatformManifest/ASystemIsRead", [] { + auto m = parse(R"([package] +name = "rows" +version = "0.1.0" + +[[platform.system]] +name = "netbsd" +uname = "NetBSD" + +[[platform.define]] +triple = "x86_64-netbsd" +clang-triple = "x86_64-unknown-netbsd10.2" +execution = "netbsd" + +[[platform.provider]] +serves = ["x86_64-netbsd"] +)"); + expect(fatal(bool(m.platformDeclared))); + expect(fatal(eq(m.platform.systems.size(), 1u))); + expect(eq(m.platform.systems[0].name, "netbsd")); + expect(eq(m.platform.systems[0].uname, "NetBSD")); +}}; + UTest refusals_name_the_key{"PlatformManifest/RefusalsNameTheKey", [] { const std::string head = "[package]\nname = \"p\"\nversion = \"1.0.0\"\n"; struct Row { std::string body; std::string_view says; }; From 9e9884112865eb31c6a216f4b1dba52468d7ca3c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 04:12:02 +0900 Subject: [PATCH 18/36] tests: 1104's runner is declared for the host's own row; 1105 states its host file for Windows too and names its copy apart from the original on case-insensitive file systems; the design record states the dirun test's cause as found --- ...26-11-16-ecosystem-gap-supplement-part2-design.md | 6 +++--- tests/e2e/run/contract.cpp | 12 +++++++----- tests/e2e/test/selection.cpp | 7 ++++--- 3 files changed, 14 insertions(+), 11 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index 04e9b08b5..b86990239 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -773,9 +773,9 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 - **mimalloc 的随机数(待定)**:openkal-uefi 的程序在 OVMF 中启动时输出 `mimalloc: warning: thread …: unable to use secure randomness`。UEFI 下 mimalloc 没有取得安全随机数的途径; 可行的做法是由 openkal-uefi 经 `EFI_RNG_PROTOCOL` 提供,不可用时静默退回。不在本轮范围内,记为后续项。 -- **dirun 的测试在 macOS 上失败**:FAT 的测试在 macOS 上读到一个文件为 0 字节。原因在测试:同一测试程序的用例可以 - 同时运行,四个用例共用一个源目录,一个用例重建目录时另一个正在写映像。改为每个用例一个源目录;写入方以 - `std::filesystem::file_size(path)` 询问大小并报告错误。 +- **dirun 的测试在 macOS 上失败**:FAT 的测试在 macOS 上读到一个文件为 0 字节。原因在测试:查找函数返回指向临时 + 列表元素的指针,列表在表达式结束时销毁,读到的是已释放的内存(Linux 上碰巧保留了原值)。改为先保存列表; + 写入方另以 `std::filesystem::file_size(path)` 询问大小并报告错误。 - **磁盘空间**:本机 `/home` 的可用空间一度降到 1 GB 以下,e2e 因 `No space left on device` 失败;清理过期的临时 目录与构建缓存之后重跑通过。磁盘映像的制作(M3、P7b)需要先确认空间。 diff --git a/tests/e2e/run/contract.cpp b/tests/e2e/run/contract.cpp index 03d0e1dbc..79c8f8f7c 100644 --- a/tests/e2e/run/contract.cpp +++ b/tests/e2e/run/contract.cpp @@ -372,11 +372,13 @@ E2E contract_machine{"contract-machine", resource{"machine"}, [] { expect(fatal(r.started && r.status == 2 && e2e::contains(s::text(r), "missing.txt does not exist"))) << s::text(r); s::write_executable(work / "runner.sh", "#!/bin/sh\necho \"runner-cwd=$(basename \"$PWD\" | cut -c1-4)\"\n" "cat \"$MCPP_RUNTIME_FILES\"\ngrep -c '\"dest\": \"IN.TXT\"' \"$MCPP_RUN_CLOSURE\"\nexec \"$@\"\n"); - auto machine = s::run(*s::on_path("uname"), {"-m"}).out; - machine.erase(machine.find_last_not_of("\r\n") + 1); - app.write("mcpp.toml", std::format("[package]\nname = \"app\"\nversion = \"0.1.0\"\n[target.{}-linux-gnu]\nrunner = [\"{}\"]\n", - machine, (work / "runner.sh").generic_string())); - r = app.mcpp({"run", "--target", machine + "-linux-gnu", "--file", "data/in.txt=IN.TXT", "--", "IN.TXT"}); + // The runner is declared for the host's own row, which every host builds. + auto host = app.mcpp({"--print-target"}).out; + host.erase(host.find_last_not_of("\r\n") + 1); + expect(fatal(!host.empty())) << "mcpp --print-target named no row"; + app.write("mcpp.toml", std::format("[package]\nname = \"app\"\nversion = \"0.1.0\"\n[target.{}]\nrunner = [\"{}\"]\n", + host, (work / "runner.sh").generic_string())); + r = app.mcpp({"run", "--target", host, "--file", "data/in.txt=IN.TXT", "--", "IN.TXT"}); expect(fatal(r.started && r.status == 3)) << s::text(r); expect(fatal(s::has_line(s::text(r), "^runner-cwd=run-"))) << "the runner did not start in the run's directory: " << s::text(r); expect(fatal(s::has_line(s::text(r), "^IN.TXT\t.*/run/run-[^/]*/IN.TXT$"))) << "MCPP_RUNTIME_FILES does not list the run file: " << s::text(r); diff --git a/tests/e2e/test/selection.cpp b/tests/e2e/test/selection.cpp index 3429723da..f7853366b 100644 --- a/tests/e2e/test/selection.cpp +++ b/tests/e2e/test/selection.cpp @@ -280,13 +280,14 @@ E2E selection{"selection", [] { {"tests/data/in.txt", "payload\n"}, {"tests/data/sub/d.txt", "deep\n"}, {"tests/host/h.txt", "host\n"}, {"tests/reads.cpp", "import std;\nint main() {\n" - " std::ifstream a(\"IN.TXT\"), b(\"in.txt\"), c(\"sub/d.txt\"), d(\"h.txt\");\n" + " std::ifstream a(\"COPY.TXT\"), b(\"in.txt\"), c(\"sub/d.txt\"), d(\"h.txt\");\n" " std::string w, x, y, z; std::getline(a, w); std::getline(b, x); std::getline(c, y); std::getline(d, z);\n" " std::ofstream(\"written\") << 1;\n" " return w == \"payload\" && x == \"payload\" && y == \"deep\" && z == \"host\" ? 0 : 1;\n}\n"}, {"tests/fails.cpp", "import std;\nint main() { std::ofstream(\"evidence.txt\") << 1; return 4; }\n"}}); - p.write("mcpp.toml", manifest("p") + "\n[test]\nfiles = [\"tests/data/**\", { from = \"tests/data/in.txt\", name = \"IN.TXT\" }]\n" - "\n[target.'cfg(unix)'.test]\nfiles = [\"tests/host/h.txt\"]\n"); + p.write("mcpp.toml", manifest("p") + "\n[test]\nfiles = [\"tests/data/**\", { from = \"tests/data/in.txt\", name = \"COPY.TXT\" }]\n" + "\n[target.'cfg(unix)'.test]\nfiles = [\"tests/host/h.txt\"]\n" + "\n[target.'cfg(windows)'.test]\nfiles = [\"tests/host/h.txt\"]\n"); auto r = p.mcpp({"test"}); expect(fatal(r.started && r.status != 0)) << "the failing test passed: " << s::text(r); expect(fatal(s::has_line(s::text(r), "^reads \\.\\.\\. ok"))) << "the test that reads its files failed: " << s::text(r); From 0f43e7d80f00bc7687bc8e2ff31ad3e437d62522 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 04:24:42 +0900 Subject: [PATCH 19/36] platform: the index's column is scanned with no lock held, so a leg that holds the home's lock does not wait on another; the scan is cached per set of roots --- ...6-ecosystem-gap-supplement-part2-design.md | 6 ++ src/build/prepare/platform.cpp | 68 +++++++++++-------- 2 files changed, 46 insertions(+), 28 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index b86990239..6e732b6f8 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -776,6 +776,12 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 - **dirun 的测试在 macOS 上失败**:FAT 的测试在 macOS 上读到一个文件为 0 字节。原因在测试:查找函数返回指向临时 列表元素的指针,列表在表达式结束时销毁,读到的是已释放的内存(Linux 上碰巧保留了原值)。改为先保存列表; 写入方另以 `std::filesystem::file_size(path)` 询问大小并报告错误。 +- **平台列的扫描与并行的腿**:全量 CI 中 e2e 的 images/1033 与 sets/987 超时(状态 137)。这两个用例构建 openkal 的行, + 这些行由内置表的 `[[platform.define]]` 定义,所以每次 prepare 都进入索引的扫描。扫描原先在一个进程级互斥锁内 + 调用配置与提供者,而它们可能等待 home 的锁;目标集合的各条腿在并行的线程中 prepare,持有 home 锁的一条腿再进入 + 扫描时可能形成死锁。这是两个用例共有的、本分支新增的路径;本机不能复现(两个用例本机通过),以 CI 的全量运行 + 验证。改为在不持锁的情况下取得根并扫描,锁只保护结果的缓存;缓存以根的集合为键,一个工作区中根不同的成员各自 + 扫描。 - **磁盘空间**:本机 `/home` 的可用空间一度降到 1 GB 以下,e2e 因 `No space left on device` 失败;清理过期的临时 目录与构建缓存之后重跑通过。磁盘映像的制作(M3、P7b)需要先确认空间。 diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index b563043de..375638a7e 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -323,21 +323,48 @@ void define_early_platform(PrepareState& state) { } std::string index_platform_for(const PrepareState& state, std::string_view triple) { - // One scan of the indices per process: the column is consulted only for a - // row the built-in table does not answer, and the answer does not change - // while one command runs. The project's own indices ([indices] of - // mcpp.toml) rank before the shared ones; one rank's claims are compared. + // One scan of a set of index roots per process: the column is consulted + // only for a row the built-in table does not answer, and the answer does + // not change while one command runs. The project's own indices ([indices] + // of mcpp.toml) rank before the shared ones; one rank's claims are + // compared. The roots are found and the scan runs with no lock held: the + // configuration and the provider may wait on the home's lock, which a + // leg prepared beside this one may hold while it reaches this function. struct Claim { int rank; std::string package; }; + using Claims = std::map>; + auto cfg = state.get_cfg(false); + if (!cfg) return {}; + // (root, rank, the namespace of a descriptor that names none) + std::vector> roots; + const auto add = [&](fs::path dir, int rank, std::string ns) { + if (std::ranges::none_of(roots, [&](auto const& r) { return std::get<0>(r) == dir; })) + roots.emplace_back(std::move(dir), rank, std::move(ns)); + }; + if (state.root && state.m) { + const auto project = state.projectIndexRoot(); + for (auto const& [name, spec] : state.m->indices) + if (spec.is_local()) add(mcpp::config::resolve_project_index_path(*state.root, spec), 0, name); + for (auto const& dir : mcpp::provider::layout().index_roots_of(project / ".mcpp")) + add(dir, 0, dir.filename().string()); + for (auto const& data : mcpp::config::project_xlings_data_roots(project)) + for (auto const& dir : mcpp::provider::layout().index_roots_of(data.parent_path())) + add(dir, 0, dir.filename().string()); + } + for (auto const& dir : mcpp::provider::open(mcpp::config::make_xlings_env(**cfg))->index_roots()) + add(dir, 1, std::string(mcpp::pm::kDefaultNamespace)); + std::string rootsKey; + for (auto const& [root, rank, defaultNs] : roots) rootsKey += std::format("{}\t{}\t{}\n", root.string(), rank, defaultNs); static std::mutex lock; - static std::optional>> claims; - std::scoped_lock guard(lock); + static std::map> scanned; + std::shared_ptr claims; + { + std::scoped_lock guard(lock); + if (auto found = scanned.find(rootsKey); found != scanned.end()) claims = found->second; + } if (!claims) { - claims.emplace(); - auto cfg = state.get_cfg(false); - if (!cfg) return {}; // ` = ` at the start of a line or after `{` or `,`: the first such // key is the package table's own (its keys precede its nested tables). auto field = [](std::string_view lua, std::string_view key) -> std::size_t { @@ -360,24 +387,7 @@ std::string index_platform_for(const PrepareState& state, std::string_view tripl if (q2 != std::string_view::npos) out = std::string(lua.substr(q1 + 1, q2 - q1 - 1)); return out; }; - // (root, rank, the namespace of a descriptor that names none) - std::vector> roots; - const auto add = [&](fs::path dir, int rank, std::string ns) { - if (std::ranges::none_of(roots, [&](auto const& r) { return std::get<0>(r) == dir; })) - roots.emplace_back(std::move(dir), rank, std::move(ns)); - }; - if (state.root && state.m) { - const auto project = state.projectIndexRoot(); - for (auto const& [name, spec] : state.m->indices) - if (spec.is_local()) add(mcpp::config::resolve_project_index_path(*state.root, spec), 0, name); - for (auto const& dir : mcpp::provider::layout().index_roots_of(project / ".mcpp")) - add(dir, 0, dir.filename().string()); - for (auto const& data : mcpp::config::project_xlings_data_roots(project)) - for (auto const& dir : mcpp::provider::layout().index_roots_of(data.parent_path())) - add(dir, 0, dir.filename().string()); - } - for (auto const& dir : mcpp::provider::open(mcpp::config::make_xlings_env(**cfg))->index_roots()) - add(dir, 1, std::string(mcpp::pm::kDefaultNamespace)); + auto built = std::make_shared(); for (auto const& [root, rank, defaultNs] : roots) { std::error_code ec; const auto pkgs = root / "pkgs"; @@ -403,13 +413,15 @@ std::string index_platform_for(const PrepareState& state, std::string_view tripl if (lua[p] != '"') continue; const auto q = lua.find('"', p + 1); if (q == std::string::npos || q > close) break; - auto& who = (*claims)[lua.substr(p + 1, q - p - 1)]; + auto& who = (*built)[lua.substr(p + 1, q - p - 1)]; if (std::ranges::none_of(who, [&](auto const& c) { return c.package == pkg; })) who.push_back({rank, pkg}); p = q; } } } + std::scoped_lock guard(lock); + claims = scanned.try_emplace(rootsKey, std::move(built)).first->second; } auto it = claims->find(std::string(triple)); if (it == claims->end() || it->second.empty()) return {}; From f8db3fdee83ba3ce9350f4531c6a125da2a2e1fc Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 04:24:42 +0900 Subject: [PATCH 20/36] skills: a pull request that needs the full CI is labelled ci:full before its push --- .agents/skills/mcpp-contributing/SKILL.md | 11 +++++++++++ .agents/skills/mcpp-release/SKILL.md | 4 ++++ 2 files changed, 15 insertions(+) diff --git a/.agents/skills/mcpp-contributing/SKILL.md b/.agents/skills/mcpp-contributing/SKILL.md index 019ac5d0c..0f45b1288 100644 --- a/.agents/skills/mcpp-contributing/SKILL.md +++ b/.agents/skills/mcpp-contributing/SKILL.md @@ -195,6 +195,17 @@ gh run view --log-failed # 查看失败日志 `ci-aarch64-fresh-install`、`measure-windows-tool-crt` 与 `pypi-publish` 仍是按路径触发的独立工作流。缓存只在 main 上由一个 job 保存;PR 只恢复。 +**全集与选集。** PR 的 push 触发的运行,e2e 只跑 `pr` 集合,即改动影响到的用例;main 的 push 与手动启动的运行跑 `full`。 +合入前需要全集的 PR(引擎改动、要随下一版发布的改动),做法是给 PR 打上 `ci:full` 标签(要测量开销时再加 +`ci:bench`),然后推送。标签在 push 启动运行时读取,所以要先打标签、后推送。这样 PR 自己的运行就是全集,结果 +直接显示在 PR 的检查中。不要另外以 `gh workflow run ci.yml --ref <分支>` 启动一次运行,也不要取消 PR 的运行: +分支上的运行不出现在 PR 的检查中,两次运行还会同时占用 runner。 + +```bash +gh pr edit --add-label ci:full # 先打标签 +git push # 再推送:这次运行跑 full +``` + **以 PR 实际 required checks 为准,所有未跳过的 required checks 必须通过。** 如果某个平台失败: 1. 下载日志分析原因 2. 修复后 push 到同一分支,CI 自动重跑 diff --git a/.agents/skills/mcpp-release/SKILL.md b/.agents/skills/mcpp-release/SKILL.md index fddedcb09..d4df8a3d3 100644 --- a/.agents/skills/mcpp-release/SKILL.md +++ b/.agents/skills/mcpp-release/SKILL.md @@ -104,6 +104,10 @@ gh pr create --title "chore: bump version to $NEW_VERSION" --body "..." # CI 绿后合入;版本 bump 同样禁止直推 main(见 mcpp-contributing) ``` +要随这一版发布的 PR,合入前必须有一次全集的 CI。做法是给 PR 打上 `ci:full` 标签(需要测量时加 `ci:bench`), +然后推送(标签在 push 启动运行时读取)。这样 PR 自己的运行就是全集,结果在 PR 的检查中可见。不要以 +`gh workflow run ci.yml --ref <分支>` 另开一次运行,也不要取消 PR 的运行(见 mcpp-contributing 第 5 节)。 + ### 3. 创建并推送 tag ```bash From 2333ce12b020a1a481db4a7fbbd7d7055e8286aa Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 05:00:29 +0900 Subject: [PATCH 21/36] platform: a row the table holds with no platform and no built-in provider of that row here (x86_64-freebsd) takes the index's platform rather than a sibling row; table_answers decides it in both places --- CHANGELOG.md | 8 ++++--- docs/35-platforms.md | 4 +++- docs/specs/platforms.md | 8 ++++--- docs/zh/35-platforms.md | 3 ++- src/build/prepare/platform.cpp | 41 +++++++++++++++++++++++++-------- tests/e2e/openkal/platforms.cpp | 27 ++++++++++++++-------- 6 files changed, 64 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 87cb6add2..1f343201a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -41,9 +41,11 @@ - A platform package's index entry lists the rows it serves, `platform_rows = { "x86_64-uefi", ... }`, and a row the built-in table has no platform for takes that package as a series (`.@*`): `mcpp build - --target x86_64-uefi` needs no `[target.x86_64-uefi] platform`, and a - `planned` row such as `i686-linux-musl` builds once a platform claims it. A - row the table answers is unchanged, a project's own statement still comes + --target x86_64-uefi` needs no `[target.x86_64-uefi] platform`, a `planned` + row such as `i686-linux-musl` builds once a platform claims it, and so does a + row the table holds with no platform and no built-in provider here, such as + `x86_64-freebsd`, rather than being completed to a sibling row. A row the + table answers, a host row included, is unchanged, a project's own statement still comes first, the project's `[indices]` rank before the shared ones, and the report names the source `index`. The entries are read once per command, and only for such a row (design 2026-11-16 §6.2, SPEC-013 §14). diff --git a/docs/35-platforms.md b/docs/35-platforms.md index e365c1a50..f5c56b415 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -218,7 +218,9 @@ takes the newest release again. ## A row the index names (2026.11.17.1+) A row the built-in table has no platform for — a row the engine does not know, -such as `x86_64-uefi`, or a `planned` one, such as `i686-linux-musl` — takes the +such as `x86_64-uefi`, a `planned` one, such as `i686-linux-musl`, or one the +table holds with no platform and no built-in provider here, such as +`x86_64-freebsd`, whose system a platform's sysroot provides — takes the platform whose index entry lists it: ```lua diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index e55bd9dd1..d790878d7 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -456,8 +456,9 @@ package = { } ``` -1. 第 5 步先查内置表。内置表没有答案的行,即引擎不认识的行、平台定义的行(SPEC-016 §2)与 `planned` 行,查索引; - 内置表有答案的行不查索引,其选择不变。 +1. 第 5 步先查内置表。内置表没有答案的行查索引:引擎不认识的行、平台定义的行(SPEC-016 §2)、`planned` 行,以及 + 内置表持有、平台列为空而本机没有内置提供者服务该行本身的行(如 `x86_64-freebsd`;选择把请求补全为另一行, + 如 `x86_64-freebsd-musl`,不算内置表回答了它)。内置表有答案的行不查索引,其选择不变,宿主的行也是如此。 2. 认领一行的平台包以系列 `<命名空间>.<名字>@*` 选择,按本节的平台系列取版本并写入锁;选择来源报告为 `index`。 3. 工程自己的索引(`[indices]`)先于共享的索引;同一层有多个平台包认领一行时取名字最小者,并给出警告列出它们。 索引的检查应保证一行只有一个平台包认领。 @@ -585,7 +586,8 @@ openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscri 15. 内置平台为确切版本:不写平台的工程构建该版本并锁定;锁中记录的其他版本不被采用并写出;工程写出的平台照常采用 (e2e 1025;提示的条件与内容见单元测试 PlatformDefault)。 16. 工程的索引在 `platform_rows` 中列出的行不需要工程的声明即选择该平台,报告来源 `index`;`planned` 行由认领的平台服务; - 工程的声明优先;两个认领被写出并取第一个;没有这一列时行未知、`planned` 行被拒绝(e2e 1106)。 + 内置表持有而本机没有提供者的行同样由索引回答;宿主的行不经索引;工程的声明优先;两个认领被写出并取第一个; + 没有这一列时行未知、`planned` 行被拒绝(e2e 1106)。 ## 变更记录 diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index 70b4d65be..586c604da 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -179,7 +179,8 @@ platform = "openkal-platform@0.28.0" # 一个版本,或一个系列:"openkal ## 索引列出的行(2026.11.17.1+) -内置表中没有平台的行,即引擎不认识的行(如 `x86_64-uefi`)与 `planned` 行(如 `i686-linux-musl`),采用索引条目中 +内置表中没有平台的行,即引擎不认识的行(如 `x86_64-uefi`)、`planned` 行(如 `i686-linux-musl`),以及内置表持有、 +平台列为空而本机没有内置提供者为之服务的行(如 `x86_64-freebsd`,其系统由平台的 sysroot 提供),采用索引条目中 列出它的平台: ```lua diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index 375638a7e..c7c6e22a6 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -444,6 +444,27 @@ std::string platform_short_name(std::string_view ref) { return r ? known_short_name(*r) : std::string{}; } +// WHETHER THE BUILT-IN TABLE ANSWERS THE PLATFORM COLUMN FOR A ROW (design +// 2026-11-16 §6.2). It does not for a row it does not hold (a platform defines +// it), holds as `planned`, or holds with no platform in its column and no +// built-in provider that serves that row here: the native FreeBSD rows, whose +// system a platform's sysroot provides (E4 of 2026-11-10; the selection takes +// them with no provider). A request the +// selection would complete to a sibling (`x86_64-freebsd` to openkal's +// `x86_64-freebsd-musl`) is not the table answering the row requested. +static bool table_answers(const PrepareState& state, const mcpp::toolchain::triple::Triple& t) { + const auto* known = mcpp::toolchain::triple::find_known_target(t); + if (!known || known->tier == "planned" || t.defined()) return false; + if (!known->platform.empty()) return true; + auto served = mcpp::toolchain::target::select(t.str(), t.arch, t.os, t.env, true, + target_context(state.m->platformOverrides, {}, {}, {}, [&] { + return state.msvc_usable_either_origin && state.msvc_usable_either_origin(); + })); + return !served.candidates.empty() && !served.fallback && served.candidates.front().available + && served.candidates.front().provider + && served.candidates.front().row && served.candidates.front().row->triple == t.str(); +} + std::string named_provider(const PrepareState& state, const mcpp::toolchain::triple::Triple& t, std::span config, std::string* platform, std::string* chosenOut) { const auto flag = !state.overrides.platform_choice.empty() @@ -464,11 +485,12 @@ std::string named_provider(const PrepareState& state, const mcpp::toolchain::tri if (!unchosen) { chosen = known_short_name(sel->ref); } else { - // Nothing chose, and the table has no platform for a `planned` row: the - // index's column names the platform that serves it (design 2026-11-16 - // §6.2), as a choice of the project would. - const auto* known = mcpp::toolchain::triple::find_known_target(t); - if (!known || known->tier != "planned" || t.defined()) return {}; + // Nothing chose, and the table has no answer for a row it holds (a + // `planned` row, or one no built-in provider serves here): the index's + // column names the platform that serves it (design 2026-11-16 §6.2), as + // a choice of the project would. A row nothing defines is answered by + // `define_early_platform`. + if (!mcpp::toolchain::triple::find_known_target(t) || table_answers(state, t)) return {}; const auto column = index_platform_for(state, t.str()); if (column.empty()) return {}; const auto ref = mp::parse_ref_text(column); @@ -538,13 +560,12 @@ std::expected step1_select_platform(PrepareState& state) { if (state.selectedTarget) column = std::string(state.selectedTarget->platform); else if (auto* row = mcpp::toolchain::triple::find_known_target(t)) column = std::string(row->platform); // THE INDEX ANSWERS WHERE THE BUILT-IN TABLE DOES NOT (design 2026-11-16 - // §6.2): a row the table does not hold (a platform defines it), or holds - // as planned. A row the table answers -- a host row included, which a - // platform's `targets` may also list -- is never taken from the index. + // §6.2, `table_answers`). A row the table answers -- a host row included, + // which a platform's `targets` may also list -- is never taken from the + // index. bool fromIndex = false; { - const auto* known = mcpp::toolchain::triple::find_known_target(t); - const bool tableAnswers = known && known->tier != "planned" && !t.defined(); + const bool tableAnswers = table_answers(state, t); if (!tableAnswers) if (auto indexed = index_platform_for(state, t.str()); !indexed.empty()) { column = std::move(indexed); diff --git a/tests/e2e/openkal/platforms.cpp b/tests/e2e/openkal/platforms.cpp index 7ffa0b052..07c316513 100644 --- a/tests/e2e/openkal/platforms.cpp +++ b/tests/e2e/openkal/platforms.cpp @@ -483,13 +483,15 @@ flags = ["-e", "_start"] expect(fatal(h.kind == s::format::elf && !h.wide)) << "the program is not ELF32"; }; "1106 the package index names the platform of a row"_test[s::elf][s::llvm][s::fresh_sandbox] = [] { - // A row no table knows, `x86_64-toyos`, and a `planned` row, `i686-linux-musl`, are listed - // in the `platform_rows` of a descriptor in the project's index: A `--target x86_64-toyos` - // selects that platform with no statement of the project, and the report says the index - // chose; E the planned row takes the platform that serves it; B a statement of the - // project stands over the index; C two descriptors of one index listing the row are - // named, and the first is taken; D without the column the row is unknown and the planned - // row is refused. + // A row no table knows, `x86_64-toyos`, a `planned` row, `i686-linux-musl`, a row the + // table holds with no platform and no provider here, `x86_64-freebsd`, and the host's + // own row are listed in the `platform_rows` of a descriptor in the project's index: A + // `--target x86_64-toyos` selects that platform with no statement of the project, and + // the report says the index chose; E the planned row takes the platform that serves it; + // F so does the row no built-in provider serves; G the host's row, which the table + // answers, is not taken from the index; B a statement of the project stands over the + // index; C two descriptors of one index listing the row are named, and the first is + // taken; D without the column the row is unknown and the planned row is refused. const auto platform = [](std::string_view name) { return std::format(R"([package] namespace = "acme" @@ -507,7 +509,7 @@ family = "unix" execution = "linux" [[platform.provider]] -serves = ["x86_64-toyos", "i686-linux-musl"] +serves = ["x86_64-toyos", "i686-linux-musl", "x86_64-freebsd"] [platform.build] cflags = ["-ffreestanding", "-fno-stack-protector", "-fno-pic", "-DTOY_NAME=\"{0}\""] @@ -531,7 +533,7 @@ driver = "compiler" }} )", name, rows); }; - const std::string rows = " platform_rows = { \"x86_64-toyos\", \"i686-linux-musl\" },\n"; + const std::string rows = " platform_rows = { \"x86_64-toyos\", \"i686-linux-musl\", \"x86_64-freebsd\", \"x86_64-linux-gnu\", \"aarch64-linux-gnu\" },\n"; const auto t = s::case_dir(); s::project app(t / "app", {{"mcpp.toml", R"([package] name = "toyapp" @@ -561,6 +563,13 @@ main = "src/main.c" r = app.mcpp({"build", "--target", "i686-linux-musl"}, inherited); expect(fatal(s::lacks(s::text(r), "not yet supported"))) << "E: the planned row the index names was refused: " << s::text(r); expect(fatal(s::has_line(s::text(r), R"(Platform +toy 0\.1\.0 \(index)"))) << "E: the planned row did not take the index's platform: " << s::text(r); + // F + r = app.mcpp({"build", "--target", "x86_64-freebsd"}, inherited); + expect(fatal(s::lacks(s::text(r), "cannot be built on this host"))) << "F: the row no provider serves was refused: " << s::text(r); + expect(fatal(s::has_line(s::text(r), R"(Platform +toy 0\.1\.0 \(index)"))) << "F: the row no provider serves did not take the index's platform: " << s::text(r); + // G + r = app.mcpp({"build"}, inherited); + expect(fatal(s::lacks(s::text(r), "(index"))) << "G: the host's row was taken from the index: " << s::text(r); // B const auto plain = app.read("mcpp.toml"); app.write("mcpp.toml", plain + "\n[target.x86_64-toyos]\nplatform = \"acme.toy2@0.1.0\"\n"); From 04b1bfcbba8655548611be7d7081c4233615c06c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 05:00:29 +0900 Subject: [PATCH 22/36] Design record: the timeouts' cause is open; the fourth kind of row the index answers; illumos and OpenBSD findings --- ...6-ecosystem-gap-supplement-part2-design.md | 30 ++++++++++++++----- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index 6e732b6f8..b7002e0ba 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -763,8 +763,11 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 - 条目的键为 `platform_rows`,值为行的列表;平台包以系列 `<命名空间>.<名字>@*` 选择,版本按 SPEC-013 §14 的 系列规则取得并锁定。 -- 内置表没有答案的三种行查索引:引擎不认识的行、平台定义的行、`planned` 行。`planned` 行与工程选择平台时相同, - 由认领它的平台服务后可以构建。 +- 内置表没有答案的四种行查索引:引擎不认识的行、平台定义的行、`planned` 行,以及内置表持有、平台列为空而本机没有 + 内置提供者服务该行本身的行(原生的 FreeBSD 行,8.7)。`planned` 行与工程选择平台时相同,由认领它的平台服务后可以 + 构建。第四种是实施中补上的:原先的条件只看行是否在内置表中,`--target x86_64-freebsd` 因此被补全为 openkal 的 + `x86_64-freebsd-musl`,8.7 所说的不写平台即用原生行并不成立。判断改为一个函数 `table_answers`,选择的补全 + (把请求换成同一系统的另一行)不算内置表回答了请求的行;e2e 1106 的 F、G 两步覆盖它与宿主行。 - 工程自己的索引(`[indices]`)先于共享的索引;同一层两个认领时取名字最小者并给出警告。 - 规范:SPEC-013 v1.11(§3.1 第 5 步、§14、判据 16),e2e 1106。 @@ -776,12 +779,23 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 - **dirun 的测试在 macOS 上失败**:FAT 的测试在 macOS 上读到一个文件为 0 字节。原因在测试:查找函数返回指向临时 列表元素的指针,列表在表达式结束时销毁,读到的是已释放的内存(Linux 上碰巧保留了原值)。改为先保存列表; 写入方另以 `std::filesystem::file_size(path)` 询问大小并报告错误。 -- **平台列的扫描与并行的腿**:全量 CI 中 e2e 的 images/1033 与 sets/987 超时(状态 137)。这两个用例构建 openkal 的行, - 这些行由内置表的 `[[platform.define]]` 定义,所以每次 prepare 都进入索引的扫描。扫描原先在一个进程级互斥锁内 - 调用配置与提供者,而它们可能等待 home 的锁;目标集合的各条腿在并行的线程中 prepare,持有 home 锁的一条腿再进入 - 扫描时可能形成死锁。这是两个用例共有的、本分支新增的路径;本机不能复现(两个用例本机通过),以 CI 的全量运行 - 验证。改为在不持锁的情况下取得根并扫描,锁只保护结果的缓存;缓存以根的集合为键,一个工作区中根不同的成员各自 - 扫描。 +- **全量 CI 中的两个超时(待查)**:e2e 的 images/1033 与 sets/987 在 Linux 的一个分片中超时(状态 137),本机两者都通过, + main 上的全量运行也通过。曾怀疑平台列的扫描:它在一个进程级互斥锁内调用配置与提供者,而目标集合的腿在并行的线程中 + prepare。但内置表的行(`x86_64-openkal` 等)由内置表回答,不进入扫描,所以这一怀疑不能解释这两个用例。扫描仍改为 + 不持锁地取得根并扫描,锁只保护以根的集合为键的缓存,因为在锁内调用外部是不必要的风险;超时的原因以下一次全量运行 + 为准继续查找。 +- **illumos 的链接(M2)**:OmniOS r151054 的 sysroot(经 IPS 取得)与磁盘映像都已完成,C 与 C++ 程序在映像中运行 + (以手工改写的链接命令)。但 clang 23 的 Solaris 驱动只经 Solaris ld 链接:程序固定为 `/usr/bin/ld`,参数是 + Solaris 的方言(`-C`、`-z ignore`),`--ld-path` 与 `-B` 不起作用;`-fuse-ld=gld` 固定为 `/usr/gnu/bin/ld`; + `-fuse-ld=` 仍给出 Solaris 方言,lld 不接受。现有的行数据没有办法让编译器的链接经过一个方言转换程序, + 所以按 17 节的退路,illumos 的行本轮不收录,原因写入收录规范。可行的两条路:在 `mcpp.platforms` 中提供一个转换 + 程序,并在引擎中增加“编译器调用的链接程序”这一行键(新机制,需要先评审);或者向上游提交 clang 的修改,使 + Solaris 驱动在 lld 下给出 GNU 方言。 +- **OpenBSD 的文件盘**:OpenBSD 的 msdos 文件系统不给文件执行权限(`-m` 无效,没有 `-x`)。映像包的 dirun 配置把 + 文件盘复制到 `/tmp/dirun` 并加上执行权限,程序从那里运行。 +- **OpenBSD 映像的体积**:`install.site` 在安装程序的 chroot 中运行,`mount` 报告的挂载点带 `/mnt` 前缀,原先的 + 填零只写到根,`/usr` 与 `/home` 的空闲块没有清零,映像 2.5 GiB(tar.xz 1.07 GB)。改为单一的根文件系统 + (disklabel 模板)并在 `/` 填零之后,映像 659 MiB(tar.xz 142 MB)。 - **磁盘空间**:本机 `/home` 的可用空间一度降到 1 GB 以下,e2e 因 `No space left on device` 失败;清理过期的临时 目录与构建缓存之后重跑通过。磁盘映像的制作(M3、P7b)需要先确认空间。 From e31ebaba284744204b68df57368ce1390ebb6cd3 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 05:03:03 +0900 Subject: [PATCH 23/36] =?UTF-8?q?SPEC-013=20=C2=A714:=20which=20rows=20an?= =?UTF-8?q?=20entry's=20platform=5Frows=20lists,=20and=20that=20a=20host's?= =?UTF-8?q?=20own=20row=20is=20never=20among=20them?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/specs/platforms.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index d790878d7..0e8d49966 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -463,8 +463,10 @@ package = { 3. 工程自己的索引(`[indices]`)先于共享的索引;同一层有多个平台包认领一行时取名字最小者,并给出警告列出它们。 索引的检查应保证一行只有一个平台包认领。 4. `planned` 行由认领它的平台服务时可以构建,与工程选择平台相同(§3.1 第 3 步)。 -5. 条目是平台包清单的派生数据:列表由索引的工具从平台包的 `[platform] targets`、`[[platform.define]]` 与 - `[[platform.provider]] serves` 生成,索引的 CI 检查二者一致。早于本版本的引擎不读这个字段。 +5. 条目是平台包清单的派生数据:列表由索引的工具从平台包最新版本的清单生成,索引的 CI 检查二者一致,且一行只有 + 一个平台包认领。列入的行:`[platform] targets`、`[[platform.define]] triple`、`[[platform.provider]] serves`, + 其中的模式(`*-linux-musl`)代表它在平台自己的行与引擎内置表的行中匹配到的行;`[platform.hosted] targets` + 中只列入引擎不认识或登记为 `planned` 的行,所以索引从不为宿主自己的行指定平台。早于本版本的引擎不读这个字段。 每条命令至多扫描一次索引的条目,且只在内置表没有答案时扫描(约 400 个条目时整个拒绝过程 0.05 s)。**已实现** From 24d6ae024680ecfc0c29bbec306f9488b5969b98 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 05:14:47 +0900 Subject: [PATCH 24/36] test: a package's programs are planned with its dev-dependencies, as its tests are, so a shared module is not compiled again over itself; a failure naming no step shows ninja's last lines (#945) --- CHANGELOG.md | 9 +++++++ src/build/execute.cppm | 15 ++++++++--- src/build/ninja_backend.cppm | 28 ++++++++++++++++++- tests/e2e/test/workspace.cpp | 45 +++++++++++++++++++++++++++++++ tests/unit/test_ninja_backend.cpp | 18 +++++++++++++ 5 files changed, 111 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1f343201a..319f5a33e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,15 @@ array of tables before the platform reader saw it, so a platform could define rows for a system but not state the system (`uname`, `resources`, `command-scripts`; SPEC-016 §2.3). +- `mcpp test` plans a package's programs with its dev-dependencies, as it plans + the tests (#945). A dev-dependency that asks a shared package for a feature + configured that package one way for the tests and another for the programs, + so one run compiled its modules again into the same paths: three compiles of + one unchanged module per `mcpp test --workspace` (e2e 1107), two per `mcpp + test` of a package, and on Windows a rewrite of a module another process had + open, which failed the run. +- A failed build whose ninja output names no failed step prints ninja's last + lines rather than a bare `error: build failed` (#945). ## [2026.11.16.1] - 2026-11-16 diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 5f7aa33c5..4e64b1c7d 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -2625,7 +2625,7 @@ std::optional run_ninja_fast(const std::string& ninjaProgram, // --verbose); what remains is the advice below. if (!reported) mcpp::ui::error("build failed"); if (!reported && !(reporting && verbose)) - mcpp::ui::block(verbose ? out : mcpp::build::filter_ninja_output(out, prefixes)); + mcpp::ui::block(verbose ? out : mcpp::build::failure_diagnostics(out, prefixes)); // Read from the RAW output, not from `diagnostics`: the filter drops // command lines, and a future filter change must not be able to // silently remove the advice along with them. @@ -7198,7 +7198,10 @@ export int run_tests(std::span passthrough, } } if (hasPrograms) { - auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, {}, overrides); + // With the dev-dependencies, as the tests' own plan (mcpp#945): the + // packages the programs share with the tests keep one configuration, + // so neither plan compiles them again over the other's output. + auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, {}, overrides); if (!binCtx) { mcpp::ui::error(binCtx.error()); return 2; } if (std::ranges::any_of(binCtx->plan.linkUnits, [](auto const& lu) { return lu.kind == LinkUnit::Binary; })) { @@ -7826,7 +7829,13 @@ export void run_workspace_tests(std::span passthrough, po.workspace_request.clear(); for (auto const& m : members) po.workspace_request.push_back(m.path); po.member_targets.clear(); - auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, {}, std::move(po)); + // WITH THE DEV-DEPENDENCIES, as the tests' plan is made (mcpp#945): a + // dev-dependency's features are unified into the packages it shares + // with the programs, so a plan without them configured those packages + // otherwise and compiled them again into the same paths -- three + // compiles of one unchanged module in a run (e2e 1107), and on Windows + // a rewrite of a module another process had open. + auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, {}, std::move(po)); if (!binCtx) { why = binCtx.error(); return false; } // `--no-programs`: planned for their names, and the environment's. if (!testOpts.noPrograms diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index b78d77909..26df9b8cc 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -112,6 +112,11 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, std::vector* placed = nullptr); std::string filter_ninja_output(std::string_view output, std::span commandPrefixes); +// What a failed build that named no failed step says (mcpp#945): the filtered +// output, or, when the filter leaves nothing, the last lines ninja printed as +// they are -- never a bare "build failed". +std::string failure_diagnostics(std::string_view output, + std::span commandPrefixes); // One ninja run read line by line into its build's report (build progress // design 2026-09-29, §6.1). `argv` must not carry `--quiet`: the status lines @@ -1368,6 +1373,27 @@ std::string graph_link_library_advice(std::string_view output, return out; } +std::string failure_diagnostics(std::string_view output, + std::span commandPrefixes) { + auto filtered = filter_ninja_output(output, commandPrefixes); + if (filtered.find_first_not_of(" \t\r\n") != std::string::npos) return filtered; + std::vector lines; + std::istringstream in{std::string(output)}; + for (std::string line; std::getline(in, line);) { + if (!line.empty() && line.back() == '\r') line.pop_back(); + const auto trimmed = ltrim_copy(line); + if (trimmed.empty() || trimmed.starts_with("\x1b[0m@@mcpp ") || trimmed.starts_with("@@mcpp ")) continue; + lines.push_back(std::move(line)); + } + if (lines.empty()) + return "the build stopped with no output: ninja named no failed step and printed nothing\n"; + constexpr std::size_t kTail = 40; + std::string out = "ninja named no failed step; its last lines:\n"; + for (std::size_t i = lines.size() > kTail ? lines.size() - kTail : 0; i < lines.size(); ++i) + out += " " + lines[i] + "\n"; + return out; +} + std::string filter_ninja_output(std::string_view output, std::span commandPrefixes) { std::string filtered; @@ -5289,7 +5315,7 @@ std::expected NinjaBackend::build(const BuildPlan& plan // What was reported as it happened is not repeated: a failed step's // diagnostics (default output), or every line (--verbose). auto diagnostics = reported || (report && opts.verbose) ? std::string{} - : opts.verbose ? out : filter_ninja_output(out, prefixes); + : opts.verbose ? out : failure_diagnostics(out, prefixes); // Appended here as well as on the fast path (execute.cppm): the two // paths report failure through different channels, and advice attached // to only one of them would appear or not depending on whether diff --git a/tests/e2e/test/workspace.cpp b/tests/e2e/test/workspace.cpp index c6306ec1a..05cc9e0c4 100644 --- a/tests/e2e/test/workspace.cpp +++ b/tests/e2e/test/workspace.cpp @@ -412,6 +412,51 @@ int main() { expect(eq(gen_edges(), 1)) << "an unchanged shared module was compiled again"; }; + "1107 a member planned alone compiles a shared module once"_test[s::gcc] = [] { + // mcpp#945: a member in a group of its own (its `[indices]` differ) whose test asks a + // shared package for a feature through a dev-dependency. Its programs are planned with + // the configuration its tests were built with, so the package's module -- one its build + // program writes -- is compiled once in a run, and an unchanged one not again in the next. + // A second compile rewrote a module another process had open (on Windows, "a file with a + // user-mapped section open"). + s::project ws{{"mcpp.toml", "[workspace]\nmembers = [\"core\", \"app\"]\n"}, + {"core/mcpp.toml", "[package]\nname = \"core\"\nversion = \"0.1.0\"\n\n[features]\nextra = []\n\n[targets.core]\nkind = \"lib\"\n"}, + {"core/src/core.cppm", "export module s1107_core;\nexport import s1107_core.gen;\nexport int core_v() { return gen_v(); }\n"}, + {"core/build.mcpp", R"(import std; +import mcpp; +int main() { + const std::string p = std::string(mcpp::out_dir()) + "/gen.cppm"; + std::ofstream(p) << "export module s1107_core.gen;\nexport int gen_v() { return 1; }\n"; + std::printf("mcpp:generated=%s\n", p.c_str()); + return 0; +} +)"}, + {"app/mcpp.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[indices]\nlocal = { path = \"idx\" }\n\n" + "[dependencies]\ncore = { path = \"../core\" }\n\n[dev-dependencies]\ncore = { path = \"../core\", features = [\"extra\"] }\n\n" + "[targets.app]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n"}, + {"app/idx/pkgs/.keep", ""}, + {"app/src/main.cpp", "import s1107_core;\nint main() { return core_v() == 1 ? 0 : 1; }\n"}, + {"app/tests/main.cpp", "import std;\nint main() { return std::getenv(\"MCPP_BIN_APP\") ? 0 : 1; }\n"}}; + auto gen_edges = [&] { + std::ptrdiff_t n = 0; + for (const auto& d : graph_dirs(ws)) + n += std::ranges::count_if(s::ninja_log_outputs(d), [](const std::string& o) { + return std::regex_search(o, std::regex(R"(/core/gen\.m\.o$)")); + }); + return n; + }; + auto first = ws.mcpp({"test", "--workspace"}); + expect(fatal(e2e::succeeded(first))) << s::text(first); + expect(eq(gen_edges(), 1)) << "the shared module was compiled more than once in one run: " << s::text(first); + expect(fatal(e2e::succeeded(ws.mcpp({"test", "--workspace"})))); + expect(eq(gen_edges(), 1)) << "an unchanged shared module was compiled again"; + // One member's tests (`-p`) take the same course: the second run compiles nothing. + expect(fatal(e2e::succeeded(ws.mcpp({"test", "-p", "app"})))); + const auto once = gen_edges(); + expect(fatal(e2e::succeeded(ws.mcpp({"test", "-p", "app"})))); + expect(eq(gen_edges(), once)) << "one member's unchanged tests compiled the shared module again"; + }; + "1095 the guide at a workspace root plans nothing"_test[s::gcc] = [] { // #932: the guide reads the frameworks from the resolved graph -- every member's -- and // plans, scans and builds nothing. diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index 8f55084e6..89435b99f 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -1327,6 +1327,24 @@ UTest filter_keeps_staging_diagnostics_and_failed_target{"NinjaBackend/FilterKee expect(eq(filtered.find("ninja: Entering directory"), std::string::npos)) << filtered; }}; +// ── mcpp#945: a failure that names no step is never reported bare ──────────── + +UTest failure_diagnostics_never_empty{"NinjaBackend/AFailureThatNamesNoStepShowsNinjasLastLines", [] { + std::vector prefixes = {"/usr/bin/clang++"}; + // A: a named step keeps the filter's answer. + const std::string named = "FAILED: obj/a.o \n/usr/bin/clang++ -c a.cpp -o obj/a.o\na.cpp:1:1: error: x\n"; + expect(eq(failure_diagnostics(named, prefixes), filter_ninja_output(named, prefixes))); + // B: output the filter removes entirely (command lines and ninja's own stop) is shown as it is. + const std::string unnamed = "ninja: Entering directory `target/x'\n[1/2] /usr/bin/clang++ -c b.cpp\n" + "ninja: build stopped: interrupted by user.\n"; + expect(fatal(eq(filter_ninja_output(unnamed, prefixes).find_first_not_of(" \n"), std::string::npos))); + const auto shown = failure_diagnostics(unnamed, prefixes); + expect(ne(shown.find("ninja named no failed step"), std::string::npos)) << shown; + expect(ne(shown.find("interrupted by user"), std::string::npos)) << shown; + // C: no output at all is said so. + expect(ne(failure_diagnostics("", prefixes).find("printed nothing"), std::string::npos)); +}}; + // ── Cache-served units: stage edges, not compile edges ────────────────────── // // The global dependency cache used to copy artifacts into the build dir from From 233ab6563060071ed291c7b45e44b8f6ef64b5a0 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 05:18:31 +0900 Subject: [PATCH 25/36] test: the programs a test runs request no [test] features (#941 stands); the testing chapter says the programs are planned with the dev-dependencies --- docs/08-testing.md | 5 ++++- docs/zh/08-testing.md | 4 +++- src/build/execute.cppm | 5 ++++- src/build/prepare.cppm | 6 ++++++ src/build/prepare/manifest.cpp | 6 +++--- 5 files changed, 20 insertions(+), 6 deletions(-) diff --git a/docs/08-testing.md b/docs/08-testing.md index d39f100a0..b236806f6 100644 --- a/docs/08-testing.md +++ b/docs/08-testing.md @@ -352,7 +352,10 @@ other than a letter or a digit written `_` (`MCPP_BIN_MY_TOOL` for `my-tool`). A test that runs the program it tests reads it from there. Under several members the programs of a configuration group are planned together, as `mcpp build --workspace` plans them, so a package they share is built with -one configuration (2026.11.16.1+). +one configuration (2026.11.16.1+). The programs are planned with the +package's dev-dependencies, as the tests are, so a feature a dev-dependency +asks of a shared package is the programs' too and the package is compiled once +(2026.11.17.1+); `[test] features` stay the tests' own. `--no-programs` (2026.11.13.1+) builds none of them: the programs are planned for their names, and each test takes each program from the environment's diff --git a/docs/zh/08-testing.md b/docs/zh/08-testing.md index c7ec90295..f784f4a83 100644 --- a/docs/zh/08-testing.md +++ b/docs/zh/08-testing.md @@ -262,7 +262,9 @@ test result ok. 7 passed; 0 failed; finished in 7.77s (plan 2.10s · programs 1. **测试按名字找到包的程序**(2026.10.27.1+)。`mcpp test` 先按 `mcpp build` 的方式构建包的程序,并给每个测试 `MCPP_BIN_`:程序 `` 的路径,名字转为大写,字母与数字之外的字符写作 `_`(`my-tool` 为 `MCPP_BIN_MY_TOOL`)。运行被测程序的测试从这里读取它。选中多个成员时,一个配置组的程序一起规划,与 -`mcpp build --workspace` 相同,因此它们共用的包以一种配置构建(2026.11.16.1+)。 +`mcpp build --workspace` 相同,因此它们共用的包以一种配置构建(2026.11.16.1+)。程序与测试一样带着包的 +dev-dependencies 规划,dev-dependency 向共用的包要求的 feature 也是程序的,包只编译一次(2026.11.17.1+); +`[test] features` 仍只属于测试。 `--no-programs`(2026.11.13.1+)不构建它们:程序只为得到名字而规划,每个测试从环境的 `MCPP_BIN_` 取得各程序,用于在别处构建的程序(一个 CI 作业测试另一个作业构建的二进制)。环境未指定的程序在任何测试运行前 diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 4e64b1c7d..88c11cb41 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -7201,7 +7201,9 @@ export int run_tests(std::span passthrough, // With the dev-dependencies, as the tests' own plan (mcpp#945): the // packages the programs share with the tests keep one configuration, // so neither plan compiles them again over the other's output. - auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, {}, overrides); + auto programs = overrides; + programs.programs_of_tests = true; + auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, {}, std::move(programs)); if (!binCtx) { mcpp::ui::error(binCtx.error()); return 2; } if (std::ranges::any_of(binCtx->plan.linkUnits, [](auto const& lu) { return lu.kind == LinkUnit::Binary; })) { @@ -7835,6 +7837,7 @@ export void run_workspace_tests(std::span passthrough, // otherwise and compiled them again into the same paths -- three // compiles of one unchanged module in a run (e2e 1107), and on Windows // a rewrite of a module another process had open. + po.programs_of_tests = true; auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, {}, std::move(po)); if (!binCtx) { why = binCtx.error(); return false; } // `--no-programs`: planned for their names, and the environment's. diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index a959b61e1..0545b6f7b 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -995,6 +995,12 @@ export struct BuildOverrides { // One says which PACKAGES enter the graph, the other which TOOLS are // installed, and `mcpp run` needs the second without the first. bool will_run = false; + // THE PROGRAMS A TEST RUNS (mcpp#945). Planned with the dev-dependencies, so + // the packages they share with the tests are configured as the tests' plan + // configures them and are not compiled again over it; but without the + // package's `[test] features`, which are the tests' own request (mcpp#941): + // the programs are the package's defaults. + bool programs_of_tests = false; // ── The packaging pass, when this prepare is one (mcpp 2026.9.11.1+) ──── // // `mcpp pack --format ` prepares TWICE, and these two fields are the diff --git a/src/build/prepare/manifest.cpp b/src/build/prepare/manifest.cpp index 0c75ef7b1..917620cc8 100644 --- a/src/build/prepare/manifest.cpp +++ b/src/build/prepare/manifest.cpp @@ -271,7 +271,7 @@ select_workspace_members(PrepareState& state, const std::filesystem::path& wsRoo // member that declares a feature of the same name is not asked for it, // and a plan without the tests (`mcpp build`, the programs the tests // run) requests none. - if (state.includeDevDeps) + if (state.includeDevDeps && !state.overrides.programs_of_tests) for (auto const& f : member->testFeatures) if (std::ranges::find(requested, f) == requested.end()) requested.push_back(f); mcpp::manifest::DependencySpec spec; @@ -598,8 +598,8 @@ std::expected phase0_manifest_and_workspace(PrepareState& sta // recorded graph names them. A plan without the tests requests none, so the // artifact and the programs the tests run are the package's defaults. A // host-tool sub-build is not the package under test. - if (state.includeDevDeps && !state.workspacePlan() && !state.overrides.preloaded_manifest - && !state.m->testFeatures.empty()) { + if (state.includeDevDeps && !state.overrides.programs_of_tests && !state.workspacePlan() + && !state.overrides.preloaded_manifest && !state.m->testFeatures.empty()) { auto tokens = mcpp::build::feature_request_tokens(state.overrides.features); for (auto const& f : state.m->testFeatures) { if (std::ranges::find(tokens, f) != tokens.end()) continue; From 25e63bf8af0e90888bcf244a8a867a69e586062b Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 05:20:08 +0900 Subject: [PATCH 26/36] tests: 1104 takes the host's row from its build directory (mcpp has no --print-target) --- tests/e2e/run/contract.cpp | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tests/e2e/run/contract.cpp b/tests/e2e/run/contract.cpp index 79c8f8f7c..e1c24ebe6 100644 --- a/tests/e2e/run/contract.cpp +++ b/tests/e2e/run/contract.cpp @@ -372,10 +372,12 @@ E2E contract_machine{"contract-machine", resource{"machine"}, [] { expect(fatal(r.started && r.status == 2 && e2e::contains(s::text(r), "missing.txt does not exist"))) << s::text(r); s::write_executable(work / "runner.sh", "#!/bin/sh\necho \"runner-cwd=$(basename \"$PWD\" | cut -c1-4)\"\n" "cat \"$MCPP_RUNTIME_FILES\"\ngrep -c '\"dest\": \"IN.TXT\"' \"$MCPP_RUN_CLOSURE\"\nexec \"$@\"\n"); - // The runner is declared for the host's own row, which every host builds. - auto host = app.mcpp({"--print-target"}).out; - host.erase(host.find_last_not_of("\r\n") + 1); - expect(fatal(!host.empty())) << "mcpp --print-target named no row"; + // The runner is declared for the host's own row, which every host builds: the triple of + // the build directory the runs above made. + std::string host; + for (const auto& e : fs::directory_iterator(app.at("target"))) + if (const auto n = e.path().filename().string(); !n.starts_with(".") && n.find('-') != std::string::npos) { host = n; break; } + expect(fatal(!host.empty())) << "could not determine the host's row"; app.write("mcpp.toml", std::format("[package]\nname = \"app\"\nversion = \"0.1.0\"\n[target.{}]\nrunner = [\"{}\"]\n", host, (work / "runner.sh").generic_string())); r = app.mcpp({"run", "--target", host, "--file", "data/in.txt=IN.TXT", "--", "IN.TXT"}); From 7185c33156a2de862ba3bba680c0e05d6b804df1 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 05:25:46 +0900 Subject: [PATCH 27/36] Design record: the timeouts did not recur with the scan's lock released; i686-openkal is the planned row a set's legs scan for --- ...2026-11-16-ecosystem-gap-supplement-part2-design.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index b7002e0ba..b2c296828 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -779,11 +779,11 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 - **dirun 的测试在 macOS 上失败**:FAT 的测试在 macOS 上读到一个文件为 0 字节。原因在测试:查找函数返回指向临时 列表元素的指针,列表在表达式结束时销毁,读到的是已释放的内存(Linux 上碰巧保留了原值)。改为先保存列表; 写入方另以 `std::filesystem::file_size(path)` 询问大小并报告错误。 -- **全量 CI 中的两个超时(待查)**:e2e 的 images/1033 与 sets/987 在 Linux 的一个分片中超时(状态 137),本机两者都通过, - main 上的全量运行也通过。曾怀疑平台列的扫描:它在一个进程级互斥锁内调用配置与提供者,而目标集合的腿在并行的线程中 - prepare。但内置表的行(`x86_64-openkal` 等)由内置表回答,不进入扫描,所以这一怀疑不能解释这两个用例。扫描仍改为 - 不持锁地取得根并扫描,锁只保护以根的集合为键的缓存,因为在锁内调用外部是不必要的风险;超时的原因以下一次全量运行 - 为准继续查找。 +- **全量 CI 中的两个超时**:e2e 的 images/1033 与 sets/987 在 Linux 的一个分片中超时(状态 137),本机两者都通过, + main 上的全量运行也通过。平台列的扫描原先在一个进程级互斥锁内调用配置与提供者,而目标集合的腿在并行的线程中 + prepare;集合 `openkal` 含内置表中 `planned` 的 `i686-openkal`,`planned` 行进入扫描,所以并行的腿会同时到达这里, + 持有 home 锁的一条腿再等待这把互斥锁即形成死锁。扫描改为不持锁地取得根并扫描,锁只保护以根的集合为键的缓存。 + 含这一修改的下一次全量运行中两个用例都通过。证据是一次运行,本机不能复现;若再出现,从挂起的进程取得栈继续查。 - **illumos 的链接(M2)**:OmniOS r151054 的 sysroot(经 IPS 取得)与磁盘映像都已完成,C 与 C++ 程序在映像中运行 (以手工改写的链接命令)。但 clang 23 的 Solaris 驱动只经 Solaris ld 链接:程序固定为 `/usr/bin/ld`,参数是 Solaris 的方言(`-C`、`-z ignore`),`--ld-path` 与 `-B` 不起作用;`-fuse-ld=gld` 固定为 `/usr/gnu/bin/ld`; From b6b4c5767f5b248c8039ae62d84e1310c69cf407 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 05:31:31 +0900 Subject: [PATCH 28/36] run files: the run's directory is removed by the interrupt watcher too, so a request that ends mcpp leaves no run-- behind --- src/build/execute.cppm | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 88c11cb41..46277f9f7 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -3881,11 +3881,19 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, o.input = mcpp::run::Input::inherit; o.streams = mcpp::run::Streams::inherit; o.report_spawn_failure = false; + // The directory is the run's: removed after it, and by the interrupt + // watcher when a request ends mcpp while the program runs, so no + // `run--` is left behind (mcpp waits here; it cannot exec). + const auto removeDir = [dir = runDir->dir] { + std::error_code rmEc; + std::filesystem::remove_all(dir, rmEc); + }; + const int cleanup = mcpp::run::add_cleanup(removeDir); auto r = mcpp::run::execute(argv, o); + mcpp::run::remove_cleanup(cleanup); rc = r.exit_code; spawnErr = r.spawn_error; - std::error_code rmEc; - std::filesystem::remove_all(runDir->dir, rmEc); + removeDir(); } else { rc = mcpp::run::run_foreground(argv, childEnv, &spawnErr); } From ede5eae03c0376ee1ae3c96c51b28a4604426215 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 06:02:07 +0900 Subject: [PATCH 29/36] Bases: a version directory with a manifest is the package root; a base's C++ library leaves the link when another package supplies the C++ layer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An OpenBSD sysroot holds only usr/ beside its mcpp.toml, and the payload resolution descended into usr/ as into an extracted tarball (payload_root, unit PmPayloadRoot). A NetBSD program built with libc++ from source (the provider's layers) still linked the base's libstdc++ through the base's ldflags (SPEC-002 §2.5, e2e 1108). --- CHANGELOG.md | 9 ++++ docs/22-target-side.md | 6 +++ docs/specs/target-side.md | 10 +++-- docs/zh/22-target-side.md | 3 ++ src/build/flags.cppm | 8 +++- src/pm/package_fetcher.cppm | 31 +++++++------ tests/e2e/openkal/layers.cpp | 61 ++++++++++++++++++++++++++ tests/unit/test_pm_package_fetcher.cpp | 26 +++++++++++ 8 files changed, 134 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 319f5a33e..b5c9066e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -79,6 +79,15 @@ open, which failed the run. - A failed build whose ninja output names no failed step prints ninja's last lines rather than a bare `error: build failed` (#945). +- A base package whose C++ layer another package supplies (a provider's + `layers`, a dependency's `provides`) no longer puts its `ldflags`, its C++ + library's, on the link: a NetBSD program built with libc++ from source also + named the base's libstdc++, and carried two C++ runtimes (SPEC-002 §2.5, + e2e 1108). +- A package whose version directory holds a `mcpp.toml` is rooted there, + whatever directories lie beside it: an OpenBSD sysroot, which holds only + `usr/`, was taken for an extracted tarball, its manifest was lost, and the + build judged `usr/` a library prefix ("no crt1.o"). ## [2026.11.16.1] - 2026-11-16 diff --git a/docs/22-target-side.md b/docs/22-target-side.md index 45ec4ee4e..f4d8eb8a3 100644 --- a/docs/22-target-side.md +++ b/docs/22-target-side.md @@ -1322,6 +1322,12 @@ platform's or a provider's `sysroot`), the base answers the layers it states: - the C++ headers, the `std` module and the C++ libraries of the link are the base's; the toolchain payload's C++ runtime and its archives, built for the payload's own host, take no part. A base that states no `std-module` leaves the build without `import std`; +- a package that supplies the `c++-abi` layer instead (a dependency's `provides`, a platform's + or a provider's `layers`) takes the base's place for that layer alone: the base's C++ headers + leave the compile and its `ldflags` leave the link, so an image has one C++ runtime, while + the C library and the compiler runtime stay the base's (2026.11.17.1+). A system whose own + C++ library ships no module sources (NetBSD's libstdc++) gets `import std` this way, from + libc++ built over its C library; - `compiler_runtime_dir` (a `[build]` key, 2026.11.10.1+) names the directory of the compiler runtime: clang reads it as its resource directory, gcc through `-B`. diff --git a/docs/specs/target-side.md b/docs/specs/target-side.md index 3da8d0c9d..4b887cdf4 100644 --- a/docs/specs/target-side.md +++ b/docs/specs/target-side.md @@ -116,7 +116,8 @@ mcpp:<层名>[=<实现名>] 它按普通包读取,不带角色标记: - **一种读法(mcpp >= 2026.11.12.1)。** 清单陈述包根即 sysroot,所以基座包**必须**按 sysroot 渲染,不论其目录怎样 - 命名;目录布局(`include/features.h` 与 `lib64`,即生态自身的 glibc 包布局)只对不带清单的目录探测。产物不由本机 + 命名;目录布局(`include/features.h` 与 `lib64`,即生态自身的 glibc 包布局)只对不带清单的目录探测。包的版本目录带 + `mcpp.toml` 时它即包根,不论其中只有哪些目录(OpenBSD 的基座只有 `usr/`;mcpp >= 2026.11.17.1)。产物不由本机 加载时的链接规则见 SPEC-016 §6。 - `[package] provides` 回答它供给的层(`mcpp:c-abi=…`、`mcpp:c++-abi=…`、`mcpp:compiler-runtime=…`)与 @@ -126,13 +127,14 @@ mcpp:<层名>[=<实现名>] | 键 | 含义 | |---|---| | `include_dirs` | C++ 库的头文件目录(规则二:`-nostdinc++` 之后加入) | -| `ldflags` | 链接的 C++ 库;供给 c++-abi 时引擎加 `-nostdlib++`,不再由驱动选择工具链的 C++ 库 | +| `ldflags` | 链接的 C++ 库;供给 c++-abi 时引擎加 `-nostdlib++`,不再由驱动选择工具链的 C++ 库;`c++-abi` 由其他包供给时(依赖的 `provides`、平台或提供者的 `layers`)不进入链接 | | `std-module`、`std-compat-module`、`std-module-flags` | §2.3 | | `compiler_runtime_dir` | 编译器运行时所在的目录;clang 读作资源目录(`-resource-dir`),gcc 经 `-B` 读取 | - 一个基座包是一个身份的 sysroot。驱动能在一个根内区分架构的系统(MinGW-w64),一个包可以服务该系统的全部身份; 否则每个身份一个包,平台以各提供者的 `sysroot` 为各身份指名(SPEC-016 §3)。 -- 基座声明 `c++-abi` 时,C++ 头文件、`std` 模块与链接的 C++ 库**必须**取自基座;工具链载荷的 C++ 运行时及为其宿主 +- 基座声明 `c++-abi` 且该层由基座供给时,C++ 头文件、`std` 模块与链接的 C++ 库**必须**取自基座;另一包供给该层时, + 基座的 C++ 头文件与 `ldflags` **禁止**进入该构建,C 库与编译器运行时仍取自基座(mcpp >= 2026.11.17.1);工具链载荷的 C++ 运行时及为其宿主 构建的归档**禁止**进入该构建。基座未给出 `std-module` 时,该构建没有 `std` 模块。 - `provides` 是声明(§2.4):测量回答不了 C 库时,`mcpp:c-abi` 回答它。 - driver-internal 的 MinGW-w64 行上,基座包是 clang 的 sysroot;静态链接取基座的归档而非其导入库(SPEC-016 §3)。 @@ -409,5 +411,5 @@ hosted 与否是 C 环境的一个声明:提供 `hosted-standard-library` 的 sy | v1.6 | 2026-11-10 | 随 mcpp 2026.11.10.1:新增 §2.5 基座包(按普通包读取:`provides` 回答层,`[build]` 陈述层,包根即 sysroot;新键 `compiler_runtime_dir`)与 §3.8 层供给者的需求不成环(`layer-cycle`)。 | | v1.7 | 2026-11-10 | 随 mcpp 2026.11.10.2:§3.4 写明段与 C 库名的对应,MSVC 载荷的 `c-abi` 回答 `ucrt`。 | | v1.2 | 2026-10-23 | 随 mcpp 2026.10.24.1:§1.3 `prebuilt` 放宽为被点名、按内容识别的预制目录;§2.4 改为由测量回答、`provides` 声明或核对,标为已实现;规则一增加基座是一个供给者与 c-abi 独占律;规则二增加基座上的编译器运行库与可测量的下限;规则三增加可叠加层;新增 §6 目标侧执行(组合、运行器契约、物理边界)。 | -| v1.9 | 2026-11-17 | 随 mcpp 2026.11.17.1(设计 2026-11-16 §5、§7):§6.2 闭包的 `run_files`(schema 2)与 `MCPP_RUNTIME_FILES` 为派生的并集;新增 §6.2.1 工作目录。 | +| v1.9 | 2026-11-17 | 随 mcpp 2026.11.17.1(设计 2026-11-16 §5、§7):§6.2 闭包的 `run_files`(schema 2)与 `MCPP_RUNTIME_FILES` 为派生的并集;新增 §6.2.1 工作目录;§2.5 另一包供给 `c++-abi` 时基座的 `ldflags` 不进入链接,包根由清单标明(版本目录带 `mcpp.toml` 即包根)。 | | v1.8 | 2026-11-12 | 随 mcpp 2026.11.12.1:§2.5 基座包只有一种读法(带清单即包根 sysroot,布局只对无清单的目录探测)。 | diff --git a/docs/zh/22-target-side.md b/docs/zh/22-target-side.md index 5b1425efc..84f81ffa4 100644 --- a/docs/zh/22-target-side.md +++ b/docs/zh/22-target-side.md @@ -1110,6 +1110,9 @@ std-module-flags = ["-D__fenv_static="] - 报告把这些层标为 `prebuilt`,并附基座的 `<名>@<版本>`; - C++ 头文件、`std` 模块与链接的 C++ 库取自基座;工具链载荷为其自身宿主构建的 C++ 运行时及其归档不参与。基座未给出 `std-module` 时,该构建没有 `import std`; +- 另一个包供给 `c++-abi` 层时(依赖的 `provides`、平台或提供者的 `layers`),它只在该层取代基座:基座的 C++ 头文件 + 不进入编译,其 `ldflags` 不进入链接,一个映像只有一个 C++ 运行时;C 库与编译器运行时仍取自基座(2026.11.17.1+)。 + 系统自带的 C++ 库不带模块源码时(NetBSD 的 libstdc++),以这种方式得到 `import std`:在其 C 库之上构建的 libc++; - `compiler_runtime_dir`(`[build]` 的键,2026.11.10.1+)指名编译器运行时所在的目录:clang 读作资源目录,gcc 经 `-B` 读取。 diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 328537325..ad2c42980 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -2020,7 +2020,11 @@ CompileFlags compute_flags_impl(const BuildPlan& plan, bool packageLinkFlags) { // layer, its `ldflags` in place of the driver's own C++ library, which // names the toolchain's. The run-time directory is as much a C link's as a // C++ one's (`kLinkDriverFlagsC` states the same of the driver's own - // selection); the C++ library's words belong to the C++ link alone. + // selection); the C++ library's words belong to the C++ link alone, and + // only while the base is what supplies that layer: when a package of the + // graph or the platform supplies it (`provides`, a platform's `layers`), + // the base's C++ library is withheld from the link as its headers are + // withheld from the compile, so one image has one C++ run-time. auto base_link_words = [&](bool cxx) { std::string words; auto const& L = plan.toolchain.base.layers; @@ -2028,7 +2032,7 @@ CompileFlags compute_flags_impl(const BuildPlan& plan, bool packageLinkFlags) { words += plan.toolchain.compiler == mcpp::toolchain::CompilerId::Clang ? " -resource-dir=" + ninjaEsc(L.compilerRuntimeDir) : " -B" + ninjaEsc(L.compilerRuntimeDir); - if (cxx && !L.cxx.empty() && !L.ldflags.empty()) { + if (cxx && !L.cxx.empty() && !L.ldflags.empty() && !plan.targetSide.cxx.fromGraph()) { words += " -nostdlib++"; for (auto const& w : L.ldflags) words += " " + ninja_command_word(w); } diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index a8af2ea0f..a2a8e1566 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -49,6 +49,22 @@ struct CallError { using CallResult = mcpp::provider::CallResult; +// The root of a payload as its provider placed it under the version +// directory. The version directory is the root when it holds `bin/`, +// `include/` or `lib/` (xim packages), or a `mcpp.toml` (a base package states +// its root by its manifest, SPEC-002 §2.5, whatever directories the system's +// layout puts beneath it: an OpenBSD sysroot holds only `usr/`); otherwise its +// one subdirectory is (an extracted source tarball, mcpplibs packages). +std::filesystem::path payload_root(const std::filesystem::path& verdir) { + std::error_code ec; + for (auto marker : {"bin", "include", "lib", "mcpp.toml"}) + if (std::filesystem::exists(verdir / marker, ec)) return verdir; + std::vector subs; + for (auto& e : std::filesystem::directory_iterator(verdir, ec)) + if (e.is_directory(ec)) subs.push_back(e.path()); + return subs.size() == 1 ? subs.front() : verdir; +} + // What a caller wants of one payload (design 2026-10-17 D3). // // A packaging revision the index raised is a newer packaging, not a broken @@ -1203,22 +1219,9 @@ Fetcher::resolve_xpkg_path(std::string_view target, p.sourceDir = (subs.size() == 1) ? subs.front() : p.root; }; - // The payload as the provider placed it: a version dir that holds bin/, - // include/ or lib/ is the root (xim packages); otherwise its one - // subdirectory (an extracted source tarball, mcpplibs packages). auto make_payload = [&](const std::filesystem::path& verdir) -> XpkgPayload { XpkgPayload payload; - std::error_code ec; - std::vector subs; - for (auto& e : std::filesystem::directory_iterator(verdir, ec)) { - if (e.is_directory()) subs.push_back(e.path()); - } - bool verdir_is_root = std::filesystem::exists(verdir / "bin") - || std::filesystem::exists(verdir / "include") - || std::filesystem::exists(verdir / "lib"); - payload.root = verdir_is_root ? verdir - : (subs.size() == 1) ? subs.front() - : verdir; + payload.root = payload_root(verdir); fill_payload(payload); return payload; }; diff --git a/tests/e2e/openkal/layers.cpp b/tests/e2e/openkal/layers.cpp index e05ed5f25..3a7f8cdfc 100644 --- a/tests/e2e/openkal/layers.cpp +++ b/tests/e2e/openkal/layers.cpp @@ -288,6 +288,67 @@ driver = "compiler" expect(fatal(!e2e::succeeded(r))) << "the musl row was served by a platform that does not serve it: " << out; expect(fatal(e2e::contains(out, "serves x86_64-freebsd, aarch64-freebsd and not x86_64-freebsd-musl"))) << "the refusal does not name what the platform serves: " << out; }; + + "1108 a base's c++ library leaves the link when a package supplies the c++ layer"_test[s::llvm] = [] { + // A base whose C++ library has no module sources (GCC's libstdc++ on NetBSD) is + // paired with libc++ from a package (the provider's `layers`): the C++ layer is the + // package's, and the base's `ldflags`, which are its C++ library's (SPEC-002 §2.5), + // leave the link as its headers leave the compile, so one image has one C++ run-time. + // The compiler run-time stays the base's. + const auto t = s::case_dir(); + fs::create_directories(t / "toy-sysroot/usr/lib"); + s::project(t / "toy-sysroot", { + {"usr/include/g++/toy_vector", "#pragma once\n"}, + {"mcpp.toml", "[package]\nname = \"toy-sysroot\"\nversion = \"1.0\"\nprovides = [\"mcpp:c-abi=toy-libc\", \"mcpp:c++-abi=libstdc++\", \"mcpp:compiler-runtime=libgcc\",\n \"hosted-standard-library\"]\n\n[build]\ninclude_dirs = [\"usr/include/g++\"]\nldflags = [\"-ltoystdcxx\"]\n"}}); + s::project(t / "cxx", {{"mcpp.toml", "[package]\nname = \"toycxx\"\nversion = \"0.1.0\"\nprovides = [\"mcpp:c++-abi=libc++\", \"hosted-standard-library\"]\n\n[build]\nsources = []\n"}}); + auto platform = [&](bool layer) { + s::project(t / "plat", {{"mcpp.toml", std::format(R"([package] +name = "toy-netbsd-platform" +version = "0.1.0" + +[platform] +toolchain = "llvm@{}" + +[[platform.system]] +name = "toynet" +uname = "ToyNet" + +[[platform.define]] +triple = "x86_64-toynet" +clang-triple = "x86_64-unknown-netbsd10.2" +object-format = "elf" +family = "unix" +execution = "toynet" +tier = "preview" + +[[platform.provider]] +serves = ["x86_64-toynet"] +sysroot = {{ path = "../toy-sysroot" }} +{}hosted = true + +[platform.link] +driver = "compiler" +)", s::llvm_version(), layer ? "layers = { \"c++-abi\" = { path = \"../cxx\" } }\n" : "")}}); + }; + s::project app(t / "app", {{"mcpp.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[targets.app]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n[target.x86_64-toynet]\nplatform = { path = \"../plat\" }\n"}, + {"src/main.cpp", "int main() { return 0; }\n"}}); + + platform(false); + auto r = app.mcpp({"why", "flags", "--target", "x86_64-toynet"}); + auto out = s::text(r); + expect(fatal(e2e::succeeded(r))) << "the plan over the base failed: " << out; + expect(fatal(s::has_line(out, R"(c\+\+-abi +libstdc\+\+ +\(toy-sysroot@1\.0, prebuilt)"))) << "the C++ layer is not the base's: " << out; + expect(fatal(s::has_line(out, R"((^| )-ltoystdcxx( |$))"))) << "the link does not name the base's C++ library: " << out; + + platform(true); + r = app.mcpp({"why", "flags", "--target", "x86_64-toynet"}); + out = s::text(r); + expect(fatal(e2e::succeeded(r))) << "the plan with the package's C++ layer failed: " << out; + expect(fatal(s::has_line(out, R"(c\+\+-abi +libc\+\+ +\(toycxx@0\.1\.0)"))) << "the C++ layer is not the package's: " << out; + expect(fatal(s::has_line(out, R"(compiler-runtime +libgcc +\(toy-sysroot@1\.0, prebuilt)"))) << "the compiler run-time is not the base's: " << out; + expect(fatal(s::no_line(out, R"((^| )-ltoystdcxx( |$))"))) << "the base's C++ library is still on the link: " << out; + expect(fatal(!e2e::contains(out, (t / "toy-sysroot/usr/include/g++").string()))) << "a unit carries the base's C++ headers: " << out; + }; }}; } // namespace diff --git a/tests/unit/test_pm_package_fetcher.cpp b/tests/unit/test_pm_package_fetcher.cpp index d3795c17e..22483cd14 100644 --- a/tests/unit/test_pm_package_fetcher.cpp +++ b/tests/unit/test_pm_package_fetcher.cpp @@ -372,3 +372,29 @@ UTest only_a_missing_package_refreshes_the_index{"PmInstallFailurePolicy/OnlyAMi expect(bool(mcpp::pm::install_failure_policy("").refreshIndex)); expect(bool(mcpp::pm::install_failure_policy("").retry)); }}; + +// ─── Where a payload's root is ─── +// +// A base package states its root by its manifest: an OpenBSD sysroot holds +// only `usr/` beside its `mcpp.toml`, and descending into `usr/` as into an +// extracted tarball lost the manifest, so the build then judged `usr/` a +// library prefix ("no crt1.o"). +UTest a_manifest_marks_the_version_directory_as_the_root{"PmPayloadRoot/AManifestMarksTheVersionDirectoryAsTheRoot", [] { + const auto base = make_tempdir("mcpp-payload-root"); + const auto sysroot = base / "sysroot"; + std::filesystem::create_directories(sysroot / "usr" / "lib"); + std::ofstream(sysroot / "mcpp.toml") << "[package]\nname = \"s\"\nversion = \"1\"\n"; + expect(eq(mcpp::pm::payload_root(sysroot), sysroot)); + + // An extracted source tarball: the one subdirectory is the root. + const auto tarball = base / "tarball"; + std::filesystem::create_directories(tarball / "pkg-1.0" / "src"); + expect(eq(mcpp::pm::payload_root(tarball), tarball / "pkg-1.0")); + + // An xim payload with the usual directories is its own root. + const auto xim = base / "xim"; + std::filesystem::create_directories(xim / "bin"); + expect(eq(mcpp::pm::payload_root(xim), xim)); + + std::filesystem::remove_all(base); +}}; From 6d8c031d4b0c6263a677626337acae20ad84ff6c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 06:06:06 +0900 Subject: [PATCH 30/36] Design record: the OpenBSD package root, the image profile, and NetBSD's import std through libc++ from source --- ...-11-16-ecosystem-gap-supplement-part2-design.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md index b2c296828..02a95bc13 100644 --- a/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md +++ b/.agents/docs/2026-11-16-ecosystem-gap-supplement-part2-design.md @@ -799,3 +799,17 @@ mcpp-index)因前置条件就绪的时间不同,必须分批,批次在表 - **磁盘空间**:本机 `/home` 的可用空间一度降到 1 GB 以下,e2e 因 `No space left on device` 失败;清理过期的临时 目录与构建缓存之后重跑通过。磁盘映像的制作(M3、P7b)需要先确认空间。 +- **OpenBSD 的包根**:OpenBSD 的 sysroot 在根下只有 `usr/` 与 `mcpp.toml`。取载荷时版本目录不含 `bin/`、`include/`、 + `lib/` 就进入其唯一的子目录(解压的源码包),清单因此丢失,构建把 `usr/` 判为库前缀("no crt1.o")。引擎改为 + 版本目录带 `mcpp.toml` 即包根(单元测试 PmPayloadRoot)。 +- **OpenBSD 映像的 dirun 配置**:配方写出的配置在 TOML 基本字符串中写了 `\$` 与 `\d`,dirun 拒绝解析。改为转义 + 反斜杠并以 revision 1、floor 1 替换已安装的 r0;xim-pkgindex 增加静态测试,解析每个配方带的 dirun 配置。 +- **NetBSD 的 `import std`**:NetBSD 10.2 的 C++ 库是 GCC 10 的 libstdc++,不带模块源码。按 openkal 的做法,层由包 + 供给:提供者以 `layers = { "c++-abi" = "llvm.libcxx@22.1.8.3" }` 在 NetBSD 的 C 库之上从源码构建 libc++,C 库与 + 编译器运行时仍取自基座。实测中链接同时带上了基座的 `-lstdc++`(程序 NEEDED libstdc++.so.9,两个 C++ 运行时): + 基座的 `ldflags` 按 SPEC-002 §2.5 是其 C++ 库的,引擎却只看基座是否声明 `c++-abi`。改为该层由其他包供给时不进入 + 链接(e2e 1108)。libc++abi 的展开器是基座的 libgcc_s,clang 的 NetBSD 驱动不论 `--rtlib`/`--unwindlib` 都不 + 写出它,所以由平台为 NetBSD 的行陈述(`[platform.build.'cfg(os = "netbsd")'] ldflags`,`--as-needed`)。 + 两个 NetBSD 行都能构建,x86_64 在 NetBSD 10.2 中运行并返回程序的状态。 +- **本机验证(P10)**:六个行都能构建(含 `import std` 与异常);x86_64 的 FreeBSD、NetBSD、OpenBSD 经 dirun 0.6.0 + 运行,参数 `one "two three"` 原样到达,状态 7 返回。 From ca3308575f97b6d25bf6be32502fcc96ddb7cc95 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 06:32:05 +0900 Subject: [PATCH 31/36] A cfg selector names the platform of a row only that platform defines MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The early selection read only tables keyed by the triple, so [target.'cfg(any(os = "netbsd", os = "openbsd"))'] platform was refused as an unknown target. Until a row defines the triple, a selector is answered by its words, -[-] (SPEC-013 §3.1, e2e 1040). --- CHANGELOG.md | 5 +++++ docs/35-platforms.md | 10 ++++++++++ docs/specs/platforms.md | 5 +++++ docs/zh/35-platforms.md | 8 ++++++++ src/build/prepare/platform.cpp | 32 ++++++++++++++++++++++++++++---- tests/e2e/openkal/platforms.cpp | 10 ++++++++++ 6 files changed, 66 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b5c9066e5..8eaf5f2e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -84,6 +84,11 @@ library's, on the link: a NetBSD program built with libc++ from source also named the base's libstdc++, and carried two C++ runtimes (SPEC-002 §2.5, e2e 1108). +- A `cfg(...)` table names the platform of a row only that platform defines: + `[target.'cfg(any(os = "netbsd", os = "openbsd"))'] platform = ...` was + refused as an unknown target, because the selection read only tables keyed + by the triple. A selector is answered by the triple's own words + (`-[-]`) until a row defines it (SPEC-013 §3.1, e2e 1040). - A package whose version directory holds a `mcpp.toml` is rooted there, whatever directories lie beside it: an OpenBSD sysroot, which holds only `usr/`, was taken for an extracted tarball, its manifest was lost, and the diff --git a/docs/35-platforms.md b/docs/35-platforms.md index f5c56b415..d7eb4c613 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -350,6 +350,16 @@ machine description is the platform's: a target whose machine differs from a built-in row is another row. `[platform]` and `[platform.hosted]` remain the two shorthands of a provider. +A table whose selector names the row by its words selects the platform for +every row it matches, though the row is defined only by the platform: before +the row is defined, `cfg(...)` is answered by the triple as written +(`-[-]`), so one table serves several rows (2026.11.17.1+): + +```toml +[target.'cfg(any(os = "netbsd", os = "openbsd"))'] +platform = { path = "../platforms" } +``` + **A row's companions and exception model (2026.11.16.1+).** `companions = [".wasm"]` names the files the link may write beside a program, named after it: they are removed before each link, staged when the link wrote one, and not an error when it did not. diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 0e8d49966..1be267b8b 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -140,6 +140,11 @@ plugin = "build.mcpp" 第 3、4 步在命中的表中取具体程度最高者(SPEC-004 §3.1.1)。**已实现** +所求的三元组尚无任何行定义时(由平台定义的行),第 3、4 步的选择器按三元组的字面成分求值: +`-[-]` 的 `arch`、`os`、`env`;族、目标格式等由行给出的成分在行被定义之前不匹配。 +于是 `[target.'cfg(any(os = "netbsd", os = "openbsd"))'] platform` 为它所定义的各行指名平台,与逐行写出 +三元组等价。**已实现**(mcpp >= 2026.11.17.1) + ### 3.2 值 | 值 | 含义 | diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index 586c604da..145fcdfc9 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -289,6 +289,14 @@ hosted = true # C 环境有 `main` 未运行」(docs/41)。行的机器描述属于平台:机器与内置行不同的目标是另一行。`[platform]` 与 `[platform.hosted]` 仍是 提供者的两种简写。 +按行的字面成分写出的选择器为它命中的每一行选择平台,即使该行只由平台定义:行被定义之前,`cfg(...)` 按三元组的写法 +(`-[-]`)求值,于是一张表可服务多行(2026.11.17.1 起): + +```toml +[target.'cfg(any(os = "netbsd", os = "openbsd"))'] +platform = { path = "../platforms" } +``` + **行的附带文件与异常模型(2026.11.16.1+)。** `companions = [".wasm"]` 指链接可能在程序旁写出、以程序命名的文件:每次链接前 删除,链接写出时被暂存,未写出不是错误。`exceptions` 是工程未陈述 `[target..abi] exceptions` 时该行构建的异常模型, 为 `"wasm"` 时 long-jump 模型亦为 `"wasm"`(内置的 `wasm32-emscripten` 行为 `"wasm"`)。`exe-suffix-fixed = true` diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index c7c6e22a6..34f5c87a6 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -271,12 +271,36 @@ void define_early_platform(PrepareState& state) { if (auto sel = mcpp::platform::parse_choice(chosenText, mp::Origin{mp::Tier::Flag, {}, 0, "--platform"}); sel && !sel->none) chosen = sel->ref; - } else { + } else if (!raw.empty()) { + // The tables whose selector the request answers by its words alone: the + // triple, or a `cfg` of its arch, os and env, which a triple no row + // defines yet (`x86_64-netbsd`) states as written. The project's tables + // before the machine's, the most specific of each (SPEC-004 §3.1.1), as + // the selection proper chooses once the row is defined (§3.1). std::vector config; if (auto cfg = state.get_cfg(false); cfg) config = (*cfg)->platformOverrides; - for (auto const* tables : { &state.m->platformOverrides, &config }) - for (auto const& o : *tables) - if (!chosen && o.selector == raw && o.platformSet && !o.platformNone) chosen = o.platform; + const auto project = project_tables(*state.m); + auto lexical = cfgpred::context_for(raw); + if (lexical.os.empty()) { + // No row knows the system yet: `-[-]` as written. + std::vector seg; + for (auto part : std::views::split(std::string_view(raw), '-')) seg.emplace_back(part.begin(), part.end()); + if (seg.size() >= 2 && seg.size() <= 3 && std::ranges::none_of(seg, [](auto const& x) { return x.empty(); })) { + lexical.os = seg[1]; + if (seg.size() == 3) lexical.env = seg[2]; + } + } + const auto words = matcher(std::move(lexical)); + const std::array*, 2> spans{&project, &config}; + for (auto const* tables : spans) { + const mp::Override* most = nullptr; + for (auto const* o : mcpp::platform::matching(*tables, words)) + if (o->platformSet) most = o; + if (most) { + if (!most->platformNone) chosen = most->platform; + break; + } + } } fs::path root = state.overrides.platform_root; if (root.empty() && chosen && !chosen->spec.path.empty()) root = chosen->spec.path; diff --git a/tests/e2e/openkal/platforms.cpp b/tests/e2e/openkal/platforms.cpp index 07c316513..c9c8ad556 100644 --- a/tests/e2e/openkal/platforms.cpp +++ b/tests/e2e/openkal/platforms.cpp @@ -384,6 +384,16 @@ cflags = ["-DTOY_SYSTEM=1"] const auto listing = s::tar_list(app.at("target/dist/toyapp-0.1.0-x86_64-toyos.tar.gz")); expect(fatal(std::ranges::any_of(listing, [](const std::string& e) { return e.ends_with("bin/toyapp.toy"); }))) << "the package holds no program"; + // A selector of the row's words names its platform as the triple does: the row is + // unknown until the platform defines it, and `cfg(os = ...)` is answered by the triple + // as written (`-[-]`). + const auto exact = app.read("mcpp.toml"); + app.write("mcpp.toml", replaced(exact, "[target.x86_64-toyos]\nplatform", "[target.'cfg(any(os = \"toyos\", os = \"toynet\"))']\nplatform")); + r = app.mcpp({"build", "--target", "x86_64-toyos"}); + expect(fatal(e2e::succeeded(r))) << "a cfg selector did not name the platform of the row it defines: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), "Platform toy-platform 0.1.0"))) << "the platform is not the one the selector names: " << s::text(r); + app.write("mcpp.toml", exact); + // `sed -i '/^\[target.x86_64-toyos\]/,/^platform/d'` app.write("mcpp.toml", replaced(app.read("mcpp.toml"), "[target.x86_64-toyos]\nplatform = { path = \"../toy\" }\n", "")); r = app.mcpp({"build", "--target", "x86_64-toyos"}); From 6e3e44bcad33279ee705dad3dcc07929caa2f8ee Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 06:50:41 +0900 Subject: [PATCH 32/36] runner = true | false: an override's consent alone, { enabled = ... }; the refusal names it and the platform-wide selector A runner's statement (a platform's, a provider's) states its command; an override table may enable or disable it by a boolean (e2e 1092 G, H). --- modules/manifest/src/platform_toml.cppm | 15 ++++++++++++++- modules/manifest/src/toml.cppm | 7 ++++--- src/build/execute.cppm | 12 +++++++++++- tests/e2e/run/runners.cpp | 12 ++++++++++++ 4 files changed, 41 insertions(+), 5 deletions(-) diff --git a/modules/manifest/src/platform_toml.cppm b/modules/manifest/src/platform_toml.cppm index 80d5f70b2..4d62c42ed 100644 --- a/modules/manifest/src/platform_toml.cppm +++ b/modules/manifest/src/platform_toml.cppm @@ -247,6 +247,18 @@ std::expected Reader::runner(const toml::Value& v, std::str Runner r; r.origin = at(v, std::string(where)); r.set = true; + // `runner = true | false`: an override's consent alone, `{ enabled = ... }`. + // A runner's statement (a platform's, a provider's) states its command. + if (v.is_bool()) { + if (!overriding) + return std::unexpected(std::format( + "{} states a runner by its command: an array of strings or {{ cmd = [...] }}; " + "`true` and `false` enable or disable a runner stated elsewhere", where)); + r.enabled = v.as_bool(); + r.enabledSet = true; + r.set = false; + return r; + } if (v.is_array()) { auto c = strings(v, where); if (!c) return std::unexpected(c.error()); @@ -305,7 +317,8 @@ std::expected Reader::runner(const toml::Value& v, std::str } } else { return std::unexpected(std::format( - "{} must be an array of strings or {{ cmd = [...], host = \"same-arch\" }}", where)); + "{} must be an array of strings or {{ cmd = [...], host = \"same-arch\" }}{}", where, + overriding ? ", or true / false to enable or disable the runner stated for the row" : "")); } // An override table adjusts the runner below it: without `cmd` it states // the keys it writes and replaces no command. diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 6b90d94c2..30421cf5a 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -4320,7 +4320,7 @@ std::expected parse_document(const t::Document& documen // The table form, `{ cmd = [...], host = "same-arch", exclusive = // true }`, is the platform reader's (SPEC-013 §2.3); its command // is the template here as well. - if (!it->second.is_array() && it->second.is_table()) { + if (!it->second.is_array() && (it->second.is_table() || it->second.is_bool())) { mcpp::manifest::platform::Reader rd{origin, mcpp::manifest::platform::Tier::Manifest, &m.schemaWarnings}; auto r = rd.runner(it->second, std::format("[target.{}].runner", triple), true); @@ -4329,8 +4329,9 @@ std::expected parse_document(const t::Document& documen runnerAdjusts = !r->set; } else if (!it->second.is_array()) { return std::unexpected(error(origin, std::format( - "[target.{}].runner must be an array of strings or {{ cmd = [...], " - "host = \"same-arch\" }}, e.g. runner = [\"qemu-system-riscv64\", \"-kernel\"]", + "[target.{}].runner must be an array of strings, {{ cmd = [...], " + "host = \"same-arch\" }}, or true / false (enable the runner stated for the row), " + "e.g. runner = [\"qemu-system-riscv64\", \"-kernel\"]", triple))); } if (it->second.is_array()) for (auto& el : it->second.as_array()) { diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 46277f9f7..e5538b7e9 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -1617,6 +1617,9 @@ struct RunnerChoice { // `[target.…]` lookup matches. Derived here, once, so the lookup and the // message it produces cannot disagree about which target they mean. std::string tripleKey; + // The `cfg(platform = ...)` name of the platform in use (SPEC-013 §5): + // the selector that enables a runner for every row of the platform. + std::string platform; }; // THE PLACES A RUNNER'S TEMPLATE NAMES (SPEC-016 §6). `{sysroot}` is the base @@ -1680,8 +1683,11 @@ std::string not_enabled_message(const RunnerChoice& c) { std::string out = what + " [runner-not-enabled]"; if (!needs.empty()) out += std::format("\n it needs: {} (installed on first use)", needs); if (c.name.empty()) { - out += std::format("\n enable: [target.{}] runner = {{ enabled = true }} # mcpp.toml, or config.toml " + out += std::format("\n enable: [target.{}] runner = true # mcpp.toml, or config.toml " "for this machine", c.tripleKey); + if (!c.platform.empty()) + out += std::format("\n [target.'cfg(platform = \"{}\")'] runner = true # every row of " + "the platform", c.platform); out += std::format("\n once: {}=true mcpp run --target {}", target_variable(c.tripleKey, "RUNNER_ENABLED"), c.tripleKey); } else { @@ -1733,6 +1739,10 @@ RunnerChoice choose_device_action(const BuildContext& ctx, const auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); if (ft) c.freestanding = ft->is_freestanding(); c.tripleKey = ft ? ft->str() : ctx.tc.targetTriple; + if (ctx.plan.platform.active) { + const auto& pkg = ctx.plan.platform.package; + c.platform = mcpp::manifest::platform::platform_name(pkg.substr(pkg.rfind('.') + 1)); + } // THE STATEMENT PREPARE MERGED (SPEC-013 §4.5): the platform's runner, a // build program's (`mcpp::runner`), this machine's, the project's // `[target.]` (a cfg selector as well as a triple, whichever diff --git a/tests/e2e/run/runners.cpp b/tests/e2e/run/runners.cpp index 88a87e7b1..15f782fab 100644 --- a/tests/e2e/run/runners.cpp +++ b/tests/e2e/run/runners.cpp @@ -418,6 +418,18 @@ int main() { p.write("mcpp.toml", base + std::format("\n[target.{}]\nrunner = [\"{}\"]\n", host, runner)); r = p.mcpp({"run"}); expect(fatal(e2e::succeeded(r) && e2e::contains(s::text(r), "RUNNER-RAN"))) << "F: an argv runner is enabled: " << s::text(r); + + // `runner = true | false` is `{ enabled = ... }`: it enables or disables the runner a + // wider table states, and the refusal names the spelling. + p.write("mcpp.toml", base + std::format( + "\n[target.'cfg(unix)']\nrunner = {{ cmd = [\"{}\"], enabled = false }}\n[target.{}]\nrunner = true\n", runner, host)); + r = p.mcpp({"run"}); + expect(fatal(e2e::succeeded(r) && e2e::contains(s::text(r), "RUNNER-RAN"))) << "G: runner = true did not enable it: " << s::text(r); + p.write("mcpp.toml", base + std::format( + "\n[target.'cfg(unix)']\nrunner = [\"{}\"]\n[target.{}]\nrunner = false\n", runner, host)); + r = p.mcpp({"run"}); + expect(fatal(r.started && r.status == 2 && e2e::contains(s::text(r), "[runner-not-enabled]"))) << "H: runner = false did not disable it: " << s::text(r); + expect(fatal(e2e::contains(s::text(r), std::format("[target.{}] runner = true", host)))) << "H: the refusal does not name the spelling: " << s::text(r); }; "1093 a build program states a runner as its table"_test[s::unix_shell] = [] { From 3f97db3cf1b1f39bd0af2c8c34cf58cf40bc531e Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 07:00:49 +0900 Subject: [PATCH 33/36] Revert the programs planned with the dev-dependencies (#945): a dev-dependency's objects reached the programs' link CI showed the change wrong: a dev-dependency that supplies main (gtest_main) was linked into the program (duplicate symbol main, e2e graph/78), and the programs' plan no longer matched mcpp build's, so the test fast path declined (e2e 1069). The programs are planned as mcpp build plans them, as before. The diagnostic for a failure that names no step stays. --- CHANGELOG.md | 7 ------ docs/08-testing.md | 5 +--- docs/zh/08-testing.md | 4 +-- src/build/execute.cppm | 16 ++---------- src/build/prepare.cppm | 6 ----- src/build/prepare/manifest.cpp | 6 ++--- tests/e2e/test/workspace.cpp | 45 ---------------------------------- 7 files changed, 7 insertions(+), 82 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8eaf5f2e1..d790732e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,13 +70,6 @@ array of tables before the platform reader saw it, so a platform could define rows for a system but not state the system (`uname`, `resources`, `command-scripts`; SPEC-016 §2.3). -- `mcpp test` plans a package's programs with its dev-dependencies, as it plans - the tests (#945). A dev-dependency that asks a shared package for a feature - configured that package one way for the tests and another for the programs, - so one run compiled its modules again into the same paths: three compiles of - one unchanged module per `mcpp test --workspace` (e2e 1107), two per `mcpp - test` of a package, and on Windows a rewrite of a module another process had - open, which failed the run. - A failed build whose ninja output names no failed step prints ninja's last lines rather than a bare `error: build failed` (#945). - A base package whose C++ layer another package supplies (a provider's diff --git a/docs/08-testing.md b/docs/08-testing.md index b236806f6..d39f100a0 100644 --- a/docs/08-testing.md +++ b/docs/08-testing.md @@ -352,10 +352,7 @@ other than a letter or a digit written `_` (`MCPP_BIN_MY_TOOL` for `my-tool`). A test that runs the program it tests reads it from there. Under several members the programs of a configuration group are planned together, as `mcpp build --workspace` plans them, so a package they share is built with -one configuration (2026.11.16.1+). The programs are planned with the -package's dev-dependencies, as the tests are, so a feature a dev-dependency -asks of a shared package is the programs' too and the package is compiled once -(2026.11.17.1+); `[test] features` stay the tests' own. +one configuration (2026.11.16.1+). `--no-programs` (2026.11.13.1+) builds none of them: the programs are planned for their names, and each test takes each program from the environment's diff --git a/docs/zh/08-testing.md b/docs/zh/08-testing.md index f784f4a83..c7ec90295 100644 --- a/docs/zh/08-testing.md +++ b/docs/zh/08-testing.md @@ -262,9 +262,7 @@ test result ok. 7 passed; 0 failed; finished in 7.77s (plan 2.10s · programs 1. **测试按名字找到包的程序**(2026.10.27.1+)。`mcpp test` 先按 `mcpp build` 的方式构建包的程序,并给每个测试 `MCPP_BIN_`:程序 `` 的路径,名字转为大写,字母与数字之外的字符写作 `_`(`my-tool` 为 `MCPP_BIN_MY_TOOL`)。运行被测程序的测试从这里读取它。选中多个成员时,一个配置组的程序一起规划,与 -`mcpp build --workspace` 相同,因此它们共用的包以一种配置构建(2026.11.16.1+)。程序与测试一样带着包的 -dev-dependencies 规划,dev-dependency 向共用的包要求的 feature 也是程序的,包只编译一次(2026.11.17.1+); -`[test] features` 仍只属于测试。 +`mcpp build --workspace` 相同,因此它们共用的包以一种配置构建(2026.11.16.1+)。 `--no-programs`(2026.11.13.1+)不构建它们:程序只为得到名字而规划,每个测试从环境的 `MCPP_BIN_` 取得各程序,用于在别处构建的程序(一个 CI 作业测试另一个作业构建的二进制)。环境未指定的程序在任何测试运行前 diff --git a/src/build/execute.cppm b/src/build/execute.cppm index e5538b7e9..294cbeaad 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -7216,12 +7216,7 @@ export int run_tests(std::span passthrough, } } if (hasPrograms) { - // With the dev-dependencies, as the tests' own plan (mcpp#945): the - // packages the programs share with the tests keep one configuration, - // so neither plan compiles them again over the other's output. - auto programs = overrides; - programs.programs_of_tests = true; - auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, {}, std::move(programs)); + auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, {}, overrides); if (!binCtx) { mcpp::ui::error(binCtx.error()); return 2; } if (std::ranges::any_of(binCtx->plan.linkUnits, [](auto const& lu) { return lu.kind == LinkUnit::Binary; })) { @@ -7849,14 +7844,7 @@ export void run_workspace_tests(std::span passthrough, po.workspace_request.clear(); for (auto const& m : members) po.workspace_request.push_back(m.path); po.member_targets.clear(); - // WITH THE DEV-DEPENDENCIES, as the tests' plan is made (mcpp#945): a - // dev-dependency's features are unified into the packages it shares - // with the programs, so a plan without them configured those packages - // otherwise and compiled them again into the same paths -- three - // compiles of one unchanged module in a run (e2e 1107), and on Windows - // a rewrite of a module another process had open. - po.programs_of_tests = true; - auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/true, {}, std::move(po)); + auto binCtx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, {}, std::move(po)); if (!binCtx) { why = binCtx.error(); return false; } // `--no-programs`: planned for their names, and the environment's. if (!testOpts.noPrograms diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 0545b6f7b..a959b61e1 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -995,12 +995,6 @@ export struct BuildOverrides { // One says which PACKAGES enter the graph, the other which TOOLS are // installed, and `mcpp run` needs the second without the first. bool will_run = false; - // THE PROGRAMS A TEST RUNS (mcpp#945). Planned with the dev-dependencies, so - // the packages they share with the tests are configured as the tests' plan - // configures them and are not compiled again over it; but without the - // package's `[test] features`, which are the tests' own request (mcpp#941): - // the programs are the package's defaults. - bool programs_of_tests = false; // ── The packaging pass, when this prepare is one (mcpp 2026.9.11.1+) ──── // // `mcpp pack --format ` prepares TWICE, and these two fields are the diff --git a/src/build/prepare/manifest.cpp b/src/build/prepare/manifest.cpp index 917620cc8..0c75ef7b1 100644 --- a/src/build/prepare/manifest.cpp +++ b/src/build/prepare/manifest.cpp @@ -271,7 +271,7 @@ select_workspace_members(PrepareState& state, const std::filesystem::path& wsRoo // member that declares a feature of the same name is not asked for it, // and a plan without the tests (`mcpp build`, the programs the tests // run) requests none. - if (state.includeDevDeps && !state.overrides.programs_of_tests) + if (state.includeDevDeps) for (auto const& f : member->testFeatures) if (std::ranges::find(requested, f) == requested.end()) requested.push_back(f); mcpp::manifest::DependencySpec spec; @@ -598,8 +598,8 @@ std::expected phase0_manifest_and_workspace(PrepareState& sta // recorded graph names them. A plan without the tests requests none, so the // artifact and the programs the tests run are the package's defaults. A // host-tool sub-build is not the package under test. - if (state.includeDevDeps && !state.overrides.programs_of_tests && !state.workspacePlan() - && !state.overrides.preloaded_manifest && !state.m->testFeatures.empty()) { + if (state.includeDevDeps && !state.workspacePlan() && !state.overrides.preloaded_manifest + && !state.m->testFeatures.empty()) { auto tokens = mcpp::build::feature_request_tokens(state.overrides.features); for (auto const& f : state.m->testFeatures) { if (std::ranges::find(tokens, f) != tokens.end()) continue; diff --git a/tests/e2e/test/workspace.cpp b/tests/e2e/test/workspace.cpp index 05cc9e0c4..c6306ec1a 100644 --- a/tests/e2e/test/workspace.cpp +++ b/tests/e2e/test/workspace.cpp @@ -412,51 +412,6 @@ int main() { expect(eq(gen_edges(), 1)) << "an unchanged shared module was compiled again"; }; - "1107 a member planned alone compiles a shared module once"_test[s::gcc] = [] { - // mcpp#945: a member in a group of its own (its `[indices]` differ) whose test asks a - // shared package for a feature through a dev-dependency. Its programs are planned with - // the configuration its tests were built with, so the package's module -- one its build - // program writes -- is compiled once in a run, and an unchanged one not again in the next. - // A second compile rewrote a module another process had open (on Windows, "a file with a - // user-mapped section open"). - s::project ws{{"mcpp.toml", "[workspace]\nmembers = [\"core\", \"app\"]\n"}, - {"core/mcpp.toml", "[package]\nname = \"core\"\nversion = \"0.1.0\"\n\n[features]\nextra = []\n\n[targets.core]\nkind = \"lib\"\n"}, - {"core/src/core.cppm", "export module s1107_core;\nexport import s1107_core.gen;\nexport int core_v() { return gen_v(); }\n"}, - {"core/build.mcpp", R"(import std; -import mcpp; -int main() { - const std::string p = std::string(mcpp::out_dir()) + "/gen.cppm"; - std::ofstream(p) << "export module s1107_core.gen;\nexport int gen_v() { return 1; }\n"; - std::printf("mcpp:generated=%s\n", p.c_str()); - return 0; -} -)"}, - {"app/mcpp.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[indices]\nlocal = { path = \"idx\" }\n\n" - "[dependencies]\ncore = { path = \"../core\" }\n\n[dev-dependencies]\ncore = { path = \"../core\", features = [\"extra\"] }\n\n" - "[targets.app]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n"}, - {"app/idx/pkgs/.keep", ""}, - {"app/src/main.cpp", "import s1107_core;\nint main() { return core_v() == 1 ? 0 : 1; }\n"}, - {"app/tests/main.cpp", "import std;\nint main() { return std::getenv(\"MCPP_BIN_APP\") ? 0 : 1; }\n"}}; - auto gen_edges = [&] { - std::ptrdiff_t n = 0; - for (const auto& d : graph_dirs(ws)) - n += std::ranges::count_if(s::ninja_log_outputs(d), [](const std::string& o) { - return std::regex_search(o, std::regex(R"(/core/gen\.m\.o$)")); - }); - return n; - }; - auto first = ws.mcpp({"test", "--workspace"}); - expect(fatal(e2e::succeeded(first))) << s::text(first); - expect(eq(gen_edges(), 1)) << "the shared module was compiled more than once in one run: " << s::text(first); - expect(fatal(e2e::succeeded(ws.mcpp({"test", "--workspace"})))); - expect(eq(gen_edges(), 1)) << "an unchanged shared module was compiled again"; - // One member's tests (`-p`) take the same course: the second run compiles nothing. - expect(fatal(e2e::succeeded(ws.mcpp({"test", "-p", "app"})))); - const auto once = gen_edges(); - expect(fatal(e2e::succeeded(ws.mcpp({"test", "-p", "app"})))); - expect(eq(gen_edges(), once)) << "one member's unchanged tests compiled the shared module again"; - }; - "1095 the guide at a workspace root plans nothing"_test[s::gcc] = [] { // #932: the guide reads the frameworks from the resolved graph -- every member's -- and // plans, scans and builds nothing. From 908934e2e6eecde21d6c82f76cf72f886031044c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 07:01:23 +0900 Subject: [PATCH 34/36] =?UTF-8?q?Docs:=20runner=20=3D=20true=20|=20false,?= =?UTF-8?q?=20and=20a=20runner's=20statement=20and=20its=20consent=20as=20?= =?UTF-8?q?two=20layers=20(SPEC-013=20=C2=A710,=20docs/41,=20docs/04)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CHANGELOG.md | 5 +++++ docs/04-mcpp-toml.md | 2 +- docs/41-devices.md | 14 +++++++++++++- docs/specs/platforms.md | 8 ++++++++ docs/zh/04-mcpp-toml.md | 2 +- docs/zh/41-devices.md | 10 +++++++++- 6 files changed, 37 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d790732e1..3ed94d674 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,11 @@ ### Added +- `runner = true` and `runner = false` in `[target.]` are + `{ enabled = true }` and `{ enabled = false }`: the consent to use the runner + a row's data states. The refusal of a runner that is not enabled names this + spelling and, for a platform's row, `[target.'cfg(platform = "")'] + runner = true` for every row of the platform (SPEC-013 §10, e2e 1092). - A `[runtime] deploy` entry is a path or a table, and its `from` is a glob of the engine's one dialect (`*`, `**`, `?`, `{a,b}`). `"share/data.bin"` is `{ from = "share/data.bin" }`, placed beside the program; a match is placed diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 630fac5f2..9999c7c3e 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1581,7 +1581,7 @@ runner = { cmd = ["okrun", "{}"], host = "same-arch", exclusive = false } | `tools.` | the package that builds a tool for this machine, or `{ program = "" }` | | `link` | `driver`, `flags`, `start`, `script`, `inputs`, `exe`, `shared`, `seal` | | `carrier` | `{ package, section, main }`: the program `--mode self-contained` builds | -| `runner` | an argv, or `{ cmd, host, hosts, exclusive, requires, enabled, longlived }`: `host = "same-arch"` runs an artifact only on a machine of its architecture; a table without `cmd` adjusts the runner below it, as `{ enabled = true }` enables a runner its data states `enabled = false` (2026.11.16.1) | +| `runner` | an argv, or `{ cmd, host, hosts, exclusive, requires, enabled, longlived }`: `host = "same-arch"` runs an artifact only on a machine of its architecture; a table without `cmd` adjusts the runner below it, as `{ enabled = true }` enables a runner its data states `enabled = false` (2026.11.16.1); `true` / `false` is `{ enabled = true / false }` (2026.11.17.1) | | `runners.` | the same argv or table, for a named runner (the table form 2026.11.16.1) | `cfg(platform = "")` selects by platform; `platform` itself is not diff --git a/docs/41-devices.md b/docs/41-devices.md index be63936b2..fe7299c5d 100644 --- a/docs/41-devices.md +++ b/docs/41-devices.md @@ -130,9 +130,21 @@ installed before it is enabled. ```toml [target.x86_64-windows-gnu] -runner = { enabled = true } # mcpp.toml, or config.toml for this machine +runner = true # mcpp.toml, or config.toml for this machine + +[target.'cfg(platform = "platforms")'] +runner = true # every row of a platform (2026.11.17.1+) ``` +What a runner is (`cmd`, `requires`, `host`) is stated by its data: the +provider, the platform, a build program, or a project or machine that replaces +it. Whether it is used is the consent of whoever runs it: the project, the +machine, the environment or `--runner`. The consent is therefore not written in +the platform's reference (`platform = {...}`): the platform may come from the +package index, a built-in row may have none, and a machine may consent alone. +`runner = true` and `runner = false` are `{ enabled = true }` and +`{ enabled = false }` (2026.11.17.1+). + A table without `cmd` adjusts the runner below it. `cmd`, `requires` and `enabled` are one statement: a tier that replaces `cmd` states another runner, enabled unless it says otherwise, so a project that writes `runner = ["wine", diff --git a/docs/specs/platforms.md b/docs/specs/platforms.md index 1be267b8b..9fc5cb7e3 100644 --- a/docs/specs/platforms.md +++ b/docs/specs/platforms.md @@ -342,6 +342,13 @@ plugin = "build.mcpp" 具名运行器 `runners.` 取与 `runner` 相同的数组或表形式(**mcpp >= 2026.11.16.1**;此前只有数组)。平台与提供者 陈述的运行器**必须**有 `cmd`;本机配置与工程的表可以只写 `enabled` 等键,调整其下一层的运行器。 +**陈述与许可分属两层。** 运行器是什么(`cmd`、`requires`、`host`、`hosts`)由其数据陈述:提供者、平台、构建程序, +或替换它的工程与本机;是否使用(`enabled`)是运行者的许可:工程、本机配置、环境变量或 `--runner`。平台包的引用 +(`platform = {...}`)**不**带运行器的键:平台可以来自索引的平台列,内置行可以没有平台,本机可以只给许可。覆盖层的 +`runner = true`、`runner = false` 即 `{ enabled = true }`、`{ enabled = false }`;陈述运行器的层写布尔值时被拒绝, +消息给出 `cmd` 的写法。为平台服务的每一行许可时写 `[target.'cfg(platform = "<名>")'] runner = true`(§5)。 +**mcpp >= 2026.11.17.1** + 模板中 `{}` 为产物,`{sysroot}` 为该目标的基座(SPEC-006),`{pkg:}` 为 `requires` 中该名字的包的根 (**mcpp >= 2026.11.12.1**)。运行器陈述的最低一级是所选提供者条目的 `runner`(SPEC-016 §6),其上依次为平台、构建程序、 本机配置、项目与环境变量。 @@ -612,3 +619,4 @@ openkal-platform 的 `targets` 因此不含 WebAssembly。Web 经 `wasm32-emscri | 1.9 | 2026-11-14 | mcpp 2026.11.13.1:§7.3 一行所带的工具,`base = "none"` 的提供者只带其自身约定指名的工具,其余身份不带只由这类提供者指名的工具。 | | 1.10 | 2026-11-16 | mcpp 2026.11.16.1(设计 2026-11-15 R1、R2):§10 运行器的 `enabled` 与 `longlived`;具名运行器与 `runner` 同一种表;§6 覆盖层不含 `cmd` 的表、`enabled` 随 `cmd` 的陈述、`MCPP_TARGET__RUNNER_ENABLED`。;运行器的程序:查找、Windows 的后缀、命令解释器启动的脚本的参数、占位符与状态(R2)。 | | 1.11 | 2026-11-16 | mcpp 2026.11.17.1(设计 2026-11-16 §6.2):§3.1 第 5 步与 §14 索引的平台列 `platform_rows`;判据 16(e2e 1106)。 | +| 1.12 | 2026-11-17 | mcpp 2026.11.17.1:§3.1 尚无行定义的三元组按字面成分求值选择器(e2e 1040);§10 运行器的陈述与许可分属两层,覆盖层的 `runner = true`/`false`(e2e 1092)。 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 96a6f511d..2fafe231d 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1448,7 +1448,7 @@ runner = { cmd = ["okrun", "{}"], host = "same-arch", exclusive = false } | `tools.` | 为本机构建一个工具的包,或 `{ program = "" }` | | `link` | `driver`、`flags`、`start`、`script`、`inputs`、`exe`、`shared`、`seal` | | `carrier` | `{ package, section, main }`:`--mode self-contained` 构建的程序 | -| `runner` | 一个 argv,或 `{ cmd, host, hosts, exclusive, requires, enabled, longlived }`:`host = "same-arch"` 只在产物架构的机器上运行它;不含 `cmd` 的表调整其下一层的 runner,如 `{ enabled = true }` 启用数据陈述 `enabled = false` 的 runner(2026.11.16.1) | +| `runner` | 一个 argv,或 `{ cmd, host, hosts, exclusive, requires, enabled, longlived }`:`host = "same-arch"` 只在产物架构的机器上运行它;不含 `cmd` 的表调整其下一层的 runner,如 `{ enabled = true }` 启用数据陈述 `enabled = false` 的 runner(2026.11.16.1);`true` / `false` 即 `{ enabled = true / false }`(2026.11.17.1) | | `runners.` | 具名 runner 的同一种 argv 或表(表形式 2026.11.16.1) | `cfg(platform = "")` 按平台选择;`platform` 本身不能写在这样的选择器下。同样的键也从 diff --git a/docs/zh/41-devices.md b/docs/zh/41-devices.md index 3fcca1fc7..aadda58b9 100644 --- a/docs/zh/41-devices.md +++ b/docs/zh/41-devices.md @@ -107,9 +107,17 @@ runner 能从任何机器抵达产物。给出执行宿主(`run_under`)时 ```toml [target.x86_64-windows-gnu] -runner = { enabled = true } # mcpp.toml,或本机的 config.toml +runner = true # mcpp.toml,或本机的 config.toml + +[target.'cfg(platform = "platforms")'] +runner = true # 平台的每一行(2026.11.17.1 起) ``` +runner 是什么(`cmd`、`requires`、`host`)由其数据陈述:提供者、平台、构建程序,或替换它的工程与本机;是否使用是运行者的 +许可:工程、本机、环境变量或 `--runner`。因此许可不写在平台的引用(`platform = {...}`)中:平台可以来自包索引,内置行可以 +没有平台,本机可以只给许可。`runner = true` 与 `runner = false` 即 `{ enabled = true }` 与 `{ enabled = false }` +(2026.11.17.1 起)。 + 不含 `cmd` 的表调整其下一层的 runner。`cmd`、`requires` 与 `enabled` 是一条陈述:替换 `cmd` 的层陈述另一个 runner, 除非写出否则为启用,因此已写 `runner = ["wine", "{}"]` 的工程行为不变。`MCPP_TARGET__RUNNER_ENABLED=true` 为一次调用启用缺省 runner,`mcpp run --runner ` 启用它指名的 runner。具名 runner 取同一种表 From b40a305f987dd4cb9a60e264d0c64fb0f964da73 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 07:12:06 +0900 Subject: [PATCH 35/36] Matrix: i686-linux-musl and i686-openkal are named a platform by the index's column (measured on all four hosts) --- tests/matrix/expected.tsv | 61 +++++++++++++++++++++------------------ 1 file changed, 33 insertions(+), 28 deletions(-) diff --git a/tests/matrix/expected.tsv b/tests/matrix/expected.tsv index 15295672e..be23e56c0 100644 --- a/tests/matrix/expected.tsv +++ b/tests/matrix/expected.tsv @@ -92,6 +92,11 @@ # # 2026.11.15.1 改变的格:引擎钉的平台为 openkal-platform 0.28.0,openkal 列为 openkal-llvm-runtime@0.20.3。 # +# 2026.11.17.1 改变的格(设计 2026-11-16 §6.2,平台列查索引):`planned` 行 `i686-linux-musl`、 +# `i686-openkal` 由索引的平台列(openkal-platform 的 `platform_rows`)指名平台:payload 体系以 llvm +# 建出;gcc 与 msvc 是 `capability-pin`;graph 体系里图中钉的版本在平台版本组之外, +# `platform-version-group`。 +# # 中间五列是报告器输出,不参与支持判定。现有报告器把 aarch64-ios 的 # c-abi 输出为 glibc(payload),这不代表 iOS 产物实际链接 glibc。Apple # 目标的实际依赖由目标构建与运行门检查,不能从该报告字段推导。 @@ -109,10 +114,10 @@ graph linux-aarch64 aarch64-windows-gnu gcc@16.1.0 - - - - - unsupported capabil graph linux-aarch64 aarch64-windows-gnu llvm@23.1.3 - - - - - unsupported std-module-precompile graph linux-aarch64 i686-linux-gnu gcc@16.1.0 - - - - - unsupported tier-planned graph linux-aarch64 i686-linux-gnu llvm@23.1.3 - - - - - unsupported tier-planned -graph linux-aarch64 i686-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned -graph linux-aarch64 i686-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned -graph linux-aarch64 i686-openkal gcc@16.1.0 - - - - - unsupported tier-planned -graph linux-aarch64 i686-openkal llvm@23.1.3 - - - - - unsupported tier-planned +graph linux-aarch64 i686-linux-musl gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-aarch64 i686-linux-musl llvm@23.1.3 - - - - - unsupported platform-version-group +graph linux-aarch64 i686-openkal gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-aarch64 i686-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph linux-aarch64 riscv64-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned graph linux-aarch64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned graph linux-aarch64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported capability-pin @@ -145,10 +150,10 @@ graph linux-x86_64 aarch64-windows-gnu gcc@16.1.0 - - - - - unsupported capabili graph linux-x86_64 aarch64-windows-gnu llvm@23.1.3 - - - - - unsupported std-module-precompile graph linux-x86_64 i686-linux-gnu gcc@16.1.0 - - - - - unsupported tier-planned graph linux-x86_64 i686-linux-gnu llvm@23.1.3 - - - - - unsupported tier-planned -graph linux-x86_64 i686-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned -graph linux-x86_64 i686-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned -graph linux-x86_64 i686-openkal gcc@16.1.0 - - - - - unsupported tier-planned -graph linux-x86_64 i686-openkal llvm@23.1.3 - - - - - unsupported tier-planned +graph linux-x86_64 i686-linux-musl gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-x86_64 i686-linux-musl llvm@23.1.3 - - - - - unsupported platform-version-group +graph linux-x86_64 i686-openkal gcc@16.1.0 - - - - - unsupported capability-pin +graph linux-x86_64 i686-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph linux-x86_64 riscv64-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned graph linux-x86_64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned graph linux-x86_64 wasm32-emscripten gcc@16.1.0 - - - - - unsupported capability-pin @@ -178,8 +183,8 @@ graph macos-arm64 aarch64-macos-musl llvm@23.1.3 - - - - - unsupported platform- graph macos-arm64 aarch64-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph macos-arm64 aarch64-windows-gnu llvm@23.1.3 - - - - - unsupported std-module-precompile graph macos-arm64 i686-linux-gnu llvm@23.1.3 - - - - - unsupported tier-planned -graph macos-arm64 i686-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned -graph macos-arm64 i686-openkal llvm@23.1.3 - - - - - unsupported tier-planned +graph macos-arm64 i686-linux-musl llvm@23.1.3 - - - - - unsupported platform-version-group +graph macos-arm64 i686-openkal llvm@23.1.3 - - - - - unsupported platform-version-group graph macos-arm64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned graph macos-arm64 wasm32-emscripten llvm@23.1.3 - - - - - unsupported capability-pin graph macos-arm64 x86_64-freebsd-musl llvm@23.1.3 - - - - - unsupported platform-version-group @@ -204,10 +209,10 @@ graph windows-x86_64 aarch64-windows-gnu llvm@23.1.3 - - - - - unsupported std-m graph windows-x86_64 aarch64-windows-gnu msvc@system - - - - - unsupported capability-pin graph windows-x86_64 i686-linux-gnu llvm@23.1.3 - - - - - unsupported tier-planned graph windows-x86_64 i686-linux-gnu msvc@system - - - - - unsupported tier-planned -graph windows-x86_64 i686-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned -graph windows-x86_64 i686-linux-musl msvc@system - - - - - unsupported tier-planned -graph windows-x86_64 i686-openkal llvm@23.1.3 - - - - - unsupported tier-planned -graph windows-x86_64 i686-openkal msvc@system - - - - - unsupported tier-planned +graph windows-x86_64 i686-linux-musl llvm@23.1.3 - - - - - unsupported platform-version-group +graph windows-x86_64 i686-linux-musl msvc@system - - - - - unsupported capability-pin +graph windows-x86_64 i686-openkal llvm@23.1.3 - - - - - unsupported platform-version-group +graph windows-x86_64 i686-openkal msvc@system - - - - - unsupported capability-pin graph windows-x86_64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned graph windows-x86_64 riscv64-linux-musl msvc@system - - - - - unsupported tier-planned graph windows-x86_64 wasm32-emscripten llvm@23.1.3 - - - - - unsupported capability-pin @@ -248,12 +253,12 @@ payload linux-aarch64 armv7a-none-eabihf gcc@16.1.0 - - - - - unsupported capabi payload linux-aarch64 armv7a-none-eabihf llvm@23.1.3 armv7a-none-eabihf payload - - - ok none payload linux-aarch64 i686-linux-gnu gcc@16.1.0 - - - - - unsupported tier-planned payload linux-aarch64 i686-linux-gnu llvm@23.1.3 - - - - - unsupported tier-planned -payload linux-aarch64 i686-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned -payload linux-aarch64 i686-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned +payload linux-aarch64 i686-linux-musl gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-aarch64 i686-linux-musl llvm@23.1.3 i686-unknown-linux-musl payload musl(platform) libc++(platform) openkal-llvm-runtime@0.20.3 ok none payload linux-aarch64 i686-none-elf gcc@16.1.0 - - - - - unsupported capability-pin payload linux-aarch64 i686-none-elf llvm@23.1.3 i686-none-elf payload - - - ok none -payload linux-aarch64 i686-openkal gcc@16.1.0 - - - - - unsupported tier-planned -payload linux-aarch64 i686-openkal llvm@23.1.3 - - - - - unsupported tier-planned +payload linux-aarch64 i686-openkal gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-aarch64 i686-openkal llvm@23.1.3 i686-unknown-none-elf payload musl(platform) libc++(platform) openkal-llvm-runtime@0.20.3 ok none payload linux-aarch64 riscv32-none-elf gcc@16.1.0 - - - - - unsupported capability-pin payload linux-aarch64 riscv32-none-elf llvm@23.1.3 riscv32-none-elf payload - - - ok none payload linux-aarch64 riscv64-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned @@ -312,12 +317,12 @@ payload linux-x86_64 armv7a-none-eabihf gcc@16.1.0 - - - - - unsupported capabil payload linux-x86_64 armv7a-none-eabihf llvm@23.1.3 armv7a-none-eabihf payload - - - ok none payload linux-x86_64 i686-linux-gnu gcc@16.1.0 - - - - - unsupported tier-planned payload linux-x86_64 i686-linux-gnu llvm@23.1.3 - - - - - unsupported tier-planned -payload linux-x86_64 i686-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned -payload linux-x86_64 i686-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned +payload linux-x86_64 i686-linux-musl gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-x86_64 i686-linux-musl llvm@23.1.3 i686-unknown-linux-musl payload musl(platform) libc++(platform) openkal-llvm-runtime@0.20.3 ok none payload linux-x86_64 i686-none-elf gcc@16.1.0 - - - - - unsupported capability-pin payload linux-x86_64 i686-none-elf llvm@23.1.3 i686-none-elf payload - - - ok none -payload linux-x86_64 i686-openkal gcc@16.1.0 - - - - - unsupported tier-planned -payload linux-x86_64 i686-openkal llvm@23.1.3 - - - - - unsupported tier-planned +payload linux-x86_64 i686-openkal gcc@16.1.0 - - - - - unsupported capability-pin +payload linux-x86_64 i686-openkal llvm@23.1.3 i686-unknown-none-elf payload musl(platform) libc++(platform) openkal-llvm-runtime@0.20.3 ok none payload linux-x86_64 riscv32-none-elf gcc@16.1.0 - - - - - unsupported capability-pin payload linux-x86_64 riscv32-none-elf llvm@23.1.3 riscv32-none-elf payload - - - ok none payload linux-x86_64 riscv64-linux-musl gcc@16.1.0 - - - - - unsupported tier-planned @@ -372,9 +377,9 @@ payload macos-arm64 aarch64-windows-gnu llvm@23.1.3 aarch64-w64-windows-gnu none payload macos-arm64 armv7a-none-eabi llvm@23.1.3 armv7a-none-eabi host - - - ok none payload macos-arm64 armv7a-none-eabihf llvm@23.1.3 armv7a-none-eabihf host - - - ok none payload macos-arm64 i686-linux-gnu llvm@23.1.3 - - - - - unsupported tier-planned -payload macos-arm64 i686-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned +payload macos-arm64 i686-linux-musl llvm@23.1.3 i686-unknown-linux-musl host musl(platform) libc++(platform) openkal-llvm-runtime@0.20.3 ok none payload macos-arm64 i686-none-elf llvm@23.1.3 i686-none-elf host - - - ok none -payload macos-arm64 i686-openkal llvm@23.1.3 - - - - - unsupported tier-planned +payload macos-arm64 i686-openkal llvm@23.1.3 i686-unknown-none-elf host musl(platform) libc++(platform) openkal-llvm-runtime@0.20.3 ok none payload macos-arm64 riscv32-none-elf llvm@23.1.3 riscv32-none-elf host - - - ok none payload macos-arm64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned payload macos-arm64 riscv64-none-elf llvm@23.1.3 riscv64-none-elf host - - - ok none @@ -416,12 +421,12 @@ payload windows-x86_64 armv7a-none-eabihf llvm@23.1.3 armv7a-none-eabihf none - payload windows-x86_64 armv7a-none-eabihf msvc@system - - - - - unsupported capability-pin payload windows-x86_64 i686-linux-gnu llvm@23.1.3 - - - - - unsupported tier-planned payload windows-x86_64 i686-linux-gnu msvc@system - - - - - unsupported tier-planned -payload windows-x86_64 i686-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned -payload windows-x86_64 i686-linux-musl msvc@system - - - - - unsupported tier-planned +payload windows-x86_64 i686-linux-musl llvm@23.1.3 i686-unknown-linux-musl none musl(platform) libc++(platform) openkal-llvm-runtime@0.20.3 ok none +payload windows-x86_64 i686-linux-musl msvc@system - - - - - unsupported capability-pin payload windows-x86_64 i686-none-elf llvm@23.1.3 - - - - - unsupported lld-required-absent payload windows-x86_64 i686-none-elf msvc@system - - - - - unsupported capability-pin -payload windows-x86_64 i686-openkal llvm@23.1.3 - - - - - unsupported tier-planned -payload windows-x86_64 i686-openkal msvc@system - - - - - unsupported tier-planned +payload windows-x86_64 i686-openkal llvm@23.1.3 i686-unknown-none-elf none musl(platform) libc++(platform) openkal-llvm-runtime@0.20.3 ok none +payload windows-x86_64 i686-openkal msvc@system - - - - - unsupported capability-pin payload windows-x86_64 riscv32-none-elf llvm@23.1.3 riscv32-none-elf none - - - ok none payload windows-x86_64 riscv32-none-elf msvc@system - - - - - unsupported capability-pin payload windows-x86_64 riscv64-linux-musl llvm@23.1.3 - - - - - unsupported tier-planned From e6fdd89ff1de64fdcc707cf7a85723fe58eabe4c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 17 Nov 2026 07:17:14 +0900 Subject: [PATCH 36/36] A test plan that configures a package otherwise than mcpp build builds in a directory of its own; a build program's unchanged output keeps its time (#945) The tests' configuration axis (#test=) enters the fingerprint when the tests ask a package for features ([test] features, or a dev-dependency entry that asks a package the manifest also depends on). One directory held both configurations, whose interfaces and objects are named by module and package, and each plan compiled over the other. A build program rerun when its cached answer was another configuration's rewrote its generated source with a new time. e2e 1107: one compile per configuration, none again. --- CHANGELOG.md | 11 +++++++++ docs/08-testing.md | 11 ++++++--- docs/zh/08-testing.md | 8 +++--- src/build/build_program.cppm | 44 +++++++++++++++++++++++++++++++++ src/build/prepare/scan.cpp | 43 ++++++++++++++++++++++++++++++++ tests/e2e/test/workspace.cpp | 48 ++++++++++++++++++++++++++++++++++++ 6 files changed, 159 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3ed94d674..7f7caf9ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -75,6 +75,17 @@ array of tables before the platform reader saw it, so a platform could define rows for a system but not state the system (`uname`, `resources`, `command-scripts`; SPEC-016 §2.3). +- A test plan that configures a package otherwise than `mcpp build` (its + `[test] features`, or a dev-dependency entry that asks a package the + manifest also depends on for features) builds in a directory of its own + (#945). The two configurations shared one directory, whose module + interfaces and objects are named by module and package, and compiled each + other's files again: three compiles of one module per `mcpp test + --workspace`, and on Windows a rewrite of an interface another compile had + open. Each configuration now compiles once (e2e 1107). +- A file a build program writes again with the same content keeps its time, + so a program that writes its outputs on every run does not recompile what + imports them (#945). - A failed build whose ninja output names no failed step prints ninja's last lines rather than a bare `error: build failed` (#945). - A base package whose C++ layer another package supplies (a provider's diff --git a/docs/08-testing.md b/docs/08-testing.md index d39f100a0..805971bb5 100644 --- a/docs/08-testing.md +++ b/docs/08-testing.md @@ -227,8 +227,13 @@ A test plan turns them on beside the defaults and `--features`, in `mcpp test`, requested on the member's own edge, so another member that declares a feature of the same name is not asked for it. `mcpp build`, the programs the tests run (`MCPP_BIN_`) and every other consumer of the package keep its defaults. -The tests and `mcpp build` share one build directory, so building one after the -other compiles again what the features change, as alternating `--features` does. +A test plan that asks a package for features this way, or through a +dev-dependency entry that asks a package the manifest also depends on for +features (or for none of its defaults), configures that package otherwise than +`mcpp build`. It builds in a build directory of its own, so each configuration +is compiled once and neither compiles over the other's module interfaces; a test +plan that asks for nothing shares the build's directory (2026.11.17.1+; before, +the two shared one directory and compiled each other's files again, mcpp#945). ### Where a test program begins @@ -445,7 +450,7 @@ A dev-dependency reaches the test units and nothing else of the package the files under `tests/`; the package's own sources are compiled under `mcpp test` with the commands `mcpp build` gives them, so the code under test is the code that ships, and `mcpp build` and `mcpp test` share one build directory -and one compile database. A kernel interface that only a dev-dependency reaches +and one compile database while the tests ask the package for no features. A kernel interface that only a dev-dependency reaches (`mcpplibs.mtest` over `mcpplibs.os` reaches openkal's) is given to the dev-dependencies and the test units alone: they see `__OPENKAL__` and openkal's headers, and the package's own sources do not. A C library or a C++ runtime that diff --git a/docs/zh/08-testing.md b/docs/zh/08-testing.md index c7ec90295..33f8b356d 100644 --- a/docs/zh/08-testing.md +++ b/docs/zh/08-testing.md @@ -166,8 +166,10 @@ features = ["ktx2"] 测试规划在默认 feature 与 `--features` 之外打开它们,见于 `mcpp test`、`mcpp test -p ` 与 `mcpp test --workspace`。在工作空间中,它们在该成员自己的边上请求,声明了同名 feature 的另一个成员不被请求。 -`mcpp build`、测试运行的程序(`MCPP_BIN_`)以及包的其它消费者保持默认 feature。测试与 `mcpp build` -共用一个构建目录,所以先后构建两者会重新编译 feature 改变的部分,与交替使用 `--features` 相同。 +`mcpp build`、测试运行的程序(`MCPP_BIN_`)以及包的其它消费者保持默认 feature。以这种方式,或经 dev 依赖条目 +(向本清单同时依赖的包要求 feature,或不要其默认 feature)向包要求 feature 的测试规划,以不同于 `mcpp build` 的配置构建该包。 +它在自己的构建目录中构建,因此每种配置只编译一次,互不覆盖对方的模块接口;不要求任何 feature 的测试规划与构建共用目录 +(2026.11.17.1 起;此前二者共用一个目录、互相重新编译,mcpp#945)。 ### 测试程序的起点 @@ -328,7 +330,7 @@ counters = { path = "../counters" } dev 依赖只到达测试单元,不到达包的其他部分(2026.11.8.1+)。它的包含目录与接口定义只给 `tests/` 下的文件;包自己的源文件在 `mcpp test` 下以 `mcpp build` 给出的命令编译,因此被 -测试的代码就是发布的代码,`mcpp build` 与 `mcpp test` 共用一个构建目录和一份编译数据库。 +测试的代码就是发布的代码,在测试不向包要求 feature 时,`mcpp build` 与 `mcpp test` 共用一个构建目录和一份编译数据库。 只有 dev 依赖到达的内核接口(`mcpplibs.mtest` 经 `mcpplibs.os` 到达 openkal 的接口)只给 dev 依赖与测试单元:它们看到 `__OPENKAL__` 与 openkal 的头文件,包自己的源文件看不到。 dev 依赖提供的 C 库或 C++ 运行时是测试所链接的每个目标文件的属性,仍属于整个配置。 diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 200a92156..00a521dba 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -1122,6 +1122,48 @@ install_hook_env(const BuildProgramEnv& env) { }; } +// A FILE A RUN WROTE AGAIN AS IT WAS KEEPS ITS TIME (mcpp#945). A build program +// commonly writes its outputs on every run, and it runs again whenever its +// cached answer is another configuration's: the run record is one per +// package, the output directory one per configuration. A generated source +// rewritten with its old content but a new time compiled its module and every +// importer again. Each regular file of the output directory is stamped before +// the run; one the run left with the same size and content gets its time back, +// so the build sees no change. +struct OutputStamp { + std::uintmax_t size = 0; + fs::file_time_type time; + std::string hash; +}; + +std::map stamp_outputs(const fs::path& outDir) { + std::map out; + std::error_code ec; + if (!fs::is_directory(outDir, ec)) return out; + for (fs::recursive_directory_iterator it(outDir, fs::directory_options::skip_permission_denied, ec), end; + !ec && it != end; it.increment(ec)) { + if (!it->is_regular_file(ec)) continue; + const auto size = it->file_size(ec); + // A large output (an archive, an image) is not read twice per run. + if (ec || size > (std::uintmax_t{64} << 20)) continue; + const auto time = it->last_write_time(ec); + if (ec) continue; + out.emplace(it->path(), OutputStamp{size, time, mcpp::toolchain::hash_file(it->path())}); + } + return out; +} + +void keep_unchanged_times(const std::map& before) { + for (auto const& [path, was] : before) { + std::error_code ec; + const auto time = fs::last_write_time(path, ec); + if (ec || time == was.time) continue; + if (fs::file_size(path, ec) != was.size || ec) continue; + if (mcpp::toolchain::hash_file(path) != was.hash) continue; + fs::last_write_time(path, was.time, ec); + } +} + std::expected run_build_program_impl( mcpp::manifest::Manifest& m, const fs::path& root, @@ -1278,8 +1320,10 @@ std::expected run_build_program_impl( } if (!placed) argv.push_back(bin.string()); } + const auto stamps = stamp_outputs(outDir); auto rres = mcpp::run::capture_exec_deadline( argv, childEnv, deadline, &timedOut, root.string()); + if (!timedOut && rres.exit_code == 0) keep_unchanged_times(stamps); if (timedOut) { // Name the manifest to edit. Without this the user edits their OWN // mcpp.toml when a dependency's build program is what timed out, and diff --git a/src/build/prepare/scan.cpp b/src/build/prepare/scan.cpp index 866c157ee..dcd8133df 100644 --- a/src/build/prepare/scan.cpp +++ b/src/build/prepare/scan.cpp @@ -1009,6 +1009,49 @@ static void step11_compute_fingerprint(PrepareState& state) { fpi.compileFlags += " #accel="; fpi.compileFlags += accel; } + // THE TESTS' CONFIGURATION (mcpp#945). A test plan configures a package + // otherwise than `mcpp build` when the tests ask it for features: the + // package's `[test] features`, or a dev-dependency entry that asks a + // package the manifest also depends on for features (or for none of its + // defaults). The module interfaces and objects of a build directory are + // named by module and package, not by configuration, so two + // configurations in one directory compile each other's files again -- + // three compiles of one module in a workspace run, and on Windows a + // rewrite of an interface another compile had open. Such a test plan + // builds in a directory of its own; one that asks for nothing shares the + // build's, which the tests' programs and `mcpp build` reuse. + if (state.includeDevDeps) { + std::vector asks; + auto add = [&](const mcpp::manifest::Manifest& m) { + const auto who = m.package.namespace_.empty() ? m.package.name + : m.package.namespace_ + "." + m.package.name; + for (auto const& f : m.testFeatures) asks.push_back(who + "[test]" + f); + for (auto const& [key, dev] : m.devDependencies) { + auto dep = m.dependencies.find(key); + if (dep == m.dependencies.end()) continue; + std::vector extra; + for (auto const& f : dev.features) + if (std::ranges::find(dep->second.features, f) == dep->second.features.end()) + extra.push_back(f); + const bool fewerDefaults = !dev.defaultFeatures && dep->second.defaultFeatures; + if (extra.empty() && !fewerDefaults) continue; + std::ranges::sort(extra); + std::string w = who + ">" + key + ":"; + for (auto const& f : extra) w += f + ","; + if (fewerDefaults) w += "-default"; + asks.push_back(std::move(w)); + } + }; + if (state.workspacePlan()) + for (auto const& [dir, member] : state.selectedMemberManifests) add(member); + else + add(*state.m); + if (!asks.empty()) { + std::ranges::sort(asks); + fpi.compileFlags += " #test="; + for (auto const& a : asks) { fpi.compileFlags += a; fpi.compileFlags += ';'; } + } + } if (state.m->cppStandard.experimental) { // c++fly gate flags are derived (not manifest-declared): fold them in // so a cppfly table change across mcpp versions re-fingerprints. diff --git a/tests/e2e/test/workspace.cpp b/tests/e2e/test/workspace.cpp index c6306ec1a..2b83150cc 100644 --- a/tests/e2e/test/workspace.cpp +++ b/tests/e2e/test/workspace.cpp @@ -412,6 +412,54 @@ int main() { expect(eq(gen_edges(), 1)) << "an unchanged shared module was compiled again"; }; + "1107 a member planned alone compiles a shared module once"_test[s::gcc] = [] { + // mcpp#945: a member in a group of its own (its `[indices]` differ) whose test asks a + // shared package for a feature through a dev-dependency. The tests' configuration of the + // package is not the build's, so the test plan builds in a directory of its own: the + // package's module -- one its build program writes on every run -- is compiled once per + // configuration, and not again by an unchanged run, a member's tests or `mcpp build`. + // Compiled over each other in one directory, it was compiled three times a run, and on + // Windows rewritten while another compile had it open. + s::project ws{{"mcpp.toml", "[workspace]\nmembers = [\"core\", \"app\"]\n"}, + {"core/mcpp.toml", "[package]\nname = \"core\"\nversion = \"0.1.0\"\n\n[features]\nextra = []\n\n[targets.core]\nkind = \"lib\"\n"}, + {"core/src/core.cppm", "export module s1107_core;\nexport import s1107_core.gen;\nexport int core_v() { return gen_v(); }\n"}, + {"core/build.mcpp", R"(import std; +import mcpp; +int main() { + const std::string p = std::string(mcpp::out_dir()) + "/gen.cppm"; + std::ofstream(p) << "export module s1107_core.gen;\nexport int gen_v() { return 1; }\n"; + std::printf("mcpp:generated=%s\n", p.c_str()); + return 0; +} +)"}, + {"app/mcpp.toml", "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[indices]\nlocal = { path = \"idx\" }\n\n" + "[dependencies]\ncore = { path = \"../core\" }\n\n[dev-dependencies]\ncore = { path = \"../core\", features = [\"extra\"] }\n\n" + "[targets.app]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n"}, + {"app/idx/pkgs/.keep", ""}, + {"app/src/main.cpp", "import s1107_core;\nint main() { return core_v() == 1 ? 0 : 1; }\n"}, + {"app/tests/main.cpp", "import std;\nint main() { return std::getenv(\"MCPP_BIN_APP\") ? 0 : 1; }\n"}}; + auto gen_edges = [&] { + std::ptrdiff_t n = 0; + for (const auto& d : graph_dirs(ws)) + n += std::ranges::count_if(s::ninja_log_outputs(d), [](const std::string& o) { + return std::regex_search(o, std::regex(R"(/core/gen\.m\.o$)")); + }); + return n; + }; + auto first = ws.mcpp({"test", "--workspace"}); + expect(fatal(e2e::succeeded(first))) << s::text(first); + expect(eq(gen_edges(), 2)) << "the shared module was not compiled once per configuration: " << s::text(first); + expect(eq(graph_dirs(ws).size(), std::size_t{2})) << "the tests' configuration has no directory of its own"; + expect(fatal(e2e::succeeded(ws.mcpp({"test", "--workspace"})))); + expect(eq(gen_edges(), 2)) << "an unchanged shared module was compiled again"; + // One member's tests (`-p`) and the build take the same directories: nothing again. + expect(fatal(e2e::succeeded(ws.mcpp({"test", "-p", "app"})))); + expect(fatal(e2e::succeeded(ws.mcpp({"test", "-p", "app"})))); + expect(eq(gen_edges(), 2)) << "one member's unchanged tests compiled the shared module again"; + expect(fatal(e2e::succeeded(ws.mcpp({"build"})))); + expect(eq(gen_edges(), 2)) << "mcpp build after the tests compiled the shared module again"; + }; + "1095 the guide at a workspace root plans nothing"_test[s::gcc] = [] { // #932: the guide reads the frameworks from the resolved graph -- every member's -- and // plans, scans and builds nothing.