diff --git a/.gitignore b/.gitignore index 749725a..b8dca1b 100644 --- a/.gitignore +++ b/.gitignore @@ -17,4 +17,5 @@ tests/java/hs_err_pid*.log # Output of the go coverage tool, specifically when used with LiteIDE *.out .claude/ -.workflows/ +.workflow/ +.pi-subagents/ diff --git a/docs/README.md b/docs/README.md index 052f37b..82f90bf 100644 --- a/docs/README.md +++ b/docs/README.md @@ -7,7 +7,7 @@ - `architecture.md`:系统架构总览,作为入口和全局导航(模块结构、入口差异、执行主流程、安全机制概览) - `ptrace-minimal-fix.md`:记录 ptrace 状态机从最小修复走到多 tracee 跟踪的演进、边界和剩余风险 - `sandbox-refactor-analysis.md`:说明沙箱统一入口和执行顺序的设计思路 -- `seccomp-bpf-migration.md`:描述从 ptrace 过滤迁移到 seccomp-bpf 的分阶段方案 +- `seccomp-bpf-migration.md`:描述从 ptrace 过滤迁移到 seccomp-bpf 的分阶段方案,以及当前 hybrid 落点 - `runner/resource-limits.md`:`case.json` 中各资源限制字段的运行期语义和判题口径 - `runner/syscalls.md`:运行期 syscall 参考清单,偏实践侧 - `runner/cgroup-v2-memory.md`:Linux runtime 的 cgroup v2 内存/进程限制设计与落地 @@ -19,7 +19,7 @@ - 想先建立全局认识:先读 `architecture.md` - 想理解当前 ptrace 逻辑的历史问题和后续扩展:读 `ptrace-minimal-fix.md` - 想理解 namespace / chroot / 降权顺序:读 `sandbox-refactor-analysis.md` -- 想看未来安全演进方向:读 `seccomp-bpf-migration.md` +- 想看 hybrid syscall 过滤和后续纯 seccomp 方向:读 `seccomp-bpf-migration.md` - 想查资源限制字段与判题口径:读 `runner/resource-limits.md` - 想查运行时 syscall 示例:看 `runner/syscalls.md` - 想了解 cgroup v2 内存模型的设计与实现:看 `runner/cgroup-v2-memory.md` diff --git a/docs/architecture.md b/docs/architecture.md index f1cd5c3..4de35b0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -114,15 +114,16 @@ setrlimit(CPU) → setitimer(alarm, CPU+5) ptrace 状态机的演进背景、相位模型和剩余风险详见 [ptrace-minimal-fix.md](ptrace-minimal-fix.md)。 -## 6. 安全机制:三段式沙箱(非 seccomp) +## 6. 安全机制:hybrid syscall 过滤与三段式沙箱 -**仓库中没有任何 seccomp-BPF 的实际代码**,seccomp 仅作为未来迁移方向出现在 [seccomp-bpf-migration.md](seccomp-bpf-migration.md)。当前安全边界完全建立在三段之上: +当前默认仍是 ptrace-only;当 `SyscallBackend` 配置为 `hybrid` 时,runner 会在 Linux 子进程完成沙箱和 `ptrace(TRACEME)` 后、`execve` 前安装 seccomp-BPF 过滤器。seccomp 负责把普通 runtime syscall 过滤下沉到内核态,ptrace 保留 bootstrap `execve` 一次性语义、多 tracee 生命周期和按名单审计职责。Phase 2 已新增 `SyscallPolicy` 归一与编译层,把 legacy 白名单和结构化 `Allow` / `Deny` / `Trace` / `Audit` 合成同一份 effective policy,再编译成 ptrace 与 seccomp 各自消费的后端输入。迁移背景见 [seccomp-bpf-migration.md](seccomp-bpf-migration.md)。 -### 6.1 ptrace 用户态白名单(主过滤层) +### 6.1 syscall 白名单 - [`TracerDetect`](../runner/tracer.go) 为每个 tracee 维护 enter/exit 相位;只在 **syscall-enter stop** 查白名单。 -- [`CallPolicy`](../runner/sec.go) 是两层白名单:`allowedCalls`(永久)+ `oneTimeCalls`(一次性,默认只有 `execve`,启动期被预先消费)。 -- ptrace 选项(`process_linux.go`):`PTRACE_O_TRACESYSGOOD | PTRACE_O_EXITKILL | TRACE{CLONE,FORK,VFORK}`。`TRACESYSGOOD` 让 syscall-stop 用 `SIGTRAP|0x80`,与真实 `SIGTRAP` 区分。 +- [`EffectiveSyscallPolicy`](../runner/syscall_policy.go) 先把 legacy `AllowedCalls` / `AdditionCalls` / `OneTimeCalls` 和结构化 `SyscallPolicy` 归一;`compileSyscallPolicy()` 再生成 ptrace 与 seccomp 的后端输入;[`CallPolicy`](../runner/sec.go) 只消费 ptrace 需要的具名 `callPolicySpec`。 +- ptrace 选项(`process_linux.go`):默认是 `PTRACE_O_TRACESYSGOOD | PTRACE_O_EXITKILL | TRACE{CLONE,FORK,VFORK}`;hybrid 模式额外打开 `PTRACE_O_TRACESECCOMP`。ptrace-only 用 `PTRACE_SYSCALL` 获得 syscall enter/exit stop;hybrid 为性能和简单性使用 `PTRACE_CONT`,只接收 ptrace event、信号停靠和退出事件。 +- hybrid 模式下,effective `Allow` 会生成 seccomp `ALLOW` 规则;`Deny` 先从 allow 结果里做减法,因此可覆盖默认或语言 fixture 继承来的 syscall;`OneTimeCalls + Trace + Audit` 生成 `SECCOMP_RET_TRACE` 规则,由 ptrace 审批或审计。普通禁止 syscall 会在内核态执行前被杀死,不再依赖 ptrace enter/exit 相位。 ### 6.2 rlimit 内核兜底层 @@ -145,6 +146,7 @@ ptrace 状态机的演进背景、相位模型和剩余风险详见 [ptrace-mini - **CPU 是总 CPU 时间**:多线程并行时该值可能明显大于 wall-clock。 - **Memory 双来源**:[`refreshPeakMemoryFromProc`](../runner/exec.go) 遍历 active pid 读 `/proc/status` 的 `VmHWM`,按 Tgid 去重取每组峰值再求和;[`process.Memory`](../runner/process.go) 来自 `wait4` 的 `ru_maxrss`,按 thread group 聚合并扣除 bootstrap 基线偏移。两者取 `max` 作为 `PeakMemory`。 +- **hybrid event-only 采样粒度较粗**:hybrid 不再在每个普通 syscall enter/exit stop 醒来,因此 `checkLimit` 只会在 ptrace event、信号停靠、退出等路径运行;最终判定仍依赖 `RLIMIT_CPU` / `alarm` / cgroup v2 memory 状态兜底。 完整字段语义、双层关系与事件归类见 [runner/resource-limits.md](runner/resource-limits.md)。 @@ -156,12 +158,13 @@ ptrace 状态机的演进背景、相位模型和剩余风险详见 [ptrace-mini - **arm64**([syscalls_arm64.go](../sec/syscalls_arm64.go)):直接用 `unix.SYS_*` 常量 map,末尾几个 6.x 新号因 `x/sys` 尚未定义而硬编码。 - 其它平台([stub_other.go](../sec/stub_other.go)):返回 "only available on linux/amd64 or linux/arm64"。 -策略语义在 [`runner/sec.go`](../runner/sec.go) 的 `CallPolicy` 实现(**纯白名单模型,无黑名单**): +策略语义先在 [`runner/syscall_policy.go`](../runner/syscall_policy.go) 归一,再交给 [`runner/sec.go`](../runner/sec.go) 的 `CallPolicy` 或 [`runner/seccomp_linux.go`](../runner/seccomp_linux.go) 的 seccomp filter 生成器: - 默认 `AllowedCalls`([config.go:42](../runner/config.go))非常保守:`read,write,brk,fstat,uname,mmap,exit_group,exit,readlinkat,faccessat,mprotect,set_tid_address,set_robust_list,rseq,prlimit64,getrandom,rt_sigreturn`。 - 默认 `OneTimeCalls`:仅 `execve`。 - Java 等运行时通过 `AdditionCalls` 追加约 40 个,示例见 [`tests/java/case.json`](../tests/java/case.json)。 - amd64 平台默认追加 `arch_prctl/readlink/access`,arm64 不追加。 +- `SyscallPolicy.Allow` 追加 runtime allow;`SyscallPolicy.Deny` 覆盖 allow 结果;`SyscallPolicy.Trace` 在 hybrid 下转成 seccomp trace 规则;`SyscallPolicy.Audit` 同样转成 trace 规则,并额外输出 audit 日志。`Deny` 不能和 `OneTimeCalls` / `Trace` / `Audit` 重叠;hybrid 启动协议保留 `write` / `close` / `exit` / `exit_group`,这些 syscall 不能放入 `Deny` / `Trace` / `Audit` / `OneTimeCalls`。 可读的 syscall 清单与 Java 案例见 [runner/syscalls.md](runner/syscalls.md)。 @@ -174,7 +177,7 @@ ptrace 状态机的演进背景、相位模型和剩余风险详见 [ptrace-mini - [sec/syscalls_test.go](../sec/syscalls_test.go):交叉校验表与内核常量、新内核 syscall 覆盖、架构差异。 - [sandbox_behavior_linux_test.go](../runner/sandbox_behavior_linux_test.go):行为级集成测试,通过 `runProcess()` 启动真实子进程读 `/proc//status` 断言 NoNewPrivs、UID/GID 切换、mount namespace、chroot(部分用例需 root)。 -### 9.2 集成测试(`tests/`,20 个 case) +### 9.2 集成测试(`tests/`,23 个 case) 每个 case = `case.json` + 源码(`.c`/`.java`)+ `makefile`(`gcc -static → ../../bin/test → clean`)。`make testall`([Makefile](../Makefile))依次进入每个目录执行。分类: @@ -183,7 +186,7 @@ ptrace 状态机的演进背景、相位模型和剩余风险详见 [ptrace-mini | 正常 | `general` | | 资源限制 | `tle` `tle2` `mle` `mle2` `mle21` `mle3` `ole` `prlimit-ole` `stack` | | 运行时异常 | `segmentfault` `sigtrap` `zero` | -| 安全/多进程 | `clone-syscall-phase` `fork` `socket` `thread` | +| 安全/多进程 | `clone-syscall-phase` `clone-syscall-phase-hybrid` `fork` `socket` `syscall-policy-deny-hybrid` `syscall-policy-trace-audit-hybrid` `thread` | | Java | `java` `java-tle` `java-mle` | `make testall` 需要 C/C++ 工具链(静态链接)、GNU Make,可选 Java(`tests/java*`)。完整前提与平台支持矩阵见 [README.md](../README.md)。 @@ -251,7 +254,7 @@ RUNTIME_ERROR=10, COMPILE_ERROR=11, COMPILE_OK=12, TEST_RUN=13 以下均为既有文档已记录的事项,此处仅做导航: -- **seccomp 迁移**(ptrace 用户态过滤的性能与复杂度):分阶段草案见 [seccomp-bpf-migration.md](seccomp-bpf-migration.md)。主要难点是 `execve once` 语义与 seccomp allowlist 不兼容。 +- **seccomp 迁移**:当前已支持显式 `SyscallBackend: "hybrid"`,默认仍为 ptrace-only。后续纯 seccomp 方向见 [seccomp-bpf-migration.md](seccomp-bpf-migration.md)。主要难点仍是 `execve once` 语义与 seccomp allowlist 不兼容。 - **ptrace 多 tracee 剩余风险**(资源统计仍是近似模型):见 [ptrace-minimal-fix.md](ptrace-minimal-fix.md)。 - **资源限制待验证项**(`MemoryReserve` 余量是否覆盖各语言初始化、短生命周期 MLE 能否被 `/proc` 采样捕获、Java 多线程下 thread group 聚合准确性、`RLIMIT_AS` 过紧致 mmap 失败):见 [todo.md](todo.md)。 - **平台支持边界**(其它 Linux 架构可编译但启动失败,arm64 集成测试需在目标主机跑):见 [README.md](../README.md) 的 Platform Support 一节。 @@ -260,7 +263,7 @@ RUNTIME_ERROR=10, COMPILE_ERROR=11, COMPILE_OK=12, TEST_RUN=13 - 想理解 ptrace 状态机的历史问题与多 tracee 演进 → [ptrace-minimal-fix.md](ptrace-minimal-fix.md) - 想理解 namespace / chroot / 降权顺序的设计 → [sandbox-refactor-analysis.md](sandbox-refactor-analysis.md) -- 想看未来从 ptrace 迁移到 seccomp-bpf 的方向 → [seccomp-bpf-migration.md](seccomp-bpf-migration.md) +- 想看 hybrid syscall 过滤和后续纯 seccomp 方向 → [seccomp-bpf-migration.md](seccomp-bpf-migration.md) - 想查运行期资源限制字段与判题口径 → [runner/resource-limits.md](runner/resource-limits.md) - 想查运行期 syscall 白名单实践与 Java 案例 → [runner/syscalls.md](runner/syscalls.md) - 想看待验证事项清单 → [todo.md](todo.md) diff --git a/docs/runner/syscalls.md b/docs/runner/syscalls.md index cfe686b..e5f4f98 100644 --- a/docs/runner/syscalls.md +++ b/docs/runner/syscalls.md @@ -3,7 +3,14 @@ > **Source of truth**: The actual defaults are in [`runner/config.go`](../../runner/config.go) > (`AllowedCalls` / `OneTimeCalls`) and the platform-specific defaults in > `runner/defaults_linux_*.go`. Per-case overrides live in each `case.json` -> under `AdditionCalls`. This document is a human-readable summary. +> under legacy `AdditionCalls` or structured `SyscallPolicy`. This document is a human-readable summary. +> With `SyscallBackend: "hybrid"`, Linux turns the effective runtime allowlist +> into seccomp-BPF `ALLOW` rules installed before the user program starts. +> `OneTimeCalls` such as bootstrap `execve`, plus `SyscallPolicy.Trace` / +> `SyscallPolicy.Audit`, use `SECCOMP_RET_TRACE`, so ptrace remains the startup +> boundary and named audit path instead of permanently allowing one-time calls. +> Hybrid uses event-only ptrace resume for performance, so ordinary seccomp +> `ALLOW` calls do not produce ptrace syscall enter/exit stops. ## Default allowed syscalls (runtime-supported Linux platforms) @@ -25,6 +32,32 @@ These are consumed on first use and then denied: execve ``` +### Structured policy fields + +`SyscallPolicy` is the Phase 2 compatibility layer over the legacy lists: + +```json +{ + "SyscallPolicy": { + "Allow": ["read"], + "Deny": ["ptrace", "bpf"], + "Trace": ["clone3"], + "Audit": ["getpid"] + } +} +``` + +- `Allow` is merged with legacy `AllowedCalls + AdditionCalls`. +- `Deny` overrides that merged allowlist, so cases can remove syscalls inherited from defaults or language fixtures. +- `Trace` is permitted under ptrace-only, and becomes a seccomp `TRACE` rule under hybrid. +- `Audit` follows the same allow/trace backend mapping as `Trace`, and additionally emits an audit log entry when observed; it is named observation, not full syscall-stream replay. +- `Deny` may overlap allow rules because it subtracts from them, but it cannot overlap `Trace`, `Audit`, or `OneTimeCalls`. +- Under hybrid, `write`, `close`, `exit`, and `exit_group` are reserved by the runner startup protocol. They may be runtime-allowed, but cannot be placed in `Deny`, `Trace`, `Audit`, or `OneTimeCalls`. + +The runner first normalizes these lists into an effective policy, then compiles +that policy into backend inputs. ptrace receives a `callPolicySpec`; hybrid +seccomp receives final `ALLOW` and `TRACE` syscall lists. + ### Platform-specific additions **linux/amd64** adds: diff --git a/docs/seccomp-bpf-migration.md b/docs/seccomp-bpf-migration.md index 58b2abd..c8d34ae 100644 --- a/docs/seccomp-bpf-migration.md +++ b/docs/seccomp-bpf-migration.md @@ -1,18 +1,20 @@ # seccomp-bpf 迁移草案 +> 当前代码已落地 Phase 1 的 hybrid 版本,并推进了 Phase 2 的兼容式策略归一层:默认仍是 `ptrace`,显式配置 `SyscallBackend: "hybrid"` 时会在 Linux child 中安装 seccomp-BPF 过滤器;普通 runtime allowlist 走 `ALLOW`,`OneTimeCalls` 走 `SECCOMP_RET_TRACE` 交给 ptrace 做启动边界判断。`SyscallPolicy` 已提供 `Allow` / `Deny` / `Trace` / `Audit` 四个显式桶,但旧的 `AllowedCalls` / `AdditionCalls` / `OneTimeCalls` 仍保持兼容。纯 seccomp 仍是后续方向。 + ## 目标 把 syscall 过滤的安全边界从用户态 ptrace 下沉到内核态 seccomp-bpf,同时尽量复用当前 runner 的配置模型和测试资产。 -这份草案的目标不是直接替换全部运行逻辑,而是给出一条低风险迁移路径。 +这份草案的目标不是直接替换全部运行逻辑,而是给出一条低风险迁移路径;当前实现覆盖了第一阶段的工程接缝。 ## 当前现状 当前 runner 的 syscall 控制面有三个关键特点: -1. 白名单来自 [runner/config.go](../runner/config.go) 的 `AllowedCalls` 和 `AdditionCalls`。 +1. 兼容白名单来自 [runner/config.go](../runner/config.go) 的 `AllowedCalls` 和 `AdditionCalls`,结构化新增规则来自 `SyscallPolicy.Allow` / `Deny` / `Trace` / `Audit`。 2. 一次性 syscall 来自 [runner/config.go](../runner/config.go) 的 `OneTimeCalls`,默认包含 `execve`。 -3. 运行时的最终判定发生在 [runner/tracer_linux.go](../runner/tracer_linux.go) 的 ptrace loop 中。 +3. ptrace-only 模式的最终判定仍发生在 [runner/tracer_linux.go](../runner/tracer_linux.go) 的 ptrace loop 中;hybrid 模式下普通禁止 syscall 由 seccomp 在执行前拒绝,ptrace 只处理 `SECCOMP_RET_TRACE` 事件和生命周期事件。 这个模型的主要问题是: @@ -52,42 +54,56 @@ 建议做法: 1. 保留当前最小修复后的 ptrace 初始化流程。 -2. 在子进程完成 namespace、rootfs、no_new_privs、凭证切换之后安装 seccomp 过滤器。 +2. 在子进程完成 namespace、rootfs、no_new_privs、凭证切换之后安装 seccomp 过滤器。当前首版 hybrid 实现将安装点放在 `ptrace(TRACEME)` 之后、`execve` 之前,这样 runtime seccomp allowlist 不需要放行用户态 `ptrace`。 3. seccomp 规则负责稳定的 runtime allowlist,例如 `read`、`write`、`mmap`、`brk`、`exit_group` 这类通用 syscall。 4. ptrace 暂时只处理两类场景: - - bootstrap `execve` 的边界语义 - - 迁移期的审计和对照验证 + - bootstrap `execve` 的边界语义:BPF 对 `OneTimeCalls` 返回 `SECCOMP_RET_TRACE`,parent 在 `PTRACE_EVENT_SECCOMP` 上确认这次启动期 `execve` + - 迁移期按 `Trace` / `Audit` 名单做审计和抽样对照 这样做的好处是: 1. 大部分危险 syscall 在内核态直接被拒绝。 2. 可以逐步收缩 ptrace 的职责,而不是一次性替换所有行为。 -3. 现有测试可以复用,并且能做 ptrace/seccomp 双轨对照。 +3. 现有测试可以复用,并且能做 ptrace-only 与 hybrid 的结果对照。 + +当前 hybrid 明确选择 event-only resume:tracee 用 `PTRACE_CONT` 继续运行,ptrace 只接收 fork/clone/vfork 生命周期事件、`PTRACE_EVENT_SECCOMP`、信号停靠和退出事件;它不再通过 `PTRACE_SYSCALL` 观察普通 syscall 的 enter/exit stop。因此,“迁移期对照”不是全量 ptrace 复核 seccomp `ALLOW` 调用,而是通过 `Trace` / `Audit` 把需要观测的 syscall 转成 seccomp trace event。 ### Phase 2: 配置模型重构 目标:把当前配置从“ptrace 时代的 syscall 名单”重构成“seccomp 时代的策略模型”。 -建议新增一个显式策略层,例如: +当前实现已经新增一个兼容式显式策略层: ```json { "SyscallPolicy": { - "DefaultAction": "errno", - "Allowed": ["read", "write", "mmap", "brk"], - "Denied": ["socket", "connect", "bpf", "ptrace"], - "Trap": ["clone", "fork", "vfork", "execve"] + "Allow": ["read", "write", "mmap", "brk"], + "Deny": ["socket", "connect", "bpf", "ptrace"], + "Trace": ["clone", "fork", "vfork"], + "Audit": ["getpid"] } } ``` 这里的含义是: -1. `Allowed` 直接转成 seccomp allowlist。 -2. `Denied` 直接返回 `EPERM` 或 `KILL_PROCESS`。 -3. `Trap` 在 hybrid 模式下先交给 ptrace 或 `SECCOMP_RET_TRACE` 处理。 +1. `Allow` 进入 runtime allowlist;hybrid 模式下生成 seccomp `ALLOW` 规则。 +2. `Trace` 在 ptrace-only 模式下作为允许规则通过 ptrace;hybrid 模式下生成 `SECCOMP_RET_TRACE` 规则,交给 ptrace 处理事件。 +3. `Audit` 的放行方式和 `Trace` 一样,但会额外输出独立 audit 日志,用于迁移期按名单抽样观测。 +4. `Deny` 会从 legacy `AllowedCalls + AdditionCalls` 和 `SyscallPolicy.Allow` 的合并结果中删除对应 syscall;在 hybrid 模式下落到 seccomp 默认 `KILL_PROCESS`,在 ptrace-only 模式下落到默认拒绝。 +5. `OneTimeCalls` 仍保留启动边界语义,默认 `execve`;`Deny` 不能和 `OneTimeCalls` / `Trace` / `Audit` 重叠。 +6. hybrid 下 `write` / `close` / `exit` / `exit_group` 属于 runner 启动协议保留 syscall。它们可以被 runtime allow,但不能放入 `Deny` / `Trace` / `Audit` / `OneTimeCalls`,否则 child 在 seccomp 安装后可能无法回报启动失败或正常退出。 +7. `AllowedCalls` 和 `AdditionCalls` 作为 legacy runtime allowlist 继续兼容,并参与同一份 effective policy。 + +#### Breaking 行为:syscall ownership fail-loud + +当前实现有意收紧 syscall policy 的 ownership 契约:同一个 syscall 不能同时属于 `OneTimeCalls`、runtime allowlist(`AllowedCalls` / `AdditionCalls` / `SyscallPolicy.Allow`)、`SyscallPolicy.Trace`、`SyscallPolicy.Audit`。`SyscallPolicy.Deny` 只允许覆盖 runtime allowlist,不能和 `OneTimeCalls` / `Trace` / `Audit` 重叠。 + +这个校验在默认 `ptrace` 路径同样生效,不只影响 `hybrid`。因此,旧版本中曾经合法的 `OneTimeCalls` 与 `AllowedCalls` / `AdditionCalls` 重叠配置,升级后会在配置加载阶段失败;旧版 ptrace 的 allow 优先语义会让这种重叠退化为永久 allow,当前版本改为明确拒绝这种模糊配置。 + +迁移规则:如果某个 syscall 同时出现在 `OneTimeCalls` 和 runtime allowlist,需要按真实意图二选一;需要永久允许时从 `OneTimeCalls` 删除,只允许启动边界或一次性调用时从 `AllowedCalls` / `AdditionCalls` / `SyscallPolicy.Allow` 删除。 -这一步的关键收益是把“一次性调用”“需要审计的调用”“可直接拒绝的调用”分层,而不是全部混在一个白名单里。 +这一步的关键收益是把“一次性调用”“永久允许调用”“需要审计的调用”“显式拒绝的调用”分层,而不是全部混在一个白名单里。当前实现采用明确优先级:先合并 allow,再用 deny 做减法;allow/trace/audit/one-time 保持互斥。代码上先得到 `EffectiveSyscallPolicy`,再编译成后端输入:ptrace 消费 `callPolicySpec`,seccomp 消费最终 `ALLOW`/`TRACE` 两组 syscall 名称,调用点不再重复拼装四个桶。 ### Phase 3: 启动模型重构 @@ -114,8 +130,9 @@ 规则生成逻辑建议分三层: 1. 名称解析层:把 syscall 名称映射到当前架构的 syscall 编号。 -2. 策略归一层:把 `AllowedCalls`、语言差异和平台差异整理成统一策略。 -3. 后端生成层:输出 seccomp filter。 +2. 策略归一层:把 legacy `AllowedCalls` / `AdditionCalls`、结构化 `SyscallPolicy`、语言差异和平台差异整理成统一 `EffectiveSyscallPolicy`。 +3. 策略编译层:把 effective policy 编译成 ptrace 与 seccomp 各自消费的后端输入。 +4. 后端生成层:输出 seccomp filter。 这三层分开后,未来如果切换架构或更新 syscall 表,不需要直接动运行时逻辑。 @@ -131,11 +148,13 @@ 6. rootfs 阶段 7. no_new_privs 阶段 8. 凭证切换阶段(`setgroups` → `setgid` → `setuid`) -9. `installSeccomp()` -10. `ptraceTraceme()` 或等价的迁移期观测逻辑 -11. `execve()` +9. `ptraceTraceme()` 或等价的迁移期观测逻辑 +10. child 用 `SIGSTOP` 等待 parent 设置 `PTRACE_O_TRACESECCOMP` +11. `installSeccomp()` +12. `execve()` 这里把 `installSeccomp()` 放在凭证切换之后,是为了避免过滤器误伤前置的特权 syscall。 +首版实现进一步把它放在 `ptraceTraceme()` 之后,是为了不把 `ptrace` 加进用户程序继承的 runtime allowlist;`execve once` 不进入永久 allowlist,而是通过 `SECCOMP_RET_TRACE` 交给 ptrace 审批。 ## 测试计划 @@ -158,14 +177,14 @@ 3. 正常题解的基础 syscall 仍然放行 4. Java/C++ 等多语言启动路径没有被误杀 -### 3. 对照测试 +### 3. 结果对照测试 在 hybrid 阶段,同一组样例分别跑: 1. ptrace only 2. seccomp + ptrace -输出应对齐到同一份判题结果和关键日志指标。 +输出应对齐到同一份判题结果和关键日志指标。hybrid event-only 模式不会生成普通 syscall enter/exit 日志;需要迁移期观测的 syscall 应显式放入 `Trace` 或 `Audit`。当前集成覆盖包括 `tests/syscall-policy-deny-hybrid` 的 Deny 覆盖继承 allow,以及 `tests/syscall-policy-trace-audit-hybrid` 的 Trace/Audit 端到端审计日志。 ## 建议的首批实现范围 diff --git a/makefile b/makefile index 1c48521..8181f8d 100644 --- a/makefile +++ b/makefile @@ -42,6 +42,9 @@ pre-commit-run: testall: prepare cd tests/general;make cd tests/clone-syscall-phase;make + cd tests/clone-syscall-phase-hybrid;make + cd tests/syscall-policy-deny-hybrid;make + cd tests/syscall-policy-trace-audit-hybrid;make cd tests/fork;make cd tests/mle;make cd tests/mle2;make diff --git a/runner/README.md b/runner/README.md index fd44d54..9e8dfac 100644 --- a/runner/README.md +++ b/runner/README.md @@ -1,13 +1,13 @@ # runner -该目录是仓库核心库,负责运行 `case.json` 描述的任务,并在 Linux 上完成资源限制、沙箱隔离、ptrace syscall 检查、时间内存采集与结果归类。 +该目录是仓库核心库,负责运行 `case.json` 描述的任务,并在 Linux 上完成资源限制、沙箱隔离、syscall 检查、时间内存采集与结果归类。 ## 主要职责 - 解析并校验运行配置 - 启动子进程并设置资源限制、工作目录、IO 重定向 - 应用沙箱隔离能力,如 `chroot`、namespace、`no_new_privs`、降权 -- 使用 ptrace 跟踪 syscall,并结合 allowlist / one-time policy 做拦截 +- 使用 ptrace 跟踪 syscall,并结合 allowlist / one-time policy 做拦截;Linux 上可通过 `SyscallBackend: "hybrid"` 额外启用 seccomp-BPF 作为主过滤边界 - 采集时间与内存数据,输出统一 `Result` ## 关键文件 @@ -19,6 +19,7 @@ - `process*.go`:对子进程 `wait`、ptrace 选项、状态判断的封装。 - `tracer*.go`:syscall 读取与判定逻辑,核心标识符是 `TracerDetect`。 - `sec.go`:把 syscall 名称列表转换成可执行的 `CallPolicy`。 +- `seccomp_linux.go`:hybrid 模式下的 seccomp-BPF `ALLOW` / `TRACE` 规则生成与安装逻辑。 - `sandbox_linux.go`:namespace、`chroot`、`no_new_privs`、降权顺序的统一入口。 - `memory*.go`:内存采集逻辑。 - `result.go`:评测状态码及 signal 到状态码的映射。 @@ -36,7 +37,7 @@ - 想新增 `case.json` 配置项:先看 `config.go` - 想修改资源限制或 IO 文件:先看 `exec_linux.go` -- 想调整 syscall 允许策略:先看 `sec.go`、`tracer_linux.go`,再看 [`../sec/`](../sec/) +- 想调整 syscall 允许策略:先看 `sec.go`、`tracer_linux.go`、`seccomp_linux.go`,再看 [`../sec/`](../sec/) - 想调整沙箱执行顺序:先看 `sandbox_linux.go` - 想调整结果状态或信号映射:先看 `result.go` diff --git a/runner/config.go b/runner/config.go index 2ddb766..6305d21 100644 --- a/runner/config.go +++ b/runner/config.go @@ -38,12 +38,14 @@ type TaskConfig struct { UseUTSNS bool `default:"false"` // Isolate hostname/domainname UseNetNS bool `default:"false"` // Isolate network stack - OneTimeCalls []string `default:"execve"` - AllowedCalls []string `default:"read,write,brk,fstat,uname,mmap,exit_group,exit,readlinkat,faccessat,mprotect,set_tid_address,set_robust_list,rseq,prlimit64,getrandom,rt_sigreturn"` - AdditionCalls []string `default:""` - Verbose bool `default:"false"` - Name string - Result int `default:"4"` + OneTimeCalls []string `default:"execve"` + AllowedCalls []string `default:"read,write,brk,fstat,uname,mmap,exit_group,exit,readlinkat,faccessat,mprotect,set_tid_address,set_robust_list,rseq,prlimit64,getrandom,rt_sigreturn"` + AdditionCalls []string `default:""` + SyscallPolicy SyscallPolicyConfig + SyscallBackend string `default:"ptrace"` + Verbose bool `default:"false"` + Name string + Result int `default:"4"` //LogPath string `default:"/var/log/runner/runner.log"` LogPath string `default:"/dev/stderr"` @@ -55,6 +57,11 @@ const namespacePrivilegeWarning = "Namespaces are enabled but no privilege drop const memoryReserveDeprecatedWarning = "MemoryReserve is deprecated and ignored by the Linux cgroup v2 runtime" +const ( + syscallBackendPtrace = "ptrace" + syscallBackendHybrid = "hybrid" +) + // Validate checks if the configuration is valid. // Returns an error if any required constraints are violated. func (tc *TaskConfig) Validate() error { @@ -90,6 +97,24 @@ func (tc *TaskConfig) Validate() error { if err := validateSyscallNames("additionCalls", tc.AdditionCalls); err != nil { return err } + if err := tc.validateSyscallPolicy(); err != nil { + return err + } + backend := tc.effectiveSyscallBackend() + if err := validateSyscallBackend(backend); err != nil { + return err + } + if backend == syscallBackendHybrid && !tc.NoNewPrivs { + return errors.New("syscallBackend hybrid requires NoNewPrivs") + } + if err := validateSyscallBackendForPlatform(backend); err != nil { + return err + } + if backend == syscallBackendHybrid { + if err := validateHybridSyscallPolicy(tc.effectiveSyscallPolicy()); err != nil { + return err + } + } return nil } @@ -124,6 +149,22 @@ func (tc *TaskConfig) LogValidationWarnings() { } } +func (tc *TaskConfig) effectiveSyscallBackend() string { + if tc.SyscallBackend == "" { + return syscallBackendPtrace + } + return tc.SyscallBackend +} + +func validateSyscallBackend(backend string) error { + switch backend { + case syscallBackendPtrace, syscallBackendHybrid: + return nil + default: + return fmt.Errorf("syscallBackend must be %q or %q (got %q)", syscallBackendPtrace, syscallBackendHybrid, backend) + } +} + func (tc *TaskConfig) GetCommand() string { if err := tc.parseCommand(); err != nil || len(tc.commands) == 0 { return "" diff --git a/runner/config_darwin.go b/runner/config_darwin.go index bc0322d..d2d748b 100644 --- a/runner/config_darwin.go +++ b/runner/config_darwin.go @@ -1,7 +1,16 @@ package runner +import "fmt" + // validateSyscallNames is a no-op on non-Linux platforms where the syscall // table is unavailable. func validateSyscallNames(_ string, _ []string) error { return nil } + +func validateSyscallBackendForPlatform(backend string) error { + if backend == syscallBackendHybrid { + return fmt.Errorf("syscallBackend %q is only supported on linux", backend) + } + return nil +} diff --git a/runner/config_linux.go b/runner/config_linux.go index c920dd9..0d978be 100644 --- a/runner/config_linux.go +++ b/runner/config_linux.go @@ -17,3 +17,7 @@ func validateSyscallNames(field string, names []string) error { } return nil } + +func validateSyscallBackendForPlatform(_ string) error { + return nil +} diff --git a/runner/config_linux_test.go b/runner/config_linux_test.go index 74288d2..2979f0f 100644 --- a/runner/config_linux_test.go +++ b/runner/config_linux_test.go @@ -14,6 +14,10 @@ func TestValidateRejectsInvalidSyscallName(t *testing.T) { {"OneTimeCalls", `{"OneTimeCalls":["not_a_real_syscall"]}`, `invalid syscall in oneTimeCalls: "not_a_real_syscall"`}, {"AllowedCalls", `{"AllowedCalls":["bogus_call"]}`, `invalid syscall in allowedCalls: "bogus_call"`}, {"AdditionCalls", `{"AdditionCalls":["fake_syscall"]}`, `invalid syscall in additionCalls: "fake_syscall"`}, + {"SyscallPolicyAllow", `{"SyscallPolicy":{"Allow":["fake_syscall"]}}`, `invalid syscall in syscallPolicy.allow: "fake_syscall"`}, + {"SyscallPolicyDeny", `{"SyscallPolicy":{"Deny":["fake_syscall"]}}`, `invalid syscall in syscallPolicy.deny: "fake_syscall"`}, + {"SyscallPolicyTrace", `{"SyscallPolicy":{"Trace":["fake_syscall"]}}`, `invalid syscall in syscallPolicy.trace: "fake_syscall"`}, + {"SyscallPolicyAudit", `{"SyscallPolicy":{"Audit":["fake_syscall"]}}`, `invalid syscall in syscallPolicy.audit: "fake_syscall"`}, } for _, tt := range tests { @@ -55,3 +59,42 @@ func TestValidateAcceptsValidSyscallNames(t *testing.T) { } }) } + +func TestValidateAcceptsHybridSyscallBackend(t *testing.T) { + restoreGlobals := preserveConfigTestGlobals() + defer restoreGlobals() + + json := `{"SyscallBackend":"hybrid","NoNewPrivs":true,"OneTimeCalls":["execve"],"AllowedCalls":["read"],"AdditionCalls":["write"]}` + runWithTempCaseJSON(t, json, func() { + SetLogger(nil) + + cfg, err := LoadConfig() + if err != nil { + t.Fatalf("LoadConfig() error = %v", err) + } + if got := cfg.effectiveSyscallBackend(); got != syscallBackendHybrid { + t.Fatalf("effectiveSyscallBackend() = %q, want %q", got, syscallBackendHybrid) + } + }) +} + +func TestValidateRejectsHybridDenyOfStartupProtocolCalls(t *testing.T) { + restoreGlobals := preserveConfigTestGlobals() + defer restoreGlobals() + + json := `{"SyscallBackend":"hybrid","NoNewPrivs":true,"OneTimeCalls":["execve"],"AllowedCalls":["read"],"SyscallPolicy":{"Deny":["write"]}}` + runWithTempCaseJSON(t, json, func() { + SetLogger(nil) + + _, err := LoadConfig() + if err == nil { + t.Fatal("LoadConfig() error = nil, want startup protocol denial rejection") + } + if !strings.Contains(err.Error(), `syscallPolicy.deny cannot include "write"`) { + t.Fatalf("LoadConfig() error = %q, want denied startup syscall name", err) + } + if !strings.Contains(err.Error(), "hybrid startup protocol") { + t.Fatalf("LoadConfig() error = %q, want startup protocol context", err) + } + }) +} diff --git a/runner/config_test.go b/runner/config_test.go index d4a2a3b..6280227 100644 --- a/runner/config_test.go +++ b/runner/config_test.go @@ -106,6 +106,58 @@ func TestLoadConfigWarnsOnDeprecatedMemoryReserve(t *testing.T) { }) } +func TestSyscallBackendDefaultsToPtrace(t *testing.T) { + cfg := &TaskConfig{} + + if got := cfg.effectiveSyscallBackend(); got != syscallBackendPtrace { + t.Fatalf("effectiveSyscallBackend() = %q, want %q", got, syscallBackendPtrace) + } +} + +func TestValidateRejectsUnknownSyscallBackend(t *testing.T) { + cfg := &TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + SyscallBackend: "unknown", + } + + err := cfg.Validate() + if err == nil { + t.Fatal("Validate() error = nil, want invalid syscall backend") + } + if !strings.Contains(err.Error(), "syscallBackend must be") { + t.Fatalf("Validate() error = %q, want syscallBackend validation", err) + } +} + +func TestValidateRejectsHybridWithoutNoNewPrivs(t *testing.T) { + cfg := &TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: false, + SyscallBackend: syscallBackendHybrid, + } + + err := cfg.Validate() + if err == nil { + t.Fatal("Validate() error = nil, want hybrid without NoNewPrivs rejection") + } + if !strings.Contains(err.Error(), "requires NoNewPrivs") { + t.Fatalf("Validate() error = %q, want NoNewPrivs validation", err) + } +} + func TestParseCommandFallbackUsesShlex(t *testing.T) { tc := &TaskConfig{Command: ` /usr/bin/java "Main Class" `} if got := tc.GetCommand(); got != "/usr/bin/java" { diff --git a/runner/exec.go b/runner/exec.go index 6ee2916..65ed75f 100644 --- a/runner/exec.go +++ b/runner/exec.go @@ -51,11 +51,13 @@ func (task *RunningTask) trace() error { tracer := TracerDetect{} tracer.RegisterTracee(task.process.Pid, false) - allowedCalls := make([]string, 0, len(task.setting.AllowedCalls)+len(task.setting.AdditionCalls)) - allowedCalls = append(allowedCalls, task.setting.AllowedCalls...) - allowedCalls = append(allowedCalls, task.setting.AdditionCalls...) - log.Debugf("allowed syscall is: %s", allowedCalls) - policy, err := makeCallPolicy(&task.setting.OneTimeCalls, &allowedCalls) + syscallPolicy, err := task.setting.compileSyscallPolicy() + if err != nil { + process.Kill() + return fmt.Errorf("compile syscall policy: %w", err) + } + log.Debugf("allowed syscall is: %s", syscallPolicy.Ptrace.AllowedCalls) + policy, err := makeCallPolicy(syscallPolicy.Ptrace) if err != nil { process.Kill() return fmt.Errorf("build call policy: %w", err) @@ -87,14 +89,16 @@ func (task *RunningTask) trace() error { } process.SetThreadGroup(process.Pid, process.Pid) process.SetRusageOffset(process.Pid, process.Rusage.Maxrss) - if err := process.SetPtraceOptions(); err != nil { + traceSeccomp := task.traceSeccompEvents() + resumeMode := task.traceResumeMode() + if err := process.SetPtraceOptions(traceSeccomp); err != nil { log.Infof("PtraceSetOptions: err %v", err) task.Result.RetCode = RUNTIME_ERROR process.Kill() task.parseRunningInfo() return task.finalizeTraceResult() } - if !process.Continue() { + if !process.ContinueWithMode(resumeMode, 0) { log.Infof("Program not alive after ptrace setup") task.parseRunningInfo() return task.finalizeTraceResult() @@ -128,14 +132,14 @@ func (task *RunningTask) trace() error { continue } if tracer.ConsumeAttachStop(process.CurrentPid, process.Status) { - if err := process.SetPtraceOptions(); err != nil { + if err := process.SetPtraceOptions(traceSeccomp); err != nil { log.Infof("PtraceSetOptions(new child): err %v", err) task.Result.RetCode = RUNTIME_ERROR process.Kill() task.parseRunningInfo() break } - if !process.Continue() { + if !process.ContinueWithMode(resumeMode, 0) { log.Infof("Program not alive after child attach stop") task.parseRunningInfo() break @@ -146,7 +150,7 @@ func (task *RunningTask) trace() error { if !task.handlePtraceEvent(process, &tracer) { break } - if !process.Continue() { + if !process.ContinueWithMode(resumeMode, 0) { log.Infof("Program not alive after ptrace event") task.parseRunningInfo() break @@ -163,7 +167,7 @@ func (task *RunningTask) trace() error { log.Debugf("forwarding signal %v to pid=%d", sig, process.CurrentPid) task.parseRunningInfo() task.checkLimit() - if !process.ContinueWithSignal(int(sig)) { + if !process.ContinueWithMode(resumeMode, int(sig)) { log.Infof("Program not alive after signal forward") task.parseRunningInfo() break @@ -197,7 +201,7 @@ func (task *RunningTask) trace() error { task.parseRunningInfo() task.checkLimit() - if !process.Continue() { + if !process.ContinueWithMode(resumeMode, 0) { log.Infof("Program not alive! break") break } @@ -206,6 +210,17 @@ func (task *RunningTask) trace() error { return task.finalizeTraceResult() } +func (task *RunningTask) traceSeccompEvents() bool { + return task.setting.effectiveSyscallBackend() == syscallBackendHybrid +} + +func (task *RunningTask) traceResumeMode() traceResumeMode { + if task.traceSeccompEvents() { + return traceResumeEventStops + } + return traceResumeSyscallStops +} + func (task *RunningTask) handlePtraceEvent(process *Process, tracer *TracerDetect) bool { switch process.PtraceEvent() { case ptraceEventClone, ptraceEventFork, ptraceEventVFork: @@ -223,6 +238,24 @@ func (task *RunningTask) handlePtraceEvent(process *Process, tracer *TracerDetec } tracer.RegisterTracee(newPid, true) log.Infof("registered traced child pid=%d from event=%d", newPid, process.PtraceEvent()) + case ptraceEventSeccomp: + checkResult := tracer.checkSeccompTrace(process.CurrentPid) + if checkResult == syscallCheckViolation { + log.Debugf("------- check seccomp-traced syscall failed") + process.Kill() + task.Result.RetCode = RUNTIME_ERROR + return false + } + if checkResult == syscallCheckTraceeGone { + log.Debugf("skip seccomp trace inspection for pid=%d because tracee is already gone", process.CurrentPid) + return true + } + if checkResult == syscallCheckTracerError { + log.Warnf("ptrace register read failed for seccomp event pid=%d", process.CurrentPid) + process.Kill() + task.Result.RetCode = RUNTIME_ERROR + return false + } default: log.Warnf("unhandled ptrace event %d on pid=%d", process.PtraceEvent(), process.CurrentPid) } diff --git a/runner/exec_linux.go b/runner/exec_linux.go index 3ff5cf6..0d03404 100644 --- a/runner/exec_linux.go +++ b/runner/exec_linux.go @@ -37,6 +37,8 @@ const ( childStageSandboxSetgid childStageSandboxSetuid childStagePtraceTraceme + childStagePtraceSync + childStageSeccomp childStageExec ) @@ -93,6 +95,10 @@ func (s childStartupStage) String() string { return "setuid" case childStagePtraceTraceme: return "ptrace traceme" + case childStagePtraceSync: + return "ptrace sync" + case childStageSeccomp: + return "install seccomp filter" case childStageExec: return "execve" default: @@ -116,6 +122,7 @@ type childProcessSpec struct { io childIOFiles exec childExecSpec sandbox childSandboxSpec + seccomp childSeccompSpec cpuLimit syscall.Rlimit outputLimit syscall.Rlimit stackLimit syscall.Rlimit @@ -133,6 +140,15 @@ func ptraceTraceme() syscall.Errno { return errno } +func stopForPtraceOptions() syscall.Errno { + pid, _, errno := syscall.RawSyscall(syscall.SYS_GETPID, 0, 0, 0) + if errno != 0 { + return errno + } + _, _, errno = syscall.RawSyscall(syscall.SYS_KILL, pid, uintptr(syscall.SIGSTOP), 0) + return errno +} + var ( writeChildCgroupGate = syscall.Write wait4ChildStartup = syscall.Wait4 @@ -208,6 +224,13 @@ func (task *RunningTask) runProcess() error { waitChildStartupFailure(pid) return fmt.Errorf("release child cgroup gate: %w", err) } + if shouldSyncSeccompTracee(spec.seccomp) { + if err := prepareHybridSeccompTracee(pid); err != nil { + _ = syscall.Close(startupPipeFDs[0]) + killChildAfterStartupError(pid) + return fmt.Errorf("prepare hybrid seccomp tracee: %w", err) + } + } failure, err := readChildStartupFailure(startupPipeFDs[0]) _ = syscall.Close(startupPipeFDs[0]) @@ -244,6 +267,11 @@ func (task *RunningTask) prepareChildProcessSpec() (childProcessSpec, error) { closeChildIOFiles(ioFiles) return childProcessSpec{}, err } + seccompSpec, err := prepareChildSeccompSpec(task.setting) + if err != nil { + closeChildIOFiles(ioFiles) + return childProcessSpec{}, err + } timeLimit := uint64(task.setting.CPU) stackLimit := uint64(task.setting.Stack) << 20 @@ -254,6 +282,7 @@ func (task *RunningTask) prepareChildProcessSpec() (childProcessSpec, error) { io: ioFiles, exec: execSpec, sandbox: sandboxSpec, + seccomp: seccompSpec, cpuLimit: syscall.Rlimit{ Max: enforcedCPULimit, Cur: enforcedCPULimit, @@ -388,6 +417,94 @@ func waitChildStartupFailure(pid int) { } } +func shouldSyncSeccompTracee(spec childSeccompSpec) bool { + return spec.enabled && len(spec.traceSyscalls) > 0 +} + +func prepareHybridSeccompTracee(pid int) error { + status, err := waitForTraceeStop(pid) + if err != nil { + return fmt.Errorf("wait ptrace sync stop: %w", err) + } + if !status.Stopped() || status.StopSignal() != syscall.SIGSTOP { + return fmt.Errorf("unexpected hybrid ptrace sync status: %s", describeWaitStatus(status)) + } + if err := setPtraceOptions(pid, true); err != nil { + return fmt.Errorf("set ptrace options before seccomp install: %w", err) + } + if err := ptraceCont(pid, 0); err != nil { + return fmt.Errorf("continue child to seccomp execve event: %w", err) + } + + status, err = waitForTraceeStop(pid) + if err != nil { + return fmt.Errorf("wait seccomp execve event: %w", err) + } + if !isPtraceEvent(status, ptraceEventSeccomp) { + return fmt.Errorf("unexpected hybrid seccomp event status: %s", describeWaitStatus(status)) + } + if err := verifyStartupSeccompEvent(pid); err != nil { + return err + } + if err := ptraceCont(pid, 0); err != nil { + return fmt.Errorf("continue child after seccomp execve event: %w", err) + } + return nil +} + +func waitForTraceeStop(pid int) (syscall.WaitStatus, error) { + var status syscall.WaitStatus + var rusage syscall.Rusage + for { + waitedPID, err := syscall.Wait4(pid, &status, 0, &rusage) + if err == syscall.EINTR { + continue + } + if err != nil { + return 0, err + } + if waitedPID != pid { + return 0, fmt.Errorf("wait4 returned pid %d, want %d", waitedPID, pid) + } + return status, nil + } +} + +func isPtraceEvent(status syscall.WaitStatus, event int) bool { + return status.Stopped() && status.StopSignal() == syscall.SIGTRAP && status.TrapCause() == event +} + +func verifyStartupSeccompEvent(pid int) error { + var regs syscall.PtraceRegs + if err := ptraceGetRegs(pid, ®s); err != nil { + return fmt.Errorf("read regs for startup seccomp event: %w", err) + } + callID := getSyscallNumber(®s) + if callID != uint64(syscall.SYS_EXECVE) { + return fmt.Errorf("unexpected startup seccomp syscall %s(%d), want execve", getName(callID), callID) + } + return nil +} + +func describeWaitStatus(status syscall.WaitStatus) string { + switch { + case status.Stopped(): + return fmt.Sprintf("stopped signal=%v trap=%d", status.StopSignal(), status.TrapCause()) + case status.Exited(): + return fmt.Sprintf("exited code=%d", status.ExitStatus()) + case status.Signaled(): + return fmt.Sprintf("signaled signal=%v", status.Signal()) + default: + return fmt.Sprintf("raw=%#x", int(status)) + } +} + +func killChildAfterStartupError(pid int) { + _ = syscall.Kill(-pid, syscall.SIGKILL) + _ = syscall.Kill(pid, syscall.SIGKILL) + waitChildStartupFailure(pid) +} + func releaseChildCgroupGate(fd int) error { defer func() { _ = syscall.Close(fd) @@ -459,6 +576,15 @@ func runChildProcess(spec childProcessSpec, startupPipeReadFD, startupPipeWriteF if errno := ptraceTraceme(); errno != 0 { reportChildStartupFailure(startupPipeWriteFD, childStagePtraceTraceme, errno) } + if shouldSyncSeccompTracee(spec.seccomp) { + if errno := stopForPtraceOptions(); errno != 0 { + reportChildStartupFailure(startupPipeWriteFD, childStagePtraceSync, errno) + } + } + // Install after PTRACE_TRACEME so runtime policy does not need to allow ptrace. + if errno := installSeccompFilter(spec.seccomp); errno != 0 { + reportChildStartupFailure(startupPipeWriteFD, childStageSeccomp, errno) + } var argvPtr uintptr if len(spec.exec.argv) > 0 { diff --git a/runner/exec_linux_test.go b/runner/exec_linux_test.go index ef0a1c6..74563a3 100644 --- a/runner/exec_linux_test.go +++ b/runner/exec_linux_test.go @@ -104,6 +104,77 @@ func TestPrepareChildProcessSpecUsesConfiguredResourceLimits(t *testing.T) { } } +func TestPrepareChildProcessSpecBuildsHybridSeccompFilter(t *testing.T) { + previousWD, err := os.Getwd() + if err != nil { + t.Fatalf("os.Getwd() error = %v", err) + } + + tempDir := t.TempDir() + if err := os.Chdir(tempDir); err != nil { + t.Fatalf("os.Chdir(%q) error = %v", tempDir, err) + } + defer func() { + if err := os.Chdir(previousWD); err != nil { + t.Fatalf("restore working directory error = %v", err) + } + }() + + if err := os.WriteFile(filepath.Join(tempDir, "user.in"), []byte(""), 0o600); err != nil { + t.Fatalf("os.WriteFile(user.in) error = %v", err) + } + + task := RunningTask{ + setting: &TaskConfig{ + CPU: 2, + Memory: 64, + Output: 16, + Stack: 8, + MaxProcs: 16, + Command: "/bin/true", + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + SyscallBackend: syscallBackendHybrid, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + AdditionCalls: []string{"write"}, + }, + memoryLimit: 64 * 1024, + } + + spec, err := task.prepareChildProcessSpec() + if err != nil { + t.Fatalf("prepareChildProcessSpec() error = %v", err) + } + defer closeChildIOFiles(spec.io) + + if !spec.seccomp.enabled { + t.Fatal("prepareChildProcessSpec() seccomp.enabled = false, want true") + } + if len(spec.seccomp.filter) == 0 { + t.Fatal("prepareChildProcessSpec() seccomp.filter is empty") + } +} + +func TestChildStageSeccompString(t *testing.T) { + if got := childStageSeccomp.String(); got != "install seccomp filter" { + t.Fatalf("childStageSeccomp.String() = %q", got) + } +} + +func TestTraceResumeModeUsesEventStopsForHybridOnly(t *testing.T) { + defaultTask := RunningTask{setting: &TaskConfig{}} + if got := defaultTask.traceResumeMode(); got != traceResumeSyscallStops { + t.Fatalf("default traceResumeMode() = %d, want syscall stops", got) + } + + hybridTask := RunningTask{setting: &TaskConfig{SyscallBackend: syscallBackendHybrid}} + if got := hybridTask.traceResumeMode(); got != traceResumeEventStops { + t.Fatalf("hybrid traceResumeMode() = %d, want event stops", got) + } +} + func TestReleaseChildCgroupGateRetriesEINTR(t *testing.T) { original := writeChildCgroupGate t.Cleanup(func() { diff --git a/runner/process.go b/runner/process.go index 28c8280..843564b 100644 --- a/runner/process.go +++ b/runner/process.go @@ -10,6 +10,13 @@ type processStop struct { rusage syscall.Rusage } +type traceResumeMode uint8 + +const ( + traceResumeSyscallStops traceResumeMode = iota + traceResumeEventStops +) + type Process struct { Pid int CurrentPid int diff --git a/runner/process_darwin.go b/runner/process_darwin.go index e9ae04e..f21dddb 100644 --- a/runner/process_darwin.go +++ b/runner/process_darwin.go @@ -5,9 +5,10 @@ package runner import "errors" const ( - ptraceEventFork = 1 - ptraceEventVFork = 2 - ptraceEventClone = 3 + ptraceEventFork = 1 + ptraceEventVFork = 2 + ptraceEventClone = 3 + ptraceEventSeccomp = 4 ) func waitOptions() int { @@ -22,6 +23,10 @@ func (process *Process) ContinueWithSignal(_ int) bool { panic("Process.ContinueWithSignal is not supported on darwin") } +func (process *Process) ContinueWithMode(_ traceResumeMode, _ int) bool { + panic("Process.ContinueWithMode is not supported on darwin") +} + func (process *Process) IsInitialTraceStop() bool { panic("Process.IsInitialTraceStop is not supported on darwin") } @@ -42,6 +47,6 @@ func (process *Process) GetEventPid() (int, error) { return 0, errors.New("Process.GetEventPid is not supported on darwin") } -func (process *Process) SetPtraceOptions() error { +func (process *Process) SetPtraceOptions(_ bool) error { return errors.New("Process.SetPtraceOptions is not supported on darwin") } diff --git a/runner/process_linux.go b/runner/process_linux.go index afde362..4d86ccd 100644 --- a/runner/process_linux.go +++ b/runner/process_linux.go @@ -4,38 +4,61 @@ package runner import ( "syscall" + + "golang.org/x/sys/unix" ) const syscallStopSignal = syscall.Signal(syscall.SIGTRAP | 0x80) const ptraceOExitKill = 0x00100000 -const ptraceTraceEvents = syscall.PTRACE_O_TRACECLONE | syscall.PTRACE_O_TRACEFORK | syscall.PTRACE_O_TRACEVFORK +const ptraceForkTraceEvents = syscall.PTRACE_O_TRACECLONE | syscall.PTRACE_O_TRACEFORK | syscall.PTRACE_O_TRACEVFORK const ( - ptraceEventFork = syscall.PTRACE_EVENT_FORK - ptraceEventVFork = syscall.PTRACE_EVENT_VFORK - ptraceEventClone = syscall.PTRACE_EVENT_CLONE + ptraceEventFork = syscall.PTRACE_EVENT_FORK + ptraceEventVFork = syscall.PTRACE_EVENT_VFORK + ptraceEventClone = syscall.PTRACE_EVENT_CLONE + ptraceEventSeccomp = unix.PTRACE_EVENT_SECCOMP ) func waitOptions() int { return syscall.WALL } +var ( + ptraceSyscallCall = syscall.PtraceSyscall + ptraceContCall = syscall.PtraceCont +) + func (process *Process) Continue() bool { return process.ContinueWithSignal(0) } func (process *Process) ContinueWithSignal(sig int) bool { + return process.ContinueWithMode(traceResumeSyscallStops, sig) +} + +func (process *Process) ContinueWithMode(mode traceResumeMode, sig int) bool { if process.IsKilled { return false } - err := syscall.PtraceSyscall(process.CurrentPid, sig) + err := resumeTracee(process.CurrentPid, mode, sig) if err != nil { - log.Infof("PtraceSyscall(sig=%d): err %v", sig, err) + log.Infof("ptrace resume mode=%d sig=%d: err %v", mode, sig, err) return false } return true } +func resumeTracee(pid int, mode traceResumeMode, sig int) error { + if mode == traceResumeEventStops { + return ptraceContCall(pid, sig) + } + return ptraceSyscallCall(pid, sig) +} + +func ptraceCont(pid int, sig int) error { + return ptraceContCall(pid, sig) +} + func (process *Process) IsInitialTraceStop() bool { return process.Status.Stopped() && process.Status.StopSignal() == syscall.SIGTRAP && process.Status.TrapCause() == 0 } @@ -57,6 +80,14 @@ func (process *Process) GetEventPid() (int, error) { return int(msg), err } -func (process *Process) SetPtraceOptions() error { - return syscall.PtraceSetOptions(process.CurrentPid, syscall.PTRACE_O_TRACESYSGOOD|ptraceOExitKill|ptraceTraceEvents) +func (process *Process) SetPtraceOptions(traceSeccomp bool) error { + return setPtraceOptions(process.CurrentPid, traceSeccomp) +} + +func setPtraceOptions(pid int, traceSeccomp bool) error { + options := syscall.PTRACE_O_TRACESYSGOOD | ptraceOExitKill | ptraceForkTraceEvents + if traceSeccomp { + options |= unix.PTRACE_O_TRACESECCOMP + } + return syscall.PtraceSetOptions(pid, options) } diff --git a/runner/process_linux_test.go b/runner/process_linux_test.go index a4588d0..3cf1243 100644 --- a/runner/process_linux_test.go +++ b/runner/process_linux_test.go @@ -32,6 +32,56 @@ func TestProcessTraceStopClassification(t *testing.T) { assert.False(t, process.IsPtraceEventStop()) } +func TestProcessContinueWithModeUsesPtraceSyscallForSyscallStops(t *testing.T) { + originalSyscall := ptraceSyscallCall + originalCont := ptraceContCall + t.Cleanup(func() { + ptraceSyscallCall = originalSyscall + ptraceContCall = originalCont + }) + + var syscallCalled bool + ptraceSyscallCall = func(pid int, sig int) error { + syscallCalled = true + assert.Equal(t, 123, pid) + assert.Equal(t, int(syscall.SIGTERM), sig) + return nil + } + ptraceContCall = func(pid int, sig int) error { + t.Fatalf("PtraceCont called with pid=%d sig=%d, want PtraceSyscall", pid, sig) + return nil + } + + process := &Process{CurrentPid: 123} + assert.True(t, process.ContinueWithMode(traceResumeSyscallStops, int(syscall.SIGTERM))) + assert.True(t, syscallCalled) +} + +func TestProcessContinueWithModeUsesPtraceContForEventStops(t *testing.T) { + originalSyscall := ptraceSyscallCall + originalCont := ptraceContCall + t.Cleanup(func() { + ptraceSyscallCall = originalSyscall + ptraceContCall = originalCont + }) + + var contCalled bool + ptraceSyscallCall = func(pid int, sig int) error { + t.Fatalf("PtraceSyscall called with pid=%d sig=%d, want PtraceCont", pid, sig) + return nil + } + ptraceContCall = func(pid int, sig int) error { + contCalled = true + assert.Equal(t, 123, pid) + assert.Equal(t, int(syscall.SIGUSR1), sig) + return nil + } + + process := &Process{CurrentPid: 123} + assert.True(t, process.ContinueWithMode(traceResumeEventStops, int(syscall.SIGUSR1))) + assert.True(t, contCalled) +} + func TestCallPolicyConsume(t *testing.T) { policy := &CallPolicy{ oneTimeCalls: map[uint64]bool{uint64(syscall.SYS_EXECVE): true}, @@ -42,6 +92,17 @@ func TestCallPolicyConsume(t *testing.T) { assert.False(t, policy.CheckID(uint64(syscall.SYS_EXECVE))) } +func TestCallPolicyAuditCalls(t *testing.T) { + policy, err := makeCallPolicy(callPolicySpec{ + AllowedCalls: []string{"read"}, + AuditCalls: []string{"read"}, + }) + assert.NoError(t, err) + assert.True(t, policy.CheckID(uint64(syscall.SYS_READ))) + assert.True(t, policy.ShouldAudit(uint64(syscall.SYS_READ))) + assert.False(t, policy.ShouldAudit(uint64(syscall.SYS_WRITE))) +} + func TestProcessReturnsPendingStopAfterTraceeRegistration(t *testing.T) { process := NewProcess(100) process.rememberPendingStop(101, syscall.WaitStatus((int(syscall.SIGSTOP)<<8)|0x7f), syscall.Rusage{Maxrss: 64}) diff --git a/runner/sec.go b/runner/sec.go index 40270f1..0a68910 100644 --- a/runner/sec.go +++ b/runner/sec.go @@ -9,30 +9,45 @@ import ( type CallPolicy struct { oneTimeCalls map[uint64]bool allowedCalls map[uint64]bool + auditCalls map[uint64]bool } -func makeCallPolicy(ones *[]string, allows *[]string) (*CallPolicy, error) { - oneTimeCalls := make(map[uint64]bool) - allowedCalls := make(map[uint64]bool) - for _, s := range *ones { - n, err := sec.SCTbl.GetID(s) - if err != nil { - return nil, fmt.Errorf("one-time syscall %q: %w", s, err) - } - oneTimeCalls[uint64(n)] = true +type callPolicySpec struct { + OneTimeCalls []string + AllowedCalls []string + AuditCalls []string +} + +func makeCallPolicy(spec callPolicySpec) (*CallPolicy, error) { + oneTimeCalls, err := syscallIDSet("one-time syscall", spec.OneTimeCalls) + if err != nil { + return nil, err + } + allowedCalls, err := syscallIDSet("allowed syscall", spec.AllowedCalls) + if err != nil { + return nil, err + } + auditCalls, err := syscallIDSet("audit syscall", spec.AuditCalls) + if err != nil { + return nil, err } - for _, s := range *allows { + + return &CallPolicy{oneTimeCalls, allowedCalls, auditCalls}, nil +} + +func syscallIDSet(category string, names []string) (map[uint64]bool, error) { + calls := make(map[uint64]bool) + for _, s := range names { if len(s) == 0 { continue } n, err := sec.SCTbl.GetID(s) if err != nil { - return nil, fmt.Errorf("allowed syscall %q: %w", s, err) + return nil, fmt.Errorf("%s %q: %w", category, s, err) } - allowedCalls[uint64(n)] = true + calls[uint64(n)] = true } - - return &CallPolicy{oneTimeCalls, allowedCalls}, nil + return calls, nil } func (c *CallPolicy) CheckID(callID uint64) bool { @@ -49,3 +64,7 @@ func (c *CallPolicy) CheckID(callID uint64) bool { func (c *CallPolicy) Consume(callID uint64) { delete(c.oneTimeCalls, callID) } + +func (c *CallPolicy) ShouldAudit(callID uint64) bool { + return c.auditCalls[callID] +} diff --git a/runner/seccomp_linux.go b/runner/seccomp_linux.go new file mode 100644 index 0000000..fe48946 --- /dev/null +++ b/runner/seccomp_linux.go @@ -0,0 +1,205 @@ +//go:build linux + +package runner + +import ( + "errors" + "fmt" + "runtime" + "syscall" + "unsafe" + + "github.com/hustoj/runner/sec" + "golang.org/x/sys/unix" +) + +const ( + seccompDataSyscallOffset = 0 + seccompDataArchOffset = 4 +) + +type childSeccompSpec struct { + enabled bool + filter []unix.SockFilter + traceSyscalls []uint32 +} + +func prepareChildSeccompSpec(setting *TaskConfig) (childSeccompSpec, error) { + if setting.effectiveSyscallBackend() != syscallBackendHybrid { + return childSeccompSpec{}, nil + } + + policy, err := setting.compileSyscallPolicy() + if err != nil { + return childSeccompSpec{}, err + } + if err := validateHybridSyscallPolicy(setting.effectiveSyscallPolicy()); err != nil { + return childSeccompSpec{}, err + } + allowedIDs, err := seccompSyscallIDs(policy.SeccompAllowedCalls) + if err != nil { + return childSeccompSpec{}, err + } + traceIDs, err := seccompSyscallIDs(policy.SeccompTracedCalls) + if err != nil { + return childSeccompSpec{}, err + } + oneTimeIDs, err := seccompSyscallIDs(policy.Ptrace.OneTimeCalls) + if err != nil { + return childSeccompSpec{}, err + } + if err := validateHybridTracePolicy(allowedIDs, traceIDs, oneTimeIDs); err != nil { + return childSeccompSpec{}, err + } + filter, err := buildSeccompHybridFilter(allowedIDs, traceIDs) + if err != nil { + return childSeccompSpec{}, err + } + return childSeccompSpec{ + enabled: true, + filter: filter, + traceSyscalls: traceIDs, + }, nil +} + +func seccompSyscallIDs(names []string) ([]uint32, error) { + seen := make(map[uint32]struct{}, len(names)) + for _, name := range names { + if name == "" { + continue + } + id, err := sec.SCTbl.GetID(name) + if err != nil { + return nil, fmt.Errorf("seccomp syscall %q: %w", name, err) + } + seen[uint32(id)] = struct{}{} + } + + ids := make([]uint32, 0, len(seen)) + for id := range seen { + ids = append(ids, id) + } + sortUint32(ids) + return ids, nil +} + +func validateHybridTracePolicy(allowedIDs, traceIDs, oneTimeIDs []uint32) error { + for _, traceID := range traceIDs { + if containsUint32(allowedIDs, traceID) { + return fmt.Errorf("seccomp hybrid requires traced syscall %s(%d) to be trace-only; remove it from the runtime allowlist", getName(uint64(traceID)), traceID) + } + } + + execveID := uint32(syscall.SYS_EXECVE) + if !containsUint32(oneTimeIDs, execveID) { + return errors.New("seccomp hybrid requires execve in OneTimeCalls") + } + return nil +} + +func buildSeccompHybridFilter(allowedSyscalls, tracedSyscalls []uint32) ([]unix.SockFilter, error) { + if len(allowedSyscalls) == 0 && len(tracedSyscalls) == 0 { + return nil, errors.New("seccomp policy cannot be empty") + } + + auditArch, err := seccompAuditArch() + if err != nil { + return nil, err + } + + filter := []unix.SockFilter{ + seccompStmt(unix.BPF_LD|unix.BPF_W|unix.BPF_ABS, seccompDataArchOffset), + seccompJump(unix.BPF_JMP|unix.BPF_JEQ|unix.BPF_K, auditArch, 1, 0), + seccompStmt(unix.BPF_RET|unix.BPF_K, unix.SECCOMP_RET_KILL_PROCESS), + seccompStmt(unix.BPF_LD|unix.BPF_W|unix.BPF_ABS, seccompDataSyscallOffset), + } + filter = appendSeccompReturnRules(filter, allowedSyscalls, unix.SECCOMP_RET_ALLOW) + filter = appendSeccompReturnRules(filter, tracedSyscalls, unix.SECCOMP_RET_TRACE) + filter = append(filter, seccompStmt(unix.BPF_RET|unix.BPF_K, unix.SECCOMP_RET_KILL_PROCESS)) + if len(filter) > unix.BPF_MAXINSNS { + return nil, fmt.Errorf("seccomp filter exceeds instruction limit: %d > %d", len(filter), unix.BPF_MAXINSNS) + } + return filter, nil +} + +func appendSeccompReturnRules(filter []unix.SockFilter, syscallIDs []uint32, action uint32) []unix.SockFilter { + for _, syscallID := range syscallIDs { + filter = append( + filter, + seccompJump(unix.BPF_JMP|unix.BPF_JEQ|unix.BPF_K, syscallID, 0, 1), + seccompStmt(unix.BPF_RET|unix.BPF_K, action), + ) + } + return filter +} + +func seccompAuditArch() (uint32, error) { + switch runtime.GOARCH { + case "amd64": + return unix.AUDIT_ARCH_X86_64, nil + case "arm64": + return unix.AUDIT_ARCH_AARCH64, nil + default: + return 0, fmt.Errorf("seccomp hybrid is not supported on linux/%s", runtime.GOARCH) + } +} + +func installSeccompFilter(spec childSeccompSpec) syscall.Errno { + if !spec.enabled { + return 0 + } + if len(spec.filter) == 0 { + return syscall.EINVAL + } + if len(spec.filter) > unix.BPF_MAXINSNS { + return syscall.EINVAL + } + + program := unix.SockFprog{ + Len: uint16(len(spec.filter)), + Filter: &spec.filter[0], + } + _, _, errno := syscall.RawSyscall6( + syscall.SYS_PRCTL, + uintptr(unix.PR_SET_SECCOMP), + uintptr(unix.SECCOMP_MODE_FILTER), + uintptr(unsafe.Pointer(&program)), + 0, + 0, + 0, + ) + return errno +} + +func seccompStmt(code, k uint32) unix.SockFilter { + return unix.SockFilter{ + Code: uint16(code), + K: k, + } +} + +func seccompJump(code, k uint32, jt, jf uint8) unix.SockFilter { + return unix.SockFilter{ + Code: uint16(code), + Jt: jt, + Jf: jf, + K: k, + } +} + +func sortUint32(values []uint32) { + for i := 1; i < len(values); i++ { + for j := i; j > 0 && values[j-1] > values[j]; j-- { + values[j-1], values[j] = values[j], values[j-1] + } + } +} + +func containsUint32(values []uint32, target uint32) bool { + for _, value := range values { + if value == target { + return true + } + } + return false +} diff --git a/runner/seccomp_linux_test.go b/runner/seccomp_linux_test.go new file mode 100644 index 0000000..38cd13c --- /dev/null +++ b/runner/seccomp_linux_test.go @@ -0,0 +1,266 @@ +//go:build linux + +package runner + +import ( + "strings" + "syscall" + "testing" + + "golang.org/x/sys/unix" +) + +func TestSeccompSyscallIDsDeduplicatesAndSorts(t *testing.T) { + ids, err := seccompSyscallIDs([]string{"write", "read", "read"}) + if err != nil { + t.Fatalf("seccompSyscallIDs() error = %v", err) + } + + want := []uint32{uint32(syscall.SYS_READ), uint32(syscall.SYS_WRITE)} + if len(ids) != len(want) { + t.Fatalf("seccompSyscallIDs() = %v, want %v", ids, want) + } + for i := range want { + if ids[i] != want[i] { + t.Fatalf("seccompSyscallIDs()[%d] = %d, want %d; full=%v", i, ids[i], want[i], ids) + } + } +} + +func TestPrepareChildSeccompSpecIncludesStartupFailureReportingCalls(t *testing.T) { + spec, err := prepareChildSeccompSpec(&TaskConfig{ + SyscallBackend: syscallBackendHybrid, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + }) + if err != nil { + t.Fatalf("prepareChildSeccompSpec() error = %v", err) + } + if !spec.enabled { + t.Fatal("prepareChildSeccompSpec() enabled = false, want true") + } + if !containsUint32(spec.traceSyscalls, uint32(syscall.SYS_EXECVE)) { + t.Fatalf("prepareChildSeccompSpec() traceSyscalls = %v, want execve", spec.traceSyscalls) + } + + filter := spec.filter + for _, syscallID := range []uint32{ + uint32(syscall.SYS_WRITE), + uint32(syscall.SYS_CLOSE), + uint32(syscall.SYS_EXIT), + uint32(syscall.SYS_EXIT_GROUP), + } { + if !seccompFilterHasSyscall(filter, syscallID) { + t.Fatalf("seccomp filter does not allow startup reporting syscall %d", syscallID) + } + } + action, ok := seccompFilterActionForSyscall(filter, uint32(syscall.SYS_EXECVE)) + if !ok { + t.Fatal("seccomp filter has no execve rule") + } + if action != unix.SECCOMP_RET_TRACE { + t.Fatalf("execve seccomp action = %#x, want TRACE", action) + } +} + +func TestPrepareChildSeccompSpecTracesStructuredPolicyCalls(t *testing.T) { + spec, err := prepareChildSeccompSpec(&TaskConfig{ + SyscallBackend: syscallBackendHybrid, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + SyscallPolicy: SyscallPolicyConfig{ + Trace: []string{"getpid"}, + Audit: []string{"getppid"}, + }, + }) + if err != nil { + t.Fatalf("prepareChildSeccompSpec() error = %v", err) + } + + for _, syscallID := range []uint32{ + uint32(syscall.SYS_EXECVE), + uint32(syscall.SYS_GETPID), + uint32(syscall.SYS_GETPPID), + } { + action, ok := seccompFilterActionForSyscall(spec.filter, syscallID) + if !ok { + t.Fatalf("seccomp filter has no rule for syscall %d", syscallID) + } + if action != unix.SECCOMP_RET_TRACE { + t.Fatalf("syscall %d seccomp action = %#x, want TRACE", syscallID, action) + } + } + + action, ok := seccompFilterActionForSyscall(spec.filter, uint32(syscall.SYS_READ)) + if !ok { + t.Fatal("seccomp filter has no read rule") + } + if action != unix.SECCOMP_RET_ALLOW { + t.Fatalf("read seccomp action = %#x, want ALLOW", action) + } +} + +func TestPrepareChildSeccompSpecRemovesDeniedAllowedCalls(t *testing.T) { + spec, err := prepareChildSeccompSpec(&TaskConfig{ + SyscallBackend: syscallBackendHybrid, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read", "write"}, + SyscallPolicy: SyscallPolicyConfig{ + Allow: []string{"fstat"}, + Deny: []string{"read", "fstat"}, + }, + }) + if err != nil { + t.Fatalf("prepareChildSeccompSpec() error = %v", err) + } + + for _, syscallID := range []uint32{uint32(syscall.SYS_READ), uint32(syscall.SYS_FSTAT)} { + if action, ok := seccompFilterActionForSyscall(spec.filter, syscallID); ok { + t.Fatalf("denied syscall %d has seccomp action %#x, want no allow/trace rule", syscallID, action) + } + } + action, ok := seccompFilterActionForSyscall(spec.filter, uint32(syscall.SYS_WRITE)) + if !ok { + t.Fatal("seccomp filter has no write rule") + } + if action != unix.SECCOMP_RET_ALLOW { + t.Fatalf("write seccomp action = %#x, want ALLOW", action) + } +} + +func TestPrepareChildSeccompSpecRejectsDeniedStartupProtocolCalls(t *testing.T) { + _, err := prepareChildSeccompSpec(&TaskConfig{ + SyscallBackend: syscallBackendHybrid, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + SyscallPolicy: SyscallPolicyConfig{Deny: []string{"write"}}, + }) + if err == nil { + t.Fatal("prepareChildSeccompSpec() error = nil, want startup protocol denial rejection") + } + if !strings.Contains(err.Error(), "hybrid startup protocol") { + t.Fatalf("prepareChildSeccompSpec() error = %q, want startup protocol context", err) + } +} + +func TestPrepareChildSeccompSpecRejectsDenyTraceOverlap(t *testing.T) { + _, err := prepareChildSeccompSpec(&TaskConfig{ + SyscallBackend: syscallBackendHybrid, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + SyscallPolicy: SyscallPolicyConfig{ + Trace: []string{"getpid"}, + Deny: []string{"getpid"}, + }, + }) + if err == nil { + t.Fatal("prepareChildSeccompSpec() error = nil, want deny/trace overlap rejection") + } +} + +func TestPrepareChildSeccompSpecRejectsPermanentExecveAllow(t *testing.T) { + _, err := prepareChildSeccompSpec(&TaskConfig{ + SyscallBackend: syscallBackendHybrid, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read", "execve"}, + }) + if err == nil { + t.Fatal("prepareChildSeccompSpec() error = nil, want execve allowlist rejection") + } +} + +func TestPrepareChildSeccompSpecRejectsAnyOneTimeAllowOverlap(t *testing.T) { + _, err := prepareChildSeccompSpec(&TaskConfig{ + SyscallBackend: syscallBackendHybrid, + OneTimeCalls: []string{"execve", "getpid"}, + AllowedCalls: []string{"read"}, + AdditionCalls: []string{"getpid"}, + }) + if err == nil { + t.Fatal("prepareChildSeccompSpec() error = nil, want one-time overlap rejection") + } +} + +func TestPrepareChildSeccompSpecRequiresExecveOneTime(t *testing.T) { + _, err := prepareChildSeccompSpec(&TaskConfig{ + SyscallBackend: syscallBackendHybrid, + AllowedCalls: []string{"read"}, + }) + if err == nil { + t.Fatal("prepareChildSeccompSpec() error = nil, want missing execve one-time rejection") + } +} + +func TestPrepareChildSeccompSpecRequiresExecveInOneTimeCalls(t *testing.T) { + _, err := prepareChildSeccompSpec(&TaskConfig{ + SyscallBackend: syscallBackendHybrid, + OneTimeCalls: []string{"getpid"}, + AllowedCalls: []string{"read"}, + SyscallPolicy: SyscallPolicyConfig{Trace: []string{"execve"}}, + }) + if err == nil { + t.Fatal("prepareChildSeccompSpec() error = nil, want execve one-time rejection") + } +} + +func TestBuildSeccompHybridFilterRejectsTooManyInstructions(t *testing.T) { + allowedSyscalls := make([]uint32, (unix.BPF_MAXINSNS-5)/2+1) + _, err := buildSeccompHybridFilter(allowedSyscalls, nil) + if err == nil { + t.Fatal("buildSeccompHybridFilter() error = nil, want instruction limit rejection") + } + if !strings.Contains(err.Error(), "seccomp filter exceeds") { + t.Fatalf("buildSeccompHybridFilter() error = %q, want instruction limit context", err) + } +} + +func TestInstallSeccompFilterRejectsTooManyInstructions(t *testing.T) { + errno := installSeccompFilter(childSeccompSpec{ + enabled: true, + filter: make([]unix.SockFilter, unix.BPF_MAXINSNS+1), + }) + if errno != syscall.EINVAL { + t.Fatalf("installSeccompFilter() errno = %v, want EINVAL", errno) + } +} + +func TestBuildSeccompAllowlistFilterRejectsWrongArchBeforeSyscallAllowlist(t *testing.T) { + filter, err := buildSeccompHybridFilter([]uint32{uint32(syscall.SYS_READ)}, []uint32{uint32(syscall.SYS_EXECVE)}) + if err != nil { + t.Fatalf("buildSeccompHybridFilter() error = %v", err) + } + + if len(filter) < 5 { + t.Fatalf("filter len = %d, want at least 5", len(filter)) + } + if filter[0].Code != uint16(unix.BPF_LD|unix.BPF_W|unix.BPF_ABS) || filter[0].K != seccompDataArchOffset { + t.Fatalf("filter[0] = %#v, want load arch", filter[0]) + } + if filter[1].Code != uint16(unix.BPF_JMP|unix.BPF_JEQ|unix.BPF_K) || filter[1].Jt != 1 || filter[1].Jf != 0 { + t.Fatalf("filter[1] = %#v, want arch equality jump", filter[1]) + } + if filter[2].Code != uint16(unix.BPF_RET|unix.BPF_K) || filter[2].K != unix.SECCOMP_RET_KILL_PROCESS { + t.Fatalf("filter[2] = %#v, want kill wrong arch", filter[2]) + } + if filter[3].Code != uint16(unix.BPF_LD|unix.BPF_W|unix.BPF_ABS) || filter[3].K != seccompDataSyscallOffset { + t.Fatalf("filter[3] = %#v, want load syscall nr", filter[3]) + } + if filter[len(filter)-1].K != unix.SECCOMP_RET_KILL_PROCESS { + t.Fatalf("last filter = %#v, want default kill", filter[len(filter)-1]) + } +} + +func seccompFilterHasSyscall(filter []unix.SockFilter, syscallID uint32) bool { + _, ok := seccompFilterActionForSyscall(filter, syscallID) + return ok +} + +func seccompFilterActionForSyscall(filter []unix.SockFilter, syscallID uint32) (uint32, bool) { + for i := 0; i < len(filter)-1; i++ { + instruction := filter[i] + if instruction.Code == uint16(unix.BPF_JMP|unix.BPF_JEQ|unix.BPF_K) && instruction.K == syscallID { + return filter[i+1].K, true + } + } + return 0, false +} diff --git a/runner/syscall_policy.go b/runner/syscall_policy.go new file mode 100644 index 0000000..582669a --- /dev/null +++ b/runner/syscall_policy.go @@ -0,0 +1,201 @@ +package runner + +import "fmt" + +type SyscallPolicyConfig struct { + Allow []string `default:"" json:"Allow"` + Deny []string `default:"" json:"Deny"` + Trace []string `default:"" json:"Trace"` + Audit []string `default:"" json:"Audit"` +} + +type EffectiveSyscallPolicy struct { + OneTime []string + Allow []string + Deny []string + Trace []string + Audit []string +} + +type compiledSyscallPolicy struct { + Ptrace callPolicySpec + SeccompAllowedCalls []string + SeccompTracedCalls []string +} + +// postSeccompStartupCalls are owned by the runner's hybrid startup protocol. +// User policy cannot deny them because the child may need them after installing +// the seccomp filter to report execve failures and exit cleanly. +var postSeccompStartupCalls = []string{"write", "close", "exit", "exit_group"} + +func (tc *TaskConfig) effectiveSyscallPolicy() EffectiveSyscallPolicy { + allow := make([]string, 0, len(tc.AllowedCalls)+len(tc.AdditionCalls)+len(tc.SyscallPolicy.Allow)) + allow = append(allow, tc.AllowedCalls...) + allow = append(allow, tc.AdditionCalls...) + allow = append(allow, tc.SyscallPolicy.Allow...) + deny := uniqueSyscallNames(tc.SyscallPolicy.Deny) + + return EffectiveSyscallPolicy{ + OneTime: uniqueSyscallNames(tc.OneTimeCalls), + Allow: removeSyscallNames(uniqueSyscallNames(allow), deny), + Deny: deny, + Trace: uniqueSyscallNames(tc.SyscallPolicy.Trace), + Audit: uniqueSyscallNames(tc.SyscallPolicy.Audit), + } +} + +func (tc *TaskConfig) validateSyscallPolicy() error { + for _, check := range []struct { + field string + names []string + }{ + {"syscallPolicy.allow", tc.SyscallPolicy.Allow}, + {"syscallPolicy.deny", tc.SyscallPolicy.Deny}, + {"syscallPolicy.trace", tc.SyscallPolicy.Trace}, + {"syscallPolicy.audit", tc.SyscallPolicy.Audit}, + } { + if err := validateSyscallNames(check.field, check.names); err != nil { + return err + } + } + + return validateSyscallPolicyOwnership(tc.effectiveSyscallPolicy()) +} + +func (tc *TaskConfig) compileSyscallPolicy() (compiledSyscallPolicy, error) { + if err := tc.validateSyscallPolicy(); err != nil { + return compiledSyscallPolicy{}, err + } + return tc.effectiveSyscallPolicy().compile(), nil +} + +func (policy EffectiveSyscallPolicy) compile() compiledSyscallPolicy { + seccompAllowed := make([]string, 0, len(policy.Allow)+len(postSeccompStartupCalls)) + seccompAllowed = append(seccompAllowed, policy.seccompAllowedCalls()...) + seccompAllowed = append(seccompAllowed, postSeccompStartupCalls...) + + return compiledSyscallPolicy{ + Ptrace: callPolicySpec{ + OneTimeCalls: append([]string(nil), policy.OneTime...), + AllowedCalls: policy.ptraceAllowedCalls(), + AuditCalls: append([]string(nil), policy.Audit...), + }, + SeccompAllowedCalls: uniqueSyscallNames(seccompAllowed), + SeccompTracedCalls: policy.seccompTracedCalls(), + } +} + +func (policy EffectiveSyscallPolicy) ptraceAllowedCalls() []string { + allowed := make([]string, 0, len(policy.Allow)+len(policy.Trace)+len(policy.Audit)) + allowed = append(allowed, policy.Allow...) + allowed = append(allowed, policy.Trace...) + allowed = append(allowed, policy.Audit...) + return uniqueSyscallNames(allowed) +} + +func (policy EffectiveSyscallPolicy) seccompAllowedCalls() []string { + return append([]string(nil), policy.Allow...) +} + +func (policy EffectiveSyscallPolicy) seccompTracedCalls() []string { + traced := make([]string, 0, len(policy.OneTime)+len(policy.Trace)+len(policy.Audit)) + traced = append(traced, policy.OneTime...) + traced = append(traced, policy.Trace...) + traced = append(traced, policy.Audit...) + return uniqueSyscallNames(traced) +} + +func validateSyscallPolicyOwnership(policy EffectiveSyscallPolicy) error { + ownerByName := map[string]string{} + for _, category := range []struct { + name string + calls []string + }{ + {"oneTimeCalls", policy.OneTime}, + {"runtime allowlist", policy.Allow}, + {"syscallPolicy.trace", policy.Trace}, + {"syscallPolicy.audit", policy.Audit}, + } { + for _, call := range category.calls { + if call == "" { + continue + } + owner, exists := ownerByName[call] + if exists { + return fmt.Errorf("syscall %q is assigned to both %s and %s", call, owner, category.name) + } + ownerByName[call] = category.name + } + } + + for _, deniedCall := range policy.Deny { + if owner, exists := ownerByName[deniedCall]; exists && owner != "runtime allowlist" { + return fmt.Errorf("syscall %q cannot be both denied and %s", deniedCall, owner) + } + } + return nil +} + +func validateHybridSyscallPolicy(policy EffectiveSyscallPolicy) error { + for _, check := range []struct { + field string + calls []string + }{ + {"oneTimeCalls", policy.OneTime}, + {"syscallPolicy.deny", policy.Deny}, + {"syscallPolicy.trace", policy.Trace}, + {"syscallPolicy.audit", policy.Audit}, + } { + for _, call := range check.calls { + if containsSyscallName(postSeccompStartupCalls, call) { + return fmt.Errorf("%s cannot include %q: syscall is reserved for hybrid startup protocol", check.field, call) + } + } + } + return nil +} + +func uniqueSyscallNames(names []string) []string { + seen := make(map[string]struct{}, len(names)) + unique := make([]string, 0, len(names)) + for _, name := range names { + if name == "" { + continue + } + if _, exists := seen[name]; exists { + continue + } + seen[name] = struct{}{} + unique = append(unique, name) + } + return unique +} + +func removeSyscallNames(names, remove []string) []string { + if len(remove) == 0 { + return append([]string(nil), names...) + } + + removeSet := make(map[string]struct{}, len(remove)) + for _, name := range remove { + removeSet[name] = struct{}{} + } + + filtered := make([]string, 0, len(names)) + for _, name := range names { + if _, shouldRemove := removeSet[name]; shouldRemove { + continue + } + filtered = append(filtered, name) + } + return filtered +} + +func containsSyscallName(names []string, target string) bool { + for _, name := range names { + if name == target { + return true + } + } + return false +} diff --git a/runner/syscall_policy_test.go b/runner/syscall_policy_test.go new file mode 100644 index 0000000..0160bc7 --- /dev/null +++ b/runner/syscall_policy_test.go @@ -0,0 +1,348 @@ +package runner + +import ( + "encoding/json" + "strings" + "testing" +) + +func TestSyscallPolicyConfigJSONContractUsesDocumentedFieldNames(t *testing.T) { + policy := SyscallPolicyConfig{ + Allow: []string{"read"}, + Deny: []string{"ptrace"}, + Trace: []string{"clone3"}, + Audit: []string{"getpid"}, + } + + data, err := json.Marshal(policy) + if err != nil { + t.Fatalf("json.Marshal(SyscallPolicyConfig) error = %v", err) + } + want := `{"Allow":["read"],"Deny":["ptrace"],"Trace":["clone3"],"Audit":["getpid"]}` + if string(data) != want { + t.Fatalf("json.Marshal(SyscallPolicyConfig) = %s, want %s", data, want) + } + + var decoded SyscallPolicyConfig + if err := json.Unmarshal(data, &decoded); err != nil { + t.Fatalf("json.Unmarshal(SyscallPolicyConfig) error = %v", err) + } + assertStringSliceEqual(t, decoded.Allow, policy.Allow) + assertStringSliceEqual(t, decoded.Deny, policy.Deny) + assertStringSliceEqual(t, decoded.Trace, policy.Trace) + assertStringSliceEqual(t, decoded.Audit, policy.Audit) +} + +func TestEffectiveSyscallPolicyMergesLegacyAndStructuredPolicy(t *testing.T) { + cfg := &TaskConfig{ + OneTimeCalls: []string{"execve", "execve"}, + AllowedCalls: []string{"read", "write"}, + AdditionCalls: []string{"mmap", "read"}, + SyscallPolicy: SyscallPolicyConfig{ + Allow: []string{"fstat", "write"}, + Deny: []string{"ptrace"}, + Trace: []string{"clone3"}, + Audit: []string{"getpid"}, + }, + } + + policy := cfg.effectiveSyscallPolicy() + + assertStringSliceEqual(t, policy.OneTime, []string{"execve"}) + assertStringSliceEqual(t, policy.Allow, []string{"read", "write", "mmap", "fstat"}) + assertStringSliceEqual(t, policy.Deny, []string{"ptrace"}) + assertStringSliceEqual(t, policy.Trace, []string{"clone3"}) + assertStringSliceEqual(t, policy.Audit, []string{"getpid"}) + assertStringSliceEqual(t, policy.ptraceAllowedCalls(), []string{"read", "write", "mmap", "fstat", "clone3", "getpid"}) + assertStringSliceEqual(t, policy.seccompAllowedCalls(), []string{"read", "write", "mmap", "fstat"}) + assertStringSliceEqual(t, policy.seccompTracedCalls(), []string{"execve", "clone3", "getpid"}) +} + +func TestEffectiveSyscallPolicyDenyOverridesLegacyAndStructuredAllow(t *testing.T) { + cfg := &TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read", "write"}, + AdditionCalls: []string{"mmap"}, + SyscallPolicy: SyscallPolicyConfig{ + Allow: []string{"fstat"}, + Deny: []string{"read", "fstat"}, + }, + } + + policy := cfg.effectiveSyscallPolicy() + + assertStringSliceEqual(t, policy.Allow, []string{"write", "mmap"}) + assertStringSliceEqual(t, policy.Deny, []string{"read", "fstat"}) + assertStringSliceEqual(t, policy.ptraceAllowedCalls(), []string{"write", "mmap"}) + assertStringSliceEqual(t, policy.seccompAllowedCalls(), []string{"write", "mmap"}) + if err := cfg.Validate(); err != nil { + t.Fatalf("Validate() error = %v, want Deny to override allow rules", err) + } +} + +func TestCompileSyscallPolicyProducesBackendInputs(t *testing.T) { + cfg := &TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read", "write"}, + AdditionCalls: []string{"mmap"}, + SyscallPolicy: SyscallPolicyConfig{ + Allow: []string{"fstat"}, + Deny: []string{"read", "fstat"}, + Trace: []string{"clone3"}, + Audit: []string{"getpid"}, + }, + } + + policy, err := cfg.compileSyscallPolicy() + if err != nil { + t.Fatalf("compileSyscallPolicy() error = %v", err) + } + + assertStringSliceEqual(t, policy.Ptrace.OneTimeCalls, []string{"execve"}) + assertStringSliceEqual(t, policy.Ptrace.AllowedCalls, []string{"write", "mmap", "clone3", "getpid"}) + assertStringSliceEqual(t, policy.Ptrace.AuditCalls, []string{"getpid"}) + assertStringSliceEqual(t, policy.SeccompAllowedCalls, []string{"write", "mmap", "close", "exit", "exit_group"}) + assertStringSliceEqual(t, policy.SeccompTracedCalls, []string{"execve", "clone3", "getpid"}) +} + +func TestValidateRejectsSyscallPolicyOwnershipOverlap(t *testing.T) { + tests := []struct { + name string + config TaskConfig + wantMsg string + }{ + { + name: "one-time overlaps structured allow", + config: TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + SyscallBackend: syscallBackendPtrace, + SyscallPolicy: SyscallPolicyConfig{Allow: []string{"execve"}}, + }, + wantMsg: `syscall "execve" is assigned to both oneTimeCalls and runtime allowlist`, + }, + { + name: "legacy one-time overlaps allowed", + config: TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read", "execve"}, + SyscallBackend: syscallBackendPtrace, + }, + wantMsg: `syscall "execve" is assigned to both oneTimeCalls and runtime allowlist`, + }, + { + name: "legacy one-time overlaps addition", + config: TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + AdditionCalls: []string{"execve"}, + SyscallBackend: syscallBackendPtrace, + }, + wantMsg: `syscall "execve" is assigned to both oneTimeCalls and runtime allowlist`, + }, + { + name: "deny overlaps audit", + config: TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + SyscallBackend: syscallBackendPtrace, + SyscallPolicy: SyscallPolicyConfig{ + Audit: []string{"getpid"}, + Deny: []string{"getpid"}, + }, + }, + wantMsg: `syscall "getpid" cannot be both denied and syscallPolicy.audit`, + }, + { + name: "trace overlaps allow", + config: TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + SyscallBackend: syscallBackendPtrace, + SyscallPolicy: SyscallPolicyConfig{Trace: []string{"read"}}, + }, + wantMsg: `syscall "read" is assigned to both runtime allowlist and syscallPolicy.trace`, + }, + { + name: "audit overlaps trace", + config: TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + SyscallBackend: syscallBackendPtrace, + SyscallPolicy: SyscallPolicyConfig{ + Trace: []string{"clone3"}, + Audit: []string{"clone3"}, + }, + }, + wantMsg: `syscall "clone3" is assigned to both syscallPolicy.trace and syscallPolicy.audit`, + }, + { + name: "one-time overlaps trace", + config: TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + SyscallBackend: syscallBackendPtrace, + SyscallPolicy: SyscallPolicyConfig{Trace: []string{"execve"}}, + }, + wantMsg: `syscall "execve" is assigned to both oneTimeCalls and syscallPolicy.trace`, + }, + { + name: "one-time overlaps deny", + config: TaskConfig{ + CPU: 1, + Memory: 1, + Output: 1, + Stack: 1, + MaxProcs: 1, + RunUID: -1, + RunGID: -1, + NoNewPrivs: true, + OneTimeCalls: []string{"execve"}, + AllowedCalls: []string{"read"}, + SyscallBackend: syscallBackendPtrace, + SyscallPolicy: SyscallPolicyConfig{Deny: []string{"execve"}}, + }, + wantMsg: `syscall "execve" cannot be both denied and oneTimeCalls`, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := tt.config.Validate() + if err == nil { + t.Fatal("Validate() error = nil, want ownership overlap rejection") + } + if !strings.Contains(err.Error(), tt.wantMsg) { + t.Fatalf("Validate() error = %q, want %q", err, tt.wantMsg) + } + }) + } +} + +func TestValidateHybridSyscallPolicyRejectsReservedStartupProtocolCalls(t *testing.T) { + tests := []struct { + name string + policy EffectiveSyscallPolicy + wantMsg string + }{ + { + name: "deny", + policy: EffectiveSyscallPolicy{Deny: []string{"write"}}, + wantMsg: `syscallPolicy.deny cannot include "write"`, + }, + { + name: "trace", + policy: EffectiveSyscallPolicy{Trace: []string{"close"}}, + wantMsg: `syscallPolicy.trace cannot include "close"`, + }, + { + name: "audit", + policy: EffectiveSyscallPolicy{Audit: []string{"exit"}}, + wantMsg: `syscallPolicy.audit cannot include "exit"`, + }, + { + name: "one-time", + policy: EffectiveSyscallPolicy{OneTime: []string{"exit_group"}}, + wantMsg: `oneTimeCalls cannot include "exit_group"`, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := validateHybridSyscallPolicy(tt.policy) + if err == nil { + t.Fatal("validateHybridSyscallPolicy() error = nil, want startup protocol rejection") + } + if !strings.Contains(err.Error(), tt.wantMsg) { + t.Fatalf("validateHybridSyscallPolicy() error = %q, want %q", err, tt.wantMsg) + } + if !strings.Contains(err.Error(), "hybrid startup protocol") { + t.Fatalf("validateHybridSyscallPolicy() error = %q, want startup protocol context", err) + } + }) + } +} + +func assertStringSliceEqual(t *testing.T, got, want []string) { + t.Helper() + if len(got) != len(want) { + t.Fatalf("slice = %v, want %v", got, want) + } + for i := range want { + if got[i] != want[i] { + t.Fatalf("slice[%d] = %q, want %q; full=%v", i, got[i], want[i], got) + } + } +} diff --git a/runner/tracer.go b/runner/tracer.go index 958935b..2e8b737 100644 --- a/runner/tracer.go +++ b/runner/tracer.go @@ -12,7 +12,7 @@ const ( ) type traceeState struct { - inSyscall bool + inSyscall bool //nolint:unused // accessed in tracer_linux.go prevSyscall uint64 //nolint:unused // accessed in tracer_linux.go pendingAttachStop bool } diff --git a/runner/tracer_darwin.go b/runner/tracer_darwin.go index b1fa257..ae32b2a 100644 --- a/runner/tracer_darwin.go +++ b/runner/tracer_darwin.go @@ -5,3 +5,7 @@ package runner func (tracer *TracerDetect) checkSyscall(pid int) syscallCheckResult { panic("checkSyscall is not supported on darwin") } + +func (tracer *TracerDetect) checkSeccompTrace(pid int) syscallCheckResult { + panic("checkSeccompTrace is not supported on darwin") +} diff --git a/runner/tracer_linux.go b/runner/tracer_linux.go index 2bf0a7f..5ec522b 100644 --- a/runner/tracer_linux.go +++ b/runner/tracer_linux.go @@ -56,6 +56,7 @@ func (tracer *TracerDetect) checkSyscall(pid int) syscallCheckResult { log.Infof("not allowed syscall %d: %16v ", callID, getName(callID)) return syscallCheckViolation } + tracer.auditSyscall(pid, callID, "ptrace") if callID != syscall.SYS_WRITE && callID != syscall.SYS_READ { log.Info(getName(callID)) } @@ -74,3 +75,38 @@ func (tracer *TracerDetect) checkSyscall(pid int) syscallCheckResult { tracer.setInSyscall(pid, !tracer.inSyscall(pid)) return syscallCheckOK } + +func (tracer *TracerDetect) checkSeccompTrace(pid int) syscallCheckResult { + _, ok := tracer.getTracee(pid) + if !ok { + log.Warnf("checkSeccompTrace called for unregistered pid %d", pid) + return syscallCheckTraceeGone + } + + var regs syscall.PtraceRegs + err := ptraceGetRegs(pid, ®s) + if err != nil { + if errors.Is(err, syscall.ESRCH) { + log.Debugf("tracee pid=%d disappeared before seccomp trace regs fetch", pid) + return syscallCheckTraceeGone + } + log.Debugf("seccomp trace regs failed for pid=%d: %v", pid, err) + return syscallCheckTracerError + } + + callID := getSyscallNumber(®s) + log.Debugf("seccomp trace event pid=%d syscall=%s(%d)", pid, getName(callID), callID) + if !tracer.callPolicy.CheckID(callID) { + log.Infof("not allowed seccomp-traced syscall %d: %16v ", callID, getName(callID)) + return syscallCheckViolation + } + tracer.auditSyscall(pid, callID, "seccomp") + return syscallCheckOK +} + +func (tracer *TracerDetect) auditSyscall(pid int, callID uint64, source string) { + if tracer.callPolicy == nil || !tracer.callPolicy.ShouldAudit(callID) || log == nil { + return + } + log.Infof("audit syscall source=%s pid=%d syscall=%s(%d)", source, pid, getName(callID), callID) +} diff --git a/runner/tracer_linux_test.go b/runner/tracer_linux_test.go index 7c82874..65d7246 100644 --- a/runner/tracer_linux_test.go +++ b/runner/tracer_linux_test.go @@ -7,6 +7,8 @@ import ( "testing" "github.com/stretchr/testify/assert" + "go.uber.org/zap" + "go.uber.org/zap/zaptest/observer" ) func TestGetNameReturnsFallbackForUnknownSyscall(t *testing.T) { @@ -14,11 +16,9 @@ func TestGetNameReturnsFallbackForUnknownSyscall(t *testing.T) { } func TestConsumeBootstrapCallConsumesExecveQuota(t *testing.T) { - oneTimeCalls := []string{"execve"} - allowedCalls := []string{} tracer := &TracerDetect{} - policy, err := makeCallPolicy(&oneTimeCalls, &allowedCalls) + policy, err := makeCallPolicy(callPolicySpec{OneTimeCalls: []string{"execve"}}) assert.NoError(t, err) tracer.setCallPolicy(policy) tracer.consumeBootstrapCall(syscall.SYS_EXECVE) @@ -78,3 +78,135 @@ func TestCheckSyscallTreatsUnexpectedPtraceErrorsAsTracerError(t *testing.T) { assert.Equal(t, syscallCheckTracerError, tracer.checkSyscall(100)) } + +func TestCheckSeccompTraceRejectsConsumedExecveBeforeEnter(t *testing.T) { + tracer := &TracerDetect{} + tracer.RegisterTracee(100, false) + + policy, err := makeCallPolicy(callPolicySpec{OneTimeCalls: []string{"execve"}}) + assert.NoError(t, err) + tracer.setCallPolicy(policy) + tracer.consumeBootstrapCall(syscall.SYS_EXECVE) + + original := ptraceGetRegs + ptraceGetRegs = func(pid int, regs *syscall.PtraceRegs) error { + assert.Equal(t, 100, pid) + setTestSyscallNumber(regs, uint64(syscall.SYS_EXECVE)) + return nil + } + t.Cleanup(func() { + ptraceGetRegs = original + }) + + assert.Equal(t, syscallCheckViolation, tracer.checkSeccompTrace(100)) +} + +func TestCheckSeccompTraceConsumesPolicyAtEventStop(t *testing.T) { + tracer := &TracerDetect{} + tracer.RegisterTracee(100, false) + + policy, err := makeCallPolicy(callPolicySpec{OneTimeCalls: []string{"getpid"}}) + assert.NoError(t, err) + tracer.setCallPolicy(policy) + + original := ptraceGetRegs + ptraceGetRegs = func(pid int, regs *syscall.PtraceRegs) error { + assert.Equal(t, 100, pid) + setTestSyscallNumber(regs, uint64(syscall.SYS_GETPID)) + return nil + } + t.Cleanup(func() { + ptraceGetRegs = original + }) + + assert.Equal(t, syscallCheckOK, tracer.checkSeccompTrace(100)) + assert.False(t, tracer.callPolicy.CheckID(uint64(syscall.SYS_GETPID))) +} + +func TestCheckSyscallLogsAuditCall(t *testing.T) { + observedLogs := useObservedLogger(t) + tracer := &TracerDetect{} + tracer.RegisterTracee(100, false) + + policy, err := makeCallPolicy(callPolicySpec{ + AllowedCalls: []string{"getpid"}, + AuditCalls: []string{"getpid"}, + }) + assert.NoError(t, err) + tracer.setCallPolicy(policy) + + original := ptraceGetRegs + ptraceGetRegs = func(pid int, regs *syscall.PtraceRegs) error { + assert.Equal(t, 100, pid) + setTestSyscallNumber(regs, uint64(syscall.SYS_GETPID)) + return nil + } + t.Cleanup(func() { + ptraceGetRegs = original + }) + + assert.Equal(t, syscallCheckOK, tracer.checkSyscall(100)) + assert.Equal(t, 1, observedLogs.FilterMessageSnippet("audit syscall source=ptrace pid=100 syscall=getpid").Len()) +} + +func TestCheckSeccompTraceLogsAuditCall(t *testing.T) { + observedLogs := useObservedLogger(t) + tracer := &TracerDetect{} + tracer.RegisterTracee(100, false) + + policy, err := makeCallPolicy(callPolicySpec{ + AllowedCalls: []string{"getpid"}, + AuditCalls: []string{"getpid"}, + }) + assert.NoError(t, err) + tracer.setCallPolicy(policy) + + original := ptraceGetRegs + ptraceGetRegs = func(pid int, regs *syscall.PtraceRegs) error { + assert.Equal(t, 100, pid) + setTestSyscallNumber(regs, uint64(syscall.SYS_GETPID)) + return nil + } + t.Cleanup(func() { + ptraceGetRegs = original + }) + + assert.Equal(t, syscallCheckOK, tracer.checkSeccompTrace(100)) + assert.Equal(t, 1, observedLogs.FilterMessageSnippet("audit syscall source=seccomp pid=100 syscall=getpid").Len()) +} + +func TestCheckSeccompTraceChecksPolicyRegardlessOfSyscallPhase(t *testing.T) { + tracer := &TracerDetect{} + tracer.RegisterTracee(100, false) + tracer.setInSyscall(100, true) + + policy, err := makeCallPolicy(callPolicySpec{}) + assert.NoError(t, err) + tracer.setCallPolicy(policy) + + original := ptraceGetRegs + ptraceGetRegs = func(pid int, regs *syscall.PtraceRegs) error { + assert.Equal(t, 100, pid) + setTestSyscallNumber(regs, uint64(syscall.SYS_GETPID)) + return nil + } + t.Cleanup(func() { + ptraceGetRegs = original + }) + + assert.Equal(t, syscallCheckViolation, tracer.checkSeccompTrace(100)) + assert.True(t, tracer.inSyscall(100)) +} + +func useObservedLogger(t *testing.T) *observer.ObservedLogs { + t.Helper() + + previousLog := log + core, observedLogs := observer.New(zap.DebugLevel) + SetLogger(zap.New(core).Sugar()) + t.Cleanup(func() { + SetLogger(previousLog) + }) + + return observedLogs +} diff --git a/runner/tracer_regs_linux_amd64_test.go b/runner/tracer_regs_linux_amd64_test.go new file mode 100644 index 0000000..171f187 --- /dev/null +++ b/runner/tracer_regs_linux_amd64_test.go @@ -0,0 +1,9 @@ +//go:build linux && amd64 + +package runner + +import "syscall" + +func setTestSyscallNumber(regs *syscall.PtraceRegs, sysno uint64) { + regs.Orig_rax = sysno +} diff --git a/runner/tracer_regs_linux_arm64_test.go b/runner/tracer_regs_linux_arm64_test.go new file mode 100644 index 0000000..c354ae2 --- /dev/null +++ b/runner/tracer_regs_linux_arm64_test.go @@ -0,0 +1,9 @@ +//go:build linux && arm64 + +package runner + +import "syscall" + +func setTestSyscallNumber(regs *syscall.PtraceRegs, sysno uint64) { + regs.Regs[8] = sysno +} diff --git a/tests/README.md b/tests/README.md index 205dd78..235aa8a 100644 --- a/tests/README.md +++ b/tests/README.md @@ -15,7 +15,7 @@ - 内存限制:`mle/`、`mle2/`、`mle21/`、`mle3/` - 时间限制:`tle/`、`tle2/`、`java-tle/` - 运行异常:`segmentfault/`、`sigtrap/`、`zero/` -- 系统调用与环境限制:`clone-syscall-phase/`、`socket/`、`stack/` +- 系统调用与环境限制:`clone-syscall-phase/`、`clone-syscall-phase-hybrid/`、`socket/`、`stack/`、`syscall-policy-deny-hybrid/`、`syscall-policy-trace-audit-hybrid/` - 输出限制:`ole/` - Java 内存场景:`java-mle/` diff --git a/tests/clone-syscall-phase-hybrid/case.json b/tests/clone-syscall-phase-hybrid/case.json new file mode 100644 index 0000000..5085a13 --- /dev/null +++ b/tests/clone-syscall-phase-hybrid/case.json @@ -0,0 +1,9 @@ +{ + "Name": "Hybrid clone event should not reset syscall phase", + "verbose": true, + "CPU": 1, + "Memory": 16, + "Result": 10, + "SyscallBackend": "hybrid", + "AdditionCalls": ["clone"] +} diff --git a/tests/clone-syscall-phase-hybrid/main.c b/tests/clone-syscall-phase-hybrid/main.c new file mode 100644 index 0000000..428e337 --- /dev/null +++ b/tests/clone-syscall-phase-hybrid/main.c @@ -0,0 +1,20 @@ +#define _GNU_SOURCE + +#include +#include +#include +#include + +int main(void) +{ + long pid = syscall(SYS_clone, SIGCHLD, 0, 0, 0, 0); + if (pid == 0) { + _exit(0); + } + if (pid < 0) { + return 2; + } + + rename("sentinel.source", "sentinel.dest"); + return 0; +} diff --git a/tests/clone-syscall-phase-hybrid/makefile b/tests/clone-syscall-phase-hybrid/makefile new file mode 100644 index 0000000..69a240f --- /dev/null +++ b/tests/clone-syscall-phase-hybrid/makefile @@ -0,0 +1,16 @@ +default: compile test assert clean + +compile: + @rm -f sentinel.dest + @touch sentinel.source + gcc main.c -o main -static + +test: + ../../bin/test + +assert: + @test -e sentinel.source + @test ! -e sentinel.dest + +clean: + @rm -f main user.err user.in user.out sentinel.source sentinel.dest diff --git a/tests/syscall-policy-deny-hybrid/case.json b/tests/syscall-policy-deny-hybrid/case.json new file mode 100644 index 0000000..68864a5 --- /dev/null +++ b/tests/syscall-policy-deny-hybrid/case.json @@ -0,0 +1,12 @@ +{ + "Name": "Hybrid syscall policy deny overrides inherited allow", + "verbose": true, + "CPU": 1, + "Memory": 16, + "Result": 10, + "SyscallBackend": "hybrid", + "AdditionCalls": ["getpid"], + "SyscallPolicy": { + "Deny": ["getpid"] + } +} diff --git a/tests/syscall-policy-deny-hybrid/main.c b/tests/syscall-policy-deny-hybrid/main.c new file mode 100644 index 0000000..1d2be4e --- /dev/null +++ b/tests/syscall-policy-deny-hybrid/main.c @@ -0,0 +1,10 @@ +#define _GNU_SOURCE + +#include +#include + +int main(void) +{ + syscall(SYS_getpid); + return 0; +} diff --git a/tests/syscall-policy-deny-hybrid/makefile b/tests/syscall-policy-deny-hybrid/makefile new file mode 100644 index 0000000..2d5f710 --- /dev/null +++ b/tests/syscall-policy-deny-hybrid/makefile @@ -0,0 +1,13 @@ +default: compile test assert clean + +compile: + gcc main.c -o main -static + +test: + @../../bin/test > test.log + +assert: + @grep -q "passed!" test.log + +clean: + @rm -f main user.err user.in user.out test.log diff --git a/tests/syscall-policy-trace-audit-hybrid/case.json b/tests/syscall-policy-trace-audit-hybrid/case.json new file mode 100644 index 0000000..8ec8e3a --- /dev/null +++ b/tests/syscall-policy-trace-audit-hybrid/case.json @@ -0,0 +1,13 @@ +{ + "Name": "Hybrid syscall policy trace and audit", + "verbose": true, + "CPU": 1, + "Memory": 16, + "Result": 4, + "SyscallBackend": "hybrid", + "LogPath": "runner.log", + "SyscallPolicy": { + "Trace": ["getpid"], + "Audit": ["getppid"] + } +} diff --git a/tests/syscall-policy-trace-audit-hybrid/main.c b/tests/syscall-policy-trace-audit-hybrid/main.c new file mode 100644 index 0000000..7e719fe --- /dev/null +++ b/tests/syscall-policy-trace-audit-hybrid/main.c @@ -0,0 +1,14 @@ +#define _GNU_SOURCE + +#include +#include + +int main(void) +{ + long pid = syscall(SYS_getpid); + long ppid = syscall(SYS_getppid); + if (pid <= 0 || ppid <= 0) { + return 1; + } + return 0; +} diff --git a/tests/syscall-policy-trace-audit-hybrid/makefile b/tests/syscall-policy-trace-audit-hybrid/makefile new file mode 100644 index 0000000..b7eca35 --- /dev/null +++ b/tests/syscall-policy-trace-audit-hybrid/makefile @@ -0,0 +1,15 @@ +default: compile test assert clean + +compile: + @rm -f runner.log test.log + gcc main.c -o main -static + +test: + @../../bin/test > test.log + +assert: + @grep -q "passed!" test.log + @grep -q "audit syscall source=seccomp.*getppid" runner.log + +clean: + @rm -f main user.err user.in user.out runner.log test.log