diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9f5e3d55..5fa3736f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -380,6 +380,18 @@ jobs: echo "healthz ok, version ${version}" docker rm -f fledge-smoke >/dev/null 2>&1 || true cleaned=1 + # #109: the FlyBot boundary is proven from inside a real container, with the + # product's own run arguments, in the one job that has a Docker daemon. The + # node test job has none, so `npm test` keeps the argument assertions and + # this step carries the container-level proof. A run without a daemon exits + # 3 instead of passing, as the check itself does everywhere else. + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 + with: + node-version: 22 + cache: npm + - run: npm ci + - name: Prove the FlyBot container boundary from inside a container + run: npx tsx scripts/isolation-check.ts --image fledge:ci audit: name: audit diff --git a/changelog.d/109-flybot-isolation.security.md b/changelog.d/109-flybot-isolation.security.md new file mode 100644 index 00000000..dd464b76 --- /dev/null +++ b/changelog.d/109-flybot-isolation.security.md @@ -0,0 +1 @@ +A per-repository FlyBot container now starts with a read-only root, a single writable tmpfs at `/tmp`, every Linux capability dropped, `no-new-privileges`, a non-root user and the existing CPU, memory and PID ceilings, and the host Docker socket is never mounted into it. A new isolation check starts a throwaway container with exactly those arguments and asserts each fact from inside it — the root refuses a write, the effective capability set is empty, a seccomp filter is loaded, the socket is absent, only the repository's own data directory is a host bind, and a container on another tenant's network is unreachable — and exits 3 where no Docker daemon exists instead of reporting a pass. The builder also refuses a data directory that is not an absolute existing directory, so the one host path a tenant container sees cannot be a relative path the daemon resolves elsewhere or a path Docker would create as root. Egress is unchanged in this slice. diff --git a/docs/deploy.md b/docs/deploy.md index eb77c5be..427915c5 100644 --- a/docs/deploy.md +++ b/docs/deploy.md @@ -175,6 +175,36 @@ long runs; lowering it makes that failure more likely, so change it together with the agent's own time budget. `JOB_STALL_MINUTES` (default 30) is the separate no-heartbeat stall sweep; a heartbeat moves only that one. +### Per-repository FlyBot containers + +Each hosted repository gets its own container, started from +`flybotContainerArgs()` in `src/flybot/isolation.ts`. The flags are the boundary, +so they are not configurable per repository: `--read-only` with one `/tmp` tmpfs +(`rw,noexec,nosuid,size=64m`), a single bind of that repository's data directory +at `/data`, `--cap-drop ALL`, `--security-opt no-new-privileges`, a non-root +`--user`, and the CPU, memory and PID ceilings. `NARB_DISABLE_NATIVE_CACHE=1` is +not optional: the DSH native-addon loader copies a binary into the temp directory +and executes it, so a `noexec` `/tmp` otherwise stops the web profile from +starting. + +Those flags reach production through the normal deploy and take effect the next +time an instance is provisioned or restarted. Before shipping, prove the boundary +on a host that runs the containers: + +```sh +npm run isolation:check # node:22-bookworm-slim probe +npm run isolation:check -- --image +``` + +The check starts its own throwaway container with exactly those arguments, then +asserts from inside it that the root refuses a write, the effective capability set +is empty, a seccomp filter is loaded, the Docker socket is absent, only the +repository data directory is a host bind, and a container on another tenant's +network is unreachable. It exits 3 where no Docker daemon exists rather than +reporting a pass. Egress is not restricted yet: a fresh data volume installs its +profile from `registry.npmmirror.com` on first start, so deny-by-default egress +with an allowlist is still open work and not something a deploy changes today. + ## 5. Run ### Docker diff --git a/docs/security.md b/docs/security.md index 2a36d13d..25257ff3 100644 --- a/docs/security.md +++ b/docs/security.md @@ -36,6 +36,46 @@ operator FlyBot is not used for user jobs. A container is limited by CPU, memory, PID and filesystem mounts; stopping or deleting a repository affects only that container. +The container's run arguments are a security boundary, so they are built in one +place (`flybotContainerArgs` in `src/flybot/isolation.ts`) and nothing else adds +a mount or a network: + +- **Read-only root.** `--read-only` plus one tmpfs at `/tmp` + (`rw,noexec,nosuid,size=64m`). Only the repository's own data directory is a + host bind; the container cannot rewrite the image the next job starts from. +- **No capabilities.** `--cap-drop ALL` with `--security-opt no-new-privileges` + and a non-root `--user`, so a process inside cannot gain one back through a + setuid binary. +- **No host Docker socket.** The arguments never carry `/var/run/docker.sock`; + the socket is root on the host, so mounting it into a tenant container would + make every other tenant reachable. +- **One checked mount source.** The data directory must be an absolute path to an + existing directory before it becomes a bind: the builder throws otherwise, + because a relative path is resolved by the daemon against its own working + directory and a missing one is created by Docker as root — either way the mount + would come from outside the tenant's tree. +- **Seccomp stays on.** The runtime's default profile is what refuses the rare + syscalls an agent runtime never needs; the isolation check asserts that a + filter really is loaded (`Seccomp: 2`) rather than replacing the profile with a + weaker hand-written one. +- **PID, CPU and memory ceilings** stay as they were, which is what keeps one + tenant from starving the host. + +`npm run isolation:check` starts a throwaway container with exactly those +arguments and asserts each fact from inside it: the root refuses a write, the +effective capability set is empty, a seccomp filter is loaded, the Docker socket +is absent, exactly one host bind is mounted, and a container on another tenant's +network cannot be reached. It exits 3 where no Docker daemon exists, because a +silent pass would hide the boundary it exists to test. CI runs it in the `docker` +job; a developer can run it on any host with a daemon. + +Egress is not restricted yet: the container still reaches the outside network, +because a fresh data volume installs its profile from `registry.npmmirror.com` on +first start. Deny-by-default egress with an allowlist is the remaining close +condition, and it needs a mechanism decision (an egress proxy on an internal +network, or a warm-store seal before the long-lived container starts) before it +can land. + ### API tokens - A personal `flg_` token is a gateway capability with two properties: a `scope` diff --git a/package.json b/package.json index 2925ff57..a6280c55 100644 --- a/package.json +++ b/package.json @@ -37,6 +37,7 @@ "mock:snapshot": "tsx scripts/mock-snapshot.ts", "mock:check": "tsx scripts/mock-snapshot.ts --check", "evolution:consent": "tsx scripts/evolution-consent.ts", + "isolation:check": "tsx scripts/isolation-check.ts", "lint:update": "node scripts/lint-warnings.mjs --update" }, "dependencies": { diff --git a/scripts/isolation-check.ts b/scripts/isolation-check.ts new file mode 100644 index 00000000..1fab2564 --- /dev/null +++ b/scripts/isolation-check.ts @@ -0,0 +1,165 @@ +// #109: the isolation boundary is proven from inside a real container that is +// started with exactly the flags the product starts, never by reading flags. +// This check needs a Docker daemon; where there is none it exits 3 with a loud +// message, because a silent pass would hide the very boundary it exists to test. +import { execFile } from 'node:child_process' +import { mkdtempSync, rmSync } from 'node:fs' +import { createServer } from 'node:net' +import { join } from 'node:path' +import { promisify } from 'node:util' +import { flybotContainerArgs } from '../src/flybot/isolation.js' + +const execFileAsync = promisify(execFile) + +const NO_DAEMON = 3 +const BOUNDARY_BROKEN = 1 + +interface Options { + docker: string + image: string +} + +function parseOptions(argv: string[]): Options { + const options: Options = { docker: 'docker', image: 'node:22-bookworm-slim' } + for (let index = 0; index < argv.length; index += 1) { + const flag = argv[index] + const value = argv[index + 1] + if (flag === '--docker' && value !== undefined) { options.docker = value; index += 1 } + else if (flag === '--image' && value !== undefined) { options.image = value; index += 1 } + else throw new Error(`unknown argument: ${flag ?? ''}`) + } + return options +} + +function docker(options: Options, argv: string[], timeout = 120_000): Promise { + return execFileAsync(options.docker, argv, { timeout, maxBuffer: 8 * 1024 * 1024 }).then((result) => result.stdout.trim()) +} + +async function freePort(): Promise { + return new Promise((resolve, reject) => { + const server = createServer() + server.on('error', reject) + server.listen(0, '127.0.0.1', () => { + const address = server.address() + const port = typeof address === 'object' && address !== null ? address.port : 0 + server.close(() => resolve(port)) + }) + }) +} + +// Runs inside the probe container. It reports what the kernel says about the +// process, not what the flags intended, and it tries one thing it must not be +// able to do: reach a sibling container on another tenant's network. +const PROBE = ` +const fs = require('node:fs') +const net = require('node:net') +const fact = {} +try { fs.writeFileSync('/__isolation_probe', 'x'); fact.rootWritable = true } catch (error) { fact.rootWritable = false; fact.rootError = error.code } +const status = fs.readFileSync('/proc/self/status', 'utf8') +fact.caps = (/^CapEff:\\s*([0-9a-f]+)/m.exec(status) || [])[1] || null +fact.seccomp = (/^Seccomp:\\s*(\\d+)/m.exec(status) || [])[1] || null +fact.dockerSocket = fs.existsSync('/var/run/docker.sock') +fact.dataMounts = fs.readFileSync('/proc/mounts', 'utf8').split('\\n') + .filter((line) => line.includes(' /data ')).map((line) => line.split(' ')[1]) +const sibling = process.argv[1] +const reach = (host, port, timeout) => new Promise((resolve) => { + const socket = net.connect({ host, port }) + const done = (value) => { socket.destroy(); resolve(value) } + socket.setTimeout(timeout, () => done(false)) + socket.once('connect', () => done(true)) + socket.once('error', () => done(false)) +}) +reach(sibling, Number(process.argv[2]), 1500).then((ok) => { fact.siblingReachable = ok; process.stdout.write(JSON.stringify(fact)) }) +` + +interface Fact { + rootWritable: boolean + rootError?: string + caps: string | null + seccomp: string | null + dockerSocket: boolean + dataMounts: string[] + siblingReachable: boolean +} + +async function main(): Promise { + const options = parseOptions(process.argv.slice(2)) + try { + await docker(options, ['info', '--format', '{{.ServerVersion}}'], 30_000) + } catch { + console.error(`isolation-check: no Docker daemon reachable through ${options.docker} — this check cannot pass without one`) + process.exit(NO_DAEMON) + } + + const tag = `${process.pid}-${Math.random().toString(16).slice(2, 8)}` + const network = `fledge-isolation-${tag}` + const siblingNetwork = `fledge-isolation-sibling-${tag}` + const probe = `fledge-isolation-probe-${tag}` + const sibling = `fledge-isolation-sibling-${tag}` + const dataDir = mkdtempSync(join(process.cwd(), '.isolation-check-')) + const failures: string[] = [] + try { + await docker(options, ['network', 'create', network]) + await docker(options, ['network', 'create', siblingNetwork]) + // The sibling stands in for another tenant: its own network, its own + // container, one open port. Nothing a job container does may reach it. + await docker(options, [ + 'run', '-d', '--name', sibling, '--network', siblingNetwork, '--entrypoint', 'node', options.image, + '-e', 'require("node:http").createServer((q, s) => s.end("sibling")).listen(8080)', + ]) + const siblingIp = await docker(options, ['inspect', '-f', '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}', sibling]) + const dshPort = await freePort() + const args = flybotContainerArgs({ + container: probe, + network, + dataDir, + uid: typeof process.getuid === 'function' ? process.getuid() : 1000, + gid: typeof process.getgid === 'function' ? process.getgid() : 1000, + dshPort, + memory: '512m', + cpus: '1', + image: options.image, + modelKey: 'isolation-probe', + command: `node -e '${PROBE.replace(/'/g, "'\\''")}' ${siblingIp} 8080`, + }) + // The product starts the container detached and waits for the web port; the + // check wants the probe's stdout, so it runs the same arguments attached. + const output = await docker(options, ['run', '--rm', ...args.slice(2)], 180_000) + .catch((error: { stdout?: string; stderr?: string }) => error.stdout || error.stderr || '') + + let fact: Fact | null = null + try { fact = JSON.parse(output) as Fact } catch { fact = null } + if (fact === null) { + console.error(`isolation-check: probe produced no facts\n${output}`) + process.exit(BOUNDARY_BROKEN) + } + + const report = (ok: boolean, line: string): void => { + console.log(`isolation-check: ${ok ? 'ok' : 'FAILED'} — ${line}`) + if (!ok) failures.push(line) + } + report(fact.rootWritable === false, `root filesystem is read-only (write failed with ${fact.rootError ?? 'an error'})`) + report(fact.caps === '0000000000000000', `every capability is dropped (CapEff ${fact.caps ?? 'unreadable'})`) + report(fact.seccomp === '2', `a seccomp filter is loaded (Seccomp ${fact.seccomp ?? 'unreadable'})`) + report(fact.dockerSocket === false, 'the host Docker socket is not visible inside the container') + report(fact.dataMounts.length === 1, `exactly one host bind is mounted, the repository's own data directory (${fact.dataMounts.join(', ') || 'none'})`) + report(fact.siblingReachable === false, 'another tenant on another Docker network is unreachable') + + if (failures.length > 0) { + console.error(`isolation-check: ${failures.length} condition(s) failed`) + process.exit(BOUNDARY_BROKEN) + } + console.log('isolation-check: boundary holds') + } finally { + await docker(options, ['rm', '-f', probe], 60_000).catch(() => '') + await docker(options, ['rm', '-f', sibling], 60_000).catch(() => '') + await docker(options, ['network', 'rm', network], 60_000).catch(() => '') + await docker(options, ['network', 'rm', siblingNetwork], 60_000).catch(() => '') + rmSync(dataDir, { recursive: true, force: true }) + } +} + +main().catch((error: unknown) => { + console.error(`isolation-check: ${error instanceof Error ? error.message : String(error)}`) + process.exit(BOUNDARY_BROKEN) +}) diff --git a/src/flybot/instances.ts b/src/flybot/instances.ts index c22c2f7b..e2f2926e 100644 --- a/src/flybot/instances.ts +++ b/src/flybot/instances.ts @@ -9,6 +9,7 @@ import { log } from '../lib/log.js' import { decryptSecret, encryptSecret, randomToken, sha256Hex } from '../lib/secrets.js' import { nowIso } from '../lib/time.js' import { PLAYBOOK_VERSION, TEAM_ROLES, playbookFingerprint, playbookFor } from './playbook.js' +import { flybotContainerArgs } from './isolation.js' import { clearWorkspaceCookie, dshCookie } from './workspace.js' import { readZipText } from '../lib/zip.js' import { parseChiefTranscript, type ChiefMessage } from '../lib/chiefTranscript.js' @@ -302,27 +303,18 @@ server.listen(3081, '0.0.0.0') const gid = typeof process.getgid === 'function' ? process.getgid() : 1000 writeFileSync(join(dataDir, 'dsh-web.log'), '') clearStaleDshLock(dataDir) - await this.docker([ - 'run', '-d', - '--name', container, - '--network', network, - '-p', `127.0.0.1:${dshPort}:3081`, - '-v', `${dataDir}:/data`, - '--user', `${uid}:${gid}`, - '--memory', this.config.flybot.memory, - '--cpus', this.config.flybot.cpus, - '--pids-limit', '512', - '--security-opt', 'no-new-privileges', - '--restart', 'no', - '--entrypoint', '/bin/sh', - '-e', 'HOME=/data', - '-e', 'DSH_HOME=/data', - '-e', 'NPM_CONFIG_REGISTRY=https://registry.npmmirror.com', - '-e', 'npm_config_registry=https://registry.npmmirror.com', - '-e', `FLEDGE_MODEL_KEY=${modelKey}`, - this.config.flybot.instanceImage, - '-c', 'node /opt/flybot/docker/entrypoint.mjs >> /data/dsh-web.log 2>&1 & exec node /data/tcp-proxy.mjs', - ], 180_000) + await this.docker(flybotContainerArgs({ + container, + network, + dataDir, + uid, + gid, + dshPort, + memory: this.config.flybot.memory, + cpus: this.config.flybot.cpus, + image: this.config.flybot.instanceImage, + modelKey, + }), 180_000) await this.waitForDsh(dshPort, row) if (this.store.getRepo(repo.id) === undefined) throw new Error('仓库已删除,停止初始化') diff --git a/src/flybot/isolation.ts b/src/flybot/isolation.ts new file mode 100644 index 00000000..0edac9b7 --- /dev/null +++ b/src/flybot/isolation.ts @@ -0,0 +1,132 @@ +// #109: a per-repository FlyBot is a container on a host shared with every other +// tenant, so its `docker run` arguments are a security boundary. They live here, +// in one function, so the isolation check can start a container with exactly the +// flags the product starts and prove the boundary from the inside. + +import { statSync } from 'node:fs' +import { isAbsolute } from 'node:path' + +export interface FlybotContainerInput { + container: string + network: string + dataDir: string + uid: number + gid: number + dshPort: number + memory: string + cpus: string + image: string + modelKey: string + /** Overrides the entrypoint command; the isolation check runs a probe instead. */ + command?: string +} + +// A read-only root has to name its writable paths. `/tmp` is the process temp +// directory; `noexec` would block the native-addon loader, which copies a binary +// into the temp directory and executes it, so the loader's cache is disabled in +// the environment below instead of leaving `/tmp` executable. +const TMPFS = '/tmp:rw,noexec,nosuid,size=64m' + +// The data directory is the one host path a tenant container sees, so it is +// checked before it becomes a mount. A relative path is resolved by the Docker +// daemon against its own working directory, so it can point at another tenant's +// tree, and a missing one is created by Docker as root, so neither may reach +// `docker run` as written. +function checkedMountSource(dataDir: string): string { + if (!isAbsolute(dataDir)) { + throw new Error(`FlyBot data directory must be absolute, got "${dataDir}"`) + } + let stat + try { + stat = statSync(dataDir) + } catch { + throw new Error(`FlyBot data directory does not exist: ${dataDir}`) + } + if (!stat.isDirectory()) throw new Error(`FlyBot data directory is not a directory: ${dataDir}`) + return dataDir +} + +// The arguments themselves are the boundary, so they are checked once more after +// they are built: a later edit that adds a bind, a capability, a writable root, a +// root user or the host Docker socket fails here in `npm test` instead of only in +// the container-level check, which needs a daemon. +const DATA_MOUNT = '/data' + +export function assertContainerBoundary(args: string[], dataDir: string): string[] { + if (!args.includes('--read-only')) { + throw new Error('FlyBot container arguments must start from a read-only root') + } + if (!args.includes('--cap-drop') || args[args.indexOf('--cap-drop') + 1] !== 'ALL') { + throw new Error('FlyBot container arguments must drop every capability') + } + if (!args.includes('--user')) throw new Error('FlyBot container arguments must name an unprivileged user') + const binds: string[] = [] + for (let index = 0; index < args.length; index += 1) { + const argument = args[index] ?? '' + if (argument === '--cap-add') throw new Error('FlyBot container arguments must not add a capability') + if (argument === '--user') { + const user = (args[index + 1] ?? '').split(':')[0] + if (user === '' || user === '0' || user === 'root') { + throw new Error(`FlyBot container must run as an unprivileged user, got "${user}"`) + } + } + if (argument === '--security-opt' && args[index + 1] !== 'no-new-privileges') { + throw new Error(`FlyBot container must keep no-new-privileges, got "${args[index + 1] ?? ''}"`) + } + if (argument === '-v') binds.push(args[index + 1] ?? '') + } + // Only a mount can reach the host socket; the probe command that proves it is + // absent from inside names the path on purpose, so argument text is not a + // mount and must not be refused. + if (binds.some((bind) => (bind.split(':')[0] ?? '').includes('docker.sock'))) { + throw new Error('FlyBot container must never mount the host Docker socket') + } + if (binds.length !== 1) { + throw new Error(`FlyBot container must mount exactly one host path, got ${binds.length}`) + } + const [source, target] = (binds[0] ?? '').split(':') + if (source !== dataDir || target !== DATA_MOUNT) { + throw new Error(`FlyBot container must mount the repository data directory at ${DATA_MOUNT}, got "${binds[0] ?? ''}"`) + } + return args +} + +// No process inside the container may raise its own limits or gain privileges: +// no capability survives, and the setuid bit cannot add one back. +export function flybotContainerArgs(input: FlybotContainerInput): string[] { + const dataDir = checkedMountSource(input.dataDir) + if (!Number.isInteger(input.uid) || input.uid === 0) { + throw new Error(`FlyBot container must run as a non-root uid, got "${input.uid}"`) + } + if (!Number.isInteger(input.gid) || input.gid === 0) { + throw new Error(`FlyBot container must run as a non-root gid, got "${input.gid}"`) + } + return assertContainerBoundary([ + 'run', '-d', + '--name', input.container, + '--network', input.network, + '-p', `127.0.0.1:${input.dshPort}:3081`, + '-v', `${dataDir}:/data`, + '--user', `${input.uid}:${input.gid}`, + '--memory', input.memory, + '--cpus', input.cpus, + '--pids-limit', '512', + '--read-only', + '--tmpfs', TMPFS, + '--cap-drop', 'ALL', + '--security-opt', 'no-new-privileges', + '--restart', 'no', + '--entrypoint', '/bin/sh', + '-e', 'HOME=/data', + '-e', 'DSH_HOME=/data', + // A hidden constraint, not a preference: the loader writes its native + // binary into `os.tmpdir()` and `require()`s it, so a `noexec` /tmp makes the + // web profile fail to start. Disabling the cache loads the binary in place. + '-e', 'NARB_DISABLE_NATIVE_CACHE=1', + '-e', 'NPM_CONFIG_REGISTRY=https://registry.npmmirror.com', + '-e', 'npm_config_registry=https://registry.npmmirror.com', + '-e', `FLEDGE_MODEL_KEY=${input.modelKey}`, + input.image, + '-c', input.command ?? 'node /opt/flybot/docker/entrypoint.mjs >> /data/dsh-web.log 2>&1 & exec node /data/tcp-proxy.mjs', + ], dataDir) +} diff --git a/test/flybotIsolation.test.ts b/test/flybotIsolation.test.ts new file mode 100644 index 00000000..acf4a295 --- /dev/null +++ b/test/flybotIsolation.test.ts @@ -0,0 +1,259 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { existsSync, mkdirSync, mkdtempSync, readFileSync } from 'node:fs' +import { execFile } from 'node:child_process' +import { promisify } from 'node:util' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { assertContainerBoundary, flybotContainerArgs, type FlybotContainerInput } from '../src/flybot/isolation.js' + +const execFileAsync = promisify(execFile) + +// #109: every per-repository FlyBot runs as a Docker container on the same host +// as every other tenant, so the run arguments are a security boundary and not +// just a convenience. These assertions are the contract the product's own run +// arguments must keep; the container-level proof — that a real container +// actually is read-only and capability-free from the inside — lives in +// `scripts/isolation-check.ts`, which runs where a Docker daemon exists and +// exits non-zero where it does not. + +function sourceOf(path: string): string { + const url = new URL(`../${path}`, import.meta.url) + return existsSync(url) ? readFileSync(url, 'utf8') : '' +} + +// The flags may live in the instance manager or in a dedicated builder module; +// either way the product starts the container with them, so both files count. +function runArgsSource(): string { + return ['src/flybot/isolation.ts', 'src/flybot/instances.ts'].map(sourceOf).join('\n') +} + +test('the job container starts with a read-only root and no host Docker socket', () => { + const source = runArgsSource() + assert.match( + source, + /'--read-only'/, + 'the container root is writable, so anything inside it can rewrite the image the next job runs from', + ) + assert.match( + source, + /NARB_DISABLE_NATIVE_CACHE/, + 'the native-addon loader writes a binary into /tmp and executes it, so a noexec /tmp needs the cache disabled', + ) + assert.match(source, /--tmpfs/, 'a read-only root must declare the writable paths explicitly instead of leaving the root writable') +}) + +test('the job container drops every capability and runs as an unprivileged user', () => { + const source = runArgsSource() + assert.match(source, /'--cap-drop',\s*'ALL'/, 'capabilities are not dropped, so a breakout keeps the default capability set') + assert.match(source, /'--security-opt',\s*'no-new-privileges'/, 'no-new-privileges is not passed') + assert.match(source, /'--user',/, 'the container must not run as root inside its own namespace') + assert.match(source, /'--pids-limit'/, 'a fork bomb inside one tenant must not starve the whole host') +}) + +test('the isolation check exists and refuses to pass without a Docker daemon', async () => { + const script = fileURLToPath(new URL('../scripts/isolation-check.ts', import.meta.url)) + assert.equal( + existsSync(script), + true, + 'scripts/isolation-check.ts must exist: the boundary is proven from inside a container, not by reading flags', + ) + const failure = await execFileAsync('npx', ['tsx', script, '--docker', '/nonexistent/docker'], { timeout: 180_000 }) + .then(() => null) + .catch((error: { code?: number; stderr?: string; stdout?: string }) => error) + assert.ok(failure !== null, 'the check passed with no Docker daemon, which is exactly the silent green it must not allow') + assert.equal(failure.code, 3, 'a missing daemon is its own outcome: exit 3, never 0') + assert.match(String(failure.stderr ?? ''), /no Docker daemon/i) +}) + +test('the isolation check proves the boundary inside the container', () => { + const source = sourceOf('scripts/isolation-check.ts') + assert.match(source, /read-only|readonly|--read-only/, 'the check does not assert a read-only root') + assert.match(source, /CapEff/, 'the check does not read the effective capability set from inside the container') + assert.match(source, /Seccomp/, 'the check does not assert that a seccomp filter is loaded') + assert.match(source, /docker\.sock/, 'the check does not assert that the host Docker socket is absent') +}) + +// The two blocks above read the source; these call the builder. They need no +// Docker daemon, so the arguments the product actually hands to `docker run` are +// asserted on every `npm test`, and each fact is proven both ways: the flag that +// must be there, and the tampered argument that must be refused. +const UNIT_DATA_DIR = mkdtempSync(join(tmpdir(), 'flybot-args-')) + +function unitArgs(overrides: Partial = {}): string[] { + return flybotContainerArgs({ + container: 'fledge-flybot-abc123', + network: 'fledge-abc123', + dataDir: UNIT_DATA_DIR, + uid: 1000, + gid: 1000, + dshPort: 4300, + memory: '1g', + cpus: '1.5', + image: 'flybot:fledge-20261031-2213b4ca', + modelKey: 'model-key-1', + ...overrides, + }) +} + +function flag(args: string[], name: string): string | undefined { + const index = args.indexOf(name) + return index === -1 ? undefined : args[index + 1] +} + +// The image and the entrypoint command are the last three arguments; a flag +// added by hand goes in front of them, where Docker reads it. +function withFlags(args: string[], ...extra: string[]): string[] { + return [...args.slice(0, -3), ...extra, ...args.slice(-3)] +} + +function replaceValue(args: string[], name: string, value: string): string[] { + const copy = [...args] + copy[copy.indexOf(name) + 1] = value + return copy +} + +function withoutPair(args: string[], name: string): string[] { + const index = args.indexOf(name) + return [...args.slice(0, index), ...args.slice(index + 2)] +} + +test('the builder keeps a read-only root and no root mount', () => { + const args = unitArgs() + assert.equal(args.includes('--read-only'), true, 'the container root must be read-only') + // A root mount would hand back the writable tree the flag exists to remove. + assert.equal( + args.filter((argument) => argument.startsWith('/')).some((argument) => argument.endsWith(':/')), + false, + 'no bind may mount the container root', + ) + assert.throws( + () => assertContainerBoundary(withoutPair(args, '--read-only'), UNIT_DATA_DIR), + /read-only root/, + 'arguments without --read-only must be refused, not silently run', + ) + assert.throws( + () => assertContainerBoundary(replaceValue(args, '-v', '/:/'), UNIT_DATA_DIR), + /repository data directory at \/data/, + 'a bind of the host root at the container root must be refused', + ) +}) + +test('the builder gives /tmp the one writable tmpfs and no second bind', () => { + const args = unitArgs() + assert.equal(flag(args, '--tmpfs'), '/tmp:rw,noexec,nosuid,size=64m', 'a read-only root declares its writable paths') + assert.equal(args.filter((argument) => argument === '-v').length, 1, 'exactly one host path may be mounted') + assert.throws( + () => assertContainerBoundary(withFlags(args, '-v', `${UNIT_DATA_DIR}:/var/tmp`), UNIT_DATA_DIR), + /exactly one host path, got 2/, + 'a second writable bind must be refused', + ) +}) + +test('the builder drops every capability and adds none', () => { + const args = unitArgs() + assert.equal(flag(args, '--cap-drop'), 'ALL', 'no capability survives into the container') + assert.equal(args.includes('--cap-add'), false, 'nothing may add a capability back') + assert.throws( + () => assertContainerBoundary(withFlags(args, '--cap-add', 'SYS_ADMIN'), UNIT_DATA_DIR), + /must not add a capability/, + 'an added capability must be refused', + ) + assert.throws( + () => assertContainerBoundary(replaceValue(args, '--cap-drop', 'NET_RAW'), UNIT_DATA_DIR), + /drop every capability/, + 'dropping only some capabilities must be refused', + ) +}) + +test('the builder keeps no-new-privileges, not a disabled variant', () => { + const args = unitArgs() + assert.equal(flag(args, '--security-opt'), 'no-new-privileges') + assert.equal(args.some((argument) => argument === '--security-opt=no-new-privileges=false'), false) + assert.throws( + () => assertContainerBoundary(replaceValue(args, '--security-opt', 'no-new-privileges=false'), UNIT_DATA_DIR), + /must keep no-new-privileges/, + 'a disabled no-new-privileges must be refused however it is spelled', + ) +}) + +test('the builder mounts exactly the repository data directory at /data', () => { + const args = unitArgs() + assert.equal(flag(args, '-v'), `${UNIT_DATA_DIR}:/data`, 'the data directory is the only host path mounted') + assert.throws( + () => assertContainerBoundary(withFlags(args, '-v', '/etc:/data:ro'), UNIT_DATA_DIR), + /exactly one host path, got 2/, + 'a second bind must be refused even when it is read-only', + ) + assert.throws( + () => assertContainerBoundary(replaceValue(args, '-v', '/etc:/data'), UNIT_DATA_DIR), + /repository data directory at \/data/, + 'a bind outside the repository data directory must be refused', + ) +}) + +test('the builder refuses a root user or no user at all', () => { + const args = unitArgs() + assert.equal(flag(args, '--user'), '1000:1000', 'the container runs as the invoking user, never root') + assert.throws(() => unitArgs({ uid: 0 }), /non-root uid/, 'uid 0 must be refused') + assert.throws(() => unitArgs({ gid: 0 }), /non-root gid/, 'gid 0 must be refused') + assert.throws( + () => assertContainerBoundary(withoutPair(args, '--user'), UNIT_DATA_DIR), + /must name an unprivileged user/, + 'arguments with no --user must be refused', + ) + assert.throws( + () => assertContainerBoundary(replaceValue(args, '--user', 'root:root'), UNIT_DATA_DIR), + /unprivileged user, got "root"/, + 'a named root user must be refused', + ) +}) + +test('the builder never mounts the host Docker socket, including as a data path', () => { + const args = unitArgs() + assert.equal( + args.filter((argument) => argument.startsWith('-v')).includes('-v'), + true, + ) + assert.throws( + () => assertContainerBoundary(replaceValue(args, '-v', '/var/run/docker.sock:/data'), UNIT_DATA_DIR), + /never mount the host Docker socket/, + 'a bind whose source is the host Docker socket must be refused', + ) + const socketDir = join(UNIT_DATA_DIR, 'docker.sock') + mkdirSync(socketDir, { recursive: true }) + assert.throws( + () => unitArgs({ dataDir: socketDir }), + /never mount the host Docker socket/, + 'a repository data path that names the socket must be refused before it becomes a bind', + ) +}) + +test('a command that inspects the socket path is not a socket mount', () => { + // The isolation check proves the socket is absent from inside the container, + // so its probe command names the path. A command is not a mount and must not + // be refused, or the check fails before it can start a container. + const probe = "node -e \"require('node:fs').existsSync('/var/run/docker.sock')\"" + const args = unitArgs({ command: probe }) + assert.equal(args.at(-1), probe, 'the probe command reaches docker run unchanged') + assert.equal(flag(args, '-v'), `${UNIT_DATA_DIR}:/data`, 'the only bind is still the data directory') +}) + +test('the builder refuses a data directory that is not an absolute existing directory', () => { + assert.throws( + () => unitArgs({ dataDir: 'repos/tenant/data' }), + /must be absolute/, + "a relative data directory is resolved by the daemon against its own working directory, so it can mount another tenant's tree", + ) + assert.throws( + () => unitArgs({ dataDir: join(UNIT_DATA_DIR, 'missing') }), + /does not exist/, + 'a missing data directory would be created by Docker as root', + ) + assert.throws( + () => unitArgs({ dataDir: fileURLToPath(new URL('../package.json', import.meta.url)) }), + /not a directory/, + 'a file is not a mount source for a tenant data directory', + ) +})