diff --git a/docs/development/testing.mdx b/docs/development/testing.mdx index caf417cb1..fa7a8af2f 100644 --- a/docs/development/testing.mdx +++ b/docs/development/testing.mdx @@ -57,6 +57,61 @@ Tests MUST isolate environment variables such as `AGENT_PROVIDER`, `AGENT_MODEL_ `copilot bugbot-analytics` aggregates content-free telemetry exported by the Action. Do not add prompt, diff, source, comment, response, credential, or rule content to the telemetry schema. +## Qualification source and runtime admission + +New Bugbot qualification runs use an explicit committed implementation pin. +Before loading application modules, the corpus and review entrypoints compare +the physical `src/`, lockfile, compiler configuration and package manifest with +that commit's regular Git blobs. Ignored files, JavaScript shadows, symlinks, +staged differences and Git's assume-unchanged hints cannot conceal another +executed source. The reusable check strips inherited `GIT_*` variables, anchors +the Git directory and worktree explicitly, disables replacement objects and +fsmonitor, and excludes global/system configuration. It also supports linked +local worktrees. + +```text +Committed plan -> physical source proof -> driver/runtime proof -> import -> dispatch +``` + +Text equivalent: prove the actual source, evaluation drivers, Node executable +and TypeScript compiler before importing the application; prove the prepared +model executable and any launcher again before dispatch. The start record +retains the implementation commit/tree, source inventory digest, lockfile and +compiler digests, driver hashes and runtime identities. A version string alone +does not establish executable identity. + +Preregister a **new** plan after reviewing the corrections. Include the current +`qualificationDrivers` inventory exported by `ev07-shadow/implementation.cjs`, +including the shared `execution-provenance.cjs`. Record the `node`, +`nodeSha256`, `typescript`, `typescriptSha256`, `git`, `gitExecutable` and +`gitSha256` observations returned by +`runtimeIdentity(root)` alongside the existing exact CLI/launcher identities. +Git proof commands use the trusted operating-system executable, never `PATH`. +Linux uses `/usr/bin/git`; macOS resolves the actual developer-tools Git with +`/usr/bin/xcrun` in a clean environment. The supported qualification bootstrap +is Linux/macOS and does not claim Windows qualification or Setup acceptance. +Commit the completed plan before measurement. The normal-flow diagnostic's +selected subject manifest similarly requires `implementation`, `runtime` and +`drivers` from its exported `normalFlowDrivers` inventory; its direct runtime +uses an absolute `runtime.executable` and optional absolute `runtime.launcher`. +These are local evaluation artifacts, with no new public Action setting. + +Run `pnpm run test:bugbot-shadow-verification` for the deterministic admission +controls. They use temporary Git repositories and blocked model/write paths. +An admission error means no application/model work was admitted. Inspect the +named source, staged inventory, driver or runtime mismatch, then restore the +selected bytes or prepare a separately reviewed and committed preregistration. +Do not regenerate expected hashes merely to suppress a failure. + +Historical manifests, ledgers, scores and unsuccessful campaigns retain their +original bytes and reproduce through their frozen commits. Their old plans +cannot admit a run through the corrected drivers. Passing deterministic +admission neither passes the 174-case corpus nor admits the 42 paired reviews. +Keep `BUGBOT_VERIFICATION_MODE=disabled`, evaluated-PR write attempts at zero, +explicit cancellation, optional usage telemetry and the existing score floors. +Latency is measured after investigation; admission introduces no research +deadline, turn count or token budget. + Workflow queue tests inject a clock, random source, and scheduler/delay port. They cover workflow-scoped active-status traversals and later-page matches, strict lower-ID/workflow-file filtering, malformed responses, fail-closed provider errors, 429 and @@ -274,3 +329,109 @@ Inspect the run's individual platform jobs before claiming full success. A queued, canceled or failed job is missing evidence, even if other jobs passed. The platform fixtures exercise local paths. Authenticated agent tool execution has separate acceptance. + +Qualification runtime admission also pins `dependencyTreeSha256` and +`dependencyEntries` for the physical installed dependency closure. Capture the +expected identity only from a reviewed isolated frozen-lockfile installation +with package integrity checked and lifecycle scripts disabled, before committing +the new preregistration. At launch compare every file/mode and internal link, +including absent platform-optional targets. External dependency links and +nested driver dependency shadows are not admitted. The normal unused portal to +this repository browser-extension workspace is inventoried as unadmitted; the +entrypoint registers synchronous Node hooks that block its real path for both +require and import. Use a separate lab rather than changing a contributor's +working dependency tree. The reviewed Node/package +manager/OS bootstrap remains a trust prerequisite, not an independently attested +claim derived from the lockfile or an inventory alone. + +External package imports wait for admission. The corpus version probe additionally +requires `runtime.executable` and optional `runtime.launcher`, validates their +existing digest pins before executing `--version`, uses the absolute invocation +under a clean environment, then compares the version. PATH cannot select this +probe. These are integrity checks, not model research time/turn/read/token caps. + +Supply read-only GitHub evidence credentials through `COPILOT_EVIDENCE_TOKEN`, +`GH_TOKEN` or `GITHUB_TOKEN` as appropriate for the private fixtures; the review +driver does not launch a PATH-resolved credential helper. Do not persist tokens +in a plan or evidence record. An absent credential blocks the required CI reader. + + +### Start qualification from a reviewed bootstrap + +Extract `quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs` from an +independently reviewed commit using trusted operating-system Git into a private +file outside the checkout. Verify its independently reviewed SHA-256 before +launching trusted Node with NODE_OPTIONS and NODE_PATH unset. Do not run the +working-tree corpus, normal-flow or full-review entrypoint directly for measured +qualification; those CLIs refuse before importing checkout code. + +The positional arguments are repository, reviewed plan commit, relative plan +path, reviewed plan SHA-256, reviewed bootstrap SHA-256, mode, then mode arguments. +`corpus` takes a new output directory and optional `--preflight`; `full-review` +takes the campaign directory; `normal-flow` takes lab, PR number and a new ledger +path. The selected plan must be a committed regular blob containing the complete +current driver/runtime pins. Its revision must contain the exact reviewed +bootstrap. Treat the reviewed digests and trusted Node/Git/package installation +as operator trust anchors; a checkout's own receipt is insufficient. + +The launcher authenticates and materializes drivers before loading its verifier, +checks original and private source/dependency/runtime bytes, forwards explicit +signals and removes its private source after exit. It records stable bootstrap +and plan identities without private temporary paths. Normal-flow authenticates +the manifest before selecting even a `--version` executable. Production stays +disabled and publication stays dry-run. Existing frozen campaigns replay with +their frozen historical drivers; this route needs a new reviewed preregistration. + + +The bootstrap also binds later application evidence reads to that admitted Git. +Its child PATH starts with a private directory containing only a link to the +verified OS Git executable; the remaining PATH and normal Node/Codex selection +are preserved. This routing is part of the reviewed bootstrap identity, rather +than a new user-configurable executable override. The actual application adapter +is included in a causal shim/no-shim deterministic control. + + +Qualification also checks the ordinarily selected CLI before calling the normal +planner: that planner probes --version during preparation. Admission uses its +production resolution/validation helpers, then delegates to the unchanged planner +and checks the final prepared runtime/profile as before. A matching version +reported by an unregistered executable cannot execute even during preparation. + +Qualification drivers now refuse direct CLI execution before importing a checkout +helper. A `.qualification-admission.json` file, even with matching hashes, supplies +no execution authority. Use the independently materialized external bootstrap +above. It performs the complete validation/materialization and calls the driver +in that same bootstrap process. The internal --admitted-worker endpoint and IPC +message handler have been removed: a parent message cannot initialize admission. +Each nested scheduler/review process enters the complete public launch again with +the same explicit reviewed plan/bootstrap pins and repeats all private byte and +runtime checks. The final review slot verifies its exact parent corpus campaign. + +Ordinary corpus/full-review/normal-flow arguments are unchanged. Nested launches +also use full-review-runner with one campaign argument, or review with campaign, +checkout, subject ID, repetition and output. Every mode requires the same explicit +reviewed plan/bootstrap identities and complete proof; there is no worker shortcut. +Explicit cancellation reaches the executing admitted driver and preserves its exit. +There are no new consumer settings or investigation limits. Seven additional +receipt/direct-entry/arbitrary-IPC/nested-mutation/cancellation cases use real Git +and owned fakes, with zero model calls and evaluated-PR writes. The frozen previous +bootstrap demonstrates the IPC bypass before the new rejection. Fixture cleanup +timeouts never bound a model investigation. Context is not independent host or +human-review accreditation: retain the external digest check before starting Node +and freeze new driver hashes before measuring. + + +The pinned `--version` metadata probe uses the ordinary production reader's +15-second health timeout and kills a stalled probe with SIGKILL, including a +launcher that ignores SIGTERM. ETIMEDOUT is a preflight failure: retain that +attempt, repair the pinned installation and review a new plan when its bytes +change. No model call or clean verdict is produced by a stalled version probe. +This timeout applies only to executable metadata, before investigation; model +research and explicit cancellation retain their existing contract. It adds no +consumer setting or automatic retry. Production source is unchanged. + +The real-process controls cover both direct and optional-launcher stalls, exact +--version arguments, timeout failure and absence of a surviving CLI. Their owned +30-second fixture cleanup never bounds research. Node's synchronous subprocess +API otherwise continues waiting if a timed-out child handles SIGTERM; see the +[pinned Node v24.20.0 subprocess documentation](https://nodejs.org/download/release/v24.20.0/docs/api/child_process.html#child_processexecfilesyncfile-args-options). diff --git a/package.json b/package.json index 4c7247b20..29e456314 100644 --- a/package.json +++ b/package.json @@ -69,7 +69,7 @@ "test": "jest", "test:watch": "jest --watch", "test:coverage": "jest --coverage --json --outputFile=coverage/setup-acceptance-jest.json && node scripts/validate-coverage-budgets.cjs && node scripts/validate-setup-assistant-acceptance.cjs coverage/setup-acceptance-jest.json && pnpm run test:extension", - "test:bugbot-shadow-verification": "node --test quality/bugbot/evidence/issue-950/shadow-verification/verification.test.cjs quality/bugbot/evidence/issue-950/normal-flow-shadow/config.test.cjs quality/bugbot/evidence/issue-950/normal-flow-shadow/paths.test.cjs quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.test.cjs quality/bugbot/evidence/issue-950/ev07-shadow/review-runtime.test.cjs quality/bugbot/evidence/issue-950/ev07-shadow/score-review.test.cjs quality/bugbot/evidence/issue-950/ev07-shadow/plan.test.cjs quality/bugbot/evidence/issue-950/status-card-publication/publisher.test.cjs", + "test:bugbot-shadow-verification": "node --test quality/bugbot/evidence/issue-950/qualification-bootstrap.test.cjs quality/bugbot/evidence/issue-950/shadow-verification/verification.test.cjs quality/bugbot/evidence/issue-950/normal-flow-shadow/config.test.cjs quality/bugbot/evidence/issue-950/normal-flow-shadow/paths.test.cjs quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.test.cjs quality/bugbot/evidence/issue-950/ev07-shadow/review-runtime.test.cjs quality/bugbot/evidence/issue-950/ev07-shadow/score-review.test.cjs quality/bugbot/evidence/issue-950/ev07-shadow/plan.test.cjs quality/bugbot/evidence/issue-950/status-card-publication/publisher.test.cjs", "eval:bugbot": "jest src/tooling/__tests__/bugbot_quality_eval.test.ts src/tooling/__tests__/bugbot_benchmark.test.ts src/tooling/__tests__/bugbot_request_transform_evidence.test.ts --runInBand", "typecheck": "tsc --noEmit && pnpm run typecheck:extension", "metrics:architecture": "node scripts/collect-architecture-metrics.cjs", diff --git a/quality/bugbot/evidence/issue-950/ev07-shadow/corpus.cjs b/quality/bugbot/evidence/issue-950/ev07-shadow/corpus.cjs index 657899ce0..95b96acf5 100644 --- a/quality/bugbot/evidence/issue-950/ev07-shadow/corpus.cjs +++ b/quality/bugbot/evidence/issue-950/ev07-shadow/corpus.cjs @@ -1,28 +1,35 @@ 'use strict'; +if (require.main === module) throw new Error('Use the independently reviewed immutable bootstrap; direct qualification runners are unsupported.'); + const assert = require('node:assert/strict'); const { createHash } = require('node:crypto'); const { execFileSync } = require('node:child_process'); const fs = require('node:fs'); const path = require('node:path'); +const { trustedGitExecutable, gitEnvironment, admitQualificationModules } = require('../execution-provenance.cjs'); const { tmpdir } = require('node:os'); -const { requirePinnedImplementation, requirePinnedRuntime, requirePinnedExecution, requirePinnedDrivers, selectedPlanProvenance } = require('./implementation.cjs'); -require('./load.cjs'); - -const { runBugbotBenchmarkAgent } = require('../../../../../src/tooling/bugbot_benchmark_runner'); -const { evaluateBugbotBenchmark, loadBugbotBenchmark } = require('../../../../../src/tooling/bugbot_benchmark'); -const { createFindingsQueryPort } = require('../../../../../src/infrastructure/composition/agent_capability_composition_root'); -const { AgentCliClient } = require('../../../../../src/data/repository/agent_cli_client'); -const { AgentExecutionPlanner } = require('../../../../../src/infrastructure/agents/agent_execution_planner'); -const { codexInvocation } = require('../../../../../src/infrastructure/bugbot/shadow/session.cjs'); +const { qualificationProvenance, requirePinnedCliProbe, requirePinnedExecution, preparePinnedExecution, selectedPlanProvenance } = require('./implementation.cjs'); const plan = require('./configuration.cjs'); const root = path.resolve(__dirname, '../../../../..'); const hash = input => createHash('sha256').update(input).digest('hex'); async function run(destination, preflight = false) { - requirePinnedDrivers(plan); - const implementation = requirePinnedImplementation(); + require('../execution-provenance.cjs').requireBootstrapAdmission(root); + const provenance = qualificationProvenance(plan); + const implementation = provenance.implementation; + admitQualificationModules(root); + const cliProbe = requirePinnedCliProbe(plan.runtime); + require('./load.cjs'); + + const { runBugbotBenchmarkAgent } = require('../../../../../src/tooling/bugbot_benchmark_runner'); + const { evaluateBugbotBenchmark, loadBugbotBenchmark } = require('../../../../../src/tooling/bugbot_benchmark'); + const { createFindingsQueryPort } = require('../../../../../src/infrastructure/composition/agent_capability_composition_root'); + const { AgentCliClient } = require('../../../../../src/data/repository/agent_cli_client'); + const { AgentExecutionPlanner } = require('../../../../../src/infrastructure/agents/agent_execution_planner'); + const { codexInvocation } = require('../../../../../src/infrastructure/bugbot/shadow/session.cjs'); + assert.equal(hash(fs.readFileSync(__filename)), plan.inspectedArtifacts?.['corpus.cjs'], 'corpus runner differs from the preregistered implementation'); assert(destination, 'usage: node corpus.cjs '); @@ -34,14 +41,13 @@ async function run(destination, preflight = false) { const configuration = plan.detector; const preregistration = selectedPlanProvenance(); const planner = new AgentExecutionPlanner(); - const cli = execFileSync('codex', ['--version'], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }).trim(); - requirePinnedRuntime(cli); + const cli = cliProbe.version; fs.mkdirSync(destination); const ledger = path.join(destination, 'corpus.jsonl'); const record = row => fs.appendFileSync(ledger, JSON.stringify(row) + '\n', { mode: 0o600 }); - record({ event: 'start', at: new Date().toISOString(), cli, configuration, + record({ event: 'start', at: new Date().toISOString(), cli, cliProbe, configuration, plan: preregistration.sha256, preregistration, runner: hash(fs.readFileSync(__filename)), - implementation, corpus: plan.corpus.sha256, gitHubMutationPorts: 0 }); + implementation, provenance, corpus: plan.corpus.sha256, gitHubMutationPorts: 0 }); const cwd = fs.mkdtempSync(path.join(tmpdir(), 'copilot-ev07-corpus-')); const oldCwd = process.cwd(); const cancel = new AbortController(); @@ -50,7 +56,7 @@ async function run(destination, preflight = false) { process.once('SIGTERM', abort); const results = new Map(); function prepare(request, calls, runtimeProofs) { - const invocation = planner.prepare(request); + const invocation = preparePinnedExecution(planner, request, plan.runtime); try { const proof = requirePinnedExecution(invocation); const flags = codexInvocation(invocation.argv); @@ -68,7 +74,7 @@ async function run(destination, preflight = false) { } let next = 0; try { - execFileSync('git', ['init', '--quiet', '--template='], { cwd }); + execFileSync(trustedGitExecutable(), ['init', '--quiet', '--template='], { cwd, env: gitEnvironment() }); process.chdir(cwd); const flags = [], runtimeProofs = []; const canary = createFindingsQueryPort({ cli: { async execute(request) { @@ -133,4 +139,4 @@ async function run(destination, preflight = false) { } } -run(process.argv[2], process.argv[3] === '--preflight').catch(error => { console.error(error.code ?? error.name, error.message); process.exitCode = 1; }); +module.exports = { run: (destination, flag) => run(destination, flag === '--preflight') }; diff --git a/quality/bugbot/evidence/issue-950/ev07-shadow/full-review.cjs b/quality/bugbot/evidence/issue-950/ev07-shadow/full-review.cjs index 4fb6c1b0d..3807a2b87 100644 --- a/quality/bugbot/evidence/issue-950/ev07-shadow/full-review.cjs +++ b/quality/bugbot/evidence/issue-950/ev07-shadow/full-review.cjs @@ -1,17 +1,21 @@ 'use strict'; -const { spawn } = require('node:child_process'); +if (require.main === module) throw new Error('Use the independently reviewed immutable bootstrap; direct qualification runners are unsupported.'); + +const { spawnQualificationDriver } = require('../execution-provenance.cjs'); const fs = require('node:fs'); const path = require('node:path'); const { requirePassingCorpus } = require('./launch-full-review.cjs'); -const { requirePinnedImplementation, selectedPlanProvenance } = require('./implementation.cjs'); +const { qualificationProvenance, selectedPlanProvenance } = require('./implementation.cjs'); const plan = require('./configuration.cjs'); const concurrency = 3; async function run(campaign) { + require('../execution-provenance.cjs').requireBootstrapAdmission(path.resolve(__dirname, '../../../../..')); campaign = requirePassingCorpus(campaign); - const implementation = requirePinnedImplementation(); + const provenance = qualificationProvenance(plan); + const implementation = provenance.implementation; const preregistration = selectedPlanProvenance(); const output = path.join(campaign, 'full-review'); const score = path.join(campaign, 'corpus-v3/corpus-score.json'); @@ -31,7 +35,7 @@ async function run(campaign) { }; process.once('SIGINT', cancel); process.once('SIGTERM', cancel); - record({ event: 'start', at: new Date().toISOString(), planned: slots.length, concurrency, implementation, + record({ event: 'start', at: new Date().toISOString(), planned: slots.length, concurrency, implementation, provenance, plan: preregistration.sha256, preregistration }); try { async function worker() { @@ -42,7 +46,7 @@ async function run(campaign) { const log = fs.openSync(path.join(output, name + '.log'), 'wx', 0o600); record({ event: 'slot-start', ...slot }); try { - const child = spawn(process.execPath, [path.join(__dirname, 'review.cjs'), path.join(checkouts, slot.id), + const child = spawnQualificationDriver('review', [path.join(checkouts, slot.id), slot.id, String(slot.repetition), destination], { stdio: ['ignore', log, log] }); running.add(child); const exit = await new Promise(resolve => { @@ -64,4 +68,4 @@ async function run(campaign) { } } -run(process.argv[2]).catch(error => { console.error(error.code ?? error.name, error.message); process.exitCode = 1; }); +module.exports = { run }; diff --git a/quality/bugbot/evidence/issue-950/ev07-shadow/implementation.cjs b/quality/bugbot/evidence/issue-950/ev07-shadow/implementation.cjs index 17836cd11..5ea81cc27 100644 --- a/quality/bugbot/evidence/issue-950/ev07-shadow/implementation.cjs +++ b/quality/bugbot/evidence/issue-950/ev07-shadow/implementation.cjs @@ -4,24 +4,23 @@ const assert = require('node:assert/strict'); const { execFileSync } = require('node:child_process'); const { createHash } = require('node:crypto'); const { readFileSync, lstatSync, realpathSync } = require('node:fs'); -const { devNull } = require('node:os'); const { isDeepStrictEqual } = require('node:util'); const path = require('node:path'); const { implementation } = require('./configuration.cjs'); +const { attestSource, attestRuntime, attestDrivers, pinnedGit } = require('../execution-provenance.cjs'); + function requirePinnedImplementation(root = path.resolve(__dirname, '../../../../..'), pin = implementation) { - assert.match(pin, /^[a-f0-9]{40}$/u, 'invalid implementation pin'); - const git = (...args) => execFileSync('git', ['-C', root, ...args], { encoding: 'utf8' }).trim(); - assert.equal(git('rev-parse', '--verify', pin + '^{commit}'), pin, - 'preregistered implementation commit unavailable'); - const source = ['src', 'pnpm-lock.yaml', 'tsconfig.json']; - assert.equal(git('diff', '--no-ext-diff', '--no-textconv', '--name-only', pin, '--', ...source), '', - 'application source or lockfile does not match the preregistered implementation'); - assert.equal(git('diff', '--cached', '--no-ext-diff', '--no-textconv', '--name-only', pin, '--', ...source), '', - 'staged application source or lockfile differs from the preregistered implementation'); - assert.equal(git('ls-files', '--others', '--', ...source), '', - 'untracked application source or lockfile'); - return pin; + return attestSource(root, pin).implementation; +} + +function qualificationProvenance(selected = require('./configuration.cjs')) { + const root = path.resolve(__dirname, '../../../../..'); + const source = attestSource(root, selected.implementation); + const drivers = requirePinnedDrivers(selected); + const runtime = attestRuntime(root, selected.runtime); + const admission = require('../execution-provenance.cjs').bootstrapAdmission(root); + return { ...source, drivers, runtime, ...(admission && { bootstrap: admission }) }; } function requirePinnedRuntime(cli, runtime = require('./configuration.cjs').runtime) { @@ -54,17 +53,47 @@ function requirePinnedExecution(execution, runtime = require('./configuration.cj return { executableSha256 }; } +function requirePinnedCliProbe(runtime = require('./configuration.cjs').runtime) { + const launcher = runtime?.launcher ? [runtime.launcher] : []; + const execution = { executable: runtime?.executable, launcherArgv: launcher, + ...(launcher.length && { launcherSha256: runtime.launcherSha256 }), + runtimeContract: { version: runtime?.codex } }; + const proof = requirePinnedExecution(execution, runtime); + // Never resolve through PATH, or run even --version before byte admission. + const version = execFileSync(execution.executable, [...launcher, '--version'], { + encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], + env: { PATH: '/usr/bin:/bin' }, timeout: 15_000, killSignal: 'SIGKILL', + }).trim(); + requirePinnedRuntime(version, runtime); + return { version, proof }; +} + +function preparePinnedExecution(planner, request, runtime = require('./configuration.cjs').runtime) { + // The ordinary planner probes the selected CLI before returning its plan. + // Resolve through its production helpers and admit those bytes first; + // then retain the ordinary planner and its post-preparation proof unchanged. + const root = path.resolve(__dirname, '../../../../..'); + const { resolveAgentExecutablePath, validateResolvedAgentInvocation } = require(path.join(root, + 'src/infrastructure/agents/agent_executable_invocation')); + const { getAgentRuntimeManifestEntry } = require(path.join(root, 'src/infrastructure/agents/agent_runtime_manifest')); + const { provider } = request.configuration, environment = request.environment ?? process.env; + const selected = request.configuration.executable?.trim() || getAgentRuntimeManifestEntry(provider).executable; + const executable = resolveAgentExecutablePath(selected, environment); + const invocation = validateResolvedAgentInvocation(executable, provider, environment); + const launcherArgv = invocation.prefixArgs; + requirePinnedExecution({ executable: invocation.executable, launcherArgv, + ...(launcherArgv.length && { launcherSha256: createHash('sha256').update(readFileSync(launcherArgv[0])).digest('hex') }), + runtimeContract: { version: runtime?.codex } }, runtime); + return planner.prepare(request); +} + const qualificationDrivers = Object.freeze(['configuration.cjs', 'implementation.cjs', 'load.cjs', 'corpus.cjs', - 'review.cjs', 'full-review.cjs', 'launch-full-review.cjs', 'score-review.py']); + 'review.cjs', 'full-review.cjs', 'launch-full-review.cjs', 'score-review.py', '../execution-provenance.cjs', '../qualification-bootstrap.cjs']); function requirePinnedDrivers(selected = require('./configuration.cjs')) { - for (const file of qualificationDrivers) { - const expected = selected.inspectedArtifacts?.[file]; - assert.match(expected ?? '', /^[a-f0-9]{64}$/u, 'preregistered driver hash unavailable: ' + file); - assert.equal(createHash('sha256').update(readFileSync(path.join(__dirname, file))).digest('hex'), expected, - 'qualification driver differs from the preregistration: ' + file); - } + const drivers = attestDrivers(__dirname, selected.inspectedArtifacts, qualificationDrivers); selectedPlanProvenance(undefined, selected); + return drivers; } function selectedPlanProvenance(file, selected = require('./configuration.cjs')) { @@ -74,13 +103,12 @@ function selectedPlanProvenance(file, selected = require('./configuration.cjs')) requireQualificationPolicy(selected); assert(!lstatSync(file).isSymbolicLink(), 'preregistered plan must be a committed regular file'); file = realpathSync(file); - const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.startsWith('GIT_'))); - Object.assign(env, { GIT_CONFIG_GLOBAL: devNull, GIT_CONFIG_NOSYSTEM: '1' }); let directory = path.dirname(file); - const git = (...args) => execFileSync('git', ['--no-pager', '--literal-pathspecs', '-C', directory, - '-c', 'log.showSignature=false', '-c', 'core.fsmonitor=false', ...args], - { env, stdio: ['ignore', 'pipe', 'ignore'], maxBuffer: 1024 * 1024 }); + while (!require('node:fs').existsSync(path.join(directory, '.git')) && path.dirname(directory) !== directory) { + directory = path.dirname(directory); + } try { + const git = pinnedGit(directory); const text = (...args) => git(...args).toString('utf8').replace(/\r?\n$/u, ''); const root = text('rev-parse', '--show-toplevel'); const relative = path.relative(root, file).split(path.sep).join('/'); @@ -128,7 +156,8 @@ function requireQualificationPolicy(selected) { function selectedPlanSha256() { return selectedPlanProvenance().sha256; } -module.exports = { requirePinnedImplementation, requirePinnedRuntime, requirePinnedExecution, requirePinnedDrivers, selectedPlanSha256, selectedPlanProvenance }; +module.exports = { requirePinnedImplementation, requirePinnedRuntime, requirePinnedExecution, requirePinnedCliProbe, + preparePinnedExecution, requirePinnedDrivers, qualificationProvenance, qualificationDrivers, selectedPlanSha256, selectedPlanProvenance }; if (require.main === module) { assert.equal(process.argv[2], '--plan-provenance'); diff --git a/quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.cjs b/quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.cjs index f23a4f9a6..9790075c2 100644 --- a/quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.cjs +++ b/quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.cjs @@ -1,16 +1,18 @@ 'use strict'; +if (require.main === module) throw new Error('Use the independently reviewed immutable bootstrap; direct qualification runners are unsupported.'); + const { readFileSync } = require('node:fs'); -const { spawn } = require('node:child_process'); +const { spawnQualificationDriver } = require('../execution-provenance.cjs'); const path = require('node:path'); const assert = require('node:assert/strict'); const { createHash } = require('node:crypto'); const { corpus } = require('./configuration.cjs'); const plan = require('./configuration.cjs'); -const { requirePinnedRuntime, requirePinnedDrivers, selectedPlanProvenance } = require('./implementation.cjs'); +const { requirePinnedRuntime, qualificationProvenance, selectedPlanProvenance } = require('./implementation.cjs'); function validateCorpusProvenance(directory, result) { - requirePinnedDrivers(plan); + const provenance = qualificationProvenance(plan); const hash = input => createHash('sha256').update(input).digest('hex'); const root = path.resolve(__dirname, '../../../../..'); const planFile = process.env.COPILOT_EV07_PLAN ? path.resolve(process.env.COPILOT_EV07_PLAN) : path.join(__dirname, 'plan.json'); @@ -27,6 +29,7 @@ function validateCorpusProvenance(directory, result) { assert.equal(start.runner, hash(readFileSync(path.join(__dirname, 'corpus.cjs')))); assert.equal(start.runner, plan.inspectedArtifacts?.['corpus.cjs']); assert.equal(start.implementation, plan.implementation); + assert.deepEqual(start.provenance, provenance); assert.equal(start.corpus, corpus.sha256); assert.equal(start.gitHubMutationPorts, 0); assert.deepEqual(start.configuration, plan.detector); @@ -78,28 +81,27 @@ function requirePassingCorpus(campaign) { return directory; } -function launchFullReview(campaign, launch = spawn) { +function launchFullReview(campaign, launch) { const directory = requirePassingCorpus(campaign); - return launch(process.execPath, [path.join(__dirname, 'full-review.cjs'), directory], { stdio: 'inherit' }); + return launch ? launch(process.execPath, [path.join(__dirname, 'full-review.cjs'), directory], { stdio: 'inherit' }) + : spawnQualificationDriver('full-review-runner', [directory], { stdio: 'inherit' }); } -if (require.main === module) { - try { - const child = launchFullReview(process.argv[2]); +async function run(campaign) { + require('../execution-provenance.cjs').requireBootstrapAdmission(path.resolve(__dirname, '../../../../..')); + const child = launchFullReview(campaign); + return new Promise((resolve, reject) => { const interrupt = () => child.kill('SIGINT'); const terminate = () => child.kill('SIGTERM'); process.once('SIGINT', interrupt); process.once('SIGTERM', terminate); - child.once('error', () => { process.exitCode = 1; }); + child.once('error', reject); child.once('close', (code, signal) => { process.removeListener('SIGINT', interrupt); process.removeListener('SIGTERM', terminate); - process.exitCode = code ?? (signal === 'SIGINT' ? 130 : signal === 'SIGTERM' ? 143 : 1); + resolve(code ?? (signal === 'SIGINT' ? 130 : signal === 'SIGTERM' ? 143 : 1)); }); - } catch (error) { - console.error(error.message); - process.exitCode = 1; - } + }); } -module.exports = { launchFullReview, requirePassingCorpus }; +module.exports = { run, launchFullReview, requirePassingCorpus }; diff --git a/quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.test.cjs b/quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.test.cjs index 0b4712bc4..64321987e 100644 --- a/quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.test.cjs +++ b/quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.test.cjs @@ -50,7 +50,7 @@ for (const [name, data] of [ assert.throws(() => launchFullReview(directory, launch), /independent corpus/); const direct = spawnSync(process.execPath, [path.join(__dirname, 'full-review.cjs'), directory], { encoding: 'utf8' }); assert.equal(direct.status, 1); - assert.match(direct.stderr, /independent corpus/u); + assert.match(direct.stderr, /immutable bootstrap/u); assert.equal(existsSync(path.join(directory, 'full-review')), false); }); } @@ -67,9 +67,11 @@ function boundCorpusFixture(t) { const { evaluateBugbotBenchmark } = require(path.join(root, 'src/tooling/bugbot_benchmark')); const directory = fixture(t); const hash = value => createHash('sha256').update(value).digest('hex'); - const plan = { ...require('./configuration.cjs'), runtime: { codex: 'codex-cli 0.169.1', codexSha256: 'a'.repeat(64) }, - inspectedArtifacts: Object.fromEntries(['configuration.cjs', 'implementation.cjs', 'load.cjs', 'corpus.cjs', - 'review.cjs', 'full-review.cjs', 'launch-full-review.cjs', 'score-review.py'] + const sourceProof = require('../execution-provenance.cjs'); + const implementation = execFileSync('git', ['-C', root, 'rev-parse', 'HEAD'], { encoding: 'utf8' }).trim(); + const plan = { ...require('./configuration.cjs'), implementation, + runtime: { ...sourceProof.runtimeIdentity(root), codex: 'codex-cli 0.169.1', codexSha256: 'a'.repeat(64) }, + inspectedArtifacts: Object.fromEntries(require('./implementation.cjs').qualificationDrivers .map(file => [file, hash(readFileSync(path.join(__dirname, file)))])) }; const planFile = path.join(directory, 'plan.json'); writeFileSync(planFile, JSON.stringify(plan)); @@ -78,7 +80,8 @@ function boundCorpusFixture(t) { const predictions = { schemaVersion: 1, predictions: Object.fromEntries(corpus.cases.map(item => [item.id, item.expected.map(finding => ({ ...finding, confidence: 1 }))])) }; const rows = [{ event: 'start', cli: plan.runtime.codex, configuration: plan.detector, plan: hash(readFileSync(planFile)), - preregistration, + preregistration, provenance: { ...sourceProof.attestSource(root, implementation), drivers: plan.inspectedArtifacts, + runtime: sourceProof.runtimeIdentity(root) }, runner: hash(readFileSync(path.join(__dirname, 'corpus.cjs'))), implementation: plan.implementation, corpus: plan.corpus.sha256, gitHubMutationPorts: 0 }, ...corpus.cases.map((item, index) => ({ event: 'case', index, id: item.id, status: 'completed', predictions: predictions.predictions[item.id], runtimeProofs: [{ executableSha256: plan.runtime.codexSha256 }], calls: [{ model: plan.detector.model, diff --git a/quality/bugbot/evidence/issue-950/ev07-shadow/review-runtime.test.cjs b/quality/bugbot/evidence/issue-950/ev07-shadow/review-runtime.test.cjs index 16af4aaac..d7538ee9a 100644 --- a/quality/bugbot/evidence/issue-950/ev07-shadow/review-runtime.test.cjs +++ b/quality/bugbot/evidence/issue-950/ev07-shadow/review-runtime.test.cjs @@ -1,12 +1,14 @@ 'use strict'; const assert = require('node:assert/strict'); -const { mkdtempSync, existsSync, rmSync, writeFileSync } = require('node:fs'); +const { mkdtempSync, existsSync, rmSync, writeFileSync, mkdirSync, readFileSync } = require('node:fs'); +const { execFileSync, spawnSync, spawn } = require('node:child_process'); const { createHash } = require('node:crypto'); const { tmpdir } = require('node:os'); const path = require('node:path'); const { test } = require('node:test'); -const { prepareDetector } = require('./review.cjs'); +const { admitPreparedDetector } = require('./review.cjs'); +const { requirePinnedCliProbe, preparePinnedExecution } = require('./implementation.cjs'); const plan = { ...require('./configuration.cjs'), runtime: { codex: 'codex-cli 0.169.1' } }; for (const mode of ['pinned', 'foreign', 'missing', 'wrong-executable', 'missing-digest', 'launcher', 'wrong-launcher', 'missing-launcher']) { @@ -30,18 +32,143 @@ for (const mode of ['pinned', 'foreign', 'missing', 'wrong-executable', 'missing runtimeContract: mode === 'missing' ? undefined : { version: mode === 'foreign' ? 'codex-cli 0.0.0' : plan.runtime.codex, } }; - const request = { prompt: 'controlled request' }, observations = []; - const planner = { prepare(input) { assert.equal(input, request); return execution; } }; + const observations = []; const selected = { ...plan, runtime }; if (['pinned', 'launcher'].includes(mode)) { - assert.equal(prepareDetector(planner, request, row => observations.push(row), selected), execution); + assert.equal(admitPreparedDetector(execution, row => observations.push(row), selected), execution); assert.equal(observations[0].runtime, plan.runtime.codex); assert.equal(observations[0].runtimeProof.executableSha256, runtime.codexSha256); assert.equal(existsSync(runtimeDirectory), true); } else { - assert.throws(() => prepareDetector(planner, request, row => observations.push(row), selected), /CLI version|digest/u); + assert.throws(() => admitPreparedDetector(execution, row => observations.push(row), selected), /CLI version|digest/u); assert.equal(existsSync(runtimeDirectory), false); assert.deepEqual(observations, []); } }); } + +for (const mode of ['direct', 'changed-direct', 'launcher', 'changed-launcher']) { + test('CLI version probe admits bytes before executing ' + mode, t => { + const directory = mkdtempSync(path.join(tmpdir(), 'ev07-probe-proof-')); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const marker = path.join(directory, 'executed'), file = path.join(directory, 'codex'); + const usesLauncher = mode.includes('launcher'); + const program = usesLauncher + ? 'require("node:fs").writeFileSync(' + JSON.stringify(marker) + ', "version");console.log("codex-cli fixture");\n' + : '#!/bin/sh\nprintf version > "' + marker + '"\nprintf "codex-cli fixture\\n"\n'; + writeFileSync(file, program, { mode: 0o700 }); + const hashFile = filename => createHash('sha256').update(require('node:fs').readFileSync(filename)).digest('hex'); + const runtime = { codex: 'codex-cli fixture', executable: usesLauncher ? process.execPath : file, + codexSha256: hashFile(usesLauncher ? process.execPath : file), + ...(usesLauncher && { launcher: file, launcherSha256: hashFile(file) }) }; + if (mode.startsWith('changed')) writeFileSync(file, program + '\n'); + if (mode.startsWith('changed')) { + assert.throws(() => requirePinnedCliProbe(runtime), /digest/u); + assert.equal(existsSync(marker), false); + } else { + assert.equal(requirePinnedCliProbe(runtime).version, runtime.codex); + assert.equal(existsSync(marker), true); + } + }); +} + +test('the old PATH version probe executes a shim, while the pinned probe invokes only the admitted CLI', t => { + const directory = mkdtempSync(path.join(tmpdir(), 'ev07-path-probe-')); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const bin = path.join(directory, 'bin'), shimMarker = path.join(directory, 'shim-ran'), admitted = path.join(directory, 'admitted'); + mkdirSync(bin); + writeFileSync(path.join(bin, 'codex'), '#!/bin/sh\ntouch "' + shimMarker + '"\nprintf "codex-cli fixture\\n"\n', { mode: 0o700 }); + writeFileSync(admitted, '#!/bin/sh\nprintf "codex-cli fixture\\n"\n', { mode: 0o700 }); + const env = { ...process.env, PATH: bin + path.delimiter + process.env.PATH }; + assert.equal(execFileSync('codex', ['--version'], { env, encoding: 'utf8' }).trim(), 'codex-cli fixture'); + assert.equal(existsSync(shimMarker), true); + rmSync(shimMarker); + const runtime = { codex: 'codex-cli fixture', executable: admitted, + codexSha256: createHash('sha256').update(readFileSync(admitted)).digest('hex') }; + const result = spawnSync(process.execPath, ['-e', 'require(process.argv[1]).requirePinnedCliProbe(JSON.parse(process.argv[2]))', + path.resolve(__dirname, 'implementation.cjs'), JSON.stringify(runtime)], { env, encoding: 'utf8' }); + assert.equal(result.status, 0, result.stderr); + assert.equal(existsSync(shimMarker), false); +}); + +for (const mode of ['pinned', 'foreign']) test('ordinary planner admits CLI bytes before its internal version probe: ' + mode, t => { + require('./load.cjs'); + const root = path.resolve(__dirname, '../../../../..'); + const { AgentExecutionPlanner } = require(path.join(root, 'src/infrastructure/agents/agent_execution_planner')); + const { resolveAgentExecutablePath, readAgentExecutableVersion } = require(path.join(root, + 'src/infrastructure/agents/agent_executable_invocation')); + const directory = mkdtempSync(path.join(tmpdir(), 'ev07-planner-probe-')); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const executable = path.join(directory, 'codex'), marker = path.join(directory, 'version-executed'); + writeFileSync(executable, '#!/bin/sh\nprintf version > "' + marker + '"\nprintf "codex-cli fixture\\n"\n', { mode: 0o700 }); + const runtime = { executable, codex: 'codex-cli fixture', + codexSha256: mode === 'pinned' ? createHash('sha256').update(readFileSync(executable)).digest('hex') : 'a'.repeat(64) }; + const planner = new AgentExecutionPlanner({ resolveExecutable: resolveAgentExecutablePath, + readVersion: readAgentExecutableVersion, resolveWorkspace: () => directory }); + const request = { configuration: { provider: 'codex', modelProvider: 'openai', model: 'gpt-6-luna', effort: 'xhigh' }, + capability: 'findings', prompt: 'controlled planner preparation; no model invocation', timeoutMs: 1000, + environment: { PATH: directory + path.delimiter + process.env.PATH }, cwd: directory }; + if (mode === 'foreign') { + const old = planner.prepare(request); + try { assert.equal(existsSync(marker), true, 'ordinary preparation already executed the unpinned version probe'); } + finally { rmSync(old.runtimeDirectory, { recursive: true, force: true }); } + rmSync(marker); + assert.throws(() => preparePinnedExecution(planner, request, runtime), /digest/u); + assert.equal(existsSync(marker), false, 'the corrected admission runs before ordinary preparation'); + } else { + const execution = preparePinnedExecution(planner, request, runtime); + try { + assert.equal(execution.runtimeContract.version, runtime.codex); + assert.equal(execution.executable, require('node:fs').realpathSync(executable)); + assert.equal(existsSync(marker), true); + } finally { rmSync(execution.runtimeDirectory, { recursive: true, force: true }); } + } +}); + + +for (const mode of ['direct-stall', 'launcher-ignores-term']) { + test('an admitted --version health probe fails closed for ' + mode, async t => { + const directory = mkdtempSync(path.join(tmpdir(), 'ev07-stalled-version-')); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const marker = path.join(directory, 'started.json'), executable = path.join(directory, 'cli'); + const launcher = mode === 'launcher-ignores-term'; + const program = (launcher ? '' : '#!' + process.execPath + '\n') + + 'require("node:fs").writeFileSync(' + JSON.stringify(marker) + + ',JSON.stringify({pid:process.pid,args:process.argv.slice(2)}));' + + (launcher ? 'process.on("SIGTERM",()=>{});' : '') + 'setInterval(()=>{},1000);\n'; + writeFileSync(executable, program, { mode: 0o700 }); + const digest = file => createHash('sha256').update(readFileSync(file)).digest('hex'); + const runtime = { executable: launcher ? process.execPath : executable, codex: 'codex-cli stalled-fixture', + codexSha256: digest(launcher ? process.execPath : executable), + ...(launcher && { launcher: executable, launcherSha256: digest(executable) }) }; + const script = 'try { require(process.argv[1]).requirePinnedCliProbe(JSON.parse(process.argv[2]));' + + 'console.log(JSON.stringify({unexpectedSuccess:true})); } catch(error) {' + + 'console.log(JSON.stringify({code:error.code,signal:error.signal})); }'; + const child = spawn(process.execPath, ['-e', script, path.resolve(__dirname, 'implementation.cjs'), + JSON.stringify(runtime)], { detached: true, stdio: ['ignore', 'pipe', 'pipe'] }); + const cleanup = () => { + if (child.exitCode === null && child.signalCode === null && child.pid) { + try { process.kill(-child.pid, 'SIGKILL'); } catch (error) { if (error.code !== 'ESRCH') throw error; } + } + }; + t.after(cleanup); + let timer, timedOut = false, stdout = '', stderr = ''; + child.stdout.on('data', bytes => { stdout += bytes; }); + child.stderr.on('data', bytes => { stderr += bytes; }); + try { + // Cleanup of these owned model-free processes only, never a research deadline. + const result = await new Promise((resolve, reject) => { + timer = setTimeout(() => { timedOut = true; cleanup(); }, 30_000); + child.once('error', reject); + child.once('close', (code, signal) => resolve({ code, signal })); + }); + assert.equal(existsSync(marker), true, 'the byte-admitted fixture actually entered --version'); + const started = JSON.parse(readFileSync(marker)); + assert.deepEqual(started.args, ['--version']); + assert.equal(timedOut, false, 'the health probe stayed blocked; owned fixture cleanup was required'); + assert.equal(result.code, 0, stderr); + assert.deepEqual(JSON.parse(stdout), { code: 'ETIMEDOUT', signal: 'SIGKILL' }); + assert.throws(() => process.kill(started.pid, 0), { code: 'ESRCH' }, 'no stalled CLI remains alive'); + } finally { clearTimeout(timer); cleanup(); } + }); +} diff --git a/quality/bugbot/evidence/issue-950/ev07-shadow/review.cjs b/quality/bugbot/evidence/issue-950/ev07-shadow/review.cjs index f6aa3426e..ce9075305 100644 --- a/quality/bugbot/evidence/issue-950/ev07-shadow/review.cjs +++ b/quality/bugbot/evidence/issue-950/ev07-shadow/review.cjs @@ -1,31 +1,18 @@ 'use strict'; +if (require.main === module) throw new Error('Use the independently reviewed immutable bootstrap; direct qualification runners are unsupported.'); + const assert = require('node:assert/strict'); const { execFileSync } = require('node:child_process'); const { createHash } = require('node:crypto'); const fs = require('node:fs'); const path = require('node:path'); -const { getOctokit } = require('@actions/github'); -const { requirePinnedImplementation, requirePinnedExecution, requirePinnedDrivers, selectedPlanProvenance } = require('./implementation.cjs'); -require('./load.cjs'); - -const { Ai } = require('../../../../../src/data/model/ai'); -const { AgentCliClient } = require('../../../../../src/data/repository/agent_cli_client'); -const { AgentExecutionPlanner } = require('../../../../../src/infrastructure/agents/agent_execution_planner'); -const { codexInvocation } = require('../../../../../src/infrastructure/bugbot/shadow/session.cjs'); -const { composeBugbotReview } = require('../../../../../src/infrastructure/composition/bugbot_review_composition'); -const { createFindingsQueryPort } = require('../../../../../src/infrastructure/composition/agent_capability_composition_root'); -const { readGithubActionAiInputs } = require('../../../../../src/actions/github_action_ai_inputs'); -const { projectBugbotReviewOperationContext } = require('../../../../../src/application/usecases/steps/commit/bugbot/bugbot_review_operation_context'); -const { createInstalledBugbotShadowVerifier } = require('../../../../../src/infrastructure/bugbot/installed_bugbot_shadow_adapter'); -const { createGitHubBugbotCiReader } = require('../../../../../src/infrastructure/bugbot/github_bugbot_ci_reader'); -const { askManagedBugbotShadowTurn } = require('../../../../../src/infrastructure/bugbot/shadow_verification_adapter'); +const { pinnedGit, admitQualificationModules } = require('../execution-provenance.cjs'); +const { qualificationProvenance, requirePinnedExecution, preparePinnedExecution, selectedPlanProvenance } = require('./implementation.cjs'); const plan = require('./configuration.cjs'); const hash = input => createHash('sha256').update(input).digest('hex'); -const git = (cwd, ...args) => execFileSync('git', ['-C', cwd, '-c', 'core.quotePath=true', ...args], { - encoding: 'utf8', maxBuffer: 16 * 1024 * 1024, -}); +const git = (cwd, ...args) => pinnedGit(cwd)('-c', 'core.quotePath=true', ...args).toString('utf8'); const snapshot = (source, value, count = 0) => ({ value, coverage: { source, status: 'complete', pagesFetched: 1, itemsFetched: count, itemsRetained: count, omittedItems: 0, truncatedItems: 0, limitReached: false }, @@ -34,9 +21,14 @@ const args = ['diff', '--no-ext-diff', '--no-textconv', '--no-renames']; const originalPullNumbers = { X1: 24, P1: 28, J1: 28 }; function prepareDetector(planner, request, log, selected = plan) { - const execution = planner.prepare(request); + const execution = preparePinnedExecution(planner, request, selected.runtime); + return admitPreparedDetector(execution, log, selected); +} + +function admitPreparedDetector(execution, log, selected = plan) { try { const runtimeProof = requirePinnedExecution(execution, selected.runtime); + const { codexInvocation } = require('../../../../../src/infrastructure/bugbot/shadow/session.cjs'); const flags = codexInvocation(execution.argv); assert.deepEqual(flags, { model: selected.detector.model, modelProvider: selected.detector.modelProvider, reasoningEffort: selected.detector.effort, sandbox: 'read-only' }); @@ -90,8 +82,25 @@ function readSubject(id, checkout) { } async function run(checkout, id, repetition, output, preflight = false) { - requirePinnedDrivers(plan); - const implementation = requirePinnedImplementation(); + require('../execution-provenance.cjs').requireBootstrapAdmission(path.resolve(__dirname, '../../../../..')); + const provenance = qualificationProvenance(plan); + const implementation = provenance.implementation; + admitQualificationModules(path.resolve(__dirname, '../../../../..')); + const { getOctokit } = require('@actions/github'); + require('./load.cjs'); + + const { Ai } = require('../../../../../src/data/model/ai'); + const { AgentCliClient } = require('../../../../../src/data/repository/agent_cli_client'); + const { AgentExecutionPlanner } = require('../../../../../src/infrastructure/agents/agent_execution_planner'); + const { codexInvocation } = require('../../../../../src/infrastructure/bugbot/shadow/session.cjs'); + const { composeBugbotReview } = require('../../../../../src/infrastructure/composition/bugbot_review_composition'); + const { createFindingsQueryPort } = require('../../../../../src/infrastructure/composition/agent_capability_composition_root'); + const { readGithubActionAiInputs } = require('../../../../../src/actions/github_action_ai_inputs'); + const { projectBugbotReviewOperationContext } = require('../../../../../src/application/usecases/steps/commit/bugbot/bugbot_review_operation_context'); + const { createInstalledBugbotShadowVerifier } = require('../../../../../src/infrastructure/bugbot/installed_bugbot_shadow_adapter'); + const { createGitHubBugbotCiReader } = require('../../../../../src/infrastructure/bugbot/github_bugbot_ci_reader'); + const { askManagedBugbotShadowTurn } = require('../../../../../src/infrastructure/bugbot/shadow_verification_adapter'); + assert(checkout && plan.repositoryCohort.order.includes(id) && [1, 2, 3].includes(repetition) && output); checkout = fs.realpathSync(checkout); output = path.resolve(output); @@ -100,7 +109,7 @@ async function run(checkout, id, repetition, output, preflight = false) { fs.writeFileSync(output, '', { flag: 'wx', mode: 0o600 }); const log = row => fs.appendFileSync(output, JSON.stringify(row) + '\n'); const preregistration = selectedPlanProvenance(); - log({ event: 'start', subject, repetition, implementation, plan: preregistration.sha256, preregistration, startedAt: new Date().toISOString() }); + log({ event: 'start', subject, repetition, implementation, provenance, plan: preregistration.sha256, preregistration, startedAt: new Date().toISOString() }); const repository = { owner: 'vypdev', name: id.startsWith('U') ? 'ha-ui' : 'domus' }; const number = originalPullNumbers[id] ?? 1; const branch = 'feature/reviewed-source'; @@ -156,8 +165,10 @@ async function run(checkout, id, repetition, output, preflight = false) { const cancel = new AbortController(); process.once('SIGINT', () => cancel.abort()); process.once('SIGTERM', () => cancel.abort()); + const evidenceToken = process.env.COPILOT_EVIDENCE_TOKEN || process.env.GH_TOKEN || process.env.GITHUB_TOKEN; + assert(preflight || evidenceToken, 'read-only GitHub evidence credential unavailable'); const ci = preflight ? undefined : createGitHubBugbotCiReader({ owner: repository.owner, repo: repository.name }, - getOctokit(execFileSync('gh', ['auth', 'token', '--hostname', 'github.com'], { encoding: 'utf8' }).trim()).rest.actions); + getOctokit(evidenceToken).rest.actions); const verifier = createInstalledBugbotShadowVerifier(repository, checkout, async turn => { assert.deepEqual(turn.configuration, plan.verifier); const result = await askManagedBugbotShadowTurn(turn); @@ -177,9 +188,5 @@ async function run(checkout, id, repetition, output, preflight = false) { if (preflight) assert.equal(result.payload.bugbotShadowVerification.publicationSimulation.status, 'complete'); } -if (require.main === module) { - run(process.argv[2], process.argv[3], Number(process.argv[4]), process.argv[5], process.argv[6] === '--preflight') - .catch(error => { console.error(error.code ?? error.name, error.message); process.exitCode = 1; }); -} - -module.exports = { prepareDetector }; +module.exports = { run: (checkout, id, repetition, destination, flag) => + run(checkout, id, Number(repetition), destination, flag === '--preflight'), prepareDetector, admitPreparedDetector }; diff --git a/quality/bugbot/evidence/issue-950/execution-provenance.cjs b/quality/bugbot/evidence/issue-950/execution-provenance.cjs new file mode 100644 index 000000000..508ca8035 --- /dev/null +++ b/quality/bugbot/evidence/issue-950/execution-provenance.cjs @@ -0,0 +1,272 @@ +'use strict'; + +const assert = require('node:assert/strict'); +const { execFileSync } = require('node:child_process'); +const { createHash } = require('node:crypto'); +const fs = require('node:fs'); +const { devNull } = require('node:os'); +const path = require('node:path'); + +const hash = bytes => createHash('sha256').update(bytes).digest('hex'); +const sourcePaths = Object.freeze(['src', 'pnpm-lock.yaml', 'tsconfig.json', 'package.json']); + +function gitEnvironment() { + const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.toUpperCase().startsWith('GIT_'))); + return { ...env, GIT_CONFIG_GLOBAL: devNull, GIT_CONFIG_NOSYSTEM: '1', GIT_NO_REPLACE_OBJECTS: '1' }; +} + +function trustedGitExecutable() { + // Bootstrap from the operating system, never PATH or a campaign-provided binary. + // On macOS resolve the actual developer-tools Git rather than hashing its shim. + const file = process.platform === 'darwin' + ? execFileSync('/usr/bin/xcrun', ['--find', 'git'], { + env: { PATH: '/usr/bin:/bin' }, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], + }).trim() + : process.platform === 'linux' ? '/usr/bin/git' : undefined; + assert(file && path.isAbsolute(file), 'qualification Git bootstrap supports Linux and macOS only'); + const executable = fs.realpathSync(file); + assert(fs.lstatSync(executable).isFile(), 'trusted Git executable unavailable'); + return executable; +} + +function pinnedGit(root) { + root = fs.realpathSync(root); + const marker = path.join(root, '.git'); + const stat = fs.lstatSync(fs.existsSync(marker) ? marker : root); + assert(!stat.isSymbolicLink(), 'repository Git identity must not be a symlink'); + let directory = fs.existsSync(marker) ? marker : root; + if (stat.isFile()) { + const match = /^gitdir: (.+)\r?\n?$/u.exec(fs.readFileSync(marker, 'utf8')); + assert(match, 'invalid worktree Git identity'); + directory = path.resolve(root, match[1]); + } else assert(stat.isDirectory(), 'repository Git identity unavailable'); + const env = gitEnvironment(); + const executable = trustedGitExecutable(); + const command = (...args) => execFileSync(executable, ['--no-pager', '--no-replace-objects', '--literal-pathspecs', + '--git-dir=' + directory, '--work-tree=' + root, '-C', root, + '-c', 'core.fsmonitor=false', '-c', 'core.hooksPath=' + devNull, '-c', 'core.quotePath=false', '-c', 'log.showSignature=false', ...args], + { env, stdio: ['pipe', 'pipe', 'pipe'], maxBuffer: 256 * 1024 * 1024 }); + command.executable = executable; + return command; +} + +function attestSource(root, implementation) { + assert.match(implementation ?? '', /^[a-f0-9]{40}$/u, 'invalid implementation pin'); + root = fs.realpathSync(root); + const git = pinnedGit(root); + const text = (...args) => git(...args).toString('utf8').trim(); + assert.equal(text('rev-parse', '--verify', implementation + '^{commit}'), implementation, + 'preregistered implementation commit unavailable'); + const rows = git('ls-tree', '-r', '-z', '--full-tree', implementation, '--', ...sourcePaths) + .toString('utf8').split('\0').filter(Boolean).map(row => { + const tab = row.indexOf('\t'); + const [mode, type, blob] = row.slice(0, tab).split(' '); + const name = row.slice(tab + 1); + assert(['100644', '100755'].includes(mode) && type === 'blob', 'source must be a regular committed file'); + assert(!path.isAbsolute(name) && !name.split('/').some(part => ['..', '.git'].includes(part)), 'unsafe source path'); + return { path: name, blob }; + }); + const expected = new Map(rows.map(row => [row.path, row])); + assert(expected.has('pnpm-lock.yaml') && expected.has('tsconfig.json') && rows.some(row => row.path.startsWith('src/')), + 'preregistered source inventory unavailable'); + // Compare the physical inventory, including ignored files and symlinked parents. + // Git's assume-unchanged, sparse/skip-worktree and stat cache are not byte evidence. + const actual = []; + function visit(relative) { + const file = path.join(root, relative); + let stat; + try { stat = fs.lstatSync(file); } + catch (error) { + if (error.code !== 'ENOENT') throw error; + assert(!expected.has(relative), 'application source or lockfile does not match the preregistered implementation'); + return; + } + assert(!stat.isSymbolicLink(), 'application source symlink is not admitted'); + if (stat.isDirectory()) { + for (const name of fs.readdirSync(file)) visit(relative + '/' + name); + } else { + assert(stat.isFile() && expected.has(relative), 'untracked application source or lockfile'); + actual.push(relative); + } + } + for (const name of sourcePaths) visit(name); + assert.equal(actual.length, expected.size, 'application source or lockfile does not match the preregistered implementation'); + assert.equal(text('diff', '--cached', '--no-ext-diff', '--no-textconv', '--name-only', implementation, '--', ...sourcePaths), '', + 'staged application source or lockfile does not match the preregistered implementation'); + // One batch reads immutable blobs without filters, attributes, textconv or a checkout. + const contents = execFileSync(git.executable, ['--no-replace-objects', '--git-dir=' + text('rev-parse', '--absolute-git-dir'), + '-c', 'core.fsmonitor=false', 'cat-file', '--batch'], { + env: gitEnvironment(), input: rows.map(row => row.blob).join('\n') + '\n', maxBuffer: 256 * 1024 * 1024, + }); + let offset = 0; + for (const row of rows) { + const end = contents.indexOf(10, offset); + const [blob, type, length] = contents.subarray(offset, end).toString('utf8').split(' '); + const size = Number(length); + assert(blob === row.blob && type === 'blob' && Number.isSafeInteger(size) && size >= 0, 'invalid pinned source blob'); + const bytes = contents.subarray(end + 1, end + 1 + size); + const local = fs.readFileSync(path.join(root, row.path)); + assert(bytes.equals(local), 'application source or lockfile does not match the preregistered implementation: ' + row.path); + row.sha256 = hash(local); + offset = end + size + 2; + } + assert.equal(offset, contents.length, 'invalid pinned source stream'); + return { implementation, tree: text('rev-parse', implementation + '^{tree}'), + sourceSha256: hash(JSON.stringify(rows)), dependencyLockSha256: expected.get('pnpm-lock.yaml').sha256, + compilerConfigurationSha256: expected.get('tsconfig.json').sha256 }; +} + +function dependencyIdentity(root) { + root = fs.realpathSync(root); + const directory = path.join(root, 'node_modules'); + assert(fs.lstatSync(directory).isDirectory() && !fs.lstatSync(directory).isSymbolicLink(), + 'qualification dependencies must have a physical installation'); + const rows = []; + function visit(relative) { + const file = path.join(directory, relative), stat = fs.lstatSync(file); + if (stat.isSymbolicLink()) { + const link = fs.readlinkSync(file), lexical = path.relative(directory, path.resolve(path.dirname(file), link)); + if (relative === path.join('.pnpm', 'node_modules', '@vypdev', 'copilot-browser-extension') + && fs.realpathSync(file) === path.join(root, 'extension')) { + // Normal pnpm workspaces expose this unused portal. Its code is + // outside admission and the entrypoint loader fence forbids it. + rows.push([relative, 'unadmitted-workspace', link, 'extension']); + return; + } + assert(lexical && !path.isAbsolute(lexical) && !lexical.split(path.sep).includes('..'), + 'dependency symlink escapes the admitted installation'); + let target; + try { target = path.relative(directory, fs.realpathSync(file)); } + catch (error) { if (error.code !== 'ENOENT') throw error; } + assert(target === undefined || target && !path.isAbsolute(target) && !target.split(path.sep).includes('..'), + 'dependency symlink escapes the admitted installation'); + // pnpm can retain links to platform-excluded optional packages. + // Bind the missing target as well; materializing it changes this proof. + rows.push([relative, 'link', link, target ?? 'missing:' + lexical]); + } else if (stat.isDirectory()) { + for (const name of fs.readdirSync(file).sort()) visit(path.join(relative, name)); + } else { + assert(stat.isFile(), 'dependency must be a regular file or internal symlink'); + rows.push([relative, 'file', stat.mode & 0o777, hash(fs.readFileSync(file))]); + } + } + visit(''); + // These ancestors could otherwise shadow the admitted root dependencies. + for (const relative of ['quality', 'quality/bugbot', 'quality/bugbot/evidence', + 'quality/bugbot/evidence/issue-950', 'quality/bugbot/evidence/issue-950/ev07-shadow', + 'quality/bugbot/evidence/issue-950/normal-flow-shadow']) { + assert(!fs.existsSync(path.join(root, relative, 'node_modules')), 'unadmitted qualification dependency shadow'); + } + assert(!process.env.NODE_PATH && !process.env.NODE_OPTIONS, 'qualification requires a clean Node loader environment'); + assert(!process.execArgv.some(arg => /^(?:--require|--import|--loader|--experimental-loader)(?:=|$)|^-r/u.test(arg)), + 'qualification Node preload is not admitted'); + return { dependencyTreeSha256: hash(JSON.stringify(rows)), dependencyEntries: rows.length }; +} + +function admitQualificationModules(root) { + root = fs.realpathSync(root); + const { registerHooks } = require('node:module'), { fileURLToPath } = require('node:url'); + assert.equal(typeof registerHooks, 'function', 'qualification requires synchronous Node module hooks'); + const admitted = ['src', 'node_modules', 'quality/bugbot/evidence/issue-950'] + .map(relative => path.join(root, relative) + path.sep); + return registerHooks({ resolve(specifier, context, nextResolve) { + const result = nextResolve(specifier, context); + if (!result.url.startsWith('node:')) { + assert(result.url.startsWith('file:'), 'qualification module must have a physical admitted path'); + const file = fs.realpathSync(fileURLToPath(result.url)); + assert(admitted.some(prefix => file.startsWith(prefix)), 'module is outside the admitted qualification closure'); + } + return result; + } }); +} + +function runtimeIdentity(root) { + const dependencies = dependencyIdentity(root); + const typescript = require.resolve('typescript', { paths: [root] }); + assert(fs.realpathSync(typescript).startsWith(fs.realpathSync(path.join(root, 'node_modules')) + path.sep), + 'compiler is outside the admitted dependency tree'); + const gitExecutable = trustedGitExecutable(); + return { ...dependencies, node: process.version, nodeSha256: hash(fs.readFileSync(process.execPath)), + gitExecutable, gitSha256: hash(fs.readFileSync(gitExecutable)), + git: execFileSync(gitExecutable, ['--version'], { env: gitEnvironment(), encoding: 'utf8' }).trim(), + typescript: JSON.parse(fs.readFileSync(require.resolve('typescript/package.json', { paths: [root] }))).version, + typescriptSha256: hash(fs.readFileSync(typescript)) }; +} + +function attestRuntime(root, selected) { + const actual = runtimeIdentity(root); + for (const [name, value] of Object.entries(actual)) { + assert.equal(value, selected?.[name], 'preregistered runtime differs: ' + name); + } + return actual; +} + +function attestDrivers(directory, expected, files) { + return Object.fromEntries(files.map(file => { + const target = path.join(directory, file); + assert(fs.lstatSync(target).isFile(), 'qualification driver must be a regular file: ' + file); + const actual = hash(fs.readFileSync(target)); + assert.equal(actual, expected?.[file], 'qualification driver differs from the preregistration: ' + file); + return [file, actual]; + })); +} + +function attestCommittedArtifact(root, file, expectedSha256, commit) { + assert.match(expectedSha256 ?? '', /^[a-f0-9]{64}$/u, 'reviewed committed artifact digest required'); + assert.match(commit ?? '', /^[a-f0-9]{40}$/u, 'reviewed artifact commit required'); + const requestedRoot = path.resolve(root); + root = fs.realpathSync(root); + const requestedFile = path.resolve(file); + const canonicalFile = requestedFile.startsWith(requestedRoot + path.sep) + ? root + requestedFile.slice(requestedRoot.length) : requestedFile; + const relative = path.relative(root, canonicalFile).split(path.sep).join('/'); + assert(relative && !path.isAbsolute(relative) && !relative.split('/').some(part => ['', '..', '.git'].includes(part)), + 'committed artifact must be inside the reviewed repository'); + let current = root; + for (const part of relative.split('/')) { + current = path.join(current, part); + assert(!fs.lstatSync(current).isSymbolicLink(), 'committed artifact symlink refused'); + } + assert(fs.lstatSync(current).isFile(), 'committed artifact must be regular'); + const git = pinnedGit(root), rows = git('ls-tree', '-z', commit, '--', relative).toString('utf8').split('\0').filter(Boolean); + assert.equal(rows.length, 1, 'reviewed committed artifact unavailable'); + const [header, name] = rows[0].split('\t'), [mode, type, blob] = header.split(' '); + assert(['100644', '100755'].includes(mode) && type === 'blob' && name === relative, + 'reviewed committed artifact must be a regular blob'); + const bytes = fs.readFileSync(current); + assert.equal(hash(bytes), expectedSha256, 'reviewed committed artifact digest differs'); + assert(git('cat-file', 'blob', blob).equals(bytes), 'reviewed committed artifact bytes differ'); + return { bytes, provenance: { commit, path: relative, blob, sha256: expectedSha256 } }; +} + +// Process-local transport installed only after the external bootstrap admits +// this immutable module. A checkout file is never execution authority. This +// context is not an independent attestation of the host or of human review. +let bootstrapContext; +function initializeBootstrapContext(root, proof, spawnDriver) { + assert(!bootstrapContext, 'bootstrap context is already installed'); + root = fs.realpathSync(root); + const main = fs.realpathSync(require.main.filename); + assert(!main.startsWith(root + path.sep), 'bootstrap must execute outside the admitted checkout'); + assert.equal(hash(fs.readFileSync(main)), proof.bootstrapSha256, 'executing bootstrap identity differs'); + assert.equal(hash(fs.readFileSync(path.join(root, 'quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs'))), + proof.bootstrapSha256, 'admitted bootstrap identity differs'); + assert.equal(typeof spawnDriver, 'function'); + bootstrapContext = Object.freeze({ root, proof: Object.freeze({ ...proof }), spawnDriver }); +} +function bootstrapAdmission(root) { + return bootstrapContext?.root === fs.realpathSync(root) ? bootstrapContext.proof : undefined; +} +function requireBootstrapAdmission(root) { + const proof = bootstrapAdmission(root); + assert(proof, 'qualification requires the independently reviewed immutable bootstrap'); + return proof; +} +function spawnQualificationDriver(mode, args, options) { + assert(bootstrapContext, 'qualification requires the independently reviewed immutable bootstrap'); + return bootstrapContext.spawnDriver(mode, args, options); +} + +module.exports = { attestSource, attestRuntime, attestDrivers, runtimeIdentity, dependencyIdentity, admitQualificationModules, + pinnedGit, gitEnvironment, trustedGitExecutable, attestCommittedArtifact, requireBootstrapAdmission, + bootstrapAdmission, initializeBootstrapContext, spawnQualificationDriver }; diff --git a/quality/bugbot/evidence/issue-950/normal-flow-shadow/paths.test.cjs b/quality/bugbot/evidence/issue-950/normal-flow-shadow/paths.test.cjs index d97a18dd5..71b5070dd 100644 --- a/quality/bugbot/evidence/issue-950/normal-flow-shadow/paths.test.cjs +++ b/quality/bugbot/evidence/issue-950/normal-flow-shadow/paths.test.cjs @@ -2,10 +2,246 @@ const assert = require('node:assert/strict'); const { execFileSync, spawnSync } = require('node:child_process'); -const { mkdtempSync, mkdirSync, writeFileSync, readFileSync, existsSync, rmSync } = require('node:fs'); +const { mkdtempSync, mkdirSync, writeFileSync, readFileSync, existsSync, rmSync, symlinkSync, cpSync, constants } = require('node:fs'); +const { createHash } = require('node:crypto'); const { tmpdir } = require('node:os'); const path = require('node:path'); const { test } = require('node:test'); +const { attestSource, attestRuntime, attestDrivers, runtimeIdentity, dependencyIdentity, trustedGitExecutable } = require('../execution-provenance.cjs'); + +function sourceFixture(t) { + const directory = mkdtempSync(path.join(tmpdir(), 'bugbot-source-proof-')); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const root = path.join(directory, 'actual'), clean = path.join(directory, 'clean'); + mkdirSync(path.join(root, 'src'), { recursive: true }); + const git = (...args) => execFileSync('git', ['-C', root, ...args], { encoding: 'utf8', + env: { ...process.env, GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_NOSYSTEM: '1' } }).trim(); + git('init', '--quiet', '--template='); + writeFileSync(path.join(root, 'src/app.ts'), 'export const value = 1;\n'); + writeFileSync(path.join(root, 'pnpm-lock.yaml'), 'lockfileVersion: 9\n'); + writeFileSync(path.join(root, 'tsconfig.json'), '{}\n'); + git('add', '.'); + git('-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '--quiet', '-m', 'Pinned fixture'); + const pin = git('rev-parse', 'HEAD'); + execFileSync('git', ['clone', '--quiet', '--template=', root, clean]); + return { directory, root, clean, pin, git, file: path.join(root, 'src/app.ts') }; +} + +test('clean admission records the actual tree, source inventory and lockfile bytes', t => { + const f = sourceFixture(t), proof = attestSource(f.root, f.pin); + assert.equal(proof.implementation, f.pin); + assert.equal(proof.tree, f.git('rev-parse', f.pin + '^{tree}')); + assert.equal(proof.dependencyLockSha256, createHash('sha256').update(readFileSync(path.join(f.root, 'pnpm-lock.yaml'))).digest('hex')); + assert.match(proof.sourceSha256, /^[a-f0-9]{64}$/u); +}); + +for (const mode of ['dirty', 'staged', 'untracked', 'ignored', 'lockfile', 'compiler', 'assume-unchanged', 'symlink']) { + test('physical source admission refuses ' + mode + ' bytes', t => { + const f = sourceFixture(t); + if (mode === 'staged') { + writeFileSync(f.file, 'staged'); f.git('add', 'src'); writeFileSync(f.file, 'export const value = 1;\n'); + } else if (mode === 'untracked' || mode === 'ignored') { + if (mode === 'ignored') { mkdirSync(path.join(f.root, '.git/info')); writeFileSync(path.join(f.root, '.git/info/exclude'), '*.js\n'); } + writeFileSync(path.join(f.root, 'src/app.js'), 'untracked runtime shadow'); + } else if (mode === 'symlink') { + rmSync(f.file); require('node:fs').symlinkSync(path.join(f.clean, 'src/app.ts'), f.file); + } else { + if (mode === 'assume-unchanged') f.git('update-index', '--assume-unchanged', 'src/app.ts'); + const file = mode === 'lockfile' ? 'pnpm-lock.yaml' : mode === 'compiler' ? 'tsconfig.json' : 'src/app.ts'; + writeFileSync(path.join(f.root, file), 'changed bytes'); + } + assert.throws(() => attestSource(f.root, f.pin), /does not match|untracked|symlink/u); + }); +} + +for (const mode of ['GIT_DIR', 'GIT_WORK_TREE', 'GIT_INDEX_FILE', 'GIT_CONFIG_COUNT', 'local-core-worktree']) { + test('sanitized admission defeats redirection through ' + mode, t => { + const f = sourceFixture(t); + const env = { ...process.env }; + writeFileSync(f.file, 'dirty actual source'); + if (mode === 'GIT_DIR') env.GIT_DIR = path.join(f.clean, '.git'); + if (mode === 'GIT_WORK_TREE') env.GIT_WORK_TREE = f.clean; + if (mode === 'GIT_INDEX_FILE') { + writeFileSync(f.file, 'staged dirty source'); f.git('add', 'src'); writeFileSync(f.file, 'export const value = 1;\n'); + env.GIT_INDEX_FILE = path.join(f.clean, '.git/index'); + } + if (mode === 'GIT_CONFIG_COUNT') Object.assign(env, { GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: 'core.worktree', GIT_CONFIG_VALUE_0: f.clean }); + if (mode === 'local-core-worktree') f.git('config', 'core.worktree', f.clean); + const child = spawnSync(process.execPath, ['-e', + 'require(process.argv[1]).attestSource(process.argv[2],process.argv[3])', + path.resolve(__dirname, '../execution-provenance.cjs'), f.root, f.pin], { env, encoding: 'utf8' }); + assert.notEqual(child.status, 0); + assert.match(child.stderr, /does not match/u); + }); +} + +test('local fsmonitor configuration cannot execute during source admission', t => { + const f = sourceFixture(t), marker = path.join(f.directory, 'hook-ran'), hook = path.join(f.directory, 'monitor'); + writeFileSync(hook, '#!/bin/sh\ntouch "' + marker + '"\n', { mode: 0o700 }); + f.git('config', 'core.fsmonitor', hook); + assert.equal(attestSource(f.root, f.pin).implementation, f.pin); + assert.equal(existsSync(marker), false); +}); + +test('a missing committed source pin cannot admit work', t => { + const f = sourceFixture(t); + assert.throws(() => attestSource(f.root, 'f'.repeat(40))); +}); + +test('driver proof records actual bytes and rejects a modified driver', t => { + const f = sourceFixture(t), file = path.join(f.directory, 'driver.cjs'); + writeFileSync(file, 'original'); + const expected = { 'driver.cjs': createHash('sha256').update(readFileSync(file)).digest('hex') }; + assert.deepEqual(attestDrivers(f.directory, expected, ['driver.cjs']), expected); + writeFileSync(file, 'changed'); + assert.throws(() => attestDrivers(f.directory, expected, ['driver.cjs']), /driver differs/u); +}); + +test('runtime proof binds Node and TypeScript versions and actual executable bytes', () => { + const root = path.resolve(__dirname, '../../../../..'), runtime = runtimeIdentity(root); + assert.deepEqual(attestRuntime(root, runtime), runtime); + assert.throws(() => attestRuntime(root, { ...runtime, nodeSha256: 'f'.repeat(64) }), /runtime differs/u); + assert.throws(() => attestRuntime(root, { ...runtime, typescriptSha256: 'f'.repeat(64) }), /runtime differs/u); + assert.throws(() => attestRuntime(root, { ...runtime, gitSha256: 'f'.repeat(64) }), /runtime differs/u); +}); + +test('clean admission never executes an inherited PATH Git shim', t => { + const f = sourceFixture(t), bin = path.join(f.directory, 'bin'), marker = path.join(f.directory, 'shim-ran'); + mkdirSync(bin); + writeFileSync(path.join(bin, 'git'), '#!' + process.execPath + '\nrequire("node:fs").writeFileSync(' + + JSON.stringify(marker) + ', "unexpected");process.exit(91);\n', { mode: 0o700 }); + const child = spawnSync(process.execPath, ['-e', + 'require(process.argv[1]).attestSource(process.argv[2],process.argv[3])', + path.resolve(__dirname, '../execution-provenance.cjs'), f.root, f.pin], { + encoding: 'utf8', env: { ...process.env, PATH: bin + path.delimiter + process.env.PATH }, + }); + assert.equal(child.status, 0, child.stderr); + assert.equal(existsSync(marker), false); +}); + +test('a Git shim can forge the old proof but cannot admit changed physical source now', t => { + const f = sourceFixture(t), bin = path.join(f.directory, 'bin'), marker = path.join(f.directory, 'shim-ran'); + mkdirSync(bin); + writeFileSync(f.file, 'export const value = 2;\n'); f.git('add', 'src'); + f.git('-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '--quiet', '-m', 'different source'); + const alternate = f.git('rev-parse', 'HEAD'); + const shim = `#!${process.execPath} +const fs=require('node:fs'),cp=require('node:child_process'); +fs.appendFileSync(${JSON.stringify(marker)},'called\\n'); +let args=process.argv.slice(2); +if(args.includes('diff'))process.exit(0); +if(args.includes('ls-tree'))args=args.map(x=>x===${JSON.stringify(f.pin)}?${JSON.stringify(alternate)}:x); +if(args.includes('rev-parse'))args=args.map(x=>x===${JSON.stringify(f.pin + '^{tree}')}?${JSON.stringify(alternate + '^{tree}')}:x); +const child=cp.spawnSync(${JSON.stringify(trustedGitExecutable())},args,{input:fs.readFileSync(0)}); +if(child.stdout)process.stdout.write(child.stdout);if(child.stderr)process.stderr.write(child.stderr);process.exit(child.status??1); +`; + writeFileSync(path.join(bin, 'git'), shim, { mode: 0o700 }); + const current = path.resolve(__dirname, '../execution-provenance.cjs'), legacy = path.join(f.directory, 'bare-git.cjs'); + const source = readFileSync(current, 'utf8'); + assert(source.includes('const executable = trustedGitExecutable();')); + // Counter-control: change only the original PATH-resolution choice. + writeFileSync(legacy, source.replace('const executable = trustedGitExecutable();', "const executable = 'git';")); + const launch = file => spawnSync(process.execPath, ['-e', + 'process.stdout.write(JSON.stringify(require(process.argv[1]).attestSource(process.argv[2],process.argv[3])))', + file, f.root, f.pin], { encoding: 'utf8', env: { ...process.env, PATH: bin + path.delimiter + process.env.PATH } }); + const bypass = launch(legacy); + assert.equal(bypass.status, 0, bypass.stderr); + assert.equal(JSON.parse(bypass.stdout).implementation, f.pin); + assert.equal(JSON.parse(bypass.stdout).tree, f.git('rev-parse', alternate + '^{tree}')); + assert.equal(existsSync(marker), true); + rmSync(marker); + const refused = launch(current); + assert.notEqual(refused.status, 0); + assert.match(refused.stderr, /does not match/u); + assert.equal(existsSync(marker), false); +}); + +for (const runner of ['normal-flow-shadow/run.cjs', 'ev07-shadow/corpus.cjs', 'ev07-shadow/review.cjs']) { + test('the real ' + runner + ' refuses a foreign pin before application imports or model work', t => { + const directory = mkdtempSync(path.join(tmpdir(), 'bugbot-before-import-')); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const marker = path.join(directory, 'application-imported'), preload = path.join(directory, 'preload.cjs'); + const plan = path.join(directory, 'plan.json'); + writeFileSync(plan, JSON.stringify({ implementation: 'f'.repeat(40) })); + writeFileSync(preload, `const Module=require('node:module'),fs=require('node:fs'),load=Module._load; +Module._load=function(name,...args){if(name.includes('/src/')||name.endsWith('/typescript')||name==='@actions/github'){ +fs.writeFileSync(process.env.IMPORT_MARKER,name);throw Error('unexpected application import');}return load.call(this,name,...args);};`); + const child = spawnSync(process.execPath, ['--require', preload, path.resolve(__dirname, '..', runner), + directory, '1', path.join(directory, 'result.jsonl'), plan], { encoding: 'utf8', + env: { ...process.env, COPILOT_EV07_PLAN: plan, IMPORT_MARKER: marker } }); + assert.notEqual(child.status, 0); + assert.equal(existsSync(marker), false, child.stderr); + assert.equal(existsSync(path.join(directory, 'result.jsonl')), false); + assert.match(child.stderr, /rev-parse|commit|pin|immutable bootstrap/u); + }); +} + +function dependencyFixture(t) { + const directory = mkdtempSync(path.join(tmpdir(), 'qualification-dependency-proof-')); + t.after(() => rmSync(directory, { recursive: true, force: true })); + const modules = path.join(directory, 'node_modules'); + mkdirSync(path.join(modules, 'typescript'), { recursive: true }); + writeFileSync(path.join(modules, 'typescript/package.json'), JSON.stringify({ name: 'typescript', version: 'fixture', main: 'index.js' })); + writeFileSync(path.join(modules, 'typescript/index.js'), 'module.exports = {};\n'); + mkdirSync(path.join(modules, '@actions/github'), { recursive: true }); + const file = path.join(modules, '@actions/github/index.js'); + writeFileSync(file, 'module.exports = {};\n'); + return { directory, modules, file }; +} + +test('dependency admission binds actual files and refuses additions, loader shadows and escaping symlinks', t => { + const f = dependencyFixture(t), original = dependencyIdentity(f.directory); + assert.match(original.dependencyTreeSha256, /^[a-f0-9]{64}$/u); + writeFileSync(path.join(f.modules, 'extra.js'), 'unadmitted'); + assert.notEqual(dependencyIdentity(f.directory).dependencyTreeSha256, original.dependencyTreeSha256); + symlinkSync('platform-optional.js', path.join(f.modules, 'optional')); + const missingOptional = dependencyIdentity(f.directory); + writeFileSync(path.join(f.modules, 'platform-optional.js'), 'materialized'); + assert.notEqual(dependencyIdentity(f.directory).dependencyTreeSha256, missingOptional.dependencyTreeSha256); + symlinkSync(f.directory, path.join(f.modules, 'outside')); + assert.throws(() => dependencyIdentity(f.directory), /symlink escapes/u); + rmSync(path.join(f.modules, 'outside')); + mkdirSync(path.join(f.directory, 'quality/node_modules'), { recursive: true }); + assert.throws(() => dependencyIdentity(f.directory), /dependency shadow/u); +}); + +test('a modified dependency executes with the old proof but is refused before import by the new proof', t => { + const f = dependencyFixture(t), runtime = runtimeIdentity(f.directory), marker = path.join(f.directory, 'dependency-executed'); + writeFileSync(f.file, 'require("node:fs").writeFileSync(' + JSON.stringify(marker) + ', "executed");\n'); + const helper = path.resolve(__dirname, '../execution-provenance.cjs'), old = path.join(f.directory, 'old-proof.cjs'); + const source = readFileSync(helper, 'utf8'); + assert(source.includes('return { ...dependencies, node:')); + writeFileSync(old, source.replace('return { ...dependencies, node:', 'return { node:')); + const run = file => spawnSync(process.execPath, ['-e', + 'require(process.argv[1]).attestRuntime(process.argv[2],JSON.parse(process.argv[3]));require(process.argv[2]+"/node_modules/@actions/github")', + file, f.directory, JSON.stringify(runtime)], { encoding: 'utf8' }); + const bypass = run(old); + assert.equal(bypass.status, 0, bypass.stderr); + assert.equal(existsSync(marker), true); + rmSync(marker); + const refused = run(helper); + assert.notEqual(refused.status, 0); + assert.match(refused.stderr, /runtime differs: dependencyTreeSha256/u); + assert.equal(existsSync(marker), false); +}); + +for (const mode of ['require', 'import']) { + test('the normal workspace portal is inventoried but cannot execute through ' + mode, t => { + const f = dependencyFixture(t), portal = path.join(f.modules, '.pnpm/node_modules/@vypdev'); + mkdirSync(portal, { recursive: true }); + mkdirSync(path.join(f.directory, 'extension')); + const marker = path.join(f.directory, 'portal-executed'), external = path.join(f.directory, 'extension/index.cjs'); + writeFileSync(external, 'require("node:fs").writeFileSync(' + JSON.stringify(marker) + ', "executed");\n'); + symlinkSync('../../../../extension', path.join(portal, 'copilot-browser-extension')); + assert.match(dependencyIdentity(f.directory).dependencyTreeSha256, /^[a-f0-9]{64}$/u); + const program = 'require(process.argv[1]).admitQualificationModules(process.argv[2]);' + + (mode === 'require' ? 'require(process.argv[3]);' : 'import(require("node:url").pathToFileURL(process.argv[3])).catch(e=>{console.error(e);process.exitCode=1;});'); + const child = spawnSync(process.execPath, ['-e', program, path.resolve(__dirname, '../execution-provenance.cjs'), f.directory, external], { encoding: 'utf8' }); + assert.notEqual(child.status, 0); + assert.match(child.stderr, /outside the admitted qualification closure/u); + assert.equal(existsSync(marker), false); + }); +} test('the actual runner resolves a relative lab before entering its private checkout, without model calls', t => { const directory = mkdtempSync(path.join(tmpdir(), 'bugbot-relative-lab-')); @@ -19,13 +255,39 @@ test('the actual runner resolves a relative lab before entering its private chec git('add', '.'); git('-c', 'user.name=Test', '-c', 'user.email=fixture@example.invalid', 'commit', '--quiet', '-m', 'fixed source'); const head = git('rev-parse', 'HEAD'), tree = git('rev-parse', 'HEAD^{tree}'); - writeFileSync(path.join(directory, 'subjects.json'), JSON.stringify({ publication: 'dry-run', effort: 'xhigh', - subjects: [{ repository: 'owner/project', number: 1, baseRef: 'master', headRef: 'review', head, base: head, - tree, baseTree: tree, model: 'gpt-6-luna' }] })); + const manifest = { publication: 'dry-run', effort: 'xhigh', subjects: [{ repository: 'owner/project', number: 1, baseRef: 'master', headRef: 'review', head, base: head, + tree, baseTree: tree, model: 'gpt-6-luna' }] }; const forbidden = path.join(directory, 'model-invoked'); writeFileSync(path.join(bin, 'codex'), '#!/bin/sh\nif [ "$1" = "--version" ]; then echo "codex-cli fixture"; exit 0; fi\n' + 'touch "$MODEL_FORBIDDEN"\nexit 91\n', { mode: 0o700 }); - const run = spawnSync(process.execPath, [path.join(__dirname, 'run.cjs'), 'lab', '1', 'result.jsonl', 'subjects.json'], { + const root = path.resolve(__dirname, '../../../../..'); + const proof = require('../execution-provenance.cjs'); + const hash = file => createHash('sha256').update(readFileSync(file)).digest('hex'); + manifest.implementation = execFileSync('git', ['-C', root, 'rev-parse', 'HEAD'], { encoding: 'utf8' }).trim(); + manifest.runtime = { ...proof.runtimeIdentity(root), executable: path.join(bin, 'codex'), + codex: 'codex-cli fixture', codexSha256: hash(path.join(bin, 'codex')) }; + manifest.drivers = Object.fromEntries(require('./run.cjs').normalFlowDrivers.map(file => [file, hash(path.join(__dirname, file))])); + const checkout = path.join(directory, 'implementation'); + execFileSync('git', ['clone', '--quiet', '--shared', '--template=', root, checkout]); + // Commit the current driver snapshot inside this disposable fixture, so + // this control also works before the contributor implementation commit. + const driverPaths = require('./run.cjs').normalFlowDrivers.map(file => { + const relative = path.relative(root, path.join(__dirname, file)); + writeFileSync(path.join(checkout, relative), readFileSync(path.join(root, relative))); + return relative; + }); + cpSync(path.join(root, 'node_modules'), path.join(checkout, 'node_modules'), + { recursive: true, verbatimSymlinks: true, mode: constants.COPYFILE_FICLONE }); + const relative = 'quality/bugbot/evidence/issue-950/normal-flow-shadow/relative-lab.fixture.json'; + writeFileSync(path.join(checkout, relative), JSON.stringify(manifest)); + execFileSync('git', ['-C', checkout, 'add', relative, ...driverPaths]); + execFileSync('git', ['-C', checkout, '-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.invalid', + '-c', 'core.hooksPath=/dev/null', 'commit', '--quiet', '-m', 'Commit synthetic fixture manifest']); + const revision = execFileSync('git', ['-C', checkout, 'rev-parse', 'HEAD'], { encoding: 'utf8' }).trim(); + const bootstrap = path.join(directory, 'reviewed-bootstrap.cjs'); + writeFileSync(bootstrap, execFileSync('git', ['-C', checkout, 'show', revision + ':quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs'])); + const run = spawnSync(process.execPath, [bootstrap, checkout, revision, relative, + hash(path.join(checkout, relative)), hash(bootstrap), 'normal-flow', 'lab', '1', 'result.jsonl'], { cwd: directory, encoding: 'utf8', env: { ...process.env, PATH: bin + path.delimiter + process.env.PATH, MODEL_FORBIDDEN: forbidden }, }); diff --git a/quality/bugbot/evidence/issue-950/normal-flow-shadow/run.cjs b/quality/bugbot/evidence/issue-950/normal-flow-shadow/run.cjs index 8f22d29cc..e79c4f008 100644 --- a/quality/bugbot/evidence/issue-950/normal-flow-shadow/run.cjs +++ b/quality/bugbot/evidence/issue-950/normal-flow-shadow/run.cjs @@ -1,5 +1,7 @@ 'use strict'; +if (require.main === module) throw new Error('Use the independently reviewed immutable bootstrap; direct qualification runners are unsupported.'); + const assert = require('node:assert/strict'); const { execFileSync } = require('node:child_process'); const { createHash } = require('node:crypto'); @@ -9,26 +11,34 @@ const path = require('node:path'); const official = require('../shadow-real-prs/official.json'); const shadowPlan = require('../shadow-verification/plan.json'); const root = path.resolve(__dirname, '../../../../..'); -const ts = require(path.join(root, 'node_modules/typescript')); -require.extensions['.ts'] = (module, filename) => module._compile(ts.transpileModule(fs.readFileSync(filename, 'utf8'), { - compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2022, esModuleInterop: true }, -}).outputText, filename); - -const load = file => require(path.join(root, 'src', file)); -const { composeBugbotReview } = load('infrastructure/composition/bugbot_review_composition'); -const { createPinnedBugbotShadowVerifier } = load('infrastructure/bugbot/shadow_verification_adapter'); -const { ask, codexInvocation } = load('infrastructure/bugbot/shadow/session.cjs'); -const { projectBugbotReviewOperationContext } = load('application/usecases/steps/commit/bugbot/bugbot_review_operation_context'); -const { buildAgentTasksFromValues } = load('actions/agent_input_builder'); -const { Ai } = load('data/model/ai'); -const { createFindingsQueryPort } = load('infrastructure/composition/agent_capability_composition_root'); -const { AgentCliClient } = load('data/repository/agent_cli_client'); -const { AgentExecutionPlanner } = load('infrastructure/agents/agent_execution_planner'); -const { resolveBugbotReviewEffort } = load('domain/bugbot/review_configuration'); +const { attestSource, attestRuntime, admitQualificationModules, attestDrivers, pinnedGit, trustedGitExecutable, + attestCommittedArtifact, requireBootstrapAdmission } = require('../execution-provenance.cjs'); +const { requirePinnedExecution, requirePinnedCliProbe, preparePinnedExecution } = require('../ev07-shadow/implementation.cjs'); +const normalFlowDrivers = Object.freeze(['run.cjs', '../execution-provenance.cjs', '../ev07-shadow/load.cjs', + '../ev07-shadow/implementation.cjs', '../ev07-shadow/configuration.cjs', '../qualification-bootstrap.cjs']); +let application; +function loadApplication() { + if (application) return application; + require('../ev07-shadow/load.cjs'); + const load = file => require(path.join(root, 'src', file)); + const { composeBugbotReview } = load('infrastructure/composition/bugbot_review_composition'); + const { createPinnedBugbotShadowVerifier } = load('infrastructure/bugbot/shadow_verification_adapter'); + const { ask, codexInvocation } = load('infrastructure/bugbot/shadow/session.cjs'); + const { projectBugbotReviewOperationContext } = load('application/usecases/steps/commit/bugbot/bugbot_review_operation_context'); + const { buildAgentTasksFromValues } = load('actions/agent_input_builder'); + const { Ai } = load('data/model/ai'); + const { createFindingsQueryPort } = load('infrastructure/composition/agent_capability_composition_root'); + const { AgentCliClient } = load('data/repository/agent_cli_client'); + const { AgentExecutionPlanner } = load('infrastructure/agents/agent_execution_planner'); + const { resolveBugbotReviewEffort } = load('domain/bugbot/review_configuration'); + application = { composeBugbotReview, createPinnedBugbotShadowVerifier, ask, codexInvocation, + projectBugbotReviewOperationContext, buildAgentTasksFromValues, Ai, createFindingsQueryPort, + AgentCliClient, AgentExecutionPlanner, resolveBugbotReviewEffort }; + return application; +} const hash = text => createHash('sha256').update(text).digest('hex'); -const git = (cwd, ...args) => execFileSync('git', ['-C', cwd, '-c', 'core.hooksPath=/dev/null', ...args], - { encoding: 'utf8', maxBuffer: 16 * 1024 * 1024 }); +const git = (cwd, ...args) => pinnedGit(cwd)(...args).toString('utf8'); const snapshot = (source, value, count = 0) => ({ value, coverage: { source, status: 'complete', pagesFetched: 1, itemsFetched: count, itemsRetained: count, omittedItems: 0, truncatedItems: 0, limitReached: false }, @@ -70,6 +80,7 @@ function selectVerifierProfiles(subject, detector) { assert(!subject.compareInheritedVerifier, 'matched comparison requires an experimental verifier'); return { primary: detector }; } + const { buildAgentTasksFromValues } = loadApplication(); const { model, effort } = subject.experimentalVerifier; assert(typeof model === 'string' && model.trim(), 'experimental verifier model required'); const { reviewer } = buildAgentTasksFromValues({ 'reviewer-model': model, 'reviewer-effort': effort }); @@ -98,8 +109,23 @@ async function verifyMatchedCandidate(request, profiles, verifyArm, recordCompar async function run(lab, number, destination, manifestPath = path.join(__dirname, 'subjects.json')) { lab = path.resolve(lab); - const manifest = fs.readFileSync(manifestPath); + const admission = requireBootstrapAdmission(root); + const authenticated = attestCommittedArtifact(root, manifestPath, admission.planSha256, admission.planCommit); + const manifest = authenticated.bytes; const frozen = JSON.parse(manifest); + const provenance = { bootstrap: admission, manifest: authenticated.provenance, ...attestSource(root, frozen.implementation), + drivers: attestDrivers(__dirname, frozen.drivers, normalFlowDrivers), runtime: attestRuntime(root, frozen.runtime) }; + admitQualificationModules(root); + const directRuntime = { executable: frozen.runtime.executable, prefixArgs: frozen.runtime.launcher ? [frozen.runtime.launcher] : [] }; + const attestDirect = () => requirePinnedExecution({ executable: directRuntime.executable, launcherArgv: directRuntime.prefixArgs, + ...(frozen.runtime.launcher && { launcherSha256: frozen.runtime.launcherSha256 }), + runtimeContract: { version: frozen.runtime.codex } }, frozen.runtime); + const cliProbe = requirePinnedCliProbe(frozen.runtime); + const cli = cliProbe.version; + provenance.cli = { ...cliProbe.proof, version: cli }; + const { composeBugbotReview, createPinnedBugbotShadowVerifier, ask, codexInvocation, + projectBugbotReviewOperationContext, buildAgentTasksFromValues, Ai, createFindingsQueryPort, + AgentCliClient, AgentExecutionPlanner, resolveBugbotReviewEffort } = loadApplication(); const { workflowInputs } = frozen; assert.equal(frozen.publication, 'dry-run'); const subject = frozen.subjects.find(item => item.number === Number(number)); @@ -129,7 +155,8 @@ async function run(lab, number, destination, manifestPath = path.join(__dirname, throw new Error('forbidden GitHub mutation'); } }); try { - git(scratch, 'clone', '-q', '--shared', '--no-checkout', source, checkout); + execFileSync(trustedGitExecutable(), ['clone', '-q', '--shared', '--no-checkout', source, checkout], + { cwd: scratch, env: require('../execution-provenance.cjs').gitEnvironment() }); git(checkout, 'remote', 'remove', 'origin'); git(checkout, 'checkout', '-q', '--detach', subject.head); assert.equal(git(checkout, 'remote'), ''); @@ -159,8 +186,9 @@ async function run(lab, number, destination, manifestPath = path.join(__dirname, if (workflowInputs) { assert.equal(process.env.REVIEWER_EFFORT, workflowInputs.REVIEWER_EFFORT); assert.equal(process.env.BUGBOT_EFFORT, workflowInputs.BUGBOT_EFFORT); - const runtime = execFileSync(process.env.PILOT_LOCAL_PYTHON, ['-c', - 'import platform,sqlite3; print(platform.python_version(), sqlite3.sqlite_version)'], { encoding: 'utf8' }).trim(); + const runtime = execFileSync(process.env.PILOT_LOCAL_PYTHON, ['-I', '-S', '-c', + 'import platform,sqlite3; print(platform.python_version(), sqlite3.sqlite_version)'], + { encoding: 'utf8', env: { PATH: '/usr/bin:/bin' } }).trim(); assert.equal(runtime, shadowPlan.localRuntime.python + ' ' + shadowPlan.localRuntime.sqlite); log({ event: 'local-runtime', runtime, scope: shadowPlan.localRuntime.scope }); } @@ -188,11 +216,17 @@ async function run(lab, number, destination, manifestPath = path.join(__dirname, }; const standardDetector = createFindingsQueryPort({ cli: new AgentCliClient({ prepare(request) { - const plan = planner.prepare(request); - const flags = codexInvocation(plan.argv); - attest(flags); - log({ event: 'detector-invocation', invocation: flags, capability: plan.capability, atMs: sinceStart() }); - return plan; + const plan = preparePinnedExecution(planner, request, frozen.runtime); + try { + const runtimeProof = requirePinnedExecution(plan, frozen.runtime); + const flags = codexInvocation(plan.argv); + attest(flags); + log({ event: 'detector-invocation', invocation: flags, capability: plan.capability, runtimeProof, atMs: sinceStart() }); + return plan; + } catch (error) { + fs.rmSync(plan.runtimeDirectory, { recursive: true, force: true }); + throw error; + } }, }, { observe(observation) { if (observation.state === 'failed') log({ event: 'detector-provider-failure', observation, atMs: sinceStart() }); @@ -209,8 +243,8 @@ async function run(lab, number, destination, manifestPath = path.join(__dirname, }); log({ event: 'plan', subject, files: diff.changes.length, comparisonSha256: hash(JSON.stringify(diff)), configuration: selected, verifierConfiguration, workflowInputs, reviewEffort, effectiveDepth, subjectsSha256: hash(manifest), - cli: execFileSync('codex', ['--version'], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }).trim(), - implementation: git(root, 'rev-parse', 'HEAD').trim(), runnerSha256: hash(fs.readFileSync(__filename)) }); + cli, + implementation: provenance.implementation, provenance, runnerSha256: hash(fs.readFileSync(__filename)) }); let invocation = 0; const detector = { async query(request) { const turn = ++invocation; @@ -239,13 +273,16 @@ async function run(lab, number, destination, manifestPath = path.join(__dirname, && input.headSha === subject.head && input.baseSha === subject.base ? { repo: source, officialDirectory: lab, catalog: official, ...(workflowInputs && { localRuntime: shadowPlan.localRuntime }) } : undefined, - transport: turn => ask({ ...turn, + transport: turn => { + attestDirect(); + return ask({ ...turn, runtime: directRuntime, onInvocation(flags) { attest(flags, expected); log(label(arm, { event: 'verifier-invocation', id: turn.candidateId, invocation: flags, continuation: Boolean(turn.threadId), atMs: sinceStart() })); }, - }), + }); + }, }); const primaryArm = profiles.experimental ? 'experimental' : 'inherited'; const adapters = { [primaryArm]: makeAdapter(primaryArm, profiles.primary) }; @@ -287,11 +324,4 @@ async function run(lab, number, destination, manifestPath = path.join(__dirname, } } -if (require.main === module) { - run(...process.argv.slice(2)).catch(error => { - console.error(error); - process.exitCode = 1; - }); -} - -module.exports = { selectVerifierProfiles, verifyMatchedCandidate }; +module.exports = { run, selectVerifierProfiles, verifyMatchedCandidate, normalFlowDrivers }; diff --git a/quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs b/quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs new file mode 100644 index 000000000..603610221 --- /dev/null +++ b/quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs @@ -0,0 +1,218 @@ +'use strict'; + +// Trust begins OUTSIDE the checkout: materialize this file from an independently +// reviewed Git revision, compare its reviewed SHA-256, then start trusted Node +// with NODE_OPTIONS/NODE_PATH unset. See issue-970/README.md. This file uses only +// Node core until the immutable verifier and every driver have been admitted. +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const os = require('node:os'); +const { createHash } = require('node:crypto'); +const { execFileSync, spawn } = require('node:child_process'); +const sha256 = bytes => createHash('sha256').update(bytes).digest('hex'); +const prefix = 'quality/bugbot/evidence/issue-950/'; +const ev07 = ['configuration.cjs', 'implementation.cjs', 'load.cjs', 'corpus.cjs', 'review.cjs', + 'full-review.cjs', 'launch-full-review.cjs', 'score-review.py', '../execution-provenance.cjs', '../qualification-bootstrap.cjs']; +const normal = ['run.cjs', '../execution-provenance.cjs', '../ev07-shadow/load.cjs', + '../ev07-shadow/implementation.cjs', '../ev07-shadow/configuration.cjs', '../qualification-bootstrap.cjs']; +const entries = Object.freeze({ corpus: 'ev07-shadow/corpus.cjs', + 'full-review': 'ev07-shadow/launch-full-review.cjs', 'normal-flow': 'normal-flow-shadow/run.cjs', + 'full-review-runner': 'ev07-shadow/full-review.cjs', review: 'ev07-shadow/review.cjs' }); + +function safePath(value) { + assert(typeof value === 'string' && value.length && !path.isAbsolute(value) + && !value.includes('\\') && !value.split('/').some(part => ['', '.', '..', '.git'].includes(part)), + 'unsafe committed artifact path'); + return value; +} + +function physical(root, relative) { + let file = root; + for (const part of safePath(relative).split('/')) { + file = path.join(file, part); + assert(!fs.lstatSync(file).isSymbolicLink(), 'qualification artifact symlink refused'); + } + assert(fs.lstatSync(file).isFile(), 'qualification artifact must be a regular file'); + return file; +} + +function gitFor(root) { + const file = process.platform === 'darwin' ? execFileSync('/usr/bin/xcrun', ['--find', 'git'], + { env: { PATH: '/usr/bin:/bin' }, encoding: 'utf8' }).trim() + : process.platform === 'linux' ? '/usr/bin/git' : undefined; + assert(file && path.isAbsolute(file), 'trusted Git requires Linux or macOS'); + const executable = fs.realpathSync(file), marker = path.join(root, '.git'); + assert(!fs.lstatSync(marker).isSymbolicLink(), 'Git identity symlink refused'); + let directory = marker; + if (fs.lstatSync(marker).isFile()) { + const match = /^gitdir: (.+)\r?\n?$/u.exec(fs.readFileSync(marker, 'utf8')); + assert(match, 'invalid worktree Git identity'); + directory = path.resolve(root, match[1]); + } + const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.toUpperCase().startsWith('GIT_'))); + Object.assign(env, { GIT_CONFIG_GLOBAL: os.devNull, GIT_CONFIG_NOSYSTEM: '1', GIT_NO_REPLACE_OBJECTS: '1' }); + const git = (...args) => execFileSync(executable, ['--no-pager', '--no-replace-objects', '--literal-pathspecs', + '--git-dir=' + directory, '--work-tree=' + root, '-C', root, + '-c', 'core.hooksPath=' + os.devNull, '-c', 'core.fsmonitor=false', ...args], + { env, maxBuffer: 256 * 1024 * 1024, stdio: ['pipe', 'pipe', 'pipe'] }); + return { git, executable, env, directory }; +} + +function committed(git, revision, relative) { + safePath(relative); + const rows = git('ls-tree', '-z', revision, '--', relative).toString('utf8').split('\0').filter(Boolean); + assert.equal(rows.length, 1, 'committed artifact unavailable'); + const [header, name] = rows[0].split('\t'), [mode, type, blob] = header.split(' '); + assert(['100644', '100755'].includes(mode) && type === 'blob' && name === relative, + 'committed artifact must be a regular blob'); + return git('cat-file', 'blob', blob); +} + +function reviewedPlan(repository, revision, planPath, planSha256, bootstrapSha256, mode) { + assert(!process.env.NODE_OPTIONS && !process.env.NODE_PATH && process.execArgv.length === 0, + 'bootstrap requires trusted Node without preloads or loader overrides'); + assert.match(revision ?? '', /^[a-f0-9]{40}$/u, 'reviewed plan commit required'); + assert.match(planSha256 ?? '', /^[a-f0-9]{64}$/u, 'independently reviewed plan digest required'); + assert.match(bootstrapSha256 ?? '', /^[a-f0-9]{64}$/u, 'independently reviewed bootstrap digest required'); + assert.equal(sha256(fs.readFileSync(__filename)), bootstrapSha256, 'reviewed bootstrap differs'); + assert(Object.hasOwn(entries, mode), 'unsupported qualification entrypoint'); + const root = fs.realpathSync(repository), trust = gitFor(root), { git } = trust; + assert.equal(git('rev-parse', '--verify', revision + '^{commit}').toString('utf8').trim(), revision); + assert.equal(sha256(committed(git, revision, prefix + 'qualification-bootstrap.cjs')), bootstrapSha256, + 'plan revision does not contain the independently reviewed bootstrap'); + const bytes = committed(git, revision, planPath); + assert.equal(sha256(bytes), planSha256, 'reviewed committed plan differs'); + assert(fs.readFileSync(physical(root, planPath)).equals(bytes), 'working plan differs from reviewed commit'); + const plan = JSON.parse(bytes); + assert.equal(plan.publication, 'dry-run', 'qualification must remain dry-run'); + assert.match(plan.implementation ?? '', /^[a-f0-9]{40}$/u); + git('merge-base', '--is-ancestor', plan.implementation, revision); + const base = prefix + (mode === 'normal-flow' ? 'normal-flow-shadow/' : 'ev07-shadow/'); + const inventory = mode === 'normal-flow' ? normal : ev07; + const expected = mode === 'normal-flow' ? plan.drivers : plan.inspectedArtifacts; + const admitted = []; + for (const driver of inventory) { + const relative = path.posix.normalize(base + driver), trusted = committed(git, revision, relative); + assert.equal(sha256(trusted), expected?.[driver], 'committed driver differs from reviewed plan: ' + driver); + assert(fs.readFileSync(physical(root, relative)).equals(trusted), 'working driver differs: ' + driver); + admitted.push([relative, trusted]); + } + return { root, trust, bytes, plan, admitted }; +} + +function spawnReviewedLaunch(request, env, options = {}) { + assert(Object.keys(options).every(key => key === 'stdio'), 'unsupported qualification child options'); + assert.equal(sha256(fs.readFileSync(__filename)), request.proof.bootstrapSha256, 'executing bootstrap changed'); + const { proof, root, mode, args, corpusCampaign } = request; + // No IPC admission endpoint: every process enters the complete public + // validation/materialization path with the same independently supplied pins. + const modeArgs = mode === 'review' ? [corpusCampaign, ...args] : args; + return spawn(process.execPath, [__filename, root, proof.planCommit, proof.planPath, + proof.planSha256, proof.bootstrapSha256, mode, ...modeArgs], + { env, stdio: options.stdio ?? 'inherit', cwd: process.cwd() }); +} + +async function executePrivate(request) { + assert(request && Array.isArray(request.args) && request.args.every(arg => typeof arg === 'string'), + 'invalid admitted invocation'); + const { root: repository, proof, mode } = request; + const { root, plan } = reviewedPlan(repository, proof.planCommit, proof.planPath, + proof.planSha256, proof.bootstrapSha256, mode); + // This function is reachable only from launch after private materialization. + // There is no message handler or exported API that initializes admission. + // Repeat private bytes before imports; a checkout receipt has no authority. + const verifier = require(path.join(root, prefix, 'execution-provenance.cjs')); + verifier.attestSource(root, plan.implementation); + verifier.attestRuntime(root, plan.runtime); + verifier.initializeBootstrapContext(root, proof, (nextMode, args, options) => { + assert(mode === 'full-review' && nextMode === 'full-review-runner' + || mode === 'full-review-runner' && nextMode === 'review', 'unadmitted qualification child route'); + const corpusCampaign = mode === 'full-review' ? request.args[0] : request.corpusCampaign; + return spawnReviewedLaunch({ root, proof, mode: nextMode, args, corpusCampaign }, process.env, options); + }); + if (mode === 'review') { + assert(typeof request.corpusCampaign === 'string', 'paired review requires its passing corpus'); + require(path.join(root, prefix, entries['full-review'])).requirePassingCorpus(request.corpusCampaign); + } + const driver = require(path.join(root, prefix, entries[mode])); + assert.equal(typeof driver.run, 'function', 'admitted driver entry is unavailable'); + return driver.run(...request.args); +} + +async function launch(repository, revision, planPath, planSha256, bootstrapSha256, mode, ...args) { + assert(Object.hasOwn(entries, mode), 'unsupported qualification entrypoint; worker admission endpoint removed'); + assert(mode === 'normal-flow' ? args.length === 3 + : mode === 'full-review' || mode === 'full-review-runner' ? args.length === 1 + : mode === 'review' ? args.length === 5 + : args.length === 1 || args.length === 2 && args[1] === '--preflight', 'invalid qualification arguments'); + const { root, trust, bytes, plan, admitted } = reviewedPlan(repository, revision, + planPath, planSha256, bootstrapSha256, mode); + assert(!fs.realpathSync(__filename).startsWith(root + path.sep), 'materialize the reviewed bootstrap outside the checkout'); + const scratch = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'bugbot-qualified-source-'))); + fs.chmodSync(scratch, 0o700); + const isolated = path.join(scratch, 'source'); + try { + // A fresh local clone does not copy source repository hooks/configuration. + execFileSync(trust.executable, ['-c', 'core.hooksPath=' + os.devNull, 'clone', '--quiet', '--shared', + '--no-checkout', '--template=', trust.directory, isolated], { env: trust.env }); + const isolatedGit = gitFor(isolated).git; + isolatedGit('checkout', '--quiet', '--detach', revision); + isolatedGit('remote', 'remove', 'origin'); + // Git attributes can transform checkout bytes even without external + // filters. Recheck the materialized closure before requiring its helper. + for (const [relative, trusted] of admitted) assert( + fs.readFileSync(physical(isolated, relative)).equals(trusted), 'materialized driver differs: ' + relative); + assert(fs.readFileSync(physical(isolated, planPath)).equals(bytes), 'materialized plan differs'); + // The helper is now the immutable blob already checked above, not a + // working-tree verifier asked to authenticate itself after execution. + const verifier = require(path.join(isolated, prefix, 'execution-provenance.cjs')); + verifier.attestSource(root, plan.implementation); + verifier.attestRuntime(root, plan.runtime); + assert(!fs.existsSync(path.join(isolated, 'node_modules')), 'tracked dependency installation refused'); + fs.cpSync(path.join(root, 'node_modules'), path.join(isolated, 'node_modules'), + { recursive: true, verbatimSymlinks: true, dereference: false, mode: fs.constants.COPYFILE_FICLONE }); + verifier.attestSource(isolated, plan.implementation); + verifier.attestRuntime(isolated, plan.runtime); + const proof = { bootstrapSha256, planCommit: revision, planPath, planSha256 }; + // The application evidence adapter also invokes the bare Git command. + // Bind that lookup to the admitted OS executable without changing how + // Node/Codex or any other normal detector executable is selected. + const gitBin = path.join(scratch, 'git-bin'); + fs.mkdirSync(gitBin, { mode: 0o700 }); + fs.symlinkSync(trust.executable, path.join(gitBin, 'git')); + const env = { ...trust.env, COPILOT_EV07_PLAN: path.join(isolated, planPath), + PATH: gitBin + path.delimiter + (trust.env.PATH ?? '/usr/bin:/bin'), + BUGBOT_VERIFICATION_MODE: 'disabled', + COPILOT_QUALIFICATION_MANIFEST_SHA256: planSha256, + COPILOT_QUALIFICATION_PLAN_COMMIT: revision }; + delete env.NODE_OPTIONS; + delete env.NODE_PATH; + // Old diagnostic plans may select an auxiliary executable from the + // environment. A future reviewed plan must bind it before any probe. + if (plan.workflowInputs) { + assert(path.isAbsolute(plan.runtime.pythonExecutable ?? ''), 'pinned auxiliary Python required'); + assert.equal(sha256(fs.readFileSync(plan.runtime.pythonExecutable)), plan.runtime.pythonSha256, + 'auxiliary Python differs from reviewed runtime'); + env.PILOT_LOCAL_PYTHON = plan.runtime.pythonExecutable; + } + const corpusCampaign = mode === 'review' || mode === 'full-review-runner' ? args[0] : undefined; + const childArgs = mode === 'normal-flow' ? [...args, path.join(isolated, planPath)] + : mode === 'review' ? args.slice(1) : args; + const previousEnvironment = process.env; + process.env = env; + try { + // Import and run in this verified bootstrap process, never through + // an internal CLI/IPC message that can bypass public admission. + return await executePrivate({ root: isolated, proof, mode, args: childArgs, corpusCampaign }); + } finally { process.env = previousEnvironment; } + + } finally { fs.rmSync(scratch, { recursive: true, force: true }); } +} + +module.exports = { launch }; +if (require.main === module) { + launch(...process.argv.slice(2)).then(code => { + process.exitCode = Number.isInteger(code) ? code : code?.status === 'interrupted' ? 130 : 0; + }).catch(error => { console.error(error.message); process.exitCode = 1; }); +} diff --git a/quality/bugbot/evidence/issue-950/qualification-bootstrap.test.cjs b/quality/bugbot/evidence/issue-950/qualification-bootstrap.test.cjs new file mode 100644 index 000000000..2122a0203 --- /dev/null +++ b/quality/bugbot/evidence/issue-950/qualification-bootstrap.test.cjs @@ -0,0 +1,331 @@ +'use strict'; + +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const os = require('node:os'); +const { createHash } = require('node:crypto'); +const { execFileSync, spawnSync, spawn } = require('node:child_process'); +const { once } = require('node:events'); +const { test } = require('node:test'); +const proof = require('./execution-provenance.cjs'); +const hash = bytes => createHash('sha256').update(bytes).digest('hex'); +const prefix = 'quality/bugbot/evidence/issue-950/'; + +function fixture(t, crlfCheckout = false, brokerGit = false, cancellation = false, bootstrapBytes) { + const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'qualified-bootstrap-control-')); + t.after(() => fs.rmSync(directory, { recursive: true, force: true })); + const root = path.join(directory, 'project'); + fs.mkdirSync(root); + const write = (file, bytes) => { + fs.mkdirSync(path.dirname(path.join(root, file)), { recursive: true }); + fs.writeFileSync(path.join(root, file), bytes); + }; + const env = { ...process.env, GIT_CONFIG_GLOBAL: os.devNull, GIT_CONFIG_NOSYSTEM: '1' }; + const git = (...args) => execFileSync(proof.trustedGitExecutable(), ['-C', root, ...args], { env, encoding: 'utf8' }).trim(); + git('init', '--quiet', '--template='); + write('.gitignore', 'node_modules/\n'); + write('src/app.ts', 'export const answer = 42;\n'); + write('package.json', '{}\n'); + write('tsconfig.json', '{}\n'); + write('pnpm-lock.yaml', 'lockfileVersion: 9\n'); + write('node_modules/typescript/package.json', JSON.stringify({ name: 'typescript', version: 'fixture', main: 'index.js' })); + write('node_modules/typescript/index.js', 'module.exports = {};\n'); + const marker = path.join(directory, 'tampered-code-executed'), output = path.join(directory, 'result.json'); + const drivers = ['run.cjs', '../execution-provenance.cjs', '../ev07-shadow/load.cjs', + '../ev07-shadow/implementation.cjs', '../ev07-shadow/configuration.cjs', '../qualification-bootstrap.cjs']; + for (const file of drivers) write(path.posix.normalize(prefix + 'normal-flow-shadow/' + file), 'module.exports = {};\n'); + write(prefix + 'execution-provenance.cjs', fs.readFileSync(path.join(__dirname, 'execution-provenance.cjs'))); + write(prefix + 'qualification-bootstrap.cjs', bootstrapBytes ?? fs.readFileSync(path.join(__dirname, 'qualification-bootstrap.cjs'))); + write(prefix + 'normal-flow-shadow/run.cjs', 'exports.run=(lab,number,output)=>{const fs=require("node:fs"),path=require("node:path"),' + + 'root=path.resolve(__dirname,"../../../../..");fs.writeFileSync(output,JSON.stringify({cwd:process.cwd(),isolated:__dirname,' + + 'mode:process.env.BUGBOT_VERIFICATION_MODE,admission:require("../execution-provenance.cjs").requireBootstrapAdmission(root),' + + 'checkoutReceipt:fs.existsSync(path.join(root,".qualification-admission.json"))}));};\n'); + if (brokerGit) { + for (const file of ['git.cjs', 'hash.cjs']) write('src/infrastructure/bugbot/shadow/' + file, + fs.readFileSync(path.resolve(__dirname, '../../../../src/infrastructure/bugbot/shadow', file))); + write(prefix + 'normal-flow-shadow/run.cjs', 'exports.run=(lab,number,output)=>{const fs=require("node:fs"),' + + '{git}=require("../../../../../src/infrastructure/bugbot/shadow/git.cjs");' + + 'fs.writeFileSync(output,JSON.stringify({head:git(lab,["rev-parse","HEAD"]).trim()}));};\n'); + } + if (cancellation) write(prefix + 'normal-flow-shadow/run.cjs', + 'exports.run=(lab,number,output)=>new Promise(resolve=>{' + + 'const keepAlive=setInterval(()=>{},1000);process.once("SIGTERM",()=>{clearInterval(keepAlive);' + + 'require("node:fs").writeFileSync(output,"interrupted");resolve(143);});' + + 'process.stdout.write(JSON.stringify({ready:true,pid:process.pid})+"\\n");});\n'); + if (crlfCheckout) write('.gitattributes', '*.cjs text eol=crlf\n'); + git('add', '.'); + git('-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '--quiet', '-m', 'Trusted fixture implementation'); + const implementation = git('rev-parse', 'HEAD'); + const plan = { implementation, publication: 'dry-run', runtime: proof.runtimeIdentity(root), + drivers: Object.fromEntries(drivers.map(file => [file, hash(fs.readFileSync(path.join(root, prefix, 'normal-flow-shadow', file)))])) }; + const planPath = prefix + 'normal-flow-shadow/fixture-plan.json'; + write(planPath, JSON.stringify(plan)); + git('add', planPath); + git('-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '--quiet', '-m', 'Reviewed synthetic preregistration'); + const revision = git('rev-parse', 'HEAD'), planSha256 = hash(fs.readFileSync(path.join(root, planPath))); + const bootstrap = path.join(directory, 'reviewed-bootstrap.cjs'); + fs.writeFileSync(bootstrap, execFileSync(proof.trustedGitExecutable(), ['-C', root, 'show', revision + ':' + prefix + 'qualification-bootstrap.cjs'], { env })); + const bootstrapSha256 = hash(fs.readFileSync(bootstrap)); + const launch = (extra = {}) => spawnSync(process.execPath, [bootstrap, root, revision, planPath, + planSha256, bootstrapSha256, 'normal-flow', brokerGit ? root : 'relative-lab', '1', output], + { cwd: directory, env: { ...env, ...extra }, encoding: 'utf8' }); + return { directory, root, marker, output, write, git, plan, planPath, revision, planSha256, launch, bootstrap }; +} + +test('the reviewed external bootstrap runs immutable drivers in a disposable private source tree', t => { + const f = fixture(t), run = f.launch(); + assert.equal(run.status, 0, run.stderr); + const result = JSON.parse(fs.readFileSync(f.output)); + assert.equal(result.cwd, fs.realpathSync(f.directory)); + assert.equal(result.mode, 'disabled'); + assert.equal(result.admission.planCommit, f.revision); + assert.equal(result.checkoutReceipt, false); + assert.notEqual(result.isolated, path.join(f.root, prefix, 'normal-flow-shadow')); + assert.equal(fs.existsSync(result.isolated), false, 'private source is removed after execution'); + assert.equal(fs.existsSync(f.marker), false); +}); + +for (const driver of ['execution-provenance.cjs', 'normal-flow-shadow/run.cjs']) { + test('a changed ' + driver + ' cannot execute before immutable admission', t => { + const f = fixture(t), file = prefix + driver; + f.write(file, 'require("node:fs").writeFileSync(' + JSON.stringify(f.marker) + ',"executed");\n' + fs.readFileSync(path.join(f.root, file))); + // Causal counter-control: directly loading that working-tree file runs it. + const old = spawnSync(process.execPath, ['-e', 'require(process.argv[1])', path.join(f.root, file)], { encoding: 'utf8' }); + assert.equal(fs.existsSync(f.marker), true, old.stderr); + fs.rmSync(f.marker); + const run = f.launch(); + assert.notEqual(run.status, 0); + assert.match(run.stderr, /working driver differs/u); + assert.equal(fs.existsSync(f.marker), false); + assert.equal(fs.existsSync(f.output), false); + }); +} + +test('an invented matching executable digest in a modified manifest cannot run even its version probe', t => { + const f = fixture(t), executable = path.join(f.directory, 'untrusted-cli'); + fs.writeFileSync(executable, '#!' + process.execPath + '\nrequire("node:fs").writeFileSync(' + + JSON.stringify(f.marker) + ',"executed");process.stdout.write("fixture-cli");\n', { mode: 0o700 }); + const runtime = { executable, codex: 'fixture-cli', codexSha256: hash(fs.readFileSync(executable)) }; + require('./ev07-shadow/implementation.cjs').requirePinnedCliProbe(runtime); + assert.equal(fs.existsSync(f.marker), true, 'the former self-declared runtime admits the executable'); + fs.rmSync(f.marker); + f.write(f.planPath, JSON.stringify({ ...f.plan, runtime })); + const run = f.launch(); + assert.notEqual(run.status, 0); + assert.match(run.stderr, /working plan differs/u); + assert.equal(fs.existsSync(f.marker), false); + assert.equal(fs.existsSync(f.output), false); +}); + +test('materialized checkout transformations are refused before loading the immutable helper', t => { + const f = fixture(t, true), run = f.launch(); + assert.notEqual(run.status, 0); + assert.match(run.stderr, /materialized driver differs/u); + assert.equal(fs.existsSync(f.output), false); +}); + +test('committed manifest authentication admits exact bytes through the supplied repository path', t => { + const f = fixture(t), result = proof.attestCommittedArtifact(f.root, path.join(f.root, f.planPath), f.planSha256, f.revision); + assert.deepEqual(JSON.parse(result.bytes), f.plan); + assert.equal(result.provenance.commit, f.revision); + assert.equal(result.provenance.sha256, f.planSha256); +}); + +for (const mode of ['uncommitted', 'symlink', 'foreign-digest', 'foreign-commit']) { + test('committed manifest authentication refuses ' + mode + ' input before using runtime data', t => { + const f = fixture(t), file = path.join(f.root, f.planPath); + let digest = f.planSha256, revision = f.revision; + if (mode === 'uncommitted') fs.appendFileSync(file, '\n'); + if (mode === 'symlink') { fs.renameSync(file, file + '.original'); fs.symlinkSync(file + '.original', file); } + if (mode === 'foreign-digest') digest = 'f'.repeat(64); + if (mode === 'foreign-commit') revision = f.plan.implementation; + assert.throws(() => proof.attestCommittedArtifact(f.root, file, digest, revision)); + assert.equal(fs.existsSync(f.marker), false); + }); +} + +test('immutable bootstrap ignores Git redirection and PATH Git shims', t => { + const f = fixture(t), bin = path.join(f.directory, 'bin'); + fs.mkdirSync(bin); + fs.writeFileSync(path.join(bin, 'git'), '#!/bin/sh\ntouch "' + f.marker + '"\nexit 91\n', { mode: 0o700 }); + const run = f.launch({ PATH: bin + path.delimiter + process.env.PATH, GIT_DIR: '/unavailable', GIT_WORK_TREE: '/unavailable', + GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: 'core.worktree', GIT_CONFIG_VALUE_0: '/unavailable' }); + assert.equal(run.status, 0, run.stderr); + assert.equal(fs.existsSync(f.marker), false); +}); + +test('changed installed dependencies are refused before the immutable entrypoint runs', t => { + const f = fixture(t); + f.write('node_modules/typescript/index.js', 'require("node:fs").writeFileSync(' + JSON.stringify(f.marker) + ',"executed");\n'); + const run = f.launch(); + assert.notEqual(run.status, 0); + assert.match(run.stderr, /runtime differs: dependencyTreeSha256/u); + assert.equal(fs.existsSync(f.marker), false); + assert.equal(fs.existsSync(f.output), false); +}); + +test('the actual application Git adapter cannot execute an inherited PATH shim after admission', t => { + const f = fixture(t, false, true), bin = path.join(f.directory, 'bin'); + fs.mkdirSync(bin); + fs.writeFileSync(path.join(bin, 'git'), '#!' + process.execPath + '\nconst fs=require("node:fs"),cp=require("node:child_process");' + + 'fs.writeFileSync(' + JSON.stringify(f.marker) + ',"executed");' + + 'process.stdout.write(cp.execFileSync(' + JSON.stringify(proof.trustedGitExecutable()) + ',process.argv.slice(2)));\n', { mode: 0o700 }); + const env = { ...process.env, PATH: bin + path.delimiter + process.env.PATH }; + const old = spawnSync(process.execPath, ['-e', 'process.stdout.write(require(process.argv[1]).git(process.argv[2],["rev-parse","HEAD"]))', + path.join(f.root, 'src/infrastructure/bugbot/shadow/git.cjs'), f.root], { env, encoding: 'utf8' }); + assert.equal(old.status, 0, old.stderr); + assert.equal(fs.existsSync(f.marker), true, 'the old inherited PATH reaches the shim'); + fs.rmSync(f.marker); + const admitted = f.launch({ PATH: env.PATH }); + assert.equal(admitted.status, 0, admitted.stderr); + assert.equal(JSON.parse(fs.readFileSync(f.output)).head, f.revision); + assert.equal(fs.existsSync(f.marker), false, 'the admitted application adapter uses the pinned OS Git'); +}); + +test('a self-consistent checkout receipt is not bootstrap admission', t => { + const f = fixture(t); + const fabricated = { bootstrapSha256: hash(fs.readFileSync(f.bootstrap)), + planCommit: f.revision, planPath: f.planPath, planSha256: f.planSha256 }; + f.write('.qualification-admission.json', JSON.stringify(fabricated)); + const previous = path.join(f.directory, 'previous-verifier.cjs'); + // Preserve the actual historical bytes without depending on an unmerged + // branch commit remaining fetchable after squash or branch cleanup. + const legacy = fs.readFileSync(path.join(__dirname, + '../issue-970/legacy-verifier-01097099.cjs.txt')); + assert.equal(hash(legacy), 'c0471ff35e10cd39ff75a84e02830f4aa98f6f97bb3b47345e241cb77d3877d8'); + fs.writeFileSync(previous, legacy); + assert.deepEqual(require(previous).requireBootstrapAdmission(f.root), fabricated, + 'the previous verifier accepts this receipt without any external bootstrap invocation'); + assert.throws(() => proof.requireBootstrapAdmission(f.root), /independently reviewed immutable bootstrap/u); +}); + +test('a direct runner refuses a forged receipt before importing a checkout verifier', t => { + const f = fixture(t); + f.write('.qualification-admission.json', JSON.stringify({ bootstrapSha256: hash(fs.readFileSync(f.bootstrap)), + planCommit: f.revision, planPath: f.planPath, planSha256: f.planSha256 })); + f.write(prefix + 'shadow-real-prs/official.json', '{}'); + f.write(prefix + 'shadow-verification/plan.json', '{}'); + f.write(prefix + 'execution-provenance.cjs', 'require("node:fs").writeFileSync(' + + JSON.stringify(f.marker) + ',"imported");\n' + fs.readFileSync(path.join(__dirname, 'execution-provenance.cjs'))); + const runner = path.join(f.root, prefix, 'normal-flow-shadow/run.cjs'); + f.write(prefix + 'normal-flow-shadow/run.cjs', fs.readFileSync(path.join(__dirname, 'normal-flow-shadow/run.cjs'))); + const run = spawnSync(process.execPath, [runner, 'lab', '1', f.output, fs.realpathSync(path.join(f.root, f.planPath))], + { cwd: f.directory, encoding: 'utf8' }); + assert.notEqual(run.status, 0); + assert.equal(fs.existsSync(f.marker), false); + assert.match(run.stderr, /independently reviewed immutable bootstrap/u); + assert.equal(fs.existsSync(f.output), false); +}); + +test('the removed internal worker entry refuses direct invocation before any admission', t => { + const f = fixture(t); + const run = spawnSync(process.execPath, [f.bootstrap, '--admitted-worker'], { encoding: 'utf8' }); + assert.notEqual(run.status, 0); + assert.match(run.stderr, /unsupported qualification entrypoint|worker admission.*removed/u); + assert.equal(fs.existsSync(f.marker), false); + assert.equal(fs.existsSync(f.output), false); +}); + +function nestedFixture(t, tamper) { + const f = fixture(t); + const names = ['configuration.cjs', 'implementation.cjs', 'load.cjs', 'corpus.cjs', 'review.cjs', + 'full-review.cjs', 'launch-full-review.cjs', 'score-review.py', '../execution-provenance.cjs', '../qualification-bootstrap.cjs']; + const base = prefix + 'ev07-shadow/'; + for (const name of names.filter(name => !name.startsWith('../'))) f.write(base + name, 'module.exports={};\n'); + const wait = 'const wait=child=>new Promise((resolve,reject)=>{child.once("error",reject);child.once("close",code=>resolve(code??1));});'; + f.write(base + 'launch-full-review.cjs', wait + + 'exports.requirePassingCorpus=campaign=>{require("node:assert/strict").equal(campaign,' + + JSON.stringify(f.directory) + ');return campaign;};exports.run=campaign=>wait(require("../execution-provenance.cjs")' + + '.spawnQualificationDriver("full-review-runner",[campaign]));\n'); + f.write(base + 'full-review.cjs', 'const fs=require("node:fs"),path=require("node:path"),helper=require("../execution-provenance.cjs");' + + wait + 'exports.run=campaign=>{' + + (tamper ? 'const file=path.resolve(__dirname,"../execution-provenance.cjs");' + + 'fs.writeFileSync(file,\'require("node:fs").writeFileSync(\'+JSON.stringify(' + + JSON.stringify(f.marker) + ')+\',"executed");\\n\'+fs.readFileSync(file));' : '') + + 'return wait(helper.spawnQualificationDriver("review",["fixture-checkout","id","1",' + + JSON.stringify(f.output) + '],{stdio:"inherit"}));};\n'); + f.write(base + 'review.cjs', 'exports.run=(checkout,id,repetition,output)=>{const fs=require("node:fs"),path=require("node:path"),' + + 'root=path.resolve(__dirname,"../../../../..");fs.writeFileSync(output,JSON.stringify({admission:' + + 'require("../execution-provenance.cjs").requireBootstrapAdmission(root),' + + 'checkoutReceipt:fs.existsSync(path.join(root,".qualification-admission.json"))}));};\n'); + const plan = { implementation: f.plan.implementation, publication: 'dry-run', runtime: proof.runtimeIdentity(f.root), + inspectedArtifacts: Object.fromEntries(names.map(name => [name, hash(fs.readFileSync(path.join(f.root, base, name)))])) }; + const relative = base + 'fixture-plan.json';f.write(relative, JSON.stringify(plan));f.git('add', '.'); + f.git('-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '--quiet', '-m', 'Reviewed nested-driver fixture'); + const revision = f.git('rev-parse', 'HEAD'); + const result = spawnSync(process.execPath, [f.bootstrap, f.root, revision, relative, + hash(fs.readFileSync(path.join(f.root, relative))), hash(fs.readFileSync(f.bootstrap)), 'full-review', f.directory], + { cwd: f.directory, encoding: 'utf8' }); + return { ...f, result, revision }; +} + +test('nested qualification processes repeat the complete reviewed external launch', t => { + const f = nestedFixture(t, false); + assert.equal(f.result.status, 0, f.result.stderr); + const result = JSON.parse(fs.readFileSync(f.output)); + assert.equal(result.admission.planCommit, f.revision); + assert.equal(result.checkoutReceipt, false); + assert.equal(fs.existsSync(f.marker), false); +}); + +test('each nested worker rechecks private driver bytes before importing a changed verifier', t => { + const f = nestedFixture(t, true); + assert.notEqual(f.result.status, 0); + assert.match(f.result.stderr, /working driver differs/u); + assert.equal(fs.existsSync(f.marker), false); + assert.equal(fs.existsSync(f.output), false); +}); + +test('explicit cancellation reaches the admitted worker and preserves its interrupted exit', { timeout: 30000 }, async t => { + // This timeout cleans up an owned synthetic fixture, never a model investigation. + const f = fixture(t, false, false, true); + const child = spawn(process.execPath, [f.bootstrap, f.root, f.revision, f.planPath, f.planSha256, + hash(fs.readFileSync(f.bootstrap)), 'normal-flow', 'relative-lab', '1', f.output], + { cwd: f.directory, stdio: ['ignore', 'pipe', 'pipe'] }); + let workerPid, stderr = ''; + child.stderr.on('data', bytes => { stderr += bytes; }); + t.after(() => { + if (child.exitCode === null) { + child.kill('SIGKILL'); + if (workerPid) try { process.kill(workerPid, 'SIGKILL'); } catch (error) { if (error.code !== 'ESRCH') throw error; } + } + }); + const [bytes] = await Promise.race([once(child.stdout, 'data'), once(child, 'close').then(() => { + throw new Error('fixture exited before readiness: ' + stderr); + })]); + const ready = JSON.parse(bytes.toString());workerPid = ready.pid;assert.equal(ready.ready, true); + child.kill('SIGTERM'); + const [code, signal] = await once(child, 'close'); + assert.equal(signal, null, stderr); + assert.equal(code, 143, stderr); + assert.equal(fs.readFileSync(f.output, 'utf8'), 'interrupted'); +}); + +test('an arbitrary IPC parent cannot initialize qualification admission, unlike the frozen previous worker', { timeout: 30000 }, async t => { + // An owned model-free process cleanup, never an investigation deadline. + const previous = fs.readFileSync(path.join(__dirname, '../issue-970/legacy-bootstrap-f583e40c.cjs.txt')); + assert.equal(hash(previous), 'e809d86426f33ac20bae21cf463253fcbf8f489aff4888c532e82afa40d2f609'); + for (const legacy of [true, false]) { + const f = fixture(t, false, false, false, legacy ? previous : undefined); + const child = require('node:child_process').fork(f.bootstrap, ['--admitted-worker'], + { cwd: f.directory, execArgv: [], stdio: ['ignore', 'pipe', 'pipe', 'ipc'] }); + let stderr = ''; child.stderr.on('data', bytes => { stderr += bytes; }); + t.after(() => { if (child.exitCode === null) child.kill('SIGKILL'); }); + child.send({ root: f.root, proof: { planCommit: f.revision, planPath: f.planPath, + planSha256: f.planSha256, bootstrapSha256: hash(fs.readFileSync(f.bootstrap)) }, + mode: 'normal-flow', args: ['relative-lab', '1', f.output] }); + const [code] = await once(child, 'close'); + if (legacy) { + assert.equal(code, 0, stderr); + const result = JSON.parse(fs.readFileSync(f.output)); + assert.equal(result.admission.planCommit, f.revision, + 'the real previous worker accepts the arbitrary IPC parent without the public launch path'); + } else { + assert.notEqual(code, 0, 'arbitrary IPC parent initialized bootstrap context: ' + stderr); + assert.match(stderr, /unsupported qualification entrypoint|worker admission.*removed/u); + assert.equal(fs.existsSync(f.output), false); + } + assert.equal(fs.existsSync(f.marker), false); + } +}); diff --git a/quality/bugbot/evidence/issue-970/README.md b/quality/bugbot/evidence/issue-970/README.md new file mode 100644 index 000000000..30e7fc823 --- /dev/null +++ b/quality/bugbot/evidence/issue-970/README.md @@ -0,0 +1,358 @@ +# Admission follow-up controls — #970 + +The second native review correctly identified two remaining gaps: installed +package bytes were outside the runtime proof, and the corpus version probe used +PATH before binary admission. Both have deterministic counter-controls. + +The complete installed dependency inventory now binds every regular file's +bytes/mode and every internal symlink/target, including absent platform-optional +targets. Materializing an absent optional target changes the proof. External +links and nested dependency shadows along qualification-driver ancestors are +rejected. The one normal pnpm portal to this repository browser-extension +workspace is recorded as unadmitted; synchronous Node resolve hooks refuse its +physical path for both require and import before application modules load. Node preload/search overrides are not admitted. The expected inventory +hash and entry count must come from the exact reviewed committed runtime plan; +launching never silently snapshots a new expected tree. + +Use an isolated lab installed from its pinned package/lockfile with the reviewed +Node/Corepack/pnpm bootstrap, frozen lockfile, ignored lifecycle/pnpm hooks and +verified store integrity. The final explicit pnpm 10.12.4 installation uses +`--frozen-lockfile --ignore-scripts --ignore-pnpmfile --verify-store-integrity +--package-import-method=copy --offline --force`; command/output is retained. +An earlier isolated control removed the unused workspace portal; the final +implementation preserves normal workspace installation and proves that portal +cannot execute through require or import. Do not mutate a user's working tree. +The trusted bootstrap/install procedure is a required operator trust anchor; +this inventory does not independently authenticate npm or attest a compromised +host. A lockfile hash alone is insufficient. + +The first dependency control run passed 36 of 38 tests and refused two actual +runtime fixtures because pnpm retains dangling links to platform-excluded +optional packages. The correction binds those absent internal targets rather +than silently ignoring them. That failed control and the subsequent 38/38 pass +are preserved. The intermediate full suite then passed **170/170** controls. Two further +workspace-portal controls cover require and dynamic import, including the normal +installed workspace layout; the final complete suite passes **172/172** controls. In the causal +negative control, removing only the new dependency fields from the old proof +lets a modified package execute; the corrected proof rejects it before import. +The real entrypoint controls now also detect an early @actions/github import; +that import was moved behind source/driver/runtime admission. + +The CLI version probe validates the preregistered absolute executable and optional +launcher bytes before running either. It uses that exact invocation with a clean +environment and then verifies the returned version. Controls cover direct and +launcher execution, changes refused before execution, and an actual PATH shim +that runs under the old bare-name probe but never under the admitted probe. +The normal-flow runner shares the admitted probe. GitHub evidence credentials +are supplied through existing read-only token environment bindings; the runner +no longer invokes a PATH-resolved gh auth subprocess. Prepared runtime checks +still run before model turns. No control calls models or +writes to an evaluated PR. No profile is qualified and production stays disabled. + +Future plans need installed-tree identity and the version-probe executable / +optional launcher paths in addition to the existing Node/Git/TypeScript and +prepared-runtime hashes. Old plans and campaigns remain immutable and must be +reproduced only at their frozen drivers. #974 remains blocked until reviewed +prerequisites merge and a new exact preregistration is committed. + +The loader fence uses the official synchronous module hook contract from the +[pinned Node v24.20.0 documentation](https://github.com/nodejs/node/blob/v24.20.0/doc/api/module.md#customization-hooks). +It rejects non-physical module URLs and real paths outside admitted source, +dependencies and qualification drivers; the workspace portal is never treated +as verified executable dependency bytes. + + +## Independently reviewed bootstrap and manifest + +The supported qualification route starts outside the source checkout. An operator +must select an independently reviewed bootstrap commit and SHA-256, materialize +its Git blob with trusted operating-system Git, verify that SHA-256 **before** +starting trusted Node, and clear NODE_OPTIONS/NODE_PATH. Running a working-tree +entrypoint or verifier directly is not qualification admission. Neither a +self-declared manifest nor a receipt generated by that checkout establishes +this trust anchor; a compromised host or substituted approved digest is outside +this local proof. + +The bootstrap takes repository, reviewed plan commit, repository-relative plan +path, externally reviewed plan SHA-256, externally reviewed bootstrap SHA-256, +mode (`corpus`, `full-review`, `normal-flow`), then that mode's arguments. +Nested scheduler/review processes use the same complete launch with +`full-review-runner ` or `review `; +they do not accept an internal worker-admission message. It +checks the immutable committed plan and the complete fixed driver inventory +against their reviewed bytes before importing any driver or helper. It rejects +changed physical drivers and symlinks, uses a sterile local Git clone to +materialize the committed revision, and loads the verifier only from there. +It verifies original source/runtime, copies physical dependencies to the private +source tree, verifies the copy again, and executes that immutable entrypoint. +The disposable directory is not part of the stable recorded identity. Explicit +signals are forwarded; no investigation time/turn/read/token ceiling is added. + +Normal-flow authenticates its exact committed manifest and externally selected +digest before using runtime fields, including the version probe. The bootstrap +records its own reviewed digest and the reviewed plan revision/path/digest; +corpus and full-review provenance compare those same values. It forces the child +production mode to disabled and preserves dry-run publication. An auxiliary +Python used by old diagnostic workflow-input plans now needs explicit executable +and digest pins; it cannot be selected from an unchecked environment override. + +Ten new deterministic controls use real Git objects, a materialized external +bootstrap, physical source/dependency installations and zero model/GitHub calls. +Two causal controls first demonstrate the old direct working-tree execution of +a tampered verifier/entrypoint, then show that the bootstrap rejects each before +its marker executes. A crafted self-declared CLI digest similarly executes under +the former unauthenticated probe but is refused as an unreviewed manifest by the +bootstrap. Additional cases cover isolated success and cleanup, uncommitted / +symlink / foreign digest / foreign commit manifests, Git environment/PATH shims, +and dependency tampering. The first run passed 9/10: the positive assertion used +macOS's logical /var path instead of its physical /private/var equivalent. Both +outputs remain byte-exact gzip; the corrected physical-path assertion passes 10/10. +The existing actual normal-flow relative-lab control now commits its synthetic +manifest and launches the immutable bootstrap rather than an untracked manifest. + +An integration attempt made before committing the new package/driver snapshot is retained as `bootstrap-integration-before-commit.log.gz`: the source guard rejects the edited package manifest and the real launch fixture cannot extract the not-yet-committed bootstrap. These are not passing results. Re-run the full controls after committing the exact implementation; no old plan or guard is changed to bypass those failures. + + +The first full committed integration passes 181/182 controls and preserves its +failed real normal-flow result in `bootstrap-committed-shadow.log.gz`. A logical +macOS /var manifest path was compared with the physical /private/var repository +path; the guard correctly refused it before a CLI probe. The bootstrap now uses +physical scratch paths and generic committed-artifact admission maps only the +supplied repository-root alias before checking every child for symlinks. A new +positive committed-manifest control covers that boundary. The real runner control +commits its current driver snapshot inside the disposable fixture, so it does +not depend on uncommitted contributor files being present in an older Git blob. + +A further deterministic case forces CRLF checkout transformation through Git +attributes. The bootstrap now compares every materialized private driver and plan +with immutable blobs **again before requiring the private helper**; transformed +checkout bytes are refused. The bootstrap suite is now 12 controls; together with +the complete source/path suite, all 40 focused controls pass, including the actual +normal-flow relative-lab journey with zero candidates/model calls/verifiers/writes. + +Final complete shadow suite: **184/184 passed**, zero skipped/cancelled/model calls/evaluated-PR writes. The exact complete output is `bootstrap-final-shadow.log.gz`. Specifications/documentation and the final graph refresh also pass. The required final build/Jest/lint hook and native final-head review remain distinct delivery checks. + + +## Actual application Git adapter after admission + +The source proof already used absolute OS Git, but the actual shipped evidence +adapter in src/infrastructure/bugbot/shadow/git.cjs resolves its later read-only +Git calls through PATH. The admitted child previously inherited the caller PATH. +A new causal control uses that actual adapter: under the old inherited path a +Git shim executes even though the OS-Git proof itself is genuine. The bootstrap +now prepends a private directory containing only a git link to the admitted OS +executable. Later application Git reads therefore use the same pinned binary; +selection of Node, Codex and other normal executables retains the remaining PATH. +No production source or new configuration setting is added. + +The same actual adapter runs successfully through the admitted child without +executing the shim. All 13 bootstrap controls pass. The private executable path +is excluded from stable provenance; its target is the recorded OS-Git executable +and digest, and the reviewed bootstrap digest binds this routing. Historical +184-control results remain immutable; the complete suite now includes one +additional application-adapter control. + + +The end-to-end probe audit also found the normal planner's own --version call: +AgentExecutionPlanner.prepare invokes its production version reader before it +returns the plan that the driver previously checked. Qualification now resolves +the selected invocation with those same production helpers and checks executable/ +launcher bytes before calling the unchanged ordinary planner. The final returned +plan still passes the existing version/digest/profile proof. No production source, +planner API, monkeypatch, model limit or configuration setting is changed. + +Two new controls run the actual production planner with only a fixture workspace +port; executable resolution, ownership validation and version reading remain real. +The former preparation executes a mismatched CLI's version marker before its +later digest rejection; the admitted preparation refuses it before that marker. +The matching runtime still prepares normally. The first run passed 14/15 runtime +controls but compared a logical macOS /var path with its physical alias. That +assertion failure remains; the corrected realpath assertion passes all 15. The +prior complete 185-control result is preserved separately; the final suite now +contains 187 controls, including the two actual planner cases. + +The final combined source/runtime suite passes **187/187** controls with no skipped/cancelled cases, model calls or evaluated-PR writes (`planner-final-shadow-187.log.gz`). Final specification/catalog/documentation validation and Graphify pass. Application source and shipping bundles remain unchanged; all routing/admission changes are in the registered lab drivers. + + +The first final planner-admission build/Jest/lint hook failed in the existing +`src/prompts/__tests__/bugbot_complete_conversation.test.ts` worker with SIGSEGV: +942 suites and 18,689 tests passed; that six-test suite did not run. The exact +unchanged six tests then pass in isolation. This is preserved as a failed full +attempt, followed by a complete hook retry; the isolated pass does not replace +the required complete validation. Both raw outputs are lossless gzip. + +- `planner-final-hook-sigsegv.log.gz`: original bytes 526237, SHA-256 `03e97b4a5a1b5e74815262999194f81a7df311359c5cfb5229790c9255010eef`. +- `planner-final-hook-focused.log.gz`: original bytes 193, SHA-256 `677fe194c74955c922514e93ef4010100248ff690cc0c8403599582800c4b98b`. + + +## Forged checkout receipt and external execution context + +The final01097099 native review found that a checkout-written admission file +could supply its own matching bootstrap/plan digests. The actual old verifier +accepts that fabricated record without any external bootstrap invocation. A +second causal control observes the direct runner importing a checkout verifier. +Both former behaviors fail the new rejection assertions; these failures remain. +The first negative run also exposed the existing logical macOS path alias in a +fixture argument; the corrected counter-control uses its physical manifest path. + +The historical f583e40c implementation below was subsequently superseded by +the complete-public-launch correction at the end of this document. + +Qualification no longer reads or creates .qualification-admission.json. Direct +driver CLIs reject before helper imports. After admitting source/dependencies, +the independently reviewed external bootstrap forks that same external file, +sends verified context through its parent IPC channel, repeats private-byte and +runtime verification in the worker, and calls the driver as a module. Context +is process-local and bound to that executing bootstrap identity. Nested paired +workers follow the same path and recheck their corpus gate. IPC and in-process +context are transport, not independent host or human-review attestation. The +operator must still verify the external Git blob/hash before starting trusted Node. + +Six new controls cover fabricated receipt, direct pre-import refusal, missing +worker IPC, actual nested positive route, changed private verifier before a nested +import, and explicit SIGTERM propagation. The nested fixture uses pinned fake +corpus/worker ports to test transport; it is no corpus or qualification result. +The cancellation fixture initially exited before handling the signal; that failure +is retained. Its owned keep-alive now models an active worker and is cleared on +cancellation. The fixture-only cleanup timeout never bounds a model investigation. + +All **193/193** combined controls pass, with zero skipped/cancelled tests, model +calls or evaluated-PR writes. The prior **192/192** run before adding the explicit +signal case is retained separately; the bootstrap minimum is now19 cases. +Application source, shipping bundles, consumer templates and frozen historical +campaign bytes remain unchanged. Qualification driver identities change and need +a new exact preregistration before any future measurement. +Full mandatory hook, specification/docs/catalog checks and new-SHA native review +remain separate delivery checks; no new campaign, installation or qualification +is claimed by these deterministic results. + +Lossless raw-log inventory: + +The real previous verifier from commit `01097099d220bd481f6e50b58d2213165bcf6194` +is also frozen as `legacy-verifier-01097099.cjs.txt` (15,205 original bytes, +SHA-256 `c0471ff35e10cd39ff75a84e02830f4aa98f6f97bb3b47345e241cb77d3877d8`). +The causal control checks that digest before using the code on its disposable +fixture. It does not rely on the managed branch remaining fetchable after squash. + +- `forged-receipt-first-negative.log.gz`: 4835 original bytes; SHA-256 `cec885537a2dd908d329fadc5a993a94d1e3d56371635901ecc2dadf62da465a`. +- `forged-receipt-causal-negative.log.gz`: 2081 original bytes; SHA-256 `a87078ab93b86fc2a61cc8e214c9e45548d09aff8bdb33afefb55ec7f62d9efb`. +- `forged-receipt-focus-pass.log.gz`: 293 original bytes; SHA-256 `a3faa77cb0e9c3def85be302a2370c1dfd05e472601bc4bd96c238668a491199`. +- `memory-bootstrap-18-pass.log.gz`: 2886 original bytes; SHA-256 `88e211735b9318dd0252d15dd3d9a72f76082892b563e73e753f493cd9116c33`. +- `memory-shadow-192-pass.log.gz`: 20375 original bytes; SHA-256 `0fe14ffcd31dc7c648b36e3459da9c535dbf2c975de6d99c251f81a88cd8c1cc`. +- `memory-cancel-fixture-negative.log.gz`: 1163 original bytes; SHA-256 `085cd38dd9830364811559c629e2771d18fa27e9729cf4c8cfaa81705737f8fc`. +- `memory-cancel-pass.log.gz`: 218 original bytes; SHA-256 `d850cb50598e801927753bb0cbc4e05abe49c4e57db5e12b5bf6735457470853`. +- `memory-shadow-193-pass.log.gz`: 20461 original bytes; SHA-256 `55fedbc9dcd2c2d62c81c4e16e3110a93d5daf27505a214d4a04d0557b0d3404`. + +The final replay using the frozen legacy source also passes **193/193** controls. + +- `memory-frozen-final-shadow-193.log.gz`: 20469 original bytes; SHA-256 `5f80b7559e8a4085089ff8d0a0b5b111e55dcc67804c29fac29219cf57aef2c4`. + + +Native Commit on f583e40c rejected the compressed historical verifier as required +binary code (validation.evidence-unavailable, non-retryable). No model progress +assessment was made and previous progress was retained. The counter-control now +stores the exact same 15,205 source bytes as reviewable .cjs.txt data; its pinned +SHA-256 remains unchanged. Current-base diff no longer includes a binary verifier. +No evidence path is excluded or reclassified to bypass that check. Historical +gzip remains in the f583 commit, and all negative outputs remain lossless. + +- `memory-native-binary-source-failure.log.gz`: 36085 original bytes; SHA-256 `e9c4c3bec1b138f99af7bbb9d04720a4452286830ee5d5aefca4266835c58f32`. + +The complete replay with the readable historical source passes **193/193**, zero skipped/cancelled/model calls/evaluated-PR writes. + +- `readable-legacy-shadow-193.log.gz`: 20493 original bytes; SHA-256 `49fd783acc93c3c17fbdb88f41dbb57d463095e7a0afb9500a4af2906201ba90`. + + +## Every process uses the complete reviewed launch + +Native review on f583e40c found that its internal --admitted-worker endpoint +accepted a self-pinned request from any IPC-connected parent. The new causal +control executes the actual frozen previous bootstrap and observes successful +admission without calling public launch. It then checks that the current code +rejects the same invocation before driver execution. The historical bootstrap +is reviewable UTF-8 data, `legacy-bootstrap-f583e40c.cjs.txt`, 14,309 bytes, +SHA-256 `e809d86426f33ac20bae21cf463253fcbf8f489aff4888c532e82afa40d2f609`. + +The worker endpoint and IPC message handler are removed. The external bootstrap +now authenticates and materializes the closure and calls the driver in that same +process. Each nested scheduler/review process starts the external bootstrap with +the same explicit reviewed plan and bootstrap pins, repeats the entire public +validation/materialization/dependency-copy proof, then installs process-local +context. Review slots retain the exact parent corpus campaign and validate its +matching passing evidence. A caller-supplied IPC object cannot initialize context. +The five closed modes have fixed argument counts; there is no acceptance of an +internal worker request or persisted admission receipt. Operator review of the +external digest remains essential; this is no independent human/host attestation. + +The strengthened nested positive control asserts the exact corpus campaign at +the final slot, in addition to the plan identity. The prior first20-case local +pass used a permissive fake corpus gate and is retained as that limited result. +Explicit SIGTERM still reaches the executing admitted driver; the same readiness, +interrupted-output/exit and cleanup control applies. No model or evaluated-PR +calls, new consumer option, cohort change or investigation limit is introduced. + +The previous mandatory hook attempt failed with a SIGSEGV in the existing +branch_sync_resolved_workspace suite: 942 suites /18,689 tests passed. Its six +unchanged tests pass in isolation. The raw failure and isolated result are +retained, and the complete hook must run again on the final correction. +The initial standalone IPC diagnostic did not load its helper; a second +standalone diagnostic rewrote a generated driver path incorrectly and therefore +refused for the wrong reason. Neither establishes the vulnerability or the fix. +The in-repository causal failure does: the unchanged real previous worker returns +zero and writes admitted output; the corrected assertion rejects that behavior. + +- `readable-legacy-hook-sigsegv.log.gz`: 463058 original bytes; SHA-256 `40dac502be2bfb2b796f45e8c35ec1a50c82a5ecbd452c2347170b4248ce7352`. +- `readable-legacy-hook-isolated-six.log.gz`: 209 original bytes; SHA-256 `11895ae53f61f01219ce0e18a30231d0f953bdefb8375c77771d9179f3b6f458`. +- `ipc-counter-import-diagnostic.log.gz`: 1351 original bytes; SHA-256 `259e2328dccacf45cbec1972e7517fc7ca29625950b4be8a226116de4563079b`. +- `ipc-counter-path-diagnostic.log.gz`: 279 original bytes; SHA-256 `0f88853d8605aecdcd40a76ee63c43c48912586f0b60c5d90eb8c9590ac0cca3`. +- `ipc-parent-causal-negative.log.gz`: 1089 original bytes; SHA-256 `ea981eb57d99675c9d3b1b96ff15fc153879f1260cac9dac6304880ac846baa7`. +- `public-launch-bootstrap-first-20.log.gz`: 3086 original bytes; SHA-256 `74abfe52a06f3d91dcd9335bfc4c3e8250b8a26b2de4e56004f913a902cf23e7`. + +Final strengthened bootstrap controls: **20/20** pass. The full combined suite +passes **194/194**, zero skipped/cancelled/model calls/evaluated-PR writes, +including the actual relative-lab zero-candidate journey. Specification/catalog, +documentation and Graphify validation pass. The mandatory complete hook and +new-SHA native review remain separate outstanding checks. + +- `public-launch-bootstrap-final-20.log.gz`: 3078 original bytes; SHA-256 `dd36c3d82104459d248551924674cbb8b0497bfce81fb19e0fc9af0766bfb3fd`. +- `public-launch-shadow-194.log.gz`: 20219 original bytes; SHA-256 `b21d59d3f604cb4168a71ee8f25b8ff3843fa19e1b1a3d4fa8ae065d7a3dfb57`. + + +## Executable version health probe + +Native Review on `7652c1c5` resolved all seven earlier findings, then identified the +unbounded synchronous --version probe (LOW, thread `PRRT_kwDONSeBW865y_KP`). Its +complete native CI/Commit/Verification/Branch Sync pass; Review remains NEUTRAL +with that finding. Both added real-process rejection assertions fail against +the unchanged `7652c1c5` implementation: the admitted direct CLI and optional launcher +actually enter --version and remain blocked until owned 30-second test cleanup. +The launcher deliberately handles SIGTERM without exiting. Neither fixture calls +a model or an evaluated PR. + +The correction uses the ordinary production version reader's 15,000 ms operational +health timeout, plus SIGKILL so a stalled probe cannot keep the synchronous call +waiting by handling SIGTERM. Bytes/launcher admission still comes first; version +text still must match the reviewed runtime. This applies only to --version +metadata before any model call. No investigation time/turn/read/token limit, +cohort/order/retry change or consumer option is added. Ordinary production source +and model/cancellation policy remain unchanged. Future measurements still require +new driver pins. + +The two real-process cases require ETIMEDOUT, exact --version arguments and no +surviving fixture CLI. Their 30-second cleanup is only for owned model-free test +processes. Together with existing controls the budget is 17 runtime cases and 196 +combined controls; final results and mandatory hook/native outcomes are separate. +The fixed timeout/SIGKILL semantics follow the +[pinned Node v24.20.0 subprocess contract](https://nodejs.org/download/release/v24.20.0/docs/api/child_process.html#child_processexecfilesyncfile-args-options). + +- `version-health-causal-negative.log.gz`: 1963 original bytes; SHA-256 `9c4385e2bce6129024b292bade23cc8ba31f76a635065f04452c5d2b0a8c2103`. + +Both focused real-process cases pass, followed by **196/196** complete controls, +zero skipped/cancelled/model calls/evaluated-PR writes. The full suite includes +all 17 runtime cases, 20 bootstrap cases and the actual relative-lab zero-candidate +normal-flow journey. Final specification/catalog/documentation validation and +Graphify pass. The complete mandatory hook and new-SHA native outcomes remain +separate delivery checks. + +- `version-health-focus-pass.log.gz`: 297 original bytes; SHA-256 `c8ca995662705c8a0159358e3c0b8c1c2c65afd722ad92e8b102e0005e39f0b3`. +- `version-health-shadow-196.log.gz`: 20413 original bytes; SHA-256 `ef572c6320a0ba3e03560695c607799074b347056af85c8ff3ad4f893611b85b`. diff --git a/quality/bugbot/evidence/issue-970/all-admission-controls.log.gz b/quality/bugbot/evidence/issue-970/all-admission-controls.log.gz new file mode 100644 index 000000000..2d60af07f Binary files /dev/null and b/quality/bugbot/evidence/issue-970/all-admission-controls.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/application-git-controls.log.gz b/quality/bugbot/evidence/issue-970/application-git-controls.log.gz new file mode 100644 index 000000000..3d833a44d Binary files /dev/null and b/quality/bugbot/evidence/issue-970/application-git-controls.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/application-git-shadow-185.log.gz b/quality/bugbot/evidence/issue-970/application-git-shadow-185.log.gz new file mode 100644 index 000000000..124310338 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/application-git-shadow-185.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/bootstrap-committed-shadow.log.gz b/quality/bugbot/evidence/issue-970/bootstrap-committed-shadow.log.gz new file mode 100644 index 000000000..4ba6aeaf5 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/bootstrap-committed-shadow.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/bootstrap-final-focused.log.gz b/quality/bugbot/evidence/issue-970/bootstrap-final-focused.log.gz new file mode 100644 index 000000000..379cfe73d Binary files /dev/null and b/quality/bugbot/evidence/issue-970/bootstrap-final-focused.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/bootstrap-final-shadow.log.gz b/quality/bugbot/evidence/issue-970/bootstrap-final-shadow.log.gz new file mode 100644 index 000000000..ae9de17fa Binary files /dev/null and b/quality/bugbot/evidence/issue-970/bootstrap-final-shadow.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/bootstrap-integration-before-commit.log.gz b/quality/bugbot/evidence/issue-970/bootstrap-integration-before-commit.log.gz new file mode 100644 index 000000000..7f039feb4 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/bootstrap-integration-before-commit.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/bootstrap-tests-1.log.gz b/quality/bugbot/evidence/issue-970/bootstrap-tests-1.log.gz new file mode 100644 index 000000000..9df8d50bd Binary files /dev/null and b/quality/bugbot/evidence/issue-970/bootstrap-tests-1.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/bootstrap-tests-2.log.gz b/quality/bugbot/evidence/issue-970/bootstrap-tests-2.log.gz new file mode 100644 index 000000000..08394a01c Binary files /dev/null and b/quality/bugbot/evidence/issue-970/bootstrap-tests-2.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/explicit-verified-install.log.gz b/quality/bugbot/evidence/issue-970/explicit-verified-install.log.gz new file mode 100644 index 000000000..85c953a65 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/explicit-verified-install.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/final-shadow.log.gz b/quality/bugbot/evidence/issue-970/final-shadow.log.gz new file mode 100644 index 000000000..8576cf770 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/final-shadow.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/forged-receipt-causal-negative.log.gz b/quality/bugbot/evidence/issue-970/forged-receipt-causal-negative.log.gz new file mode 100644 index 000000000..d407d9525 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/forged-receipt-causal-negative.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/forged-receipt-first-negative.log.gz b/quality/bugbot/evidence/issue-970/forged-receipt-first-negative.log.gz new file mode 100644 index 000000000..e177f07ce Binary files /dev/null and b/quality/bugbot/evidence/issue-970/forged-receipt-first-negative.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/forged-receipt-focus-pass.log.gz b/quality/bugbot/evidence/issue-970/forged-receipt-focus-pass.log.gz new file mode 100644 index 000000000..0850951e2 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/forged-receipt-focus-pass.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/ipc-counter-import-diagnostic.log.gz b/quality/bugbot/evidence/issue-970/ipc-counter-import-diagnostic.log.gz new file mode 100644 index 000000000..07b7ec7f0 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/ipc-counter-import-diagnostic.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/ipc-counter-path-diagnostic.log.gz b/quality/bugbot/evidence/issue-970/ipc-counter-path-diagnostic.log.gz new file mode 100644 index 000000000..6995b3ab4 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/ipc-counter-path-diagnostic.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/ipc-parent-causal-negative.log.gz b/quality/bugbot/evidence/issue-970/ipc-parent-causal-negative.log.gz new file mode 100644 index 000000000..92a4b1cf5 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/ipc-parent-causal-negative.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/legacy-bootstrap-f583e40c.cjs.txt b/quality/bugbot/evidence/issue-970/legacy-bootstrap-f583e40c.cjs.txt new file mode 100644 index 000000000..612c7634e --- /dev/null +++ b/quality/bugbot/evidence/issue-970/legacy-bootstrap-f583e40c.cjs.txt @@ -0,0 +1,224 @@ +'use strict'; + +// Trust begins OUTSIDE the checkout: materialize this file from an independently +// reviewed Git revision, compare its reviewed SHA-256, then start trusted Node +// with NODE_OPTIONS/NODE_PATH unset. See issue-970/README.md. This file uses only +// Node core until the immutable verifier and every driver have been admitted. +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const os = require('node:os'); +const { createHash } = require('node:crypto'); +const { execFileSync, fork } = require('node:child_process'); +const sha256 = bytes => createHash('sha256').update(bytes).digest('hex'); +const prefix = 'quality/bugbot/evidence/issue-950/'; +const ev07 = ['configuration.cjs', 'implementation.cjs', 'load.cjs', 'corpus.cjs', 'review.cjs', + 'full-review.cjs', 'launch-full-review.cjs', 'score-review.py', '../execution-provenance.cjs', '../qualification-bootstrap.cjs']; +const normal = ['run.cjs', '../execution-provenance.cjs', '../ev07-shadow/load.cjs', + '../ev07-shadow/implementation.cjs', '../ev07-shadow/configuration.cjs', '../qualification-bootstrap.cjs']; +const entries = Object.freeze({ corpus: 'ev07-shadow/corpus.cjs', + 'full-review': 'ev07-shadow/launch-full-review.cjs', 'normal-flow': 'normal-flow-shadow/run.cjs', + 'full-review-runner': 'ev07-shadow/full-review.cjs', review: 'ev07-shadow/review.cjs' }); + +function safePath(value) { + assert(typeof value === 'string' && value.length && !path.isAbsolute(value) + && !value.includes('\\') && !value.split('/').some(part => ['', '.', '..', '.git'].includes(part)), + 'unsafe committed artifact path'); + return value; +} + +function physical(root, relative) { + let file = root; + for (const part of safePath(relative).split('/')) { + file = path.join(file, part); + assert(!fs.lstatSync(file).isSymbolicLink(), 'qualification artifact symlink refused'); + } + assert(fs.lstatSync(file).isFile(), 'qualification artifact must be a regular file'); + return file; +} + +function gitFor(root) { + const file = process.platform === 'darwin' ? execFileSync('/usr/bin/xcrun', ['--find', 'git'], + { env: { PATH: '/usr/bin:/bin' }, encoding: 'utf8' }).trim() + : process.platform === 'linux' ? '/usr/bin/git' : undefined; + assert(file && path.isAbsolute(file), 'trusted Git requires Linux or macOS'); + const executable = fs.realpathSync(file), marker = path.join(root, '.git'); + assert(!fs.lstatSync(marker).isSymbolicLink(), 'Git identity symlink refused'); + let directory = marker; + if (fs.lstatSync(marker).isFile()) { + const match = /^gitdir: (.+)\r?\n?$/u.exec(fs.readFileSync(marker, 'utf8')); + assert(match, 'invalid worktree Git identity'); + directory = path.resolve(root, match[1]); + } + const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.toUpperCase().startsWith('GIT_'))); + Object.assign(env, { GIT_CONFIG_GLOBAL: os.devNull, GIT_CONFIG_NOSYSTEM: '1', GIT_NO_REPLACE_OBJECTS: '1' }); + const git = (...args) => execFileSync(executable, ['--no-pager', '--no-replace-objects', '--literal-pathspecs', + '--git-dir=' + directory, '--work-tree=' + root, '-C', root, + '-c', 'core.hooksPath=' + os.devNull, '-c', 'core.fsmonitor=false', ...args], + { env, maxBuffer: 256 * 1024 * 1024, stdio: ['pipe', 'pipe', 'pipe'] }); + return { git, executable, env, directory }; +} + +function committed(git, revision, relative) { + safePath(relative); + const rows = git('ls-tree', '-z', revision, '--', relative).toString('utf8').split('\0').filter(Boolean); + assert.equal(rows.length, 1, 'committed artifact unavailable'); + const [header, name] = rows[0].split('\t'), [mode, type, blob] = header.split(' '); + assert(['100644', '100755'].includes(mode) && type === 'blob' && name === relative, + 'committed artifact must be a regular blob'); + return git('cat-file', 'blob', blob); +} + +function reviewedPlan(repository, revision, planPath, planSha256, bootstrapSha256, mode) { + assert(!process.env.NODE_OPTIONS && !process.env.NODE_PATH && process.execArgv.length === 0, + 'bootstrap requires trusted Node without preloads or loader overrides'); + assert.match(revision ?? '', /^[a-f0-9]{40}$/u, 'reviewed plan commit required'); + assert.match(planSha256 ?? '', /^[a-f0-9]{64}$/u, 'independently reviewed plan digest required'); + assert.match(bootstrapSha256 ?? '', /^[a-f0-9]{64}$/u, 'independently reviewed bootstrap digest required'); + assert.equal(sha256(fs.readFileSync(__filename)), bootstrapSha256, 'reviewed bootstrap differs'); + assert(Object.hasOwn(entries, mode), 'unsupported qualification entrypoint'); + const root = fs.realpathSync(repository), trust = gitFor(root), { git } = trust; + assert.equal(git('rev-parse', '--verify', revision + '^{commit}').toString('utf8').trim(), revision); + assert.equal(sha256(committed(git, revision, prefix + 'qualification-bootstrap.cjs')), bootstrapSha256, + 'plan revision does not contain the independently reviewed bootstrap'); + const bytes = committed(git, revision, planPath); + assert.equal(sha256(bytes), planSha256, 'reviewed committed plan differs'); + assert(fs.readFileSync(physical(root, planPath)).equals(bytes), 'working plan differs from reviewed commit'); + const plan = JSON.parse(bytes); + assert.equal(plan.publication, 'dry-run', 'qualification must remain dry-run'); + assert.match(plan.implementation ?? '', /^[a-f0-9]{40}$/u); + git('merge-base', '--is-ancestor', plan.implementation, revision); + const base = prefix + (mode === 'normal-flow' ? 'normal-flow-shadow/' : 'ev07-shadow/'); + const inventory = mode === 'normal-flow' ? normal : ev07; + const expected = mode === 'normal-flow' ? plan.drivers : plan.inspectedArtifacts; + const admitted = []; + for (const driver of inventory) { + const relative = path.posix.normalize(base + driver), trusted = committed(git, revision, relative); + assert.equal(sha256(trusted), expected?.[driver], 'committed driver differs from reviewed plan: ' + driver); + assert(fs.readFileSync(physical(root, relative)).equals(trusted), 'working driver differs: ' + driver); + admitted.push([relative, trusted]); + } + return { root, trust, bytes, plan, admitted }; +} + +function spawnAdmitted(request, env, options = {}) { + assert(Object.keys(options).every(key => key === 'stdio'), 'unsupported qualification child options'); + assert.equal(sha256(fs.readFileSync(__filename)), request.proof.bootstrapSha256, 'executing bootstrap changed'); + const io = options.stdio ?? 'inherit'; + const stdio = Array.isArray(io) ? [...io, 'ipc'] : [io, io, io, 'ipc']; + const child = fork(__filename, ['--admitted-worker'], { env, stdio, execArgv: [], cwd: process.cwd() }); + child.send(request, error => { if (error) child.emit('error', error); }); + return child; +} + +async function executeAdmitted(request) { + assert(request && Array.isArray(request.args) && request.args.every(arg => typeof arg === 'string'), + 'invalid admitted invocation'); + const { root: repository, proof, mode } = request; + const { root, plan } = reviewedPlan(repository, proof.planCommit, proof.planPath, + proof.planSha256, proof.bootstrapSha256, mode); + // The executing file is still the externally materialized bootstrap. Repeat + // private-byte/source/runtime checks before loading any private driver in + // each child. No receipt in this checkout is read as execution authority. + const verifier = require(path.join(root, prefix, 'execution-provenance.cjs')); + verifier.attestSource(root, plan.implementation); + verifier.attestRuntime(root, plan.runtime); + verifier.initializeBootstrapContext(root, proof, (nextMode, args, options) => { + assert(mode === 'full-review' && nextMode === 'full-review-runner' + || mode === 'full-review-runner' && nextMode === 'review', 'unadmitted qualification child route'); + const corpusCampaign = mode === 'full-review' ? request.args[0] : request.corpusCampaign; + return spawnAdmitted({ root, proof, mode: nextMode, args, corpusCampaign }, process.env, options); + }); + if (mode === 'review') { + assert(typeof request.corpusCampaign === 'string', 'paired review requires its passing corpus'); + require(path.join(root, prefix, entries['full-review'])).requirePassingCorpus(request.corpusCampaign); + } + const driver = require(path.join(root, prefix, entries[mode])); + assert.equal(typeof driver.run, 'function', 'admitted driver entry is unavailable'); + return driver.run(...request.args); +} + +async function launch(repository, revision, planPath, planSha256, bootstrapSha256, mode, ...args) { + assert(['corpus', 'full-review', 'normal-flow'].includes(mode), 'unsupported public qualification entrypoint'); + assert(mode === 'normal-flow' ? args.length === 3 : mode === 'full-review' ? args.length === 1 + : args.length === 1 || args.length === 2 && args[1] === '--preflight', 'invalid qualification arguments'); + const { root, trust, bytes, plan, admitted } = reviewedPlan(repository, revision, + planPath, planSha256, bootstrapSha256, mode); + assert(!fs.realpathSync(__filename).startsWith(root + path.sep), 'materialize the reviewed bootstrap outside the checkout'); + const scratch = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'bugbot-qualified-source-'))); + fs.chmodSync(scratch, 0o700); + const isolated = path.join(scratch, 'source'); + try { + // A fresh local clone does not copy source repository hooks/configuration. + execFileSync(trust.executable, ['-c', 'core.hooksPath=' + os.devNull, 'clone', '--quiet', '--shared', + '--no-checkout', '--template=', trust.directory, isolated], { env: trust.env }); + const isolatedGit = gitFor(isolated).git; + isolatedGit('checkout', '--quiet', '--detach', revision); + isolatedGit('remote', 'remove', 'origin'); + // Git attributes can transform checkout bytes even without external + // filters. Recheck the materialized closure before requiring its helper. + for (const [relative, trusted] of admitted) assert( + fs.readFileSync(physical(isolated, relative)).equals(trusted), 'materialized driver differs: ' + relative); + assert(fs.readFileSync(physical(isolated, planPath)).equals(bytes), 'materialized plan differs'); + // The helper is now the immutable blob already checked above, not a + // working-tree verifier asked to authenticate itself after execution. + const verifier = require(path.join(isolated, prefix, 'execution-provenance.cjs')); + verifier.attestSource(root, plan.implementation); + verifier.attestRuntime(root, plan.runtime); + assert(!fs.existsSync(path.join(isolated, 'node_modules')), 'tracked dependency installation refused'); + fs.cpSync(path.join(root, 'node_modules'), path.join(isolated, 'node_modules'), + { recursive: true, verbatimSymlinks: true, dereference: false, mode: fs.constants.COPYFILE_FICLONE }); + verifier.attestSource(isolated, plan.implementation); + verifier.attestRuntime(isolated, plan.runtime); + const proof = { bootstrapSha256, planCommit: revision, planPath, planSha256 }; + // The application evidence adapter also invokes the bare Git command. + // Bind that lookup to the admitted OS executable without changing how + // Node/Codex or any other normal detector executable is selected. + const gitBin = path.join(scratch, 'git-bin'); + fs.mkdirSync(gitBin, { mode: 0o700 }); + fs.symlinkSync(trust.executable, path.join(gitBin, 'git')); + const env = { ...trust.env, COPILOT_EV07_PLAN: path.join(isolated, planPath), + PATH: gitBin + path.delimiter + (trust.env.PATH ?? '/usr/bin:/bin'), + BUGBOT_VERIFICATION_MODE: 'disabled', + COPILOT_QUALIFICATION_MANIFEST_SHA256: planSha256, + COPILOT_QUALIFICATION_PLAN_COMMIT: revision }; + delete env.NODE_OPTIONS; + delete env.NODE_PATH; + // Old diagnostic plans may select an auxiliary executable from the + // environment. A future reviewed plan must bind it before any probe. + if (plan.workflowInputs) { + assert(path.isAbsolute(plan.runtime.pythonExecutable ?? ''), 'pinned auxiliary Python required'); + assert.equal(sha256(fs.readFileSync(plan.runtime.pythonExecutable)), plan.runtime.pythonSha256, + 'auxiliary Python differs from reviewed runtime'); + env.PILOT_LOCAL_PYTHON = plan.runtime.pythonExecutable; + } + const childArgs = mode === 'normal-flow' ? [...args, path.join(isolated, planPath)] : args; + const child = spawnAdmitted({ root: isolated, proof, mode, args: childArgs }, env); + const interrupt = () => child.kill('SIGINT'), terminate = () => child.kill('SIGTERM'); + process.once('SIGINT', interrupt); + process.once('SIGTERM', terminate); + try { + return await new Promise((resolve, reject) => { + child.once('error', reject); + child.once('close', (code, signal) => resolve(code ?? (signal === 'SIGINT' ? 130 : signal === 'SIGTERM' ? 143 : 1))); + }); + } finally { + process.removeListener('SIGINT', interrupt); + process.removeListener('SIGTERM', terminate); + } + } finally { fs.rmSync(scratch, { recursive: true, force: true }); } +} + +module.exports = { launch }; +if (require.main === module) { + const fail = error => { console.error(error.message); process.exitCode = 1; }; + if (process.argv[2] === '--admitted-worker') { + if (!process.send || !process.connected) fail(new Error('admitted worker requires the external bootstrap IPC channel')); + else process.once('message', request => { + process.disconnect(); + executeAdmitted(request).then(code => { + process.exitCode = Number.isInteger(code) ? code : code?.status === 'interrupted' ? 130 : 0; + }).catch(fail); + }); + } else launch(...process.argv.slice(2)).then(code => { process.exitCode = code; }).catch(fail); +} diff --git a/quality/bugbot/evidence/issue-970/legacy-verifier-01097099.cjs.txt b/quality/bugbot/evidence/issue-970/legacy-verifier-01097099.cjs.txt new file mode 100644 index 000000000..9f7864fb5 --- /dev/null +++ b/quality/bugbot/evidence/issue-970/legacy-verifier-01097099.cjs.txt @@ -0,0 +1,255 @@ +'use strict'; + +const assert = require('node:assert/strict'); +const { execFileSync } = require('node:child_process'); +const { createHash } = require('node:crypto'); +const fs = require('node:fs'); +const { devNull } = require('node:os'); +const path = require('node:path'); + +const hash = bytes => createHash('sha256').update(bytes).digest('hex'); +const sourcePaths = Object.freeze(['src', 'pnpm-lock.yaml', 'tsconfig.json', 'package.json']); + +function gitEnvironment() { + const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.toUpperCase().startsWith('GIT_'))); + return { ...env, GIT_CONFIG_GLOBAL: devNull, GIT_CONFIG_NOSYSTEM: '1', GIT_NO_REPLACE_OBJECTS: '1' }; +} + +function trustedGitExecutable() { + // Bootstrap from the operating system, never PATH or a campaign-provided binary. + // On macOS resolve the actual developer-tools Git rather than hashing its shim. + const file = process.platform === 'darwin' + ? execFileSync('/usr/bin/xcrun', ['--find', 'git'], { + env: { PATH: '/usr/bin:/bin' }, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], + }).trim() + : process.platform === 'linux' ? '/usr/bin/git' : undefined; + assert(file && path.isAbsolute(file), 'qualification Git bootstrap supports Linux and macOS only'); + const executable = fs.realpathSync(file); + assert(fs.lstatSync(executable).isFile(), 'trusted Git executable unavailable'); + return executable; +} + +function pinnedGit(root) { + root = fs.realpathSync(root); + const marker = path.join(root, '.git'); + const stat = fs.lstatSync(fs.existsSync(marker) ? marker : root); + assert(!stat.isSymbolicLink(), 'repository Git identity must not be a symlink'); + let directory = fs.existsSync(marker) ? marker : root; + if (stat.isFile()) { + const match = /^gitdir: (.+)\r?\n?$/u.exec(fs.readFileSync(marker, 'utf8')); + assert(match, 'invalid worktree Git identity'); + directory = path.resolve(root, match[1]); + } else assert(stat.isDirectory(), 'repository Git identity unavailable'); + const env = gitEnvironment(); + const executable = trustedGitExecutable(); + const command = (...args) => execFileSync(executable, ['--no-pager', '--no-replace-objects', '--literal-pathspecs', + '--git-dir=' + directory, '--work-tree=' + root, '-C', root, + '-c', 'core.fsmonitor=false', '-c', 'core.hooksPath=' + devNull, '-c', 'core.quotePath=false', '-c', 'log.showSignature=false', ...args], + { env, stdio: ['pipe', 'pipe', 'pipe'], maxBuffer: 256 * 1024 * 1024 }); + command.executable = executable; + return command; +} + +function attestSource(root, implementation) { + assert.match(implementation ?? '', /^[a-f0-9]{40}$/u, 'invalid implementation pin'); + root = fs.realpathSync(root); + const git = pinnedGit(root); + const text = (...args) => git(...args).toString('utf8').trim(); + assert.equal(text('rev-parse', '--verify', implementation + '^{commit}'), implementation, + 'preregistered implementation commit unavailable'); + const rows = git('ls-tree', '-r', '-z', '--full-tree', implementation, '--', ...sourcePaths) + .toString('utf8').split('\0').filter(Boolean).map(row => { + const tab = row.indexOf('\t'); + const [mode, type, blob] = row.slice(0, tab).split(' '); + const name = row.slice(tab + 1); + assert(['100644', '100755'].includes(mode) && type === 'blob', 'source must be a regular committed file'); + assert(!path.isAbsolute(name) && !name.split('/').some(part => ['..', '.git'].includes(part)), 'unsafe source path'); + return { path: name, blob }; + }); + const expected = new Map(rows.map(row => [row.path, row])); + assert(expected.has('pnpm-lock.yaml') && expected.has('tsconfig.json') && rows.some(row => row.path.startsWith('src/')), + 'preregistered source inventory unavailable'); + // Compare the physical inventory, including ignored files and symlinked parents. + // Git's assume-unchanged, sparse/skip-worktree and stat cache are not byte evidence. + const actual = []; + function visit(relative) { + const file = path.join(root, relative); + let stat; + try { stat = fs.lstatSync(file); } + catch (error) { + if (error.code !== 'ENOENT') throw error; + assert(!expected.has(relative), 'application source or lockfile does not match the preregistered implementation'); + return; + } + assert(!stat.isSymbolicLink(), 'application source symlink is not admitted'); + if (stat.isDirectory()) { + for (const name of fs.readdirSync(file)) visit(relative + '/' + name); + } else { + assert(stat.isFile() && expected.has(relative), 'untracked application source or lockfile'); + actual.push(relative); + } + } + for (const name of sourcePaths) visit(name); + assert.equal(actual.length, expected.size, 'application source or lockfile does not match the preregistered implementation'); + assert.equal(text('diff', '--cached', '--no-ext-diff', '--no-textconv', '--name-only', implementation, '--', ...sourcePaths), '', + 'staged application source or lockfile does not match the preregistered implementation'); + // One batch reads immutable blobs without filters, attributes, textconv or a checkout. + const contents = execFileSync(git.executable, ['--no-replace-objects', '--git-dir=' + text('rev-parse', '--absolute-git-dir'), + '-c', 'core.fsmonitor=false', 'cat-file', '--batch'], { + env: gitEnvironment(), input: rows.map(row => row.blob).join('\n') + '\n', maxBuffer: 256 * 1024 * 1024, + }); + let offset = 0; + for (const row of rows) { + const end = contents.indexOf(10, offset); + const [blob, type, length] = contents.subarray(offset, end).toString('utf8').split(' '); + const size = Number(length); + assert(blob === row.blob && type === 'blob' && Number.isSafeInteger(size) && size >= 0, 'invalid pinned source blob'); + const bytes = contents.subarray(end + 1, end + 1 + size); + const local = fs.readFileSync(path.join(root, row.path)); + assert(bytes.equals(local), 'application source or lockfile does not match the preregistered implementation: ' + row.path); + row.sha256 = hash(local); + offset = end + size + 2; + } + assert.equal(offset, contents.length, 'invalid pinned source stream'); + return { implementation, tree: text('rev-parse', implementation + '^{tree}'), + sourceSha256: hash(JSON.stringify(rows)), dependencyLockSha256: expected.get('pnpm-lock.yaml').sha256, + compilerConfigurationSha256: expected.get('tsconfig.json').sha256 }; +} + +function dependencyIdentity(root) { + root = fs.realpathSync(root); + const directory = path.join(root, 'node_modules'); + assert(fs.lstatSync(directory).isDirectory() && !fs.lstatSync(directory).isSymbolicLink(), + 'qualification dependencies must have a physical installation'); + const rows = []; + function visit(relative) { + const file = path.join(directory, relative), stat = fs.lstatSync(file); + if (stat.isSymbolicLink()) { + const link = fs.readlinkSync(file), lexical = path.relative(directory, path.resolve(path.dirname(file), link)); + if (relative === path.join('.pnpm', 'node_modules', '@vypdev', 'copilot-browser-extension') + && fs.realpathSync(file) === path.join(root, 'extension')) { + // Normal pnpm workspaces expose this unused portal. Its code is + // outside admission and the entrypoint loader fence forbids it. + rows.push([relative, 'unadmitted-workspace', link, 'extension']); + return; + } + assert(lexical && !path.isAbsolute(lexical) && !lexical.split(path.sep).includes('..'), + 'dependency symlink escapes the admitted installation'); + let target; + try { target = path.relative(directory, fs.realpathSync(file)); } + catch (error) { if (error.code !== 'ENOENT') throw error; } + assert(target === undefined || target && !path.isAbsolute(target) && !target.split(path.sep).includes('..'), + 'dependency symlink escapes the admitted installation'); + // pnpm can retain links to platform-excluded optional packages. + // Bind the missing target as well; materializing it changes this proof. + rows.push([relative, 'link', link, target ?? 'missing:' + lexical]); + } else if (stat.isDirectory()) { + for (const name of fs.readdirSync(file).sort()) visit(path.join(relative, name)); + } else { + assert(stat.isFile(), 'dependency must be a regular file or internal symlink'); + rows.push([relative, 'file', stat.mode & 0o777, hash(fs.readFileSync(file))]); + } + } + visit(''); + // These ancestors could otherwise shadow the admitted root dependencies. + for (const relative of ['quality', 'quality/bugbot', 'quality/bugbot/evidence', + 'quality/bugbot/evidence/issue-950', 'quality/bugbot/evidence/issue-950/ev07-shadow', + 'quality/bugbot/evidence/issue-950/normal-flow-shadow']) { + assert(!fs.existsSync(path.join(root, relative, 'node_modules')), 'unadmitted qualification dependency shadow'); + } + assert(!process.env.NODE_PATH && !process.env.NODE_OPTIONS, 'qualification requires a clean Node loader environment'); + assert(!process.execArgv.some(arg => /^(?:--require|--import|--loader|--experimental-loader)(?:=|$)|^-r/u.test(arg)), + 'qualification Node preload is not admitted'); + return { dependencyTreeSha256: hash(JSON.stringify(rows)), dependencyEntries: rows.length }; +} + +function admitQualificationModules(root) { + root = fs.realpathSync(root); + const { registerHooks } = require('node:module'), { fileURLToPath } = require('node:url'); + assert.equal(typeof registerHooks, 'function', 'qualification requires synchronous Node module hooks'); + const admitted = ['src', 'node_modules', 'quality/bugbot/evidence/issue-950'] + .map(relative => path.join(root, relative) + path.sep); + return registerHooks({ resolve(specifier, context, nextResolve) { + const result = nextResolve(specifier, context); + if (!result.url.startsWith('node:')) { + assert(result.url.startsWith('file:'), 'qualification module must have a physical admitted path'); + const file = fs.realpathSync(fileURLToPath(result.url)); + assert(admitted.some(prefix => file.startsWith(prefix)), 'module is outside the admitted qualification closure'); + } + return result; + } }); +} + +function runtimeIdentity(root) { + const dependencies = dependencyIdentity(root); + const typescript = require.resolve('typescript', { paths: [root] }); + assert(fs.realpathSync(typescript).startsWith(fs.realpathSync(path.join(root, 'node_modules')) + path.sep), + 'compiler is outside the admitted dependency tree'); + const gitExecutable = trustedGitExecutable(); + return { ...dependencies, node: process.version, nodeSha256: hash(fs.readFileSync(process.execPath)), + gitExecutable, gitSha256: hash(fs.readFileSync(gitExecutable)), + git: execFileSync(gitExecutable, ['--version'], { env: gitEnvironment(), encoding: 'utf8' }).trim(), + typescript: JSON.parse(fs.readFileSync(require.resolve('typescript/package.json', { paths: [root] }))).version, + typescriptSha256: hash(fs.readFileSync(typescript)) }; +} + +function attestRuntime(root, selected) { + const actual = runtimeIdentity(root); + for (const [name, value] of Object.entries(actual)) { + assert.equal(value, selected?.[name], 'preregistered runtime differs: ' + name); + } + return actual; +} + +function attestDrivers(directory, expected, files) { + return Object.fromEntries(files.map(file => { + const target = path.join(directory, file); + assert(fs.lstatSync(target).isFile(), 'qualification driver must be a regular file: ' + file); + const actual = hash(fs.readFileSync(target)); + assert.equal(actual, expected?.[file], 'qualification driver differs from the preregistration: ' + file); + return [file, actual]; + })); +} + +function attestCommittedArtifact(root, file, expectedSha256, commit) { + assert.match(expectedSha256 ?? '', /^[a-f0-9]{64}$/u, 'reviewed committed artifact digest required'); + assert.match(commit ?? '', /^[a-f0-9]{40}$/u, 'reviewed artifact commit required'); + const requestedRoot = path.resolve(root); + root = fs.realpathSync(root); + const requestedFile = path.resolve(file); + const canonicalFile = requestedFile.startsWith(requestedRoot + path.sep) + ? root + requestedFile.slice(requestedRoot.length) : requestedFile; + const relative = path.relative(root, canonicalFile).split(path.sep).join('/'); + assert(relative && !path.isAbsolute(relative) && !relative.split('/').some(part => ['', '..', '.git'].includes(part)), + 'committed artifact must be inside the reviewed repository'); + let current = root; + for (const part of relative.split('/')) { + current = path.join(current, part); + assert(!fs.lstatSync(current).isSymbolicLink(), 'committed artifact symlink refused'); + } + assert(fs.lstatSync(current).isFile(), 'committed artifact must be regular'); + const git = pinnedGit(root), rows = git('ls-tree', '-z', commit, '--', relative).toString('utf8').split('\0').filter(Boolean); + assert.equal(rows.length, 1, 'reviewed committed artifact unavailable'); + const [header, name] = rows[0].split('\t'), [mode, type, blob] = header.split(' '); + assert(['100644', '100755'].includes(mode) && type === 'blob' && name === relative, + 'reviewed committed artifact must be a regular blob'); + const bytes = fs.readFileSync(current); + assert.equal(hash(bytes), expectedSha256, 'reviewed committed artifact digest differs'); + assert(git('cat-file', 'blob', blob).equals(bytes), 'reviewed committed artifact bytes differ'); + return { bytes, provenance: { commit, path: relative, blob, sha256: expectedSha256 } }; +} + +function requireBootstrapAdmission(root) { + const file = path.join(root, '.qualification-admission.json'); + assert(fs.existsSync(file), 'qualification requires the independently reviewed immutable bootstrap'); + assert(fs.lstatSync(file).isFile() && !fs.lstatSync(file).isSymbolicLink(), 'invalid bootstrap admission'); + const proof = JSON.parse(fs.readFileSync(file)); + assert.match(proof.bootstrapSha256 ?? '', /^[a-f0-9]{64}$/u); + assert.equal(hash(fs.readFileSync(path.join(root, 'quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs'))), + proof.bootstrapSha256, 'bootstrap admission identity differs'); + attestCommittedArtifact(root, path.join(root, proof.planPath), proof.planSha256, proof.planCommit); + return proof; +} + +module.exports = { attestSource, attestRuntime, attestDrivers, runtimeIdentity, dependencyIdentity, admitQualificationModules, + pinnedGit, gitEnvironment, trustedGitExecutable, attestCommittedArtifact, requireBootstrapAdmission }; diff --git a/quality/bugbot/evidence/issue-970/loader-fence-controls.log.gz b/quality/bugbot/evidence/issue-970/loader-fence-controls.log.gz new file mode 100644 index 000000000..a6ddee3f3 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/loader-fence-controls.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/memory-bootstrap-18-pass.log.gz b/quality/bugbot/evidence/issue-970/memory-bootstrap-18-pass.log.gz new file mode 100644 index 000000000..8b82cde48 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/memory-bootstrap-18-pass.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/memory-cancel-fixture-negative.log.gz b/quality/bugbot/evidence/issue-970/memory-cancel-fixture-negative.log.gz new file mode 100644 index 000000000..8f8d9c141 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/memory-cancel-fixture-negative.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/memory-cancel-pass.log.gz b/quality/bugbot/evidence/issue-970/memory-cancel-pass.log.gz new file mode 100644 index 000000000..06da63cc8 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/memory-cancel-pass.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/memory-frozen-final-shadow-193.log.gz b/quality/bugbot/evidence/issue-970/memory-frozen-final-shadow-193.log.gz new file mode 100644 index 000000000..8364b487d Binary files /dev/null and b/quality/bugbot/evidence/issue-970/memory-frozen-final-shadow-193.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/memory-native-binary-source-failure.log.gz b/quality/bugbot/evidence/issue-970/memory-native-binary-source-failure.log.gz new file mode 100644 index 000000000..f52f508c1 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/memory-native-binary-source-failure.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/memory-shadow-192-pass.log.gz b/quality/bugbot/evidence/issue-970/memory-shadow-192-pass.log.gz new file mode 100644 index 000000000..8f7fdb44a Binary files /dev/null and b/quality/bugbot/evidence/issue-970/memory-shadow-192-pass.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/memory-shadow-193-pass.log.gz b/quality/bugbot/evidence/issue-970/memory-shadow-193-pass.log.gz new file mode 100644 index 000000000..1beb117e3 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/memory-shadow-193-pass.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/planner-final-hook-focused.log.gz b/quality/bugbot/evidence/issue-970/planner-final-hook-focused.log.gz new file mode 100644 index 000000000..5a12703df Binary files /dev/null and b/quality/bugbot/evidence/issue-970/planner-final-hook-focused.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/planner-final-hook-sigsegv.log.gz b/quality/bugbot/evidence/issue-970/planner-final-hook-sigsegv.log.gz new file mode 100644 index 000000000..feb7bc7b7 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/planner-final-hook-sigsegv.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/planner-final-shadow-187.log.gz b/quality/bugbot/evidence/issue-970/planner-final-shadow-187.log.gz new file mode 100644 index 000000000..47b8fe0de Binary files /dev/null and b/quality/bugbot/evidence/issue-970/planner-final-shadow-187.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/planner-probe-controls-1.log.gz b/quality/bugbot/evidence/issue-970/planner-probe-controls-1.log.gz new file mode 100644 index 000000000..c19867317 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/planner-probe-controls-1.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/planner-probe-controls-pass.log.gz b/quality/bugbot/evidence/issue-970/planner-probe-controls-pass.log.gz new file mode 100644 index 000000000..c73f7b7e3 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/planner-probe-controls-pass.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/public-launch-bootstrap-final-20.log.gz b/quality/bugbot/evidence/issue-970/public-launch-bootstrap-final-20.log.gz new file mode 100644 index 000000000..12f4a2e0e Binary files /dev/null and b/quality/bugbot/evidence/issue-970/public-launch-bootstrap-final-20.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/public-launch-bootstrap-first-20.log.gz b/quality/bugbot/evidence/issue-970/public-launch-bootstrap-first-20.log.gz new file mode 100644 index 000000000..52a0af822 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/public-launch-bootstrap-first-20.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/public-launch-shadow-194.log.gz b/quality/bugbot/evidence/issue-970/public-launch-shadow-194.log.gz new file mode 100644 index 000000000..d7b821cb8 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/public-launch-shadow-194.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/public-loader-controls.log.gz b/quality/bugbot/evidence/issue-970/public-loader-controls.log.gz new file mode 100644 index 000000000..eb6e3ddc0 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/public-loader-controls.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/readable-legacy-hook-isolated-six.log.gz b/quality/bugbot/evidence/issue-970/readable-legacy-hook-isolated-six.log.gz new file mode 100644 index 000000000..bc130a262 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/readable-legacy-hook-isolated-six.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/readable-legacy-hook-sigsegv.log.gz b/quality/bugbot/evidence/issue-970/readable-legacy-hook-sigsegv.log.gz new file mode 100644 index 000000000..319e30359 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/readable-legacy-hook-sigsegv.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/readable-legacy-shadow-193.log.gz b/quality/bugbot/evidence/issue-970/readable-legacy-shadow-193.log.gz new file mode 100644 index 000000000..3b954d752 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/readable-legacy-shadow-193.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/runtime-dependencies-controls-2.log.gz b/quality/bugbot/evidence/issue-970/runtime-dependencies-controls-2.log.gz new file mode 100644 index 000000000..488beb9d0 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/runtime-dependencies-controls-2.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/runtime-dependencies-controls.log.gz b/quality/bugbot/evidence/issue-970/runtime-dependencies-controls.log.gz new file mode 100644 index 000000000..6df6c14da Binary files /dev/null and b/quality/bugbot/evidence/issue-970/runtime-dependencies-controls.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/verified-dependency-install.log.gz b/quality/bugbot/evidence/issue-970/verified-dependency-install.log.gz new file mode 100644 index 000000000..88a589707 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/verified-dependency-install.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/version-health-causal-negative.log.gz b/quality/bugbot/evidence/issue-970/version-health-causal-negative.log.gz new file mode 100644 index 000000000..1805c9273 Binary files /dev/null and b/quality/bugbot/evidence/issue-970/version-health-causal-negative.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/version-health-focus-pass.log.gz b/quality/bugbot/evidence/issue-970/version-health-focus-pass.log.gz new file mode 100644 index 000000000..1edf0b63d Binary files /dev/null and b/quality/bugbot/evidence/issue-970/version-health-focus-pass.log.gz differ diff --git a/quality/bugbot/evidence/issue-970/version-health-shadow-196.log.gz b/quality/bugbot/evidence/issue-970/version-health-shadow-196.log.gz new file mode 100644 index 000000000..f68f4a47e Binary files /dev/null and b/quality/bugbot/evidence/issue-970/version-health-shadow-196.log.gz differ diff --git a/specs/CATALOG.md b/specs/CATALOG.md index ab1f8e793..1882b6e19 100644 --- a/specs/CATALOG.md +++ b/specs/CATALOG.md @@ -13,7 +13,7 @@ debt or convert unknown historic intent into a design decision. | `github-communication-experience` | Implemented | English-default, localized, semantic, bounded and idempotent product messages across GitHub and repository-aware operator surfaces; complete acceptance batches and cards with stable source identities, verified published SDD provenance, shared complete-inventory text allocation, precise response diagnostics and same-head requirement evidence before Commit progress publication; shared bounded complete-batch recovery for attributable completed acceptance-field length or completion failures; bounded closed-fact recovery and compact large-inventory assessment, exact original hard allocations, explicit admission counts and non-retryable length exhaustion with separate live-provider qualification; exact JSON-byte allocation and one shared source-part size replacement for both framed profiles, captured request contracts, receipt quota revalidation and nonretryable unit-specific exhaustion; shared same-profile complete-batch correction for otherwise-valid repeated citations with current-request origin binding, unchanged evidence gates and nonretryable citation exhaustion | [Semantic GitHub communication and repository localization](./semantic-github-publication-and-notification.md) + 11 companion | 462 paths · 2026-11-18 | | `release-orchestration` | Implemented | Release and hotfix promotion, publication, bounded merge readiness recovery, and receipt-verified reconciliation without republication; explicit native-guarded replacement of closed unpublished candidates with complete fresh validation and bounded audit history | [Configurable production-first release orchestration](./configurable-release-orchestration.md) + 3 companion | 91 paths · 2026-10-30 | | `merge-queue-readiness` | Implemented | Fail-closed validation of required checks and merge-group workflow support; optional manual-only RepoWise analysis with independent CI and verified required-check migration; cached RepoWise exports and bounded full-history secret processing with truthful completeness and resource evidence | [Merge queue readiness and effective target rules](./merge-queue-readiness.md) | 29 paths · 2026-10-21 | -| `bugbot-review-state-reconciliation` | Implemented | Reconcile review snapshots, findings, threads, comments, and check conclusions | [Bugbot review-state reconciliation](./bugbot-review-state-reconciliation.md) | 106 paths · 2026-11-19 | +| `bugbot-review-state-reconciliation` | Implemented | Reconcile review snapshots, findings, threads, comments, and check conclusions; bind qualification execution to independently reviewed external bootstrap code with complete public admission in every process and bounded executable version health probes | [Bugbot review-state reconciliation](./bugbot-review-state-reconciliation.md) | 118 paths · 2026-11-20 | | `execution-lifecycle` | Implemented | Shared GitHub Action lifecycle with pinned native resource/lane coordination proof, conservative predecessor fencing, routing and durable user-facing results | [Execution admission, queueing, routing, and result publication](./execution-admission-queue-and-publication.md) + 4 companion | 161 paths · 2026-10-29 | | `architecture-quality-hardening` | Implemented | Close verified concurrency, error-contract, context-coupling, fan-out, setup/doctor, and provider-policy risks in dependency order | [Architecture quality and scalability hardening](./architecture-quality-and-scalability-hardening.md) + 1 companion | 72 paths · 2026-10-13 | | `setup-and-doctor` | Implemented | Plan, validate, provision, and audit a repository installation without exposing credentials | [Setup, configuration, credentials, and doctor](./setup-configuration-credentials-and-doctor.md) + 6 companion | 334 paths · 2026-10-29 | @@ -71,13 +71,13 @@ debt or convert unknown historic intent into a design decision. ### `bugbot-review-state-reconciliation` — Bugbot review-state reconciliation - Owner: Copilot maintainers -- Last verified: 2026-11-19 +- Last verified: 2026-11-20 - Specifications: [`specs/bugbot-review-state-reconciliation.md`](./bugbot-review-state-reconciliation.md) - Workflows: [`.github/workflows/copilot_commit.yml`](../.github/workflows/copilot_commit.yml) · [`.github/workflows/copilot_pull_request.yml`](../.github/workflows/copilot_pull_request.yml) · [`.github/workflows/copilot_pull_request_comment.yml`](../.github/workflows/copilot_pull_request_comment.yml) · [`.github/workflows/copilot_pull_request_review_state.yml`](../.github/workflows/copilot_pull_request_review_state.yml) · [`setup/workflows/copilot_pull_request.yml`](../setup/workflows/copilot_pull_request.yml) · [`setup/workflows/copilot_pull_request_review_state.yml`](../setup/workflows/copilot_pull_request_review_state.yml) · [`.github/workflows/copilot_bugbot_status_card.yml`](../.github/workflows/copilot_bugbot_status_card.yml) - Entrypoints: [`src/application/usecases/steps/commit/detect_potential_problems_workflow.ts`](../src/application/usecases/steps/commit/detect_potential_problems_workflow.ts) · [`src/application/usecases/steps/commit/bugbot/reconcile_bugbot_review_state_use_case.ts`](../src/application/usecases/steps/commit/bugbot/reconcile_bugbot_review_state_use_case.ts) · [`src/actions/github_action_completion.ts`](../src/actions/github_action_completion.ts) · [`src/api.ts`](../src/api.ts) -- Core code: [`scripts/coverage-budgets.json`](../scripts/coverage-budgets.json) · [`src/domain/bugbot/review_state.ts`](../src/domain/bugbot/review_state.ts) · [`src/domain/bugbot/review_projection.ts`](../src/domain/bugbot/review_projection.ts) · [`src/application/policies/bugbot_reconciliation_policy.ts`](../src/application/policies/bugbot_reconciliation_policy.ts) · [`src/application/policies/bugbot_event_ownership_policy.ts`](../src/application/policies/bugbot_event_ownership_policy.ts) · [`src/application/policies/bugbot_result_finding_state_projection_policy.ts`](../src/application/policies/bugbot_result_finding_state_projection_policy.ts) · [`src/application/policies/bugbot_telemetry_projection_policy.ts`](../src/application/policies/bugbot_telemetry_projection_policy.ts) · [`src/application/policies/bugbot_message_catalog.ts`](../src/application/policies/bugbot_message_catalog.ts) · [`src/application/policies/bugbot_finding_marker_policy.ts`](../src/application/policies/bugbot_finding_marker_policy.ts) · [`src/application/policies/bugbot_review_presentation_policy.ts`](../src/application/policies/bugbot_review_presentation_policy.ts) · [`src/application/policies/action_summary_policy.ts`](../src/application/policies/action_summary_policy.ts) · [`src/application/policies/copilot_evidence_policy.ts`](../src/application/policies/copilot_evidence_policy.ts) · [`src/application/policies/lifecycle_state_policy.ts`](../src/application/policies/lifecycle_state_policy.ts) · [`src/application/policies/status_command_policy.ts`](../src/application/policies/status_command_policy.ts) · [`src/application/usecases/steps/common/publish_resume_workflow.ts`](../src/application/usecases/steps/common/publish_resume_workflow.ts) · [`src/application/usecases/steps/commit/bugbot/load_bugbot_context_use_case.ts`](../src/application/usecases/steps/commit/bugbot/load_bugbot_context_use_case.ts) · [`src/application/usecases/steps/commit/bugbot/schema.ts`](../src/application/usecases/steps/commit/bugbot/schema.ts) · [`src/application/usecases/steps/commit/bugbot/prepare_bugbot_findings_policy.ts`](../src/application/usecases/steps/commit/bugbot/prepare_bugbot_findings_policy.ts) · [`src/application/usecases/steps/commit/bugbot/publish_pr_review_comments.ts`](../src/application/usecases/steps/commit/bugbot/publish_pr_review_comments.ts) · [`src/application/usecases/steps/commit/bugbot/synchronize_bugbot_review_presentation_use_case.ts`](../src/application/usecases/steps/commit/bugbot/synchronize_bugbot_review_presentation_use_case.ts) · [`src/application/policies/bugbot_status_count_marker_policy.ts`](../src/application/policies/bugbot_status_count_marker_policy.ts) · [`src/application/errors/bugbot_mutation_error.ts`](../src/application/errors/bugbot_mutation_error.ts) · [`src/data/repository/pull_request/pull_request_review_error_metadata.ts`](../src/data/repository/pull_request/pull_request_review_error_metadata.ts) · [`src/domain/bugbot/finding.ts`](../src/domain/bugbot/finding.ts) · [`src/application/usecases/steps/commit/bugbot/analyze_bugbot_revision_use_case.ts`](../src/application/usecases/steps/commit/bugbot/analyze_bugbot_revision_use_case.ts) · [`src/application/usecases/steps/commit/bugbot/publish_findings_use_case.ts`](../src/application/usecases/steps/commit/bugbot/publish_findings_use_case.ts) · [`scripts/validate-workflow-contract.cjs`](../scripts/validate-workflow-contract.cjs) · [`src/application/policies/bugbot_resolution_eligibility_policy.ts`](../src/application/policies/bugbot_resolution_eligibility_policy.ts) · [`src/application/usecases/steps/commit/bugbot/resolve_pull_request_finding.ts`](../src/application/usecases/steps/commit/bugbot/resolve_pull_request_finding.ts) · [`src/application/usecases/steps/commit/bugbot/mark_findings_resolved_workflow.ts`](../src/application/usecases/steps/commit/bugbot/mark_findings_resolved_workflow.ts) · [`src/application/usecases/steps/commit/bugbot/verify_reopened_finding_thread.ts`](../src/application/usecases/steps/commit/bugbot/verify_reopened_finding_thread.ts) · [`src/application/ports/bugbot_pull_request_resolution_ports.ts`](../src/application/ports/bugbot_pull_request_resolution_ports.ts) · [`src/application/ports/bugbot_pull_request_write_ports.ts`](../src/application/ports/bugbot_pull_request_write_ports.ts) · [`src/application/ports/pull_request_review_comment_ports.ts`](../src/application/ports/pull_request_review_comment_ports.ts) · [`src/infrastructure/composition/bugbot_scm_port_factory.ts`](../src/infrastructure/composition/bugbot_scm_port_factory.ts) · [`src/application/policies/bugbot_commit_progress_link_policy.ts`](../src/application/policies/bugbot_commit_progress_link_policy.ts) · [`src/infrastructure/github/github_bugbot_interrupted_check.ts`](../src/infrastructure/github/github_bugbot_interrupted_check.ts) · [`src/application/policies/bugbot_progress_check_observation_policy.ts`](../src/application/policies/bugbot_progress_check_observation_policy.ts) · [`src/application/policies/bugbot_status_card_publication_policy.ts`](../src/application/policies/bugbot_status_card_publication_policy.ts) · [`src/infrastructure/github/github_bugbot_status_card_workflow.ts`](../src/infrastructure/github/github_bugbot_status_card_workflow.ts) · [`src/infrastructure/github/github_bugbot_status_card_dispatch.ts`](../src/infrastructure/github/github_bugbot_status_card_dispatch.ts) · [`src/infrastructure/github/native_list_pagination.ts`](../src/infrastructure/github/native_list_pagination.ts) -- Tests: [`src/domain/bugbot/__tests__/review_state.test.ts`](../src/domain/bugbot/__tests__/review_state.test.ts) · [`src/domain/bugbot/__tests__/review_projection.test.ts`](../src/domain/bugbot/__tests__/review_projection.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/schema.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/schema.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/prepare_bugbot_findings.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/prepare_bugbot_findings.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/reconcile_bugbot_review_state_integration.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/reconcile_bugbot_review_state_integration.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/synchronize_bugbot_review_presentation_use_case.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/synchronize_bugbot_review_presentation_use_case.test.ts) · [`src/application/policies/__tests__/bugbot_result_finding_state_projection_policy.test.ts`](../src/application/policies/__tests__/bugbot_result_finding_state_projection_policy.test.ts) · [`src/application/policies/__tests__/bugbot_event_ownership_policy.test.ts`](../src/application/policies/__tests__/bugbot_event_ownership_policy.test.ts) · [`src/application/policies/__tests__/bugbot_telemetry_projection_policy.test.ts`](../src/application/policies/__tests__/bugbot_telemetry_projection_policy.test.ts) · [`src/application/policies/__tests__/bugbot_message_catalog.test.ts`](../src/application/policies/__tests__/bugbot_message_catalog.test.ts) · [`src/application/policies/__tests__/bugbot_finding_marker_policy.test.ts`](../src/application/policies/__tests__/bugbot_finding_marker_policy.test.ts) · [`src/application/policies/__tests__/bugbot_review_presentation_policy.test.ts`](../src/application/policies/__tests__/bugbot_review_presentation_policy.test.ts) · [`src/application/policies/__tests__/action_summary_policy.test.ts`](../src/application/policies/__tests__/action_summary_policy.test.ts) · [`src/application/policies/__tests__/copilot_evidence_policy.test.ts`](../src/application/policies/__tests__/copilot_evidence_policy.test.ts) · [`src/application/policies/__tests__/lifecycle_state_policy.test.ts`](../src/application/policies/__tests__/lifecycle_state_policy.test.ts) · [`src/application/policies/__tests__/status_command_policy.test.ts`](../src/application/policies/__tests__/status_command_policy.test.ts) · [`src/actions/__tests__/github_action_completion.test.ts`](../src/actions/__tests__/github_action_completion.test.ts) · [`src/application/usecases/steps/common/__tests__/publish_resume_use_case.test.ts`](../src/application/usecases/steps/common/__tests__/publish_resume_use_case.test.ts) · [`src/application/usecases/actions/__tests__/synchronize_lifecycle_state_use_case.test.ts`](../src/application/usecases/actions/__tests__/synchronize_lifecycle_state_use_case.test.ts) · [`src/application/usecases/steps/commit/__tests__/detect_potential_problems_use_case.test.ts`](../src/application/usecases/steps/commit/__tests__/detect_potential_problems_use_case.test.ts) · [`src/infrastructure/composition/__tests__/pull_request_use_case_composition_root.test.ts`](../src/infrastructure/composition/__tests__/pull_request_use_case_composition_root.test.ts) · [`src/tooling/__tests__/validate_workflow_contract.test.ts`](../src/tooling/__tests__/validate_workflow_contract.test.ts) · [`src/application/errors/__tests__/bugbot_mutation_error.test.ts`](../src/application/errors/__tests__/bugbot_mutation_error.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/finding_mutation_recovery.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/finding_mutation_recovery.test.ts) · [`src/domain/bugbot/__tests__/finding.test.ts`](../src/domain/bugbot/__tests__/finding.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/finding_identity_replay.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/finding_identity_replay.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/analyze_bugbot_revision_use_case.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/analyze_bugbot_revision_use_case.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/publish_findings_use_case.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/publish_findings_use_case.test.ts) · [`src/application/policies/__tests__/bugbot_resolution_eligibility_policy.test.ts`](../src/application/policies/__tests__/bugbot_resolution_eligibility_policy.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/reopened_dismissal_reconciliation.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/reopened_dismissal_reconciliation.test.ts) · [`src/infrastructure/composition/__tests__/bugbot_scm_port_factory.test.ts`](../src/infrastructure/composition/__tests__/bugbot_scm_port_factory.test.ts) · [`src/application/ports/__tests__/bugbot_port_boundaries.test.ts`](../src/application/ports/__tests__/bugbot_port_boundaries.test.ts) · [`src/application/policies/__tests__/bugbot_commit_progress_link_policy.test.ts`](../src/application/policies/__tests__/bugbot_commit_progress_link_policy.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_interrupted_check.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_interrupted_check.test.ts) · [`src/application/policies/__tests__/bugbot_progress_check_observation_policy.test.ts`](../src/application/policies/__tests__/bugbot_progress_check_observation_policy.test.ts) · [`src/data/repository/__tests__/copilot_evidence_repository.test.ts`](../src/data/repository/__tests__/copilot_evidence_repository.test.ts) · [`quality/bugbot/evidence/issue-950/status-card-publication/publisher.test.cjs`](../quality/bugbot/evidence/issue-950/status-card-publication/publisher.test.cjs) · [`quality/bugbot/evidence/issue-950/status-card-publication/packaged-canary.cjs`](../quality/bugbot/evidence/issue-950/status-card-publication/packaged-canary.cjs) · [`src/application/policies/__tests__/bugbot_status_card_publication_policy.test.ts`](../src/application/policies/__tests__/bugbot_status_card_publication_policy.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_workflow.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_workflow.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_lane.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_lane.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_dispatch.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_dispatch.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_observation.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_observation.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_handoff.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_handoff.test.ts) -- User documentation: [`docs/bugbot/detection.mdx`](../docs/bugbot/detection.mdx) · [`docs/bugbot/finding-publication.mdx`](../docs/bugbot/finding-publication.mdx) · [`docs/bugbot/quality-observability.mdx`](../docs/bugbot/quality-observability.mdx) · [`docs/bugbot/failure-scenarios.mdx`](../docs/bugbot/failure-scenarios.mdx) · [`docs/bugbot/configuration.mdx`](../docs/bugbot/configuration.mdx) · [`docs/bugbot/how-it-works.mdx`](../docs/bugbot/how-it-works.mdx) · [`docs/bugbot/programmatic-api.mdx`](../docs/bugbot/programmatic-api.mdx) · [`docs/pull-requests/workflow-setup.mdx`](../docs/pull-requests/workflow-setup.mdx) · [`docs/issues/comment-commands.mdx`](../docs/issues/comment-commands.mdx) +- Core code: [`scripts/coverage-budgets.json`](../scripts/coverage-budgets.json) · [`src/domain/bugbot/review_state.ts`](../src/domain/bugbot/review_state.ts) · [`src/domain/bugbot/review_projection.ts`](../src/domain/bugbot/review_projection.ts) · [`src/application/policies/bugbot_reconciliation_policy.ts`](../src/application/policies/bugbot_reconciliation_policy.ts) · [`src/application/policies/bugbot_event_ownership_policy.ts`](../src/application/policies/bugbot_event_ownership_policy.ts) · [`src/application/policies/bugbot_result_finding_state_projection_policy.ts`](../src/application/policies/bugbot_result_finding_state_projection_policy.ts) · [`src/application/policies/bugbot_telemetry_projection_policy.ts`](../src/application/policies/bugbot_telemetry_projection_policy.ts) · [`src/application/policies/bugbot_message_catalog.ts`](../src/application/policies/bugbot_message_catalog.ts) · [`src/application/policies/bugbot_finding_marker_policy.ts`](../src/application/policies/bugbot_finding_marker_policy.ts) · [`src/application/policies/bugbot_review_presentation_policy.ts`](../src/application/policies/bugbot_review_presentation_policy.ts) · [`src/application/policies/action_summary_policy.ts`](../src/application/policies/action_summary_policy.ts) · [`src/application/policies/copilot_evidence_policy.ts`](../src/application/policies/copilot_evidence_policy.ts) · [`src/application/policies/lifecycle_state_policy.ts`](../src/application/policies/lifecycle_state_policy.ts) · [`src/application/policies/status_command_policy.ts`](../src/application/policies/status_command_policy.ts) · [`src/application/usecases/steps/common/publish_resume_workflow.ts`](../src/application/usecases/steps/common/publish_resume_workflow.ts) · [`src/application/usecases/steps/commit/bugbot/load_bugbot_context_use_case.ts`](../src/application/usecases/steps/commit/bugbot/load_bugbot_context_use_case.ts) · [`src/application/usecases/steps/commit/bugbot/schema.ts`](../src/application/usecases/steps/commit/bugbot/schema.ts) · [`src/application/usecases/steps/commit/bugbot/prepare_bugbot_findings_policy.ts`](../src/application/usecases/steps/commit/bugbot/prepare_bugbot_findings_policy.ts) · [`src/application/usecases/steps/commit/bugbot/publish_pr_review_comments.ts`](../src/application/usecases/steps/commit/bugbot/publish_pr_review_comments.ts) · [`src/application/usecases/steps/commit/bugbot/synchronize_bugbot_review_presentation_use_case.ts`](../src/application/usecases/steps/commit/bugbot/synchronize_bugbot_review_presentation_use_case.ts) · [`src/application/policies/bugbot_status_count_marker_policy.ts`](../src/application/policies/bugbot_status_count_marker_policy.ts) · [`src/application/errors/bugbot_mutation_error.ts`](../src/application/errors/bugbot_mutation_error.ts) · [`src/data/repository/pull_request/pull_request_review_error_metadata.ts`](../src/data/repository/pull_request/pull_request_review_error_metadata.ts) · [`src/domain/bugbot/finding.ts`](../src/domain/bugbot/finding.ts) · [`src/application/usecases/steps/commit/bugbot/analyze_bugbot_revision_use_case.ts`](../src/application/usecases/steps/commit/bugbot/analyze_bugbot_revision_use_case.ts) · [`src/application/usecases/steps/commit/bugbot/publish_findings_use_case.ts`](../src/application/usecases/steps/commit/bugbot/publish_findings_use_case.ts) · [`scripts/validate-workflow-contract.cjs`](../scripts/validate-workflow-contract.cjs) · [`src/application/policies/bugbot_resolution_eligibility_policy.ts`](../src/application/policies/bugbot_resolution_eligibility_policy.ts) · [`src/application/usecases/steps/commit/bugbot/resolve_pull_request_finding.ts`](../src/application/usecases/steps/commit/bugbot/resolve_pull_request_finding.ts) · [`src/application/usecases/steps/commit/bugbot/mark_findings_resolved_workflow.ts`](../src/application/usecases/steps/commit/bugbot/mark_findings_resolved_workflow.ts) · [`src/application/usecases/steps/commit/bugbot/verify_reopened_finding_thread.ts`](../src/application/usecases/steps/commit/bugbot/verify_reopened_finding_thread.ts) · [`src/application/ports/bugbot_pull_request_resolution_ports.ts`](../src/application/ports/bugbot_pull_request_resolution_ports.ts) · [`src/application/ports/bugbot_pull_request_write_ports.ts`](../src/application/ports/bugbot_pull_request_write_ports.ts) · [`src/application/ports/pull_request_review_comment_ports.ts`](../src/application/ports/pull_request_review_comment_ports.ts) · [`src/infrastructure/composition/bugbot_scm_port_factory.ts`](../src/infrastructure/composition/bugbot_scm_port_factory.ts) · [`src/application/policies/bugbot_commit_progress_link_policy.ts`](../src/application/policies/bugbot_commit_progress_link_policy.ts) · [`src/infrastructure/github/github_bugbot_interrupted_check.ts`](../src/infrastructure/github/github_bugbot_interrupted_check.ts) · [`src/application/policies/bugbot_progress_check_observation_policy.ts`](../src/application/policies/bugbot_progress_check_observation_policy.ts) · [`src/application/policies/bugbot_status_card_publication_policy.ts`](../src/application/policies/bugbot_status_card_publication_policy.ts) · [`src/infrastructure/github/github_bugbot_status_card_workflow.ts`](../src/infrastructure/github/github_bugbot_status_card_workflow.ts) · [`src/infrastructure/github/github_bugbot_status_card_dispatch.ts`](../src/infrastructure/github/github_bugbot_status_card_dispatch.ts) · [`src/infrastructure/github/native_list_pagination.ts`](../src/infrastructure/github/native_list_pagination.ts) · [`quality/bugbot/evidence/issue-950/execution-provenance.cjs`](../quality/bugbot/evidence/issue-950/execution-provenance.cjs) · [`quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs`](../quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs) · [`quality/bugbot/evidence/issue-950/ev07-shadow/implementation.cjs`](../quality/bugbot/evidence/issue-950/ev07-shadow/implementation.cjs) · [`quality/bugbot/evidence/issue-950/normal-flow-shadow/run.cjs`](../quality/bugbot/evidence/issue-950/normal-flow-shadow/run.cjs) · [`quality/bugbot/evidence/issue-950/ev07-shadow/corpus.cjs`](../quality/bugbot/evidence/issue-950/ev07-shadow/corpus.cjs) · [`quality/bugbot/evidence/issue-950/ev07-shadow/review.cjs`](../quality/bugbot/evidence/issue-950/ev07-shadow/review.cjs) · [`quality/bugbot/evidence/issue-950/ev07-shadow/full-review.cjs`](../quality/bugbot/evidence/issue-950/ev07-shadow/full-review.cjs) · [`quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.cjs`](../quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.cjs) +- Tests: [`src/domain/bugbot/__tests__/review_state.test.ts`](../src/domain/bugbot/__tests__/review_state.test.ts) · [`src/domain/bugbot/__tests__/review_projection.test.ts`](../src/domain/bugbot/__tests__/review_projection.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/schema.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/schema.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/prepare_bugbot_findings.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/prepare_bugbot_findings.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/reconcile_bugbot_review_state_integration.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/reconcile_bugbot_review_state_integration.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/synchronize_bugbot_review_presentation_use_case.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/synchronize_bugbot_review_presentation_use_case.test.ts) · [`src/application/policies/__tests__/bugbot_result_finding_state_projection_policy.test.ts`](../src/application/policies/__tests__/bugbot_result_finding_state_projection_policy.test.ts) · [`src/application/policies/__tests__/bugbot_event_ownership_policy.test.ts`](../src/application/policies/__tests__/bugbot_event_ownership_policy.test.ts) · [`src/application/policies/__tests__/bugbot_telemetry_projection_policy.test.ts`](../src/application/policies/__tests__/bugbot_telemetry_projection_policy.test.ts) · [`src/application/policies/__tests__/bugbot_message_catalog.test.ts`](../src/application/policies/__tests__/bugbot_message_catalog.test.ts) · [`src/application/policies/__tests__/bugbot_finding_marker_policy.test.ts`](../src/application/policies/__tests__/bugbot_finding_marker_policy.test.ts) · [`src/application/policies/__tests__/bugbot_review_presentation_policy.test.ts`](../src/application/policies/__tests__/bugbot_review_presentation_policy.test.ts) · [`src/application/policies/__tests__/action_summary_policy.test.ts`](../src/application/policies/__tests__/action_summary_policy.test.ts) · [`src/application/policies/__tests__/copilot_evidence_policy.test.ts`](../src/application/policies/__tests__/copilot_evidence_policy.test.ts) · [`src/application/policies/__tests__/lifecycle_state_policy.test.ts`](../src/application/policies/__tests__/lifecycle_state_policy.test.ts) · [`src/application/policies/__tests__/status_command_policy.test.ts`](../src/application/policies/__tests__/status_command_policy.test.ts) · [`src/actions/__tests__/github_action_completion.test.ts`](../src/actions/__tests__/github_action_completion.test.ts) · [`src/application/usecases/steps/common/__tests__/publish_resume_use_case.test.ts`](../src/application/usecases/steps/common/__tests__/publish_resume_use_case.test.ts) · [`src/application/usecases/actions/__tests__/synchronize_lifecycle_state_use_case.test.ts`](../src/application/usecases/actions/__tests__/synchronize_lifecycle_state_use_case.test.ts) · [`src/application/usecases/steps/commit/__tests__/detect_potential_problems_use_case.test.ts`](../src/application/usecases/steps/commit/__tests__/detect_potential_problems_use_case.test.ts) · [`src/infrastructure/composition/__tests__/pull_request_use_case_composition_root.test.ts`](../src/infrastructure/composition/__tests__/pull_request_use_case_composition_root.test.ts) · [`src/tooling/__tests__/validate_workflow_contract.test.ts`](../src/tooling/__tests__/validate_workflow_contract.test.ts) · [`src/application/errors/__tests__/bugbot_mutation_error.test.ts`](../src/application/errors/__tests__/bugbot_mutation_error.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/finding_mutation_recovery.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/finding_mutation_recovery.test.ts) · [`src/domain/bugbot/__tests__/finding.test.ts`](../src/domain/bugbot/__tests__/finding.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/finding_identity_replay.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/finding_identity_replay.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/analyze_bugbot_revision_use_case.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/analyze_bugbot_revision_use_case.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/publish_findings_use_case.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/publish_findings_use_case.test.ts) · [`src/application/policies/__tests__/bugbot_resolution_eligibility_policy.test.ts`](../src/application/policies/__tests__/bugbot_resolution_eligibility_policy.test.ts) · [`src/application/usecases/steps/commit/bugbot/__tests__/reopened_dismissal_reconciliation.test.ts`](../src/application/usecases/steps/commit/bugbot/__tests__/reopened_dismissal_reconciliation.test.ts) · [`src/infrastructure/composition/__tests__/bugbot_scm_port_factory.test.ts`](../src/infrastructure/composition/__tests__/bugbot_scm_port_factory.test.ts) · [`src/application/ports/__tests__/bugbot_port_boundaries.test.ts`](../src/application/ports/__tests__/bugbot_port_boundaries.test.ts) · [`src/application/policies/__tests__/bugbot_commit_progress_link_policy.test.ts`](../src/application/policies/__tests__/bugbot_commit_progress_link_policy.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_interrupted_check.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_interrupted_check.test.ts) · [`src/application/policies/__tests__/bugbot_progress_check_observation_policy.test.ts`](../src/application/policies/__tests__/bugbot_progress_check_observation_policy.test.ts) · [`src/data/repository/__tests__/copilot_evidence_repository.test.ts`](../src/data/repository/__tests__/copilot_evidence_repository.test.ts) · [`quality/bugbot/evidence/issue-950/status-card-publication/publisher.test.cjs`](../quality/bugbot/evidence/issue-950/status-card-publication/publisher.test.cjs) · [`quality/bugbot/evidence/issue-950/status-card-publication/packaged-canary.cjs`](../quality/bugbot/evidence/issue-950/status-card-publication/packaged-canary.cjs) · [`src/application/policies/__tests__/bugbot_status_card_publication_policy.test.ts`](../src/application/policies/__tests__/bugbot_status_card_publication_policy.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_workflow.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_workflow.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_lane.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_lane.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_dispatch.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_dispatch.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_observation.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_observation.test.ts) · [`src/infrastructure/github/__tests__/github_bugbot_status_card_handoff.test.ts`](../src/infrastructure/github/__tests__/github_bugbot_status_card_handoff.test.ts) · [`quality/bugbot/evidence/issue-950/qualification-bootstrap.test.cjs`](../quality/bugbot/evidence/issue-950/qualification-bootstrap.test.cjs) · [`quality/bugbot/evidence/issue-950/normal-flow-shadow/paths.test.cjs`](../quality/bugbot/evidence/issue-950/normal-flow-shadow/paths.test.cjs) · [`quality/bugbot/evidence/issue-950/ev07-shadow/review-runtime.test.cjs`](../quality/bugbot/evidence/issue-950/ev07-shadow/review-runtime.test.cjs) +- User documentation: [`docs/bugbot/detection.mdx`](../docs/bugbot/detection.mdx) · [`docs/bugbot/finding-publication.mdx`](../docs/bugbot/finding-publication.mdx) · [`docs/bugbot/quality-observability.mdx`](../docs/bugbot/quality-observability.mdx) · [`docs/bugbot/failure-scenarios.mdx`](../docs/bugbot/failure-scenarios.mdx) · [`docs/bugbot/configuration.mdx`](../docs/bugbot/configuration.mdx) · [`docs/bugbot/how-it-works.mdx`](../docs/bugbot/how-it-works.mdx) · [`docs/bugbot/programmatic-api.mdx`](../docs/bugbot/programmatic-api.mdx) · [`docs/pull-requests/workflow-setup.mdx`](../docs/pull-requests/workflow-setup.mdx) · [`docs/issues/comment-commands.mdx`](../docs/issues/comment-commands.mdx) · [`docs/development/testing.mdx`](../docs/development/testing.mdx) ### `execution-lifecycle` — Execution admission, queueing, routing, and result publication diff --git a/specs/bugbot-review-state-reconciliation.md b/specs/bugbot-review-state-reconciliation.md index 338e51651..cc3996f51 100644 --- a/specs/bugbot-review-state-reconciliation.md +++ b/specs/bugbot-review-state-reconciliation.md @@ -2502,3 +2502,177 @@ path forms and one generated-card ownership case. Reuse actual workflow, adapter, Git and packaged entrypoints; no model calls or evaluated-PR writes. Update the operator documentation and catalog evidence, run specification, architecture and focused checks, and report final-head CI separately. + + +### Independent qualification bootstrap — #970, 2026-11-19 + +Before Node loads any qualification driver, the operator MUST extract the +bootstrap blob with trusted operating-system Git from an independently reviewed +revision, check its independently reviewed SHA-256, and start trusted Node with +no preload or loader/search overrides. The bootstrap MUST authenticate the exact +reviewed committed plan by explicit revision/path/SHA-256, compare every closed +inventory driver with its immutable committed blob and reject changed physical +bytes/symlink parents before loading the verifier. A working-tree verifier cannot +establish its own integrity after executing. Direct working-tree entrypoints do +not constitute qualification admission. + +The bootstrap materializes that immutable Git revision in a private source tree, +loads the already checked verifier there, verifies the original source and +installed dependency/runtime identity, copies dependencies, rechecks the private +source and installation, and runs only that admitted entrypoint. Node, OS Git, +reviewed bootstrap/plan digests and the verified package installation are the +operator trust anchors. A receipt or run/artifact identity is not independent +accreditation of a compromised bootstrap or host. + +Normal-flow MUST authenticate its manifest as that exact reviewed committed +regular blob before reading runtime selection for any executable probe or model +turn. The external digest cannot be supplied solely by that manifest. Record the +stable bootstrap SHA-256 and plan commit/path/digest with the source, driver and +runtime proof, and compare them between corpus and paired reviews. Exclude +private ephemeral directory names from equality. Preserve dry-run publication, +disabled production, explicit cancellation and immutable historical attempts. +No score/latency gate or research policy changes; no consumer setting is added. + +```mermaid +flowchart LR + A[Operator reviewed Git blob and SHA-256] --> B[External trusted Node bootstrap] + B --> C[Authenticate committed plan and all driver blobs] + C --> D[Materialize private immutable source] + D --> E[Verified helper checks source and installed runtime] + E --> F[Run admitted corpus or paired review] +``` + +Minimum **12 additional deterministic controls**: one isolated successful launch +and cleanup; two changed-driver causal counter-controls; one unauthenticated CLI +manifest counter-control; four committed-manifest integrity failures; one combined +Git-redirection/PATH-shim case; and one installed-dependency rejection; plus one materialized-checkout transformation +rejection before helper loading and one positive committed-manifest path control. +Recheck private plan/driver bytes after materialization, before loading any helper. +Retain the +existing actual relative-lab normal-flow control with a committed fixture manifest +and immutable launch. Use real Git and blocked model/write ports. All 174 corpus +cases and the 42 paired slots remain unstarted until merged prerequisites and a +new committed preregistration meet their existing independent acceptance gates. +Update operator testing documentation and catalogue evidence together, validate +specifications/docs and refresh the graph. Local controls do not qualify a profile. + + +Qualification Git identity also applies after admission to the actual evidence +adapter. The trusted bootstrap MUST route bare Git calls in its admitted child +to the same verified OS executable. Use a private directory containing only that +Git link before the original PATH, preserving ordinary Node/Codex selection; do +not introduce a consumer setting or modify production source for this lab routing. +Add one causal case using the actual shipped adapter: an inherited shim executes +under the former child environment, while the admitted child reads the real Git +identity without executing the shim. This adds one case to the twelve-control +bootstrap minimum (thirteen total), with no models or evaluated-PR writes. + + +Before invoking the ordinary planner, qualification MUST resolve its selected +executable and optional launcher using the same production resolution/validation +helpers and verify their registered digests. The planner performs its own version +probe before returning a prepared plan, so post-preparation checking alone is too +late. Preserve the unchanged ordinary planner and its final invocation/version/ +profile proof. Add two actual-planner controls: an admitted runtime prepares, and +a mismatched executable that the old version probe ran is rejected before probe +execution. Use the real resolution/validation/version ports and only a controlled +workspace; no model calls, research ceilings or production settings are added. + +### Bootstrap admission stays outside the checkout — #970, 2026-11-20 + +This amendment removes checkout-receipt authority from the preceding bootstrap +contract. A self-consistent `.qualification-admission.json` MUST NOT establish +that the independently reviewed bootstrap executed. No qualification driver may +read that file to obtain execution authority or attach bootstrap provenance. +Direct corpus, normal-flow, launcher, scheduler and review-slot CLIs MUST refuse +before importing any checkout helper, including when such a receipt is present. +The operator's reviewed Git blob and independently selected digests remain the +trust anchors; this change adds no independent host or human-review attestation. + +The external bootstrap authenticates and materializes the reviewed closure as +above, then installs immutable process-local context and calls the admitted +module in that same process. There MUST be no internal worker CLI or IPC message +that initializes context from caller-selected proof fields. Each nested scheduler +or review process MUST enter the complete public bootstrap validation and private +materialization with the same independently selected plan/bootstrap pins. Its +private source/dependencies and fixed driver inventory are checked before import. +The closure permits launcher-to-scheduler and scheduler-to-review-slot spawning; +a review slot MUST receive and verify the exact parent independent corpus gate. +Neither an IPC parent nor a serialized checkout object authorizes admission. + +The ordinary public modes remain corpus/full-review/normal-flow. The same launch +also admits full-review-runner with one campaign argument and review with campaign, +checkout, subject ID, repetition and output arguments. Unknown modes/argument +shapes fail before materialization. These paths require all explicit reviewed +plan/bootstrap identities and the complete source/runtime proof, including for a +direct invocation; they are no admission shortcut. Reviewed digest selection and +the trusted operator/host remain external prerequisites, without a new claim of +independent human-review or host attestation. + +```mermaid +flowchart LR + A[Reviewed external bootstrap and explicit pins] --> B[Verify plan and immutable closure] + B --> C[Private source and dependency proof] + C --> D[Repeat private-byte and runtime checks] + D --> E[Install process-local context and call driver] + E -->|nested process with same explicit pins| A +``` + +Text equivalent: independently checked external code admits the closure, copies +and verifies the private bytes, then imports a driver in that same process. Each +nested process repeats the complete launch. The removed internal worker endpoint +rejects both a direct call and an arbitrary IPC-connected parent before imports. + +Normal CLI selection, verifier profiles, cohort/order/retry policies, explicit +signals and existing scoring/latency gates are preserved. Parent cancellation +continues to reach the worker; its interrupted exit and cleanup remain observable. +No consumer option, production setting or research timeout/turn/read/token cap is +introduced. Existing frozen evidence requires its original historical drivers; +future measurements require the new exact driver identities and preregistration. + +Additional numeric budget: **seven deterministic security/lifecycle cases**, making +**20 bootstrap cases and 194 combined shadow controls**. Two cases reproduce the +old real verifier accepting a fabricated receipt and a direct runner importing a +checkout helper, then verify rejection. One rejects the removed worker entry without IPC. One executes the frozen f583 +bootstrap with an arbitrary IPC parent, proves its former admission, then verifies +current rejection. Two exercise the complete nested launch, including exact corpus +campaign propagation and private verifier mutation rejected before import. One sends SIGTERM after synthetic-worker readiness and +checks its interrupted exit and output. Its bounded fixture cleanup applies only +to that owned model-free test, never an investigation. Existing source/runtime, +normal-flow and independent-corpus controls remain required; no model calls or +evaluated-PR writes are needed for these controls. + +Acceptance requires all seven controls, the actual relative-lab zero-candidate +journey and complete combined controls to pass. Preserve the old bypass, failed +fixture attempts and successful outputs byte-exactly. Update operator guidance, +catalog evidence and Graphify; run specification/docs validation and the complete +required build/Jest/lint hook. Report native review/checks for the newly published +SHA separately. Neither a local pass nor an earlier SHA's review qualifies a profile. + + +### Pinned executable version health probe — #970, 2026-11-20 + +The admitted executable's `--version` health probe MUST fail closed after the +same fixed 15,000 ms operational timeout as the ordinary production version reader. +It MUST use SIGKILL for that probe process so handling SIGTERM cannot keep the +synchronous caller blocked. Executable/optional-launcher digest admission remains +before execution; successful version text must still match the reviewed runtime. +Timeout leaves qualification unstarted and propagates a preflight error rather +than a clean model verdict. No retry is implied; preserve its attempt/output. + +This rule applies only to executable metadata before a model is called. It MUST +NOT impose a time, turn, token or read limit on investigation, trim a verdict, +change cohort/order/retry policies, or replace explicit operation cancellation. +Production source and existing ordinary-planner behavior remain unchanged. The +fixed health rule is not a new consumer option or plan-controlled override. + +Additional numeric budget: **two real-process runtime cases**, making 17 focused +runtime cases and 196 combined controls. A byte-admitted direct CLI and an optional +launcher both enter --version and stall; the launcher also handles SIGTERM without +exiting. Each MUST yield ETIMEDOUT, leave no live fixture CLI and dispatch no model. +Owned test-process cleanup is 30 seconds and never constrains an investigation. +Run these assertions against the previous code first and retain the causal +failures, then the corrected results. Existing fast success, changed-byte refusal, +20 bootstrap cases and explicit cancellation remain required. Update operator +guidance/catalog, run specification/docs validation and Graphify, and pass the +complete mandatory hook plus actual final-head native checks/review. diff --git a/specs/catalog.json b/specs/catalog.json index 328a4386d..26a108cc4 100644 --- a/specs/catalog.json +++ b/specs/catalog.json @@ -664,9 +664,9 @@ "id": "bugbot-review-state-reconciliation", "title": "Bugbot review-state reconciliation", "status": "implemented", - "scope": "Reconcile review snapshots, findings, threads, comments, and check conclusions", + "scope": "Reconcile review snapshots, findings, threads, comments, and check conclusions; bind qualification execution to independently reviewed external bootstrap code with complete public admission in every process and bounded executable version health probes", "owner": "Copilot maintainers", - "lastVerified": "2026-11-19", + "lastVerified": "2026-11-20", "specs": [ "specs/bugbot-review-state-reconciliation.md" ], @@ -727,7 +727,15 @@ "src/application/policies/bugbot_status_card_publication_policy.ts", "src/infrastructure/github/github_bugbot_status_card_workflow.ts", "src/infrastructure/github/github_bugbot_status_card_dispatch.ts", - "src/infrastructure/github/native_list_pagination.ts" + "src/infrastructure/github/native_list_pagination.ts", + "quality/bugbot/evidence/issue-950/execution-provenance.cjs", + "quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs", + "quality/bugbot/evidence/issue-950/ev07-shadow/implementation.cjs", + "quality/bugbot/evidence/issue-950/normal-flow-shadow/run.cjs", + "quality/bugbot/evidence/issue-950/ev07-shadow/corpus.cjs", + "quality/bugbot/evidence/issue-950/ev07-shadow/review.cjs", + "quality/bugbot/evidence/issue-950/ev07-shadow/full-review.cjs", + "quality/bugbot/evidence/issue-950/ev07-shadow/launch-full-review.cjs" ], "tests": [ "src/domain/bugbot/__tests__/review_state.test.ts", @@ -773,7 +781,10 @@ "src/infrastructure/github/__tests__/github_bugbot_status_card_lane.test.ts", "src/infrastructure/github/__tests__/github_bugbot_status_card_dispatch.test.ts", "src/infrastructure/github/__tests__/github_bugbot_status_card_observation.test.ts", - "src/infrastructure/github/__tests__/github_bugbot_status_card_handoff.test.ts" + "src/infrastructure/github/__tests__/github_bugbot_status_card_handoff.test.ts", + "quality/bugbot/evidence/issue-950/qualification-bootstrap.test.cjs", + "quality/bugbot/evidence/issue-950/normal-flow-shadow/paths.test.cjs", + "quality/bugbot/evidence/issue-950/ev07-shadow/review-runtime.test.cjs" ], "documentation": [ "docs/bugbot/detection.mdx", @@ -784,7 +795,8 @@ "docs/bugbot/how-it-works.mdx", "docs/bugbot/programmatic-api.mdx", "docs/pull-requests/workflow-setup.mdx", - "docs/issues/comment-commands.mdx" + "docs/issues/comment-commands.mdx", + "docs/development/testing.mdx" ] }, {