Skip to content

fix: bind Bugbot qualification to the executed source and runtime - #983

Open
efraespada wants to merge 7 commits into
developfrom
bugfix/970-bind-bugbot-qualification-execution-to-the-pinned-source-and-sanitized-git-environment
Open

efraespada wants to merge 7 commits into
developfrom
bugfix/970-bind-bugbot-qualification-execution-to-the-pinned-source-and-sanitized-git-environment

Conversation

@efraespada

@efraespada efraespada commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Qualification now starts only through an independently reviewed bootstrap Git blob and SHA-256 outside the checkout. It authenticates the committed plan and complete driver inventory before imports, materializes private source, verifies physical source/dependencies/runtime and executes admitted code. Trusted absolute OS Git and sanitized Git configuration also bind the actual application adapter; CLI/launcher bytes are checked before both the driver's and ordinary planner's version probes.

The final native review found that a checkout-written .qualification-admission.json could supply self-consistent plan/bootstrap hashes without that bootstrap executing. The actual old verifier accepts this fabricated record. This correction removes receipt-file authority: direct driver CLIs refuse before importing checkout helpers, and every worker starts through the same reviewed external bootstrap over parent IPC. Each worker repeats private plan/driver/source/runtime checks before receiving process-local context and loading its driver. Nested paired workers follow that same route and preserve their independent corpus gate.

IPC/context are transport, not independent attestation of the host or human review. Reviewed Node/Git/bootstrap/plan/package installation remain explicit operator trust anchors. External digest verification must happen before Node starts. Explicit cancellation, normal planner behavior, all score/latency gates and research policy remain unchanged.

Validation at f583e40c1bdb9c2e0154e10ea346cb42d58a9fe7: 193/193 combined shadow controls pass with zero skipped/cancelled tests, including six new forgery, direct-entry, IPC, nested-mutation and explicit-signal cases. The real relative-lab zero-candidate normal-flow journey remains covered. The old verifier is frozen byte-exactly for replay after squash; both old bypasses, failed fixture attempts and passing outputs remain as compressed logs with original hashes. Specification/catalog/docs validation and Graphify pass. The mandatory complete hook passes 943 suites / 18,695 tests, build and lint.

Final-SHA native checks/review are pending. The earlier01097099 controls and native medium finding are historical evidence, not validation of this correction; the bot must re-evaluate that finding. Application source, shipping bundles, consumer templates and frozen historical campaigns/labels/preregistrations remain unchanged. No new qualification campaign or evaluated-PR write occurred. Production remains disabled. #974 needs maintainer/Action integration of all prerequisites, combined verification and a new exact committed preregistration.

Closes #970. Refs #950, #974, #975.

…nd-sanitized-git-environment: bind qualification execution to physical pinned source (#970)
@docs-page

docs-page Bot commented Nov 19, 2026

Copy link
Copy Markdown

To preview the documentation for this pull request, visit the following URL:

docs.page/vypdev/copilot~983

Documentation is deployed and generated using docs.page

@vypbot
vypbot self-requested a review November 19, 2026 17:58
@vypbot vypbot added this to vypdev Nov 19, 2026
@vypbot vypbot moved this to In Progress in vypdev Nov 19, 2026
@vypbot
vypbot changed the base branch from develop to master November 19, 2026 17:59
@vypbot
vypbot changed the base branch from master to develop November 19, 2026 17:59
@vypbot vypbot added the size: XL Indicates that the issue or task is extra large in scope and complexity. label Nov 19, 2026
@efraespada
efraespada marked this pull request as ready for review November 19, 2026 18:00
@vypbot

vypbot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Copilot approval · Human review needed

Status: This pull request is outside the configured approval scope.
Completed: 4/4 observed checks succeeded; Bugbot complete.
Action required: Ask a human reviewer to decide.
Impact: No new approval is confirmed for this revision.
See checks

Technical evidence

Revision: 01097099d220; base: ccefb27627cd; reason: scope.

@vypbot vypbot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current status: All findings originating in this review are resolved.
Last reconciled on 0109709. See aggregate Bugbot status.

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit 43c8fd1 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

  • medium: Git proof commands use an unpinned executable from PATH — quality/bugbot/evidence/issue-950/execution-provenance.cjs:30

@vypbot

vypbot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Bugbot: 1 finding needs attention

Current status: 1 finding requires attention on 0109709.

Action: Review the linked threads or comment /copilot fix all.

Findings

Pull request · Verified commit · Current review checks · Workflow run

Previous implementation assessment · a1d4a63d

@codecov-commenter

codecov-commenter commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.66%. Comparing base (239a04f) to head (0109709).
⚠️ Report is 291 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #983      +/-   ##
===========================================
+ Coverage    95.25%   95.66%   +0.41%     
===========================================
  Files         1172     1563     +391     
  Lines        39236    49137    +9901     
  Branches     11685    15352    +3667     
===========================================
+ Hits         37373    47006    +9633     
- Misses         698      794      +96     
- Partials      1165     1337     +172     

see 321 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…nd-sanitized-git-environment: bind provenance to the operating-system Git executable (#970)

@vypbot vypbot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current status: All findings originating in this review are resolved.
Last reconciled on 0109709. See aggregate Bugbot status.

🤖 Bugbot review snapshot

Bugbot reported 2 potential problems when commit bdf9b50 was analyzed. This snapshot is historical; use the status block above for current state. 2 findings are linked to changed code.

Findings

  • high: Installed dependencies are outside the admission proof — quality/bugbot/evidence/issue-950/execution-provenance.cjs:11
  • high: The CLI version probe executes before its binary is pinned — quality/bugbot/evidence/issue-950/ev07-shadow/corpus.cjs:17

…nd-sanitized-git-environment: admit installed dependencies and CLI probes before execution (#970)
@vypbot vypbot added state:ai-processing 85% Progress: 85% and removed 70% Progress: 70% labels Nov 19, 2026

@vypbot vypbot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current status: All findings originating in this review are resolved.
Last reconciled on 0109709. See aggregate Bugbot status.

🤖 Bugbot review snapshot

Bugbot reported 2 potential problems when commit a1d4a63 was analyzed. This snapshot is historical; use the status block above for current state. 2 findings are linked to changed code.

Findings

  • high: Authenticate the runtime manifest before probing its executable — quality/bugbot/evidence/issue-950/normal-flow-shadow/run.cjs:118
  • high: The admission verifier executes before its own integrity check — quality/bugbot/evidence/issue-950/ev07-shadow/corpus.cjs:8

…nd-sanitized-git-environment: admit immutable drivers before loading the verifier (#970)
…nd-sanitized-git-environment: verify materialized driver blobs and physical manifest paths (#970)
…nd-sanitized-git-environment: pin application Git and admit CLI before planner probes (#970)
@vypbot vypbot added size: XXL Indicates an extremely large task or issue that requires significant time and effort to complete. state:ai-processing and removed size: XL Indicates that the issue or task is extra large in scope and complexity. labels Nov 19, 2026

@vypbot vypbot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current status: 1 finding originating in this review requires attention.
Last reconciled on 0109709. See aggregate Bugbot status.

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit 0109709 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

  • medium: Checkout can forge the bootstrap admission receipt — quality/bugbot/evidence/issue-950/execution-provenance.cjs:248

Comment on lines +248 to +249
assert.equal(hash(fs.readFileSync(path.join(root, 'quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs'))),
proof.bootstrapSha256, 'bootstrap admission identity differs');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checkout can forge the bootstrap admission receipt

Severity: medium

Category: security · Confidence: 88%

Location: quality/bugbot/evidence/issue-950/execution-provenance.cjs:248-249

requireBootstrapAdmission takes bootstrapSha256 from the checkout-writable .qualification-admission.json and compares it with the bootstrap file in that same checkout. The receipt also supplies the plan commit, path, and digest used by the subsequent artifact check. A direct invocation can therefore provide a self-consistent receipt without the independently reviewed bootstrap having launched the runner. normal-flow-shadow/run.cjs accepts this result before authenticating its manifest, so qualification output can claim bootstrap provenance without the external trust check; a modified checkout can self-preregister its own source and drivers as well.
Evidence:
The receipt is parsed from <root>/.qualification-admission.json; these lines compare its declared bootstrap digest only to <root>/quality/bugbot/evidence/issue-950/qualification-bootstrap.cjs. No value is checked against an independently supplied trust anchor. The runner calls this helper at normal-flow-shadow/run.cjs:110, then uses the receipt’s plan fields to authenticate the manifest.

Suggested fix:
Keep admission enforcement in the external bootstrap and prevent a working-tree runner from treating a checkout-writable receipt as proof that the reviewed bootstrap launched it.

…nd-sanitized-git-environment: keep admission in the external bootstrap (#970)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

65% Progress: 65% size: XXL Indicates an extremely large task or issue that requires significant time and effort to complete. state:ai-processing state:awaiting-issue-author state:changes-requested

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

🐛🧑‍💻 - Bind Bugbot qualification execution to the pinned source and sanitized Git environment

3 participants