Skip to content

chore: audit the frozen EV-07 failures and preserve causal evidence - #984

Open
efraespada wants to merge 2 commits into
developfrom
chore/971-audit-ev-07-precision-and-calibration-failures-before-another-measured-campaign
Open

efraespada wants to merge 2 commits into
developfrom
chore/971-audit-ev-07-precision-and-calibration-failures-before-another-measured-campaign

Conversation

@efraespada

@efraespada efraespada commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

The EV-07 audit reproduces the current and prior failed scores without changing labels, thresholds or predictions. All 17 scorer FP and 11 FN now have independently reviewable causal evidence and a next action.

What changed

Validation

  • Three deterministic scoring controls pass: exact current result, exact prior result and refusal to reinterpret a diagnostic category change as a qualifying campaign.
  • Immutable owned Node oracles reproduce the bypassed stub, missing no-store and selected opaque-token failures. Shell quoting and Python short-circuit controls ran. Actual Python discovery runs 1 unrelated method / 0 export methods for the wrong directory, against 19 / 5 in its passing controls. Kotlin 2.4.10 runs its compiler on JRE 26.0.1; OpenJDK 21.0.11 executes the exception-mapping primitive.
  • Installed commit hook passed build, 943 Jest suites (18,695 tests passed; 35 existing skips), and lint. Documentation/specification validation, diff checks and AST-only Graphify passed.
  • Audit, all rows and reproduction.

Review notes

No production prompt correction is supported solely by these taxonomy differences. The malformed workflow has a separately reviewed fixture/rubric follow-up in #985; unresolved cases name their missing contracts. No current corpus labels or historical results change. A recovered local artifact is not independent server timestamp attestation. Synthetic primitives do not execute private consumers or qualify a model profile.

The native-review runtime attribution is corrected at 1f3fbddeb4b84dea231ddeb7dc18af54fa33a565. Its full installed hook passed; await final-SHA native checks and review.

This audit makes zero model calls and zero evaluated-PR writes. Production remains disabled. #974 still requires merged prerequisites and a new committed exact preregistration; all 42 #975 slots remain unstarted.

Closes #971.

…er-measured-campaign: audit frozen EV-07 discrepancies without relabeling (#971)
@docs-page

docs-page Bot commented Nov 19, 2026

Copy link
Copy Markdown

To preview the documentation for this pull request, visit the following URL:

docs.page/vypdev/copilot~984

Documentation is deployed and generated using docs.page

@vypbot vypbot added the size: XXL Indicates an extremely large task or issue that requires significant time and effort to complete. label Nov 19, 2026
@efraespada
efraespada marked this pull request as ready for review November 19, 2026 18:07
@vypbot vypbot added the 55% Progress: 55% label Nov 19, 2026
@vypbot

vypbot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Copilot approval · Human review needed

Status: Evidence passed; a human reviewer must decide.
Completed: 4/4 observed checks succeeded; Bugbot complete.
Action required: Ask a human reviewer to decide.
Impact: No new approval is confirmed for this revision.
See checks

Technical evidence

Revision: 1f3fbddeb4b8; base: ccefb27627cd; reason: recommend-mode.

@vypbot vypbot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current status: All findings originating in this review are resolved.
Last reconciled on 1f3fbdd. See aggregate Bugbot status.

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit 6417074 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

  • low: The Kotlin probe's execution runtime is misidentified — quality/bugbot/evidence/issue-971/README.md:28

@vypbot

vypbot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Bugbot: review complete

Current status: No active findings on 1f3fbdd.

Pull request · Verified commit · Current review checks · Workflow run

…er-measured-campaign: distinguish Kotlin compiler and probe runtimes (#971)
@codecov-commenter

codecov-commenter commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.66%. Comparing base (239a04f) to head (1f3fbdd).
⚠️ Report is 291 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #984      +/-   ##
===========================================
+ Coverage    95.25%   95.66%   +0.41%     
===========================================
  Files         1172     1563     +391     
  Lines        39236    49137    +9901     
  Branches     11685    15352    +3667     
===========================================
+ Hits         37373    47006    +9633     
- Misses         698      794      +96     
- Partials      1165     1337     +172     

see 321 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

60% Progress: 60% size: XXL Indicates an extremely large task or issue that requires significant time and effort to complete. state:awaiting-maintainer state:ready

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants