Repository navigation
Description
Is there an existing issue for this?
- I have searched the existing issues.
What area does this task affect?
CI/CD
Task description
Preregister and evaluate the Luna xhigh detector on the unchanged 174-case EV-07 corpus. Deliver one independently reviewable result at the qualification phase 1 stage.
Scope and acceptance criteria
- Start only after the deterministic prerequisites and independently justified corrections from the audit are merged. Commit the exact final implementation, actual CLI/launcher digests, driver and corpus hashes, prompts/schemas, order and retry policy before measured calls.
- Use the ordinary detector and proposed Codex/OpenAI gpt-6-luna xhigh configuration. This measures the detector part of the proposed Luna-to-Sol profile; it is not evidence that Sol ran.
- Account for all 174 outer cases and every actual generated-literal follow-up, interruption and failed call. Preserve optional unknown usage and provider-reported model identity; do not silently substitute models or retry to chase a pass.
- Apply every unchanged gate: precision >=90%, recall >=85%, F1 >=87%, exact location/severity/category each >=80%, Brier <=0.16. Publish the complete ledger and exact score, including failure.
- At least four deterministic admission controls reject foreign source/runtime, reject a failed corpus at the next phase, preserve incomplete attempts, and accept unknown usage as telemetry. Evaluated PR write attempts must be zero.
- A failed or incomplete corpus keeps the full-review phase unstarted. A passing corpus only admits that next phase and does not qualify or activate the verifier profile.
Current issues or inefficiencies
The last campaign failed the independent detection gate; no Sol verification or full-review qualification followed. A campaign measured at an old source pin cannot qualify the merged code or a future corrective implementation.
Observed baseline: unreleased develop ccefb27627cd627ccdb7d5666fd174c2ac1da07b from merged PR #951.
- quality/bugbot/evidence/issue-950/ev07-shadow/preregistration.md
- quality/bugbot/evidence/issue-950/ev07-shadow/campaign-provenance-results.md
Expected impact
Make this one pending result independently inspectable and reproducible, with its actual passed, failed or unavailable evidence.
Completion does not authorize release, producer trust, profile qualification from partial measurements, or production activation.
Additional context or comments
Priority and dependencies
- Priority: high. Stage: Qualification phase 1.
- Blocked by: 🐛🧑💻 - Bind Bugbot qualification execution to the pinned source and sanitized Git environment #970, 🔧🧑💻 - Audit EV 07 precision and calibration failures before another measured campaign #971, 🐛🧑💻 - Forward explicit cancellation into the normal flow Bugbot shadow review operation #972, 🐛🧑💻 - Keep deletion only diff hunks out of changed line citation provenance #973
- Follow-up to ✨🧑💻 - Verify Bugbot findings against repository evidence before publication #950 and merged PR feat: Verify Bugbot findings against repository evidence before publication (#950) #951, not a request to reopen or extend that delivery.
- Final closure and observed limitations.
Documentation and implementation boundary
Documentation: docs/bugbot/quality-observability.mdx.
Keep Clean Architecture and current coverage/contract gates. When a later implementation changes a public contract, update its catalogued SDD, tests and docs together and validate specifications; update Graphify after code changes.
Read the effective #950 contract in the owning SDD and its current reconciliation amendments; historical pilot deadlines are not current investigation limits.
Fixed invariants and planning state
Backlog only: no in-progress, branch creation, model execution, deployment or activation requested by this planning step.
Keep BUGBOT_VERIFICATION_MODE=disabled. Evaluated PRs must have zero write attempts; retain frozen pilots and independent evaluator/oracle evidence.
Token usage is optional telemetry and cannot stop investigation or reject a supported verdict. Elapsed time and turn count do not impose research cutoffs; explicit cancellation and evidence/SHA integrity remain mandatory. Post-run latency qualification criteria remain distinct from session deadlines.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity
[-]Preregister and evaluate the Luna xhigh detector on the unchanged 174-case EV-07 corpus[/-][+]🔧 - Preregister and evaluate the Luna xhigh detector on the unchanged 174 case EV 07 corpus[/+]vypbot commentedon Nov 19, 2026
Hi! I’m @vypbot, the Copilot assistant for this repository.
I can answer questions, explain the codebase, propose implementation and test plans, review issues and pull requests for potential bugs or security problems, and help authorized maintainers apply changes.
Try
/copilot helpto see the available commands, or mention @vypbot with your question.