Skip to content

🔧 - Preregister and evaluate the Luna xhigh detector on the unchanged 174 case EV 07 corpus #974

Description

@efraespada

Is there an existing issue for this?

  • I have searched the existing issues.

What area does this task affect?

CI/CD

Task description

Preregister and evaluate the Luna xhigh detector on the unchanged 174-case EV-07 corpus. Deliver one independently reviewable result at the qualification phase 1 stage.

Scope and acceptance criteria

  • Start only after the deterministic prerequisites and independently justified corrections from the audit are merged. Commit the exact final implementation, actual CLI/launcher digests, driver and corpus hashes, prompts/schemas, order and retry policy before measured calls.
  • Use the ordinary detector and proposed Codex/OpenAI gpt-6-luna xhigh configuration. This measures the detector part of the proposed Luna-to-Sol profile; it is not evidence that Sol ran.
  • Account for all 174 outer cases and every actual generated-literal follow-up, interruption and failed call. Preserve optional unknown usage and provider-reported model identity; do not silently substitute models or retry to chase a pass.
  • Apply every unchanged gate: precision >=90%, recall >=85%, F1 >=87%, exact location/severity/category each >=80%, Brier <=0.16. Publish the complete ledger and exact score, including failure.
  • At least four deterministic admission controls reject foreign source/runtime, reject a failed corpus at the next phase, preserve incomplete attempts, and accept unknown usage as telemetry. Evaluated PR write attempts must be zero.
  • A failed or incomplete corpus keeps the full-review phase unstarted. A passing corpus only admits that next phase and does not qualify or activate the verifier profile.

Current issues or inefficiencies

The last campaign failed the independent detection gate; no Sol verification or full-review qualification followed. A campaign measured at an old source pin cannot qualify the merged code or a future corrective implementation.

Observed baseline: unreleased develop ccefb27627cd627ccdb7d5666fd174c2ac1da07b from merged PR #951.

Expected impact

Make this one pending result independently inspectable and reproducible, with its actual passed, failed or unavailable evidence.
Completion does not authorize release, producer trust, profile qualification from partial measurements, or production activation.

Additional context or comments

Priority and dependencies

Documentation and implementation boundary

Documentation: docs/bugbot/quality-observability.mdx.
Keep Clean Architecture and current coverage/contract gates. When a later implementation changes a public contract, update its catalogued SDD, tests and docs together and validate specifications; update Graphify after code changes.
Read the effective #950 contract in the owning SDD and its current reconciliation amendments; historical pilot deadlines are not current investigation limits.

Fixed invariants and planning state

Backlog only: no in-progress, branch creation, model execution, deployment or activation requested by this planning step.
Keep BUGBOT_VERIFICATION_MODE=disabled. Evaluated PRs must have zero write attempts; retain frozen pilots and independent evaluator/oracle evidence.
Token usage is optional telemetry and cannot stop investigation or reject a supported verdict. Elapsed time and turn count do not impose research cutoffs; explicit cancellation and evidence/SHA integrity remain mandatory. Post-run latency qualification criteria remain distinct from session deadlines.

Activity

  1. moved this to Todo in vypdevon Nov 19, 2026
  2. changed the title [-]Preregister and evaluate the Luna xhigh detector on the unchanged 174-case EV-07 corpus[/-] [+]🔧 - Preregister and evaluate the Luna xhigh detector on the unchanged 174 case EV 07 corpus[/+] on Nov 19, 2026
  3. vypbot commented on Nov 19, 2026

    @vypbot
    Contributor

    Hi! I’m @vypbot, the Copilot assistant for this repository.

    I can answer questions, explain the codebase, propose implementation and test plans, review issues and pull requests for potential bugs or security problems, and help authorized maintainers apply changes.

    Try /copilot help to see the available commands, or mention @vypbot with your question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

  • Status
    Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions