Skip to content

🔧 - Measure the proposed Luna to Sol xhigh profile in the 42 paired full review EV 07 slots #975

Description

@efraespada

Is there an existing issue for this?

  • I have searched the existing issues.

What area does this task affect?

CI/CD

Task description

Measure the proposed Luna to Sol xhigh profile in the 42 paired full-review EV-07 slots. Deliver one independently reviewable result at the qualification phase 2 stage.

Scope and acceptance criteria

  • Admission requires an actually passing corpus for the identical implementation/driver/runtime pins, not merely closure of the preceding issue. Freeze the paired phase before its first measured call; any changed source requires a new preregistration and applicable corpus evaluation.
  • Retain all 14 fixed-tree subjects and three repetitions (42 slots). Run the ordinary detector/preparation entrypoint; compare its ordinary dry-run decision with shadow verification of the same actual prepared candidates. Zero candidates opens zero verifier sessions; seeded replays cannot count as detection.
  • Use Codex/OpenAI Luna detection and Sol verification at xhigh through the existing REVIEWER_* / optional verifier inheritance. Report resolved and provider-reported models separately. Do not pool another profile or historical campaign into qualification.
  • Require >=3 reproduced baseline alerts across >=2 historical premises, strict paired improvement in >=2 repetitions with no regression in the third, every positive control at the exact anchor, complete valid citations and integrity, and zero evaluated-PR write attempts.
  • Measure added subject p95 <=90 seconds after all work finishes. This is a qualification criterion, never a per-session deadline. Time/tokens/turns do not stop investigation, and unknown usage cannot reject a verdict.
  • At least eight deterministic preflight controls cover write-blocked ports, exact model/effort inheritance, zero candidates, integrity rejection, cancellation, incomplete evidence, stale SHA and complete receipt delivery. Official/dependency/CI evidence must preserve its proven version and revision scope.
  • Publish every slot and first failed stage. Apply all effective SDD requirements, including applicable profile comparisons; identify any unmeasured comparison explicitly. If baseline reproduction is insufficient, preserve originals and preregister any permitted expansion separately.
  • Close with an explicit qualified/not-qualified decision for this exact profile and all remaining criteria. Even a pass keeps production disabled; enabling requires a separate explicit maintainer decision.

Current issues or inefficiencies

All 42 paired full-review slots remained unstarted after the failed corpus gate. False-positive reduction, positive-control retention, evidence grounding and added verifier latency for this exact profile are unmeasured.

Observed baseline: unreleased develop ccefb27627cd627ccdb7d5666fd174c2ac1da07b from merged PR #951.

Expected impact

Make this one pending result independently inspectable and reproducible, with its actual passed, failed or unavailable evidence.
Completion does not authorize release, producer trust, profile qualification from partial measurements, or production activation.

Additional context or comments

Priority and dependencies

Documentation and implementation boundary

Documentation: docs/bugbot/quality-observability.mdx, docs/bugbot/configuration.mdx.
Keep Clean Architecture and current coverage/contract gates. When a later implementation changes a public contract, update its catalogued SDD, tests and docs together and validate specifications; update Graphify after code changes.
Read the effective #950 contract in the owning SDD and its current reconciliation amendments; historical pilot deadlines are not current investigation limits.

Fixed invariants and planning state

Backlog only: no in-progress, branch creation, model execution, deployment or activation requested by this planning step.
Keep BUGBOT_VERIFICATION_MODE=disabled. Evaluated PRs must have zero write attempts; retain frozen pilots and independent evaluator/oracle evidence.
Token usage is optional telemetry and cannot stop investigation or reject a supported verdict. Elapsed time and turn count do not impose research cutoffs; explicit cancellation and evidence/SHA integrity remain mandatory. Post-run latency qualification criteria remain distinct from session deadlines.

Activity

  1. moved this to Todo in vypdevon Nov 19, 2026
  2. changed the title [-]Measure the proposed Luna to Sol xhigh profile in the 42 paired full-review EV-07 slots[/-] [+]🔧 - Measure the proposed Luna to Sol xhigh profile in the 42 paired full review EV 07 slots[/+] on Nov 19, 2026
  3. vypbot commented on Nov 19, 2026

    @vypbot
    Contributor

    Hi! I’m @vypbot, the Copilot assistant for this repository.

    I can answer questions, explain the codebase, propose implementation and test plans, review issues and pull requests for potential bugs or security problems, and help authorized maintainers apply changes.

    Try /copilot help to see the available commands, or mention @vypbot with your question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

  • Status
    Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions