feat(analysis): bind Pareto candidate-K selection to an analysis-run profile - #409
seonghobae wants to merge 12 commits into
Conversation
…profile GAP-004 leftover / ADR 0053. Bind existing model_selection::select_candidate_k and selected_k_root_mean_square_error to cutoff-safe pareto_candidate_k_v1. LLM votes cannot define the numerical optimum. Not Schwarz fitted selection, not joint Laplace draws, not a Bayesian sampler, and not implemented-main.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
seonghobae
left a comment
There was a problem hiding this comment.
Operator COMMENT on exact head 7f24e43.
This is not an APPROVE. Author COMMENTED is not an independent review. Ruleset 18156473 still needs two independent current-head APPROVEs from non-author, non-bot reviewers.
Bind is Pareto select_candidate_k / selected-K RMSE as pareto_candidate_k_v1 (ADR 0053). Distinct from Schwarz fitted candidate-K (#404 / ADR 0049), interpreter/verifier (#405 / ADR 0050), topic activity (#407 / ADR 0051), and joint Laplace draws (#408 / ADR 0052). LLM votes cannot define the numerical optimum. Not a Bayesian sampler. Not implemented-main.
Do not self-approve. Do not merge without two independent approvals. Predecessor Checks do not transfer.
ADR 0053 is already live on #409 (Pareto candidate-K vs main). Rename this stack's decision identity rather than collide.
Resolve the CHANGELOG.md append conflict by keeping both entries. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Restack on protected main (a243f18)Non-force merge of Local evidence on the pushed head (toolchain 1.98.0): 🤖 Generated with Claude Code |
…the Pareto candidate-K profile An invalid completed_at timestamp is the only input that makes AnalysisRunTerminalResult::succeeded fail after execute_* validation. This exercises the previously uncovered '?' at that call site. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Coverage gap on this branch: located, one of three lines repaired
The last two are the "unreachable The Python job failure on this head is main's #492-owned hourly-cron test, unrelated to this branch. 🤖 Generated with Claude Code |
Summary
ADR 0053 / GAP-004 child. This Draft binds protected-main
model_selection::select_candidate_kandselected_k_root_mean_square_errortopareto_candidate_k_v1without copying Pareto dominance or RMSE arithmetic.Current exact head:
ce546f4da50dbbf7bb7f8780a9c0637d932f2379.Protected base:
main@a243f18da4a4ca8a8d068c39922537f1f8ed6ad0.Repaired contracts
The original branch was not actually cutoff-safe: candidate diagnostics carried no availability provenance, cutoff binding used RFC 3339 string equality, the quadratic Pareto scan had no application-path input ceiling, and terminal
evidence_countwas populated with candidate count.Current branch requires
ParetoCandidateKInputto bind the complete source-evidenceAvailableTimevector to an immutable snapshot and typedKnowledgeCutoff. Post-cutoff contributing evidence fails before numerical selection; equivalent RFC 3339 spellings bind by temporal instant; source evidence/replications are bounded byMAX_EVIDENCE_UNITS; candidate population is capped at 256 before the O(n²) scan; terminal evidence count is the actual source-evidence denominator; provider status isvalidatedwhile the artifact retains the scientific claim boundarypareto_statistical_front_not_fitted_schwarz_sampler.Completed artifact fields are private and externally read through accessors. The untrusted
from_jsonpath retains the 256 KiB admission cap; a maximal-valid escaping proof shows canonical validated egress is structurally below that cap, so the unreachable post-validation egress-size branch was removed without weakening inbound admission.RED / proof → repair lineage:
34ad6d4072a5af5136f91471d7d08b9b28f38ef9→c3173908a3ac90049886e0fd198564001404ace5: temporal provenance, equivalent-instant cutoff, resource envelope, real evidence denominator, validation/domain-claim separation.1b1bc4b26a02fac85a859ebe6fce891a79291a23→3cc15b76ab4f359850d3658349366d7cb2ef5af0: immutable completed-artifact public contract.1cd3bdf49a02dec84e7d4986d627394b3e24732d: ADR 0053 returned toProposed.03a2a9bbe2225b6110d0f1f715332cb01493fbfd: exact/+1 provenance and resource-boundary coverage.71086e9eacff6e1a4db025a73132d497902027ff: maximal-valid wire proof plus removal of the unreachable egress-size branch while preserving the inbound cap.ce546f4da50dbbf7bb7f8780a9c0637d932f2379: doctoring currentized to the complete repair lineage.LLM-vote-only sets remain non-authoritative. This is not Schwarz fitted candidate-
K, joint Gauss-Newton Laplace draws, a Bayesian sampler, GPU execution, or topic birth/split/merge.Scientific acceptance gap
Issue #500 owns the still-missing profile-level recovery evidence. Hand-built RMSE contracts do not establish true-
Krecovery frequency, bias/RMSE with Monte Carlo uncertainty, convergence/failure denominators, design stability, or leakage-safe temporal validation. This PR must not be described as scientifically accepted or release-ready while #500 is open without equivalent checked-in evidence.Consolidation
This PR is a Draft fold child of #416, not an independent landing authority. Relative to #416's current
03f8de2ed0a0fb842d2022d411814e440df7cfb4, the histories diverge from protected main. Do not Close #409 until #416 or a verified successor has inherited every valid source/test/ADR/doctoring/provenance/resource/acceptance-boundary delta.Shared
docs/TRACEABILITY.md,docs/product-technical-gap-baseline.md, and ADR-index currentization belongs to the #435 single-writer documentation lane. Comments are handoff evidence, not checked-in completion.Verification and merge gate
Required exact-head verification includes Rust tests/clippy/format, documentation validation, coverage, security/SAST/CodeQL and the organization required workflows. Predecessor receipts do not transfer.
Live ruleset 18156473 requires one qualifying current-head approval, dismisses stale approvals on push, requires review-thread resolution and central required workflows, and disallows non-fast-forward updates. Author/bot COMMENTED reviews are not a qualifying approval. Keep Draft until the fold/consolidation and scientific/release gates are actually satisfied.