Skip to content

feat(analysis): bind Pareto candidate-K selection to an analysis-run profile - #409

Draft
seonghobae wants to merge 12 commits into
mainfrom
feat/pareto-candidate-k-analysis-run-gap-004
Draft

seonghobae wants to merge 12 commits into
mainfrom
feat/pareto-candidate-k-analysis-run-gap-004

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

ADR 0053 / GAP-004 child. This Draft binds protected-main model_selection::select_candidate_k and selected_k_root_mean_square_error to pareto_candidate_k_v1 without copying Pareto dominance or RMSE arithmetic.

Current exact head: ce546f4da50dbbf7bb7f8780a9c0637d932f2379.
Protected base: main@a243f18da4a4ca8a8d068c39922537f1f8ed6ad0.

Repaired contracts

The original branch was not actually cutoff-safe: candidate diagnostics carried no availability provenance, cutoff binding used RFC 3339 string equality, the quadratic Pareto scan had no application-path input ceiling, and terminal evidence_count was populated with candidate count.

Current branch requires ParetoCandidateKInput to bind the complete source-evidence AvailableTime vector to an immutable snapshot and typed KnowledgeCutoff. Post-cutoff contributing evidence fails before numerical selection; equivalent RFC 3339 spellings bind by temporal instant; source evidence/replications are bounded by MAX_EVIDENCE_UNITS; candidate population is capped at 256 before the O(n²) scan; terminal evidence count is the actual source-evidence denominator; provider status is validated while the artifact retains the scientific claim boundary pareto_statistical_front_not_fitted_schwarz_sampler.

Completed artifact fields are private and externally read through accessors. The untrusted from_json path retains the 256 KiB admission cap; a maximal-valid escaping proof shows canonical validated egress is structurally below that cap, so the unreachable post-validation egress-size branch was removed without weakening inbound admission.

RED / proof → repair lineage:

  • 34ad6d4072a5af5136f91471d7d08b9b28f38ef9 → c3173908a3ac90049886e0fd198564001404ace5: temporal provenance, equivalent-instant cutoff, resource envelope, real evidence denominator, validation/domain-claim separation.
  • 1b1bc4b26a02fac85a859ebe6fce891a79291a23 → 3cc15b76ab4f359850d3658349366d7cb2ef5af0: immutable completed-artifact public contract.
  • 1cd3bdf49a02dec84e7d4986d627394b3e24732d: ADR 0053 returned to Proposed.
  • 03a2a9bbe2225b6110d0f1f715332cb01493fbfd: exact/+1 provenance and resource-boundary coverage.
  • 71086e9eacff6e1a4db025a73132d497902027ff: maximal-valid wire proof plus removal of the unreachable egress-size branch while preserving the inbound cap.
  • ce546f4da50dbbf7bb7f8780a9c0637d932f2379: doctoring currentized to the complete repair lineage.

LLM-vote-only sets remain non-authoritative. This is not Schwarz fitted candidate-K, joint Gauss-Newton Laplace draws, a Bayesian sampler, GPU execution, or topic birth/split/merge.

Scientific acceptance gap

Issue #500 owns the still-missing profile-level recovery evidence. Hand-built RMSE contracts do not establish true-K recovery frequency, bias/RMSE with Monte Carlo uncertainty, convergence/failure denominators, design stability, or leakage-safe temporal validation. This PR must not be described as scientifically accepted or release-ready while #500 is open without equivalent checked-in evidence.

Consolidation

This PR is a Draft fold child of #416, not an independent landing authority. Relative to #416's current 03f8de2ed0a0fb842d2022d411814e440df7cfb4, the histories diverge from protected main. Do not Close #409 until #416 or a verified successor has inherited every valid source/test/ADR/doctoring/provenance/resource/acceptance-boundary delta.

Shared docs/TRACEABILITY.md, docs/product-technical-gap-baseline.md, and ADR-index currentization belongs to the #435 single-writer documentation lane. Comments are handoff evidence, not checked-in completion.

Verification and merge gate

Required exact-head verification includes Rust tests/clippy/format, documentation validation, coverage, security/SAST/CodeQL and the organization required workflows. Predecessor receipts do not transfer.

Live ruleset 18156473 requires one qualifying current-head approval, dismisses stale approvals on push, requires review-thread resolution and central required workflows, and disallows non-fast-forward updates. Author/bot COMMENTED reviews are not a qualifying approval. Keep Draft until the fold/consolidation and scientific/release gates are actually satisfied.


Devin Review

…profile

GAP-004 leftover / ADR 0053. Bind existing model_selection::select_candidate_k
and selected_k_root_mean_square_error to cutoff-safe pareto_candidate_k_v1.
LLM votes cannot define the numerical optimum. Not Schwarz fitted selection,
not joint Laplace draws, not a Bayesian sampler, and not implemented-main.
@coderabbitai

coderabbitai Bot commented Aug 31, 2026 •

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operator COMMENT on exact head 7f24e43.

This is not an APPROVE. Author COMMENTED is not an independent review. Ruleset 18156473 still needs two independent current-head APPROVEs from non-author, non-bot reviewers.

Bind is Pareto select_candidate_k / selected-K RMSE as pareto_candidate_k_v1 (ADR 0053). Distinct from Schwarz fitted candidate-K (#404 / ADR 0049), interpreter/verifier (#405 / ADR 0050), topic activity (#407 / ADR 0051), and joint Laplace draws (#408 / ADR 0052). LLM votes cannot define the numerical optimum. Not a Bayesian sampler. Not implemented-main.

Do not self-approve. Do not merge without two independent approvals. Predecessor Checks do not transfer.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 6 potential issues.

Devin Review

Comment thread crates/analysis_engine/src/pareto_candidate_k_artifact.rs
Comment thread crates/analysis_engine/src/pareto_candidate_k_artifact.rs
Comment thread crates/analysis_engine/src/pareto_candidate_k_artifact.rs Outdated
Comment thread crates/analysis_engine/src/pareto_candidate_k_artifact.rs
Comment thread crates/analysis_engine/src/pareto_candidate_k_artifact.rs Outdated
seonghobae added a commit that referenced this pull request Aug 31, 2026
ADR 0053 is already live on #409 (Pareto candidate-K vs main).
Rename this stack's decision identity rather than collide.
@seonghobae
seonghobae marked this pull request as draft September 1, 2026 16:36
Resolve the CHANGELOG.md append conflict by keeping both entries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@seonghobae

Copy link
Copy Markdown
Contributor Author

Restack on protected main (a243f18)

Non-force merge of origin/main (merge commit 7c2180eb); the only conflict was the CHANGELOG.md append, both entries kept. ADR 0053 already carries an admitted maturity value (active-PR).

Local evidence on the pushed head (toolchain 1.98.0): cargo test -p analysis_engine 39 passed / 0 failed, cargo clippy -p analysis_engine --all-targets -D warnings clean, cargo fmt --all --check clean, documentation/workspace/docstring contracts PASS, git diff --check clean.

🤖 Generated with Claude Code

…the Pareto candidate-K profile

An invalid completed_at timestamp is the only input that makes
AnalysisRunTerminalResult::succeeded fail after execute_* validation.
This exercises the previously uncovered '?' at that call site.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@seonghobae

Copy link
Copy Markdown
Contributor Author

Coverage gap on this branch: located, one of three lines repaired

Production line and branch coverage fails on 7c2180eb for three lines in crates/analysis_engine/src/pareto_candidate_k_artifact.rs (local cargo llvm-cov -p analysis_engine):

  • 235 — ? on AnalysisRunTerminalResult::succeeded(...): reachable only via an invalid completed_at. Repaired in the pushed commit with invalid_completed_at_fails_terminal_result_construction (asserts Err(Api(InvalidWirePayload))); 5/5 execution-contract tests pass, clippy/fmt/docstrings clean.
  • 226 — ? on AnalysisResultSummary::new(…, candidate_count, 2, …): only fails above the 1e9 summary bound, unreachable after execute_* validation.
  • 119 — post-serialize payload.len() > PARETO_CANDIDATE_K_ARTIFACT_BYTE_LIMIT in to_json(): unreachable with validate()-bounded fields.

The last two are the "unreachable ? / dead byte-limit branch" categories recorded on #416 and need an owner decision.

The Python job failure on this head is main's #492-owned hourly-cron test, unrelated to this branch.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant