Skip to content

Audit PDE benchmark with numerical controls and reproducible figures - #2

Open
Hydrodynamical wants to merge 4 commits into
mainfrom
polish-paper-kaggle-figures
Open

Hydrodynamical wants to merge 4 commits into
mainfrom
polish-paper-kaggle-figures

Conversation

@Hydrodynamical

@Hydrodynamical Hydrodynamical commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

The draft reported model recovery and behavioral diagnostics without simple query-policy controls or joint numerical-validity checks. This revision turns it into a reproducible benchmark audit: a fixed sine policy recovers all 30 reconstructed tasks at the strict threshold using the minimum permitted solve budget, which narrows the supported claims about adaptive model measurement selection.

The public preprint uses the official ICML format, with eight main-text pages and detailed appendices. It includes the five regenerated Kaggle figures, three new research figures, fixed/polynomial/random controls at three grid resolutions, an exact nonuniqueness proposition, joint-matrix and quantization audits, paired task-level comparisons, and run-balanced confidence summaries. Historical model outcomes remain distinct from reconstructed controls; missing attempts and historical solve discrepancies are documented.

Reproduction scripts validate the pinned archive and record source/runtime provenance. Full controls are stored as deterministic compressed JSON, with a compact summary; CI rejects stale derived outputs.

Validation: 78 offline tests pass; all 30 truths regenerate exactly; all 2,874 archived query RHS values match; 1,080 policy/grid runs and 360 solver comparisons completed; the 16-page PDF compiles without overfull boxes or unresolved references and was visually checked. A subsequent fixed-design validation retains every one of 300 fresh task seeds, including ten zero-forcing cases. Minimum-budget successes are 92/100 easy, 91/100 medium, and 98/100 hard; full-budget counts are 92/85/93. This bounds the original 30/30 finding and exposes zero-excitation and finite-precision failures. No provider calls or API costs were incurred.

The exposition now addresses PDE and ML readers together: forward versus inverse problems, weak-test regularity, measured source moments, internal regression rows, minimum-norm recovery, and the consistency-defect/pseudoinverse stability bound. It explicitly separates residuals, coefficient errors, exact identifiability, and numerical rank, and clarifies the model’s information access in the recovered prompt implementation.

@Hydrodynamical Hydrodynamical changed the title Polish paper with audited Kaggle results and figures Audit PDE benchmark with numerical controls and reproducible figures Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant