Skip to content

scientific(validation): bind recovery promotion to immutable versioned profile provenance #627

Description

@seonghobae

Finding

#623–#626 repaired practical-target/Monte-Carlo separation, extreme binary64 uncertainty composition, pseudo-replication, and survivor-only denominators. The remaining Validation Evidence defect was provenance: request-time planned_replications, max_rmse, and se_multiplier scalars did not bind the data-generating process, seed stream, estimand/state composition, or failure policy to the promoted authority. The same recovery rows could therefore be re-submitted under a different design/acceptance policy without producing a different durable scientific-authority identity.

Morris, White, & Crowther (2019, DOI 10.1002/sim.8086) treats simulation studies as empirical experiments and recommends pre-specifying aims, data-generating mechanisms, estimands, methods, performance measures, n_sim, and Monte Carlo precision (ADEMP). Exact candidate-head binding proves which source was tested; it does not by itself prove which scientific design was declared.

RED → causal repair

Canonical owner vehicle is Draft #488.

Ordinary-forward lineage from #626 head 59ab6f907c19726290f40cc6ac390fe32d2bfbeb:

  • public RED f50edf3564f9f3f033f0ebbc693531c6411a4be6 requires one versioned profile identity to survive promotion, different denominator/target profiles to produce different digests, and malformed/noncanonical dependency digests to fail closed;
  • dependency commit b68780b840857208236be4b1744d242670077712 adds SHA-256 support;
  • causal production repair 004391a4f6dc562d680d9e3620b6ae6c40e5ffea introduces ScientificRecoveryProfileV1, domain-separated derived profile SHA-256, and ScientificRecoveryPromotionV1 carrying that identity beside the promoted claim;
  • root export 7f4c2bf2f336ec2bebcf346cf2352d54d1e0ae0f;
  • public contract migrations ce0f8304644899c7a016de67d74ec7a2d4c7b8c0, 854b8e49c9820816f0b3896908388969567a4b5c, 3cd6c78f6a32bfae6154c1b5521aec9aaba82d5a, d0fcf3d6d52513630bc89a4329f4399780e5d891, 4202181d238bc2939e5fdb35625f013f3dec3ecb;
  • profile CHANGELOG/documentation 6f8edddade85e1a8660557745d057e602c2c2805.

The profile now binds:

  • planned independent replication count;
  • practical max_rmse;
  • Monte Carlo se_multiplier;
  • immutable DGP/config SHA-256;
  • immutable seed-manifest SHA-256;
  • immutable estimand SHA-256;
  • immutable within-replication state-composition SHA-256;
  • explicit RequireAllPlannedRecovered failure policy.

The digest is derived by domain-separated, length-prefixed SHA-256 over canonical represented fields; callers cannot provide a detached profile digest. Unsupported schema, unknown JSON fields, and noncanonical dependency digests fail closed. Promotion accepts the profile object rather than request-time design scalars and returns its exact derived digest with the scientific claim.

#628 subsequently found and repaired one canonical-representation edge in this profile: IEEE 754 -0.0 was scientifically equivalent to +0.0 as a zero uncertainty multiplier but had different raw bits and therefore a different profile digest. #628 keeps canonical +0.0 and rejects sign-negative zero; that refinement is separate from this issue's content-binding repair.

Remaining acceptance / chronology gap

Current #488 exact head is ccc9f45e94638695ae40831ceba03ceb5b1fb308, Draft/open/mergeable on protected main@a243f18da4a4ca8a8d068c39922537f1f8ed6ad0. Predecessor receipts do not transfer. Current workflows are live and non-terminal; qualifying independent current-head APPROVED review is absent.

The repair proves profile content binding, not publication chronology. A buyer/integration path must still prove that the exact profile artifact was persisted/approved before simulation execution began. Do not infer chronology from the in-memory value object, caller timestamps, commit time alone, or a post-hoc profile with matching contents.

Keep reusable metric arithmetic in its existing owners. No LLM judgment, mutable sibling source, cross-service SQL, universal RMSE cutoff, tolerance, or hidden effective-sample-size heuristic is introduced. Keep #623 practical-target semantics, #624 strict conservative bound, #625 independent-replication grouping, and #626 planned-denominator refusal intact.

Keep this issue open through exact-head Rust/coverage/security/CodeQL/proof-budget, qualifying independent review, normal prerequisite/protected-main integration, and code-current TRACEABILITY/product-gap authority. Source repair alone is not completion.

Refs #488 #623 #624 #625 #626 #628 #492.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions