perf(semantic): drop eager member byte hashing from lifecycle authority

[ScriptedAlchemy]

Crate-only change in crates/tracedecay-semantic making semantic serving/census-adjacent work cheaper on the successful path, plus the coordinator-required serving guard: a same-length digest-mismatched model can never become Current/serving. Stacked on origin/codex/tracedecay-total-redesign-plan (tip 2accc616).

1. Files + symbols changed

• crates/tracedecay-semantic/src/fastembed_adapter.rs — AdmittedProjectionArtifactV1::from_lifecycle_install, with the shared structural check factored into LifecycleInstallArtifactV1::member_pin_path (used by both construction and read_member_bytes).
• crates/tracedecay-semantic/src/lib.rs — DaemonSemanticRuntimeHandleV1::schedule_generation install paths now call a new warm_candidate_for_install before staging publication (see §2b).

2a. Why it is cheaper

from_lifecycle_install eagerly read and SHA-256 hashed all five artifact members — including the entire model file (default ceiling 700 MiB) — on every authority construction, then discarded the bytes. Its own comment admitted the redundancy: "Each future session open repeats the same check when it reads the bytes." Session open (open_session → required_member_bytes → read_member_bytes) re-reads and digest-verifies every member before any byte reaches ORT.

This constructor runs via LoadedSemanticArtifactV1::from_lifecycle / from_lifecycle_projection on the hot serving paths behind the live degraded reasons:

• every schedule_generation (load_artifact per saved-edit projection request), and
• every serving restore attempt (prepare_restore via the daemon's prepare_restore_current) — exactly the retry loop that runs while "no code generation is currently serving", where each attempt previously paid the full model read + hash twice (once at construction, once at the warm_query_session cold open).

Construction now performs only the byte-free structural pin checks per member (declared pin, normalized relative path, regular non-symlink file, exact length via one symlink_metadata call) — O(members) metadata instead of O(model bytes) read + hash. This matches the artifact-store authority (AdmittedProjectionArtifactV1::admit), whose admission already defers digest checks to reads. Plan 31 length AND SHA-256 both remain enforced: length eagerly at construction, digest at every session open, where the bytes are consumed. Install-time trust is unchanged (model_lifecycle/acquisition.rs verifies downloaded bytes against catalog pins).

Investigated and discarded the labeled hypothesis: first-publish schedule_generation does not re-embed — embedding is batch-checkpointed and completed_batch_offset skips committed batches; current/prepare_current_observation/query_factory are already pointer-cheap.

2b. Must-fix: warm before the serving pointer installs (review r3817700311)

With structural-only construction, schedule_generation's AlreadyPublished resume installed the serving pointer via install_candidate_on_success without ever opening a session — so a same-length digest-mismatched model could have become Current (and downstream mark_ready). The resume that reports every batch already committed had the same hole: the embed loop runs zero iterations, so no session opens before install.

Both install paths now call warm_candidate_for_install (a spawn_blocking warm_query_session) before publication staging, exactly as prepare_restore warms before commit. A cold open reads and digest-verifies every member; failure maps to SemanticRuntimeScheduleFailureV1::Runtime → Failed/degraded — never Current, never serving, never census_complete. After real embedding the warm reuses the idle pooled session with no re-read, so the cheaper-serving win is preserved.

3. Isolated test evidence

All tests are in-crate, hermetic (tempdir lifecycle-install fixtures), no live profile, no network model. cargo test -p tracedecay-semantic: 170 passed, 0 failed (169 lib + 1 integration). With --features semantic-fastembed (production FastEmbed/ORT runtime): 178 passed, 0 failed (177 lib + 1 integration, 2 pre-existing ignored).

Less work (operation-count proof — zero member byte reads at construction):
fastembed_adapter::tests::lifecycle_authority_construction_reads_no_member_bytes — the fixture's model member has the pinned length but not the pinned digest. Construction succeeds, which is only possible if it read zero member bytes (a digest verdict requires reading every byte). On the same authority, required_member_bytes(Model) and (under semantic-fastembed) open_session fail with typed CorruptArtifact.

Less time (baseline vs treatment on the same fixture):
fastembed_adapter::tests::lifecycle_authority_construction_is_cheaper_than_member_byte_verification — on one 16 MiB fixture, the baseline is a single read_member_bytes pass over all five members (byte-identical to the work the old constructor performed on top of what the new one does; still the per-session-open verification). The assert is construction < baseline, and construction is already the elapsed total of the 10-loop (review r3817700318). Ten constructions cheaper than one read+hash. Measured:

lifecycle authority construction: 10 constructions 176.695µs vs one member byte verification 17.217619ms

176.7 µs for the ten-construction total vs 17.2 ms for one baseline pass.

Serving guard (must-fix) — digest-mismatched model never becomes Current:

• scheduling_tests::already_published_resume_with_digest_mismatched_model_never_becomes_current — AlreadyPublished resume over the mismatched authority ends Failed { reason: Runtime, prior_generation: None }, current() stays None, and the publication-staging callback provably never runs. Falsified against the hole: with the warm removed, this test fails under semantic-fastembed (the pointer becomes Current) — verified locally, then the fix restored.
• scheduling_tests::restore_with_digest_mismatched_model_never_becomes_current — prepare_restore returns Err(Runtime); status stays Unavailable, current() stays None.

Fail-closed retained:

• lifecycle_authority_construction_rejects_structural_pin_violations — missing member, length-pin mismatch, and (unix) symlinked member still fail construction eagerly with typed CorruptArtifact.
• lifecycle_install_authority_verifies_member_bytes_at_read — valid pins construct; corrupt bytes fail at read; production runtime admits the install and open_session fails typed on non-ONNX fixture bytes.

4. Plan lock confirmation

• Publication ≠ activation: enforced more strongly than before — a finished/published projection batch no longer installs a serving pointer until one session provably opens over the artifact bytes. schedule_generation remains enqueue-only; exact/lexical/graph search stay non-blocking.
• Incomplete census stays Plan 38 degraded: no census, retention, or degraded-status logic was moved, silenced, or short-circuited; the daemon store_maintenance / code_index_scheduler log lines are unmodified. Not a faster degrade — the cheaper path is the successful authority construction.
• DeadlineExceeded is not serving/census_complete: deadline handling untouched; a deadline expiring during the pre-install warm is re-checked and returns the typed failure before staging. deadline_authority integration test green. Query embed keeps one-in-flight admission, non-blocking acquire, and deadline honoring (all existing runtime_query/session_pool tests pass unchanged).

  

[changeset-bot]

⚠️ No Changeset found

Latest commit: 45bdc4a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[chatgpt-codex-connector]

💡 Codex Review

tracedecay/crates/tracedecay-agent-hosts/build.rs

Line 27 in fe90ba8

#[path = "../../src/version/build_identity.rs"]

Bundle build-script inputs in the agent-host package

The release path in scripts/check-distribution-acceptance.sh packages every workspace crate, extracts the archives as flat siblings, and then checks the packaged root with those crates patched in. The tracedecay-agent-hosts archive cannot contain or resolve this ../../src/version/build_identity.rs path; the same build script also reads ../../Cargo.toml and ../../plugin. Consequently, compiling the packaged distribution fails before the release acceptance can complete. Package these inputs inside the crate or remove the repository-relative build-time dependencies.

---

tracedecay/crates/tracedecay-semantic/src/fastembed_adapter.rs

Line 473 in fe90ba8

lifecycle_install.member_pin_path(role)?;

Verify digests before installing an already-published runtime

When an installed member is corrupted without changing its length, this structural check now admits it, and the AlreadyPublished path in crates/tracedecay-semantic/src/lib.rs installs the candidate without opening or warming a session. Production then observes Current and calls mark_ready, so TraceDecay advertises semantic serving as ready even though the first query fails while opening the corrupt artifact. Retain digest verification on that publication path, or warm the candidate before installing it.

AGENTS.md reference: AGENTS.md:L98-L100

---

tracedecay/build.rs

Lines 72 to 74 in fe90ba8

	if !dashboard.join("node_modules").is_dir() {
	run_npm(dashboard, &["ci"])?;
	}

Refresh dashboard dependencies when the lockfile changes

In an existing checkout where dashboard/node_modules predates this commit's package and lockfile changes, this condition skips npm ci and immediately builds with the stale dependency tree. That can make ordinary Cargo builds fail on newly added packages or embed output generated with dependency versions different from the lockfile until the developer manually reinstalls. Track the installed lockfile state or refresh dependencies before automatic dashboard builds.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ScriptedAlchemy]

td query cross-review. Crate-only fastembed_adapter.rs change. I did not edit the crate, merge, or flatten the stack.

Holds from this side

• Incomplete stays degraded: no census/retention/degraded-status rewrite. Cheaper path is successful authority construction, not a faster fail.
• Construction is no longer a digest verdict. prepare_restore still warm_query_session() (open/read/hash) before commit, so a same-length digest-mismatched model cannot become current/serving. Query retrieve_semantic still cannot see that as Complete.
• Semantic miss/BudgetExceeded still must not take down exact/lexical/graph. Untouched here.
• DeadlineExceeded / Failed mapping untouched.

Nits only — inline. Stay draft until you want CI. #545 is up for your review.

[ScriptedAlchemy]

td query re-check on 32b5c8f. The construction≠serving hole is closed: AlreadyPublished and committed-batch now warm_candidate_for_install before stage_projection. Digest-mismatch stays Failed/Runtime, current() stays None, staging does not run. Query cannot see that as Complete. Incomplete stays degraded. No edit, no merge, no flatten. Not touching #545's base.

[ScriptedAlchemy]

td query deep review of #548 @ 32b5c8f (not #545). No crate edits, no restack/relink/flatten, no base change.

Lock holds. Construction is structural-only. prepare_restore, AlreadyPublished, and the all-batches-committed resume all warm_query_session before install. Featured open_session → fastembed_model → required_member_bytes → digest. Digest-mismatch stays Failed, current() stays None, stage_projection does not run. Query cannot treat that as Complete. Exact/lexical/graph stay unblocked (warm is spawn_blocking on the scheduler task). Incomplete stays degraded.

No serving bug found. Residuals only, inline. Ready watch stays #538. Left #509 alone.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fd6ada3c30

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Restore digest verification for generation inspection

When an installed model or tokenizer is corrupted without changing its length, this structural-only check now lets LoadedSemanticArtifactV1::from_lifecycle_projection succeed. ProductionSemanticRuntimeV1::inspect_generation then uses that constructor at crates/tracedecay-usecases/src/semantic_runtime/production.rs:2123-2132 and unconditionally emits SemanticExecutableGenerationV1 with cold_offline_ready and rollback_executable set to true at lines 2164-2169; the configuration backend trusts that evidence before committing activation and rollback. Thus a digest-mismatched artifact can be certified as byte-exact and rollback-ready even though a subsequent session open will reject it. Preserve the constructor optimization if needed, but make this production inspection path explicitly read and verify every pinned member before issuing executable evidence.

AGENTS.md reference: AGENTS.md:L119-L121

Useful? React with 👍 / 👎.