Skip to content

feat: add V2 foundation crates as stacked delivery base - #509

Open
ScriptedAlchemy wants to merge 9 commits into
masterfrom
feat/v2-foundation-crates
Open

feat: add V2 foundation crates as stacked delivery base#509
ScriptedAlchemy wants to merge 9 commits into
masterfrom
feat/v2-foundation-crates

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Aug 19, 2026

Copy link
Copy Markdown
Owner

Summary

First slice of the stacked delivery for #421. This PR carves the fully implemented low-layer V2 crates out of the consolidated branch so they can merge to master independently; #421 is restacked on top of this branch and shrinks accordingly.

  • Adds ten new workspace crates from the delivery branch: tracedecay-api, -application, -host-integration, -hooks, -policy, -private-fs, -rusqlite-runtime, -store, -temporal-query, -tool-catalog.
  • Adopts the V2 tracedecay-domain contract modules. Protobuf node kinds become unconditional domain vocabulary, so tracedecay-code-extraction stops forwarding the removed lang-protobuf domain feature.
  • Keeps the seven master-era result structs (EditResult, MultiEditResult, InsertResult, AstGrepResult, MoveHint, MoveResult, CostTurn) in domain::code_intelligence because master's runtime-core/root crate still consume them; the stacked branch deletes them together with their callers.
  • Includes the host-event fixtures, the one consumed provider-normalization fixture, and the rusqlite runtime suite module sources the crate tests compile against. The suite's root-package harness (main.rs) stays on the stacked branch: it needs the V2 root crate, and Cargo would otherwise auto-discover it as a root test target here.
  • Updates master's workspace architecture contract for the ten new members (with their dependency edges and a scoped rusqlite exemption for the sanctioned engine crate) and teaches its reachability scanner rustc's #[path] sibling-resolution semantics.

Parity with the delivery branch

The carve started byte-identical; the following deliberate, disclosed divergences now exist (each is a fix #421 should adopt verbatim on restack — see the review comment for the conflict inventory):

  • 43260d5 — repo-toolchain fmt/clippy gate fixes (the delivery branch lints under rustc 1.97 and never sees these).
  • b57b35b — platform-absolute fixture roots so the work/placement/multi-root/store-locator tests pass on Windows.
  • b728605/ea0fa23/fd1f2e3 — typed catalog-identity errors instead of expect in Result paths, canonical hex reuse in generate_node_id, and only the consumed provider fixture carried (the other 36 arrive with their consumers in feat: consolidate TraceDecay V2 PR8-PR13 delivery #421).

Behavior notes

  • NodeKind::ProtoMessage/ProtoService/ProtoRpc are now always part of the domain vocabulary: graph payloads containing protobuf node kinds deserialize on every build, not only under lang-protobuf. Protobuf extraction remains feature-gated in tracedecay-code-extraction; only the contract vocabulary broadened.
  • The workspace gains [workspace.package] edition = "2024"; new crates inherit it while existing crates keep their explicit edition keys until the stacked branch unifies them.

Verification

  • cargo check --workspace (default and --all-features) — pass
  • cargo nextest run over the carved crates — pass (latest: 1,261 across domain/application/store; 1,677 across the four crates incl. rusqlite-runtime with one known load-sensitive failure, below)
  • cargo fmt --check, cargo clippy --all-targets --all-features -D warnings — pass
  • Hawk closed-world visibility audit vs master baseline — zero new findings in shared code (see comments)
  • 12-agent review battery (quality, performance, reuse, thermo-nuclear audits) — ship verdicts; all fix-first findings addressed in the commits above

Known remaining issue

tracedecay-rusqlite-runtime repository::graph_publication::tests::relational::retired_cleanup_materializes_only_the_near_limit_record_admitted_to_each_page fails under CI load: the cfg(test) exact-SQL budgets (250ms/statement, 500ms/transaction) cannot absorb three ~4MiB near-limit appends on slow runners. This is delivery-branch code needing a design decision (the AuthorizedLongLease escape hatch is deliberately reserved); tracked as the one red test expected on CI.

Stack

Carve the low-layer V2 crates out of the consolidated delivery branch so
they can merge to master first, with the remaining V2 work stacked on top:

- new workspace members: api, application, host-integration, hooks,
  policy, private-fs, rusqlite-runtime, store, temporal-query,
  tool-catalog
- tracedecay-domain: adopt the V2 contract modules; protobuf node kinds
  become unconditional domain vocabulary (the lang-protobuf feature is
  gone, so code-extraction no longer forwards it)
- keep the seven master-era result structs (EditResult, MultiEditResult,
  InsertResult, AstGrepResult, MoveHint, MoveResult, CostTurn) in
  domain::code_intelligence until the stacked branch replaces their
  callers
- include the host-event/provider fixtures and the rusqlite runtime
  suite sources the crate tests compile against

Verified: cargo check --workspace (default and --all-features) and
cargo nextest for the eleven touched crates (2118 passed).
@changeset-bot

changeset-bot Bot commented Aug 19, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: fd1f2e3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 06047de72e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +592 to +596
match fs::symlink_metadata(root) {
Ok(metadata) if metadata.file_type().is_symlink() || !metadata.is_dir() => {
return Err(HookSpoolError::UnsafePath);
}
Ok(_) => return Ok(()),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject non-private existing hook spool directories

When the configured spool root already exists with group/world-write permissions or a permissive Windows ACL, this branch accepts it after checking only that it is a directory. Another local account can then delete or replace records.v1.bin, metadata, or lease files despite their per-file modes, allowing captured hook envelopes to be corrupted or disclosed. Validate the existing directory's ownership and private permissions—or tighten them—before returning success.

Useful? React with 👍 / 👎.

Cargo auto-discovers tests/storage_runtime_rusqlite_suite/main.rs as a
root-package test target, and that harness needs the stacked V2 root
crate. The rusqlite-runtime crate tests include the five module files
directly, so only the harness leaves this slice; the stacked delivery
reintroduces it with the V2 root.
@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Hawk closed-world visibility audit: this branch vs master

cargo +1.97.1 hawk check (astral-sh hawk, delivery-branch hawk.toml, production root = tracedecay --bin tracedecay --all-features), both trees green builds:

run total dead_public unnecessary_public unnecessary_restricted_visibility
master baseline 1,653 68 1,462 123
this branch 11,560 1,546 9,674 340

Every pre-existing crate reports identical counts in both runs — the slice introduces zero new findings in master's code (tracedecay 38/497/73, agent-hosts, code-extraction, lsp, migrate, runtime-core, sessions, etc. all unchanged).

The entire delta sits in the carved V2 surface, which master's binary does not consume yet by design — its production callers arrive in #421 (which adopted this same hawk audit with a clean burndown):

  • tracedecay-store 596 dead-public, tracedecay-application 433, tracedecay-domain 237 (the new V2 contract modules), plus the other seven new crates.

Integration checks on the compatibility work specifically:

  • The seven kept master-era structs in domain::code_intelligence::graph (EditResult, MultiEditResult, InsertResult, AstGrepResult, MoveHint, MoveResult, CostTurn) and the is_false helper produce zero hawk findings — all verified reachable from the shipped binary.
  • The earlier hawk run also caught tests/storage_runtime_rusqlite_suite/main.rs being auto-discovered as a root test target needing the V2 root crate; fixed in 16db1df by leaving the harness to the stacked branch.

The delivery branch lints under rustc 1.97; master CI gates with the
repo toolchain, which formats three carved files differently and raises
five clippy warnings the stacked branch never sees:

- collapse the invalid-activity check into a match guard (domain)
- simplify handoff-expiry and recovery-fence boolean forms
  (application, rusqlite-runtime)
- drop an explicit auto-deref on the reader snapshot transaction
- box the MCP tools-call future: the V2 domain contracts push it past
  the configured large-future threshold in master's dispatcher

Verified: cargo clippy --workspace --all-targets --all-features
-D warnings, cargo fmt --all --check, and nextest for domain,
application, and rusqlite-runtime (1514 passed).
@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

Preexisting hawk findings: disposition

Investigated all 1,653 master-baseline findings against the delivery branch (#421), which gates CI with cargo hawk check -D warnings and no allowlist — i.e. it must be hawk-clean.

68 dead_public findings (per-symbol cross-reference):

1,462 unnecessary_public + 123 unnecessary_restricted_visibility: burned down wholesale on #421 under its hawk CI gate.

Conclusion: every preexisting finding already has its fix up-stack. Re-deleting them here would duplicate #421's burndown and manufacture modify/delete merge conflicts, so this PR intentionally leaves master's preexisting surface untouched — hawk confirms it introduces zero new findings in shared code.

Split-crate adoption

  • On feat: consolidate TraceDecay V2 PR8-PR13 delivery #421, the root tracedecay binary depends directly on all ten carved crates plus the V2 domain (root [dependencies]), and its hawk closed-world gate proves reachability from the shipped binary.
  • On this branch, the V2 domain is adopted by master's runtime-core/root crate today (compat block: zero hawk findings); the ten crates' production callers arrive with the stacked merge.

Also landed 43260d5: fmt + five clippy fixes the repo-toolchain CI gates caught in the carved surface (delivery branch lints under rustc 1.97, so these were invisible there).

The workspace contract now lists the ten carved foundation crates as
internal members with their real dependency edges, keeps the remaining
stacked-delivery crates in the omitted set, and exempts
tracedecay-rusqlite-runtime — the one sanctioned boundary around the
bundled SQLite engine — from the workspace rusqlite ban.

The reachability resolver also learns that a module loaded through an
explicit #[path] attribute owns the directory containing the loaded
file, so its children resolve as siblings (mod-rs semantics); the
carved application crate registers test modules exactly that way.
The work, placement, multi-root, and store-locator contracts require
Path::is_absolute, which bare /... literals fail on Windows, so every
carved-crate fixture that feeds those validators now builds its root
through a platform-absolute helper. This clears the
InvalidExecutionEnvelope, InvalidTargetRoot, and registered-root
rejection cascades across the Windows CI shards.
…cking

work_manifest and the executable-binding builder return
CatalogValidationError but built operation, codec, binding, capability,
and use-case identities from caller-supplied operation names with
expect. A non-canonical name now surfaces as a typed InvalidValue
instead of a panic inside a Result-returning path.
generate_node_id hand-rolled a nibble table for the digest-to-text
encoding that canonical_text::encode_lowercase_hex already owns; the
output is byte-identical.
The store canonical-projection test includes exactly one
provider-normalization fixture; the other 36 arrive with their
consumers on the stacked delivery branch. Also note why the tools-call
dispatch arm alone is boxed.
@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

12-agent review battery: consolidated results

Quality (4), performance (2), reuse (2), and thermo-nuclear audits (4) over the authored surface. Verdicts: ship across the board once the fix-first items landed (they have).

Verified clean, no action:

  • Compat block in domain::code_intelligence::graph is byte-identical to master (docs, derives, serde attrs), all seven structs have live master callers, end-of-file placement makes feat: consolidate TraceDecay V2 PR8-PR13 delivery #421's removal a single tail-chop.
  • All five clippy fixes proven semantics-preserving (match-guard fall-through checked against the full match; boolean rewrites are exact De Morgan/total-order equivalents; auto-deref and Box::pin are behavior-free). The Box::pin matches what feat: consolidate TraceDecay V2 PR8-PR13 delivery #421's rewritten dispatcher independently does at its equivalent arm, and its cost is noise next to the per-call JSON clones and config reads already on that path.
  • All ten crate manifests byte-identical to the delivery branch; Cargo.lock movements converge on delivery's versions.
  • Keeping the structs in domain beats a runtime-core reexport: delivery deletes runtime-core/src/types.rs outright (guaranteed modify/delete conflict), the application types are the V2 wire authority (JsonSchema derives), and CostTurn has no V2 successor to point at.

Merge simulation (git merge-tree against the delivery tip): 11 conflicting paths, all mechanical — seven add/add conflicts that are exactly the 1–6-line lint fixes (take base), graph.rs and src/mcp/server.rs resolve wholesale to delivery, plus a member-list union and a lock regen. No modify/delete conflicts; the deleted harness re-adds cleanly. Advisory for the #421 owner: cherry-pick the base's fix commits onto the delivery branch first and most of the conflict surface vanishes.

Fix-first findings, now landed:

  • b728605work_catalog built identities from caller-supplied operation names with expect inside Result-returning paths; now typed InvalidValue errors.
  • ea0fa23generate_node_id hand-rolled the hex nibble table that canonical_text::encode_lowercase_hex canonically owns.
  • b728605/fd1f2e3 — 36 of 37 provider-normalization fixtures had no consumer in the base; only the one the store canonical-projection test includes is carried.
  • Body correction (see updated description): the ten crates are no longer claimed byte-identical — the disclosed lint/Windows/typed-error fixes are deliberate divergences feat: consolidate TraceDecay V2 PR8-PR13 delivery #421 adopts on restack.

Deferred to the delivery lane (byte-identical inherited code):

  • Domain's deleted deny(clippy::unwrap_used/expect_used) header has no workspace replacement; restoring it flags ~99 sites (mostly the infallible static-ID pattern) and belongs with feat: consolidate TraceDecay V2 PR8-PR13 delivery #421's lint posture.
  • #[path = "../../../tests/…"] includes couple crates to repo layout; a test-support crate is the durable shape.
  • Unbounded snapshot-path wait in reader pool-wide memory release; per-call telemetry config re-read in master's dispatcher (already fixed in feat: consolidate TraceDecay V2 PR8-PR13 delivery #421's rewrite).
  • Two carved files already exceed 1k lines (api/src/http.rs, application/src/context_scout.rs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant