SRE-989: Carry crate source hashes without implying a build - #9543
SRE-989: Carry crate source hashes without implying a build#9543TimDiekmann wants to merge 5 commits into
Conversation
`build:types`, `build:wasm` and `build:openapi` depended on `^build` so that a change in an upstream crate reaches their hash and `affectedTasks`. For a crate's mirror package `build` is a no-op today, but under `futureFlags.experimentalCargoWorkspaces` it becomes `cargo build`, and the chain would compile every upstream crate before generating types. `manifest` is a task nothing implements. Depending on `^manifest` folds the inputs of every transitive dependency into the hash and keeps `affectedTasks` walking the chain, while running nothing: a change in `error-stack`, two levels up, moves `@rust/hash-codec#build:types` from 5b22f4cb to 8145afd3 and still selects the sibling `@blockprotocol/type-system` package. The one edge that disappears from the documents is `hash-graph-api#build:openapi` waiting for `@local/status#build`. The mirror of the crate `hash-status` lists the TypeScript `@local/status` as its only dependency, so `^build` reached a `tsc --build` two tasks without a command away.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9543 +/- ##
==========================================
- Coverage 60.89% 60.89% -0.01%
==========================================
Files 1461 1461
Lines 146725 146725
Branches 6744 6744
==========================================
- Hits 89343 89342 -1
- Misses 56264 56265 +1
Partials 1118 1118 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will not alter performance
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
as_constant |
< 1 ns | < 1 ns | N/A | |
constant_equal |
< 1 ns | < 1 ns | N/A | |
constant_not_equal |
< 1 ns | < 1 ns | N/A | |
access |
< 1 ns | < 1 ns | N/A | |
runtime_equal |
< 1 ns | < 1 ns | N/A | |
runtime_not_equal |
< 1 ns | < 1 ns | N/A |
Comparing t/sre-989-carry-crate-source-hashes-without-a-build (c0decf5) with main (4854d5f)1
Footnotes
The client's codegen and the hickory binding's build depended on the crates' `build`, a task neither crate has, and carried TODO markers for it. They now depend on the crates' `manifest`, so a change in the crate still selects the package without asking it for a build.
Three files import the `Status` type from `@local/status`, which the package never declared. Its build had still run first, reached through `@local/hash-graph-client#codegen → @rust/hash-graph-api#build → @rust/hash-status#build → ^build`, a chain of tasks that do not exist. The manifest edge no longer walks it, and `tsc` found no declarations.
`import/no-extraneous-dependencies` skips `import type` unless `includeTypes` is set, so a package could import types from a package it never declared and resolve them through hoisting or a dependency's build. The rule now checks type imports everywhere, with the `devDependencies` globs `eslint-config-airbnb-base` had supplied. Three packages had such imports and now declare them: the API and the AI worker import the internal API client's types, both workers the Temporal client's, and the AI worker also imports from `@blockprotocol/core`. The API's and the AI worker's tasks now run the internal API client's build before them instead of relying on the backend utils to pull it in.
`includeTypes` on `import/no-extraneous-dependencies` also applied airbnb's split between `dependencies` and `devDependencies` to type imports, so a type used in nothing but a cast had to move into `dependencies`. The plugin is registered a second time as `import-declared`, and that instance of the rule runs with `devDependencies: true` and `includeTypes: true`: it asks whether an import is declared at all. The original rule keeps its options for value imports, and the package configurations that override it leave the second instance in place. Two imports of a package by its own name become relative imports, and the AI worker declares the `vite` its vitest config had imported behind a disable directive.
PR SummaryLow Risk Overview
Smaller follow-ons in the same diff: declare Reviewed by Cursor Bugbot for commit c0decf5. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
🟢 Approval recommended
The review found no unresolved or approval-blocking issues.
Pull request overview
Introduces no-op manifest tasks to propagate Rust source hashes without compiling upstream crates.
Changes:
- Replaces upstream build dependencies with hash-only manifest chains.
- Adds missing dependency declarations and corrects internal imports.
- Refreshes generated task graphs and lockfile metadata.
File summaries
| File | Description |
|---|---|
yarn.lock |
Updates dependency lock metadata. |
turbo.json |
Defines the recursive manifest task. |
libs/@local/hash-isomorphic-utils/package.json |
Declares the status dependency. |
libs/@local/hash-isomorphic-utils/docs/task-dependencies.json |
Refreshes generated dependency metadata. |
libs/@local/graph/store/rust/turbo.json |
Uses manifest hashes for type generation. |
libs/@local/graph/client/typescript/turbo.json |
Replaces a build edge with a manifest edge. |
libs/@local/graph/client/typescript/docs/task-dependencies.json |
Removes the obsolete executable edge. |
libs/@local/graph/authorization/rust/turbo.json |
Uses manifest hashes for type generation. |
libs/@local/graph/api/turbo.json |
Uses manifest hashes for OpenAPI generation. |
libs/@local/graph/api/docs/task-dependencies.json |
Updates the generated task graph. |
libs/@local/eslint/src/deprecated/base.ts |
Checks type-only imports for declared dependencies. |
libs/@local/effect-dns/hickory/typescript/turbo.json |
Replaces a Rust build edge with a manifest edge. |
libs/@local/codec/rust/turbo.json |
Uses manifest hashes for type generation. |
libs/@blockprotocol/type-system/typescript/src/native/time.ts |
Replaces a package self-import with a relative import. |
libs/@blockprotocol/type-system/rust/turbo.json |
Uses manifest hashes for WASM and type generation. |
libs/@blockprotocol/graph/src/internal/mutate-subgraph/edge.ts |
Replaces a package self-import with a relative import. |
apps/hash-integration-worker/package.json |
Declares the Temporal client dependency. |
apps/hash-api/package.json |
Declares the internal API client dependency. |
apps/hash-api/docs/task-dependencies.json |
Refreshes generated dependency metadata. |
apps/hash-ai-worker-ts/vitest.config.ts |
Removes an obsolete lint suppression. |
apps/hash-ai-worker-ts/package.json |
Declares previously implicit dependencies. |
apps/hash-ai-worker-ts/docs/task-dependencies.json |
Refreshes generated dependency metadata. |
Review details
- Files reviewed: 21/22 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1002 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1527 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 108 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 27 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |
🌟 What is the purpose of this PR?
The crates'
build:types,build:wasmandbuild:openapitasks depend on^buildso that a change in an upstream crate reaches their hash andaffectedTasks. For a crate's mirror packagebuildis a no-op today, but underfutureFlags.experimentalCargoWorkspacesit becomescargo build— the chain would compile every upstream crate before generating a single type.This replaces the chain with a dependency that carries the source hashes and runs nothing.
🔗 Related links
🔍 What does this change?
manifestis a root task nothing implements, chained over^manifest. Depending on it folds the inputs of every transitive dependency into a task's hash and letsaffectedTaskswalk the chain, without executing anything.build:types,build:wasmandbuild:openapidepend on^manifestinstead of^build, which resolves theTODO(SRE-989)markers SRE-988 left.hash-graph-api#build:openapino longer waits for@local/status#build. The cratehash-statusis mirrored with the TypeScript@local/statusas its only dependency, so^buildreached atsc --buildtwo tasks without a command away.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
turbo.json's have been updated to reflect thisThe
manifesttask is an empty carrier: it appears inturbo run --dryoutput and inturbo query, and has no command. A reader meeting@rust/error-stack#manifestin a dependency list has to know it exists only for its hash. The rootturbo.jsonsays so where it is declared, and the checked-in task dependencies do not list it at all, since they record what executes.🐾 Next steps
The root defaults still carry the recursive
codegen → ^buildandtest:unit → ^buildchains for the JavaScript packages. Dissolving those is the next PR, stacked on this one, and the checked-in task dependencies will show its effect edge by edge.@rust/hash-statusdepending on the TypeScript@local/statusis a leftover worth removing, in a change of its own.🛡 What tests cover this?
The checked-in task dependencies are regenerated in CI and diffed; the one changed document is part of this PR. Verified by measurement: a probe in
error-stackmoves@rust/hash-codec#build:typesfrom5b22f4cbto8145afd3and back, andaffectedTasksfor that probe still selects the sibling@blockprotocol/type-systempackage withcodegenandlint:tsc.❓ How to test this?
libs/error-stack/srcturbo run build:types --filter @rust/hash-codec --dry=jsonand confirm the task's hash differs from a run without the changeturbo query 'query { affectedTasks(base: "HEAD") { items { fullName } } }'and confirm@blockprotocol/type-system#codegenis listed