SRE-1033: Declare task dependencies where they hold instead of at the root - #9555
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9555 +/- ##
=======================================
Coverage 65.93% 65.93%
=======================================
Files 1884 1884
Lines 198146 198146
Branches 8230 8230
=======================================
+ Hits 130643 130644 +1
Misses 65973 65973
+ Partials 1530 1529 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will degrade performance by 15.93%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | bit_matrix/dense/iter_row[64] |
137.2 ns | 166.4 ns | -17.53% |
| ❌ | bit_matrix/dense/iter_row[200] |
174.7 ns | 203.9 ns | -14.31% |
as_constant |
< 1 ns | < 1 ns | N/A | |
constant_equal |
< 1 ns | < 1 ns | N/A | |
constant_not_equal |
< 1 ns | < 1 ns | N/A | |
access |
< 1 ns | < 1 ns | N/A | |
runtime_equal |
< 1 ns | < 1 ns | N/A | |
runtime_not_equal |
< 1 ns | < 1 ns | N/A |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing t/sre-1033-declare-turbo-task-dependencies-where-they-hold-instead-of (9d76d36) with main (50327ec)
PR SummaryMedium Risk Overview The root no longer chains Per-app and per-lib Net effect: fewer phantom tasks (like building Reviewed by Cursor Bugbot for commit 9d76d36. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
🟡 Changes recommended
Package-level overrides currently discard root cache, persistence, and environment settings for multiple tasks.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Moves Turborepo dependency edges from global defaults to the packages that consume them, reducing unnecessary cross-workspace builds.
Changes:
- Replaces broad root dependencies with package-specific task edges.
- Adds explicit build, codegen, test, development, and service dependencies.
- Regenerates task-dependency snapshots.
File summaries
| File | Description |
|---|---|
turbo.json |
Redefines root task defaults. |
tests/hash-playwright/turbo.json |
Adds targeted codegen and lint dependencies. |
tests/hash-playwright/docs/task-dependencies.json |
Updates generated task graph. |
tests/hash-backend-integration/turbo.json |
Localizes codegen and integration-test edges. |
tests/hash-backend-integration/docs/task-dependencies.json |
Updates generated task graph. |
tests/graph/integration/turbo.json |
Uses manifest dependency hashing. |
tests/graph/integration/docs/task-dependencies.json |
Updates generated task graph. |
tests/graph/http/turbo.json |
Adds Graph health-check dependency. |
tests/graph/http/docs/task-dependencies.json |
Updates generated task graph. |
tests/graph/benches/turbo.json |
Adds benchmark and test service dependencies. |
tests/graph/benches/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/status/typescript/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/repo-chores/node/turbo.json |
Adds test build dependencies. |
libs/@local/petrinaut-python/turbo.json |
Adds generated-model test dependency. |
libs/@local/petrinaut-python/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/petrinaut-optimizer-client/turbo.json |
Localizes generated-client test dependencies. |
libs/@local/petrinaut-optimizer-client/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/petrinaut-arch-docs/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/hashql/eval/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/hashql/compiletest/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/hash-isomorphic-utils/turbo.json |
Moves build dependencies from codegen. |
libs/@local/hash-isomorphic-utils/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/hash-backend-utils/turbo.json |
Adds dependency builds for build and tests. |
libs/@local/harpc/client/typescript/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/graph/store/typescript/turbo.json |
Narrows codegen and build prerequisites. |
libs/@local/graph/store/typescript/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/graph/sdk/typescript/turbo.json |
Adds build prerequisites. |
libs/@local/graph/postgres-store/turbo.json |
Adds manifest hashing to integration tests. |
libs/@local/graph/postgres-store/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/graph/client/typescript/turbo.json |
Separates codegen and build dependencies. |
libs/@local/graph/authorization/typescript/turbo.json |
Localizes generated-type build dependencies. |
libs/@local/graph/authorization/typescript/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/graph/api/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/effect-dns/core/docs/task-dependencies.json |
Updates generated task graph. |
libs/@local/codec/typescript/turbo.json |
Separates Rust codegen from build hashing. |
libs/@local/advanced-types/docs/task-dependencies.json |
Updates generated task graph. |
libs/@hashintel/petrinaut/turbo.json |
Adds development and test build edges. |
libs/@hashintel/petrinaut-cli/docs/task-dependencies.json |
Updates generated task graph. |
libs/@hashintel/ds-components/turbo.json |
Moves dependency builds to consuming tasks. |
libs/@hashintel/ds-components/docs/task-dependencies.json |
Updates generated task graph. |
libs/@hashintel/brunch-agent/packages/plugin-sdcpn/turbo.json |
Adds test build dependencies. |
libs/@hashintel/brunch-agent/packages/plugin-gherkin/turbo.json |
Adds test build dependencies. |
libs/@hashintel/brunch-agent/packages/plugin-dafny/turbo.json |
Adds test build dependencies. |
libs/@hashintel/brunch-agent/packages/core/turbo.json |
Removes unnecessary test prerequisites. |
libs/@hashintel/brunch-agent/packages/binding-flue/turbo.json |
Adds test build dependencies. |
libs/@blockprotocol/type-system/typescript/turbo.json |
Localizes codegen and codec dependencies. |
libs/@blockprotocol/type-system/typescript/docs/task-dependencies.json |
Updates generated task graph. |
libs/@blockprotocol/graph/turbo.json |
Adds dependency builds. |
apps/plugin-browser/turbo.json |
Localizes codegen, build, and development edges. |
apps/plugin-browser/docs/task-dependencies.json |
Updates generated task graph. |
apps/petrinaut-website/turbo.json |
Adds development generation and build edges. |
apps/petrinaut-website/docs/task-dependencies.json |
Updates generated task graph. |
apps/petrinaut-opt/turbo.json |
Connects generated Python models to service tasks. |
apps/petrinaut-opt/docs/task-dependencies.json |
Updates generated task graph. |
apps/hash-integration-worker/turbo.json |
Adds build and Graph-service dependencies. |
apps/hash-integration-worker/docs/task-dependencies.json |
Updates generated task graph. |
apps/hash-graph/turbo.json |
Localizes service startup chains. |
apps/hash-graph/docs/task-dependencies.json |
Updates generated task graph. |
apps/hash-frontend/turbo.json |
Adds targeted generation, build, and service edges. |
apps/hash-frontend/docs/task-dependencies.json |
Updates generated task graph. |
apps/hash-api/turbo.json |
Adds aggregate build and startup dependencies. |
apps/hash-api/docs/task-dependencies.json |
Updates generated task graph. |
apps/hash-ai-worker-ts/turbo.json |
Adds build and Graph-service dependencies. |
.claude/hooks/docs/task-dependencies.json |
Updates generated task graph. |
Review details
- Files reviewed: 64/64 changed files
- Comments generated: 1
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
ec9e9d0 to
2fd4b03
Compare
… root The root `turbo.json` chained `codegen → ^build`, `build → codegen` and `test:unit → codegen, ^build` for every package, whether or not the package had a `codegen`, imported anything from `dist`, or was a Rust crate. The effect was a recursion that made every task wait for every dependency's build, reached real tasks through phantoms — a `tsc --build` of `@local/status` before generating the Graph's OpenAPI spec — and put the linter's own build ahead of unrelated builds. The names shared between the JavaScript packages and the Rust crates now carry no edges at the root; each package declares what its tasks consume. Names a single toolchain uses keep their defaults: `lint:tsc`, `lint:eslint` and `fix:eslint` depend on `^build` because type-aware linting resolves workspace imports from `dist`, and the Rust-only names depend on `^manifest`. `test:unit` and `test:integration` keep a root default of `^manifest`: a test reads its dependencies' sources through its toolchain, so their inputs belong in its hash whether or not a build runs first. Measured on `@rust/error-stack#test:unit`, which no longer depends on `^build`: a change in `error-stack-macros` still moves its hash, and back. Packages that import from `dist` add `^build` themselves. Edges the recursion supplied by accident and that are true are now explicit: `petrinaut-opt`'s codegen reads the models `petrinaut-python` generates, the frontend's codegen loads its GraphQL config and both Panda presets from `dist`, and `dev` and `start` wait for what they serve. Explicit edges the recursion would have covered stay where `^build` does not reach them, such as the brunch plugins that are not dependencies of the agent. `hash-api` gains a `build` that runs nothing of its own and gathers its codegen and its dependencies' builds, which is what its image and the packages that import it need from it; the recursion had never included `@local/hash-backend-utils#build` there. The checked-in task dependencies change in 30 packages. Every `codegen` and `test:unit` sheds predecessors — 76 and 34 executed tasks, none of them read by the task — and the lint tasks trade 46 predecessors reached through phantom chains for 9 direct ones. The executed set of the four service images is unchanged apart from the `hash-backend-utils` build.
2fd4b03 to
d0b9805
Compare
`@rust/hash-graph-atlas` landed on main with a document generated under the root recursion: its `start` and `test:*` tasks reached `@local/status#build` through `@rust/hash-status#build`, a task that does not exist.
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1002 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1527 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 108 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 27 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |
🌟 What is the purpose of this PR?
The root
turbo.jsonchainscodegen → ^build,build → codegenandtest:unit → codegen, ^buildfor every package — whether or not the package has acodegen, imports anything fromdist, or is a Rust crate. Every task therefore waits for every dependency's build, phantoms reach real tasks two hops away (atsc --buildof@local/statusbefore generating the Graph's OpenAPI spec), and the linter's own build sits ahead of unrelated builds.This declares each edge where it holds. The checked-in task dependencies show the effect package by package.
🔗 Related links
🔍 What does this change?
codegen,build,dev,start:*,test:*) carry no edges at the root. Each package declares what its tasks consume: acodegenthat loads its config from a dependency'sdistnames that build, abuildthat compiles againstdistdepends on^build, astartnames the service it waits for.lint:tsc,lint:eslintandfix:eslintdepend on^build, since type-aware linting resolves workspace imports fromdist.lint:clippy, the benches, codspeed and miri depend on^manifest.test:unitandtest:integrationkeep a root default of^manifest: a test reads its dependencies' sources through its toolchain, so their inputs belong in its hash whether or not a build runs first. Packages that import fromdistadd^buildthemselves.petrinaut-opt's codegen reads the modelspetrinaut-pythongenerates; the frontend's codegen loads its GraphQL config and both Panda presets fromdist;devandstartwait for what they serve. Explicit edges stay where^builddoes not reach them, such as the brunch plugins that are not dependencies of the agent.hash-apideclares abuildthat runs nothing of its own and gathers its codegen and its dependencies' builds. Its image runsturbo build --filter '@apps/hash-api', and the integration tests import the package, so both now also get@local/hash-backend-utils#build, which the recursion had never included.hash-graphkeeps itsstart:*chain in its ownturbo.json; the health-check edges of the Rust test packages move from the root default intotests/graph/benchesandtests/graph/http.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
turbo.json's have been updated to reflect thisThe
oxlintpackages inherit thelint:eslintdefault, including@local/eslint#build, which they do not use. SRE-1018 decides how those tasks are named; this PR leaves them as they are.🐾 Next steps
@rust/hash-statusdepends on the TypeScript@local/status, the leftover that put atsc --buildin front of Rust tasks. It deserves a change of its own.🛡 What tests cover this?
The checked-in task dependencies are regenerated in CI and diffed; the 30 changed documents are part of this PR. Verified by measurement against the SRE-1034 tip: the set of executable tasks is unchanged; every predecessor a task no longer executes is one it never read, and every predecessor it gains is one it imports or waits for; the closure over the documents equals turbo's execution plan for thirteen sampled tasks across apps, libraries, the integration tests and the Rust test packages; the four service images plan the same tasks as before except for the added
hash-backend-utilsbuild; a change inerror-stack-macrosstill moves@rust/error-stack#test:unit's hash through^manifest; a change in@local/hash-isomorphic-utilsstill selectscodegen,test:unitandlint:tscofhash-apiandhash-frontendthrough the named edges.❓ How to test this?
turbo run test:unit --filter @local/hash-graph-sdk --dry=jsonand compare the planned tasks withlibs/@local/graph/sdk/typescript/docs/task-dependencies.jsonlibs/@local/hash-isomorphic-utils/srcand runturbo query 'query { affectedTasks(base: "HEAD") { items { fullName } } }';@apps/hash-api#codegenis listedturbo run codegen --filter @apps/hash-frontend --dry=json; the plan contains the three named builds and nothing elseturbo build --filter @apps/hash-api --dry=json; the plan ishash-api's codegen plus its dependencies' builds