Skip to content

fix(release): report the floors a deferred crate inherited - #2515

Draft
iunanua wants to merge 1 commit into
igor/versioning/semver-level-dep-and-featfrom
igor/versioning/release-path-manifest-commits
Draft

fix(release): report the floors a deferred crate inherited#2515
iunanua wants to merge 1 commit into
igor/versioning/semver-level-dep-and-featfrom
igor/versioning/release-path-manifest-commits

Conversation

@iunanua

@iunanua iunanua commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Stacked on #2506, which it needs for semver-level.sh --list-affected.

A dependency floor raised only in the root manifest's [workspace.dependencies] leaves every crate inheriting the entry with no commits of its own, so release-version-bumps.sh defers it — level none, "pending_release": "true" — and the release proposal reads as though nothing happened to it. semver-level.sh, and after #2506 the PR-title gate too, call that same change a minor.

Deferring is the right outcome, and this PR does not change it. The crate's code is unchanged; the requirement its published version states is still true of that code; and requirements are minimums, so a consumer combining it with a freshly published sibling that asks for the raised floor resolves the newer dependency anyway. Deferring moves neither the version nor the tag, so the raise stays in range and the crate's next release is still scored a minor for it — late, not wrong. The case where a crate's code genuinely needs the raised version always arrives with a change to a file of its own, which the existing path search already finds. Releasing every inheritor of a shared entry instead would mean 12 crates published for one bytes bump, none of whose code needed it, for teams that pin libdatadog versions.

What was missing is the saying so:

  • commits-since-release.sh reports, per crate, the commits that moved what a consumer of it resolves while touching no file of its own, as manifest_moves beside an untouched commits.
  • release-version-bumps.sh prints them where it defers, and carries them on the deferred row.
No commits since last release for libdd-capabilities; deferring to the libdd-* major-bump check
  Its resolved requirements moved even so, which its next release will carry as a minor:
    - 2afbfe68 build(deps): bump bytes floor to 1.12

An operator who wants the bump now can pull the crate in by hand; otherwise its next release carries it for the same reason.

Motivation

#2506 makes CI score a root-only floor raise as a minor for every inheriting crate. Without this, the release proposal for those same crates says level none, and a reviewer comparing the two has nothing to reconcile them with.

Additional Notes

  • Only commits that moved that crate's resolved requirements are listed. Another crate's entry, a new member, a dev-dependency or a comment in the root manifest are not its business. The answers come from semver-level.sh --list-affected between each root-manifest commit and its parent, so what counts as a manifest fact has one definition shared with the passes that score it.
  • Cached per commit, so the cost follows the length of the release range and not the range times the number of crates: ~16s for a range holding 62 root-manifest commits (libdd-alloc, months since its last release). A commit whose parent the clone lacks is reported rather than passed over, since a silent skip is the failure this reporting exists to end.
  • manifest_moves never overlaps commits. A commit touching both the crate and the root manifest is listed once, among its own commits.
  • The root manifest is the only path searched. It is the one file outside a crate's directory that can move a crate's resolved requirements: sibling crates are not declared in [workspace.dependencies], so a dependency on one carries its version in the depending crate's own manifest, which the path search already sees.
  • resolve_baseline now takes a full commit SHA already in the clone as it stands, which is what lets the above ask about a commit and its parent: a bare SHA is not a ref name the remote serves, and origin/<sha> is not a revision. A branch or tag still resolves through the fetch, so a stale local copy cannot stand in for the remote's.
  • The exact requirement delta is not in the output — the commit that moved it is. --list-affected reports crate names; pairing old and new requirements per crate would need either a per-crate semver-level.sh run (rustdoc passes, minutes) or a new detail mode. The commit subject carries it in practice for dependency bumps; happy to add the mode if the delta is wanted literally.

Verified in a throwaway clone against a fixture tag:

case commits manifest_moves
root-only bytes floor raise [], still deferred the bytes commit
clap raise, an entry the crate does not inherit [] absent
comment-only edit to the root manifest [] absent
one commit touching both the crate and the root manifest listed once not duplicated
libdd-alloc, real history, inherits no moved entry its 4 real commits, unchanged []

shellcheck reports nothing in any of the three scripts that was not already there.

🤖 Generated with Claude Code

A dependency floor raised only in the root manifest's [workspace.dependencies]
leaves every crate inheriting the entry with no commits of its own, so
release-version-bumps.sh defers it -- level "none", "pending_release": "true"
-- and the proposal reads as though nothing happened to it. semver-level.sh,
and now the PR-title gate, call the same change a minor.

Deferring is right: the crate's code is unchanged, the requirement its
published version states is still true of that code, and requirements are
minimums, so a consumer combining it with a freshly published sibling that
asks for the raised floor resolves the newer dependency anyway. Releasing
every inheritor of a shared entry would be churn for crates that needed
nothing. The case where a crate's code does need the raised version always
comes with a change to a file of its own, which the existing path search
already finds.

What was missing is the saying so. commits-since-release.sh now reports, per
crate, the commits that moved what a consumer of it resolves while touching
no file of its own, as `manifest_moves` beside the untouched `commits`, and
release-version-bumps.sh prints them where it defers and carries them on the
row. An operator who wants the bump now can pull the crate in by hand;
otherwise its next release is scored a minor for the same reason, since
deferring moves neither its version nor its tag.

Only commits that moved that crate's own resolved requirements are listed --
another crate's entry, a new member, a dev-dependency or a comment in the
root manifest are not its business -- as semver-level.sh --list-affected
reports them, cached per commit so the cost follows the release range and not
the range times the number of crates.

resolve_baseline takes a full commit SHA already in the clone as it stands,
which is what lets the above ask about a commit and its parent: a bare SHA is
not a ref name the remote serves and `origin/<sha>` is not a revision. A
branch or tag still resolves through the fetch, so a stale local copy cannot
stand in for the remote's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Sep 11, 2026

Copy link
Copy Markdown

Pipelines

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 77.60% (-0.14%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 18c59d8 | Docs | View more details | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.96 MB 8.96 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 95.57 MB 95.57 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 12.10 MB 12.10 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 106.92 MB 106.92 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 28.91 MB 28.91 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 96.08 KB 96.08 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 191.08 MB 191.10 MB +.01% (+24.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 810.01 MB 810.01 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 9.65 MB 9.65 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 96.08 KB 96.08 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 27.37 MB 27.37 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 55.31 MB 55.31 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 25.26 MB 25.26 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 97.58 KB 97.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 196.30 MB 196.33 MB +.01% (+24.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 795.90 MB 795.90 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 7.46 MB 7.46 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 97.58 KB 97.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 29.44 MB 29.44 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 52.22 MB 52.22 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 85.61 MB 85.61 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.99 MB 9.99 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 101.52 MB 101.52 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 12.20 MB 12.20 MB 0% (0 B) 👌

@pr-commenter

pr-commenter Bot commented Sep 11, 2026

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-09-11 15:45:46

Comparing candidate commit 18c59d8 in PR branch igor/versioning/release-path-manifest-commits with baseline commit c49fb4f in branch igor/versioning/semver-level-dep-and-feat.

📊 Benchmarking dashboard

Found 0 performance improvements and 3 performance regressions! Performance is the same for 153 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:ddsketch_encode/encode_to_vec/clustered_near_zero

  • 🟥 execution_time [+61.878ns; +67.660ns] or [+8.850%; +9.677%]

scenario:ddsketch_encode/encode_to_vec/large_values

  • 🟥 execution_time [+61.398ns; +66.473ns] or [+7.283%; +7.885%]

scenario:ddsketch_encode/encode_to_vec/mixed

  • 🟥 execution_time [+67.383ns; +73.231ns] or [+5.506%; +5.984%]

Benchmark execution time: 2026-09-11 15:47:49

Comparing candidate commit 18c59d8 in PR branch igor/versioning/release-path-manifest-commits with baseline commit c49fb4f in branch igor/versioning/semver-level-dep-and-feat.

📊 Benchmarking dashboard

Found 6 performance improvements and 3 performance regressions! Performance is the same for 156 metrics, 11 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:datadog_sample_span/unicode_uppercase_service_rule/wall_time

  • 🟥 execution_time [+32.390ns; +32.603ns] or [+9.349%; +9.411%]

scenario:glob_matcher/unicode_exact_match/wall_time

  • 🟩 execution_time [-4.502ns; -4.444ns] or [-5.759%; -5.685%]

scenario:glob_matcher/unicode_pattern_wildcard_match/wall_time

  • 🟩 execution_time [-5.416ns; -5.287ns] or [-4.100%; -4.002%]

scenario:profiler_attached/fast_path_system/4096

  • 🟩 execution_time [-8.649ns; -8.486ns] or [-8.161%; -8.007%]

scenario:profiler_attached/slow_path_system/4096

  • 🟩 execution_time [-7.862ns; -7.703ns] or [-5.009%; -4.907%]

scenario:trace_buffer/2_senders/no_delay

  • 🟥 execution_time [+107.445µs; +123.099µs] or [+6.486%; +7.431%]
  • 🟥 throughput [-75983.084op/s; -66171.496op/s] or [-6.988%; -6.085%]

scenario:trace_buffer/4_senders/no_delay

  • 🟩 execution_time [-234.350µs; -220.640µs] or [-6.190%; -5.827%]
  • 🟩 throughput [+58976.280op/s; +62626.999op/s] or [+6.202%; +6.586%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:datadog_sample_span/parent_not_sampled_short_circuit/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+556.700%; -556.189%]

scenario:datadog_sample_span/parent_sampled_short_circuit/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+556.701%; -556.190%]

scenario:flagevaluation_evp/coalescer/typical/100flags_50users_10fields

  • unstable execution_time [-12.940µs; +7.471µs] or [-6.840%; +3.949%]

scenario:glob_matcher/ascii_case_insensitive_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+556.847%; -556.258%]

scenario:glob_matcher/ascii_exact_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+555.735%; -555.735%]

scenario:glob_matcher/ascii_exact_miss/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+558.230%; -556.911%]

scenario:glob_matcher/ascii_wildcard_backtrack_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+554.967%; -555.373%]

scenario:glob_matcher/ascii_wildcard_heavy_backtrack/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+553.185%; -554.536%]

scenario:glob_matcher/ascii_wildcard_question_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+555.735%; -555.735%]

scenario:glob_matcher/ascii_wildcard_star_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+555.735%; -555.735%]

scenario:glob_matcher/star_short_circuit/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+561.028%; -558.234%]

Candidate

Omitted due to size.

Baseline

Omitted due to size.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant