Skip to content

feat(release): allow overriding a crate's bump level from the dispatch - #2508

Draft
iunanua wants to merge 2 commits into
mainfrom
igor/versioning/release-proposal-level-overrides
Draft

feat(release): allow overriding a crate's bump level from the dispatch#2508
iunanua wants to merge 2 commits into
mainfrom
igor/versioning/release-proposal-level-overrides

Conversation

@iunanua

@iunanua iunanua commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds a level_overrides input to the release-proposal dispatch, so a wrong bump level can be corrected by re-running the workflow from the Actions tab instead of checking the proposal branch out and running cargo-release by hand.

level_overrides: libdd-capabilities=minor, libdd-ipc=major

release-version-bumps.sh gains --level-overrides CRATE=LEVEL,... and uses the given level for those crates instead of calling semver-level.sh — which is also where the run's time goes, so an overridden crate gets cheaper as well as corrected.

It also records the dispatch inputs in an HTML comment at the top of the generated PR body:

<!-- release-proposal-inputs: {"crates":"libdd-capabilities-impl, libdd-ipc","main_start_ref":"","level_overrides":"libdd-capabilities=minor","bypass_standard_checks":false} -->

so a follow-up run can reconstruct them without an operator retyping the crate list.

The second commit is unrelated to the feature and can be reviewed on its own: the cargo-release job installed its four tools with taiki-e/cache-cargo-install-action, which runs cargo install and caches the result. On the last dispatch those four steps cost 703s of the job's 3660s, all cache misses. pr-title-semver-check installs two of the same tools with taiki-e/install-action in 77s. Switched to install-action, and pinned all four — cargo-release and git-cliff were unpinned, which let the tools that choose version numbers and render changelogs drift under the workflow that cuts releases.

Motivation

Reviewing release proposal #2482 turned up libdd-capabilities proposed as patch when its http requirement had moved from ^1 to ^1.1. Fixing one crate's level meant a local checkout, a manual cargo release version, regenerating that crate's changelog, and a push — for a branch whose whole diff is Cargo.toml and CHANGELOG.md files.

Overriding the input rather than patching the branch is the point: the cascade, the changelogs and the publication order are all recomputed from the corrected level, so a level raised to major can pull further crates into the release, which hand-editing the branch would miss.

Additional Notes

  • Names are validated against the computed release set, not the crates input. The crate you most often need to correct was never named in crates — in chore(release): proposal for libdd-capabilities-impl, libdd-crashtracker, libdd-data-pipeline, li... #2482, libdd-capabilities, libdd-ipc-macros and libdd-tinybytes were all pulled in transitively. A name that is not a candidate fails with the candidate list rather than silently shipping the computed level.
  • level_overrides is carried in the inputs block so successive corrections accumulate instead of the newest one reverting the last. That is a correctness requirement, not a convenience.
  • The block is -->-proof. > is escaped to the JSON sequence \u003e, which decodes to the same character for a parser but can no longer terminate the HTML comment. main_start_ref is the one value with no charset validation upstream.
  • An override on the deferred or initial-release path warns rather than passing unnoticed, since it cannot apply there.
  • A raised major floor is still capped by semver-level.sh; this input can set major explicitly, and the row records the requested level so the PR body can show that a level was set by hand even if the cascade later raises it.

⚠️ Not yet usable on an open proposal. check-proposal-ongoing rejects a dispatch while any proposal branch exists — including the branches of the proposal being corrected — so today this works on a fresh proposal, or after cancelling the open one by hand. A follow-up branch makes the guard permit replacing a proposal it can prove is the same release, and retires it once the replacement exists. Worth knowing when judging whether this is complete.

Also not covered by tests: release-version-bumps.bats exists on a separate branch (#2507) and has no case for the override path yet.

How to test the change?

shellcheck -e SC2086 --severity=warning scripts/*.sh

The validation runs before any cargo release, so it can be exercised directly against a hand-written candidate file:

--level-overrides result
libdd-alpha ERROR: not CRATE=LEVEL
libdd-alpha=minr ERROR: must be major, minor or patch
=minor ERROR: has no crate name
nope=minor ERROR + lists the candidates
libdd-alpha=minor on a deferred crate WARNING, run continues
" a=minor , b=major " both parsed
omitted behaviour unchanged

The PR-body block was rendered through the real jq filter, including a hostile main_start_ref of ref--> <!-- evil, and round-trips back through jq with exactly one --> on the line.

🤖 Generated with Claude Code

iunanua and others added 2 commits September 10, 2026 15:33
cargo-public-api, cargo-release, git-cliff and cargo-semver-checks were installed
with taiki-e/cache-cargo-install-action, which runs `cargo install` and caches the
result; its own README recommends install-action for any tool install-action can
fetch. On the last dispatch those four steps cost 703s of the job's 3660s, all
cache misses. The sibling pr-title-semver-check installs two of the same tools
with install-action in 77s.

cargo-semver-checks and git-cliff come from install-action's own manifest.
cargo-release and cargo-public-api are not in it and resolve through the
cargo-binstall fallback, so `fallback: none` is deliberately not set.

Pin all four: cargo-release and git-cliff were unpinned, letting the tools that
choose version numbers and render changelogs drift under the workflow that cuts
releases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Correcting a wrong level on a release proposal meant checking the branch out and
running cargo-release by hand. Add a `level_overrides` input (CRATE=LEVEL,...);
release-version-bumps.sh uses the given level for those crates instead of calling
semver-level.sh, so a correction is a re-dispatch.

Overriding the input rather than patching the branch keeps the cascade, the
changelogs and the publication order consistent with the new level -- a level
corrected to major can pull further crates into the release.

Names are validated against the computed release set rather than the crates
input, so a typo, or a crate the dependency closure never pulled in, fails with
the candidate list instead of silently shipping the computed level. An override
on the deferred or initial-release path warns rather than passing unnoticed, and
the requested level is recorded in the row so the PR body can show that a level
was set by hand.

Also record the dispatch inputs in an HTML comment in the PR body, so a follow-up
run can reconstruct them without an operator retyping the crate list.
level_overrides is included so successive corrections accumulate instead of the
newest one reverting the last.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@datadog-official

datadog-official Bot commented Sep 11, 2026

Copy link
Copy Markdown

Pipelines  Tests

⚠️ Warnings

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 1 Pipeline job failed

Required checks pass | allchecks

View more details · View in GitHub Actions

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 77.76% (-0.02%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4e23227 | Docs | View more details | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 95.57 MB 95.57 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.96 MB 8.96 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 106.92 MB 106.92 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 12.10 MB 12.10 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 28.91 MB 28.91 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 96.08 KB 96.08 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 191.08 MB 191.07 MB -0% (-8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 810.01 MB 810.01 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 9.65 MB 9.65 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 96.08 KB 96.08 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 27.37 MB 27.37 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 55.31 MB 55.31 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 25.26 MB 25.26 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 97.58 KB 97.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 196.30 MB 196.27 MB --.01% (-32.00 KB) 💪
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 795.90 MB 795.90 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 7.46 MB 7.46 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 97.58 KB 97.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 29.44 MB 29.44 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 52.22 MB 52.22 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 85.61 MB 85.61 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.99 MB 9.99 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 101.52 MB 101.52 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 12.20 MB 12.20 MB 0% (0 B) 👌

@pr-commenter

pr-commenter Bot commented Sep 11, 2026

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-09-11 10:50:55

Comparing candidate commit 4e23227 in PR branch igor/versioning/release-proposal-level-overrides with baseline commit 132842f in branch main.

📊 Benchmarking dashboard

Found 15 performance improvements and 7 performance regressions! Performance is the same for 134 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:ddsketch_read/count/large_values

  • 🟥 execution_time [+54.009ns; +54.525ns] or [+6.423%; +6.485%]

scenario:otlp/e2e_json/1x1000

  • 🟥 execution_time [+368.298µs; +370.187µs] or [+9.210%; +9.257%]

scenario:otlp/encode_json/1x1000

  • 🟥 execution_time [+365.239µs; +365.878µs] or [+21.866%; +21.904%]

scenario:otlp/encode_protobuf/1x1000

  • 🟩 execution_time [-53.645µs; -52.468µs] or [-4.819%; -4.713%]

scenario:profile_add_sample2_frames_x1000

  • 🟥 execution_time [+30.414µs; +30.605µs] or [+4.285%; +4.312%]

scenario:vec_map/as_deduped_map/already_deduped/8

  • 🟩 execution_time [-0.607ns; -0.591ns] or [-4.191%; -4.075%]

scenario:vec_map/contains_key/128

  • 🟩 execution_time [-1.154µs; -1.142µs] or [-7.350%; -7.273%]
  • 🟩 throughput [+639833.123op/s; +646922.094op/s] or [+7.846%; +7.933%]

scenario:vec_map/contains_key/16

  • 🟩 execution_time [-24.425ns; -23.886ns] or [-9.582%; -9.370%]
  • 🟩 throughput [+6497261.418op/s; +6645870.975op/s] or [+10.351%; +10.588%]

scenario:vec_map/contains_key/64

  • 🟩 execution_time [-296.676ns; -292.376ns] or [-7.133%; -7.030%]
  • 🟩 throughput [+1163845.277op/s; +1181979.452op/s] or [+7.563%; +7.681%]

scenario:vec_map/contains_key/8

  • 🟩 execution_time [-3.708ns; -3.546ns] or [-4.969%; -4.752%]
  • 🟩 throughput [+5360544.214op/s; +5600440.218op/s] or [+5.001%; +5.224%]

scenario:vec_map/get_hit/16

  • 🟩 execution_time [-35.465ns; -35.213ns] or [-15.041%; -14.935%]
  • 🟩 throughput [+11923884.279op/s; +12002923.374op/s] or [+17.571%; +17.688%]

scenario:vec_map/get_hit/8

  • 🟩 execution_time [-9.674ns; -9.631ns] or [-14.885%; -14.818%]
  • 🟩 throughput [+21424054.996op/s; +21514176.379op/s] or [+17.405%; +17.478%]

scenario:vec_map/get_miss/128

  • 🟩 execution_time [-4.148ns; -3.762ns] or [-5.137%; -4.658%]

scenario:vec_map/get_miss/16

  • 🟥 execution_time [+18.085ns; +18.517ns] or [+190.468%; +195.020%]

scenario:vec_map/get_miss/64

  • 🟥 execution_time [+22.191ns; +22.514ns] or [+77.157%; +78.280%]

scenario:vec_map/get_miss/8

  • 🟥 execution_time [+11.156ns; +11.381ns] or [+148.422%; +151.423%]

Benchmark execution time: 2026-09-11 10:44:21

Comparing candidate commit 4e23227 in PR branch igor/versioning/release-proposal-level-overrides with baseline commit 132842f in branch main.

📊 Benchmarking dashboard

Found 10 performance improvements and 15 performance regressions! Performance is the same for 141 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:alloc_free/system/4096

  • 🟩 execution_time [-10.260ns; -10.053ns] or [-11.076%; -10.852%]

scenario:credit_card/is_card_number/x371413321323331

  • 🟩 execution_time [-370.007ns; -367.446ns] or [-5.424%; -5.387%]
  • 🟩 throughput [+8347902.008op/s; +8405438.282op/s] or [+5.695%; +5.734%]

scenario:credit_card/is_card_number_no_luhn/x371413321323331

  • 🟩 execution_time [-370.708ns; -367.885ns] or [-5.434%; -5.392%]
  • 🟩 throughput [+8356046.702op/s; +8420296.282op/s] or [+5.701%; +5.745%]

scenario:glob_matcher/ascii_pattern_unicode_subject/wall_time

  • 🟩 execution_time [-7.999ns; -7.913ns] or [-5.514%; -5.454%]

scenario:glob_matcher/ascii_wildcard_backtrack_match/wall_time

  • 🟥 execution_time [+3.542ns; +3.565ns] or [+9.508%; +9.571%]

scenario:glob_matcher/ascii_wildcard_heavy_backtrack/wall_time

  • 🟥 execution_time [+3.749ns; +3.792ns] or [+9.119%; +9.226%]

scenario:glob_matcher/ascii_wildcard_question_match/wall_time

  • 🟥 execution_time [+3.357ns; +3.379ns] or [+8.963%; +9.023%]

scenario:glob_matcher/ascii_wildcard_star_match/wall_time

  • 🟥 execution_time [+3.458ns; +3.476ns] or [+9.261%; +9.310%]

scenario:glob_matcher/unicode_pattern_ascii_subject/wall_time

  • 🟩 execution_time [-9.700ns; -9.599ns] or [-9.622%; -9.521%]

scenario:glob_matcher/unicode_pattern_wildcard_match/wall_time

  • 🟩 execution_time [-7.670ns; -7.464ns] or [-5.789%; -5.633%]

scenario:no_profiler/short_circuit/4096

  • 🟥 execution_time [+4.953ns; +5.121ns] or [+5.102%; +5.275%]

scenario:normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...

  • 🟥 execution_time [+19.692µs; +19.920µs] or [+10.563%; +10.685%]
  • 🟥 throughput [-517904.046op/s; -512349.093op/s] or [-9.655%; -9.551%]

scenario:normalization/normalize_name/normalize_name/bad-name

  • 🟥 execution_time [+1.058µs; +1.087µs] or [+5.999%; +6.166%]
  • 🟥 throughput [-3294801.364op/s; -3207397.611op/s] or [-5.810%; -5.656%]

scenario:normalization/normalize_name/normalize_name/good

  • 🟥 execution_time [+865.870ns; +886.135ns] or [+8.771%; +8.976%]
  • 🟥 throughput [-8347164.272op/s; -8162357.965op/s] or [-8.240%; -8.058%]

scenario:normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...

  • 🟥 execution_time [+36.697µs; +37.122µs] or [+7.364%; +7.450%]
  • 🟥 throughput [-139220.656op/s; -137589.692op/s] or [-6.937%; -6.856%]

scenario:normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters

  • 🟥 execution_time [+21.110µs; +21.215µs] or [+12.471%; +12.534%]
  • 🟥 throughput [-658177.009op/s; -654902.259op/s] or [-11.141%; -11.085%]

scenario:profiler_attached/slow_path_system/4096

  • 🟩 execution_time [-7.739ns; -7.577ns] or [-4.992%; -4.887%]

scenario:sql/obfuscate_sql_string

  • 🟩 execution_time [-13.869µs; -13.651µs] or [-4.735%; -4.661%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:datadog_sample_span/parent_not_sampled_short_circuit/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+555.735%; -555.735%]

scenario:datadog_sample_span/parent_sampled_short_circuit/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+555.735%; -555.735%]

scenario:glob_matcher/ascii_case_insensitive_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+548.397%; -552.295%]

scenario:glob_matcher/ascii_exact_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+545.262%; -550.834%]

scenario:glob_matcher/ascii_exact_miss/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+578.330%; -566.496%]

scenario:glob_matcher/ascii_wildcard_backtrack_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+551.137%; -553.576%]

scenario:glob_matcher/ascii_wildcard_heavy_backtrack/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+553.185%; -554.536%]

scenario:glob_matcher/ascii_wildcard_question_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+554.967%; -555.373%]

scenario:glob_matcher/ascii_wildcard_star_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+555.735%; -555.735%]

scenario:glob_matcher/star_short_circuit/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+555.735%; -555.735%]

Candidate

Omitted due to size.

Baseline

Omitted due to size.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant