ci: run benchmarks only for the top of a PR stack - #822
Conversation
Stacked PRs chain head -> base, so a branch with an open PR using it as base is below another PR in the stack and gets no Benchmarking-Platform pipeline. The check runs in generate-benchmarks-child-pipeline, which emits either the real child pipeline (BP bridge + version-inputs guard) or a noop skip child; run-benchmarks triggers it with strategy: depend. - check-stack-top.sh: GitHub PR API lookup, fail-open on any error, token passed via 0600 header file (never argv), HTTP status surfaced - CANDIDATE_VERSION/BASELINE_VERSION redeclared in run-benchmarks' variables block so they reach the child as trigger-job variables; child guard fails fast on empty versions - shared .benchmarks-skip-rules anchor; deploy-artifact need optional (release branches); PARENT_COMMIT_BRANCH falls back to CI_COMMIT_REF_NAME - generate job auto-runs in web pipelines (run-benchmarks stays manual) - unit tests for the decision script wired into shell-unit-tests
CI Test ResultsRun: #36137762283 | Commit:
Status Overview
Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled Summary: Total: 32 | Passed: 32 | Failed: 0 Updated: 2026-09-25 13:13:49 UTC |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a8548a0e83
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![datadog-datadog-us1-prod[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
The most critical issue is that moving benchmarks-trigger into a child pipeline breaks the report downloader, which searches only the parent pipeline and silently leaves GitHub Pages stale. Separately, the Bash :- fallback syntax used in a GitLab trigger variable is not evaluated by GitLab, causing incorrect branch values to be passed downstream.
🤖 Bits Code Review · Commit a8548a0 · @DataDog review to ask questions
|
Fixed in c3576d3 (addressing general review comment). |
1 similar comment
|
Fixed in c3576d3 (addressing general review comment). |
1 participant
What does this PR do?:
Runs the Benchmarking-Platform pipeline only for the top branch of a PR stack instead of for every pushed branch. A branch is considered mid-stack when an open PR uses it as base branch (stacked PRs chain head -> base);
mainalways runs. The decision lives in.gitlab/benchmarks/check-stack-top.shand gates the existing benchmarks child pipeline via the same generated-YAML pattern the reliability pipeline already uses.Motivation:
A stack of N PRs currently triggers N identical BP benchmark pipelines per push. Only the top of the stack needs benchmark coverage.
Additional Notes:
CANDIDATE_VERSION/BASELINE_VERSIONare redeclared inrun-benchmarksvariables:(trigger-job variables, documented forwarding path) and the child pipeline has a fail-fast guard, so BP never consumes empty versions.CANCELLEDrule so a manual retry cannot bypass parent-level gating.shell-unit-tests.How to test the change?:
bash .gitlab/scripts/tests/test_check_stack_top.sh(also runs in CI viashell-unit-tests).jb/rc-7-tooling(PR Reference-chains design and architecture docs #802 stacked on it) -> skip;jb/rc-8-docs(stack top) -> run;main-> run.For Datadog employees:
dd:platform-security-reviewskill, or file a request via the PSEC review form).bewairealso runs automatically on every PR.