feat(data-pipeline)!: generate agentless trace stats by BridgeAR · Pull Request #2488 · DataDog/libdatadog

BridgeAR · 2026-09-08T23:50:57Z

Agentless v0.4 export now generates trace stats from the decoded trace payload. Traces and stats share one exporter and one WASM artifact.

github-actions · 2026-09-08T23:53:04Z

📚 Documentation Check Results

⚠️ 3149 documentation warning(s) found

📦 `libdd-data-pipeline-core` - 921 warning(s)

📦 `libdd-data-pipeline` - 1304 warning(s)

📦 `libdd-trace-stats` - 924 warning(s)

Updated: 2026-09-11 14:06:11 UTC | Commit: 2f93588 | missing-docs job results

github-actions · 2026-09-08T23:54:40Z

🔒 Cargo Deny Results

✅ No issues found!

📦 `libdd-data-pipeline-core` - ✅ No issues

📦 `libdd-data-pipeline` - ✅ No issues

📦 `libdd-trace-stats` - ✅ No issues

Updated: 2026-09-11 14:07:28 UTC | Commit: 2f93588 | dependency-check job results

datadog-prod-us1-5 · 2026-09-09T00:03:37Z

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 93.62%
• Overall Coverage: 77.95% (+0.17%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 0f77aad | Docs | View more details | Give us feedback!}

dd-octo-sts · 2026-09-09T03:05:26Z

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	96.01 MB	95.93 MB	--.08% (-87.26 KB) 💪
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	9.02 MB	9.02 MB	0% (0 B) 👌

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	107.42 MB	107.34 MB	--.07% (-84.21 KB) 💪
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	12.19 MB	12.18 MB	--.01% (-2.27 KB) 💪

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	29.04 MB	29.02 MB	--.05% (-17.50 KB) 💪
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	96.08 KB	96.08 KB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	191.77 MB	191.55 MB	--.11% (-232.00 KB) 💪
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	818.19 MB	818.73 MB	+.06% (+557.40 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	9.69 MB	9.69 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	96.08 KB	96.08 KB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	27.51 MB	27.48 MB	--.08% (-24.00 KB) 💪
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	55.57 MB	55.55 MB	--.05% (-28.90 KB) 💪

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	25.41 MB	25.40 MB	--.05% (-13.50 KB) 💪
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	97.58 KB	97.58 KB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	197.08 MB	196.83 MB	--.13% (-264.00 KB) 💪
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	800.48 MB	800.96 MB	+.05% (+486.10 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	7.51 MB	7.51 MB	--.01% (-1.00 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	97.58 KB	97.58 KB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	29.63 MB	29.61 MB	--.07% (-24.00 KB) 💪
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	52.55 MB	52.52 MB	--.05% (-28.87 KB) 💪

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	85.97 MB	85.90 MB	--.08% (-75.44 KB) 💪
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	10.03 MB	10.03 MB	0% (0 B) 👌

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	101.85 MB	101.78 MB	--.07% (-75.31 KB) 💪
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	12.25 MB	12.25 MB	--.04% (-5.66 KB) 💪

pr-commenter · 2026-09-09T03:20:41Z

Benchmarks

Comparison

Benchmark execution time: 2026-09-11 14:47:58

Comparing candidate commit 0f77aad in PR branch BridgeAR/2026-09-08-native-agentless-stats with baseline commit 132842f in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 176 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Benchmark execution time: 2026-09-11 14:42:08

Comparing candidate commit 0f77aad in PR branch BridgeAR/2026-09-08-native-agentless-stats with baseline commit 132842f in branch main.

📊 Benchmarking dashboard

Found 3 performance improvements and 0 performance regressions! Performance is the same for 163 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:no_profiler/short_circuit/4096

🟩 execution_time [-5.765ns; -5.596ns] or [-5.745%; -5.577%]

scenario:trace_buffer/2_senders/no_delay

🟩 execution_time [-153.802µs; -137.321µs] or [-8.684%; -7.754%]
🟩 throughput [+86276.196op/s; +96906.206op/s] or [+8.485%; +9.531%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:datadog_sample_span/parent_not_sampled_short_circuit/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+544.800%; -550.619%]

scenario:datadog_sample_span/parent_sampled_short_circuit/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+555.157%; -555.463%]

scenario:glob_matcher/ascii_case_insensitive_match/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+555.167%; -555.468%]

scenario:glob_matcher/ascii_exact_match/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+554.607%; -555.204%]

scenario:glob_matcher/ascii_exact_miss/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+553.626%; -554.743%]

scenario:glob_matcher/ascii_wildcard_backtrack_match/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+556.511%; -556.100%]

scenario:glob_matcher/ascii_wildcard_heavy_backtrack/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+558.326%; -556.956%]

scenario:glob_matcher/ascii_wildcard_question_match/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+554.200%; -555.013%]

scenario:glob_matcher/ascii_wildcard_star_match/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+558.837%; -557.197%]

scenario:glob_matcher/star_short_circuit/allocated_bytes

unstable execution_time [-0.000ns; +0.000ns] or [+551.134%; -553.575%]

Candidate

Omitted due to size.

Baseline

Omitted due to size.

bengl

Overall looks fine, but I'd prefer if someone from @DataDog/apm-common-components-core approved.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 63e7f235b3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Stats aggregation runs before priority-zero chunks are removed, so a payload can produce stats without a trace request.

Using a compile-time destination strategy keeps the agent branch out of agentless WASM builds while both exporters share sequencing and bucket splitting.

1. Preserve the existing Agent destination shape to avoid breaking callers. 2. Honor precomputed-stats metadata so agentless traces do not request duplicate stats. 3. Reject zero bucket sizes before timestamp alignment can divide by zero.

paullegranddc

Approved with some nits. Overall I have a gut feeling that it shouldn't be this hard to do what you are doing.

Also, you're basically rebuilding a small TracerExporter which target just agentless. Instead of maintaining the same pipeline twice we should see if we can refactor the TraceExporter even if we have to feature gate some code for package size

paullegranddc · 2026-09-11T14:30:28Z

@@ -674,7 +674,8 @@ impl StatsBucket {
                    return;
                }
                // Within the max entry limit, admit key as a new distinct entry.
-                e.insert_with_key(OwnedAggregationKey::from(&key), GroupedStats::default())
+                e.insert(OwnedAggregationKey::from(&key), GroupedStats::default())
+                    .1


I don't think this is necessary

paullegranddc · 2026-09-11T16:43:57Z

+trait StatsPayloadDestination<Cap: HttpClientCapability + SleepCapability> {
+    async fn send_payload(
+        &self,
+        capabilities: &Cap,
+        meta: &StatsMetadata,
+        sequence_id: &AtomicU64,
+        buckets: Vec<pb::ClientStatsBucket>,
+        obfuscated: bool,
+        #[cfg(feature = "stats-obfuscation")] supported_obfuscation_version: &'static str,
+    ) -> anyhow::Result<()>;
+}


I don't really get why we need to separate just agentless vs agent/agentless with a trait. Does it allow reducing package size?

github-actions Bot added the data-pipeline label Sep 8, 2026

BridgeAR changed the title ~~feat(data-pipeline): generate agentless trace stats~~ feat(data-pipeline)!: generate agentless trace stats Sep 9, 2026

BridgeAR marked this pull request as ready for review September 9, 2026 20:11

BridgeAR requested review from a team as code owners September 9, 2026 20:11

bengl approved these changes Sep 9, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread libdd-data-pipeline-core/src/agentless/v04.rs Outdated

BridgeAR force-pushed the BridgeAR/2026-09-08-native-agentless-stats branch from 63e7f23 to 7691ae8 Compare September 9, 2026 21:02

BridgeAR added 3 commits September 11, 2026 15:53

feat(data-pipeline): generate agentless stats from v0.4 traces

a95edc1

Stats aggregation runs before priority-zero chunks are removed, so a payload can produce stats without a trace request.

refactor(trace-stats): share stats sending

e698182

Using a compile-time destination strategy keeps the agent branch out of agentless WASM builds while both exporters share sequencing and bucket splitting.

BridgeAR force-pushed the BridgeAR/2026-09-08-native-agentless-stats branch from 7691ae8 to 0f77aad Compare September 11, 2026 14:03

paullegranddc reviewed Sep 11, 2026

View reviewed changes

paullegranddc approved these changes Sep 11, 2026

View reviewed changes

Uh oh!

Conversation

BridgeAR commented Sep 8, 2026

Uh oh!

github-actions Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📚 Documentation Check Results

📦 libdd-data-pipeline-core - 921 warning(s)

📦 libdd-data-pipeline - 1304 warning(s)

📦 libdd-trace-stats - 924 warning(s)

Uh oh!

github-actions Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Cargo Deny Results

📦 libdd-data-pipeline-core - ✅ No issues

📦 libdd-data-pipeline - ✅ No issues

📦 libdd-trace-stats - ✅ No issues

Uh oh!

datadog-prod-us1-5 Bot commented Sep 9, 2026 • edited by datadog-prod-us1-3 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact Size Benchmark Report

Uh oh!

pr-commenter Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Comparison

Explanation

More details about the CI and significant changes

Explanation

More details about the CI and significant changes

scenario:no_profiler/short_circuit/4096

scenario:trace_buffer/2_senders/no_delay

Unstable benchmarks

scenario:datadog_sample_span/parent_not_sampled_short_circuit/allocated_bytes

scenario:datadog_sample_span/parent_sampled_short_circuit/allocated_bytes

scenario:glob_matcher/ascii_case_insensitive_match/allocated_bytes

scenario:glob_matcher/ascii_exact_match/allocated_bytes

scenario:glob_matcher/ascii_exact_miss/allocated_bytes

scenario:glob_matcher/ascii_wildcard_backtrack_match/allocated_bytes

scenario:glob_matcher/ascii_wildcard_heavy_backtrack/allocated_bytes

scenario:glob_matcher/ascii_wildcard_question_match/allocated_bytes

scenario:glob_matcher/ascii_wildcard_star_match/allocated_bytes

scenario:glob_matcher/star_short_circuit/allocated_bytes

Candidate

Baseline

Uh oh!

bengl left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

paullegranddc left a comment

Choose a reason for hiding this comment

Uh oh!

paullegranddc Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

paullegranddc Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Sep 8, 2026 •

edited

Loading

📦 `libdd-data-pipeline-core` - 921 warning(s)

📦 `libdd-data-pipeline` - 1304 warning(s)

📦 `libdd-trace-stats` - 924 warning(s)

github-actions Bot commented Sep 8, 2026 •

edited

Loading

📦 `libdd-data-pipeline-core` - ✅ No issues

📦 `libdd-data-pipeline` - ✅ No issues

📦 `libdd-trace-stats` - ✅ No issues

datadog-prod-us1-5 Bot commented Sep 9, 2026 •

edited by datadog-prod-us1-3 Bot

Loading

dd-octo-sts Bot commented Sep 9, 2026 •

edited

Loading

pr-commenter Bot commented Sep 9, 2026 •

edited

Loading