Skip to content

C++ unit tests for reference-chain tracking - #799

Draft
jbachorik wants to merge 19 commits into
jb/rc-4-profiler-wiringfrom
jb/rc-5-gtest
Draft

jbachorik wants to merge 19 commits into
jb/rc-4-profiler-wiringfrom
jb/rc-5-gtest

Conversation

@jbachorik

@jbachorik jbachorik commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

What does this PR do?:
Adds the C++ unit tests for reference chains (622 cases): frontier-table behavior, BFS expansion and cursor advancement, chain resolution/improvement (including the cycle guard), leak-candidate selection and the heap-floor OOM projection, referencechains arguments parsing, stale-leaf handling, and JFR round-tripping of chain events. Also extends the existing liveness-tracker and line-number-table tests.

Motivation:
Test layer for PROF-15341, stacked on the engine it verifies.

Additional Notes:
Stacked on #798. All gtest binaries pass (2 skipped by design).

How to test the change?:
./gradlew :ddprof-lib:gtestDebug — the tests in this PR are exactly the ones it adds/extends.

For Datadog employees:

  • This PR doesn't touch any of that.
  • JIRA: PROF-15341

@dd-octo-sts

dd-octo-sts Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervmlun5p
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 18.1.3 (1ubuntu1)
Date:Thu Sep 17 20:07:36 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs4
C++ move semantics
Use-after-move1
Logic error
Dereference of null pointer1
Result of operation is garbage or undefined1
Unused code
Dead increment1

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Unused codeDead incrementreferenceChains.cppcollectStaticFieldAnchorsForRotation36771
Logic errorDereference of null pointerfaultInjection.cppcrashNow242
Logic errorResult of operation is garbage or undefinedlivenessTracker.cppsecondsToOOM138324
C++ move semanticsUse-after-movereferenceChains.cppbuildCanaryChainEvent608368

@datadog-datadog-prod-us1

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #35750635785 | Commit: d2ab245 | Duration: 17m 24s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-09-22 16:16:40 UTC

@dd-octo-sts

dd-octo-sts Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 cb329700

* Model crashNow's intentional crash as a trap under scan-build

The null-pointer store is the deliberate never-returning crash, but
clang scan-build reports a null dereference. Under __clang_analyzer__,
replace it with __builtin_trap() - the analyzer understands the trap as
a halt and explores no false path past it; the real build keeps the
store.

* Add JFR event types for reference chains

Introduces the ReferenceChainEvent/ReferenceChainAbandonedEvent payloads
(event.h), their JFR metadata (jfrMetadata.*), and the FlightRecorder
emission paths that serialize chain events into JFR recording buffers,
including the constant-pool handling for per-hop edge labels. Emission
is pull-style: profiler.cpp snapshots events and hands them to
FlightRecorder; this layer does not depend on the tracker itself.

* Keep chain events inside the recording buffer; keep truncated labels

Review findings on the JFR plumbing layer:

- MAX_REFERENCE_CHAIN_EVENT_HOPS was a fixed 4096, permitting a ~438 KB
  worst-case event (near-limit edge labels) against a ~61 KB recording
  buffer - the reservation flushed first but the margin underflowed, so
  the write ran past the buffer (debug assert, release corruption). The
  cap is now derived from RECORDING_BUFFER_LIMIT minus the event's fixed
  fields, divided by the per-hop worst case, so a full-cap event always
  fits.
- Truncation dropped ALL edge labels: the label count was gated on
  _edges.size() == emitted_size, which only holds for untruncated
  chains. Labels align with the chain's leaf-first element order, so
  truncation now emits the first emitted_size labels and loses only the
  root-side ones.
- ObjectLivenessEvent::leak_tag is default-initialized to 0 so any
  construction path that forgets to set it serializes a defined
  untagged value (flush_table() overwrites it from the entry, which
  track() zeroes at insert).

Moves the JFR round-trip and arguments parsing unit tests into this
layer (they test exactly this code), rewrites the round-trip test to
construct events directly instead of through the tracker, and adds
byte-level boundary tests: oversize-chain truncation with label
preservation, the size-prefix invariant, and the default leak tag.

* Drop transient and stale line-number references from comments

Uncommitted plan documents, rotted .cpp:NNN line references, and a
nonexistent j9WallClock.cpp path replaced with symbol references that
stay valid as the code moves.

* Address review: merge chain hops, trim comments, drop doc/jira refs

- ReferenceChainEvent carries one vector of ReferenceChainHop
  (klass id + retention-edge label) instead of two parallel vectors
- Compress the sub-option floor/ceiling rationale and the provisional
  default constant comments to one concise statement each
- Drop design-doc and Jira references from code comments; revert the
  unrelated LineNumberTable comment rewrite

* Make comments layer-local: no references to later stack layers

The event/argument comments named collector classes, methods and files
that do not exist at this layer of the stack; describe the contracts
without those forward references instead.

* Drop a later-layer class name from the arguments test comment

* Clamp reference-chain edge labels to the reserved cap when writing

* Reuse reservation constants and drop edge_labels stack array in recordReferenceChain
ReferenceChainTracker: per-klass class tags, the frontier table of
retained references, the BFS expansion thread, chain resolution into
per-sample ReferenceChainEvent payloads, and leak-tag correlation.
LivenessTracker: the per-klass population table, heap-floor ring with
time-to-OOM projection, and leak-candidate selection feeding the
tracker's search gate. Adds the referencechains Arguments block, the
string-dictionary generation counter used to invalidate the class-tag
cache, and the container-memory/os queries the projection needs.
hopLabelClassFor() was the only GetObjectsWithTags() call site that did
not Deallocate() the returned object/tag arrays - a per-cache-miss leak
on the BFS thread's hop-label path (caught by the asan gtest run). Free
both right after the class object is extracted, and null-guard the error
path, matching the file's other call sites.
- collectStaticFieldAnchorsForRotation: the other tier is the last
  consumer of the anchor budget - stop accumulating its leftover back
  into budget_left (dead store).
- buildCanaryChainEvent: capture the chain size before moving the vector
  into the event instead of reading the moved-from object for the log.
- secondsToOOM: check ringThirdsStats()'s return for the time ring
  instead of reading time_stats uninitialized on its (unreachable-in-
  practice, but analyzer-visible) failure path - same head/fill/min-fill
  gate as the byte call, so it cannot trigger once the byte call passed.
- Leak-tag pool: start() reset the free list while the preserved tracking
  table still had tagged entries (the table survives stop()/start()), so a
  new object could receive a tag another live object owns and a later
  double release could write past the free list. Reclaim owned tags first
  and keep their correlation info.
- track() published a reserved slot before initializing it while shared-mode
  scanners (tagLeakInstances, getLiveTraceIds) could read the uninitialized
  malloc storage; slots now carry a release/acquire ready flag and fresh
  table regions start unpublished.
- secondsToOOM projects BOTH the heap and the container boundary and takes
  the shorter time instead of picking a ring by the raw limit comparison -
  container usage includes native memory and siblings, so a container with
  a numerically larger limit can still be closer to exhaustion.
- cleanup_table() claimed the GC epoch before acquiring the table lock, so
  a newer epoch's fold could enter the population history before an older
  one's; the claim now happens under the lock (the pre-lock check remains
  as an advisory early exit).
- threadLoop's urgency ramp multiplied the budget by four on every rounded
  pause-target change and never restored it; the boost now applies once
  per urgency episode and the configured budget is restored when it ends.
- The terminal restart gate now charges the finished search's accumulated
  safepoint cost BEFORE checking affordability, so an expensive search no
  longer earns one free immediate successor (restartSearch() no longer
  spends it itself; the pain-budget test asserts the new order).
- hopLabelClassFor() deleted cls twice on the superclass-walk path.
- The class-shape reconciliation loop never deleted the class-object local
  refs GetObjectsWithTags() returned (BFS thread - pins classes against
  unload).
- walkStaticFieldAnchors() early breaks left later anchors' local refs
  undeleted; a cleanup pass now releases them.
- The static-field sweep's truncation cursor resumed by a visited-count
  index that assumes HotSpot's LIFO FollowReferences order; it now redoes
  the chunk, which is order-independent (shared code must not rely on
  HotSpot internals).
- buildDiscoveredInstanceChains() treated a cache hit from an earlier
  search generation as current; the generation check now mirrors the
  representative-refresh paths.
- cacheResolvedChain() reports success so coverage accounting (found bits,
  resolved counts) only advances for a chain that was actually stored.
- os_linux: container usage is now read from the same cgroup level that
  supplied the selected limit (an ancestor limit covers sibling cgroups
  whose usage the leaf excludes).

Moves referenceChains_ut.cpp and livenessTracker_ut.cpp into this layer -
they test exactly this code, and the pain-budget ordering change requires
its test to land with it.
A stale full scratch surviving klassPopulationResetForTest() lets the
first post-reset GC fold fill the population table in one pass, making
the synthetic-epoch seeded entry the permanent LRU-eviction victim - a
fold landing mid-seeding (few-ms GC cadence on slow runners) resets the
seeded ring and breaks the trend gate. Observed as the
shouldSelectSeededKlassAsLeakCandidateOnPositiveSlope flake on
musl-aarch64 (librca 21 and 11).
Comments pointing at locally-kept plan documents, rotted .cpp:NNN
references, and a duplicated four-times comment block consolidated to
the constant it documents.
fillHopEdgeLabels() fills hop edge labels in place; buildChainEvent()
and the canary builder assemble ReferenceChainEvent::_hops from the
parallel internal vectors.
No references to Java integration tests, stresstest repros, or
uncommitted plan documents from this layer; the gtest files carry the
comment cleanups with the code they test.
staleLeaf handling and the line-number-table copy boundary tests.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant