Commit 2f88349
authored
* feat(schema): oracle for L2 call_graph edges and external_symbols
* feat(l1): synthesize record accessors and enum values()/valueOf
* feat(l2): declaring-type hint plumbing for callee backfill
Record the callee's declaring-type binary name per resolved call site so
L2's tree walk can map it to a callee id without keeping ASTs alive. The
binary name comes from a structural containerType() walk (BinaryNames),
the spelling WALA emits and the L2 index will key on. Anonymous creations
are left unhinted so L2 can bind them to the anon's own constructor by
node identity rather than fabricating a super/interface endpoint.
The hint is @cacheonly: a V2Json.cache() Gson keeps it while the payload
writers exclude it in both directions, and both L1Cache paths switch to
it so a warm-cache run does not silently drop the backfill. No payload
change.
* feat(l2): declared call_graph, callee backfill, and external_symbols
L2CallGraph is a pure walk over the emitted L1 tree: it derives a
binary-name type index from the tree, backfills each resolved call
node's callee, accumulates one declared edge per (src, dst) with
weight = call-site count, and homes out-of-project targets as
external_symbols so no edge dangles. Self-edges are kept. Anonymous
creations stay unhinted (no fabricated endpoint); their node-identity
edges are a follow-up.
-a 2 now emits under --schema v2: the level gate lifts to 2 and
analyzeV2 runs the pass after saving the L1 cache, since callee is an
L2 refinement that must not persist into an L1 artifact. declared
edges need only the dependency jars, never a build.
Adds JCallEdge/JExternalSymbol models, call_graph/external_symbols on
JApplication, and CanId.externalId. The call-graph-test fixture gains
a second package (Greeter) for a cross-package edge and a JDK call for
external homing. L2CallGraphGateTest covers every definition-of-done
item except rta: no dangling endpoints, non-empty prov, named and
cross-package edges, weight equals backfilled-site count, L1 subset of
L2, determinism, no in-project type homed as external, and schema
conformance.
* feat(l2): anonymous-class node-identity edges (resolution case 1)
Complete the callee backfill for anonymous creations. L1 threads each
anon creation's own generated-constructor can-id through the builders:
CallableBuilder computes a body's local classes first, keyed to the
creation by AST node identity, and hands that map to CallSiteBuilder,
which records the constructor id as the site's declaring-type hint — the
discriminated-union hint L2 dispatches on the can:// prefix. L2 uses it
directly (case 1), checking membership so the endpoint never dangles.
This points new Runnable(){} at the anonymous class's own <init>()
rather than fabricating java.lang.Runnable.<init>() — an interface has
no constructor. L2CallGraphTest covers the edge, the avoided
fabrication, and that self-edges are kept.
* feat(l2): RTA overlay — prov set-union over declared and WALA rta edges
RtaCallGraph builds WALA's RTA call graph (mirroring the v1 scope/CHA/RTA
construction) and reduces it to binary-name + erased-signature endpoint
pairs; L2CallGraph joins them through the same tree-derived index the
declared pass uses. One edge per (src, dst): prov is the set-union of the
attesting analyses (sorted), weight is the declared count when declared
attests it, else the RTA count.
The overlay adds external symbols but never in-project nodes: an in-project
WALA endpoint absent from the tree (a bridge/access$/lambda$ synthetic, or
an $anon$N-vs-Outer$1 identity-join failure) is dropped via the same
membership check, so no-dangling stays structural. A library target is
homed external; the edge is kept.
-a 2 attempts the build to feed RTA; --no-rta or a build failure degrades
to declared-only rather than failing the level. RtaCallGraph defaults the
CodeAnalyzer.projectRootPom global the build machinery reads, so a direct
caller need not know that coupling.
Tests: RtaCallGraphTest pins the descriptor converters; L2CallGraphTest
covers the join (prov union, rta-only, external homing, synthetic/library
drops) against synthetic endpoints; a realworld-tagged test asserts a real
application yields at least one rta-attested edge.
* docs(notes): L2 v1-vs-v2 call-graph comparison
Adds the L2 companion to the L1 comparison note, generated from a full
v1/v2 -a 2 run over the ten real-world fixtures. Both emitters run the
same WALA RTA over the same bytecode, so the note reconciles v1's WALA-
only, application-target-only graph against v2's declared+rta overlay:
v1's edges match v2's rta in-project edges; v2 adds the declared
(JavaParser) analysis and homes the external/library targets v1 drops.
Records the app-dependent value of RTA (negligible on commons-lang,
dominant dispatch fan-out on daytrader8), the 164 fabricated <<implicit>>
callables v2 no longer emits, the callee backfill, and that v1 -a 2
failed on two apps where v2 degraded to declared-only.
* docs(design): L2 spec amendments and ledger entries D17-D23
Amend schema-v2-l3-l4-design.md for the inverted producer posture: the L2
level table and precision-posture note name both analyses and their prov
tags (JavaParser declares, WALA RTA attests + extends), and the L4
points-to note records that its dependency is on the rta overlay.
Add SCHEMA_DECISIONS D17-D23: one edge per (src,dst) with prov set-union
and kept self-edges; JavaParser-declares/WALA-attests production; the
binary-name @external symbol map; true recursive erasure; implicit
constructor synthesis; the cache-only declaring-type hint plumbing; and
implicit record-accessor / enum values()/valueOf synthesis.
* fix(wala): exclude dependency jars that shadow the project's own classes
A project that depends on a released copy of itself (a common benchmark
setup — e.g. commons-lang's test-scoped commons-lang3) had its own
compiled classes duplicated in a dependency jar. WALA binds a class to
the loader that defines it, so those duplicates loaded under the Extension
(library) loader and were excluded from isApplicationClass — dropping them
from entrypoints, the call graph, and IR construction. On commons-lang
this left only 14 of 435 application classes in the hierarchy (73
entrypoints), so WALA's call graph covered ~3% of the project.
ScopeUtils now builds the application classes first and skips any
dependency jar that redefines one, keeping the project's own bytecode
authoritative. commons-lang recovers all 435 application classes (5,141
entrypoints) and its RTA app->app edges rise from 37 to 6,944. This is
shared scope construction, so v1's call graph and future L3/L4 (built on
the same class hierarchy) benefit identically. RtaCallGraph now logs the
application-class count so a future shortfall surfaces rather than
silently thinning the graph.
* feat(l2): --external-calls opt-in for external edges (default off, v1 parity)
v1's L2 kept only application-target edges; default v2 -a 2 now matches by
gating external_symbols and edges to out-of-project targets behind
--external-calls (off by default). When off, a call resolving outside the
project is dropped like an unresolved one (no callee, no edge, no external
symbol); when on, targets are homed so no edge dangles. The gate is
threaded through both the declared pass and the rta join.
V2Emitter now omits an empty call_graph/external_symbols rather than
emitting []/{} , so parity is a missing key (absence = no fact), not an
empty one. The L2CallGraph.build library default keeps external on (it is
intrinsic to L2); only the CLI defaults it off. Ledger D24.
* docs(notes): regenerate L2 comparison after the WALA scope fix
Re-run against the fixed jar and both --external-calls modes. Leads with
the scope bug this comparison surfaced (test-scoped self-dependency
shadowing 421 of commons-lang's 435 classes into WALA's Extension loader)
and its fix, which lifts commons-lang from 21 to 16,183 v1 edges and
restores WALA coverage for v1, v2, and future L3/L4 alike. Totals now show
v2 in both modes: default (external off, v1 parity) and --external-calls
(external targets homed). Concrete post-fix example edges included.
* fix(l2): address pre-merge code-review findings
- ScopeUtils: exclude module-info/package-info from the dependency-shadow
check — every modular jar carries its own module-info, so counting it as
an application class excluded every modular dependency from WALA scope;
and skip a class ASM cannot read rather than aborting the whole scope.
- RtaCallGraph: mute stdout/stderr across the entire WALA phase (scope,
hierarchy, entrypoints, call graph), not just call-graph construction, so
--stdout mode's JSON data channel is never corrupted; log the class-count
diagnostic after restoring the streams.
- L2CallGraph: clear a call node's callee when a run resolves it to nothing,
so build() is idempotent on a reused tree and no stale @external callee
dangles across a re-run with different --external-calls.
- Tests: null-safe external_symbols access in the gate; cover the
clear-on-reuse path and that unresolved local-class calls are never homed
external.
1 parent b3b46bd commit 2f88349
33 files changed
Lines changed: 2420 additions & 52 deletions
File tree
- .claude
- docs/design
- notes
- specs
- src
- main/java/com/ibm/cldk
- schema
- syntactic_analysis
- utils
- test
- java/com/ibm/cldk
- schema
- syntactic_analysis
- resources
- schema
- test-applications/call-graph-test/src/main/java/org/example
- greeting
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
287 | 287 | | |
288 | 288 | | |
289 | 289 | | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
290 | 373 | | |
291 | 374 | | |
292 | 375 | | |
| |||
0 commit comments