You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The defuse linker was designed as a backstop behind tsc, and on TypeScript that is exactly what
it is. On JavaScript it silently becomes the primary resolver, while still being shaped like a
fallback.
Measured provenance split, analyzer v1.1.0, -a 2:
corpus
tsc
import
defuse
vscode (TypeScript, 9,351 modules)
84.8%
8.4%
6.8%
nodejs/node whole repo (1,777 modules)
28.0%
18.6%
53.3%
nodejs/node lib/ (pure JS, 406 modules)
16.1%
17.6%
66.1%
odoo web frontend (650 modules)
10.1%
11.7%
78.2%
On Node's standard library — high-quality, heavily JSDoc'd JavaScript — the checker resolves about
one edge in six. Two thirds of the graph comes from tiers whose job was to catch what tsc missed,
not to carry it.
This is not a parsing problem. tsc parsed all 21,811 JS files in nodejs/node with zero load
failures, at lib/ scale (406 modules, 9,248 callables, 33,837 edges) in 15.5s / 2.4 GB.
Scope boundary
Strengthen the existing tiers in src/semantic_analysis/defuseLinker.ts for the low-tsc-resolution
case, and audit the precision of what they produce.
Explicitly not in scope:
Replacing or supplementing tsc with Babel. Joern uses astgen + @babel/parser because it
is a JVM tool with no TypeScript checker available; that is why it needs XTypeRecovery at all.
Babel contributes no type information, so adopting it would trade 16-28% real resolution for 0%
and then require rebuilding the difference. We have no parsing problem to solve.
Adopting Joern's fan-out. Its recovery emits candidate lists; the vscode ledger classes
131,710 rows as "multi-candidate callee lists ... candidate enumeration, not resolution" (one .toString() row links a 131 KB candidate string). cants resolves to a single callee, which is
what makes an edge mean something. Keep that.
Goals
An assignment-driven type-propagation tier that runs when tsc resolution is weak, in the
spirit of Joern's XTypeRecovery (joern-cli/frontends/x2cpg/.../XTypeRecovery.scala):
flow-insensitive, SSA-symbol-table style, a fixed small iteration count rather than a
fixpoint (Joern defaults to 2), so it stays fast and deterministic. Scan signatures, walk
assignments to infer variable types, and when a variable is a call receiver set the call's
type from it.
A precision audit over the result, not just a coverage number: sample resolved JS edges
and verify each against source. The model is the existing Joern ledger's exception
classification, applied to our own output rather than to theirs.
Re-run the Joern comparison on a JavaScript corpus (nodejs/node/lib is a good size:
406 modules, self-contained, no vendored tree) and record it in docs/design/specs/defuse-linker-joern-ledger.md.
Caveat the existing superset claim in that ledger as a TypeScript result (see below).
Caveats and known risks
More heuristic resolution means more wrong edges, and we have a worked example of the reverse.
The allowJs fix in feat(callgraph)!: defuse linker, propagation tiers, and the repository-artifact layer #103 removed 89 edges from the vscode graph, 74 of them prov: defuse. They
were not lost coverage: extensions/media-preview/media/imagePreview.js is a standalone webview
IIFE with no imports whose e.preventDefault() on a DOM event had been matched by name to vs/base/browser/keyboardEvent.ts's StandardKeyboardEvent. Letting tsc type the receiver
deleted the guess. This issue pushes in the opposite direction, so the precision audit is not
optional — it is the acceptance criterion.
The ledger's 99.72% superset figure is a TypeScript result. It was measured on vscode, where
tsc carries 84.8% of edges — a checker-backed analyzer against one with no checker. On JS we fall
back to the same heuristic class Joern occupies while Joern loses nothing, so the margin should be
expected to narrow and may not hold. It is currently stated without that qualification.
Determinism must survive.--jobs N output is byte-identical to --jobs 1 today and that
gate must keep passing; a propagation tier with iteration-order sensitivity would break it.
Joern is not currently installed on the dev machine used for these measurements, so the
comparison run needs it re-provisioned (v4 distribution, jssrc2cpg).
Cost: node/lib is 15.5s / 2.4 GB today. A propagation pass over every assignment will not be
free, and the JS repos where it matters most are often the largest.
Definition of done
A measurable rise in resolved JS edges on nodejs/node/libtogether with a precision audit
showing the false-positive rate did not increase. More edges alone is not success — the failure mode
this issue risks is precisely a larger, more confident, wronger graph.
Problem
The defuse linker was designed as a backstop behind tsc, and on TypeScript that is exactly what
it is. On JavaScript it silently becomes the primary resolver, while still being shaped like a
fallback.
Measured provenance split, analyzer v1.1.0,
-a 2:lib/(pure JS, 406 modules)On Node's standard library — high-quality, heavily JSDoc'd JavaScript — the checker resolves about
one edge in six. Two thirds of the graph comes from tiers whose job was to catch what tsc missed,
not to carry it.
This is not a parsing problem. tsc parsed all 21,811 JS files in nodejs/node with zero load
failures, at
lib/scale (406 modules, 9,248 callables, 33,837 edges) in 15.5s / 2.4 GB.Scope boundary
Strengthen the existing tiers in
src/semantic_analysis/defuseLinker.tsfor the low-tsc-resolutioncase, and audit the precision of what they produce.
Explicitly not in scope:
astgen+@babel/parserbecause itis a JVM tool with no TypeScript checker available; that is why it needs
XTypeRecoveryat all.Babel contributes no type information, so adopting it would trade 16-28% real resolution for 0%
and then require rebuilding the difference. We have no parsing problem to solve.
131,710 rows as "multi-candidate callee lists ... candidate enumeration, not resolution" (one
.toString()row links a 131 KB candidate string).cantsresolves to a single callee, which iswhat makes an edge mean something. Keep that.
Goals
spirit of Joern's
XTypeRecovery(joern-cli/frontends/x2cpg/.../XTypeRecovery.scala):flow-insensitive, SSA-symbol-table style, a fixed small iteration count rather than a
fixpoint (Joern defaults to 2), so it stays fast and deterministic. Scan signatures, walk
assignments to infer variable types, and when a variable is a call receiver set the call's
type from it.
and verify each against source. The model is the existing Joern ledger's exception
classification, applied to our own output rather than to theirs.
nodejs/node/libis a good size:406 modules, self-contained, no vendored tree) and record it in
docs/design/specs/defuse-linker-joern-ledger.md.Caveats and known risks
The
allowJsfix in feat(callgraph)!: defuse linker, propagation tiers, and the repository-artifact layer #103 removed 89 edges from the vscode graph, 74 of themprov: defuse. Theywere not lost coverage:
extensions/media-preview/media/imagePreview.jsis a standalone webviewIIFE with no imports whose
e.preventDefault()on a DOM event had been matched by name tovs/base/browser/keyboardEvent.ts'sStandardKeyboardEvent. Letting tsc type the receiverdeleted the guess. This issue pushes in the opposite direction, so the precision audit is not
optional — it is the acceptance criterion.
tsc carries 84.8% of edges — a checker-backed analyzer against one with no checker. On JS we fall
back to the same heuristic class Joern occupies while Joern loses nothing, so the margin should be
expected to narrow and may not hold. It is currently stated without that qualification.
--jobs Noutput is byte-identical to--jobs 1today and thatgate must keep passing; a propagation tier with iteration-order sensitivity would break it.
comparison run needs it re-provisioned (v4 distribution,
jssrc2cpg).node/libis 15.5s / 2.4 GB today. A propagation pass over every assignment will not befree, and the JS repos where it matters most are often the largest.
Definition of done
A measurable rise in resolved JS edges on
nodejs/node/libtogether with a precision auditshowing the false-positive rate did not increase. More edges alone is not success — the failure mode
this issue risks is precisely a larger, more confident, wronger graph.