Skip to content

Strengthen the defuse tiers for JavaScript, with a precision audit #113

Description

@rahlk

Problem

The defuse linker was designed as a backstop behind tsc, and on TypeScript that is exactly what
it is. On JavaScript it silently becomes the primary resolver, while still being shaped like a
fallback.

Measured provenance split, analyzer v1.1.0, -a 2:

corpus tsc import defuse
vscode (TypeScript, 9,351 modules) 84.8% 8.4% 6.8%
nodejs/node whole repo (1,777 modules) 28.0% 18.6% 53.3%
nodejs/node lib/ (pure JS, 406 modules) 16.1% 17.6% 66.1%
odoo web frontend (650 modules) 10.1% 11.7% 78.2%

On Node's standard library — high-quality, heavily JSDoc'd JavaScript — the checker resolves about
one edge in six. Two thirds of the graph comes from tiers whose job was to catch what tsc missed,
not to carry it.

This is not a parsing problem. tsc parsed all 21,811 JS files in nodejs/node with zero load
failures, at lib/ scale (406 modules, 9,248 callables, 33,837 edges) in 15.5s / 2.4 GB.

Scope boundary

Strengthen the existing tiers in src/semantic_analysis/defuseLinker.ts for the low-tsc-resolution
case, and audit the precision of what they produce.

Explicitly not in scope:

  • Replacing or supplementing tsc with Babel. Joern uses astgen + @babel/parser because it
    is a JVM tool with no TypeScript checker available; that is why it needs XTypeRecovery at all.
    Babel contributes no type information, so adopting it would trade 16-28% real resolution for 0%
    and then require rebuilding the difference. We have no parsing problem to solve.
  • Adopting Joern's fan-out. Its recovery emits candidate lists; the vscode ledger classes
    131,710 rows as "multi-candidate callee lists ... candidate enumeration, not resolution" (one
    .toString() row links a 131 KB candidate string). cants resolves to a single callee, which is
    what makes an edge mean something. Keep that.

Goals

  • An assignment-driven type-propagation tier that runs when tsc resolution is weak, in the
    spirit of Joern's XTypeRecovery (joern-cli/frontends/x2cpg/.../XTypeRecovery.scala):
    flow-insensitive, SSA-symbol-table style, a fixed small iteration count rather than a
    fixpoint
    (Joern defaults to 2), so it stays fast and deterministic. Scan signatures, walk
    assignments to infer variable types, and when a variable is a call receiver set the call's
    type from it.
  • A precision audit over the result, not just a coverage number: sample resolved JS edges
    and verify each against source. The model is the existing Joern ledger's exception
    classification, applied to our own output rather than to theirs.
  • Re-run the Joern comparison on a JavaScript corpus (nodejs/node/lib is a good size:
    406 modules, self-contained, no vendored tree) and record it in
    docs/design/specs/defuse-linker-joern-ledger.md.
  • Caveat the existing superset claim in that ledger as a TypeScript result (see below).

Caveats and known risks

  • More heuristic resolution means more wrong edges, and we have a worked example of the reverse.
    The allowJs fix in feat(callgraph)!: defuse linker, propagation tiers, and the repository-artifact layer #103 removed 89 edges from the vscode graph, 74 of them prov: defuse. They
    were not lost coverage: extensions/media-preview/media/imagePreview.js is a standalone webview
    IIFE with no imports whose e.preventDefault() on a DOM event had been matched by name to
    vs/base/browser/keyboardEvent.ts's StandardKeyboardEvent. Letting tsc type the receiver
    deleted the guess. This issue pushes in the opposite direction, so the precision audit is not
    optional — it is the acceptance criterion.
  • The ledger's 99.72% superset figure is a TypeScript result. It was measured on vscode, where
    tsc carries 84.8% of edges — a checker-backed analyzer against one with no checker. On JS we fall
    back to the same heuristic class Joern occupies while Joern loses nothing, so the margin should be
    expected to narrow and may not hold. It is currently stated without that qualification.
  • Determinism must survive. --jobs N output is byte-identical to --jobs 1 today and that
    gate must keep passing; a propagation tier with iteration-order sensitivity would break it.
  • Joern is not currently installed on the dev machine used for these measurements, so the
    comparison run needs it re-provisioned (v4 distribution, jssrc2cpg).
  • Cost: node/lib is 15.5s / 2.4 GB today. A propagation pass over every assignment will not be
    free, and the JS repos where it matters most are often the largest.

Definition of done

A measurable rise in resolved JS edges on nodejs/node/lib together with a precision audit
showing the false-positive rate did not increase. More edges alone is not success — the failure mode
this issue risks is precisely a larger, more confident, wronger graph.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions