Merge pull request #151 from codellm-devkit/feat/issue-150-whole-program-propagation

feat(callgraph): whole-program type propagation in the defuse linker

Author: rahlk

CHANGELOG.md

@@ -27,7 +27,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `--format msgpack` CLI choice, the `analysis.msgpack` artifact, the msgpack
   serialization mixin on schema models, and the `msgpack` dependency are gone.
   `analysis.json` is the single wire format; anyone passing `--format msgpack`
-  must drop the flag. `--format json` still works unchanged.
+  must drop the flag. With one format left, the `--format` flag itself is now
+  removed too — `analysis.json` is always written; anyone passing
+  `--format json` must simply drop the flag.
 - **`--emit neo4j` now enforces its always-full-depth contract** (#119): it
   runs at level 4 with every graph section regardless of defaults, and
   explicitly passing `-a`/`--graphs` alongside it is now the documented

README.md

@@ -165,9 +165,6 @@ $ canpy --help
 │                                                                    --emit schema).               │
 │ --output             -o                     <path>                 Output directory for          │
 │                                                                    artifacts.                    │
-│ --format             -f                     <json>                 Output format for --emit      │
-│                                                                    json: json.                   │
-│                                                                    [default: json]               │
 │ --emit                                      <json|neo4j|schema>    Output target: json           │
 │                                                                    (analysis.json, default) |    │
 │                                                                    neo4j (graph.cypher or live   │
@@ -273,13 +270,14 @@ $ canpy --help
    canpy --input ./my-python-project --output ./out --format msgpack   # → ./out/analysis.msgpack
    ```
 
-3. **Enrich the call graph with PyCG (level 2):**
+3. **Enrich the call graph with the defuse linker (level 2):**
    ```sh
    canpy --input ./my-python-project -a 2
    ```
-   Level 1 edges come from Jedi's lexical resolution. `-a 2` runs **PyCG** and merges its
-   flow-sensitive edges in (RPC / third-party / dynamically-dispatched targets), backfilling
-   callees Jedi could not resolve. Every edge is provenance-tagged (e.g. `jedi`, `pycg`).
+   Level 1 edges come from Jedi's lexical resolution. `-a 2` runs the **defuse linker** —
+   per-callable resolution over lexical scopes, import bindings, class hierarchies, and a
+   bounded type-propagation round — and merges its edges with Jedi's, backfilling the
+   callees Jedi could not resolve. Every edge is provenance-tagged (`jedi`, `defuse`).
 
 4. **Emit a Neo4j snapshot, or push to a live database:**
    ```sh
@@ -321,7 +319,7 @@ levels are cumulative and additive — `analysis.json(-a 1) ⊆ … ⊆ analysis
 | Level | Flag | What it adds | Where it lands |
 | --- | --- | --- | --- |
 | **1** | `-a 1` (default) | Symbol table, Jedi call graph, and `call` nodes in each callable's `body` | `body` calls (`callee: null`) |
-| **2** | `-a 2` | PyCG call-graph enrichment; each call's `callee` backfilled to a `can://` id | `call_graph`, `body` callees |
+| **2** | `-a 2` | Defuse-linker call-graph enrichment; each call's `callee` backfilled to a `can://` id | `call_graph`, `body` callees |
 | **3** | `-a 3` | Native **intraprocedural** CFG/CDG/DDG (syntactic, name-equality, `prov: ["ssa"]`) | `cfg`, `cdg`, `ddg`, `@entry`/`@exit` on each callable |
 | **4** | `-a 4` | **Interprocedural** SDG: synthetic param vertices, alias-aware DDG (`prov: ["points-to"]`) | `param_in`, `param_out`, `summary`, semantic `ddg` |
 
@@ -350,7 +348,7 @@ symbol-table signature by construction
   external dependency to install; the analyzer falls back to the built-in `TypeBasedAliasOracle`
   (Jedi-inferred types; unknown types conservatively alias) only when Scalpel can't resolve a
   construct or a per-callable build fails, keeping the `may_alias` interface total. Call dispatch
-  comes from the merged Jedi(+PyCG) call graph, treated as a frozen oracle.
+  comes from the merged Jedi + defuse-linker call graph, treated as a frozen oracle.
 - **Summaries:** relational formal-in → formal-out flows composed bottom-up over the Tarjan SCC
   condensation of the call graph, a monotone fixpoint within SCCs; globals ride as extra formals,
   closure captures bind at definition sites.
@@ -389,7 +387,7 @@ just populate more of the same tree:
       }
     },
     "call_graph": [ { "src": "can://…/main(a)", "dst": "can://…/helper(x)",
-                      "weight": 1, "prov": ["jedi", "pycg"] } ],
+                      "weight": 1, "prov": ["defuse", "jedi"] } ],
     "external_symbols": {         // imported/builtin call targets, keyed by id
       "can://python/<app>/@external/os/getcwd":
         { "id": "can://python/<app>/@external/os/getcwd", "kind": "external",

codeanalyzer/__main__.py

@@ -38,7 +38,6 @@ def _pin_hash_seed() -> None:
 
 from codeanalyzer.core import Codeanalyzer
 from codeanalyzer.utils import _set_log_level, logger
-from codeanalyzer.config import OutputFormat
 from codeanalyzer.schema import model_dump_json, strip_internal_only
 from codeanalyzer.options import AnalysisOptions, EmitTarget
 
@@ -81,15 +80,6 @@ def main(
         Optional[Path],
         typer.Option("-o", "--output", help="Output directory for artifacts."),
     ] = None,
-    format: Annotated[
-        OutputFormat,
-        typer.Option(
-            "-f",
-            "--format",
-            help="Output format for --emit json: json.",
-            case_sensitive=False,
-        ),
-    ] = OutputFormat.JSON,
     emit: Annotated[
         EmitTarget,
         typer.Option(
@@ -299,7 +289,7 @@ def main(
     options = AnalysisOptions(
         input=input,
         output=output,
-        format=format,
+
         emit=emit,
         app_name=app_name,
         neo4j_uri=neo4j_uri,
@@ -384,22 +374,21 @@ def main(
             )
         else:
             options.output.mkdir(parents=True, exist_ok=True)
-            _write_output(artifacts, options.output, options.format)
-
-
-def _write_output(artifacts, output_dir: Path, format: OutputFormat):
-    """Write artifacts to file in the specified format."""
-    if format == OutputFormat.JSON:
-        output_file = output_dir / "analysis.json"
-        # Use Pydantic's model_dump_json() for compact output
-        # Strip internal-only fields here rather than with a field-level Pydantic
-        # `exclude`: the analysis cache shares the serializer and must keep them.
-        json_str = json.dumps(
-            strip_internal_only(artifacts.model_dump(mode="json", exclude_none=True))
-        )
-        with output_file.open("w") as f:
-            f.write(json_str)
-        logger.info(f"Analysis saved to {output_file}")
+            _write_output(artifacts, options.output)
+
+
+def _write_output(artifacts, output_dir: Path):
+    """Write analysis.json (the single wire format since #118)."""
+    output_file = output_dir / "analysis.json"
+    # Use Pydantic's model_dump_json() for compact output
+    # Strip internal-only fields here rather than with a field-level Pydantic
+    # `exclude`: the analysis cache shares the serializer and must keep them.
+    json_str = json.dumps(
+        strip_internal_only(artifacts.model_dump(mode="json", exclude_none=True))
+    )
+    with output_file.open("w") as f:
+        f.write(json_str)
+    logger.info(f"Analysis saved to {output_file}")
 
 
 app = typer.Typer(

codeanalyzer/config/__init__.py

@@ -1,3 +1,3 @@
-from .options import AnalysisOptions, EmitTarget, OutputFormat
+from .options import AnalysisOptions, EmitTarget
 
-__all__ = ["AnalysisOptions", "EmitTarget", "OutputFormat"]
+__all__ = ["AnalysisOptions", "EmitTarget"]

codeanalyzer/config/config.py

@@ -4,10 +4,6 @@
 from enum import Enum
 
 
-class OutputFormat(str, Enum):
-    JSON = "json"
-
-
 class EmitTarget(str, Enum):
     """Output target selected by ``--emit``.
 
@@ -26,7 +22,6 @@ class EmitTarget(str, Enum):
 class AnalysisOptions:
     input: Path
     output: Optional[Path] = None
-    format: OutputFormat = OutputFormat.JSON
     emit: EmitTarget = EmitTarget.JSON
     app_name: Optional[str] = None
     neo4j_uri: Optional[str] = None