feat(artifacts): source is the whole file; scope and gate every Bolt delete (#174)

Two contract fixes on the Neo4j/artifact surface.

fix(neo4j): the per-module Bolt purge matched `MATCH (x {_module: $m})` with no
label. `_module` is a shared convention -- codeanalyzer-java and
codeanalyzer-typescript set it on their nodes too -- so wherever a python module
and a sibling analyzer's module shared a file key, a python push detach-deleted
that sibling's nodes and their relationships, silently. Both statements are now
anchored on the python-owned labels, derived from the schema catalog
(`MODULE_OWNED_LABELS`) so a new module-scoped label is covered automatically.
A push also no longer deletes anything by default: the per-module purge and the
full-run orphan prune both run under `--eager` only, so a default `--lazy` push
is purely additive. Adds an `_module` index per module-owned label, the purge
having been an unindexed scan once per changed module.

BREAKING (released as a minor by maintainer decision -- see CHANGELOG):
`PyArtifact.text_truncated`, `--artifact-text-max-bytes`, and the Neo4j
`:Artifact.text_truncated` property are removed. An artifact's `source` is the
whole file, or `""` because the file is binary or `--no-artifact-text` was
passed -- never a prefix. A truncated `source` read exactly like a complete
small file, and the flag meant to distinguish them conflated "whole file" with
"capture off"; measured on microsoft/vscode the cap fired on 32 of 4,953
artifacts (0.6%). python's `dependency-manifest` exemption goes with the cap.
Matches codeanalyzer-typescript, which removed both in its #117.

Migration: drop any `text_truncated` check (`source` is trustworthy whenever
non-empty); drop `--artifact-text-max-bytes` (use `--no-artifact-text` to omit
the payload); pass `--eager` if you relied on a push removing stale nodes.

Specs: codellm-devkit/.github docs/design/specs/2026-09-02-artifact-source-whole-file.md
Epic: codellm-devkit/.github#51

Closes #171
Closes #172

Author: rahlk

CHANGELOG.md

@@ -5,6 +5,46 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Removed
+
+- **BREAKING:** the artifact-text byte cap, its `--artifact-text-max-bytes`
+  flag, and the `PyArtifact.text_truncated` field (also the Neo4j `:Artifact`
+  property). An artifact's `source` is now the whole file, or `""` because it
+  is binary or `--no-artifact-text` was passed -- never a prefix. A truncated
+  `source` read exactly like a complete small file, and the flag meant to
+  distinguish them conflated "whole file" with "capture off"; measured on
+  microsoft/vscode the cap fired on 32 of 4,953 artifacts (0.6%). Matches
+  codeanalyzer-typescript, which removed both in its #117. `--no-artifact-text`
+  is unchanged. python's `dependency-manifest` exemption from the cap is gone
+  with the cap -- every decodable file is captured in full (#172).
+
+### Fixed
+
+- Neo4j incremental push no longer deletes a sibling analyzer's nodes. The
+  per-module purge matched `MATCH (x {_module: $m})` with no label, and
+  `_module` is a shared convention -- `codeanalyzer-java` and
+  `codeanalyzer-typescript` set it on their nodes too -- so where a file key
+  collided, a python push silently detach-deleted their graph. Both statements
+  are now anchored on the python-owned labels, derived from the schema catalog
+  (`MODULE_OWNED_LABELS`) so a new module-scoped label is covered
+  automatically (#171).
+
+### Changed
+
+- A Neo4j Bolt push no longer deletes anything by default. The per-module purge
+  and the full-run orphan prune are the only destructive steps, and both now run
+  under `--eager` only; a default `--lazy` push is purely additive. The cost of
+  the default is staleness -- a declaration or call edge the source no longer has
+  survives until an `--eager` push reconciles it -- and the gain is that an
+  incremental push into a shared database cannot destroy anything (#171).
+
+### Added
+
+- An `_module` index per module-owned Neo4j label. The per-module purge ran as
+  an unindexed scan once per changed module -- quadratic on a full push (#171).
+
 ## [1.3.0] - 2026-08-29
 
 ### Added

README.md

@@ -175,7 +175,10 @@ def main(
         bool,
         typer.Option(
             "--eager/--lazy",
-            help="Enable eager or lazy analysis. Defaults to lazy.",
+            help="Enable eager or lazy analysis. Defaults to lazy. Also gates every "
+            "destructive step of a '--emit neo4j' Bolt push: a lazy push only adds and "
+            "updates, an eager one also removes declarations and edges the source no "
+            "longer has.",
         ),
     ] = False,
     skip_tests: Annotated[
@@ -239,19 +242,11 @@ def main(
         typer.Option(
             "--artifact-text/--no-artifact-text",
             help="Capture verbatim `source` text on discovered artifacts. "
-            "--no-artifact-text empties `source` everywhere (inventory unchanged).",
+            "`source` is the whole file; --no-artifact-text empties it "
+            "everywhere (inventory unchanged). sha256/size_bytes always "
+            "reflect the full file.",
         ),
     ] = True,
-    artifact_text_max_bytes: Annotated[
-        int,
-        typer.Option(
-            "--artifact-text-max-bytes",
-            help="Per-file byte cap on captured artifact `source`; a decodable "
-            "file over the cap is truncated (text_truncated=True). "
-            "sha256/size_bytes always reflect the full file.",
-            min=1,
-        ),
-    ] = 262144,
 ):
     # Determinism: pin the interpreter hash seed before any analysis (no-op
     # when PYTHONHASHSEED is already set; --version exits before this).
@@ -336,7 +331,6 @@ def main(
         verbosity=verbosity,
         entrypoint_rules=tuple(entrypoint_rules or ()),
         artifact_text=artifact_text,
-        artifact_text_max_bytes=artifact_text_max_bytes,
     )
 
     _set_log_level(options.verbosity)

codeanalyzer/__main__.py

@@ -81,9 +81,9 @@ def _resolve_ref(manifest_path: str, ref: str) -> Optional[str]:
 
 def _full_text(project_dir: Path, path: str, art: PyArtifact) -> str:
     """Manifest/lock extraction must never depend on the stored ``source`` --
-    that's capped by ``text_max_bytes`` and emptied by ``capture_text=False``
-    (both payload-size controls on the JSON/Neo4j payload, not extraction
-    controls). Read the real file fresh instead; fall back to ``art.source``
+    that's emptied by ``capture_text=False`` (a payload-size control on the
+    JSON/Neo4j payload, not an extraction control). Read the real file fresh
+    instead; fall back to ``art.source``
     only if it is gone (e.g. a synthetic artifact in a unit test, or the file
     vanished mid-run).
 

codeanalyzer/artifacts/dependencies.py

@@ -76,27 +76,11 @@ def _classify(rel_posix: str) -> Tuple[str, List[str]] | None:
     return None
 
 
-def _capture_source(
-    raw: bytes, text: str, capture_text: bool, text_max_bytes: int
-) -> Tuple[str, bool]:
-    """Decide ``(source, text_truncated)`` for a decodable file.
-
-    Slices ``raw`` (not ``text``) for the cap, so it is a true byte cap even
-    when it lands inside a multi-byte character -- ``errors="ignore"`` drops
-    the dangling partial char at the cut, so this never raises."""
-    if not capture_text:
-        return "", False
-    if len(raw) <= text_max_bytes:
-        return text, False
-    return raw[:text_max_bytes].decode("utf-8", errors="ignore"), True
-
-
 def discover_artifacts(
     project_dir: Path,
     app_name: str,
     *,
     capture_text: bool = True,
-    text_max_bytes: int = 262144,
 ) -> Dict[str, PyArtifact]:
     """Walk the project and return every file as an artifact, sorted by path.
 
@@ -109,15 +93,11 @@ def discover_artifacts(
     deliberate exception -- it IS rule-matched (a dependency-manifest), so it
     is captured like any other manifest despite the `.py` suffix.
 
-    ``capture_text=False`` empties ``source`` everywhere (inventory otherwise
-    identical); a decodable file over ``text_max_bytes`` gets a truncated
-    ``source`` and ``text_truncated=True`` -- except a ``dependency-manifest``
-    role artifact, which is always captured in full when decodable and
-    ``capture_text`` is on: its source is what ``build_dependency_view``
-    parses, not bulk/incidental content, so the byte cap does not apply to
-    it (``capture_text=False`` still empties it like everything else).
-    ``sha256``/``size_bytes`` always reflect the full file regardless of
-    either knob."""
+    ``source`` is the WHOLE file or nothing -- never a prefix (#172). A
+    decodable file is captured in full; ``capture_text=False`` empties
+    ``source`` everywhere (inventory otherwise identical), and an undecodable
+    file gets ``""`` as ``binary``. ``sha256``/``size_bytes`` always reflect
+    the full file regardless."""
     out: Dict[str, PyArtifact] = {}
     for path in sorted(project_dir.rglob("*")):
         if not path.is_file():
@@ -148,20 +128,14 @@ def discover_artifacts(
             if decodable and "." not in name and text.startswith("#!"):
                 roles = ["script"]
         if decodable:
-            # A dependency-manifest's source IS the extracted meaning (build_
-            # dependency_view parses it) -- the byte cap targets bulk/incidental
-            # assets, never the files extraction depends on, so manifests are
-            # exempt from it. capture_text=False still empties source (handled
-            # inside _capture_source); only the byte CAP is bypassed here.
-            cap = len(raw) if "dependency-manifest" in roles else text_max_bytes
-            source, text_truncated = _capture_source(raw, text, capture_text, cap)
+            source = text if capture_text else ""
         else:
-            fmt, source, text_truncated = "binary", "", False
+            fmt, source = "binary", ""
 
         out[rel_posix] = PyArtifact(
             id=artifact_id(app_name, rel_posix), path=rel_posix, format=fmt,
             roles=list(roles), size_bytes=len(raw),
             sha256=hashlib.sha256(raw).hexdigest(),
-            source=source, text_truncated=text_truncated,
+            source=source,
         )
     return out

codeanalyzer/artifacts/discovery.py

@@ -40,9 +40,9 @@
 def _artifact_full_text(project_dir: Path, path: str, art) -> str:
     """Mirrors ``artifacts.dependencies._full_text`` verbatim (not imported
     -- that name is module-private to ``dependencies.py``): config-key
-    extraction (#152) must never depend on the stored ``source`` -- capped
-    by ``text_max_bytes`` and emptied by ``capture_text=False`` (payload-size
-    controls, not extraction controls). Read the real file fresh instead;
+    extraction (#152) must never depend on the stored ``source`` -- emptied
+    by ``capture_text=False`` (a payload-size control, not an extraction
+    control). Read the real file fresh instead;
     fall back to ``art.source`` only if it's gone (e.g. a synthetic artifact
     in a unit test, or the file vanished mid-run). Keep the two in sync if
     this logic changes."""
@@ -673,7 +673,6 @@ def analyze(self) -> Analysis:
         app.artifacts = discover_artifacts(
             self.project_dir, app_name,
             capture_text=self.options.artifact_text,
-            text_max_bytes=self.options.artifact_text_max_bytes,
         )
         app.dependencies, app.unresolved_imports = build_dependency_view(
             app.artifacts,

codeanalyzer/core.py

@@ -27,11 +27,24 @@
   4. upsert edges owned by changed modules (+ the shared edges).
   5. on a FULL run only, prune modules whose source file vanished.
 
+**A push never deletes by default** (#171). Steps 3 and 5 are the only destructive
+ones and both run on ``eager`` (``--eager``) only; a default ``--lazy`` push is purely
+additive — MERGE-upsert of nodes and edges, nothing removed. The cost of the default is
+staleness: a declaration or a call edge the source no longer has stays in the graph until
+an ``--eager`` push reconciles it. That is the deliberate trade — an incremental push into
+a shared database should not be able to destroy anything, and the destructive rebuild is
+opt-in under the same flag that already forces a clean analysis rebuild.
+
 Nodes are MERGE-upserted, never blindly deleted, so a declaration another
 (unchanged) module still references survives and its incoming edges stay valid.
 ``:PyExternal`` / ``:PyPackage`` / ``:PyDecorator`` are shared (no ``_module``) and are
 MERGE-only.
 
+Every ``_module`` match is anchored on the python-owned labels
+(``schema.MODULE_OWNED_PATTERN``). ``_module`` is a shared convention, not a python-private
+one -- codeanalyzer-java and codeanalyzer-typescript set it on their nodes too -- so an
+unlabelled match reaches a sibling analyzer's graph in a shared database (#171).
+
 The ``neo4j`` driver is imported lazily so it stays an optional dependency and
 off the default (json) output path entirely.
 """
@@ -41,7 +54,7 @@
 from typing import Dict, List, Optional
 
 from codeanalyzer.neo4j.rows import EdgeRow, GraphRows, NodeRow, chunk
-from codeanalyzer.neo4j.schema import CONSTRAINTS, INDEXES
+from codeanalyzer.neo4j.schema import CONSTRAINTS, INDEXES, MODULE_OWNED_PATTERN
 from codeanalyzer.utils import logger
 
 DESCENDANTS = (
@@ -59,7 +72,7 @@ class BoltConfig:
     database: Optional[str] = None
 
 
-def bolt_writer(rows: GraphRows, cfg: BoltConfig, full_run: bool) -> None:
+def bolt_writer(rows: GraphRows, cfg: BoltConfig, full_run: bool, eager: bool = False) -> None:
     try:
         import neo4j  # noqa: WPS433 (lazy, optional dependency)
     except ImportError as exc:  # pragma: no cover - exercised only without the extra
@@ -119,15 +132,26 @@ def session():
         _upsert_nodes(session, neo4j, shared)
 
         # 4. per changed module: purge owned edges + vanished decls, then upsert its nodes.
+        # The purge is the only destructive step in a push, so it runs on --eager only.
         for m in changed:
             nodes = by_module[m]
             keys = [n.value for n in nodes]
+            if not eager:
+                _upsert_nodes(session, neo4j, nodes)
+                continue
             with session() as s:
                 def _purge(tx, module=m, node_keys=keys):
-                    tx.run("MATCH (x {_module: $m})-[r]->() DELETE r", m=module)
+                    # Anchored on python-owned labels: `_module` is also set by the java
+                    # and typescript analyzers, so an unlabelled match would delete a
+                    # sibling's nodes wherever a file key collides (#171).
                     tx.run(
-                        "MATCH (x {_module: $m}) "
-                        "WHERE NOT coalesce(x.signature, x.id, x.file_key) IN $keys "
+                        f"MATCH (x:{MODULE_OWNED_PATTERN}) WHERE x._module = $m "
+                        "MATCH (x)-[r]->() DELETE r",
+                        m=module,
+                    )
+                    tx.run(
+                        f"MATCH (x:{MODULE_OWNED_PATTERN}) WHERE x._module = $m "
+                        "AND NOT coalesce(x.signature, x.id, x.file_key) IN $keys "
                         "DETACH DELETE x",
                         m=module,
                         keys=node_keys,
@@ -147,7 +171,7 @@ def _purge(tx, module=m, node_keys=keys):
         # 6. orphan prune — only safe on a full run (a targeted run can't tell deleted from untargeted).
         # Scope to THIS application's anchor so a full run for application B never
         # deletes application A's modules from a shared database.
-        if full_run and app_name is not None:
+        if full_run and eager and app_name is not None:
             present = list(by_module.keys())
             with session() as s:
                 res = s.run(
@@ -161,6 +185,11 @@ def _purge(tx, module=m, node_keys=keys):
                 pruned = res.single()
                 pruned_count = pruned["pruned"] if pruned else 0
                 logger.info(f"neo4j(bolt): pruned {pruned_count} vanished module(s)")
+        elif not eager:
+            logger.info(
+                "neo4j(bolt): additive push (--lazy) — nothing deleted; "
+                "re-run with --eager to reconcile removed declarations and edges"
+            )
         else:
             logger.info(
                 "neo4j(bolt): targeted run — orphan pruning skipped (deleted files not removed)"

codeanalyzer/neo4j/bolt.py

@@ -67,9 +67,10 @@ def emit_neo4j(analysis: Analysis, options: AnalysisOptions) -> None:
             password=options.neo4j_password,
             database=options.neo4j_database,
         )
-        # A full run (no single-file restriction) makes orphan pruning safe.
+        # A full run (no single-file restriction) makes orphan pruning safe; --eager
+        # is what permits any deletion at all (#171).
         full_run = options.file_name is None
-        bolt_writer(rows, cfg, full_run)
+        bolt_writer(rows, cfg, full_run, eager=options.rebuild_analysis)
         return
 
     out_dir = options.output if options.output is not None else Path.cwd()

codeanalyzer/neo4j/emit.py

@@ -309,7 +309,6 @@ def _project_artifacts(b: RowBuilder, app: PyApplication, app_name: str, app_ref
                     "size_bytes": art.size_bytes,
                     "sha256": art.sha256,
                     "source": art.source,
-                    "text_truncated": art.text_truncated,
                     "extraction": art.extraction,
                 }
             ),

codeanalyzer/neo4j/project.py

@@ -210,7 +210,7 @@ class RelType:
     NodeLabel("Artifact", "Artifact", "id", {
         "id": "string", "path": "string", "format": "string",
         "roles": "string[]", "size_bytes": "integer", "sha256": "string",
-        "source": "string", "text_truncated": "boolean", "extraction": "string",
+        "source": "string", "extraction": "string",
     }),
     NodeLabel("Package", "Package", "id", {
         "id": "string", "ecosystem": "string", "name": "string",
@@ -332,10 +332,27 @@ def uniqueness_constraints() -> list[str]:
 
 CONSTRAINTS: List[str] = uniqueness_constraints()
 
+# The labels this analyzer owns per module -- the ones carrying the internal ``_module``
+# provenance property. Derived from NODE_LABELS so a new module-scoped label is covered
+# without a second list to maintain. `_module` is NOT python-private: codeanalyzer-java
+# and codeanalyzer-typescript set the same property on their nodes, so every statement
+# matching on it must be anchored to these labels or it matches a sibling analyzer's graph
+# in a shared database (#171).
+MODULE_OWNED_LABELS: List[str] = [n.label for n in NODE_LABELS if "_module" in n.properties]
+
+# The label disjunction to anchor such a statement with: ``MATCH (x:PyModule|PyClass|...)``.
+MODULE_OWNED_PATTERN: str = "|".join(MODULE_OWNED_LABELS)
+
 INDEXES: List[str] = [
     "CREATE INDEX py_callable_name IF NOT EXISTS FOR (c:PyCallable) ON (c.name)",
     "CREATE INDEX py_class_name IF NOT EXISTS FOR (c:PyClass) ON (c.name)",
     "CREATE FULLTEXT INDEX py_code_fts IF NOT EXISTS FOR (c:PyCallable) ON EACH [c.code, c.docstring]",
+] + [
+    # One per module-owned label: the incremental writer's per-module purge matches on
+    # `_module` once per changed module, which without these is a label scan per label per
+    # module -- quadratic on a full push (#171).
+    f"CREATE INDEX {label.lower()}_module IF NOT EXISTS FOR (x:{label}) ON (x._module)"
+    for label in MODULE_OWNED_LABELS
 ]