Skip to content

fix(neo4j): write content_hash, so the incremental push actually diffs - #138

Merged
rahlk merged 1 commit into
mainfrom
fix/issue-118-content-hash-v2
Sep 2, 2026
Merged

fix(neo4j): write content_hash, so the incremental push actually diffs#138
rahlk merged 1 commit into
mainfrom
fix/issue-118-content-hash-v2

Conversation

@rahlk

@rahlk rahlk commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Closes #118. Replaces #119, which GitHub auto-closed when its base branch was deleted on merging #117 — same commit, cherry-picked onto main.

The bug

bolt.ts reads each module's stored content_hash to decide which modules changed:

const res = await s.run("MATCH (m:TSModule) RETURN m._module AS k, m.content_hash AS h");

Nothing wrote it. finalizeAnalysis stripped the field from the wire envelope, and the Neo4j
projection — built from that envelope — omitted it, with project.ts naming the strip as the
reason. The diff compared against NULL for every module and concluded everything had changed, so
every push was a full re-upsert.

Three parts of the system disagreed: schema.ts declared content_hash on :TSModule,
bolt.ts read it, project.ts never wrote it. The contract needed no change — only the
writer was missing, so schema.neo4j.json is untouched.

Parity

codeanalyzer-python is the reference and does this correctly: content_hash is wire payload
(schema/py_schema.py:450), projected (neo4j/project.py:689), read back (neo4j/bolt.py:105).

Verification

Live Neo4j, superset-frontend (1,841 modules):

result
push 1 1841 modules (1841 changed) — 55s
push 2, unchanged source 1841 modules (0 changed) — 25s
content_hash stored 1841 / 1841 (was 0)

Also confirmed on sample-app that editing exactly one file yields (1 changed).

Suite: 240 pass, 0 fail.

About the test

The regression test asserts every projected :TSModule carries a non-null content_hash, and is
break-checked — reverting the projection line fails it.

That shape is deliberate. The existing bolt test hand-seeded content_hash:'stale' into a fixture
node, exercising the diff against data the real projection could never produce. A test that supplies
the value it is verifying cannot catch this class of bug.

Risk worth naming

This enables a code path that has never run in production — the permanent full re-upsert was
masking anything latent in the incremental route. The superset double-push is evidence it holds at
real scale, but it is a first execution.

bolt.ts reads each module's stored `content_hash` to find what changed, but
nothing wrote it: finalizeAnalysis stripped the field from the wire envelope,
and the Neo4j projection -- built from that envelope -- omitted it, with a
comment in project.ts noting the strip as the reason. So the diff compared
against NULL for every module and concluded everything had changed. Every push
was a full re-upsert.

Three parts of the system disagreed: schema.ts DECLARED `content_hash` on
:TSModule, bolt.ts READ it, project.ts never WROTE it. The contract needed no
change -- the property was already declared, only the writer was missing.

codeanalyzer-python is the reference and does this correctly: content_hash is
wire payload (schema/py_schema.py:450), projected (neo4j/project.py:689), and
read back (neo4j/bolt.py:105).

Verified on a live Neo4j against superset-frontend:

  push 1              1841 modules (1841 changed)   55s
  push 2, unchanged   1841 modules (   0 changed)   25s
  content_hash stored 1841 / 1841 (was 0)

The regression test asserts every projected :TSModule carries a non-null hash,
and is break-checked. That shape matters here: the existing bolt test seeded
`content_hash:'stale'` into a fixture node by hand, so it exercised the diff
against data the projection could not produce, which is how this survived.

Note this enables a path that has never run in anger. A permanent full
re-upsert was masking any latent bug in the incremental route; the superset
double-push above is the evidence that it holds at scale.

Closes #118.
@rahlk
rahlk merged commit 94293d9 into main Sep 2, 2026
1 check passed
@rahlk
rahlk deleted the fix/issue-118-content-hash-v2 branch September 2, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

content_hash is never written, so every Neo4j push is a full re-upsert

1 participant