feat(neo4j): derive code onto declarations, matching python and java - #139
Merged
Conversation
TypeScript source text was unreachable from the graph. :TSCallable carried no
`code`, and :TSModule carried no `source` to slice from -- nodes had only
start_line/end_line, not the `span.bytes` the tree uses. So neither route
worked, while python and java hand a consumer `code` directly.
The JSON model was never the disagreement: python and typescript both store a
module's text ONCE and address nodes into it by UTF-8 byte span, with no
per-callable `code` field. Python then derives `code` at projection time
(neo4j/project.py::_span_code, "schema v2 stores source once per module, so the
graph's `code` property ... is derived here"). TypeScript simply stopped short
of that step. This adds it.
`spanCode` slices a Buffer, not the string: `span.bytes` are UTF-8 byte offsets
and JS strings are UTF-16, so string slicing would cut multi-byte characters --
the same trap that had to be fixed in artifact text capture.
Also adds `ts_code_fts`, a fulltext index over TSCallable.code, mirroring
python's `py_code_fts`. Text on a node that cannot be searched is only half the
feature.
`code` is declared on every label that receives it -- TSCallable plus the five
type labels, since typeProps is shared -- rather than only the two python
declares, so the conformance gate stays honest.
Verified on a live graph: 41 of 42 callables carry sliced source (the one
without is a bodiless declaration), e.g. Get -> "function Get(path: string):
MethodDecorator {\n return () =>...". Full suite with containers: 237 pass.
Java's JSON stores `code` per callable instead of slicing; filed as
codellm-devkit/codeanalyzer-java#215.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #136 (same commit, cherry-picked onto
mainafter the stack below it merged).The gap
TypeScript source text was unreachable from the graph:
:TSCallablehad nocode, and:TSModulehad nosourceto slice from — nodes carriedstart_line/end_line, not thespan.bytesthe tree uses. Neither route worked. python and java hand a consumercodedirectly.The JSON model was never the disagreement
codesourceonce +span.bytes; no per-callablecodesourceonce +span.bytes; no per-callablecodecodestored per callablepython and TypeScript already agree on the JSON model. python then completes the contract on the
graph side (
neo4j/project.py::_span_code):TypeScript stopped short of that step. This adds it — no JSON change, no text stored twice.
Details matched deliberately
spanCodeslices aBuffer, not the string.span.bytesare UTF-8 byte offsets and JSstrings are UTF-16, so
source.slice()would cut multi-byte characters.ts_code_fts, a fulltext index overTSCallable.code, mirroring python'spy_code_fts.Text on a node that cannot be searched is half a feature.
codeis declared on every label that receives it —TSCallableplus the five type labels,since
typePropsis shared. python declares it only onPyClass/PyCallable; declaring it whereit is actually emitted keeps the conformance gate meaningful.
Verification
Live graph,
sample-app: 41 of 42 callables carrycode(the holdout is a bodilessdeclaration). Sample value:
WHERE c.code CONTAINS "..."now works. Suite: 240 pass, 0 fail;gen:schemaclean.Related
codeper callable rather than slicing —analysis.json stores declaration text per callable; python and typescript store it once per module codeanalyzer-java#215, filed as a decision rather than a defect.
docstringis still divergent: python and java carry it, TypeScript's JSON hascommentsoncallables. That needs a shape decision, not a straight port, so it is not in this PR.