Skip to content

feat(gleam): add full Gleam language support and Erlang FFI resolution - #1546

Open
zhoumao wants to merge 1 commit into
colbymchenry:mainfrom
zhoumao:feat/gleam-language-support
Open

feat(gleam): add full Gleam language support and Erlang FFI resolution#1546
zhoumao wants to merge 1 commit into
colbymchenry:mainfrom
zhoumao:feat/gleam-language-support

Conversation

@zhoumao

@zhoumao zhoumao commented Aug 13, 2026

Copy link
Copy Markdown

Summary

  • Add tree-sitter-backed Gleam extraction for .gleam files, including functions, external functions, constants, custom types, constructors, type aliases, imports, calls, and type references.
  • Add package-aware Gleam module resolution across src/ and test/, including aliases, selective imports, namespace calls, constructor visibility, opaque types, prelude terminals, and _build exclusion.
  • Add explicit Erlang FFI resolution by module, function, and arity.
  • Extract Erlang function arities and persist foreign-reference metadata through unresolved-reference retries and resolved edges.
  • Update grammar assets, language metadata, extraction invalidation, schema migration, documentation, and regression tests.

Extraction and Graph Validation on Public GitHub Repositories

Repo Commit Gleam files Indexed files Nodes Edges
Pevensie/valkyrie 9aa6f14 7 10 524 1,699
enveriumhq/yamleam c9732ba 19 19 373 1,488
  • Both repositories indexed with state=complete, zero pending references, and zero pending changes.
  • valkyrie: 464 of 900 call edges are cross-file.
  • yamleam: 599 of 830 call edges are cross-file.
  • valkyrie's monotonic_now/0 resolves from Gleam to src/valkyrie_ffi.erl with persisted Erlang FFI metadata.
  • Wrong-arity, missing-target, and ambiguous FFI cases are covered by synthetic regression tests.

Counts are whole-index totals, not only .gleam files. Validation used the local CodeGraph build from this PR: CodeGraph 1.5.0, extraction version 26, Node 24.18.0. PASS indicates a complete index with no pending references or changes; the repositories' own test suites were not executed.

Automated Verification

  • Gleam-focused tests pass: 26/26.
  • npm run build passes under Node 24.18.0.
  • git diff --check passes.
  • Full-suite failures were limited to unrelated sandbox resource issues involving socket permissions, filesystem-watch limits, and timing-sensitive integration tests.

Authorship

This PR was created with the help of OpenAI Codex. Please review the implementation and validation results before merging.

@codegraph-impact codegraph-impact Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeGraph review

Overall risk: 🟠 Medium — Gleam support lands by changing shared extraction and resolution hooks with many upstream dependents, and several of those hot paths still show NO TEST REACHES THIS.

Worth double-checking

  • Shared function extraction path
  • Gleam import and external-name resolution boundaries
  • FFI metadata persistence across upgrade and reindex flows
What to look for in each
  • Shared function extraction pathextractFunction now calls this.extractor.afterExtractFunction?.(node, funcNode, this.makeExtractorContext()) in the generic path before body traversal. Because that path is shared by many languages and the graph flags LanguageExtractor, extractFunction, and isExported as widely depended on with no direct test reach, scrutinize whether existing extractors still emit identical nodes and edges outside Gleam.
  • Gleam import and external-name resolution boundariesresolveViaImport now resolves Gleam imports references directly to file nodes, while isBuiltInOrExternal now short-circuits gleam/* imports and prelude names as external. Scrutinize same-name collision cases where a project symbol resembles a compiler-provided module or prelude constructor, because these branches decide whether resolution stops early or falls through to normal matching.
  • FFI metadata persistence across upgrade and reindex flows — Schema version 10 adds unresolved_refs.metadata, rowToUnresolvedReference round-trips it, and resurrectRefFromDroppedEdge only reconstructs metadata when ffi, targetLanguage === 'erlang', module, function, and numeric arity are all present. Scrutinize upgrade, sync, and edge-drop/rebuild paths to confirm older indexes and reindexed projects retain enough metadata to relink foreign-function calls.
Business rules — 2 honored · 10 not applicable
Status Rule Note
✔ Honored The graph model The schema and query changes extend unresolved_refs with nullable metadata and preserve its role as the staging table for post-extraction resolution.
✔ Honored Changelog and release contract CHANGELOG.md gains a user-facing Gleam feature entry under the unreleased feature section, and package.json is not version-bumped.
— Not applicable Impact radius (blast radius) The diff does not modify getImpactRadius or its traversal semantics; the blast radius here is consumer data, not changed implementation.
— Not applicable Risk policy: surface, don't score No getNodeMetrics or engine-side risk-scoring logic is changed in the provided diff.
— Not applicable Test selection (affected tests) The affected-test CLI behavior in src/bin/codegraph.ts is untouched; this PR only adds tests and underlying extraction/resolution behavior.
— Not applicable Surfaces The existing CLI/MCP/library surfaces remain the same; this PR changes what those surfaces can index and resolve, not which surfaces exist.
— Not applicable CodeGraph business rules The README language-support updates do not modify the separate business-rules source of truth described by this rule.
— Not applicable Installer contract Installer behavior and __tests__/installer-targets.test.ts are untouched by this change.
— Not applicable Agent interface rules No MCP initialize behavior or agent-facing interface rules are modified in the provided diff.
— Not applicable Server instructions are the single source of truth The change does not edit src/mcp/server-instructions.ts or duplicate agent guidance into other files.
— Not applicable Call budget by repo size No explore-call budgeting code in src/mcp/tools.ts is changed.
— Not applicable Source strings must exclude interpolated template literals Although src/extraction/index.ts changes, the provided source slice shows deleted-file/edge resurrection work, not changes to extractSourceStrings or interpolated-template indexing.
Full assessment

This PR adds full Gleam support across indexing, import resolution, constructor/type handling, and Erlang FFI linking, and it extends unresolved_refs so structured FFI metadata survives persistence and migration. The implementation is not isolated to a new parser file: it also changes shared extraction and resolution paths such as LanguageExtractor, extractFunction, import resolution, and built-in/external filtering. I do not see a concrete source-level defect in the provided slices, but those shared paths sit under broad blast radius and several are explicitly marked as having no direct test reach, so the merge should focus on regression containment rather than on a single obvious bug.

QA checklist — 4 things to verify in the running product
  • cli — Create a tiny Gleam project with two modules where one imports and calls a public function from the other, run codegraph init, then inspect callers/callees for that function. (Before this PR, .gleam files were not part of CodeGraph's supported-language flow, so this cross-file call graph should only appear with the new change.)
  • cli — Create a Gleam project that imports a compiler-provided module function while the repo also defines a same-named local helper, then inspect the caller's outgoing links. (This PR adds explicit filtering for Gleam standard-library and prelude references; with the change, the compiler-provided call should no longer falsely bind to a same-named project symbol.)
  • cli — Create a Gleam wrapper annotated to call an Erlang function, add the matching Erlang module/function with the same arity, run codegraph init, and inspect the wrapper's outgoing call edge. (The new behavior persists FFI metadata and resolves by module, function, and arity; without this PR, that cross-language edge would not resolve.)
  • cli — In a git-tracked Gleam project, index once, delete a tracked .gleam file, rerun full indexing, and then list/query that path to confirm it is gone and the run reports no file error. (This PR adds a regression test for tracked-but-deleted Gleam files during full indexing; the observable difference is clean removal instead of stale file data or an indexing error.)
Blast radius: 219 files affected beyond the diff · 1161 symbols · 133 test files selected

Tests to run:

  • __tests__/adaptive-explore-sizing.test.ts
  • __tests__/android-res-exclusion.test.ts
  • __tests__/arkts-resolution.test.ts
  • __tests__/batched-ref-cleanup.test.ts
  • __tests__/c-fnptr-kernel-sweep.test.ts
  • __tests__/c-fnptr-synthesizer.test.ts
  • __tests__/celery-dispatch-synthesizer.test.ts
  • __tests__/cfml-inheritance-resolution.test.ts
  • __tests__/cfml-receiver-inference.test.ts
  • __tests__/cli-affected-paths.test.ts
  • __tests__/cli-no-color.test.ts
  • __tests__/cli-node-command.test.ts
  • __tests__/cli-query-command.test.ts
  • __tests__/closure-collection-synthesizer.test.ts
  • __tests__/concurrent-locking.test.ts
  • __tests__/config-secret-redaction.test.ts
  • __tests__/context-ranking.test.ts
  • __tests__/context.test.ts
  • __tests__/cooperative-yield.test.ts
  • __tests__/daemon-bind-failure.test.ts

Full report

Add tree-sitter-backed extraction for Gleam modules, symbols, imports, calls, constructors, and type references, with package-aware src/test resolution.

Resolve explicit Erlang FFI targets by module, function, and arity, persisting foreign-reference metadata and extracting Erlang function arities.

Handle opaque constructor visibility, prelude and external terminals, _build and deleted-file indexing, extraction invalidation, and regression coverage.
@zhoumao
zhoumao force-pushed the feat/gleam-language-support branch from a7e2c47 to 7d29502 Compare August 13, 2026 07:32

@codegraph-impact codegraph-impact Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeGraph review

Overall risk: 🟠 Medium — Gleam support lands by touching shared extractor and import-resolution paths, and several of those core helpers still show no direct test reach.

Worth double-checking

  • Shared import and symbol resolution
  • Database upgrade and retry paths
  • Gleam external and foreign-function classification
What to look for in each
  • Shared import and symbol resolution — Scrutinize the new Gleam branches in the shared resolver, especially the gleam entry in EXTENSION_RESOLUTION, the Gleam file-edge branch in resolveViaImport, and the memoized export walk in findExportedSymbol; these sit on common resolution infrastructure, and the blast radius flags resolveViaImport and findExportedSymbol as core helpers with no direct test reach.
  • Database upgrade and retry paths — Check the version-10 migration plus all unresolved-ref readers/writers together: CURRENT_SCHEMA_VERSION moved to 10, migration 10 conditionally adds metadata TEXT, and every unresolved-ref fetch now round-trips metadata through rowToUnresolvedReference; the important thing to verify is that upgraded databases and failed-ref retry flows preserve that metadata instead of silently dropping it.
  • Gleam external and foreign-function classification — Review the interaction between Gleam builtin/external filtering and FFI resolution: isBuiltInOrExternal now short-circuits Gleam stdlib/prelude refs, gleamErlangFfiMetadata accepts only structured Erlang-call metadata, and resolveGleamErlangFfi resolves only when exactly one Erlang function carries the matching erlang-arity:N decorator.
Full assessment

This PR adds first-class Gleam indexing, import resolution, constructor/type handling, Erlang FFI linking, and persistence of unresolved-reference metadata for that new resolution path. The schema also advances to version 10 so older databases can store the new metadata column. I do not see a concrete defect in the provided source slices, but the change reaches shared extraction and resolution code paths, so regression risk comes from breadth rather than an obvious break.

QA checklist — 4 things to verify in the running product
  • cli — Index a tiny Gleam package with one module importing another, then use the CLI to inspect the imported function and a custom-type constructor from the caller file. (Before this PR, .gleam files were not supported end to end, so the new language should now produce searchable symbols and cross-file call/reference links that would have been missing.)
  • cli — Create a repo with a real src/*.gleam file and a generated _build/default/.../*.gleam file, run a full index, and confirm only the real source shows up in file/symbol results. (This PR adds default exclusion for Gleam build output; reverting it would cause generated _build files to be indexed.)
  • cli — Track a Gleam source file in git, index it, delete the file, run a full index again, and confirm the deleted file disappears from results without the run reporting an extraction error for that path. (The PR adds explicit coverage for tracked-but-deleted Gleam files during full indexing, so this behavior should differ from pre-change failure or stale-file retention.)
  • cli — Index a small project containing a Gleam external declaration and a matching Erlang module, then inspect the caller/callee or explore output; repeat once with the Erlang function using the wrong arity and confirm the cross-language link disappears. (This PR introduces structured unresolved-ref metadata plus arity-based Erlang FFI resolution, so only the matching-arity case should now link across languages.)
Blast radius: 228 files affected beyond the diff · 1182 symbols · 141 test files selected

Tests to run:

  • __tests__/adaptive-explore-sizing.test.ts
  • __tests__/android-res-exclusion.test.ts
  • __tests__/arkts-resolution.test.ts
  • __tests__/batched-ref-cleanup.test.ts
  • __tests__/c-fnptr-kernel-sweep.test.ts
  • __tests__/c-fnptr-synthesizer.test.ts
  • __tests__/celery-dispatch-synthesizer.test.ts
  • __tests__/cfml-inheritance-resolution.test.ts
  • __tests__/cfml-receiver-inference.test.ts
  • __tests__/cli-affected-paths.test.ts
  • __tests__/cli-no-color.test.ts
  • __tests__/cli-node-command.test.ts
  • __tests__/cli-query-command.test.ts
  • __tests__/closure-collection-synthesizer.test.ts
  • __tests__/concurrent-locking.test.ts
  • __tests__/config-secret-redaction.test.ts
  • __tests__/context-ranking.test.ts
  • __tests__/context.test.ts
  • __tests__/cooperative-yield.test.ts
  • __tests__/daemon-bind-failure.test.ts

Full report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants