Skip to content

feat: build completion on a cursor context API of the parser - #267

Merged
joe-re merged 1 commit into
releasefrom
feat/completion-context
Sep 27, 2026
Merged

joe-re merged 1 commit into
releasefrom
feat/completion-context

Conversation

@joe-re

@joe-re joe-re commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Summary

Moves sql-language-server completion off the syntax-error workaround onto a new cursor context API of the parser.

Before

complete() parsed the SQL truncated at the cursor and relied on the failure: keywords came from the error's expected, tables from a second parseFromClause over the whole text, broken FROM subqueries were completed by recursing on their text, and column-name positions were detected by an EXPECTED COLUMN NAME error message. When the truncated SQL happened to parse, a separate path produced a different set of candidates.

After

getCompletionContext(sql, offset) in @joe-re/sql-parser returns:

  • keywords expected at the cursor, and roles: table, column (expression), columnName (INSERT column list, UPDATE ... SET, DROP COLUMN, ...), alias, newName
  • word being typed and its qualifier (u in u.na|, including quoted names and subscripts)
  • tables visible from the innermost query at the cursor (FROM / JOIN, subqueries with their columns) or the target of INSERT / UPDATE / DELETE / ALTER TABLE, and ctes with their columns
  • inCommentOrString, statementType, errorBeforeCursor

The parser records the roles next to its expectations (they never appear in syntax errors). When the SQL before the cursor is broken, the context is computed again from the clause that contains the cursor (SELECT, FROM, JOIN, ON, WHERE, HAVING, SET, BY, or an opening parenthesis), e.g. SELECT jo. FROM employees jo JOIN jobs j ON jo.| still completes columns.

complete() now only maps roles to the existing candidate builders: tables where a table can appear, functions / aliases / scoped columns where an expression can appear, and target-table columns for column-name positions.

Behavior changes

  • Tables and functions are no longer offered everywhere (e.g. right after FROM t )
  • Only the statement at the cursor is used (tables of another statement no longer leak in)
  • INSERT / UPDATE / ALTER TABLE column names come from the target table instead of all tables
  • CTE names and columns are offered
  • ; is not offered as a candidate

Evaluation

  • All existing server tests pass unchanged (128), sqlint 36, sql-parser 117 (18 new context tests)
  • All complete() calls made by the server tests on feat(sql-parser)!: reimplement the parser by hand in TypeScript #266 were recorded and replayed: 103/106 identical; 2 differ only in order and 1 is an artifact of the recording (the old createFunctionCandidates mutated the shared schema, fixed here)
  • Next-word hit rate: for every word of the SQL used in the test suites, at the position right before it and in the middle of it (4,114 positions), is the word that actually comes next among the candidates?
    • release (old parser + old completion): 41.4%
    • feat(sql-parser)!: reimplement the parser by hand in TypeScript #266 (new parser + old completion): 43.9%
    • this PR: 48.8%
    • The remaining positions where only release hit are artifacts (e.g. the old code used the first statement's FROM for a second statement) or suggesting an alias the user is currently defining
  • LSP stdio sessions give the same diagnostics, code actions and completion as feat(sql-parser)!: reimplement the parser by hand in TypeScript #266

Test plan

  • build / lint / typecheck / vsc-compile / all tests
  • replay of recorded completion calls, hit-rate comparison, LSP stdio sessions
  • VS Code e2e (CI)

Base automatically changed from feat/v2-parser to release September 27, 2026 23:16
sql-language-server completed by parsing the SQL truncated at the cursor
and reading the `expected` list of the resulting syntax error, plus a
separate FROM clause parser, a recursive completion for broken subqueries
and the EXPECTED COLUMN NAME message. It is replaced by
getCompletionContext(sql, offset) in @joe-re/sql-parser.

Parser:
- Record roles (table, expression, column name, alias, new name) along
  with the expected keywords; they are not exposed in syntax errors
- getCompletionContext() returns the expected keywords and roles at the
  cursor, the word being typed and its qualifier, the statement type, the
  tables visible from the innermost query (FROM / JOIN, or the target of
  INSERT / UPDATE / DELETE / ALTER TABLE) and the CTEs
- When SQL before the cursor is broken, parse again from the clause that
  contains the cursor (SELECT, FROM, JOIN, ON, WHERE, HAVING, SET, BY or
  an opening parenthesis)

Server:
- Offer tables where a table can appear, columns / functions / aliases
  where an expression can appear, and target table columns for INSERT,
  UPDATE SET and ALTER TABLE column names
- Offer CTE names and columns
- Fix createFunctionCandidates mutating the shared schema
- Remove helpers that located AST nodes by position
@joe-re
joe-re force-pushed the feat/completion-context branch from bc04f3f to 6bfe91b Compare September 27, 2026 23:18
@joe-re
joe-re merged commit f75e157 into release Sep 27, 2026
5 checks passed
@joe-re
joe-re deleted the feat/completion-context branch September 27, 2026 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant