Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions docs/design/specs/js-language-namespace.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Per-module language namespace in `can://` ids

- **Status:** accepted, implementation pending
- **Scope:** `codeanalyzer-typescript`; schema v2 **id-shape change** (breaking for JS modules)
- **Tracking:** see the work item filed alongside this spec

## Problem

`src/schema/ids.ts:11` hardcodes `const LANGUAGE = "typescript"`, and every code id descends from it:

```
applicationIdOf(app) -> can://typescript/<app>
moduleIdOf(appId, fileKey) -> <appId>/<fileKey>
idFromSig(moduleId, prefix, sig)-> <moduleId>/<tail>
```

A JavaScript file therefore emits `can://typescript/<app>/lib/foo.js/fn`. The first `can://`
segment is defined as a language namespace, so a `.js` module is currently labelled as TypeScript.
The analyzer handles both languages — JS discovery landed in #98 — and nothing in the id records
which one a module actually is.

Measured: on `nodejs/node/lib` (406 modules, pure JavaScript) every id reads `can://typescript/…`.

This also sits against the repository-artifact layer, which deliberately went language-NEUTRAL
(`can://artifact/<app>/<path>`) so sibling analyzers over one repository mint identical ids for the
same file. Code nodes make the opposite choice and encode a language that may be wrong.

## Contract-impact triage

| Question | Answer |
| --- | --- |
| Schema v2 output | **id shape changes** for `.js`/`.jsx`/`.mjs`/`.cjs` modules and every descendant id |
| Levels / monotonicity | unaffected — ids change, structure does not |
| Repos | `codeanalyzer-typescript`; consumers of JS ids (Neo4j stores, caches, saved queries). No python change: python is single-language and has no analogue |
| schema_version | unchanged by standing decision — the contract re-baselines once every analyzer's schema is stable, not per release |
| Shared vocabulary | adds `javascript` as a language namespace alongside `typescript`; no new prov token, no new node or edge kind |

## Decision

**Per-module language in the id. The application anchor stays `can://typescript/<app>`.**

```
can://typescript/<app> :Application anchor (unchanged)
can://typescript/<app>/src/bar.ts/fn TypeScript module and descendants
can://javascript/<app>/lib/foo.js/fn JavaScript module and descendants
```

Extension mapping:

| Extension | Namespace |
| --- | --- |
| `.ts` `.tsx` `.mts` `.cts` `.d.ts` | `typescript` |
| `.js` `.jsx` `.mjs` `.cjs` | `javascript` |

### Accepted inconsistency

The application id keeps saying `typescript` while owning `javascript` children. This was chosen
knowingly over the two alternatives:

- A **neutral application anchor** (`can://app/<app>`, mirroring the artifact layer) is the more
coherent end state, but changes *every* id in every projection rather than only JS ones.
- **Two application anchors** would break the single `:Application` invariant that carries analyzer
identity (issue #43).

A mixed repository has no single language, so any single-anchor scheme must either name one
language or name none. Naming the analyzer's own language is the smaller break today; moving to a
neutral anchor stays open as a follow-up.

## Consequences

- **Breaking for JS ids.** Neo4j `MERGE` keys on id, so a re-projection creates new nodes for JS
modules rather than updating existing ones; a store analyzed with an earlier version needs a
rebuild. Saved Cypher and any persisted id references to JS nodes must be updated.
- **`modulePrefixOf` collision risk is unchanged.** It strips `.ts` and `.js` alike, so `a.ts` and
`a.js` still share a module prefix. That collision is prevented upstream by discovery's sibling
rule (a `.js` beside a real same-prefix `.ts` is treated as compiled output and skipped), not by
the id grammar. Splitting the namespace does not fix it and does not worsen it.
- **The cache is unaffected.** Ids are per-run (`assignIds` stamps them fresh because they embed
`--app-name`); the cached tree is id-free.
- **One construction site.** `assignIds.ts:54` is the only caller of `moduleIdOf`, so the change is
contained: `moduleIdOf` takes the app NAME and the file key and derives the namespace itself.

## Definition of done

- A `.js` module and its callables emit `can://javascript/…`; a `.ts` module emits
`can://typescript/…`; the `:Application` id is unchanged.
- The `.d.ts` case is asserted explicitly (it is `typescript`, and `.d.ts` must not be read as a
`.ts` suffix on a `.d` file).
- Neo4j projection and JSON agree, and the conformance and count-parity gates stay green.
- A fixture with both a `.ts` and a `.js` module asserts both namespaces in one run.
2 changes: 1 addition & 1 deletion src/schema/assignIds.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ export function assignIds(app: AnalysisInternal, appName: string): AssignedIds {
};

for (const [fileKey, mod] of Object.entries(app.symbol_table)) {
const moduleId = moduleIdOf(appId, fileKey);
const moduleId = moduleIdOf(appName, fileKey);
const modulePrefix = modulePrefixOf(fileKey);
mod.id = moduleId;
// Module-scope execution is a call-graph SOURCE (python #131 parity: a call in module scope
Expand Down
23 changes: 21 additions & 2 deletions src/schema/ids.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,32 @@
*/

const LANGUAGE = "typescript";
const JS_EXTS = /\.(jsx|js|mjs|cjs)$/;

/**
* A module's language namespace, from its file key (#114).
*
* The analyzer owns both languages (JS discovery, #98), so this is per MODULE rather than per run:
* a `.js` file is JavaScript and must not be labelled `typescript`. `.d.ts` falls through to the
* default rather than matching, so a declaration file stays TypeScript.
*/
export function languageOf(fileKey: string): string {
return JS_EXTS.test(fileKey) ? "javascript" : LANGUAGE;
}

/**
* The :Application anchor keeps the analyzer's own language even when it owns `javascript`
* children. A mixed repository has no single language, and the alternatives were a neutral anchor
* (moves every id in every projection) or two anchors (breaks the single-anchor invariant, #43).
* See docs/design/specs/js-language-namespace.md.
*/
export function applicationIdOf(appName: string): string {
return `can://${LANGUAGE}/${appName}`;
}

export function moduleIdOf(appId: string, fileKey: string): string {
return `${appId}/${fileKey}`;
/** Takes the app NAME, not the app id: a module's namespace is its own, not the application's. */
export function moduleIdOf(appName: string, fileKey: string): string {
return `can://${languageOf(fileKey)}/${appName}/${fileKey}`;
}

/** The module/signature prefix: the file key without its TS/JS extension. */
Expand Down
12 changes: 7 additions & 5 deletions test/checker-guard.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ import { checkerFailures, resetCheckerFailures, symbolAt } from "../src/schema/c
import type { AnalysisOptions } from "../src/options";

const FIXTURE = path.resolve(import.meta.dir, "fixtures/unresolvable-js-app");
const ID = "can://typescript/unresolvable-js-app";
// Two namespaces in one run (#114): a .js module is `javascript`, a .ts module is `typescript`.
const TS = "can://typescript/unresolvable-js-app";
const JS = "can://javascript/unresolvable-js-app";

function options(over: Partial<AnalysisOptions> = {}): AnalysisOptions {
return {
Expand All @@ -43,8 +45,8 @@ describe("a .js file outside tsconfig's include still resolves (#103)", () => {
expect(js).toBeDefined();
expect(Object.keys(js!.types ?? {})).toContain("defaultDpapi");
const methods = Object.values((js!.types ?? {})["defaultDpapi"]?.callables ?? {}).map((c) => c.id);
expect(methods).toContain(`${ID}/mocks/dpapi.js/defaultDpapi/protectData`);
expect(methods).toContain(`${ID}/mocks/dpapi.js/defaultDpapi/unprotectData`);
expect(methods).toContain(`${JS}/mocks/dpapi.js/defaultDpapi/protectData`);
expect(methods).toContain(`${JS}/mocks/dpapi.js/defaultDpapi/unprotectData`);
});

test("its call edges resolve — with allowJs off, the checker threw before reaching them", async () => {
Expand All @@ -53,9 +55,9 @@ describe("a .js file outside tsconfig's include still resolves (#103)", () => {

// Module-scope `new defaultDpapi()`, attributed to the MODULE. This edge does not merely go
// missing without the fix — resolving it is what threw.
expect(edges).toContain(`${ID}/mocks/dpapi.js -> ${ID}/mocks/dpapi.js/defaultDpapi/constructor`);
expect(edges).toContain(`${JS}/mocks/dpapi.js -> ${JS}/mocks/dpapi.js/defaultDpapi/constructor`);
// The healthy TypeScript file is unaffected either way.
expect(edges).toContain(`${ID}/src/index.ts/run -> ${ID}/src/index.ts/greet`);
expect(edges).toContain(`${TS}/src/index.ts/run -> ${TS}/src/index.ts/greet`);
});

test("nothing is skipped any more — the checker resolves the file cleanly", async () => {
Expand Down
38 changes: 38 additions & 0 deletions test/ids.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/**
* The `can://` language namespace is per MODULE (#114).
*
* The analyzer owns both languages (JS discovery, #98), so a `.js` module must not be labelled
* `typescript`. The :Application anchor deliberately keeps the analyzer's own language even when it
* owns `javascript` children — a mixed repository has no single language, and the alternatives
* moved every id in every projection or broke the single-anchor invariant (#43).
* See docs/design/specs/js-language-namespace.md.
*/
import { describe, expect, test } from "bun:test";
import { applicationIdOf, languageOf, moduleIdOf } from "../src/schema/ids";

describe("per-module language namespace (#114)", () => {
test.each([["a.ts"], ["a.tsx"], ["a.mts"], ["a.cts"]])("%s is typescript", (f) => {
expect(languageOf(f)).toBe("typescript");
});

test.each([["b.js"], ["b.jsx"], ["b.mjs"], ["b.cjs"]])("%s is javascript", (f) => {
expect(languageOf(f)).toBe("javascript");
});

// A declaration file is TypeScript. The suffix must not be read as `.ts` on a file named `a.d`.
test("a .d.ts declaration file is typescript", () => {
expect(languageOf("types/x.d.ts")).toBe("typescript");
expect(moduleIdOf("app", "types/x.d.ts")).toBe("can://typescript/app/types/x.d.ts");
});

// The match is anchored: a DIRECTORY named `foo.js` must not make its .ts children javascript.
test("a directory named *.js does not change its children's namespace", () => {
expect(languageOf("vendor.js/index.ts")).toBe("typescript");
});

test("module ids carry their own namespace; the application anchor keeps typescript", () => {
expect(applicationIdOf("app")).toBe("can://typescript/app");
expect(moduleIdOf("app", "src/bar.ts")).toBe("can://typescript/app/src/bar.ts");
expect(moduleIdOf("app", "lib/foo.js")).toBe("can://javascript/app/lib/foo.js");
});
});
Loading