diff --git a/CHANGELOG.md b/CHANGELOG.md index b09daac8..cba44969 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,19 @@ All notable changes to EigenScript are documented here. ## [Unreleased] +### Breaking changes + +- **File resolution is independent of the process working directory (#1056).** + `load_file` and `import` search the containing file's directory, the existing + `eigs_modules` walk, the nearest `eigs.json` project root, then the stdlib + locations. Absolute paths remain as-is. The bare cwd step and one-parent + fallback are removed; consumers using root-relative paths from subdirectory + files need an `eigs.json` at their project root. Nested loads and functions + retain the containing file's directory. Errors identify the roots tried. + Imported top-level `for` bodies now retain their plain `is` bindings in the + module, matching main and load_file; the loop binder remains loop-scoped. + `tools/road_diff.sh` gates all three roads and proves its own failure paths. + ### Added - **`is_file of path` (#1058).** 1 iff the path names a REGULAR file @@ -28,7 +41,7 @@ All notable changes to EigenScript are documented here. hand-written expectations (the one-sided-verifiability class), which is how #1063's JIT half — an inline `SET_LOCAL` that recorded no history — went unnoticed. Every corpus program now runs with the JIT off recording a tape, - then replays that tape under the default JIT and under forced OSR + then replays that tape under the default JIT and with the OSR threshold lowered (`EIGS_JIT_OSR_THRESHOLD=1`); stdout, stderr and exit code must be byte-identical. A divergence is adjudicated by determinism first (both sides rerun; a stable pair is the JIT's) and by tape replay second (the @@ -42,6 +55,32 @@ All notable changes to EigenScript are documented here. ### Fixed +- **Imported loop locals and runtime `eval` keep their scope and file (#1056).** + A plain `is` inside an imported module's `for` first updates an existing + loop-local, while fresh bindings remain in the module. The original f29 + fixtures stopped native compilation at `LOOP_ENV_CLEAR`; their lowered-threshold + runs did not test native writes. Six new `native_*` road fixtures now run + under interpreter, default JIT, and a lowered OSR threshold on x86-64. + The same compiled chunk is exercised with the OSR threshold lowered; both + native arms compile code and must agree. Statistics do not count OSR entries + or prove that the two native arms use distinct entry mechanisms. + For `native_inline` on x86-64: interpreter `scanned=0 compiled=0`, both + native tiers `scanned=2 compiled=1`. Creating a local mid-thunk previously + yielded 19999 instead of 14999; imported inline stores now use the helper + whenever an intervening scope can hold a nearer binding. + Runtime `eval` in a function retains the function's defining directory + during another module's import and `eigs_eval_file`. The embed API now + scopes its directory override to compilation, matching import and load_file. + `tools/embed_roads.py` checks both embed eval APIs and the override during + execution. Its C test reuses a uniquely identified make variant, or builds + from the plain source list when the CLI is standalone (build.sh) or its + variant is ambiguous; ASAN_OPTIONS requests an ASan fallback. It never + relinks the CLI. Selftests cover these layouts, an execution-scope override, + invalid/duplicate native headers, and the lowered threshold reaching the child. + The oracle requires completion after readback and names malformed metadata; + its selftest rejects early-exit snapshot forgery. Generated test modules use + their test file's canonical directory in both in-tree and installed layouts. + - **A descriptor reading observer state of an unrecorded host binding raises instead of answering a rest value (#1027).** With the #915 gate closed for the host program (nothing compiled into it reads the observer), a chunk run @@ -136,7 +175,7 @@ All notable changes to EigenScript are documented here. (leaving the call-site ip there resumed the interpreter misaligned — a constants[-1] read after an OSR'd loop called `adler32`). Pinned by `tests/test_host_frame_line.eigs`. #1071 was the same stale read seen from - the JIT side (interpreter, JIT and forced-OSR each printed a different wrong + the JIT side (interpreter, default JIT and lowered-threshold JIT each printed a different wrong host line for `test_sandbox_budget`); the three tiers now agree, and its row leaves `tests/jit_diff_expected.txt`, which is empty. Found by the AOT's byte-exact corpus: the compiled program printed the correct line and the VM @@ -6479,7 +6518,7 @@ that don't shadow your model, and a `menu_bar` that owns its own z-order. the *enclosing* loop's body, compiled against the enclosing loop's stack frame — but the thunk had entered mid-nest at the inner header, so running that code natively read the wrong (reserved-null / stale) stack slots and - corrupted execution. It surfaced only under forced OSR + corrupted execution. It surfaced only with the OSR threshold lowered (`EIGS_JIT_OSR_THRESHOLD=1`) on nested-loop, index-heavy programs (e.g. the dynamics lab's Gauss–Seidel solver) as a nondeterministic `cannot index num` (a null `INDEX_GET` target), `index must be an integer`, or double diff --git a/Makefile b/Makefile index 8f5b64a8..2114c2a5 100644 --- a/Makefile +++ b/Makefile @@ -70,7 +70,7 @@ define AUX_REFRESH done endef -.PHONY: all build full http net gfx zlib lib amalgamation tsan test sandbox-intern-test install install-gfx clean coverage coverage-clean fuzz fuzz-run lsp dap jit-smoke embed-smoke embed-smoke-gfx embed-concurrent asan valgrind pgo poison freestanding-check freestanding-libc-diff asan-http asan-gfx nativefn-test print-% +.PHONY: all build full http net gfx zlib lib amalgamation tsan test sandbox-intern-test install install-gfx clean coverage coverage-clean fuzz fuzz-run lsp dap jit-smoke embed-smoke embed-smoke-gfx embed-concurrent asan valgrind pgo poison freestanding-check freestanding-libc-diff asan-http asan-gfx nativefn-test embed-roads print-% # ---- Per-variant objdir engine (#740) ------------------------------------- # The engine's rules are defined before `all`, so pin the default goal. @@ -212,6 +212,14 @@ $(NATIVEFN_TEST): $(NATIVEFN_TEST_OBJ) $(filter-out build/release/main.o build/r nativefn-test: $(NATIVEFN_TEST) @echo "Native-fn identity test built: $(NATIVEFN_TEST)" +# #1056: use the same variant as the CLI under test, without relinking it. +ROAD_VARIANT ?= release +EMBED_ROADS_OBJ := $(filter-out build/$(ROAD_VARIANT)/main.o,$(OBJ_$(ROAD_VARIANT))) +build/$(ROAD_VARIANT)/embed_roads: $(SRC_DIR)/embed_roads.c $(EMBED_ROADS_OBJ) $(wildcard $(SRC_DIR)/*.h) Makefile + $(CC) $(FLAGS_$(ROAD_VARIANT)) -I$(SRC_DIR) -o $@ $< $(EMBED_ROADS_OBJ) $(LIBS_$(ROAD_VARIANT)) +embed-roads: build/$(ROAD_VARIANT)/embed_roads + @echo "Embed road test built: $<" + full: build/full/eigenscript $(call RELINK,full) @echo "EigenScript $(VERSION) (full) built. Binary: $$(du -sh build/full/eigenscript | cut -f1)" diff --git a/README.md b/README.md index f78500a8..d5fe910e 100644 --- a/README.md +++ b/README.md @@ -352,6 +352,15 @@ define double as: # functions take one argument, n doubled is map of [[1, 2, 3], double] # [2, 4, 6] ``` +File loading is relative to the containing file, then the `eigs_modules` walk, +then the nearest `eigs.json` project root, then stdlib locations; absolute paths +are used as-is. There is no process cwd search. The REPL (including piped input) +and the embed API without a file path use their working directory as the base. +Add `eigs.json` at the root +when subdirectory files use root-relative paths. See the exact +[shared import/load_file resolution chain](docs/SPEC.md#modules). + + See [docs/STDLIB.md](docs/STDLIB.md) for the full library guide — start at its **"Finding Things"** index ("I need to..." → module) so you reach for `stats.median` or `data.df_group_by` instead of hand-rolling it. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index a15c608b..cb01e9c5 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -256,10 +256,13 @@ The minimal build (`make build`) sets all flags to 0. The full build ## Standard Library The 77 modules in `lib/` are pure EigenScript — no C code. They are loaded at -runtime via `load_file of "lib/module.eigs"`. Path resolution searches in -order: the current working directory, the script file's directory, the script's -parent directory, directories relative to the executable (`exe_dir/..` and the -installed stdlib beside it), then `~/.local/lib/eigenscript`. +runtime via `load_file of "lib/module.eigs"`. Both loaders use absolute paths +as-is; relative paths search the containing file's directory, the `eigs_modules` +walk, the nearest `eigs.json` project root, then the executable-relative and +HOME stdlib roots. Nested loads and deferred functions retain their own file's +directory. There is no process cwd or one-parent fallback; only the REPL +(including piped input) and embedding without a file path use their working +directory as the base. See [the ordered chain](SPEC.md#modules). The meta-circular interpreter (`lib/eigen.eigs`) implements tokenization, parsing, and evaluation of EigenScript source code in EigenScript itself. diff --git a/docs/BUILTINS.md b/docs/BUILTINS.md index 73e0853f..191f46b1 100644 --- a/docs/BUILTINS.md +++ b/docs/BUILTINS.md @@ -304,7 +304,7 @@ Boolean keywords that check the most recently observed value: | Name | Signature | Description | |------|-----------|-------------| -| `load_file` | `load_file of "path.eigs"` | Load and execute EigenScript file. A missing/unreadable path raises a catchable `io` error (matching `import`); a parse/compile failure in the file raises `parse`. | +| `load_file` | `load_file of "path.eigs"` | Execute a file in the current scope, yielding its top-level return value. Uses the same file-based resolution chain as `import` (below). A missing/unreadable path raises a catchable `io` error naming the roots tried; a parse/compile failure raises `parse`. | | `file_exists` | `file_exists of "path"` | 1 if the path exists (any kind: file, directory, device, fifo), 0 otherwise. A `stat` probe — never blocks (#1070: the old `fopen` probe hung on a reader-less fifo). Trace-recorded, so replay is deterministic (#585) | | `is_dir` | `is_dir of "path"` | 1 if the path names a directory, 0 for a plain file / missing path (#576 — replaces the `file_exists of "path/."` probe). Trace-recorded, so replay is deterministic | | `is_file` | `is_file of "path"` | 1 iff the path names a REGULAR file (`S_ISREG`); 0 for a directory, a device/fifo/socket, a missing path, or a non-string. `read_file_util` admits only regular files, so this is the probe a driver uses to match that contract (#1058). Trace-recorded, so replay is deterministic | @@ -344,6 +344,19 @@ producing tensors too large to materialise in memory. | `stream_write` | `stream_write of value` | Append one float64 to the open stream. 1 on success, 0 on failure | | `stream_close` | `stream_close of null` | Close the stream. 1 on success, 0 on failure | +`load_file` and `import` resolve an absolute path as-is; otherwise they try the +containing file's directory, the `eigs_modules` walk (stopping at `eigs.json`), +the nearest `eigs.json` project root, then `/../`, +`/../lib/eigenscript/` and its leading-`lib/`-stripped form, then +`$HOME/.local/lib/eigenscript/` and its leading-`lib/`-stripped form. +`` is the executable's directory. There is no process cwd lookup or +one-parent fallback. The REPL (including piped input) and the embed API without +a file path use their working directory as the containing directory. Loaded files +and their functions retain their own directory. Consumers using root-relative +paths from subdirectory files need an `eigs.json` at their root. Errors name +the containing directory, project root (or `no eigs.json above `), and +stdlib roots. See [Modules](SPEC.md#modules) for import collision handling. + ## Path Manipulation | Name | Signature | Description | diff --git a/docs/COMPARISON.md b/docs/COMPARISON.md index 89e1b82c..7e144314 100644 --- a/docs/COMPARISON.md +++ b/docs/COMPARISON.md @@ -503,3 +503,28 @@ make the argument a single value, so the literal list arrives whole. A bare `of ["ada", "grace"]` would also work here — two arguments to a one-parameter function pack back into a list — but the parenthesised form says "one list" directly and works for any arity.) + +## File loading and block scope + +Unlike a working-directory-based include path, EigenScript resolves imports +and loads from the file containing the call, then the `eigs_modules` walk, +then the nearest `eigs.json` project root, then stdlib locations. Absolute +paths are used as-is. There is no process cwd search; the REPL (including piped +input) and the embed API without a file path use their working directory as the +containing directory. A function retains its defining file's directory through +`eval`, including calls made while another module is being imported or an +embedding host is executing `eigs_eval_file`. The complete chain is in +[SPEC, Modules](SPEC.md#modules). + +Main, import and load_file share these rules: a `for` binder is loop-scoped +and never writes an outer binding. Plain `is` in its body updates the nearest +existing binding, including a loop-local; otherwise it creates a binding in +the enclosing scope, like other blocks. In an imported module this search +stops at the module boundary, preserving the importer's bindings. Top-level `return` ends the file: +load_file yields its value, import finishes its namespace, and main discards +its value. + +The existing function-slot exception remains: a binder with no prior binding +inside a function retains its final value after the loop on every road. A +pre-existing parameter or local is restored. This change preserves that +exception; see the scope notes in LANGUAGE_CONTRACT.md. diff --git a/docs/EMBEDDING.md b/docs/EMBEDDING.md index b012b596..2550ea13 100644 --- a/docs/EMBEDDING.md +++ b/docs/EMBEDDING.md @@ -121,9 +121,13 @@ accumulate across calls and are visible to `eigs_get_global`. Returns a counted ref to the script's last expression value, or `NULL` on parse / runtime error. On error, `eigs_last_error_message()` returns -the most recent message. `eigs_eval_file` also updates `script_dir` so -`import` / `load_file` inside the source resolves relative paths against -the file's directory. +the most recent message. `eigs_eval_file` compiles with the named file's +canonical containing directory. +The override ends before execution; each helper, including a helper that calls +`eval`, retains its own defining file's directory. Subsequent `eigs_eval_string` +calls without a file use the working directory when no script frame is +executing, and can call previously loaded functions without changing those functions' provenance. The shared search chain +is documented in [SPEC, Modules](SPEC.md#modules). ```c EigsValue *r = eigs_eval_string("greeting is \"hi\"\n3 * 14"); diff --git a/docs/LANGUAGE_CONTRACT.md b/docs/LANGUAGE_CONTRACT.md index ac435d2a..4975ac76 100644 --- a/docs/LANGUAGE_CONTRACT.md +++ b/docs/LANGUAGE_CONTRACT.md @@ -76,10 +76,9 @@ bound only its message string.) **Promise:** `import name` executes the module once and binds its top-level definitions as a **dict named `name`** — nothing enters the importing scope besides that one binding, and module names starting -with `_` are private (omitted from the dict). Resolution order: -`lib/name.eigs` (the standard library) first, then `name.eigs` -script-relative and the other standard locations; the not-found error -names both tried paths. `load_file of "path.eigs"` is the +with `_` are private (omitted from the dict). Import tries `name.eigs` +before `lib/name.eigs`, warns on a project/stdlib collision, and chooses +the project file. `load_file of "path.eigs"` is the non-namespaced form: it executes the file directly in the current scope. **Module functions never write the loader's bindings** (issue #373): a module function's bare assignment to a name that isn't its @@ -92,7 +91,34 @@ fields. A **parse error** in a loaded file (via `import`, `load_file`, or `eval`) raises a catchable runtime error rather than silently executing a partial AST — consistent with the **Errors** promise. -**Status:** Enforced — `tests/test_import.eigs`, +**One file, three roads (main / import / load_file, #1056):** + +- Resolution belongs to the file containing the call, including nested loads + and `eval` inside functions, even when called during another module's + import, load, or `eigs_eval_file`. The defining file remains the base. The shared chain is: absolute + path as-is; containing directory; the `eigs_modules` walk; project root + (nearest ancestor, including that directory, with `eigs.json`); executable + and HOME stdlib locations. There is no process cwd search or one-parent + fallback. The REPL (including piped input) and the embed API without a file + path use their working directory as the containing directory. The full ordered + stdlib chain and error contract are in [SPEC, Modules](SPEC.md#modules). +- A `for` binder is loop-scoped everywhere and never writes a same-named + outer binding. A `for` body's plain `is` updates the nearest existing + binding, including a loop-local; otherwise it creates in the enclosing scope + like `if`, `loop while`, and `try`, on every road. An imported module's + search stops at its boundary, so fresh names appear in its namespace and + never write through to the importer. No function write boundary changes. +- A top-level `return value` ends the current file and yields its value, + skipping later statements. `load_file` returns it to the caller, who + continues; import finishes the module; the main program discards the value + and exits successfully. + +The existing function-slot exception remains: a binder with no prior binding +inside a function retains its final value after the loop on every road. A +pre-existing parameter or local is restored. This change preserves that +exception; see the scope notes in LANGUAGE_CONTRACT.md. + +**Status:** Enforced — `tools/road_diff.sh` and `tests/roads/`, `tests/test_import.eigs`, `tests/test_import_errors.eigs` (parse-error surfacing for `import` / `load_file` / `eval`) (stdlib + user modules, namespacing, `_` privacy, missing-module error), docs/SPEC.md Modules diff --git a/docs/OBSERVER.md b/docs/OBSERVER.md index 108f79d5..27c8a5f0 100644 --- a/docs/OBSERVER.md +++ b/docs/OBSERVER.md @@ -506,10 +506,14 @@ this program's earlier assignments ran — they have no recorded history... ``` You will see this if a program **rewrites a module between the compile and the -load**, or creates a file that **shadows** the one the compile-time scan -resolved (resolution tries the cwd before the script directory), or `chdir`s so -the same literal path resolves elsewhere. All three are the same shape: the file -the gate inspected is not the file that ran. +load**, or creates a nearer file that **shadows** the one the compile-time scan +resolved. Both loaders search the containing file's directory, the +`eigs_modules` walk, the nearest `eigs.json` project root, then the executable +and HOME stdlib roots (absolute paths are used as-is). For example, a newly +created sibling can replace a project-root or stdlib target. Changing the +process working directory does not redirect a file's loads. The failure is +that the file the gate inspected is not the file that ran; see the +[shared resolution chain](SPEC.md#modules). Re-run with `EIGS_OBS_FORCE=1` to disable the gate for that program. That is always safe — it restores the pre-gate behaviour exactly. diff --git a/docs/PACKAGE_DESIGN.md b/docs/PACKAGE_DESIGN.md index 4ad759c3..bc2648c6 100644 --- a/docs/PACKAGE_DESIGN.md +++ b/docs/PACKAGE_DESIGN.md @@ -31,26 +31,22 @@ namespace), native/C extensions in packages, build steps, and version constraint *solving* (pin exact versions; a solver can come later if real projects demand ranges). -## Current state (as-built, 0.13.0) - -- `import name` tries `lib/name.eigs` (the stdlib), then `name.eigs`, - each through the full resolution chain in `resolve_eigenscript_file` - (builtins.c): cwd → `$script_dir` → `$script_dir/..` → `$exe_dir/..` - → `$exe_dir/../lib/eigenscript` → `~/.local/lib/eigenscript`. - Public top-level names bind into a dict named `name`; `_`-prefixed - names stay private ([SPEC.md — Modules](SPEC.md#modules)). -- **Every `import` re-executes the module.** There is no module cache: - two importers get two copies of the module's state, and a diamond - (app → a → c, app → b → c) would run `c` twice with divergent state. -- Resolution is anchored to the **main script's** directory - (`g_script_dir` is global). A module imported from another directory - resolves *its* imports relative to the app, not itself — harmless - today (modules sit next to the script), wrong for packages. -- `lib/name.eigs` is tried before `name.eigs`, so the stdlib shadows - user modules of the same name — but a project-local `lib/` directory - shadows the *installed* stdlib (the chain hits `cwd/lib/` first). - This is how the repo runs its own tests; it's also an existing - footgun the package design must not widen. +## Runtime resolution (shipped, #1056) + +- `import name` requests `name.eigs`, then `lib/name.eigs`; project modules + take precedence over stdlib matches and collisions warn. Public names bind + into the module namespace. Import caching is described in + [SPEC.md — Modules](SPEC.md#modules). +- `import` and `load_file` share `resolve_eigenscript_file_from_ex` + (`builtins_host.c`). Absolute paths are used as-is. Relative paths search + the containing file's directory → the `eigs_modules` walk → the nearest + `eigs.json` project root → executable-relative and HOME stdlib roots. + There is no process cwd or one-parent fallback. A project-local `lib/` + can answer through the containing-directory or project-root steps before + the installed stdlib; it does not depend on where the process was launched. +- Nested loaded files and functions called after loading retain their own + containing directory. The REPL (including piped input) and the embed API + without a file path use their working directory as the base. ## Design @@ -95,42 +91,27 @@ dependencies are resolved by the tool into the **app's** flat `eigs_modules/` — one version of a name per project; two pins that disagree are an error naming both requirers, not a silent pick. -### Runtime change 1: one resolver step - -`import name` gains one step. Proposed order: - -1. `lib/name.eigs` — stdlib first, **unchanged** -2. `eigs_modules/name/name.eigs` — searched from the importing file's - directory upward to the project root (so packages find *their* - dependencies in the app's flat `eigs_modules/`) -3. `name.eigs` script-relative — unchanged - -Stdlib-first means a future stdlib module can collide with an existing -package name; the tool errors at `add` time when a dep name matches a -stdlib module, and package naming guidance is "prefix it" (`alice_vec`, -not `vec`). The alternative (packages shadow stdlib) trades that -papercut for a supply-chain hole — a dep silently becoming your `math` -— and loses. - -### Runtime change 2: import becomes cached and module-relative - -Two prerequisites that are worth doing even if nothing else ships: - -- **Module cache**: first `import` of a resolved real path executes - the module; subsequent imports bind the same dict. Diamond deps - share one instance of module state. The cache holds counted refs - (Value dict + module Env) released at teardown — the closure-cycle - collector's ownership rules apply (every edge counted, walker + - `gc_clear_node` updated in lockstep). -- **Per-file resolution base**: an import executing inside a module - resolves relative paths against *that module's* directory, not the - main script's. `g_script_dir` becomes a stack (or a parameter - threaded through the import path), with `load_file` keeping its - current main-script-relative behavior for back-compat. - -Both are observable behavior changes (re-import today re-executes; -side-effecting modules can tell) — minor-version territory with -CHANGELOG + SPEC.md updates per the stability contract. +### Runtime resolver: implemented order + +Both loaders use the same chain, documented in full in +[SPEC.md — Modules](SPEC.md#modules): absolute path as-is; containing file's +canonical directory; `eigs_modules//.eigs` walking upward through +the nearest `eigs.json` directory; that project root; then +`/../`, `/../lib/eigenscript/` and its leading-`lib/`-stripped +form, followed by `$HOME/.local/lib/eigenscript/` and its stripped form. +The package walk stops at the project root. A project without an `eigs.json` +has no project-root-relative fallback. Project/package matches precede stdlib +roots; import collisions produce a warning. + +### Runtime cache and containing-file context + +- **Module cache**: first `import` of a resolved canonical path executes + the module; subsequent imports bind the same dict. Diamond dependencies + share one instance of module state. +- **Per-file resolution base**: imports and loads inside a module resolve + from that module's directory. The compiled source retains this directory + for nested files and functions called later. `load_file` follows the same + resolver as `import` while executing the file in the caller's scope. ### The tool: `eigenscript --pkg` diff --git a/docs/SPEC.md b/docs/SPEC.md index 75fb52c1..6a797754 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -1109,12 +1109,33 @@ rm of "spec_shapes.eigs" (In a project, the idiom is simply `import shapes` with `shapes.eigs` sitting next to `app.eigs`.) -An `import` inside a module resolves relative to *that module's* own -directory, not the main script's. A submodule can safely -`import its_peer` and the peer is looked up next to the importer, -flattening symlinks and `..` segments. The other steps in the resolver -chain (cwd, exe-relative, `$HOME/.local/lib/eigenscript`) are -unchanged. +`import` and `load_file` use one resolution chain. `import name` first +requests `name.eigs`, then `lib/name.eigs`; a project/stdlib collision warns +and uses the project file. For each request, the order is: + +1. An absolute path is used as-is. +2. Relative to the directory of the **file containing the call**, with + symlinks and `..` canonicalized. This is the loaded file's directory for + nested loads, and remains the defining file's directory inside a function, + including `eval` in that function while its caller is running through + `import`, `load_file`, or an embedding host's `eigs_eval_file` call. The entry + file's compile directory does not override a helper's runtime `eval`. +3. The `eigs_modules` walk described below. +4. Relative to the **project root**: the nearest ancestor of that containing + directory with an `eigs.json`, including the containing directory itself. + If none exists, this step is skipped. +5. The existing stdlib locations, in order: `/../`, + `/../lib/eigenscript/`, the latter again with a leading `lib/` + stripped, then `$HOME/.local/lib/eigenscript/` and its `lib/`-stripped + form. Here `` is the executable's directory. + +There is **no process cwd search step**, and no containing-directory-parent +fallback. The REPL (including piped input) and the embed API without a file +path use their working directory as the containing directory; this is the +only way the working directory enters resolution. Files using project-root +paths from subdirectories need an `eigs.json` at their root. Failed resolution +raises an `io` error naming the containing directory, project root (or +`no eigs.json above `), and stdlib roots tried. Project-local dependencies live under `eigs_modules//.eigs` at the project root (any directory containing `eigs.json`). The @@ -1147,6 +1168,23 @@ executes a file directly **in the current scope**. The standard library's helper modules (`lib/test.eigs`'s `assert_eq`, ...) are conventionally loaded this way. +The same file has the same block and return rules on all three roads (main, +`load_file`, `import`). A `for` binder is loop-scoped and never writes a +same-named outer binding. A `for` body's plain `is` updates the nearest existing +binding, including a `local` in the current or an enclosing loop. Otherwise it +creates a binding in the enclosing scope, like `if`, `loop while`, and `try`. +At an imported module's top level the search stops at the module boundary; +fresh bindings belong to the module and are exported normally, without writing +to the importer. +A top-level `return value` ends the current file, skipping all later statements: +`load_file` yields the value to its caller; import finishes its namespace; +the main program discards the value and exits successfully. + +The existing function-slot exception remains: a binder with no prior binding +inside a function retains its final value after the loop on every road. A +pre-existing parameter or local is restored. This change preserves that +exception; see the scope notes in LANGUAGE_CONTRACT.md. + **Module write boundary.** A loaded (or imported) module's *functions* can read the loader's globals and call its functions, but they can never bind a write through to them — a bare `name is expr` inside a diff --git a/docs/STDLIB.md b/docs/STDLIB.md index 02f62872..3aeb3612 100644 --- a/docs/STDLIB.md +++ b/docs/STDLIB.md @@ -95,13 +95,26 @@ load_file of "lib/math.eigs" load_file of "lib/list.eigs" ``` -Path resolution order: -1. Relative to the **current working directory** -2. Relative to the **script file's directory** -3. Relative to the **script file's parent directory** -4. Relative to the **EigenScript executable's parent directory** -5. Relative to the installed stdlib beside the executable -6. Relative to `~/.local/lib/eigenscript` +`load_file` and `import` share this path resolution order: + +1. Absolute paths are used as-is. +2. Relative to the **containing file's directory**, including nested loaded + files and functions called later. Symlinks and `..` are canonicalized. +3. Walk upward for `eigs_modules//.eigs` (bare module names), + stopping after checking the nearest directory containing `eigs.json`. +4. Relative to that **project root**, if there is one: the nearest ancestor + (including the containing directory) with an `eigs.json`. +5. `/../`, then `/../lib/eigenscript/`, then the latter + with a leading `lib/` stripped; `` means the executable's directory. +6. `$HOME/.local/lib/eigenscript/`, then with leading `lib/` stripped. + +There is no process cwd lookup and no containing-file-parent fallback. The +REPL (including piped input) and the embed API without a file path use their +working directory as the containing directory. Add an `eigs.json` at the root of a +project whose subdirectory files use root-relative paths. Failure raises an +`io` error listing the containing directory, project root (or its absence), +and stdlib roots tried. Import tries `name.eigs` before `lib/name.eigs` and +warns when a project file shadows a stdlib module. This means `load_file of "lib/math.eigs"` works whether you run the script from the project root, from an external project while using a source diff --git a/docs/SYNTAX.md b/docs/SYNTAX.md index 31f785f1..54355c6c 100644 --- a/docs/SYNTAX.md +++ b/docs/SYNTAX.md @@ -373,19 +373,21 @@ The loaded file's definitions are added to the global environment. ### Path resolution -For non-absolute paths, `load_file` searches (in order): +For non-absolute paths, `load_file` and `import` search (in order): -1. The path as given, relative to the current working directory. -2. `/` — relative to the script being executed. -3. `/../` — relative to the script's parent directory. +1. The directory of the file containing the call, including nested loaded + files and functions called later. +2. The `eigs_modules` walk, stopping at the nearest `eigs.json` directory. +3. That project root, if present, for project-root-relative paths. 4. `/../` — relative to the EigenScript binary. -5. `/../lib/eigenscript/` — installed stdlib layout. -6. `~/.local/lib/eigenscript/` — user-local stdlib fallback. - -The third step is what lets a script in `examples/` pick up `lib/foo.eigs` -without the caller having to `cd` to the repository root. The executable -relative steps let external projects use the source-tree or installed stdlib -without copying `lib/*.eigs` into each project. `..` segments +5. `/../lib/eigenscript/`, then with leading `lib/` stripped. +6. `$HOME/.local/lib/eigenscript/`, then with leading `lib/` stripped. + +There is no process cwd search or one-parent fallback. The REPL (including +piped input) and the embed API without a file path use their working directory +as the base. An `eigs.json` at the project root lets a subdirectory file use +project-root-relative paths. The executable-relative steps let external +projects use the source-tree or installed stdlib without copying it. `..` segments embedded in the `load_file` argument itself are resolved by the OS normally — there is no sandbox, so a script can read any file the invoking user can read. diff --git a/docs/llms.txt b/docs/llms.txt index 728bd159..cc384205 100644 --- a/docs/llms.txt +++ b/docs/llms.txt @@ -68,8 +68,10 @@ not any outer `n`. - Two `local` traps: each **sibling `if`/`elif`/`else` branch** needs its own `local` for a first assignment (a `local` in one branch doesn't run for another), and a first assignment inside a `loop while` body needs `local`. -- `if` blocks do NOT create a scope (assignments leak out). `for` loops DO - (body-only names and the loop variable stay local). Comprehension variables +- `if` blocks do NOT create a scope (assignments leak out). a `for` binder is loop-scoped + and never writes an outer binding; plain `is` in its body binds in the + enclosing scope, including an imported module. A fresh function-local + binder currently retains its slot after the loop (the existing exception). Comprehension variables DO leak. ## Reserved and soft keywords (cannot be plain variable names) diff --git a/install.sh b/install.sh index 4d32b0cc..318271ed 100755 --- a/install.sh +++ b/install.sh @@ -9,42 +9,45 @@ set -e cd "$(dirname "$0")" -mkdir -p ~/.local/bin -mkdir -p ~/.local/lib/eigenscript +# Override for isolated install-layout verification; the normal user prefix +# remains ~/.local. Keep HOME untouched when exercising a temporary install. +INSTALL_PREFIX="${EIGENSCRIPT_INSTALL_PREFIX:-$HOME/.local}" +mkdir -p "$INSTALL_PREFIX/bin" +mkdir -p "$INSTALL_PREFIX/lib/eigenscript" VERSION=$(cat VERSION) # Build and install minimal ./build.sh -cp src/eigenscript ~/.local/bin/eigenscript -chmod +x ~/.local/bin/eigenscript +cp src/eigenscript "$INSTALL_PREFIX/bin/eigenscript" +chmod +x "$INSTALL_PREFIX/bin/eigenscript" # Build and install the language server alongside it — the toolchain is one # artifact (the VS Code extension in editors/vscode/ auto-launches `eigenlsp`). ./build.sh lsp -cp src/eigenlsp ~/.local/bin/eigenlsp -chmod +x ~/.local/bin/eigenlsp -echo "Language server installed: ~/.local/bin/eigenlsp" +cp src/eigenlsp "$INSTALL_PREFIX/bin/eigenlsp" +chmod +x "$INSTALL_PREFIX/bin/eigenlsp" +echo "Language server installed: $INSTALL_PREFIX/bin/eigenlsp" # Install stdlib -cp -r lib/*.eigs ~/.local/lib/eigenscript/ -echo "Stdlib installed to ~/.local/lib/eigenscript/" +cp -r lib/*.eigs "$INSTALL_PREFIX/lib/eigenscript/" +echo "Stdlib installed to $INSTALL_PREFIX/lib/eigenscript/" # Build and install full only when explicitly requested. if [ "${1:-}" = "full" ]; then ./build.sh full - cp src/eigenscript ~/.local/bin/eigenscript-full - chmod +x ~/.local/bin/eigenscript-full + cp src/eigenscript "$INSTALL_PREFIX/bin/eigenscript-full" + chmod +x "$INSTALL_PREFIX/bin/eigenscript-full" echo "" echo "Installed:" - echo " ~/.local/bin/eigenscript (v$VERSION, minimal, $(du -sh ~/.local/bin/eigenscript | cut -f1))" - echo " ~/.local/bin/eigenscript-full (v$VERSION, with extensions, $(du -sh ~/.local/bin/eigenscript-full | cut -f1))" + echo " $INSTALL_PREFIX/bin/eigenscript (v$VERSION, minimal, $(du -sh "$INSTALL_PREFIX/bin/eigenscript" | cut -f1))" + echo " $INSTALL_PREFIX/bin/eigenscript-full (v$VERSION, with extensions, $(du -sh "$INSTALL_PREFIX/bin/eigenscript-full" | cut -f1))" else echo "" - echo "Installed: ~/.local/bin/eigenscript (v$VERSION, minimal)" - echo " ~/.local/bin/eigenlsp (v$VERSION, language server)" + echo "Installed: $INSTALL_PREFIX/bin/eigenscript (v$VERSION, minimal)" + echo " $INSTALL_PREFIX/bin/eigenlsp (v$VERSION, language server)" echo "Run './install.sh full' to also install eigenscript-full." fi echo "" -echo "Make sure ~/.local/bin is in your PATH:" -echo ' export PATH="$HOME/.local/bin:$PATH"' +echo "Make sure $INSTALL_PREFIX/bin is in your PATH:" +printf ' export PATH="%s/bin:$PATH"\n' "$INSTALL_PREFIX" diff --git a/src/builtins.c b/src/builtins.c index 67b6c10f..94d57c35 100644 --- a/src/builtins.c +++ b/src/builtins.c @@ -2450,8 +2450,16 @@ Value* builtin_json_path(Value *arg) { } +const char *eigs_current_file_dir(void) { + if (eigs_current && eigs_current->vm && g_vm.frame_count > 0) { + EigsChunk *chunk = g_vm.frames[g_vm.frame_count - 1].chunk; + if (chunk->src && chunk->src->resolve_dir) return chunk->src->resolve_dir; + } + return g_import_resolve_dir[0] ? g_import_resolve_dir : g_script_dir; +} + int resolve_eigenscript_file(const char *path, char *resolved, size_t resolved_cap) { - return resolve_eigenscript_file_from(g_script_dir, path, resolved, resolved_cap); + return resolve_eigenscript_file_from(eigs_current_file_dir(), path, resolved, resolved_cap); } diff --git a/src/builtins_host.c b/src/builtins_host.c index 08b1a8af..21c39e82 100644 --- a/src/builtins_host.c +++ b/src/builtins_host.c @@ -859,6 +859,58 @@ static int try_resolve_path(const char *candidate, char *resolved, size_t resolv return 1; } +/* Canonical file provenance: symlink entry points and nested loads agree with + * import. Heap-owned because this helper also runs on compiler paths. */ +char *eigs_file_directory(const char *path) { + char *dir = realpath(path, NULL); + if (!dir) dir = xstrdup(path); + char *slash = strrchr(dir, '/'); + if (slash == dir) dir[1] = '\0'; + else if (slash) *slash = '\0'; + else { free(dir); dir = xstrdup("."); } + return dir; +} + +static int parent_directory(char *dir) { + char *slash = strrchr(dir, '/'); + if (!slash || strcmp(dir, "/") == 0) return 0; + if (slash == dir) dir[1] = '\0'; + else *slash = '\0'; + return 1; +} + +/* The nearest eigs.json is the project boundary, for both package lookup + * and root-relative paths. No cwd or one-parent fallback participates. */ +static char *project_directory(const char *base) { + char *dir = realpath(base, NULL); + if (!dir) return NULL; + char *marker = xmalloc(strlen(dir) + sizeof("/eigs.json")); + do { + size_t dir_len = strlen(dir); + memcpy(marker, dir, dir_len); + memcpy(marker + dir_len, "/eigs.json", sizeof("/eigs.json")); + if (access(marker, F_OK) == 0) { free(marker); return dir; } + } while (parent_directory(dir)); + free(marker); + free(dir); + return NULL; +} + +void eigs_file_resolve_error(const char *operation, const char *base, + const char *path, int line) { + char *project = project_directory(base); + const char *home = getenv("HOME"); + rt_error(EK_IO, line, + "%s: cannot read '%s' (not found or unreadable); tried containing directory '%s', " + "eigs_modules walk, %s%s; stdlib roots '%s/../', " + "'%s/../lib/eigenscript', '%s/.local/lib/eigenscript' " + "(also stripping lib/; absolute paths are used as-is)", + operation, path, base, project ? "project root " : "no eigs.json above ", + project ? project : base, g_exe_dir, g_exe_dir, + home ? home : ""); + free(project); +} + /* Phase 0c: walk from `base` upward looking for * /eigs_modules//.eigs * at each level. Stop at the project root (a directory containing @@ -892,9 +944,7 @@ static int try_eigs_modules_walk(const char *base, const char *path, snprintf(marker, sizeof(marker), "%.4000s/eigs.json", cur); if (access(marker, F_OK) == 0) return 0; - char *slash = strrchr(cur, '/'); - if (!slash || slash == cur) return 0; - *slash = '\0'; + if (!parent_directory(cur)) return 0; } return 0; } @@ -915,21 +965,23 @@ int resolve_eigenscript_file_from_ex(const char *base, const char *path, if (origin) *origin = EIGS_RESOLVE_PROJECT; if (!path || !resolved || resolved_cap == 0) return 0; - if (!base || !base[0]) base = g_script_dir; + if (!base || !base[0]) base = eigs_current_file_dir(); if (path[0] == '/') { return try_resolve_path(path, resolved, resolved_cap); } - if (try_resolve_path(path, resolved, resolved_cap)) return 1; - - if (try_eigs_modules_walk(base, path, resolved, resolved_cap)) return 1; - snprintf(candidate, sizeof(candidate), "%.4000s/%.4000s", base, path); if (try_resolve_path(candidate, resolved, resolved_cap)) return 1; - snprintf(candidate, sizeof(candidate), "%.4000s/../%.4000s", base, path); - if (try_resolve_path(candidate, resolved, resolved_cap)) return 1; + if (try_eigs_modules_walk(base, path, resolved, resolved_cap)) return 1; + + char *project = project_directory(base); + if (project) { + snprintf(candidate, sizeof(candidate), "%.4000s/%.4000s", project, path); + free(project); + if (try_resolve_path(candidate, resolved, resolved_cap)) return 1; + } snprintf(candidate, sizeof(candidate), "%.4000s/../%.4000s", g_exe_dir, path); if (try_resolve_path(candidate, resolved, resolved_cap)) return 1; @@ -981,7 +1033,7 @@ Value* builtin_load_file(Value *arg) { /* #490: match import's severity — a missing path is a catchable io * error, not a stderr-warn + silent null (rc=0). Callers that ignore * the return otherwise run on half-initialized state. */ - rt_error(EK_IO, 0, "load_file: cannot read '%s'", arg->data.str); + eigs_file_resolve_error("load_file", eigs_current_file_dir(), arg->data.str, 0); return make_null(); } @@ -1040,9 +1092,8 @@ Value* builtin_load_file(Value *arg) { * target and compiled it then. The file it read and the file being compiled * now are two separate reads with the whole program running in between, so * they can differ: the program can rewrite the module (`write_text` then - * `load_file`), or create a file in the cwd that SHADOWS the one the - * pre-pass resolved (resolve_eigenscript_file tries cwd before the script - * dir). Both were executed and both produced a silently wrong answer — + * `load_file`), or create a nearer file that shadows the resolved target. + * Both shapes can otherwise produce a silently wrong answer — * `report of x` read `equilibrium` under the gate and `moving` without it. * * ASK THE ACTUAL QUESTION. A first draft compared the observer bit before @@ -1068,7 +1119,13 @@ Value* builtin_load_file(Value *arg) { /* ACQUIRE: this is the one read that pairs with eigs_obs_enable's * store ORDER (gap then needed) — see obs_flag_store in eigenscript.h. */ int obs_before_module = obs_flag_load_acquire(obs_needed); + char *saved_resolve_dir = xstrdup(g_import_resolve_dir); + char *loaded_dir = eigs_file_directory(abs_key); + snprintf(g_import_resolve_dir, sizeof(g_import_resolve_dir), "%s", loaded_dir); + free(loaded_dir); EigsChunk *lf_chunk = compile_ast(ast, target, source); + snprintf(g_import_resolve_dir, sizeof(g_import_resolve_dir), "%s", saved_resolve_dir); + free(saved_resolve_dir); g_compile_module_boundary = saved_boundary; if (lf_chunk && chunk_reads_observer(lf_chunk) && (!obs_before_module || g_obs_history_gap)) { diff --git a/src/chunk.c b/src/chunk.c index c9501af7..3b391fe5 100644 --- a/src/chunk.c +++ b/src/chunk.c @@ -38,6 +38,7 @@ void srcbuf_decref(EigsSrcBuf *sb) { rc = --sb->refcount; if (rc > 0) return; free(sb->text); + free(sb->resolve_dir); free(sb); } diff --git a/src/compiler.c b/src/compiler.c index ac232d42..270f8462 100644 --- a/src/compiler.c +++ b/src/compiler.c @@ -1855,8 +1855,19 @@ static void emit_assign_for_tos(Compiler *c, const char *name, uint32_t name_has * doesn't set g_compile_import_toplevel, so its top-level * writes keep walking the chain into the caller's scope, * per its documented "current scope" contract. */ - if (local_only || g_compile_import_toplevel) { + if (local_only) { set_op = OP_SET_NAME_LOCAL; set_arg = (uint16_t)idx; + } else if (lev_has(c, name)) { + /* An outer loop's binder may be written inside an inner + * loop: follow the binding, rather than creating a shadow + * in the innermost loop env. It cannot escape the file. */ + set_op = OP_SET_NAME; set_arg = (uint16_t)idx; + } else if (g_compile_import_toplevel) { + /* #1056: update the nearest binding within this module, + * including an existing loop-local; create in the module + * entry env only when no nearer binding exists. The entry + * chunk's module_scope_writes tag bounds the VM lookup. */ + set_op = OP_SET_FN_NAME_LOCAL; set_arg = (uint16_t)idx; } else { set_op = OP_SET_NAME; set_arg = (uint16_t)idx; } @@ -2380,7 +2391,7 @@ static void compile_node_inner(Compiler *c, ASTNode *node) { } int lev_pushed = 0; - if (c->enclosing && !can_skip_env) { lev_push(c, loop_var); lev_pushed = 1; } /* #1074 */ + if (!can_skip_env) { lev_push(c, loop_var); lev_pushed = 1; } /* #1074, #1056 */ compile_block(c, node->data.forloop.body, node->data.forloop.body_count); if (lev_pushed) c->lev_count--; emit(c, OP_POP, node->line); /* discard body result */ @@ -3374,13 +3385,19 @@ enum { OBS_GATE_MAX_LOADS = 64, OBS_GATE_MAX_DEPTH = 8 }; * covers this repo only — the consumer trees above are why the headroom is * 3.5x rather than the 1.8x that would suffice for lib/ alone. */ #define OBS_GATE_SPECULATIVE_BUDGET (1024L * 1024) -typedef struct { char **paths; int count; int cap; int overflow; } ObsLoadList; +typedef struct { + char **paths; + char **bases; /* directory of the file containing each load */ + const char *base; /* borrowed while collecting one file */ + int count, cap, overflow; +} ObsLoadList; static void obs_gate_note_load(const char *path, void *ud) { /* Collect into a bounded, owned list; resolving here would re-enter the * compiler while chunk_scan_static_loads is still walking the chunk. */ ObsLoadList *L = ud; if (L->count >= L->cap) { L->overflow = 1; return; } /* caller treats as opaque */ + if (L->bases) L->bases[L->count] = xstrdup(L->base); L->paths[L->count++] = xstrdup(path); } @@ -3483,7 +3500,7 @@ static void obs_gate_unmute_stderr(int saved) { * the load). The memo can make the gate conservative-late, never silently wrong. * Released by eigs_obs_memo_release() at thread detach. * - * KEYED ON (st_dev, st_ino), NOT ON THE PATH SPELLING. resolve_eigenscript_file + * FILE IDENTITY (st_dev, st_ino), NOT PATH SPELLING. resolve_eigenscript_file * does not canonicalize — try_resolve_path is access(2) plus a copy — so a * string key gave one file N entries for N spellings, and the pass then read, * compiled and CHARGED it N times. Executed on one 55,180-byte module written @@ -3495,27 +3512,33 @@ static void obs_gate_unmute_stderr(int saved) { * through the KEY instead of the ORDERING — and no existing check saw it, * because check 31's diamond writes the identical literal in every parent. * - * The inode pair beats realpath() here: it is the true identity (it also folds - * hard links, which realpath does not) and it needs no PATH_MAX buffer. The - * cost is one stat PER REFERENCE, memo hits included — the string key stat'd - * only on misses, so this trades ~one syscall per repeated reference (measured: - * 48 newfstatat vs 1 on a 24-reference diamond) for correct identity. That is + * The inode pair identifies the source bytes, including hard links. Since + * #1056 the key ALSO includes the containing directory's identity: hard links + * in different directories can reach different children, while symlink and + * ./ aliases of the same file still share its canonical directory. The + * current cost includes a file stat and a directory stat PER REFERENCE, memo + * hits included; the old string key stat'd only on misses. This trades extra + * metadata queries on repeated references for correct identity. That is * the deliberate price of keying on identity: the stat is what YIELDS the key, * it is microseconds against a read+compile, and unlike the read it charges * nothing against the budget. A stat FAILURE on a later reference of an * already-scanned file now takes the conservative reject path rather than the * memo hit, which is also the direction we want. */ -typedef struct { dev_t dev; ino_t ino; } ObsMemoKey; +/* #1056: hard links share bytes but can resolve different relative children. + * Keep alias deduplication within a canonical containing directory; include + * that directory's identity when memoizing a transitive scan. */ +typedef struct { dev_t dev, dir_dev; ino_t ino, dir_ino; } ObsMemoKey; static __thread ObsMemoKey *g_obs_memo = NULL; static __thread int g_obs_memo_n = 0, g_obs_memo_cap = 0; static __thread long g_obs_spec_bytes = 0; /* see OBS_GATE_SPECULATIVE_BUDGET */ -static int obs_memo_seen(dev_t dev, ino_t ino) { +static int obs_memo_seen(dev_t dev, ino_t ino, dev_t dir_dev, ino_t dir_ino) { for (int i = 0; i < g_obs_memo_n; i++) - if (g_obs_memo[i].dev == dev && g_obs_memo[i].ino == ino) return 1; + if (g_obs_memo[i].dev == dev && g_obs_memo[i].ino == ino && + g_obs_memo[i].dir_dev == dir_dev && g_obs_memo[i].dir_ino == dir_ino) return 1; return 0; } -static void obs_memo_add(dev_t dev, ino_t ino) { +static void obs_memo_add(dev_t dev, ino_t ino, dev_t dir_dev, ino_t dir_ino) { if (g_obs_memo_n == g_obs_memo_cap) { int nc = g_obs_memo_cap ? g_obs_memo_cap * 2 : 16; ObsMemoKey *np = realloc(g_obs_memo, (size_t)nc * sizeof(ObsMemoKey)); @@ -3524,6 +3547,8 @@ static void obs_memo_add(dev_t dev, ino_t ino) { } g_obs_memo[g_obs_memo_n].dev = dev; g_obs_memo[g_obs_memo_n].ino = ino; + g_obs_memo[g_obs_memo_n].dir_dev = dir_dev; + g_obs_memo[g_obs_memo_n].dir_ino = dir_ino; g_obs_memo_n++; } void eigs_obs_memo_release(void); @@ -3661,9 +3686,10 @@ static int obs_ast_scan_d(ASTNode *n, ObsLoadList *L, int depth) { } static void obs_gate_resolve_static_loads(EigsChunk *chunk) { - ObsLoadList L; + ObsLoadList L = {0}; char *slots[OBS_GATE_MAX_LOADS]; char *resolved = NULL; + char *module_dir = NULL; L.paths = slots; L.count = 0; L.cap = OBS_GATE_MAX_LOADS; L.overflow = 0; /* The eager pass informs a RUNTIME decision. Entry points that compile @@ -3677,6 +3703,10 @@ static void obs_gate_resolve_static_loads(EigsChunk *chunk) { if (g_obs_gate_depth >= OBS_GATE_MAX_DEPTH) { eigs_obs_enable(); return; } + L.bases = xcalloc_array(OBS_GATE_MAX_LOADS, sizeof(char *)); + L.base = chunk->src && chunk->src->resolve_dir + ? chunk->src->resolve_dir : eigs_current_file_dir(); + if (chunk_scan_static_loads(chunk, obs_gate_note_load, &L)) { eigs_obs_enable(); goto done; } if (L.overflow) { eigs_obs_enable(); goto done; } /* more loads than slots — see above */ @@ -3713,6 +3743,8 @@ static void obs_gate_resolve_static_loads(EigsChunk *chunk) { for (int i = 0; i < L.count && !g_obs_needed; i++) { long size = 0; char *source = NULL; + free(module_dir); + module_dir = NULL; #if !EIGENSCRIPT_FREESTANDING /* Guarded on the VALUE, and the guard must cover the CALLEES, not only * the helpers. builtins_host.c is a whole-TU carve-out in this profile, @@ -3722,7 +3754,7 @@ static void obs_gate_resolve_static_loads(EigsChunk *chunk) { * why a green release suite and a green ASan suite could not see it — * and it is the same defect class as the #ifdef-vs-#if mistake this file * already records, made a second time while fixing the first. */ - int resolved_ok = resolve_eigenscript_file(L.paths[i], resolved, 8192); + int resolved_ok = resolve_eigenscript_file_from(L.bases[i], L.paths[i], resolved, 8192); #else int resolved_ok = 0; #endif @@ -3761,7 +3793,10 @@ static void obs_gate_resolve_static_loads(EigsChunk *chunk) { st.st_size > OBS_GATE_MAX_MODULE_BYTES) { eigs_obs_enable(); break; } - if (obs_memo_seen(st.st_dev, st.st_ino)) continue; + module_dir = eigs_file_directory(resolved); + struct stat dir_st; + if (stat(module_dir, &dir_st) != 0) { eigs_obs_enable(); break; } + if (obs_memo_seen(st.st_dev, st.st_ino, dir_st.st_dev, dir_st.st_ino)) continue; if (g_obs_spec_bytes + st.st_size > OBS_GATE_SPECULATIVE_BUDGET) { eigs_obs_enable(); break; } @@ -3770,7 +3805,7 @@ static void obs_gate_resolve_static_loads(EigsChunk *chunk) { #endif if (!source) { eigs_obs_enable(); break; } #if !EIGENSCRIPT_FREESTANDING - obs_memo_add(st.st_dev, st.st_ino); + obs_memo_add(st.st_dev, st.st_ino, dir_st.st_dev, dir_st.st_ino); #endif int muted = obs_gate_mute_stderr(); @@ -3813,6 +3848,7 @@ static void obs_gate_resolve_static_loads(EigsChunk *chunk) { * L.count), and an overflow of L is opaque as it is for the host * chunk's own list. Nothing compiles, so nothing arms trace recording * in the parent (the trace_arm_snapshot dance is gone with it). */ + L.base = module_dir; if (obs_ast_scan(mast, &L) || L.overflow) eigs_obs_enable(); g_first_error_line = saved_fe_line; g_first_error_col = saved_fe_col; g_first_error_len = saved_fe_len; g_first_error_col_known = saved_fe_known; @@ -3825,12 +3861,18 @@ static void obs_gate_resolve_static_loads(EigsChunk *chunk) { } done: + free(module_dir); free(resolved); - for (int i = 0; i < L.count; i++) free(L.paths[i]); + for (int i = 0; i < L.count; i++) { + free(L.paths[i]); + free(L.bases[i]); + } + free(L.bases); } EigsChunk *compile_ast(ASTNode *ast, Env *env, const char *src) { EigsChunk *chunk = chunk_new(""); + chunk->module_scope_writes = g_compile_import_toplevel != 0; /* #830: the arming below is compile-time evidence about THIS chunk, so * only this chunk (and the fn chunks compiled under it) may use the * armed-name filter. See EigsChunk.compiler_scanned in vm.h. */ @@ -3839,6 +3881,16 @@ EigsChunk *compile_ast(ASTNode *ast, Env *env, const char *src) { * Owned copy — callers free their buffers while closures can keep * chunks alive indefinitely. Nested fn chunks share the blob. */ chunk->src = srcbuf_new(src); + if (chunk->src) { + /* Compile context wins over the still-running loader's frame. + * Nested functions share this owned blob after the loader returns. */ + const char *base = g_import_resolve_dir[0] ? g_import_resolve_dir + : eigs_current_file_dir(); +#if !EIGENSCRIPT_FREESTANDING + chunk->src->resolve_dir = realpath(base, NULL); +#endif + if (!chunk->src->resolve_dir) chunk->src->resolve_dir = xstrdup(base); + } Compiler compiler; memset(&compiler, 0, sizeof(compiler)); @@ -3888,6 +3940,7 @@ EigsChunk *compile_ast(ASTNode *ast, Env *env, const char *src) { name_set_free(&module_names); name_set_free(&compiler.module_slot_names); free(compiler.locals); + free(compiler.lev_names); /* Opt-in self-check: every chunk this compiler emits must satisfy the * verifier that gates untrusted chunks. Off unless EIGS_VERIFY_SELF=1 (the diff --git a/src/eigenscript.h b/src/eigenscript.h index 8c467c5e..6fd6b94a 100644 --- a/src/eigenscript.h +++ b/src/eigenscript.h @@ -1602,9 +1602,12 @@ void rt_error(ErrKind kind, int line, const char *fmt, ...) __attribute__((format(printf, 3, 4))); char* read_file_util(const char *path, long *out_size); int resolve_eigenscript_file(const char *path, char *resolved, size_t resolved_cap); -/* Same chain, but the "script-relative" and "../" steps anchor at - * `base` instead of `g_script_dir`. Used by OP_IMPORT to resolve a - * module's own imports relative to that module's directory. */ +/* File provenance is retained by the executing chunk, including closures. */ +const char *eigs_current_file_dir(void); +char *eigs_file_directory(const char *path); /* hosted; caller frees */ +void eigs_file_resolve_error(const char *operation, const char *base, + const char *path, int line); /* hosted */ +/* One chain for import/load_file; base is the containing file's directory. */ int resolve_eigenscript_file_from(const char *base, const char *path, char *resolved, size_t resolved_cap); /* #904: which half of the chain answered. The chain's tail steps are the diff --git a/src/eigs_embed.c b/src/eigs_embed.c index 84e250cd..c444c6a2 100644 --- a/src/eigs_embed.c +++ b/src/eigs_embed.c @@ -66,7 +66,7 @@ void eigs_close(EigsState *st) { /* ---- Eval --------------------------------------------------------- */ -EigsValue *eigs_eval_string(const char *src) { +static EigsValue *eval_source(const char *src, const char *file_dir) { if (!src || !eigs_current || !g_global_env) return NULL; Env *global = g_global_env; @@ -106,7 +106,17 @@ EigsValue *eigs_eval_string(const char *src) { * can interrogate a binding an earlier call assigned. The host can also read * observer state directly. Nothing here can see the next call, so observe. */ eigs_obs_enable(); /* #915: via the helper, so a mid-run flip records the gap */ + /* A file's explicit base must beat a caller frame while compiling, but + * must never outlive compilation: runtime eval belongs to its own frame. + * Keep this pair at the compile boundary for both embed entry points. */ + char *saved_dir = file_dir ? xstrdup(g_import_resolve_dir) : NULL; + if (file_dir) + snprintf(g_import_resolve_dir, sizeof(g_import_resolve_dir), "%s", file_dir); EigsChunk *chunk = compile_ast(ast, global, src); + if (saved_dir) { + snprintf(g_import_resolve_dir, sizeof(g_import_resolve_dir), "%s", saved_dir); + free(saved_dir); + } Value *result = vm_execute(chunk, global); chunk_free(chunk); @@ -120,28 +130,22 @@ EigsValue *eigs_eval_string(const char *src) { return result; } +EigsValue *eigs_eval_string(const char *src) { + return eval_source(src, NULL); +} + EigsValue *eigs_eval_file(const char *path) { #if EIGENSCRIPT_FREESTANDING (void)path; return NULL; /* no filesystem — embed callers pass source strings */ #else if (!path || !eigs_current) return NULL; - /* Update script_dir so `import` / `load_file` inside the source can - * resolve relative paths the same way the CLI does. */ - const char *last_slash = strrchr(path, '/'); - if (last_slash) { - size_t dir_len = (size_t)(last_slash - path); - if (dir_len >= sizeof(g_script_dir)) dir_len = sizeof(g_script_dir) - 1; - memcpy(g_script_dir, path, dir_len); - g_script_dir[dir_len] = '\0'; - } else { - memcpy(g_script_dir, ".", 2); - } - long size = 0; char *src = read_file_util(path, &size); if (!src) return NULL; - EigsValue *r = eigs_eval_string(src); + char *dir = eigs_file_directory(path); + EigsValue *r = eval_source(src, dir); + free(dir); free(src); return r; #endif /* !EIGENSCRIPT_FREESTANDING */ diff --git a/src/embed_roads.c b/src/embed_roads.c new file mode 100644 index 00000000..eacaac12 --- /dev/null +++ b/src/embed_roads.c @@ -0,0 +1,110 @@ +/* #1056: exercise file provenance through the real embedding API. */ +#include "eigs_embed.h" +#include "eigenscript.h" +#include +#include +#include +#include + +static int checks, failures, scope_checks; + +static EigsValue *scope_clean(EigsValue *arg) { + scope_checks++; + checks++; + if (g_import_resolve_dir[0]) { + failures++; + printf("embed_roads: FAIL: execution override at %s: %s\n", + eigs_value_as_string(arg), g_import_resolve_dir); + } + return eigs_value_new_null(); +} + +static EigsValue *host_file(EigsValue *path) { + /* Internal provenance stress probe, not a general API reentrancy promise. */ + return eigs_eval_file(eigs_value_as_string(path)); +} + +static void pair(const char *label, EigsValue *value) { + const char *actual[2] = {"", ""}; + EigsValue *items[2] = {NULL, NULL}; + if (value && eigs_value_type(value) == EIGS_TYPE_LIST && + eigs_value_list_len(value) == 2) { + for (int i = 0; i < 2; i++) { + items[i] = eigs_value_list_get(value, i); + if (items[i] && eigs_value_type(items[i]) == EIGS_TYPE_STR) + actual[i] = eigs_value_as_string(items[i]); + } + } + printf("embed_roads: %s: %s / %s\n", label, actual[0], actual[1]); + checks++; + if (eigs_has_error() || strcmp(actual[0], "HELPER") || strcmp(actual[1], "HELPER")) { + failures++; + printf("embed_roads: FAIL: %s expected HELPER / HELPER\n", label); + } + for (int i = 0; i < 2; i++) eigs_value_release(items[i]); + eigs_value_release(value); +} + +static void string_peer(const char *label) { + EigsValue *v = eigs_eval_string("eval of \"load_file of \\\"peer.eigs\\\"\""); + checks++; + if (eigs_has_error() || !v || eigs_value_type(v) != EIGS_TYPE_STR || + strcmp(eigs_value_as_string(v), "STRING")) { + failures++; + printf("embed_roads: FAIL: %s expected STRING\n", label); + } + eigs_value_release(v); +} + +int main(int argc, char **argv) { + int plant_override = argc == 3 && strcmp(argv[2], "--plant-override") == 0; + if (argc != 2 && !plant_override) return 2; + char *root = realpath(argv[1], NULL); + if (!root) { puts("embed_roads: FAIL: missing fixture tree"); return 1; } + char *entry = malloc(strlen(root) + 32); + if (!entry) return 2; + sprintf(entry, "%s/nofile", root); + if (chdir(entry)) return 2; + EigsState *state = eigs_open(); + if (!state) return 2; + eigs_register_function("host_scope_clean", scope_clean); + eigs_register_function("host_file", host_file); + string_peer("no-file string before file eval"); + sprintf(entry, "%s/entry.eigs", root); + pair("eval_file", eigs_eval_file(entry)); + pair("eval_string deferred functions", eigs_eval_string("[go of [], direct of []]")); + /* The host passes the absolute name as a value, not quoted source text. */ + EigsValue *path = eigs_value_new_string(entry); + eigs_set_global("entry_path", path); + eigs_value_release(path); + pair("eval_file from helper callback", eigs_eval_string("call_file of entry_path")); + pair("eval_string load", eigs_eval_string("load_file of entry_path")); + sprintf(entry, "%s/import_entry.eigs", root); + pair("eval_file import", eigs_eval_file(entry)); + path = eigs_value_new_string(entry); + eigs_set_global("entry_path", path); + eigs_value_release(path); + pair("eval_string import", eigs_eval_string("load_file of entry_path")); + string_peer("no-file string after file eval"); + sprintf(entry, "%s/missing.eigs", root); + EigsValue *missing = eigs_eval_file(entry); + checks++; + if (missing) { failures++; puts("embed_roads: FAIL: missing file accepted"); } + eigs_value_release(missing); + string_peer("no-file string after missing file"); + checks++; + if (scope_checks < 12) { failures++; puts("embed_roads: FAIL: too few execution scope probes"); } + if (plant_override) { + /* Only the registered execution probe can see this fault: no file + * lookup runs while the override is set, and ordinary results match. */ + snprintf(g_import_resolve_dir, sizeof(g_import_resolve_dir), "planted-override"); + EigsValue *probe = eigs_eval_string("host_scope_clean of \"planted execution\""); + g_import_resolve_dir[0] = '\0'; + eigs_value_release(probe); + } + eigs_close(state); + free(entry); + free(root); + printf("embed_roads: checks=%d scope_checks=%d failures=%d\n", checks, scope_checks, failures); + return failures != 0; +} diff --git a/src/jit.c b/src/jit.c index 86885d8a..30898a20 100644 --- a/src/jit.c +++ b/src/jit.c @@ -1624,6 +1624,11 @@ static uint8_t *emit_cmp_rdx_disp32_rax(uint8_t *w, int32_t disp) { *w++ = 0x48; *w++ = 0x39; *w++ = 0x90; return emit_u32(w, (uint32_t)disp); } +/* cmp %rdx, disp32(%r15) — active env versus the cached function/module home. */ +static uint8_t *emit_cmp_rdx_disp32_r15(uint8_t *w, int32_t disp) { + *w++ = 0x49; *w++ = 0x39; *w++ = 0x97; + return emit_u32(w, (uint32_t)disp); +} /* mov disp32(%rdx), %esi (6 bytes) — env->binding_version load. */ static uint8_t *emit_mov_disp32_rdx_to_esi(uint8_t *w, int32_t disp) { *w++ = 0x8B; *w++ = 0xB2; @@ -2695,7 +2700,7 @@ static void jit_compile_to_thunk(struct EigsChunk *chunk, ? (int)offsetof(CallFrame, fn_env) : (int)offsetof(CallFrame, env); EnvIC *ic = &chunk->env_ic[sidx]; - uint8_t *slow_p[6]; + uint8_t *slow_p[7]; int slow_n = 0; /* Trace gate: address baked, flag is process-global. The STORAGE * symbol, not the macro — the macro is an atomic load expression @@ -2709,6 +2714,14 @@ static void jit_compile_to_thunk(struct EigsChunk *chunk, w = emit_jne_rel32(w, &slow_p[slow_n]); slow_n++; /* IC identity + starting version. */ w = emit_mov_disp32_r15_to_rdx(w, frame_env_off); + /* #1056: an imported entry can acquire a nearer local without + * changing fn_env's version (e.g. eval creates it mid-thunk). + * Inline only when there is no intervening env. Otherwise the + * shared bounded lookup in the helper decides the write target. */ + if (op == OP_SET_FN_NAME_LOCAL && chunk->module_scope_writes) { + w = emit_cmp_rdx_disp32_r15(w, (int32_t)offsetof(CallFrame, env)); + w = emit_jne_rel32(w, &slow_p[slow_n]); slow_n++; + } w = emit_movabs_rax(w, (uint64_t)(uintptr_t)ic); w = emit_cmp_rdx_disp32_rax(w, (int32_t)offsetof(EnvIC, starting_env)); w = emit_jne_rel32(w, &slow_p[slow_n]); slow_n++; diff --git a/src/main.c b/src/main.c index d3f560ac..996e9ccc 100644 --- a/src/main.c +++ b/src/main.c @@ -295,15 +295,9 @@ int main(int argc, char **argv) { /* Extract script directory for load_file resolution. g_script_dir * is an EigsState bridge macro — state is already attached above. */ { - const char *last_slash = strrchr(argv[1], '/'); - if (last_slash) { - int dir_len = (int)(last_slash - argv[1]); - if (dir_len >= (int)sizeof(g_script_dir)) dir_len = sizeof(g_script_dir) - 1; - memcpy(g_script_dir, argv[1], dir_len); - g_script_dir[dir_len] = '\0'; - } else { - memcpy(g_script_dir, ".", 2); - } + char *dir = eigs_file_directory(argv[1]); + snprintf(g_script_dir, sizeof(g_script_dir), "%s", dir); + free(dir); } long src_size = 0; diff --git a/src/vm.c b/src/vm.c index ece0babc..92b105c3 100644 --- a/src/vm.c +++ b/src/vm.c @@ -1234,13 +1234,32 @@ void jit_helper_set_name_local(EigsChunk *chunk, int idx) { } } +/* Imported entry chunks must respect explicit locals in active loop envs, + * while never writing through the module boundary into the importer. Compute + * this before consulting the IC: a newly introduced nearer binding must beat + * a previously cached module target. Function chunks keep their pinned target. + * Shared by the interpreter and JIT; the borrowed env needs no ownership edge. */ +static Env *fn_name_write_target(EigsChunk *chunk, CallFrame *frame, int idx) { + Env *home = frame->fn_env; + if (chunk->module_scope_writes && frame->env != home) { + const char *name = chunk->const_interns[idx]; + uint32_t h = chunk->const_hashes ? chunk->const_hashes[idx] : 0; + if (!h) h = env_hash_name(name); + int slot_idx, depth; + Env *found = env_resolve_chain(frame->env, name, h, &slot_idx, &depth); + for (Env *e = frame->env; e && e != home; e = e->parent) + if (e == found) return e; + } + return home; +} + void jit_helper_set_fn_name_local(EigsChunk *chunk, int idx) { if (__builtin_expect(g_trace_hist, 0)) { vm_trace_assign(chunk, chunk->const_interns[idx], g_vm.stack[g_vm.sp - 1]); } EnvIC *ic = &chunk->env_ic[idx]; CallFrame *frame = &g_vm.frames[g_vm.frame_count - 1]; - Env *target = frame->fn_env; + Env *target = fn_name_write_target(chunk, frame, idx); EigsSlot s = g_vm.stack[g_vm.sp - 1]; if (__builtin_expect(ic->starting_env == target && ic->starting_ver == target->binding_version && @@ -3629,13 +3648,15 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume, * local_only names inside a function so that assignments from nested * loop or scope envs still update the function's binding (see #129b: * unobserved+for+interrogated accumulator otherwise wrote to the - * per-iteration loop env and the outer binding never moved). */ + * per-iteration loop env and the outer binding never moved). + * Imported entry chunks instead honor nearer loop-local bindings; + * their bounded lookup still cannot escape into the importer. */ uint16_t idx = read_u16(ip); ip += 2; if (__builtin_expect(g_trace_hist, 0)) { vm_trace_assign(chunk, chunk->const_interns[idx], g_vm.stack[g_vm.sp - 1]); } EnvIC *ic = &chunk->env_ic[idx]; - Env *target = frame->fn_env; + Env *target = fn_name_write_target(chunk, frame, idx); EigsSlot s = g_vm.stack[g_vm.sp - 1]; if (__builtin_expect(ic->starting_env == target && ic->starting_ver == target->binding_version && @@ -5819,13 +5840,9 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume, extern char *read_file_util(const char *path, long *size); - /* Per-file resolution base (Phase 0b): an `import` inside a - * module anchors at *that* module's directory, not the main - * script's. `g_import_resolve_dir` is empty at the main-script - * level, in which case the chain falls back to g_script_dir. */ - const char *resolve_base = g_import_resolve_dir[0] - ? g_import_resolve_dir - : g_script_dir; + /* #1056: functions retain their containing file's directory + * even when called after the importing/loading frame returns. */ + const char *resolve_base = eigs_current_file_dir(); /* #821: PROJECT-FIRST resolution. The user module `.eigs` * (script-relative, plus the chain's other locations and the @@ -5859,8 +5876,8 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume, user_hit = 0; if (!user_hit && !stdlib_hit) { - rt_error(EK_IO, current_line, "import: module '%s' not found " - "(tried %s.eigs and lib/%s.eigs)", name, name, name); + snprintf(request, sizeof(request), "%.1024s.eigs and lib/%.1024s.eigs", name, name); + eigs_file_resolve_error("import", resolve_base, request, current_line); vm_push(make_null()); DISPATCH(); } @@ -5979,11 +5996,14 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume, EigsChunk *mod_chunk = compile_ast(ast, mod_env, source); g_compile_module_boundary = saved_boundary; g_compile_import_toplevel = saved_import_toplevel; + /* The override belongs to compilation, not module execution. Runtime + * eval must inherit its executing function's retained source directory, + * even when that function is called by this imported module. */ + memcpy(g_import_resolve_dir, saved_resolve_dir, sizeof(saved_resolve_dir)); if (g_parse_errors > 0) { g_parse_errors = saved_errors; chunk_free(mod_chunk); g_load_env = saved_load; - memcpy(g_import_resolve_dir, saved_resolve_dir, sizeof(saved_resolve_dir)); free_ast(ast); free_tokenlist(&tl); free(source); @@ -5999,7 +6019,6 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume, if (mod_result) val_decref(mod_result); chunk_free(mod_chunk); /* creator ref; module fns hold their own */ g_load_env = saved_load; - memcpy(g_import_resolve_dir, saved_resolve_dir, sizeof(saved_resolve_dir)); free_ast(ast); free_tokenlist(&tl); free(source); diff --git a/src/vm.h b/src/vm.h index 476da398..9b564614 100644 --- a/src/vm.h +++ b/src/vm.h @@ -275,6 +275,8 @@ typedef struct { typedef struct EigsSrcBuf { int refcount; char *text; + char *resolve_dir; /* owned canonical containing-file directory; + * shared with nested functions, not a GC edge */ } EigsSrcBuf; EigsSrcBuf *srcbuf_new(const char *text); @@ -358,6 +360,10 @@ typedef struct EigsChunk { * trace_assign anyway. */ char *name; /* function name or "" */ + uint8_t module_scope_writes; /* imported entry chunk only: SET_FN_NAME_LOCAL + * updates a nearer loop binding if present, + * otherwise binds in the module entry env. + * Nested functions retain function-local rules. */ int param_count; int first_default; /* slot index of first param with a default; == * param_count when no defaults. Calls with diff --git a/tests/embed_roads/eigs_modules/entrywrapper/entrywrapper.eigs b/tests/embed_roads/eigs_modules/entrywrapper/entrywrapper.eigs new file mode 100644 index 00000000..915baa0b --- /dev/null +++ b/tests/embed_roads/eigs_modules/entrywrapper/entrywrapper.eigs @@ -0,0 +1,3 @@ +host_scope_clean of "imported wrapper" +load_file of "../../helper/functions.eigs" +result is [go of [], direct of []] diff --git a/tests/embed_roads/eigs_modules/entrywrapper/peer.eigs b/tests/embed_roads/eigs_modules/entrywrapper/peer.eigs new file mode 100644 index 00000000..6aa2bd4d --- /dev/null +++ b/tests/embed_roads/eigs_modules/entrywrapper/peer.eigs @@ -0,0 +1 @@ +return "WRAPPER" diff --git a/tests/embed_roads/entry.eigs b/tests/embed_roads/entry.eigs new file mode 100644 index 00000000..0d583da4 --- /dev/null +++ b/tests/embed_roads/entry.eigs @@ -0,0 +1,3 @@ +host_scope_clean of "entry" +load_file of "helper/functions.eigs" +return [go of [], direct of []] diff --git a/tests/embed_roads/helper/functions.eigs b/tests/embed_roads/helper/functions.eigs new file mode 100644 index 00000000..0c01b099 --- /dev/null +++ b/tests/embed_roads/helper/functions.eigs @@ -0,0 +1,7 @@ +host_scope_clean of "loaded helper" +define go() as: + return eval of "load_file of \"peer.eigs\"" +define direct() as: + return load_file of "peer.eigs" +define call_file(path) as: + return host_file of path diff --git a/tests/embed_roads/helper/peer.eigs b/tests/embed_roads/helper/peer.eigs new file mode 100644 index 00000000..8906cb02 --- /dev/null +++ b/tests/embed_roads/helper/peer.eigs @@ -0,0 +1,2 @@ +host_scope_clean of "helper peer" +return "HELPER" diff --git a/tests/embed_roads/import_entry.eigs b/tests/embed_roads/import_entry.eigs new file mode 100644 index 00000000..d9c608e6 --- /dev/null +++ b/tests/embed_roads/import_entry.eigs @@ -0,0 +1,3 @@ +host_scope_clean of "import entry" +import entrywrapper +return entrywrapper.result diff --git a/tests/embed_roads/nofile/peer.eigs b/tests/embed_roads/nofile/peer.eigs new file mode 100644 index 00000000..8863f360 --- /dev/null +++ b/tests/embed_roads/nofile/peer.eigs @@ -0,0 +1 @@ +return "STRING" diff --git a/tests/embed_roads/peer.eigs b/tests/embed_roads/peer.eigs new file mode 100644 index 00000000..a9f0691a --- /dev/null +++ b/tests/embed_roads/peer.eigs @@ -0,0 +1 @@ +return "MAIN" diff --git a/tests/roads/README.md b/tests/roads/README.md new file mode 100644 index 00000000..81dc8db0 --- /dev/null +++ b/tests/roads/README.md @@ -0,0 +1,227 @@ +# The three-road oracle + +`bash tools/road_diff.sh` enumerates every `*.eigs` directly in this directory. +Each fixture runs as main, through `load_file`, and through `import`, from two +working directories. Support files live under `assets/` and `eigs_modules/`, reached by the +fixtures; they are not independent oracle programs. Each run gets a private copy +of the fixture tree and an empty HOME. Children are bounded to 30 seconds. +The second cwd run invokes the entry point through a symlink in a third directory, +so main-program provenance must agree with import's canonical-file rule. + +Every fixture declares `# road-bind: name ...` and has a nonempty `.out` file +containing its expected prints followed by `[name, 1, value]` snapshots for +present bindings, or `[name, 0]` for absent bindings. Presence is structural: +neither `null` nor the literal string `""` can impersonate absence. The main +wrapper appends snapshots; the load wrapper snapshots after the call returns; +the import wrapper reads those names back from the namespace, checking `has_key` +before access. The golden decides whether a particular binding may be absent. +Functions can be checked through their results instead of printing identities. + +Each driver captures `print`, `has_key`, `load_file` and `throw` before any +fixture code runs, including before the main-road source splice. Captures and +temporary bindings use a fresh UUID prefix. Readback calls only the captured +builtins; it never consults fixture-rebindable `print` or `has_key`, and uses no +`keys` call. The UUID is driver hygiene, not part of the compared output or runtime +semantics; this is an oracle for fixtures, not a sandbox against malicious code +that reads and rewrites its generated driver. + +For a top-level return, the main wrapper inserts snapshots immediately before +each unindented `return`; `# road-return: expression` also checks the load's +returned value. Fixtures with a return nested in a block need a separate wrapper +extension: this instrument currently supports unindented file returns only. +It changes no fixture statement, and retains the containing directory. Snapshot +reads are observations, so these fixtures test values and scope, not observer +history or exact diagnostic line numbers. `# road-cwd: relative/directory` lines +override the default fixture-directory/unrelated-directory pair. +`# road-hardlink: source target` recreates a hard link in each private tree +(Git stores file contents, not hard-link relationships). + +After the snapshots, the captured `print` emits a fresh UUID completion marker. +Each run must contain that marker exactly once, at the end of stdout. The gate +removes only that marker before comparing fixture output. Early `exit of 0` +cannot skip readback and substitute forged snapshot lines. Invalid binding +identifiers produce named failures before any child runs. + +Stdout is compared byte for byte against the expected file AND between roads and +directories; every child must exit zero and have empty stderr (including under +sanitizers). An error on all three roads cannot masquerade as agreement. Missing +metadata, missing expected files, timeouts and zero fixtures fail. `--fixture +blocks` selects a single diagnostic repro; the suite always runs the whole set. + +`--selftest` starts with five green controls: a numeric value, a literal `""` +created in a `for` body, and fixtures rebinding `print`, `has_key`/`keys`, and +`throw`, plus a native loop with compilation statistics, an ARM64-policy control, +and a check that the lowered threshold reaches the child. On x86-64, sixteen +faults must go red: cwd divergence; deletion of the sentinel +assignment; forged absence goldens for both readback-rebinding fixtures; +incorrect return metadata despite a rebound `throw`; genuine absence where +present `null` is expected; a nonzero exit alone; stderr alone; exit before readback; an invalid binding +identifier; forced-off native arms; missing JIT statistics; a removed lowered +threshold; an invalid native-header value; duplicate native headers; and zero fixtures. +The membership control also calls the shared namespace-snapshot emitter from +within a scope that rebinds `has_key`/`keys`, so module isolation cannot conceal +a missing capture. The suite runs the ordinary gate and selftest. + +`--selftest --bad-binary /path/to/known-bad/eigenscript` replaces the assignment +deletion and two forged goldens with execution of the unchanged sentinel, +print-rebinding and membership-rebinding fixtures on a runtime that drops +imported `for`-body bindings. Each plant must have clean child exits and exactly +two import-only presence mismatches, so an unavailable or crashing binary is +not accepted as a detected regression. The default selftest uses no external +checkout or compiler build. + +The exit/stderr plants run the real binary through a bounded Python wrapper +that changes only its process status or stderr after successful evaluation. +Stdout must still match exactly. All six exit-plant runs return the same 17: +cross-road status comparison must not hide a missing absolute exit check. + +Bought in #1056 round 3: rebinding `print` forged an absent-binding snapshot on +the known-bad runtime, and deleting either the exit or stderr check survived +the old selftest. The driver captures and independent process-result plants +close those holes. Reverting the print or membership capture, delaying main's +captures until after the fixture, using the rebound `throw`, or removing either +process-result check now fails selftest. Existing fixture goldens are unchanged +in this round. + +Bought in #1056 round 2: the old `[name, ""]` representation gave a +false green on the known-bad runtime when the actual value was that same string. +The structural presence field and both missing-value plants close that hole. +The 11 existing goldens changed only their snapshot encoding (37 rows); their +fixture prints and value expectations were retained. + +The `blocks` fixture is red on c1684bc: import has no `from_for` key while main +and load_file expose 4. `shadow` exercises the same A/prog.eigs from directories +A and B; only A/inc.eigs may run. `nested_load` checks sibling and project-root +loads, calls functions after the loaded file returns, and checks restored caller +provenance. `eigs.json` in that support tree is the project-root marker. + +`importer_scope` gives the importer an `outer` binding before importing a helper +whose `for` body assigns that same name. It requires the importer to retain 1 +and the helper to export its own binding. Replacing the compiler's imported +block assignment with `OP_SET_NAME` passed all 11 older fixtures but clobbered +the main/load importer's `outer` to 2 and omitted the helper's key; this fixture +rejects that plant. It also rejects c1684bc's loop-local assignment, which keeps +the importer at 1 but still omits the helper's key. + +`resolution_order` pins sibling > package > project-root precedence; +`resolution_errors` rejects the removed cwd and one-parent steps and checks +both loaders' diagnostics. `observer_nested` has an observer-free decoy at the +old scan base: resolving nested eager reads from the main file must go red. +`hardlink_observer` gives one hard-linked source two different sibling modules. +An inode-only scan memo missed the observing sibling and raised at runtime; +the memo must include the containing directory. This was reproduced during +#1056, and is why file provenance belongs in the scan as well as execution. + +`binders` deliberately retains the existing function-slot exception documented +in LANGUAGE_CONTRACT.md: a binder with no prior binding in a function remains +readable after the loop. On c1684bc `define f(): for z in [7, 8]: ...; return z` +returns 8. This differs from module scope, but is uniform across roads; #1056 +does not change it. Pre-existing parameters, locals and module bindings are +protected on every road. + +The sanitizer run also checks compiler ownership: extending loop-binder +tracking from functions to modules requires freeing the root compiler's +`lev_names` array. Before that cleanup, `blocks` produced correct stdout but +all six executions failed with a 32-byte leak. The gate rejects those exits +instead of accepting matching output from leaking children. + +`f29_loop_local` pins writes to a current loop-local, an enclosing loop-local, +and locals in module-level `if`/`loop while` bodies. `f29_loop_local_cache` +alternates between a nearer local and module state over 80 iterations. These +f29 fixtures test interpreter scope behavior: `LOOP_ENV_CLEAR` prevents native +compilation, even with `EIGS_JIT_OSR_THRESHOLD=1`. A direct cache-fixture run +reports `[jit] scanned=1 compiled=0 cache_used=0`, with `LOOP_ENV_CLEAR` as its +only bailout. The earlier claim that this demonstrated native stores was wrong. +`f30_eval_dir` calls a helper's eval and direct load from main, a loaded file, +an imported wrapper, and a nested import; every call must load the helper's peer. + +Bought in #1056 round 4: pinning every imported block write to the module +repaired missing exports but skipped existing loop locals. Check both sides +of a scope boundary: stop outward writes at it, and preserve nearer bindings +inside it. Import's compile-only directory override also leaked into execution +and redirected another file's eval. Its lifetime must end before module code +runs. The new fixtures fail with either respective fix removed. + +The installed-layout subset symlinks test sources, so their canonical containing +directory differs from the temporary runner's cwd. Both suite runners export +`EIGS_TEST_DIR`; generated module writers use it for writes, loads, and cleanup. +The standalone fallback assumes the documented `src/` cwd. To reproduce the +actual installed layout without modifying the normal installation, set +`EIGENSCRIPT_INSTALL_PREFIX` when running `install.sh`, then put its `bin` on PATH +and pass its interpreter as `EIGENSCRIPT` to `tests/run_install_smoke_subset.sh`. +That lane failed two sections before the canonical-directory migration. +Existing fixture goldens are unchanged in this round. + +## Native scope coverage (#1056 round 5) + +`# road-native: required` fixtures run on all three roads and both cwds under +`EIGS_JIT_OFF=1`, default JIT, and `EIGS_JIT_OSR_THRESHOLD=1`. Each run must emit +exactly one JIT statistics line. These are configurations, not three measured +entry mechanisms: the same compiled chunk is exercised with the OSR threshold +lowered in the `osr` arm. Both native arms compile code and must agree. The +statistics count compiled chunks, not OSR entries; default JIT may already use +OSR. The reference requires `compiled=0`; each +native arm requires `compiled>0`. On ARM64, which has no JIT emitter, the gate +prints an explicit notice and runs only the reference tier (still requiring +its stats and `compiled=0`) on all roads/cwds. This does not waive a zero-compilation +native arm on x86-64. A separate selftest simulates this ARM64 policy. +The gate strips only that recognized stats +line from stderr; every other diagnostic still fails. The selftest runs a +known native loop, then forces JIT off or removes its stats through child +wrappers and requires named failures with matching stdout. Another wrapper +checks the child environment for the lowered threshold; removing that setting +must fail even when compilation statistics and stdout stay identical. + +`native_alternate`, `native_late`, `native_outer`, `native_match`, and +`native_catch` use the critic's compilable inner loops to exercise the helper +lookup through alternating, late, nested, match, and catch locals. +`native_inline` creates a local with eval after a native inner loop has cached +a module target. It writes without reading that name first, so a GET_NAME +cannot refresh the caller's IC and conceal a stale inline store. +For `native_inline` on x86-64, the measured import-road lines are: + +```text +ref: [jit] scanned=0 compiled=0 cache_used=0 +jit: [jit] scanned=2 compiled=1 cache_used=19878 +osr: [jit] scanned=2 compiled=1 cache_used=19878 +``` + +All three return 14999. Without the inline scope guard, the native arms +return 19999. Removing only the JIT helper's bounded lookup instead breaks +`native_alternate` (and the other helper probes). The interpreter is the +independent value oracle; existing goldens are unchanged. + +## Embed provenance and override audit + +`python3 tools/embed_roads.py --selftest` builds `make embed-roads` without +relinking the CLI. A unique objdir inode match reuses that build variant. +A standalone `build.sh` CLI or ambiguous match uses the plain SOURCES list +through the release objects, or ASan objects when ASAN_OPTIONS is set. +The test covers provenance semantics with either layout; it does not infer +an unidentified CLI's compiler flags. Four metadata controls exercise zero, +one, and multiple matches, including the sanitizer fallback. +Its C harness checks `eigs_eval_file`, successive `eigs_eval_string` calls, +loaded helpers, imported wrappers, and restoration to no-file string eval. +A registered host probe checks the compile override while each file executes. +A wrong helper peer, a missing fixture tree, a nonzero exit, stderr, zero +checks, and a compile override planted only during a host probe must fail its +selftest. The scope plant leaves file lookup and ordinary values untouched, +so gutting host_scope_clean makes the selftest fail. Process plants must retain the healthy C result +and produce exactly their intended symptom. + +| Directory state | Lifetime and regression coverage | +|---|---| +| `builtins_host.c` load_file override | Saved/set/restored around compile_ast; embed loaded-helper probes and f30_eval_dir. | +| `vm.c` import override | Restored before vm_execute; embed imported-wrapper probe and f30_eval_dir's nested import. | +| `eigs_embed.c` file override | Shared eval_source scopes it around compile_ast; embed file/string and execution probes. No script_dir mutation remains. | +| `main.c` script_dir | Entry-file base for the state, with canonical file provenance captured in chunks; shadow/chdir/nested_load and the native fixtures. | +| `state.c` initial script_dir | No-file `.` base; embed string eval before and after file eval checks its cwd peer. | +| `lint_host.c` E003.base_dir | Private lint traversal context, not a runtime global override. | +| `bundle.c` | Rewrites argv to the extracted entry; main establishes its base. No resolver-global writes. Existing bundle suite covers execution. | + +Bought in round 5: a lowered OSR threshold was mistaken for evidence of compilation, +and the embed setter retained the same override lifetime import had just fixed. +The compilation assertions and execution-time embed probes enforce those +claims. Round 6 corrects the narrower overstatement: compiled>0 does not +distinguish default JIT entry from OSR entry. No runtime semantics change is +needed to describe the measured configurations accurately. diff --git a/tests/roads/assets/eval_loaded/caller.eigs b/tests/roads/assets/eval_loaded/caller.eigs new file mode 100644 index 00000000..17f86f60 --- /dev/null +++ b/tests/roads/assets/eval_loaded/caller.eigs @@ -0,0 +1,5 @@ +try: + loaded_eval is evalhelper.go of [] +catch err: + loaded_eval is "WRONG-DIR-ERROR" +loaded_direct is evalhelper.direct of [] diff --git a/tests/roads/assets/eval_loaded/peer.eigs b/tests/roads/assets/eval_loaded/peer.eigs new file mode 100644 index 00000000..5698e2c3 --- /dev/null +++ b/tests/roads/assets/eval_loaded/peer.eigs @@ -0,0 +1 @@ +return "LOADED-PEER" diff --git a/tests/roads/assets/hardlinks/a/child.eigs b/tests/roads/assets/hardlinks/a/child.eigs new file mode 100644 index 00000000..b7e8d30a --- /dev/null +++ b/tests/roads/assets/hardlinks/a/child.eigs @@ -0,0 +1 @@ +print of "idle" diff --git a/tests/roads/assets/hardlinks/a/entry.eigs b/tests/roads/assets/hardlinks/a/entry.eigs new file mode 100644 index 00000000..12933947 --- /dev/null +++ b/tests/roads/assets/hardlinks/a/entry.eigs @@ -0,0 +1 @@ +load_file of "child.eigs" diff --git a/tests/roads/assets/hardlinks/b/child.eigs b/tests/roads/assets/hardlinks/b/child.eigs new file mode 100644 index 00000000..cff94bea --- /dev/null +++ b/tests/roads/assets/hardlinks/b/child.eigs @@ -0,0 +1 @@ +print of (report of metric) diff --git a/tests/roads/assets/hardlinks/b/entry.eigs b/tests/roads/assets/hardlinks/b/entry.eigs new file mode 100644 index 00000000..12933947 --- /dev/null +++ b/tests/roads/assets/hardlinks/b/entry.eigs @@ -0,0 +1 @@ +load_file of "child.eigs" diff --git a/tests/roads/assets/no_project/child/entry.eigs b/tests/roads/assets/no_project/child/entry.eigs new file mode 100644 index 00000000..a2c344c1 --- /dev/null +++ b/tests/roads/assets/no_project/child/entry.eigs @@ -0,0 +1,20 @@ +no_parent is 0 +try: + load_file of "parent.eigs" +catch err: + no_parent is (type of err) == "dict" and err.kind == "io" +no_cwd is 0 +try: + load_file of "cwd_only.eigs" +catch err: + no_cwd is (type of err) == "dict" and err.kind == "io" +loaded_error is 0 +try: + load_file of "roads_missing_1056.eigs" +catch err: + loaded_error is (contains of [err.message, "containing directory"]) and (contains of [err.message, "no eigs.json above"]) and (contains of [err.message, "stdlib roots"]) +import_error is 0 +try: + import roads_missing_1056 +catch err: + import_error is (contains of [err.message, "containing directory"]) and (contains of [err.message, "no eigs.json above"]) and (contains of [err.message, "stdlib roots"]) diff --git a/tests/roads/assets/no_project/cwd_only.eigs b/tests/roads/assets/no_project/cwd_only.eigs new file mode 100644 index 00000000..e80d7d8f --- /dev/null +++ b/tests/roads/assets/no_project/cwd_only.eigs @@ -0,0 +1 @@ +throw of "FORBIDDEN cwd fallback" diff --git a/tests/roads/assets/no_project/parent.eigs b/tests/roads/assets/no_project/parent.eigs new file mode 100644 index 00000000..a36b01db --- /dev/null +++ b/tests/roads/assets/no_project/parent.eigs @@ -0,0 +1 @@ +throw of "FORBIDDEN parent fallback" diff --git a/tests/roads/assets/observer/assets/observer/target.eigs b/tests/roads/assets/observer/assets/observer/target.eigs new file mode 100644 index 00000000..cff94bea --- /dev/null +++ b/tests/roads/assets/observer/assets/observer/target.eigs @@ -0,0 +1 @@ +print of (report of metric) diff --git a/tests/roads/assets/observer/entry.eigs b/tests/roads/assets/observer/entry.eigs new file mode 100644 index 00000000..9d2666e6 --- /dev/null +++ b/tests/roads/assets/observer/entry.eigs @@ -0,0 +1,3 @@ +# The eager scan must anchor this path here too. Resolving from the parent +# program finds the observer-free decoy, closes the gate, and loses history. +load_file of "assets/observer/target.eigs" diff --git a/tests/roads/assets/observer/target.eigs b/tests/roads/assets/observer/target.eigs new file mode 100644 index 00000000..f8758f3e --- /dev/null +++ b/tests/roads/assets/observer/target.eigs @@ -0,0 +1,2 @@ +# Decoy for a scanner that resolves all nested loads from the main file. +0 diff --git a/tests/roads/assets/order/eigs.json b/tests/roads/assets/order/eigs.json new file mode 100644 index 00000000..3f19fc11 --- /dev/null +++ b/tests/roads/assets/order/eigs.json @@ -0,0 +1 @@ +{"name":"road-order","version":"0.0.0"} diff --git a/tests/roads/assets/order/eigs_modules/nearby/nearby.eigs b/tests/roads/assets/order/eigs_modules/nearby/nearby.eigs new file mode 100644 index 00000000..4d143ea5 --- /dev/null +++ b/tests/roads/assets/order/eigs_modules/nearby/nearby.eigs @@ -0,0 +1 @@ +return 91 diff --git a/tests/roads/assets/order/eigs_modules/package/package.eigs b/tests/roads/assets/order/eigs_modules/package/package.eigs new file mode 100644 index 00000000..3ee3fcaa --- /dev/null +++ b/tests/roads/assets/order/eigs_modules/package/package.eigs @@ -0,0 +1 @@ +return 2 diff --git a/tests/roads/assets/order/package.eigs b/tests/roads/assets/order/package.eigs new file mode 100644 index 00000000..82b8fd3a --- /dev/null +++ b/tests/roads/assets/order/package.eigs @@ -0,0 +1 @@ +return 92 diff --git a/tests/roads/assets/order/root_relative.eigs b/tests/roads/assets/order/root_relative.eigs new file mode 100644 index 00000000..32d0c308 --- /dev/null +++ b/tests/roads/assets/order/root_relative.eigs @@ -0,0 +1 @@ +return 3 diff --git a/tests/roads/assets/order/sub/entry.eigs b/tests/roads/assets/order/sub/entry.eigs new file mode 100644 index 00000000..fff3d779 --- /dev/null +++ b/tests/roads/assets/order/sub/entry.eigs @@ -0,0 +1,8 @@ +nearby is load_file of "nearby.eigs" +package is load_file of "package.eigs" +root_relative is load_file of "root_relative.eigs" +root_error is 0 +try: + load_file of "roads_missing_1056.eigs" +catch err: + root_error is (contains of [err.message, "project root"]) and not (contains of [err.message, "no eigs.json above"]) diff --git a/tests/roads/assets/order/sub/nearby.eigs b/tests/roads/assets/order/sub/nearby.eigs new file mode 100644 index 00000000..a4325f62 --- /dev/null +++ b/tests/roads/assets/order/sub/nearby.eigs @@ -0,0 +1 @@ +return 1 diff --git a/tests/roads/assets/project/eigs.json b/tests/roads/assets/project/eigs.json new file mode 100644 index 00000000..755e59b4 --- /dev/null +++ b/tests/roads/assets/project/eigs.json @@ -0,0 +1 @@ +{"name":"road-project","version":"0.0.0"} diff --git a/tests/roads/assets/project/root/target.eigs b/tests/roads/assets/project/root/target.eigs new file mode 100644 index 00000000..843d4a34 --- /dev/null +++ b/tests/roads/assets/project/root/target.eigs @@ -0,0 +1,2 @@ +print of "project root" +project_result is 22 diff --git a/tests/roads/assets/project/sub/entry.eigs b/tests/roads/assets/project/sub/entry.eigs new file mode 100644 index 00000000..83ce83d6 --- /dev/null +++ b/tests/roads/assets/project/sub/entry.eigs @@ -0,0 +1,7 @@ +load_file of "peer.eigs" +load_file of "root/target.eigs" +define delayed() as: + return load_file of "value.eigs" +define delayed_import() as: + import peer_module + return peer_module.value diff --git a/tests/roads/assets/project/sub/peer.eigs b/tests/roads/assets/project/sub/peer.eigs new file mode 100644 index 00000000..623f678a --- /dev/null +++ b/tests/roads/assets/project/sub/peer.eigs @@ -0,0 +1,2 @@ +print of "sibling" +sibling_result is 21 diff --git a/tests/roads/assets/project/sub/peer_module.eigs b/tests/roads/assets/project/sub/peer_module.eigs new file mode 100644 index 00000000..a4df6c50 --- /dev/null +++ b/tests/roads/assets/project/sub/peer_module.eigs @@ -0,0 +1 @@ +value is 24 diff --git a/tests/roads/assets/project/sub/value.eigs b/tests/roads/assets/project/sub/value.eigs new file mode 100644 index 00000000..a8135de0 --- /dev/null +++ b/tests/roads/assets/project/sub/value.eigs @@ -0,0 +1 @@ +return 23 diff --git a/tests/roads/assets/restore.eigs b/tests/roads/assets/restore.eigs new file mode 100644 index 00000000..075b8520 --- /dev/null +++ b/tests/roads/assets/restore.eigs @@ -0,0 +1 @@ +restored_result is 25 diff --git a/tests/roads/assets/shadow/A/inc.eigs b/tests/roads/assets/shadow/A/inc.eigs new file mode 100644 index 00000000..850d2022 --- /dev/null +++ b/tests/roads/assets/shadow/A/inc.eigs @@ -0,0 +1 @@ +print of "SCRIPTDIR-COPY" diff --git a/tests/roads/assets/shadow/A/prog.eigs b/tests/roads/assets/shadow/A/prog.eigs new file mode 100644 index 00000000..79db3e7e --- /dev/null +++ b/tests/roads/assets/shadow/A/prog.eigs @@ -0,0 +1 @@ +load_file of "inc.eigs" diff --git a/tests/roads/assets/shadow/B/inc.eigs b/tests/roads/assets/shadow/B/inc.eigs new file mode 100644 index 00000000..cd74ce02 --- /dev/null +++ b/tests/roads/assets/shadow/B/inc.eigs @@ -0,0 +1 @@ +print of "CWD-COPY" diff --git a/tests/roads/binders.eigs b/tests/roads/binders.eigs new file mode 100644 index 00000000..0a4cd6ff --- /dev/null +++ b/tests/roads/binders.eigs @@ -0,0 +1,37 @@ +# road-bind: y x module_result function_result parameter_result local_result nested_result fresh_result k +y is 100 +for y in [1, 2, 3]: + y is y + 10 +module_result is y +x is 5 +define over_module(x) as: + for x in [7, 8]: + x is x + 10 + return x +parameter_result is over_module of 1 +define over_local() as: + local x is 42 + for x in [7, 8]: + x is x + 10 + return x +local_result is over_local of [] +define ordinary() as: + i is 20 + for i in [7, 8]: + i is i + 10 + return i +function_result is ordinary of [] +define nested() as: + p is 30 + for p in [7, 8]: + for p in [9, 10]: + p is p + 10 + return p +nested_result is nested of [] +define fresh() as: + for z in [7, 8]: + 0 + return z +fresh_result is fresh of [] +for k in [1, 2]: + print of k diff --git a/tests/roads/binders.out b/tests/roads/binders.out new file mode 100644 index 00000000..ac2fe93c --- /dev/null +++ b/tests/roads/binders.out @@ -0,0 +1,11 @@ +1 +2 +["y", 1, 100] +["x", 1, 5] +["module_result", 1, 100] +["function_result", 1, 20] +["parameter_result", 1, 1] +["local_result", 1, 42] +["nested_result", 1, 30] +["fresh_result", 1, 8] +["k", 0] diff --git a/tests/roads/block_nested.eigs b/tests/roads/block_nested.eigs new file mode 100644 index 00000000..0d6347ed --- /dev/null +++ b/tests/roads/block_nested.eigs @@ -0,0 +1,8 @@ +# road-bind: created accumulator kept j +accumulator is 0 +kept is 100 +for kept in [1, 2]: + for j in [3, 4]: + created is kept + j + accumulator is accumulator + 1 + kept is kept + 10 diff --git a/tests/roads/block_nested.out b/tests/roads/block_nested.out new file mode 100644 index 00000000..8c316156 --- /dev/null +++ b/tests/roads/block_nested.out @@ -0,0 +1,4 @@ +["created", 1, 16] +["accumulator", 1, 4] +["kept", 1, 100] +["j", 0] diff --git a/tests/roads/blocks.eigs b/tests/roads/blocks.eigs new file mode 100644 index 00000000..c32beefc --- /dev/null +++ b/tests/roads/blocks.eigs @@ -0,0 +1,18 @@ +# road-bind: from_if from_loop from_try from_for from_fn +if 1 > 0: + from_if is 1 +loop while from_if < 2: + from_loop is 2 + from_if is from_if + 1 +try: + from_try is 3 +catch e: + from_try is -3 +for k in range of 1: + from_for is 4 +define fn() as: + for j in range of 2: + fin is j + return fin +from_fn is fn of [] +print of "blocks body" diff --git a/tests/roads/blocks.out b/tests/roads/blocks.out new file mode 100644 index 00000000..db4b469c --- /dev/null +++ b/tests/roads/blocks.out @@ -0,0 +1,6 @@ +blocks body +["from_if", 1, 2] +["from_loop", 1, 2] +["from_try", 1, 3] +["from_for", 1, 4] +["from_fn", 1, 1] diff --git a/tests/roads/chdir.eigs b/tests/roads/chdir.eigs new file mode 100644 index 00000000..b0cf1a12 --- /dev/null +++ b/tests/roads/chdir.eigs @@ -0,0 +1,6 @@ +# road-bind: value +# road-cwd: . +# road-cwd: . +chdir of "assets/shadow/B" +load_file of "assets/shadow/A/prog.eigs" +value is 1 diff --git a/tests/roads/chdir.out b/tests/roads/chdir.out new file mode 100644 index 00000000..6436b02f --- /dev/null +++ b/tests/roads/chdir.out @@ -0,0 +1,2 @@ +SCRIPTDIR-COPY +["value", 1, 1] diff --git a/tests/roads/eigs_modules/clobber_helper/clobber_helper.eigs b/tests/roads/eigs_modules/clobber_helper/clobber_helper.eigs new file mode 100644 index 00000000..41d260e2 --- /dev/null +++ b/tests/roads/eigs_modules/clobber_helper/clobber_helper.eigs @@ -0,0 +1,2 @@ +for k in range of 1: + outer is 2 diff --git a/tests/roads/eigs_modules/evalhelper/evalhelper.eigs b/tests/roads/eigs_modules/evalhelper/evalhelper.eigs new file mode 100644 index 00000000..761ad827 --- /dev/null +++ b/tests/roads/eigs_modules/evalhelper/evalhelper.eigs @@ -0,0 +1,4 @@ +define go() as: + return eval of "load_file of \"peer.eigs\"" +define direct() as: + return load_file of "peer.eigs" diff --git a/tests/roads/eigs_modules/evalhelper/peer.eigs b/tests/roads/eigs_modules/evalhelper/peer.eigs new file mode 100644 index 00000000..9d601c4e --- /dev/null +++ b/tests/roads/eigs_modules/evalhelper/peer.eigs @@ -0,0 +1 @@ +return "EVALHELPER-PEER" diff --git a/tests/roads/eigs_modules/evalleaf/evalleaf.eigs b/tests/roads/eigs_modules/evalleaf/evalleaf.eigs new file mode 100644 index 00000000..6ffc53b7 --- /dev/null +++ b/tests/roads/eigs_modules/evalleaf/evalleaf.eigs @@ -0,0 +1,3 @@ +import evalhelper +eval_value is evalhelper.go of [] +direct_value is evalhelper.direct of [] diff --git a/tests/roads/eigs_modules/evalleaf/peer.eigs b/tests/roads/eigs_modules/evalleaf/peer.eigs new file mode 100644 index 00000000..ad6442b0 --- /dev/null +++ b/tests/roads/eigs_modules/evalleaf/peer.eigs @@ -0,0 +1 @@ +return "LEAF-PEER" diff --git a/tests/roads/eigs_modules/evalnested/evalnested.eigs b/tests/roads/eigs_modules/evalnested/evalnested.eigs new file mode 100644 index 00000000..0b24b205 --- /dev/null +++ b/tests/roads/eigs_modules/evalnested/evalnested.eigs @@ -0,0 +1,3 @@ +import evalleaf +eval_value is evalleaf.eval_value +direct_value is evalleaf.direct_value diff --git a/tests/roads/eigs_modules/evalnested/peer.eigs b/tests/roads/eigs_modules/evalnested/peer.eigs new file mode 100644 index 00000000..599b154a --- /dev/null +++ b/tests/roads/eigs_modules/evalnested/peer.eigs @@ -0,0 +1 @@ +return "NESTED-PEER" diff --git a/tests/roads/eigs_modules/evalwrapper/evalwrapper.eigs b/tests/roads/eigs_modules/evalwrapper/evalwrapper.eigs new file mode 100644 index 00000000..6fc32889 --- /dev/null +++ b/tests/roads/eigs_modules/evalwrapper/evalwrapper.eigs @@ -0,0 +1,3 @@ +import evalhelper +print of (evalhelper.go of []) +print of (evalhelper.direct of []) diff --git a/tests/roads/eigs_modules/evalwrapper/peer.eigs b/tests/roads/eigs_modules/evalwrapper/peer.eigs new file mode 100644 index 00000000..3ee0a049 --- /dev/null +++ b/tests/roads/eigs_modules/evalwrapper/peer.eigs @@ -0,0 +1 @@ +return "WRAPPER-PEER" diff --git a/tests/roads/f29_loop_local.eigs b/tests/roads/f29_loop_local.eigs new file mode 100644 index 00000000..e0009b7a --- /dev/null +++ b/tests/roads/f29_loop_local.eigs @@ -0,0 +1,21 @@ +# road-bind: direct nested from_if from_while +for k in [1]: + local value is 10 + value is value + 1 + print of value + direct is value +for outer in [1]: + if 1: + local held is 20 + for inner in [1]: + held is held + 1 + nested is held +if 1: + local from_if is 30 + for k in [1]: + from_if is from_if + 1 +loop while 1: + local from_while is 40 + for k in [1]: + from_while is from_while + 1 + break diff --git a/tests/roads/f29_loop_local.out b/tests/roads/f29_loop_local.out new file mode 100644 index 00000000..201b02e0 --- /dev/null +++ b/tests/roads/f29_loop_local.out @@ -0,0 +1,5 @@ +11 +["direct", 1, 11] +["nested", 1, 21] +["from_if", 1, 31] +["from_while", 1, 41] diff --git a/tests/roads/f29_loop_local_cache.eigs b/tests/roads/f29_loop_local_cache.eigs new file mode 100644 index 00000000..53fecfb3 --- /dev/null +++ b/tests/roads/f29_loop_local_cache.eigs @@ -0,0 +1,9 @@ +# road-bind: item seen +item is 100 +seen is [] +for turn in range of 80: + if turn % 2 == 0: + local item is 10 + item is item + 1 + if turn < 4: + append of [seen, item] diff --git a/tests/roads/f29_loop_local_cache.out b/tests/roads/f29_loop_local_cache.out new file mode 100644 index 00000000..8f490a83 --- /dev/null +++ b/tests/roads/f29_loop_local_cache.out @@ -0,0 +1,2 @@ +["item", 1, 140] +["seen", 1, [11, 101, 11, 102]] diff --git a/tests/roads/f30_eval_dir.eigs b/tests/roads/f30_eval_dir.eigs new file mode 100644 index 00000000..fb73bdb8 --- /dev/null +++ b/tests/roads/f30_eval_dir.eigs @@ -0,0 +1,12 @@ +# road-bind: main_eval main_direct loaded_eval loaded_direct nested_eval nested_direct +import evalhelper +import evalwrapper +import evalnested +try: + main_eval is evalhelper.go of [] +catch err: + main_eval is "WRONG-DIR-ERROR" +main_direct is evalhelper.direct of [] +load_file of "assets/eval_loaded/caller.eigs" +nested_eval is evalnested.eval_value +nested_direct is evalnested.direct_value diff --git a/tests/roads/f30_eval_dir.out b/tests/roads/f30_eval_dir.out new file mode 100644 index 00000000..be4fd1fc --- /dev/null +++ b/tests/roads/f30_eval_dir.out @@ -0,0 +1,8 @@ +EVALHELPER-PEER +EVALHELPER-PEER +["main_eval", 1, "EVALHELPER-PEER"] +["main_direct", 1, "EVALHELPER-PEER"] +["loaded_eval", 1, "EVALHELPER-PEER"] +["loaded_direct", 1, "EVALHELPER-PEER"] +["nested_eval", 1, "EVALHELPER-PEER"] +["nested_direct", 1, "EVALHELPER-PEER"] diff --git a/tests/roads/hardlink_observer.eigs b/tests/roads/hardlink_observer.eigs new file mode 100644 index 00000000..2d3b6f6d --- /dev/null +++ b/tests/roads/hardlink_observer.eigs @@ -0,0 +1,6 @@ +# road-bind: metric +# road-hardlink: assets/hardlinks/a/entry.eigs assets/hardlinks/b/entry.eigs +metric is 0 +metric is 10 +load_file of "assets/hardlinks/a/entry.eigs" +load_file of "assets/hardlinks/b/entry.eigs" diff --git a/tests/roads/hardlink_observer.out b/tests/roads/hardlink_observer.out new file mode 100644 index 00000000..4509edeb --- /dev/null +++ b/tests/roads/hardlink_observer.out @@ -0,0 +1,3 @@ +idle +moving +["metric", 1, 10] diff --git a/tests/roads/importer_scope.eigs b/tests/roads/importer_scope.eigs new file mode 100644 index 00000000..4af35887 --- /dev/null +++ b/tests/roads/importer_scope.eigs @@ -0,0 +1,4 @@ +# road-bind: outer helper_has_outer +outer is 1 +import clobber_helper +helper_has_outer is has_key of [clobber_helper, "outer"] diff --git a/tests/roads/importer_scope.out b/tests/roads/importer_scope.out new file mode 100644 index 00000000..e3ab955d --- /dev/null +++ b/tests/roads/importer_scope.out @@ -0,0 +1,2 @@ +["outer", 1, 1] +["helper_has_outer", 1, 1] diff --git a/tests/roads/native_alternate.eigs b/tests/roads/native_alternate.eigs new file mode 100644 index 00000000..422afbc3 --- /dev/null +++ b/tests/roads/native_alternate.eigs @@ -0,0 +1,13 @@ +# road-bind: v total +# road-native: required +v is 100 +total is 0 +for i in range of 1200: + if i % 2 == 0: + local v is 10 + j is 0 + loop while j < 12: + v is v + 1 + total is total + v + j is j + 1 +print of [v, total] diff --git a/tests/roads/native_alternate.out b/tests/roads/native_alternate.out new file mode 100644 index 00000000..1e387a0a --- /dev/null +++ b/tests/roads/native_alternate.out @@ -0,0 +1,3 @@ +[7300, 26762400] +["v", 1, 7300] +["total", 1, 26762400] diff --git a/tests/roads/native_catch.eigs b/tests/roads/native_catch.eigs new file mode 100644 index 00000000..811943c1 --- /dev/null +++ b/tests/roads/native_catch.eigs @@ -0,0 +1,14 @@ +# road-bind: v total +# road-native: required +v is 100 +total is 0 +for i in range of 1200: + try: + throw of 10 + catch v: + k is 0 + loop while k < 12: + v is v + 1 + total is total + v + k is k + 1 +print of [v, total] diff --git a/tests/roads/native_catch.out b/tests/roads/native_catch.out new file mode 100644 index 00000000..83a84dff --- /dev/null +++ b/tests/roads/native_catch.out @@ -0,0 +1,3 @@ +[100, 237600] +["v", 1, 100] +["total", 1, 237600] diff --git a/tests/roads/native_inline.eigs b/tests/roads/native_inline.eigs new file mode 100644 index 00000000..473b5353 --- /dev/null +++ b/tests/roads/native_inline.eigs @@ -0,0 +1,11 @@ +# road-bind: v +# road-native: required +v is 100 +for i in [1]: + j is 0 + loop while j < 20000: + if j == 15000: + eval of "local v is 10" + v is j + j is j + 1 +print of v diff --git a/tests/roads/native_inline.out b/tests/roads/native_inline.out new file mode 100644 index 00000000..12f180dd --- /dev/null +++ b/tests/roads/native_inline.out @@ -0,0 +1,2 @@ +14999 +["v", 1, 14999] diff --git a/tests/roads/native_late.eigs b/tests/roads/native_late.eigs new file mode 100644 index 00000000..fb728a2b --- /dev/null +++ b/tests/roads/native_late.eigs @@ -0,0 +1,13 @@ +# road-bind: v total +# road-native: required +v is 100 +total is 0 +for i in range of 1200: + if i > 500: + local v is 10 + j is 0 + loop while j < 12: + v is v + 1 + total is total + v + j is j + 1 +print of [v, total] diff --git a/tests/roads/native_late.out b/tests/roads/native_late.out new file mode 100644 index 00000000..fbeebb42 --- /dev/null +++ b/tests/roads/native_late.out @@ -0,0 +1,3 @@ +[6112, 18814680] +["v", 1, 6112] +["total", 1, 18814680] diff --git a/tests/roads/native_match.eigs b/tests/roads/native_match.eigs new file mode 100644 index 00000000..d1f97d3f --- /dev/null +++ b/tests/roads/native_match.eigs @@ -0,0 +1,14 @@ +# road-bind: v total +# road-native: required +v is 100 +total is 0 +for i in range of 1200: + local v is 10 + match 10: + case v: + k is 0 + loop while k < 12: + v is v + 1 + total is total + v + k is k + 1 +print of [v, total] diff --git a/tests/roads/native_match.out b/tests/roads/native_match.out new file mode 100644 index 00000000..83a84dff --- /dev/null +++ b/tests/roads/native_match.out @@ -0,0 +1,3 @@ +[100, 237600] +["v", 1, 100] +["total", 1, 237600] diff --git a/tests/roads/native_outer.eigs b/tests/roads/native_outer.eigs new file mode 100644 index 00000000..c63abc7c --- /dev/null +++ b/tests/roads/native_outer.eigs @@ -0,0 +1,14 @@ +# road-bind: v total +# road-native: required +v is 100 +total is 0 +for i in range of 1200: + if i % 2 == 0: + local v is 10 + for b in [0, 1]: + k is 0 + loop while k < 12: + v is v + 1 + total is total + v + k is k + 1 +print of [v, total] diff --git a/tests/roads/native_outer.out b/tests/roads/native_outer.out new file mode 100644 index 00000000..a0fef0be --- /dev/null +++ b/tests/roads/native_outer.out @@ -0,0 +1,3 @@ +[14500, 105451200] +["v", 1, 14500] +["total", 1, 105451200] diff --git a/tests/roads/nested_load.eigs b/tests/roads/nested_load.eigs new file mode 100644 index 00000000..a8860a32 --- /dev/null +++ b/tests/roads/nested_load.eigs @@ -0,0 +1,5 @@ +# road-bind: sibling_result project_result delayed_result imported_result restored_result +load_file of "assets/project/sub/entry.eigs" +delayed_result is delayed of [] +imported_result is delayed_import of [] +load_file of "assets/restore.eigs" diff --git a/tests/roads/nested_load.out b/tests/roads/nested_load.out new file mode 100644 index 00000000..fd4fbda7 --- /dev/null +++ b/tests/roads/nested_load.out @@ -0,0 +1,7 @@ +sibling +project root +["sibling_result", 1, 21] +["project_result", 1, 22] +["delayed_result", 1, 23] +["imported_result", 1, 24] +["restored_result", 1, 25] diff --git a/tests/roads/observer_nested.eigs b/tests/roads/observer_nested.eigs new file mode 100644 index 00000000..1888d8a8 --- /dev/null +++ b/tests/roads/observer_nested.eigs @@ -0,0 +1,4 @@ +# road-bind: metric +metric is 0 +metric is 10 +load_file of "assets/observer/entry.eigs" diff --git a/tests/roads/observer_nested.out b/tests/roads/observer_nested.out new file mode 100644 index 00000000..dbf6293e --- /dev/null +++ b/tests/roads/observer_nested.out @@ -0,0 +1,2 @@ +moving +["metric", 1, 10] diff --git a/tests/roads/resolution_errors.eigs b/tests/roads/resolution_errors.eigs new file mode 100644 index 00000000..2f1829dd --- /dev/null +++ b/tests/roads/resolution_errors.eigs @@ -0,0 +1,4 @@ +# road-bind: no_parent no_cwd loaded_error import_error +# road-cwd: assets/no_project +# road-cwd: __unrelated_cwd +load_file of "assets/no_project/child/entry.eigs" diff --git a/tests/roads/resolution_errors.out b/tests/roads/resolution_errors.out new file mode 100644 index 00000000..8dedb170 --- /dev/null +++ b/tests/roads/resolution_errors.out @@ -0,0 +1,4 @@ +["no_parent", 1, 1] +["no_cwd", 1, 1] +["loaded_error", 1, 1] +["import_error", 1, 1] diff --git a/tests/roads/resolution_order.eigs b/tests/roads/resolution_order.eigs new file mode 100644 index 00000000..7049705a --- /dev/null +++ b/tests/roads/resolution_order.eigs @@ -0,0 +1,2 @@ +# road-bind: nearby package root_relative root_error +load_file of "assets/order/sub/entry.eigs" diff --git a/tests/roads/resolution_order.out b/tests/roads/resolution_order.out new file mode 100644 index 00000000..1b6ef8fe --- /dev/null +++ b/tests/roads/resolution_order.out @@ -0,0 +1,4 @@ +["nearby", 1, 1] +["package", 1, 2] +["root_relative", 1, 3] +["root_error", 1, 1] diff --git a/tests/roads/ret.eigs b/tests/roads/ret.eigs new file mode 100644 index 00000000..dbf51a5d --- /dev/null +++ b/tests/roads/ret.eigs @@ -0,0 +1,7 @@ +# road-bind: before after +# road-return: 42 +before is 7 +print of "before return" +return 42 +after is 99 +print of "unreachable" diff --git a/tests/roads/ret.out b/tests/roads/ret.out new file mode 100644 index 00000000..79552bd8 --- /dev/null +++ b/tests/roads/ret.out @@ -0,0 +1,3 @@ +before return +["before", 1, 7] +["after", 0] diff --git a/tests/roads/shadow.eigs b/tests/roads/shadow.eigs new file mode 100644 index 00000000..bf2c69fa --- /dev/null +++ b/tests/roads/shadow.eigs @@ -0,0 +1,5 @@ +# road-bind: loaded +# road-cwd: assets/shadow/A +# road-cwd: assets/shadow/B +load_file of "assets/shadow/A/prog.eigs" +loaded is 1 diff --git a/tests/roads/shadow.out b/tests/roads/shadow.out new file mode 100644 index 00000000..2045e8c8 --- /dev/null +++ b/tests/roads/shadow.out @@ -0,0 +1,2 @@ +SCRIPTDIR-COPY +["loaded", 1, 1] diff --git a/tests/run_all_tests.sh b/tests/run_all_tests.sh index 09a065f4..b45522a5 100755 --- a/tests/run_all_tests.sh +++ b/tests/run_all_tests.sh @@ -5,6 +5,7 @@ # intentionally hits a missing builtin in the minimal build) would abort the # whole suite. TESTS_DIR="$(cd "$(dirname "$0")" && pwd)" +export EIGS_TEST_DIR="$TESTS_DIR" cd "$(dirname "$0")/../src" || { echo "cannot cd to src"; exit 1; } PASS=0 @@ -3898,14 +3899,8 @@ check "one computed load poisons a unit that also has a literal one" "$OBS_G10" printf 'local lf is load_file\nlf of "%s/mfree.eigs"\nprint of (lf_helper of 1)\n' "$OBS_GATE_TMP" > "$OBS_GATE_TMP/lf_alias.eigs" OBS_G11=$(EIGS_OBS_GATE_STATS=1 $EIGS_BIN "$OBS_GATE_TMP/lf_alias.eigs" 2>&1 | grep -q 'obs-gate: observed' && echo open || echo closed) check "an ALIASED load_file keeps the gate open" "$OBS_G11" "open" -# 13. Resolver parity, via chdir — the THIRD route into the time-of-check / -# time-of-use family checked at 18-20. `chdir` used to be a one-element -# denylist in chunk_scan_static_loads that forced the gate open; that was the -# wrong population key (the same state is reachable by write_text, rename, -# mkdir, remove_file or a subprocess), so the denylist is gone and the -# outcome check at the load covers all of them. This asserts the ROUTE still -# ends soundly: cwd moves, the literal resolves to a DIFFERENT file, and that -# file observes -> raise, never a quiet `equilibrium`. +# 13. #1056: chdir cannot redirect a file's literal load. The observer-free +# containing-file copy must run, even with an observing copy in the new cwd. mkdir -p "$OBS_GATE_TMP/cdsub" printf 'print of "outer"\n' > "$OBS_GATE_TMP/cd_m.eigs" printf 'print of (report of y)\n' > "$OBS_GATE_TMP/cdsub/cd_m.eigs" @@ -3915,8 +3910,9 @@ printf 'y is 1.0\ny is 2.0\ny is 4.0\nlocal ok is chdir of "cdsub"\nload_file of # "the guard did not fire" rather than "the probe did not run" (§64: a probe # that cannot execute is not a probe). Resolve it to an absolute path first. OBS_ABS_BIN=$(cd "$(dirname "$EIGS_BIN")" && pwd)/$(basename "$EIGS_BIN") -OBS_G12=$( cd "$OBS_GATE_TMP" && "$OBS_ABS_BIN" "$OBS_GATE_TMP/lf_chdir.eigs" 2>&1 | grep -c 'reads observer state, but the observer gate was closed' ) -check "chdir resolving a literal to an OBSERVING file raises, not answers" "$OBS_G12" "1" +OBS_G12=$( cd "$OBS_GATE_TMP" && "$OBS_ABS_BIN" "$OBS_GATE_TMP/lf_chdir.eigs" 2>&1 ); OBS_G12_RC=$? +if ! rc_ok "$OBS_G12_RC" "$OBS_G12"; then OBS_G12="died-rc$OBS_G12_RC"; fi +check "chdir cannot redirect a file-relative literal load" "$OBS_G12" "outer" # 14. TRANSITIVE: the parent's literal load reaches an observer two modules down. # Asserted on the VALUE — the gate's own stats cannot see a wrong answer. printf 'print of (report of q)\n' > "$OBS_GATE_TMP/lf_inner.eigs" @@ -3984,7 +3980,7 @@ check "a module that fails to compile reports IDENTICALLY under the gate" "$OBS_ # when the parent COMPILES; load_file reads it again when the call RUNS, and # the whole program runs in between. Found by a blind critic with two # executed repros, both silently wrong (`equilibrium` under the gate, -# `moving` without it) — a rewrite of the module, and a cwd file SHADOWING +# `moving` without it) — a rewrite of the module, and a nearer file SHADOWING # the resolved one. An earlier draft tried to enumerate the causes and # shipped a one-element `chdir` denylist; the guard is now on the OUTCOME # (the observer bit flipping 0->1 at the load) and needs no such list. @@ -4258,9 +4254,9 @@ check "control: that leaf loaded once also closes" "$OBS_G31C" "closed" # then lost the gate anyway. This pins the budget to the population rather # than to the number (§60) — if lib/ui grows past it, or someone lowers the # budget, this fails and the value gets re-picked deliberately. -# (cwd here is src/, so the stdlib tree is ../lib — ui.eigs's own internal -# loads resolve through the exe-relative stdlib mechanism.) -printf 'load_file of "../lib/ui.eigs" +# #1056: use a stdlib request, not a cwd-relative path from the temporary +# program. ui.eigs's internal loads use the same resolver. +printf 'load_file of "lib/ui.eigs" print of "ok" ' > "$OBS_GATE_TMP/uitree.eigs" OBS_G32=$(obs_gate_closed_verdict "$OBS_GATE_TMP/uitree.eigs" ok 120) @@ -5701,6 +5697,22 @@ else fi echo "" +# The road gate enumerates disk fixtures and checks each road against a golden +# stdout as well as its peers. Its selftest must prove both divergence and +# empty enumeration fail. A selected-fixture diagnostic run is never used here. +echo "[99z] File semantics across main/load_file/import (#1056)" +TOTAL=$((TOTAL + 1)) +if bash "$TESTS_DIR/../tools/road_diff.sh" && \ + bash "$TESTS_DIR/../tools/road_diff.sh" --selftest && \ + python3 "$TESTS_DIR/../tools/embed_roads.py" --selftest; then + PASS=$((PASS + 1)) + echo " PASS: road differential and planted faults" +else + FAIL=$((FAIL + 1)) + echo " FAIL: road differential or planted faults" +fi +echo "" + echo "[99b] Stdlib/builtin discoverability (#393)" TOTAL=$((TOTAL + 1)) if bash "$TESTS_DIR/../tools/stdlib_index_check.sh" && bash "$TESTS_DIR/../tools/stdlib_index_check.sh" --selftest >/dev/null; then diff --git a/tests/run_install_smoke_subset.sh b/tests/run_install_smoke_subset.sh index b77cbff3..aaaed668 100755 --- a/tests/run_install_smoke_subset.sh +++ b/tests/run_install_smoke_subset.sh @@ -5,6 +5,7 @@ set -euo pipefail TESTS_DIR="$(cd "$(dirname "$0")" && pwd)" +export EIGS_TEST_DIR="$TESTS_DIR" EIGS="${EIGENSCRIPT:-$HOME/.local/bin/eigenscript}" SUITE_ROOT=$(mktemp -d /tmp/eigs_install_subset_suite.XXXXXX) external_home="" diff --git a/tests/test_import.eigs b/tests/test_import.eigs index 8b8abbce..27690958 100644 --- a/tests/test_import.eigs +++ b/tests/test_import.eigs @@ -1,4 +1,10 @@ # Import system tests +# The runners provide the canonical source directory, including for symlinked +# installed-layout tests. Standalone invocation from src/ keeps the same path. +_module_test_dir is env_get of "EIGS_TEST_DIR" +if _module_test_dir == "": + _module_test_dir is (getcwd of null) + "/../tests" + load_file of "lib/test.eigs" # Import creates a dict namespace @@ -43,31 +49,31 @@ assert_eq of [result, "3.14", "import format.fmt_num"] # --- User modules: .eigs resolved relative to the script is tried # FIRST, before the stdlib's lib/.eigs (#821). Same namespacing. -# (The suite runs from src/, so the script dir is ../tests.) -write_text of ["../tests/tmp_user_module.eigs", "FACTOR is 7\ndefine scaled(x) as:\n return x * FACTOR\n"] +# Generated modules use the canonical containing directory in both layouts. +write_text of [(_module_test_dir + "/tmp_user_module.eigs"), "FACTOR is 7\ndefine scaled(x) as:\n return x * FACTOR\n"] import tmp_user_module assert_eq of [type of tmp_user_module, "dict", "user module imports as dict"] assert_eq of [tmp_user_module.scaled of 6, 42, "user module function works"] assert_eq of [tmp_user_module.FACTOR, 7, "user module constant"] -rm of "../tests/tmp_user_module.eigs" +rm of (_module_test_dir + "/tmp_user_module.eigs") # Names starting with _ are module-private: omitted from the dict. -write_text of ["../tests/tmp_priv_module.eigs", "_secret is 1\nvisible is 2\n"] +write_text of [(_module_test_dir + "/tmp_priv_module.eigs"), "_secret is 1\nvisible is 2\n"] import tmp_priv_module assert_eq of [has_key of [tmp_priv_module, "visible"], 1, "public name exported"] assert_eq of [has_key of [tmp_priv_module, "_secret"], 0, "_name stays private"] -rm of "../tests/tmp_priv_module.eigs" +rm of (_module_test_dir + "/tmp_priv_module.eigs") # --- #821: a project file with a stdlib module's name wins (project-first). # Before #821, `import physics` here silently bound the stdlib's # lib/physics.eigs and every member access on the intended module read # null (dynamics F-DYN-8). The collision warning itself is asserted at # the shell level in run_all_tests.sh (it goes to stderr). -write_text of ["../tests/physics.eigs", "SHADOW_MARKER is 821\n"] +write_text of [(_module_test_dir + "/physics.eigs"), "SHADOW_MARKER is 821\n"] import physics assert_eq of [has_key of [physics, "SHADOW_MARKER"], 1, "project file shadows stdlib (#821)"] assert_eq of [physics.SHADOW_MARKER, 821, "shadowing module's own binding read"] -rm of "../tests/physics.eigs" +rm of (_module_test_dir + "/physics.eigs") # Missing module raises a catchable error naming both tried paths. import_err is "" diff --git a/tests/test_import_errors.eigs b/tests/test_import_errors.eigs index 2917fb77..a068964b 100644 --- a/tests/test_import_errors.eigs +++ b/tests/test_import_errors.eigs @@ -1,9 +1,15 @@ # Import error-path coverage tests -# Exercises eval.c lines 539-562: fallback not-found, cannot-read, parse-errors +# Exercises import/load_file not-found and parse-error paths. +# The runners provide the canonical source directory, including for symlinked +# installed-layout tests. Standalone invocation from src/ keeps the same path. +_module_test_dir is env_get of "EIGS_TEST_DIR" +if _module_test_dir == "": + _module_test_dir is (getcwd of null) + "/../tests" + load_file of "lib/test.eigs" # ---- Test 1: Import a module that does not exist anywhere ---- -# Searches lib/, script_dir/lib/, script_dir/../lib/, ~/.local/lib/eigenscript/ +# Searches the containing directory, eigs_modules, project root, and stdlib roots. # All will fail for a nonsense module name. err1 is "" try: @@ -13,9 +19,8 @@ catch e: assert_true of [contains of [err1.message, "not found"], "import nonexistent triggers 'not found' error"] # ---- Test 2: Import module with parse errors ---- -# Write a file with invalid syntax into script_dir/../lib/ which is where -# the runtime finds stdlib modules. Clean up after. -bad_path is "../lib/_test_bad_syntax_tmp.eigs" +# Write invalid syntax beside this test file. Clean up after. +bad_path is (_module_test_dir + "/_test_bad_syntax_tmp.eigs") write_text of [bad_path, "this is not valid eigenscript @@@"] err2 is "" try: @@ -40,7 +45,7 @@ import math assert_eq of [type of math, "dict", "import valid module returns dict"] # ---- Test 5: Import module that is empty (no exports) ---- -empty_path is "../lib/_test_empty_tmp.eigs" +empty_path is (_module_test_dir + "/_test_empty_tmp.eigs") write_text of [empty_path, "# empty module\n_private is 42"] import _test_empty_tmp rm of empty_path @@ -51,7 +56,7 @@ assert_eq of [type of _test_empty_tmp, "dict", "import empty module returns dict # parse — here an expression-rooted lvalue `(call).field is x`, which direct # execution rejects — was silently accepted under load_file and its effect # dropped. load_file now raises a catchable parse error, like eval and import. -bad_lf_path is "../lib/_test_bad_loadfile_tmp.eigs" +bad_lf_path is (_module_test_dir + "/_test_bad_loadfile_tmp.eigs") write_text of [bad_lf_path, "define mk as:\n return {\"x\": 1}\n(mk of null).x is 99\n"] err_lf is "" try: @@ -63,7 +68,7 @@ assert_true of [(len of err_lf) > 0, "load_file parse-error raises (not a silent assert_true of [contains of [err_lf.message, "parse error"], "load_file parse-error message contains 'parse error'"] # ---- Test 7: a clean load_file is unaffected ---- -ok_lf_path is "../lib/_test_ok_loadfile_tmp.eigs" +ok_lf_path is (_module_test_dir + "/_test_ok_loadfile_tmp.eigs") write_text of [ok_lf_path, "_lf_ok_marker is 1234\n"] load_file of ok_lf_path rm of ok_lf_path diff --git a/tests/test_import_toplevel_scope.eigs b/tests/test_import_toplevel_scope.eigs index 1d08d301..6a6527c1 100644 --- a/tests/test_import_toplevel_scope.eigs +++ b/tests/test_import_toplevel_scope.eigs @@ -15,7 +15,13 @@ # --- Case 1 (import): module top-level state is insulated from a # same-named importer variable, both at import time and across repeated # calls into the module's own functions. --- -write_text of ["tmp_589_cmod.eigs", "counter is 0\ndefine bump() as:\n counter is counter + 1\n return counter\n"] +# The runners provide the canonical source directory, including for symlinked +# installed-layout tests. Standalone invocation from src/ keeps the same path. +_module_test_dir is env_get of "EIGS_TEST_DIR" +if _module_test_dir == "": + _module_test_dir is (getcwd of null) + "/../tests" + +write_text of [(_module_test_dir + "/tmp_589_cmod.eigs"), "counter is 0\ndefine bump() as:\n counter is counter + 1\n return counter\n"] counter is [9, 9, 9] import tmp_589_cmod @@ -27,22 +33,22 @@ assert of [(tmp_589_cmod.bump of null) == 3, "I5 module state keeps accumulating assert of [(type of counter) == "list", "I6 importer's counter still untouched after repeated module calls"] assert of [counter == [9, 9, 9], "I7 importer's counter value still unchanged"] -remove_file of "tmp_589_cmod.eigs" +remove_file of (_module_test_dir + "/tmp_589_cmod.eigs") # --- Case 2 (load_file): documented contract is UNCHANGED — a load_file'd # file's top-level statements still execute directly in the CURRENT # (caller's) scope, so a same-named top-level assignment DOES bind through # and DOES clobber the caller's existing binding. This is load_file's older, # intentional semantics, not the #589 bug, and the fix must not touch it. --- -write_text of ["tmp_589_lfmod.eigs", "counterLF is 0\ndefine bumpLF() as:\n counterLF is counterLF + 1\n return counterLF\n"] +write_text of [(_module_test_dir + "/tmp_589_lfmod.eigs"), "counterLF is 0\ndefine bumpLF() as:\n counterLF is counterLF + 1\n return counterLF\n"] counterLF is [9, 9, 9] -load_file of "tmp_589_lfmod.eigs" +load_file of (_module_test_dir + "/tmp_589_lfmod.eigs") assert of [(type of counterLF) == "num", "L1 load_file's top-level init binds through to the current scope"] assert of [counterLF == 0, "L2 current-scope counterLF now holds the loaded file's value"] assert of [(bumpLF of null) == 1, "L3 bumpLF reads/writes the (now shared) current-scope binding"] assert of [counterLF == 1, "L4 the write is visible in the current scope too — same binding"] -remove_file of "tmp_589_lfmod.eigs" +remove_file of (_module_test_dir + "/tmp_589_lfmod.eigs") print of "All import top-level scope tests passed" diff --git a/tests/test_module_scope.eigs b/tests/test_module_scope.eigs index 7cd3a097..e81d5c46 100644 --- a/tests/test_module_scope.eigs +++ b/tests/test_module_scope.eigs @@ -5,7 +5,14 @@ # statements still execute in the current scope, and same-file outward # mutation is unchanged. -fixture is "tmp_mod_373.eigs" +# Generated modules live beside this source file (#1056). +# The runners provide the canonical source directory, including for symlinked +# installed-layout tests. Standalone invocation from src/ keeps the same path. +_module_test_dir is env_get of "EIGS_TEST_DIR" +if _module_test_dir == "": + _module_test_dir is (getcwd of null) + "/../tests" + +fixture is (_module_test_dir + "/tmp_mod_373.eigs") write_text of [fixture, "define mod_touch() as:\n y373 is 777\n pos373 is 777\ndefine mod_read() as:\n return cfg373\ndefine mod_call() as:\n return helper373 of null\nmod_count is 0\ndefine mod_bump() as:\n mod_count is mod_count + 1\n return mod_count\ndefine mod_poke(xs) as:\n xs[0] is 99\n"] # Case 1: caller globals declared BEFORE the load — the order that used to @@ -45,12 +52,12 @@ samefile_touch of null assert of [x373 == 99, "M8 same-file outward mutation still works"] # import direction: an imported module's fn is insulated the same way -write_text of ["tmp_mod_373i.eigs", "define imp_touch() as:\n z373 is 888\n"] +write_text of [(_module_test_dir + "/tmp_mod_373i.eigs"), "define imp_touch() as:\n z373 is 888\n"] z373 is 7 import tmp_mod_373i tmp_mod_373i.imp_touch of null assert of [z373 == 7, "M9 imported module fn cannot clobber caller global"] remove_file of fixture -remove_file of "tmp_mod_373i.eigs" +remove_file of (_module_test_dir + "/tmp_mod_373i.eigs") print of "All module-scope tests passed" diff --git a/tools/embed_roads.py b/tools/embed_roads.py new file mode 100644 index 00000000..cef083b7 --- /dev/null +++ b/tools/embed_roads.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +"""Run the embedding provenance test with either CLI build layout.""" +import argparse +import os +from pathlib import Path +import re +import shutil +import subprocess +import sys +import tempfile + +ROOT = Path(__file__).resolve().parent.parent + + +def select_variant(root, sanitize=False): + # build.sh creates a standalone CLI; make uses an objdir hard link. + # With no unique match, build from the same SOURCES via embed-roads. + cli = root / 'src/eigenscript' + cli.stat() # A missing CLI is still a setup failure, not a fallback case. + variants = [p.parent.name for p in (root / 'build').glob('*/eigenscript') + if p.samefile(cli)] + variant = variants[0] if len(variants) == 1 else ('asan' if sanitize else 'release') + return variant, len(variants) + + +def run(binary, fixtures, *extra): + try: + result = subprocess.run([str(binary), str(fixtures), *extra], capture_output=True, + text=True, timeout=30) + except (OSError, subprocess.TimeoutExpired) as error: + return 1, f"embed_roads: FAIL: {error}\n" + totals = re.findall(r"^embed_roads: checks=(\d+) scope_checks=(\d+) failures=(\d+)$", + result.stdout, re.M) + good = (result.returncode == 0 and not result.stderr and len(totals) == 1 and + int(totals[0][0]) >= 20 and int(totals[0][1]) >= 12 and int(totals[0][2]) == 0) + output = result.stdout + result.stderr + if not good: + output += f"embed_roads: FAIL: child rc={result.returncode}, expected clean exit and nonempty checks\n" + return int(not good), output + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--binary', type=Path) + parser.add_argument('--selftest', action='store_true') + args = parser.parse_args() + binary = args.binary + if binary is None: + try: + variant, matches = select_variant(ROOT, bool(os.environ.get('ASAN_OPTIONS'))) + except OSError as error: + print(f'embed_roads: FAIL: cannot inspect CLI variant: {error}') + return 1 + print(f'embed_roads: build={variant} ({matches} matching CLI variants; ' + + ('matching objects)' if matches == 1 else 'dedicated source build)')) + build = subprocess.run(['make', '-s', '-C', str(ROOT), 'embed-roads', + 'ROAD_VARIANT=' + variant], capture_output=True, text=True) + if build.returncode: + print('embed_roads: FAIL: build failed\n' + build.stdout + build.stderr) + return 1 + binary = ROOT / 'build' / variant / 'embed_roads' + binary = binary.resolve() + status, output = run(binary, ROOT / 'tests/embed_roads') + print(output, end='') + if status or not args.selftest: + return status + with tempfile.TemporaryDirectory(prefix='embed-road-plants-') as tmp: + layout = Path(tmp) / 'layout' + (layout / 'src').mkdir(parents=True) + cli = layout / 'src/eigenscript' + cli.touch() # Metadata control, never a copied runtime binary. + if select_variant(layout) != ('release', 0): + print('embed_roads selftest: FAIL: standalone build.sh layout') + return 1 + for name in ('asan', 'release'): + target = layout / 'build' / name / 'eigenscript' + target.parent.mkdir(parents=True) + os.link(cli, target) + expected = ('asan', 1) if name == 'asan' else ('release', 2) + if select_variant(layout) != expected: + print(f'embed_roads selftest: FAIL: {name} layout: {expected}') + return 1 + if select_variant(layout, sanitize=True) != ('asan', 2): + print('embed_roads selftest: FAIL: ambiguous sanitizer layout') + return 1 + print('embed_roads selftest: GREEN: standalone, unique, ambiguous, sanitizer layouts') + status, output = run(binary, ROOT / 'tests/embed_roads', '--plant-override') + if (status == 0 or 'FAIL: execution override at planted execution: planted-override' not in output or + 'checks=32 scope_checks=21 failures=1' not in output or + 'FAIL: child rc=1,' not in output): + print('embed_roads selftest: FAIL: execution-scope probe missed planted override\n' + output) + return 1 + print('embed_roads selftest: RED: execution-scope override') + tree = Path(tmp) / 'fixtures' + shutil.copytree(ROOT / 'tests/embed_roads', tree) + peer = tree / 'helper/peer.eigs' + peer.write_text(peer.read_text().replace('"HELPER"', '"WRONG"')) + status, output = run(binary, tree) + if status == 0 or 'embed_roads: FAIL: eval_file expected HELPER / HELPER' not in output: + print('embed_roads selftest: FAIL: wrong helper peer survived\n' + output) + return 1 + print('embed_roads selftest: RED: wrong helper peer') + status, output = run(binary, tree / 'missing') + if status == 0 or 'embed_roads: FAIL: missing fixture tree' not in output: + print('embed_roads selftest: FAIL: missing fixture tree survived\n' + output) + return 1 + print('embed_roads selftest: RED: missing fixture tree') + # A correct C result cannot excuse a bad process envelope or zero work. + for symptom in ('exit', 'stderr', 'zero_checks'): + wrapper = Path(tmp) / symptom + wrapper.write_text(f'#!{sys.executable}\n' + 'import re, subprocess, sys\n' + f'r = subprocess.run([{str(binary)!r}, *sys.argv[1:]], capture_output=True, timeout=20)\n' + 'if r.returncode or r.stderr: raise SystemExit(99)\n' + 'out = r.stdout\n' + + ('out = re.sub(rb"checks=\\d+ scope_checks=\\d+", b"checks=0 scope_checks=0", out)\n' + if symptom == 'zero_checks' else '') + + 'sys.stdout.buffer.write(out)\n' + + ('raise SystemExit(17)\n' if symptom == 'exit' else + 'sys.stderr.write("planted warning\\n")\n' if symptom == 'stderr' else '')) + wrapper.chmod(0o755) + status, output = run(wrapper, ROOT / 'tests/embed_roads') + expected_rc = 17 if symptom == 'exit' else 0 + expected_checks = 'checks=0 scope_checks=0' if symptom == 'zero_checks' else 'checks=31 scope_checks=20' + if (status == 0 or f'embed_roads: FAIL: child rc={expected_rc},' not in output or + expected_checks + ' failures=0' not in output or + ('planted warning\n' in output) != (symptom == 'stderr')): + print(f'embed_roads selftest: FAIL: {symptom} survived\n' + output) + return 1 + print(f'embed_roads selftest: RED: {symptom}') + print('embed_roads selftest: controls=5 plants=6 failures=0') + return 0 + + +if __name__ == '__main__': + raise SystemExit(main()) diff --git a/tools/road_diff.py b/tools/road_diff.py new file mode 100644 index 00000000..a1c62dc2 --- /dev/null +++ b/tools/road_diff.py @@ -0,0 +1,533 @@ +#!/usr/bin/env python3 +"""Compare a file's prints and declared final bindings on all three roads. + +See tests/roads/README.md for the wrapper/fixture contract. Only the driver's +completion marker is removed from stdout. No tolerated child errors or fixture +allowlist. All children have a deadline. +""" +import argparse +import contextlib +import difflib +import io +import os +import platform +from pathlib import Path +import re +import shutil +import subprocess +import sys +import tempfile +import time +import uuid + +ROOT = Path(__file__).resolve().parent.parent +BINDING_NAME = re.compile(r"[A-Za-z][A-Za-z0-9_]*") + + +def metadata(source, tag): + return re.findall(r"^# road-" + tag + r": (.*)$", source, re.M) + + +def driver_context(): + # Every generated binding gets a fresh, private name, including temporaries. + # Capture all driver builtins BEFORE the fixture can rebind their names. + prefix = "_road_" + uuid.uuid4().hex + "_" + captures = {name: prefix + name for name in + ("print", "has_key", "load_file", "throw", "result", "error")} + prelude = "".join(f"{captures[name]} is {name}\n" for name in + ("print", "has_key", "load_file", "throw")) + return captures, prelude + + +def snapshot(names, captures, module=None): + lines = [] + for name in names: + if not BINDING_NAME.fullmatch(name): + raise ValueError(f"invalid snapshot name: {name}") + # An absent namespace key is null; a missing bare binding raises. + # Presence is a separate field, never a value-domain sentinel. A + # present value (including null or "") occupies a third field. + if module: + lines += [f'if {captures["has_key"]} of [{module}, "{name}"]:', + f' {captures["print"]} of ["{name}", 1, {module}.{name}]', + 'else:', f' {captures["print"]} of ["{name}", 0]'] + else: + lines += ['try:', f' {captures["print"]} of ["{name}", 1, {name}]', + f'catch {captures["error"]}:', f' {captures["print"]} of ["{name}", 0]'] + return "\n".join(lines) + "\n" + + +def run_gate(binary, fixtures, only=None, *, architecture=None): + # Explicit architecture is for the selftest of the ARM64 CI policy. + architecture = architecture or platform.machine().lower() + paths = sorted(fixtures.glob("*.eigs")) + if only: + paths = [p for p in paths if p.stem == only] + if not paths: + print("road_diff: FAIL: zero fixtures", flush=True) + return 1 + failures = runs = 0 + started = time.monotonic() + with tempfile.TemporaryDirectory(prefix="eigs-roads-") as tmp: + scratch = Path(tmp) + for fixture in paths: + source = fixture.read_text() + bindings = metadata(source, "bind") + if len(bindings) != 1 or not bindings[0].split(): + print(f"road_diff: FAIL: {fixture.name}: missing/nonunique road-bind") + failures += 1 + continue + names = bindings[0].split() + invalid = next((name for name in names if not BINDING_NAME.fullmatch(name)), None) + if invalid is not None: + print(f"road_diff: FAIL: {fixture.name}: invalid snapshot name: {invalid}") + failures += 1 + continue + expected = fixture.with_suffix(".out") + if not expected.is_file() or not expected.read_bytes(): + print(f"road_diff: FAIL: {fixture.name}: missing/empty expected stdout") + failures += 1 + continue + native_tags = metadata(source, "native") + if native_tags and native_tags != ["required"]: + print(f"road_diff: FAIL: {fixture.name}: invalid road-native metadata") + failures += 1 + continue + native = bool(native_tags) + outputs = [] + cwds = metadata(source, "cwd") or [".", "__unrelated_cwd"] + tiers = ("ref", "jit", "osr") if native else ("ambient",) + if native and architecture in ("arm64", "aarch64"): + # jit.h has no ARM64 emitter. Still run all roads, explicitly + # interpreter-only; no non-running native arm is called green. + tiers = ("ref",) + print(f"road_diff: {fixture.name}: native tiers unavailable on ARM64; interpreter only") + for tier in tiers: + for road in ("main", "load_file", "import"): + for ci, cwd in enumerate(cwds): + tree = scratch / f"{fixture.stem}-{tier}-{road}-{ci}" + shutil.copytree(fixtures, tree) + for pair in metadata(source, "hardlink"): + src, dst = pair.split() + (tree / dst).unlink() + os.link(tree / src, tree / dst) + own = tree / fixture.name + captures, prelude = driver_context() + report = snapshot(names, captures, fixture.stem if road == "import" else None) + marker = "__road_complete_" + uuid.uuid4().hex + "__" + marker_bytes = (marker + "\n").encode() + report += f'{captures["print"]} of "{marker}"\n' + if road == "main": + # A top-level return bypasses a suffix. Snapshot just before + # each unindented return, and at normal end of the file. + body = re.sub(r"(?m)^(return(?:\s.*)?)$", lambda m: report + m[0], source) + own.write_text(prelude + body + "\n" + report) + entry = own + else: + entry = tree / "_road_driver.eigs" + if road == "import": + body = f"import {fixture.stem}\n" + else: + body = f'{captures["result"]} is {captures["load_file"]} of "{fixture.name}"\n' + returns = metadata(source, "return") + if returns: + body += (f'if {captures["result"]} != ({returns[0]}):\n' + f' {captures["throw"]} of "road return value mismatch"\n') + entry.write_text(prelude + body + report) + workdir = tree / cwd + workdir.mkdir(parents=True, exist_ok=True) + if ci == 1: + alias = tree / "__entry_alias" / "entry.eigs" + alias.parent.mkdir() + alias.symlink_to(entry) + entry = alias + env = os.environ.copy() + # Do not inherit a tape from the caller. These are deterministic + # fixtures; sanitizers and execution-tier flags remain enabled. + for key in ("EIGS_TRACE", "EIGS_REPLAY"): + env.pop(key, None) + env["HOME"] = str(scratch / "empty-home") + if native: + for key in ("EIGS_JIT_OFF", "EIGENSCRIPT_JIT_FORCE_OFF", + "EIGS_JIT_OSR_THRESHOLD", "EIGS_JIT_OSR_OFF", "EIGS_JIT_STATS", "EIGS_JIT_STOPS"): + env.pop(key, None) + env["EIGS_JIT_STATS"] = "1" + if tier == "ref": + env["EIGS_JIT_OFF"] = "1" + elif tier == "osr": + env["EIGS_JIT_OSR_THRESHOLD"] = "1" + runs += 1 + try: + result = subprocess.run([str(binary), str(entry)], cwd=workdir, + env=env, capture_output=True, timeout=30) + except (OSError, subprocess.TimeoutExpired) as err: + print(f"road_diff: FAIL: {fixture.name} {road} cwd={cwd}: {err}") + failures += 1 + continue + stderr = result.stderr + tier_ok = True + if native: + stats = re.findall(rb"(?m)^\[jit\] scanned=(\d+) compiled=(\d+) cache_used=(\d+)\n", stderr) + tier_ok = len(stats) == 1 and (int(stats[0][1]) == 0 if tier == "ref" else int(stats[0][1]) > 0) + if len(stats) == 1: + stats_line = b"[jit] scanned=%s compiled=%s cache_used=%s\n" % stats[0] + stderr = stderr.replace(stats_line, b"", 1) + print(f"road_diff: native {fixture.name} {road} tier={tier} cwd={cwd}: " + + stats_line.decode().strip()) + if not tier_ok: + print(f"road_diff: FAIL: {fixture.name} {road} tier={tier} cwd={cwd}: " + "missing/wrong native mechanism (ref requires compiled=0; jit/osr require compiled>0)") + complete = (result.stdout.endswith(marker_bytes) and + result.stdout.count(marker_bytes) == 1) + stdout = result.stdout[:-len(marker_bytes)] if complete else result.stdout + outputs.append((road, cwd, result.returncode, stdout)) + # Strictly require clean stderr as well: an ASan/UBSan warning + # at exit 0 must never get hidden behind a matching stdout. + if result.returncode or stderr or not complete or not tier_ok or stdout != expected.read_bytes(): + failures += 1 + print(f"road_diff: FAIL: {fixture.name} {road} cwd={cwd} rc={result.returncode}" + + (f" tier={tier}" if native else "")) + if not complete: + print("missing driver completion marker") + print(stderr.decode(errors="replace"), end="") + print("".join(difflib.unified_diff( + expected.read_text().splitlines(True), + stdout.decode(errors="replace").splitlines(True), + fromfile="expected", tofile=f"{road} stdout")), end="") + if outputs and any(row[2:] != outputs[0][2:] for row in outputs[1:]): + failures += 1 + print(f"road_diff: FAIL: {fixture.name}: roads/cwds diverge") + elif len(outputs) == len(tiers) * 3 * len(cwds): + print(f"road_diff: compared {fixture.name} ({len(outputs)} runs)") + print(f"road_diff: fixtures={len(paths)} runs={runs} failures={failures} " + f"seconds={time.monotonic() - started:.2f}", flush=True) + return int(failures != 0) + + +def selftest(binary, bad_binary=None): + controls = plants = 0 + with tempfile.TemporaryDirectory(prefix="eigs-road-plants-") as tmp: + tree = Path(tmp) + + def require_green(label): + nonlocal controls + if run_gate(binary, tree): + print(f"road_diff selftest: FAIL: {label}") + return False + controls += 1 + print(f"road_diff selftest: GREEN: {label}") + return True + + def require_red(label, fixture_name, missing=None, runner=binary, *, + roads=("main", "load_file", "import"), rc=0, + divergence=False, actual_line=None, stdout_matches=False, + diagnostic=None): + nonlocal plants + captured = io.StringIO() + with contextlib.redirect_stdout(captured): + status = run_gate(runner, tree) + output = captured.getvalue() + rows = re.findall(r"^road_diff: FAIL: " + re.escape(fixture_name) + + r" (main|load_file|import) cwd=.* rc=(-?\d+)$", + output, re.M) + # Require the exact road/status/diagnostic pattern of the plant, + # not a dead binary, timeout, or unrelated parse failure. + expected_rows = sorted((road, str(rc)) for road in roads for _ in range(2)) + expected_runs = len(expected_rows) + valid = status != 0 and sorted(rows) == expected_rows + if missing: + actual_line = f'["{missing}", 0]' + if actual_line: + valid = valid and output.count(f'+{actual_line}\n') == expected_runs + if divergence: + valid = valid and f"{fixture_name}: roads/cwds diverge" in output + if stdout_matches: + valid = valid and "--- expected\n" not in output + valid = valid and "roads/cwds diverge" not in output + if diagnostic: + valid = valid and output.count(diagnostic + "\n") == expected_runs + if not valid: + print(f"road_diff selftest: FAIL: {label}\n{output}") + return False + print(f"road_diff selftest: RED: {label}") + plants += 1 + return True + + fixture = tree / "planted.eigs" + fixture.write_text('# road-bind: value\nvalue is 7\n') + fixture.with_suffix('.out').write_text('["value", 1, 7]\n') + if not require_green("numeric binding"): + return 1 + # Each road runs in an isolated directory. A cwd-printing fixture is + # deliberately outside the deterministic fixture contract and MUST + # produce a named cross-road disagreement (not merely a golden diff). + fixture.write_text('# road-bind: value\nvalue is getcwd of null\n') + if not require_red("planted.eigs: roads/cwds diverge", fixture.name, divergence=True): + return 1 + fixture.unlink() + + fixture = tree / "sentinel.eigs" + fixture.write_text('# road-bind: from_for\nfor k in range of 1:\n from_for is ""\n') + fixture.with_suffix('.out').write_text('["from_for", 1, ""]\n') + if not require_green('literal "" is present'): + return 1 + # Default: delete the assignment, retaining the present-value golden. + # For an actual compiler regression proof, --bad-binary runs the same + # unmodified fixture against a binary that drops the import binding. + # The default stays portable and does not depend on a stale checkout. + if not bad_binary: + fixture.write_text('# road-bind: from_for\nfor k in range of 1:\n 0\n') + plant = "known-bad binary, import only" if bad_binary else "assignment deleted" + bad_roads = ("import",) if bad_binary else ("main", "load_file", "import") + if not require_red(f'literal "" binding dropped ({plant})', + fixture.name, "from_for", bad_binary or binary, roads=bad_roads): + return 1 + fixture.unlink() + + fixture = tree / "rebind_print.eigs" + fixture.write_text('# road-bind: print\nemit is print\nfor k in range of 1:\n' + ' print is (args) => emit of ["print", 0]\n') + fixture.with_suffix('.out').write_text('["print", 1, >]\n') + if not require_green("rebinding print cannot forge readback"): + return 1 + if bad_binary: + valid = require_red("print rebinding: dropped import binding (known-bad binary)", + fixture.name, "print", bad_binary, roads=("import",)) + else: + fixture.with_suffix('.out').write_text('["print", 0]\n') + valid = require_red("print rebinding: forged absence golden", fixture.name, + actual_line='["print", 1, >]') + if not valid: + return 1 + fixture.unlink() + + fixture = tree / "rebind_membership.eigs" + captures, prelude = driver_context() + # Exercise the SAME namespace-readback emitter in a scope where the + # fixture owns has_key/keys. Import isolation itself must not be what + # makes this control green: reverting the has_key capture must fail. + fixture.write_text('# road-bind: marker\n' + prelude + + 'subject is {"bound": 7}\n' + 'has_key is (args) => 0\nkeys is (args) => []\n' + 'for k in range of 1:\n marker is 7\n' + + snapshot(["bound"], captures, "subject")) + fixture.with_suffix('.out').write_text('["bound", 1, 7]\n["marker", 1, 7]\n') + if not require_green("rebinding has_key/keys cannot forge readback"): + return 1 + if bad_binary: + valid = require_red("has_key/keys rebinding: dropped import binding (known-bad binary)", + fixture.name, "marker", bad_binary, roads=("import",)) + else: + fixture.with_suffix('.out').write_text('["bound", 1, 7]\n["marker", 0]\n') + valid = require_red("has_key/keys rebinding: forged absence golden", fixture.name, + actual_line='["marker", 1, 7]') + if not valid: + return 1 + fixture.unlink() + + fixture = tree / "rebind_throw.eigs" + source = ('# road-bind: value\n# road-return: 8\nvalue is 7\n' + 'throw is (args) => 0\nreturn 8\n') + fixture.write_text(source) + fixture.with_suffix('.out').write_text('["value", 1, 7]\n') + if not require_green("rebinding throw preserves return validation"): + return 1 + fixture.write_text(source.replace('# road-return: 8', '# road-return: 9')) + if not require_red("throw rebinding cannot suppress a return mismatch", fixture.name, + roads=("load_file",), rc=1, divergence=True, + diagnostic="road return value mismatch"): + return 1 + fixture.unlink() + + fixture = tree / "missing.eigs" + fixture.write_text('# road-bind: absent\npresent is 1\n') + fixture.with_suffix('.out').write_text('["absent", 1, null]\n') + if not require_red("genuinely missing binding cannot impersonate present null", + fixture.name, "absent"): + return 1 + fixture.unlink() + + fixture = tree / "process_status.eigs" + fixture.write_text('# road-bind: value\nvalue is 7\n') + fixture.with_suffix('.out').write_text('["value", 1, 7]\n') + # Run a real successful child, then perturb ONLY its process envelope. + # All six rc-plant exits are identical, so cross-road comparison cannot + # conceal a deleted absolute rc check. No mock of run_gate is involved. + for symptom in ("returncode", "stderr"): + wrapper = tree / f"plant_{symptom}" + warning = "road selftest planted stderr warning" + wrapper.write_text(f'#!{sys.executable}\n' + 'import subprocess, sys\n' + f'r = subprocess.run([{str(binary)!r}, *sys.argv[1:]], capture_output=True, timeout=20)\n' + 'sys.stdout.buffer.write(r.stdout)\n' + 'sys.stderr.buffer.write(r.stderr)\n' + 'if r.returncode or r.stderr:\n' + ' raise SystemExit(99)\n' + + ('raise SystemExit(17)\n' if symptom == "returncode" else + f'sys.stderr.write({warning!r} + "\\n")\n')) + wrapper.chmod(0o755) + if not require_red("nonzero rc with matching stdout" if symptom == "returncode" else + "stderr only with matching stdout and rc=0", fixture.name, + runner=wrapper, rc=17 if symptom == "returncode" else 0, + stdout_matches=True, + diagnostic=warning if symptom == "stderr" else None): + return 1 + fixture.unlink() + + fixture = tree / "native_control.eigs" + fixture.write_text('# road-bind: count\n# road-native: required\n' + 'count is 0\nloop while count < 20000:\n count is count + 1\n') + fixture.with_suffix('.out').write_text('["count", 1, 20000]\n') + captured = io.StringIO() + with contextlib.redirect_stdout(captured): + status = run_gate(binary, tree) + output = captured.getvalue() + supported = platform.machine().lower() not in ("arm64", "aarch64") + tier_names = ('ref', 'jit', 'osr') if supported else ('ref',) + if (status or f'fixtures=1 runs={6 * len(tier_names)} failures=0 ' not in output or + any(output.count(f' tier={tier} ') != 6 for tier in tier_names)): + print('road_diff selftest: FAIL: reference and native configurations\n' + output) + return 1 + controls += 1 + print('road_diff selftest: GREEN: ' + ('reference and two native configurations (18 runs)' if supported + else 'ARM64 interpreter arm (6 runs; no native emitter)')) + captured = io.StringIO() + with contextlib.redirect_stdout(captured): + status = run_gate(binary, tree, architecture="arm64") + output = captured.getvalue() + if (status or 'fixtures=1 runs=6 failures=0 ' not in output or + 'native tiers unavailable on ARM64; interpreter only' not in output or + output.count(' tier=ref ') != 6): + print('road_diff selftest: FAIL: ARM64 policy\n' + output) + return 1 + controls += 1 + print('road_diff selftest: GREEN: ARM64 policy explicitly runs only the interpreter') + if supported: + # Measure the configuration, not an OSR-entry count. The default + # JIT can already enter the same chunk through OSR; stats do not + # distinguish that from the lowered-threshold configuration. + for drop in (False, True): + wrapper = tree / 'threshold_guard' + wrapper.write_text(f'#!{sys.executable}\n' + 'import os, pathlib, subprocess, sys\n' + 'env = os.environ.copy()\n' + + ('env.pop("EIGS_JIT_OSR_THRESHOLD", None)\n' if drop else '') + + f'r = subprocess.run([{str(binary)!r}, *sys.argv[1:]], env=env, capture_output=True, timeout=20)\n' + 'sys.stdout.buffer.write(r.stdout)\n' + 'sys.stderr.buffer.write(r.stderr)\n' + 'lowered = any(p.startswith("native_control-osr-") for p in pathlib.Path(sys.argv[1]).parts)\n' + 'if lowered and env.get("EIGS_JIT_OSR_THRESHOLD") != "1":\n' + ' sys.stderr.write("missing lowered OSR threshold\\n")\n' + ' raise SystemExit(19)\n' + 'raise SystemExit(r.returncode)\n') + wrapper.chmod(0o755) + captured = io.StringIO() + with contextlib.redirect_stdout(captured): + status = run_gate(wrapper, tree) + output = captured.getvalue() + if not drop: + valid = status == 0 and 'fixtures=1 runs=18 failures=0 ' in output + else: + valid = (status != 0 and 'fixtures=1 runs=18 failures=7 ' in output and + output.count('missing lowered OSR threshold') == 6 and + output.count('rc=19 tier=osr') == 6 and '--- expected' not in output and + 'missing/wrong native mechanism' not in output) + if not valid: + print('road_diff selftest: FAIL: lowered-threshold configuration\n' + output) + return 1 + if drop: + plants += 1 + print('road_diff selftest: RED: lowered OSR threshold removed (stdout still matches)') + else: + controls += 1 + print('road_diff selftest: GREEN: lowered OSR threshold reaches the child') + for symptom in (('force_off', 'drop_stats') if supported else ('drop_stats',)): + + wrapper = tree / symptom + wrapper.write_text(f'#!{sys.executable}\n' + 'import os, re, subprocess, sys\n' + 'env = os.environ.copy()\n' + + ('env["EIGS_JIT_OFF"] = "1"\n' if symptom == 'force_off' else '') + + f'r = subprocess.run([{str(binary)!r}, *sys.argv[1:]], env=env, capture_output=True, timeout=20)\n' + 'sys.stdout.buffer.write(r.stdout)\n' + + ('sys.stderr.buffer.write(re.sub(rb"(?m)^\\[jit\\] scanned=.*\\n", b"", r.stderr))\n' + if symptom == 'drop_stats' else 'sys.stderr.buffer.write(r.stderr)\n') + + 'raise SystemExit(r.returncode)\n') + wrapper.chmod(0o755) + captured = io.StringIO() + with contextlib.redirect_stdout(captured): + status = run_gate(wrapper, tree) + output = captured.getvalue() + expected_failures = 12 if symptom == 'force_off' else 6 * len(tier_names) + if (status == 0 or + f'fixtures=1 runs={6 * len(tier_names)} failures={expected_failures} ' not in output or + output.count('missing/wrong native mechanism') != expected_failures or + '--- expected' in output or 'roads/cwds diverge' in output): + print(f'road_diff selftest: FAIL: native {symptom} plant\n' + output) + return 1 + plants += 1 + print('road_diff selftest: RED: ' + ('native tiers compiled nothing' if symptom == 'force_off' + else 'native mechanism statistics missing')) + healthy_native = fixture.read_text() + for label, header in (('invalid value', 'maybe'), + ('duplicate header', 'required\n# road-native: required')): + fixture.write_text(healthy_native.replace('# road-native: required', '# road-native: ' + header)) + captured = io.StringIO() + with contextlib.redirect_stdout(captured): + status = run_gate(binary, tree) + output = captured.getvalue() + if (status == 0 or 'FAIL: native_control.eigs: invalid road-native metadata' not in output or + 'fixtures=1 runs=0 failures=1 ' not in output or 'Traceback' in output): + print(f'road_diff selftest: FAIL: native metadata {label}\n' + output) + return 1 + plants += 1 + print(f'road_diff selftest: RED: native metadata {label}') + fixture.unlink() + + fixture = tree / "exit_forge.eigs" + fixture.write_text('# road-bind: value\nprint of ["value", 1, 7]\nexit of 0\n') + fixture.with_suffix('.out').write_text('["value", 1, 7]\n') + if not require_red("exit before readback cannot forge completion", fixture.name, + stdout_matches=True, diagnostic="missing driver completion marker"): + return 1 + fixture.unlink() + + fixture = tree / "invalid_name.eigs" + fixture.write_text('# road-bind: x.y\nvalue is 7\n') + fixture.with_suffix('.out').write_text('["value", 1, 7]\n') + captured = io.StringIO() + with contextlib.redirect_stdout(captured): + status = run_gate(binary, tree) + output = captured.getvalue() + if (status == 0 or + "road_diff: FAIL: invalid_name.eigs: invalid snapshot name: x.y\n" not in output or + "fixtures=1 runs=0 failures=1 " not in output or "Traceback" in output): + print(f"road_diff selftest: FAIL: invalid binding name is a named failure\n{output}") + return 1 + print("road_diff selftest: RED: invalid binding name is a named failure") + plants += 1 + fixture.unlink() + + if run_gate(binary, tree) == 0: + print("road_diff selftest: FAIL: zero-fixture plant survived") + return 1 + print("road_diff selftest: RED: zero fixtures") + plants += 1 + print(f"road_diff selftest: controls={controls} plants={plants} failures=0") + return 0 + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--binary", type=Path, default=Path(os.environ.get("EIGENSCRIPT", ROOT / "src/eigenscript"))) + parser.add_argument("--fixtures", type=Path, default=ROOT / "tests/roads") + parser.add_argument("--fixture", help="run one named fixture (diagnostic only)") + parser.add_argument("--selftest", action="store_true") + parser.add_argument("--bad-binary", type=Path, + help="with --selftest: prove sentinel detection against an import-binding regression") + args = parser.parse_args() + if args.bad_binary and not args.selftest: + parser.error("--bad-binary requires --selftest") + binary = args.binary.absolute() + bad_binary = args.bad_binary.absolute() if args.bad_binary else None + raise SystemExit(selftest(binary, bad_binary) if args.selftest else run_gate(binary, args.fixtures.resolve(), args.fixture)) diff --git a/tools/road_diff.sh b/tools/road_diff.sh new file mode 100644 index 00000000..59f7d289 --- /dev/null +++ b/tools/road_diff.sh @@ -0,0 +1,5 @@ +#!/usr/bin/env bash +# The Python driver owns enumeration, bounded subprocesses, and fault plants. +set -euo pipefail +ROOT=$(cd "$(dirname "$0")/.." && pwd) +exec python3 "$ROOT/tools/road_diff.py" "$@" diff --git a/tools/werror_switch_check.sh b/tools/werror_switch_check.sh index 2eb48a2f..21bdefe2 100755 --- a/tools/werror_switch_check.sh +++ b/tools/werror_switch_check.sh @@ -104,7 +104,7 @@ MIN_LINES=100 # coverage-clean/fuzz-run (no compiles). TARGETS="build full http zlib net gfx asan asan-http asan-gfx tsan valgrind poison \ lsp dap jit-smoke lib embed-smoke embed-smoke-gfx embed-concurrent pgo coverage \ - fuzz fuzz-libfuzzer freestanding-libc-diff sandbox-intern-test errline-test nativefn-test" + fuzz fuzz-libfuzzer freestanding-libc-diff sandbox-intern-test errline-test nativefn-test embed-roads" # GNU make emits a shared prerequisite only once when several goals are in # one invocation. `embed-smoke-gfx` depends on `gfx`, so keeping that goal in @@ -118,6 +118,7 @@ TARGET_BATCHES=( "sandbox-intern-test" "errline-test" "nativefn-test" + "embed-roads" ) # TARGET_BATCHES must cover TARGETS exactly. Keep the hand-written batches @@ -276,6 +277,7 @@ dap 1 jit-smoke 1 lib 1 embed-smoke 1 +embed-roads 25 embed-concurrent 1 embed-smoke-gfx 27 pgo 2 @@ -1075,6 +1077,21 @@ if [ "${1:-}" = "--selftest" ]; then # which is what the failure actually reported (#1007, adding asan-gfx). # The check must be keyed to the tree under test, not to HEAD. cp Makefile "$root/Makefile" + # New auxiliary targets can also depend on not-yet-committed C inputs. + # Copying only their recipe/enrollment left `embed-roads` without its + # source and aborted the dry run before the planted header (#1056). + # Overlay tracked and nonignored C/header inputs, including deletions; + # never carry the working tree's binaries into the scratch tree. + local input + while IFS= read -r -d '' input; do + if [ -e "$input" ] || [ -L "$input" ]; then + mkdir -p "$root/$(dirname "$input")" + cp -P "$input" "$root/$input" + else + rm -f "$root/$input" + fi + done < <(git -c safe.directory="$PWD" ls-files -z --cached --others \ + --exclude-standard -- '*.c' '*.h') } # Generated-header family: prove that the planted generator source really