Repository navigation
Conversation
Two changes so the runtime can be linked into the compiler itself, which the JIT (xbasic64 run) needs; compiled programs behave as before. - The runtime named three symbols the program defined: _data_table, _data_count and _data_ptr. Now a program with DATA hands its table and item count to the new _rt_data_init as main starts, and the runtime keeps the READ position itself. The table is a local, read-only object, and a program without DATA has none. rt.o now needs nothing of a program -- only the C library, which its nm test asserts. - A host feature drops the runtime's #[panic_handler], which would clash with std's in the compiler. build.rs's staticlib builds without it. - A debug compiler builds rt optimized, without overflow checks or debug assertions, as build.rs builds the copy programs link. Differential run against 705d7e0: identical output on 648 of 651 programs, the rest RND/TIMER. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pure refactor, toward the JIT. CodeGen<M: ?Sized = dyn Module> keeps the module as its last field, so CodeGen<ObjectModule> -- or a JITModule's -- is used as CodeGen<dyn Module>, and the rest of codegen is unchanged and written once. lower() builds one around any module and lowers the program into it; generate() is the executable's: an ObjectModule, then its bytes. A colocate flag keeps the two overrides that make every reference PC-relative. An executable sets it; the JIT will not, since the runtime there is the compiler's own, possibly beyond reach of a 32-bit offset. The -S CLIF of all 651 examples and corpus programs is byte-identical to d0e77d4's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The half step to a JIT, on the shared codegen: the same lowering as an executable's, into a cranelift-jit JITModule, and the same runtime -- rt, now also linked into the compiler. The whole program is compiled before it starts; nothing is interpreted. src/jit.rs. - xbasic64 run prog.bas [--unsafe] is a subcommand; plain xbasic64 prog.bas still compiles an executable. Compile errors are reported as before; otherwise the program's exit is the process's, and the compiler prints nothing of its own. - The runtime and the C library are the compiler's, anywhere in its address space, so generated code reaches them by absolute address (colocate off) and the JIT is given every one's address: each rtsig helper, the trapping words and messages, and the C functions codegen calls (codegen::LIBC_FUNCTIONS). A unit test holds the table to rtsig. - The program's own code and data stay PC-relative, as in an executable: they come from one arena (ArenaMemoryProvider), 64 MB to start, doubled and the program lowered again if it runs out, up to 1 GB -- within the 2 GB a 32-bit offset reaches. (A fixed large arena would commit that much on Windows, and fail under a ulimit.) - When main returns, the C library's streams are flushed as an executable's C runtime would, and the process exits with main's status. - Symbols is Clone, for the arena's retry. tests/jit: output and nothing else, END in a SUB, STOP, a runtime error's report and exit 1, a compile error, INPUT from stdin, DATA/READ/RESTORE, ON ERROR/RESUME NEXT, GOSUB, --unsafe, files in the working directory, a huge function (define_function_bytes), and the megatest byte-identical to the executable's. Executables are unchanged: the -S CLIF of all 651 examples and corpus programs is byte-identical to before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With XB_JIT=1 the harness runs every program by xbasic64 run instead of as an executable: compile_and_run_env, compile_and_run_with_files and the new run_with_address_space_limit check that the program compiles (with -S, so a compile error is still an Err apart from the run) and then run xbasic64 run on it, with the same directory, stdin and environment. The two ulimit -v memory tests share that helper now. CI runs the integration suite a second time that way on all four jobs, and runs each example through xbasic64 run as well as compiled. Checked here: - x86-64 Linux: 599 of 599 tests pass both ways; the two memory tests fit the compiler, the JIT's 64 MB arena and the program in 256 MB. - The 651 examples and corpus programs give the same stdout, stderr and exit code by xbasic64 run as compiled (3 differ: RND seeded by TIMER). - Valgrind on 95 of them run through the JIT: no errors. - aarch64 under qemu-user: 599 of 599 compiled, 597 of 599 by run. The two memory tests cannot reserve the arena there because qemu's own guest reservation counts against the ulimit; CI's native ARM job is the check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README: usage and architecture mention xbasic64 run. CLAUDE.md: jit.rs,
the run subcommand, the code model (absolute references to the runtime,
the program's own code and data PC-relative within one arena), the rule
that a new helper needs a JIT symbol, and the XB_JIT suite mode.
Measured, x86-64 Linux (best of 3):
source to finished run, compile+link+exec vs run
mega.bas 0.05s vs 0.04s, a 2000-line program 0.15s vs 0.15s,
a 10000-line program 0.64s vs 0.62s -- compiling dominates; the link
is about 10 ms.
compiled code's run time, executable vs run (which includes compiling)
array loops 0.12s vs 0.13s, fib(32) 0.01s vs 0.01s, strings
0.07s vs 0.07s, PRINT 2M numbers 0.23s vs 0.24s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The opening said only "a native code compiler" and promised executables, with no word of xbasic64 run. It now says what the compiler does both ways -- an executable, or compiled into memory and run at once -- and on which platforms; usage leads with run; the pipeline shows both ends; requirements note that run needs no linker; the platforms are a table of OS, architecture and the two modes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
It introduces a substantial new JIT execution path with unsafe FFI, in-process runtime state, and an arena-growth heuristic relying on an untested error-string match, which warrants human verification.
1 open finding
What changed in this PR
This PR adds a run subcommand (xbasic64 run program.bas) that compiles a BASIC program into memory and executes it in the compiler's own process via a Cranelift JIT, rather than emitting and linking an executable. The same lowering (codegen::lower) feeds both paths; the JIT differs only in that the runtime/libc are reached by absolute address (colocation off) while the program's own code/data stay PC-relative in a growable arena. To make the runtime work in-process, DATA is no longer referenced by exported program symbols — the program now hands its DATA table to the runtime via a new _rt_data_init helper at main startup.
Changes:
- New
src/jit.rsJIT driver (arena sizing/growth, in-processsymbols()table, flush/exit semantics) wired through a new clapRunsubcommand inmain.rs. - Refactored
codegensoCodeGenis generic over the CraneliftModuleand lowering is shared viacodegen::lower; added acolocateflag; reworked DATA to use runtime_rt_data_initinstead of_data_table/_data_count/_data_ptrexports. - Runtime
hostfeature (std panic handler), newtests/jit/, a JIT CI pass (XB_JIT=1), and test-harness helpers (run_jit,run_with_address_space_limit).
| File | Description |
|---|---|
| src/jit.rs | New JIT runner: arena growth, in-process symbol table, flush and process-exit handling |
| src/main.rs | Adds run subcommand and dispatch to jit::run |
| src/codegen/mod.rs | Generic CodeGen<M>, shared lower/generate, colocate flag, DATA table reworked (minor doc nit) |
| src/codegen/stmt.rs | Emits _rt_data_init at main start when DATA exists |
| src/rtsig.rs | Declares _rt_data_init signature |
| rt/src/data.rs | DATA now initialized via _rt_data_init; tests updated |
| rt/src/lib.rs, rt/Cargo.toml | host feature so std supplies the panic handler in-process |
| src/runtime.rs | Test asserts rt.o references nothing of a program |
| src/sema.rs | Symbols derives Clone (needed for JIT re-lowering) |
| tests/common/mod.rs, tests/jit/mod.rs, tests/integration.rs, tests/strings/mod.rs | JIT test harness and coverage |
| .github/workflows/TestingCI.yml, README.md, CLAUDE.md, Cargo.toml, Cargo.lock | JIT CI pass, docs, and dependency additions |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The intermittent Windows CI failure -- "Permission denied" from inside the runtime's unit tests, first seen at 4ad0166 -- is now named, by the panic its fatal error makes in a test: a_field_window_outlives_its_file closing a file and opening it again. The C library's handles are inheritable by default on Windows, and the unit tests that expect a fatal error run in child processes, beside the file tests. A child started while a file was open for writing kept a write handle of its own to it, so reopening the file -- which refuses other writers (_SH_DENYWR, as the assembly runtime's CreateFileA did) -- failed for as long as the child lived. sys::open now adds MSVC's N mode flag, so the handle stays in the process that opened it. The same would have bitten a program the compiler runs, had the compiler started a child. an_open_file_is_not_inherited_by_a_child reproduces it: under Wine it failed with Permission denied before, and passes now. Also: LIBC_FUNCTIONS had taken libc_sig's doc comment above its own; each has its own again (review feedback). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.