The primary development track is neural program synthesis: a learner that constructs executable programs and improves from independently checked training experience. The programmer roadmap defines the milestones. The edit-value experiment passes 1/6 fresh array functions, tied with untrained, classical and incumbent routes, and 0/3 on the original array tasks. Accepted programs qualified on 20 actual cases per language (89 executions per backend). No learning benefit or broader-family gain is established. The first runner campaign ran both development recipes at 0/3 and stopped NO_MEASURED_GAIN without opening its audit; its adapter supports only bounded integer-list programs. Three bounded list-output trials construct 0/3 programs each, matching untrained controls. Four supplied controls pass 20 Python and CURLEE cases each, including 512 outputs from 256 inputs. See the frozen-transfer result: 11/18 unseen functions pass in both languages (20 held cases each), matching untrained 11/18 against classical 14/18. Full measured controller CPU is 71.614 s trained, 26.600 s untrained and 21.127 s classical, excluding historical donor-library creation. No training or cost benefit is established. The branch-composition result accepts all 11 development tasks in Python and CURLEE, with 40 held cases each per language. It adds largest magnitude, capped-value sum and capped count by composing verified branches and choosing shared expressions. The host supplies the composition grammar; the learner's weights stay fixed during search. A stronger classical control also solves all eleven. General programming and a matched-total-cost advantage remain unproven. The preceding query-repair result accepts 8/11 development tasks in Python and CURLEE: six earlier utilities, maximum and minimum, each with 40 held cases per language. Three more complex tasks remain unsolved. The learner resynthesizes conditions from visible counterexamples in verified contexts; weights stay fixed during repair. This remains bounded component transfer, not general programming. The preceding context-growth trial constructed maximum from a qualified minimum donor, passing 40 Python and 40 CURLEE cases. Its retained-model route keeps the six earlier utilities but fails to retain minimum, for 7/11 accepted tasks. The minimum remains separately qualified in the component-transfer trial. Training contains these algorithmic components inside other functions. The preceding stateful follow-up solved 0/5 new tasks, including maximum and capped-count programs. A broader curriculum did not improve that result and retained only 5/6 earlier tasks. Supplied control programs passed all five tasks in Python and CURLEE; the learner's generalization is the current limitation. The preceding verified-fragment trial retains four statement-generated utilities and repairs two more. All six pass 40 held cases each in Python and CURLEE. The learner constructs new conditions within contexts mined from previously checked programs; the emitter supplies syntax lowering and file handling. General programming and a neural cost advantage remain unproven. The preceding statement-construction trial retains its failures and development results. The earlier list-composition result is retained separately.
The work is separated into three research tracks:
| Track | Current focus | Checks |
|---|---|---|
| Programmer | Primary; generated Python and CURLEE code that executes correctly | just test-programmer |
| Sensing | Preserved timed prediction and adaptation experiments | just test-sensing |
| Sound Companion | Preserved native supervised sound-learning app | just test-companion |
just tracks lists the declared scopes. Track separation preserves existing
runtime import paths, fixtures, evidence and aggregate test commands.
Earlier experiments study a spiking neural network (SNN) as a ranking policy for arithmetic program synthesis. Those policies rank candidate operations; classical search assembles selected steps into typed program graphs and computes their outputs on task rows. A separate interpreter checks accepted programs against the frozen training and held-out expectations.
The repository includes a trace-trained SearchPolicy, author and beam
search, a modular policy, a bit-slice lattice, task-family generators,
and promotion/evaluation loops. The implementation and its limits are
described in architecture.md. Historical performance
figures in earlier reports used older feature encodings and work
counters; they should not be read as measurements of the current code.
The audit follow-up records subsequent fixes.
The architecture follow-ups track the
pre-evaluation search experiment and its evaluation gates.
The code-v45 amendments and validation retain
that release's exact test results and three-seed measurement.
The historical adversarial
review includes reproductions for issues
#51–#71 and the baseline measurement at 5c6701b.
rsi_programmer/— policies, search, typed-DAG validation, training, task fixtures, and evaluation utilities.fixtures/— frozen task families used for training and evaluation.scripts/train.py— canonical ranker training entry point.tests/— unit and integration checks for search, training, and independent acceptance.
just setup # create .venv and install project dependencies
just test # run the full test suite
just train # seed 0, 50 epochs; writes weights/spikeforge is a source dependency (Capsize-Games/spikeforge); just setup uses a local checkout. For another location, pass its absolute
checkout root as a positional argument: just setup /path/to/spikeforge.
Torch CPU wheels come from the PyTorch index (see requirements.txt).
For lazy evaluation, import pre_evaluation_search from
rsi_programmer.pre_eval. It accepts the same training-row specification
and ranking interface as guided search, but budget limits candidate
signature evaluations. Pass policy=None for streaming generation order.
Policies rank candidate descriptors using cached operand results before
the candidates are executed. Descriptor scoring still has a cost, which
the result reports separately. A returned training match requires an
independent held-out check before it counts as an accepted solution.
scripts/train.py writes a weight file and a JSON manifest to the
selected output directory (the default is weights/).
For independent first-solver-position measurements, run:
.venv/bin/python scripts/measure_ranker.py --out /tmp/ranker-resultsThis fits shadow final-layer trace groups with three seeds and reports untrained, 5-, 40-, and 50-epoch positions on shadow, novel, and plain OOD fixtures. It uses the exact expected random position and writes per-task data, source/fixture hashes, and signature instrumentation. Signature counters measure root vector evaluations; runtime and neural work are outside that metric. Its early-stop streaming comparator differs from the probe enumerator, which finishes each layer before checking it.
The frozen pre-evaluation benchmark compares
streaming, heuristic, random and neural controls on a separately generated
corpus. Its protocol and corpus were published in code-v61 before any
solver measurements. Run the complete fixed matrix into a new directory:
flock /tmp/rsi-programmer-builder.lock .venv/bin/python \
scripts/benchmark_pre_eval.py --output /tmp/pre-eval-resultsThe runner preserves all task/seed/budget attempts, independently checks returned programs, and rejects changed frozen inputs. Candidate work, policy overhead, validation and training costs are reported separately.
BSD-3-Clause, copyright Capsize LLC; see LICENSE.