Skip to content

Whole-program search: parts re-search, re-search the 3 best (0.7.0) - #1

Merged
fMindstormer merged 11 commits into
mainfrom
whole-program
Nov 10, 2026
Merged

fMindstormer merged 11 commits into
mainfrom
whole-program

Conversation

@fMindstormer

@fMindstormer fMindstormer commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Whole-program search for 0.7.0: the re-search now compares finished programs instead of committing to one shape at a time.

What changes

  • Redraw a color whose region falls into parts (search.parts, search.add_parts, a new improve neighborhood). The greedy search takes one shape over a whole region (two circles stacked on an oval become one tall box plus patches) because that shape fixes the most pixels at its step. The new step splits the region where its distance to the edge comes apart, climbs one shape per part against that part alone, refills, and keeps the result only if the finished program scores better.
  • Re-search the 3 best results, not only the best (compiler.RESEARCH = 3, --research N). They run in parallel when there are several processes. --research 1 keeps the old behavior. The browser demo passes its checkbox as on/off, so it still re-searches one result.
  • Code generation: inset drops the lengths CSS repeats (inset:100px), without breaking a padding that elements share.
  • Benchmarks:
    • bench/run.py --research N and a matching input on the Benchmark workflow;
    • ablation rows no-parts and research-1;
    • evaluate.py methods v0.6 / v0.6-N, for the held-out baseline at equal compute;
    • a Held-out workflow.
  • Version 0.7.0: committed Linux results, ablation, heldout4 programs and known optima. The README, method notes, blog and results page are updated.

Results

The 66 development targets in Chromium (Benchmark workflow):

Total J Characters ≥ 99% match Shorter than human
0.6.0 52,153 23,329 66 / 66 56 / 66
0.7.0 47,263 (−9.4%) 21,175 66 / 66 60 / 66

The lowest match stays at 99.21%, and the mean goes from 99.82% to 99.84%.

Ablation (cssynth's own scores):

  • switching the whole re-search off now costs 14.4%;
  • the parts step alone, 6.1%;
  • re-searching only the best result, 2.4%.

heldout4, spent once on this version, with all methods on one runner:

Solved Characters, on the 43 all solve CPU
0.6.0's search 43 / 48 16,722 4,667 s
0.6.0's search, 48 starts (equal compute) 43 / 48 16,728 9,143 s
0.7.0 43 / 48 14,696 (−12.1%; 95% interval −22.7% to −1.4%) 8,141 s

Known optima: unchanged; optimal on 33 of the 35 reproduced exactly.

Cost: a compile takes 1.5 to 3 times the CPU of 0.6.0, depending on the picture. With several processes the wall time grows much less (the lotus: 16.9 s against 17.8 s on 8 cores).

After merging

The Pages workflow publishes the results page, blog and demo from main. Then tag v0.7.0 on main and create the GitHub release with the wheel, sdist, examples, benchmark zip and checksums. No PyPI.

🤖 Generated with Claude Code

fMindstormer and others added 11 commits November 9, 2026 21:08
The greedy search takes one shape over a whole region (two circles
stacked on an oval become one tall box plus patches), because that shape
fixes the most pixels at its step. A new re-search neighborhood splits a
color's region where its distance to the edge comes apart, climbs one
shape per part against that part alone, refills, and keeps the result
only if the finished program scores better.

On cssynth's own scores (no Chromium yet), total J drops 4.6% on the
development split (9 targets better, none worse) and 3.4% on training
(4 better, none worse), for 5-8% more compile CPU. 2024-02-27 goes from
695 characters to 200. bench/evaluate.py gains a no-parts method.

Also: inset drops the lengths CSS repeats (inset:100px).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A result that starts a little worse sometimes re-searches to a much
better program. compile_image(research=N) and --research N re-search the
N best distinct results, in parallel when there are several processes,
and keep the best; 1 stays the default.

On cssynth's own scores, the 3 best lower total J by 2.3% on the
development split and 2.5% on training, for 1.5-2x the compile CPU and
little more wall time on 8 cores. The lotus drops from 484 characters to
291 (human: 418). bench/evaluate.py gains a top3 method.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bench/run.py --research N, and the Benchmark workflow's research input
(default 1), so --research 3 can be measured in Chromium too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… numbers

Since inset drops repeated lengths, it can be the shortest encoding for
one element and break a padding the elements would share (ks-04-upsndowns
went from 207 to 214 characters). Code generation now also tries every
element's shortest encoding without inset: about 55 characters shorter per
benchmark run, never longer.

The Benchmark workflow on this branch, against 0.6.0: total J 52,153 ->
48,985 (-6.1%), 23,329 -> 21,817 characters, 59 targets shorter than the
human code instead of 56, all 66 still at >= 99% match; re-searching the 3
best results, 47,179 (-9.5%) and 21,283 characters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
compiler.RESEARCH = 3, used by compile_image, the CLI, bench/run.py, the
ablation and the Benchmark workflow's default input. Measured in Chromium
it lowered total J by 3.7% against re-searching only the best, for 1.5 to
2 times the CPU; with several processes the re-searches run in parallel.
--research 1 (and evaluate.py's top1) keeps the old behavior. The browser
demo passes its checkbox as on/off, so it still re-searches one result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- pyproject: 0.7.0.
- Ablation: no-parts and research-1 (re-search only the best) rows.
- evaluate.py: v0.6 and v0.6-N, the search of 0.6.0 with N starts per
  step, for the held-out comparison at equal compute.
- A Held-out workflow: generates one set, compiles it with each method
  one after another on one runner, renders in Chromium and prints the
  summary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On every third training target, 0.7.0 took 1,456 s of CPU against 503 s
for 0.6.0's search; each extra start per step costs about 33 s there, so
48 starts spend at least as much (20 and 24 starts: 823 s and 964 s).
More starts didn't lower J (22,640 at 10, 22,655 at 20, 22,673 at 24).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the 66 development targets, rendered in Chromium: 60 shorter than the
human code (was 56), 21,175 characters (was 23,329), all 66 at >= 99%
match (lowest 99.21%), mean match 99.84%. Ablation: the re-search is now
worth 14.4% of J, redrawing a color in parts 6.1%, re-searching the 3
best rather than the best 2.4%.

The blog's "where it still loses" example (two circles on an oval) is now
solved; it moves to a section on comparing whole programs, and the pin
(2024-03-26, no gradients on shapes) takes its place. Captions follow the
new programs for the sunburst and the rings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generated and compiled once in the Held-out workflow, 0.6.0's search, the
same with 48 starts per step (at least 0.7.0's CPU) and 0.7.0 one after
another on one runner. All three solve the same 43 of 48. On those, 0.7.0
writes 14,696 characters against 16,722 and 16,728 (-12.1%; 95% bootstrap
interval -22.7% to -1.4%), mostly on rings and petals, for 1.74 times the
CPU of 0.6.0's search (8,141 s against 4,667 s and 9,143 s). The
programs are committed as for the earlier sets; the pictures weren't
looked at.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shorter inset lowered five of the optima; cssynth still matches them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fMindstormer
fMindstormer merged commit a7500c1 into main Nov 10, 2026
4 checks passed
@fMindstormer
fMindstormer deleted the whole-program branch November 10, 2026 22:07

This branch was successfully deployed

1 active deployment
github-pages — a7500c1a Deployed Nov 10, 2026 by fMindstormer via deploy #13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant