Repository navigation
Whole-program search: parts re-search, re-search the 3 best (0.7.0) - #1
Merged
Merged
Conversation
The greedy search takes one shape over a whole region (two circles stacked on an oval become one tall box plus patches), because that shape fixes the most pixels at its step. A new re-search neighborhood splits a color's region where its distance to the edge comes apart, climbs one shape per part against that part alone, refills, and keeps the result only if the finished program scores better. On cssynth's own scores (no Chromium yet), total J drops 4.6% on the development split (9 targets better, none worse) and 3.4% on training (4 better, none worse), for 5-8% more compile CPU. 2024-02-27 goes from 695 characters to 200. bench/evaluate.py gains a no-parts method. Also: inset drops the lengths CSS repeats (inset:100px). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A result that starts a little worse sometimes re-searches to a much better program. compile_image(research=N) and --research N re-search the N best distinct results, in parallel when there are several processes, and keep the best; 1 stays the default. On cssynth's own scores, the 3 best lower total J by 2.3% on the development split and 2.5% on training, for 1.5-2x the compile CPU and little more wall time on 8 cores. The lotus drops from 484 characters to 291 (human: 418). bench/evaluate.py gains a top3 method. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bench/run.py --research N, and the Benchmark workflow's research input (default 1), so --research 3 can be measured in Chromium too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… numbers Since inset drops repeated lengths, it can be the shortest encoding for one element and break a padding the elements would share (ks-04-upsndowns went from 207 to 214 characters). Code generation now also tries every element's shortest encoding without inset: about 55 characters shorter per benchmark run, never longer. The Benchmark workflow on this branch, against 0.6.0: total J 52,153 -> 48,985 (-6.1%), 23,329 -> 21,817 characters, 59 targets shorter than the human code instead of 56, all 66 still at >= 99% match; re-searching the 3 best results, 47,179 (-9.5%) and 21,283 characters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
compiler.RESEARCH = 3, used by compile_image, the CLI, bench/run.py, the ablation and the Benchmark workflow's default input. Measured in Chromium it lowered total J by 3.7% against re-searching only the best, for 1.5 to 2 times the CPU; with several processes the re-searches run in parallel. --research 1 (and evaluate.py's top1) keeps the old behavior. The browser demo passes its checkbox as on/off, so it still re-searches one result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- pyproject: 0.7.0. - Ablation: no-parts and research-1 (re-search only the best) rows. - evaluate.py: v0.6 and v0.6-N, the search of 0.6.0 with N starts per step, for the held-out comparison at equal compute. - A Held-out workflow: generates one set, compiles it with each method one after another on one runner, renders in Chromium and prints the summary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On every third training target, 0.7.0 took 1,456 s of CPU against 503 s for 0.6.0's search; each extra start per step costs about 33 s there, so 48 starts spend at least as much (20 and 24 starts: 823 s and 964 s). More starts didn't lower J (22,640 at 10, 22,655 at 20, 22,673 at 24). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the 66 development targets, rendered in Chromium: 60 shorter than the human code (was 56), 21,175 characters (was 23,329), all 66 at >= 99% match (lowest 99.21%), mean match 99.84%. Ablation: the re-search is now worth 14.4% of J, redrawing a color in parts 6.1%, re-searching the 3 best rather than the best 2.4%. The blog's "where it still loses" example (two circles on an oval) is now solved; it moves to a section on comparing whole programs, and the pin (2024-03-26, no gradients on shapes) takes its place. Captions follow the new programs for the sunburst and the rings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generated and compiled once in the Held-out workflow, 0.6.0's search, the same with 48 starts per step (at least 0.7.0's CPU) and 0.7.0 one after another on one runner. All three solve the same 43 of 48. On those, 0.7.0 writes 14,696 characters against 16,722 and 16,728 (-12.1%; 95% bootstrap interval -22.7% to -1.4%), mostly on rings and petals, for 1.74 times the CPU of 0.6.0's search (8,141 s against 4,667 s and 9,143 s). The programs are committed as for the earlier sets; the pictures weren't looked at. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shorter inset lowered five of the optima; cssynth still matches them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Whole-program search for 0.7.0: the re-search now compares finished programs instead of committing to one shape at a time.
What changes
search.parts,search.add_parts, a newimproveneighborhood). The greedy search takes one shape over a whole region (two circles stacked on an oval become one tall box plus patches) because that shape fixes the most pixels at its step. The new step splits the region where its distance to the edge comes apart, climbs one shape per part against that part alone, refills, and keeps the result only if the finished program scores better.compiler.RESEARCH = 3,--research N). They run in parallel when there are several processes.--research 1keeps the old behavior. The browser demo passes its checkbox as on/off, so it still re-searches one result.insetdrops the lengths CSS repeats (inset:100px), without breaking apaddingthat elements share.bench/run.py --research Nand a matching input on the Benchmark workflow;no-partsandresearch-1;evaluate.pymethodsv0.6/v0.6-N, for the held-out baseline at equal compute;Results
The 66 development targets in Chromium (Benchmark workflow):
The lowest match stays at 99.21%, and the mean goes from 99.82% to 99.84%.
Ablation (cssynth's own scores):
heldout4, spent once on this version, with all methods on one runner:
Known optima: unchanged; optimal on 33 of the 35 reproduced exactly.
Cost: a compile takes 1.5 to 3 times the CPU of 0.6.0, depending on the picture. With several processes the wall time grows much less (the lotus: 16.9 s against 17.8 s on 8 cores).
After merging
The Pages workflow publishes the results page, blog and demo from main. Then tag
v0.7.0on main and create the GitHub release with the wheel, sdist, examples, benchmark zip and checksums. No PyPI.🤖 Generated with Claude Code