Skip to content

Generated datasets on several images, and exercism-polyglot@1.0 - #9

Merged
rzonyx merged 7 commits into
mainfrom
generated-exercism-polyglot
Oct 22, 2026
Merged

rzonyx merged 7 commits into
mainfrom
generated-exercism-polyglot

Conversation

@rzonyx

@rzonyx rzonyx commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor
  • Schema: a built version may name environment.task_images, a glob over task names for each built image, instead of one image. Each task names its image in docker_image (by tag until the lock pins it), and the adapter writes environment/, which hbench copies into the image. Locks fill one image at a time; publishing waits for all. The single-image form is unchanged.
  • exercism-polyglot@1.0: 210 tasks, 35 each from Exercism's C++, Go, Java, JavaScript, Python and Rust tracks (MIT), generated from pinned archives. One image per language, exercism-polyglot-<lang>-1.0, amd64.
  • Left out: food-chain (lyrics copyright), and ledger (Go, JS), markdown (Go), tree-building (Java), refactoring exercises whose stubs already pass.
  • Java runs javac and the JUnit console launcher, not Gradle: 1.3 s against 5.2 s per exercise in CI.
  • CI: the new Exercism polyglot workflow builds each language's image and runs every task's oracle (must pass) and untouched stub (must fail) with hbench.

After merging, the six images need the Build images workflow before the version can be published.

A built version names one image today. environment.task_images is the
additive alternative: a glob over task names for each image, every task
matching exactly one. Each task names its image in docker_image, by tag
until images.lock.json pins it, and its adapter writes environment/, the
files hbench copies into the image when the task starts; a per-task
build context would make hbench build instead of using the image.

The Build images workflow locks one image per run, so lock-image now pins
the tasks on its tag and keeps the other images' entries. A lock may pin
some images before the others, always all of an image's tasks; such a
version validates with a warning, and publishing waits until every image
is pinned. The single-image form is unchanged.
210 tasks, 35 practice exercises from each of Exercism's C++, Go, Java,
JavaScript, Python and Rust tracks, written by adapters/exercism_polyglot
from the problem specifications and the six tracks at pinned commits, each
archive pinned by sha256. All seven are MIT; NOTICE carries both of
Exercism's notices and Catch2's BSL-1.0 licence.

Each task starts with the exercise in /app as the track ships it, tests
included, since most C++ stubs are empty. The verifier restores the tests
and every other non-solution file from tests/, enables the skipped tests
and runs them offline; the oracle copies the track's example solution.
Java compiles with javac and runs on the JUnit console launcher instead of
Gradle, whose wrapper is left out, and its image carries the public-domain
org.json 20260814 rather than the JSON-licensed 20190722. food-chain is
left out for its lyrics' copyright.

One image per language, built from official Docker Hub bases copied to
GHCR. The Exercism polyglot workflow builds each language's image on a
GitHub-hosted runner and requires, with hbench, that every oracle passes
and every untouched stub fails.
The first CI run found three problems. Exercism's C++ CMakeLists.txt names
the build after its directory, so a C++ exercise now sits in /app/<slug>.
The canary comment went above a #! line in JavaScript grep's example; it
now goes below it. And four refactoring exercises ship stubs that already
pass every test, ledger in Go and JavaScript, markdown in Go and
tree-building in Java, so they are left out and the ranking fills their
places: affine-cipher in JavaScript, go-counting and rectangles in Go,
state-of-tic-tac-toe in Java.

The Java job also times Gradle against javac on five exercises, once.
#![feature(test)] starts parallel-letter-frequency's benchmark, and the
canary comment went below it as if it were a #! line. A shebang is now #!
not followed by [, and the adapter's own check finds the canary comment on
the first or second line.
On the GitHub-hosted runner, in the Java image, an offline Gradle run with
warm caches and no daemon took 5.2 to 5.3 s for bowling, forth and react,
and javac with the JUnit console launcher 1.3 to 1.4 s. Gradle's first run,
which downloads it, took 11.2 s. The verifier keeps javac, which also
spares the image Gradle and its caches.
…lyglot

# Conflicts:
#	README.md
#	docs/licensing.md
#	images/bases.yaml
#	images/images.yaml
rzonyx added a commit that referenced this pull request Oct 22, 2026
Built-locally versions and versions on several images both go through
GenerationContext: base_images for the first, task_docker_images for
the second. versions_built_on skips built-locally versions, which have
no built image, and --list skips versions on several images and takes
--policy.
@rzonyx
rzonyx merged commit 6a351b4 into main Oct 22, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant