Repository navigation
Add toolchain like features - #7
Draft
UnsignedArduino wants to merge 49 commits into
Draft
UnsignedArduino wants to merge 49 commits into
UnsignedArduino wants to merge 49 commits into
Conversation
UnsignedArduino
marked this pull request as draft
August 1, 2026 00:15
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZcpdvCZoLyyheJd2FF
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZcpdvCZoLyyheJd2FF
- Add the required `target:` to the e2e fixture config. - Install the toolchain and target in a session fixture before building. - Remove `cache=False` so builds are incremental. - Check the second build by parsing redun's job status table (every job CACHED, none DONE) instead of wall-clock time. - Update the project context doc, including the newly exposed stale-cache issue for inputs redun can't see. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the main-only base filter on pull_request so stacked PRs (dag, toolchain) get CI. Pushes still only trigger on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- MKCD2APP_HOME moves data/state dirs to $MKCD2APP_HOME/{data,state}/<version>.
The e2e tests install into .pytest_cache/d/mkcd2app-home with it, so they
don't touch the real user dirs and reruns hit the install cache.
- Rename `build --clear-cache` to `--full-rebuild` and explain in its help
text that it's how to pick up changes to source files, icons, or GitHub
branches, which aren't tracked automatically.
- Update the project context doc (CI green on all OSes, roadmap, decision
to rely on --full-rebuild for out-of-band inputs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- download_sim decodes downloads as UTF-8 instead of res.text (which falls back to ISO-8859-1 without a charset), and all text file reads/writes pass encoding="utf-8" so Windows doesn't use cp1252. - get_resource_template_path(name) now actually joins name. - load_config_from_yaml raises NotImplementedError for a missing/unknown version instead of AttributeError. - Fix uninstall_toolchain and fill_website_template docstrings, drop a stray comment in "Racers offline.yaml", and update the project context doc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…file install_deps_and_build_website* hashed the whole filled site, so any game or asset change re-ran npm ci. install_website_deps now runs npm ci once from only the template's package.json and package-lock.json, and the renamed build_website* tasks copy that node_modules into their own site copy and run only npm run build*. static and static-singlefile share the one install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cloning and checking out a `github` code source now uses dulwich (pure Python) in `utils/git.py`, so a globally installed `git` is no longer required. Branches and tags get a shallow `depth=1` + `branch=` clone. Anything else is treated as a (possibly short) commit SHA and gets a full clone followed by a checkout. That clone checks out the default branch first, because dulwich's checkout only writes files that differ from the current HEAD tree and leaves the worktree empty when the target's tree matches HEAD. Clone progress is only shown with --debug, since dulwich's \r-separated progress floods non-TTY logs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZcxTDte5nuYr92ZaBg
Add a `git` code source taking any URL dulwich can clone (https, http, git, file, or ssh with an `ssh` binary) and an optional `ref` (branch, tag, or commit SHA; omitted means the remote's default branch), so games can come from GitLab, Codeberg, or self-hosted servers. The `github` source stays as an alias with its existing `value`/`checkout` fields, and `fetch_code` handles both in one case. Add the first unit tests: config parsing for both source types, and `clone_and_checkout` against a local repo built with dulwich so they run offline, including the same-tree-as-HEAD empty worktree regression. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZcxTDte5nuYr92ZaBg
Decode the MakeCode project embedded in a shareable PNG or UF2 (format detected by magic bytes) into the usual -source folder, warning when the saved Arcade version differs from the configured target. Adds the repo's first unit tests, covering synthetic PNG (incl. overflow rows) and UF2 round trips, error cases, and path traversal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZdVK71NaCgpyqUP1Wf
Falling duck PNG and UF2 (4.1.27) for the unit tests, and the empty game exported as a PNG from 4.0.14 for a new e2e test that builds it with a file code source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZdVK71NaCgpyqUP1Wf
Compare the extracted main.ts against the PNG's embedded bytes, and against the checked-in fixture as text, since git's autocrlf gives the fixture CRLF on Windows while extraction keeps the embedded LF. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZdVK71NaCgpyqUP1Wf
A ZIP (e.g. GitHub's Download ZIP) is detected by magic bytes like PNG and UF2. The project root is wherever pxt.json is, at the top level or inside one wrapping folder, which is stripped. Directories and symlinks are skipped, entry count and uncompressed size are capped, and names go through the existing traversal check. Adds the Racers Download ZIP as a fixture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZdVK71NaCgpyqUP1Wf
The test zips tests/fixtures/empty_game under an empty_game-master/ wrapper at run time and builds it as a file source, checking the extracted files byte for byte against the checkout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZdVK71NaCgpyqUP1Wf
- is_strict_semver uses fullmatch so "4.0.14\n" is rejected; BuildConfig.target
is validated as strict semver when the config loads.
- project.title is rendered with a single-pass substitution of {NAME},
{VERSION} and {AUTHOR} instead of str.format, so other braces no longer
crash the build and substituted values aren't expanded again.
- The title is HTML-escaped before it goes into index.html.
- The index.html title and favicon markers must each appear exactly once;
template drift now raises instead of silently doing nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mbb9ZbdDZ5goMxMi7N2f4j
pytest-cov runs on every pytest invocation (--no-cov to skip), and tests/conftest.py marks each test unit or e2e by its directory, so `uv run pytest -m unit` runs only the fast offline tests. CI is unchanged: `uv run pytest` still runs everything. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZdHu1tFP2ZhM62pbgq
- test_code.py: fetch_code for every source type, the target mismatch warning, build_binary_js, and copy_support_files with path/URL icons - test_website.py: copy/fill the real bundled template (title, favicon, package.json, copy-on-write), install_website_deps, and both website builds including node_modules symlinks - test_redun_compat.py: the get_proto patch and content-based dir hashing - extract_meta_comment, utils/paths, run_cmd, rmtree_robust, resources Tasks run through .func with run_cmd and network calls monkeypatched, so everything stays offline. Unit coverage goes from 42% to 66%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZdHu1tFP2ZhM62pbgq
unit.yaml runs `pytest -m unit` with coverage and no Node.js setup, so unit results come back in seconds instead of after the toolchain install. e2e.yaml now runs only `pytest -m e2e --no-cov`, since coverage can't see the mkcd2app subprocess the e2e tests drive. Both run on all three OSes and neither waits for the other. Because CI now selects by marker, tests/conftest.py fails collection for a test outside tests/unit and tests/e2e, which would otherwise run in neither workflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZdHu1tFP2ZhM62pbgq
Cover target and toolchain install, cli.py, _clear_build_dir, config loading and resolve_config_paths, build_project's early errors, and the logger, plus the uncovered branches in embedded_source and utils/git. Unit coverage goes from 70% to 94%; what's left mostly needs the scheduler. Also treat warnings as errors in pytest, move FakeResponse to the shared helpers, and fix the warm_npm_cache_for_templates docstring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/integration runs the real main() and redun scheduler in-process with run_cmd faked, so it needs no Node.js. It covers the build_project wiring, exactly which tasks and commands each kind of edit reruns, deleted outputs, the copy-on-write rule, failure and retry, and toolchain and target installs. It runs in its own integration.yaml workflow, and helpers.py moves to tests/ so both tiers can use it. Bugs it found: - Deleting a ContentFile output crashed the next run from redun's cache check instead of rerunning its task, so target uninstall then target install failed. A deleted file now counts as changed. - Uninstalling a toolchain or target that isn't installed crashed. It now warns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a git source build from a local repo, an opt-in share_link build (--run-network), title and icon checks, checks that both outputs contain binary.js and the simulator and load nothing from the internet, and one real rebuild after a title edit. block_network.cjs makes Node fail any network connection, as Node ignores HTTP_PROXY by default, which shows a build and a title edit work offline. mkc's once-a-day update check is the one attempt allowed: when it fails, mkc logs that and builds from its cache. Builds now run in temp folders instead of examples/testing, and run_mkcd2app shows the command's output when it fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a PR, [skip-<tier>-tests], [no-<tier>-tests], [skip-<tier>] or [no-<tier>] in the last commit's message skips that tier, and [skip-tests] or [no-tests] skips all three, so a small change doesn't have to wait for expensive CI. Pushes to main ignore these, so what's merged still gets tested. The check is a reusable workflow, as a pull_request event doesn't include the commit message, so it has to be looked up. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
index.html used absolute /assets/ paths, so the site only worked from a domain root and broke in a subfolder like GitHub Pages' /<repo>/. Set a relative base in vite.config.ts, and check the paths in the existing shared e2e build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 4, 2026
Each push to a PR started a full set of runs (13 jobs) even while the previous push's runs were still going, so two quick pushes went over the 20-job / 5-macOS concurrency limit and queued. A concurrency group per workflow and PR now cancels the older runs. Pushes to main aren't cancelled, so every merged commit still gets a full run. check-skip.yaml is left alone, since a called workflow would share its caller's group. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbb9ZbezDZgN12u51PZVFz
run_cmd only stored a command's output in BuildError.output, and nothing printed it, so compile errors and npm failures couldn't be diagnosed, even with --debug. It now logs the full output at error level on failure and at debug level on success. The full output rather than the tail, since the first compiler error is usually the root cause. Output is decoded as UTF-8, replacing invalid bytes, instead of with the locale's encoding. main() sets stdout to errors="backslashreplace", as Python already does for stderr: on Windows, a redirected stdout is cp1252, so a debug record with Vite's checkmark in it was lost to a UnicodeEncodeError. Fixes #15 and #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move MakeCode projects and exports to tests/fixtures/mkcd_projects and the LZMA-JS payloads to tests/fixtures/lzma-js, and rename the two build configs to say which code source they use. Update the tests, configs and project context doc to match. Name the empty game fixture empty_game in its pxt.json, which now matches empty-game.png, so the test comparing them no longer ignores the name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The name is joined onto build_dir to name every stage directory, and those directories are deleted before being rewritten, so a name like `../victim` made a build delete and write outside build_dir. It's also the website's package.json name, which npm requires to be lowercase and URL-safe. Require a single lowercase, URL-safe path component of at most 64 characters that isn't reserved on Windows. The reserved check uses ntpath, so a config is rejected the same way on every OS. Fixes #17 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A path source containing build_dir, such as `path: "."` with `build_dir: build`, copied the build into itself until the OS rejected the path. A source inside a stage directory was deleted before it was copied, and --full-rebuild could delete a source inside build_dir. resolve_config_paths now rejects either folder being inside the other, or both being the same one. This runs on resolved paths before any task or --full-rebuild, so nothing is built or deleted first. A config kept in the game's folder needs a build_dir outside it, like `../build`. Split the full rebuild test, as `build_dir: "."` now fails this check before reaching the "Refusing to clear" guard, which is tested on an unrelated folder. Also make fetch_code warn about a target version mismatch for every code source, not only `file`. Fixes #16 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the fixed-bug history, finished roadmap items and per-file test descriptions. Keep status, decisions, plans and open issues, with short layout and command sections. [skip-tests] Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The integration tests spend most of their time in redun's SQLite calls, one test at a time. With -n 4 they take about 60s instead of 94s on a Windows laptop, with the same coverage. More workers didn't help, and the unit tests are no faster in parallel, so they stay serial. The e2e tests can't use xdist: each worker would run the session fixture, all installing into the same dir at once. They now fail with a clear message under it instead of racing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.