Skip to content

Expand unit tests, add coverage, and split unit/e2e CI - #8

Merged
UnsignedArduino merged 3 commits into
toolchainfrom
ccr-209d5b24-ah720e
Nov 4, 2026
Merged

UnsignedArduino merged 3 commits into
toolchainfrom
ccr-209d5b24-ah720e

Conversation

@UnsignedArduino

@UnsignedArduino UnsignedArduino commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Summary

Unit coverage of src/mkcd2app goes from 42% to 66%. All new tests run offline: redun tasks are called through .func(...), and run_cmd, clone_and_checkout, requests.get and get_sim_html_path are monkeypatched, so no npm, mkc or network access is needed. Unit and e2e tests now also run as separate CI workflows.

Tooling

  • Adds pytest-cov as a dev dependency. Coverage runs on every pytest run (--no-cov turns it off), configured in pyproject.toml.
  • Adds tests/conftest.py, which marks each test unit or e2e by its directory, so uv run pytest -m unit runs only the fast tests (~3s). Also turns on --strict-markers. A test outside tests/unit and tests/e2e fails collection, because it would otherwise run in neither CI workflow.
  • There is only one conftest.py, because mypy fails when two modules are both named conftest. Shared test helpers are in tests/unit/helpers.py.

CI

  • New unit.yaml ("Unit tests"): uv run pytest -m unit with coverage, on ubuntu/windows/macos. It skips the Node.js setup step, since the unit tests don't need Node.
  • e2e.yaml ("E2E tests") now runs only uv run pytest -m e2e --no-cov. Coverage would read ~0% there anyway, because the tests drive the mkcd2app executable in a subprocess.
  • The two workflows run in parallel and neither waits for the other. Check names are now unit (<os>) and, unchanged, e2e (<os>). If branch protection requires specific checks, add the unit ones.

New tests

File Covers Module coverage
test_website.py copying and filling the real bundled template (title, favicon inlining, package.json, input left unmodified, old output replaced), install_website_deps, both website builds including relative node_modules/.bin symlinks website.py 23% → 100%
test_code.py fetch_code for all five source types, _warn_on_target_mismatch, build_binary_js, copy_support_files with path icon, URL icon and HTTP error code.py 25% → 100%
test_redun_compat.py Windows drive-letter get_proto patch; folder hashes ignore mtime but change on file edit, add, remove, rename and path change (checked that the mtime tests fail on unpatched redun) _redun_compat.py 58% → 100%
test_meta_comment.py, test_paths.py, test_run.py, test_filesystem.py, test_resources.py extract_meta_comment, the MKCD2APP_HOME override and every path helper, run_cmd (success, cwd, BuildError, which_path), rmtree_robust with read-only files, bundled resources each at 100% except text.py (82%)

Deliberately not covered here: is_strict_semver, and title formatting and escaping in fill_website_template. Those are known bugs being fixed separately, and that fix will add their tests. Still untested: cli.py, main.py, the build_project wiring, and the toolchain/target install tasks. These are listed in PROJECT_CONTEXT.md.

CLAUDE.md and docs/PROJECT_CONTEXT.md are updated to match.

Test plan

  • uv run ruff check ., uv run ruff format --check ., uv run mypy .: clean
  • uv run pytest locally on Linux: 153 passed (148 unit + 5 e2e)
  • CI on ubuntu/windows/macos green at 06130e9, before the workflow split
  • A stray tests/test_stray.py fails collection with a clear error
  • CI green with the split workflows at 8925378: unit and e2e on all three OSes, plus lint

🤖 Generated with Claude Code

https://claude.ai/code/session_01Mbb9ZdHu1tFP2ZhM62pbgq

pytest-cov runs on every pytest invocation (--no-cov to skip), and
tests/conftest.py marks each test unit or e2e by its directory, so
`uv run pytest -m unit` runs only the fast offline tests. CI is unchanged:
`uv run pytest` still runs everything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mbb9ZdHu1tFP2ZhM62pbgq
- test_code.py: fetch_code for every source type, the target mismatch
  warning, build_binary_js, and copy_support_files with path/URL icons
- test_website.py: copy/fill the real bundled template (title, favicon,
  package.json, copy-on-write), install_website_deps, and both website
  builds including node_modules symlinks
- test_redun_compat.py: the get_proto patch and content-based dir hashing
- extract_meta_comment, utils/paths, run_cmd, rmtree_robust, resources

Tasks run through .func with run_cmd and network calls monkeypatched, so
everything stays offline. Unit coverage goes from 42% to 66%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mbb9ZdHu1tFP2ZhM62pbgq
unit.yaml runs `pytest -m unit` with coverage and no Node.js setup, so unit
results come back in seconds instead of after the toolchain install.
e2e.yaml now runs only `pytest -m e2e --no-cov`, since coverage can't see
the mkcd2app subprocess the e2e tests drive. Both run on all three OSes and
neither waits for the other.

Because CI now selects by marker, tests/conftest.py fails collection for a
test outside tests/unit and tests/e2e, which would otherwise run in
neither workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mbb9ZdHu1tFP2ZhM62pbgq
@UnsignedArduino UnsignedArduino changed the title Expand unit tests and add coverage reporting Expand unit tests, add coverage, and split unit/e2e CI Nov 4, 2026
@UnsignedArduino
UnsignedArduino merged commit c39140a into toolchain Nov 4, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants