Expand unit tests, add coverage, and split unit/e2e CI

.github/workflows/e2e.yaml

@@ -1,4 +1,4 @@
-name: CI
+name: E2E tests
 
 on:
   push:
@@ -35,5 +35,7 @@ jobs:
       - name: Install dependencies
         run: uv sync
 
-      - name: Run tests with pytest
-        run: uv run pytest
+      # Coverage is skipped: the tests run the mkcd2app executable in a subprocess,
+      #  which coverage doesn't measure
+      - name: Run e2e tests
+        run: uv run pytest -m e2e --no-cov

.github/workflows/unit.yaml

@@ -0,0 +1,35 @@
+name: Unit tests
+
+on:
+  push:
+    branches: [ main ]
+  # Any base branch, so stacked PRs get CI too
+  pull_request:
+
+jobs:
+  unit:
+    strategy:
+      matrix:
+        os: [ ubuntu-latest, windows-latest, macos-latest ]
+    runs-on: ${{ matrix.os }}
+    permissions:
+      contents: read
+    steps:
+      - name: Check out repo
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+          version: "latest"
+
+      - name: Set up Python
+        run: uv python install 3.14
+
+      - name: Install dependencies
+        run: uv sync
+
+      # Offline and fast; no Node.js or toolchain install needed
+      - name: Run unit tests with coverage
+        run: uv run pytest -m unit

.gitignore

@@ -5,3 +5,5 @@ examples/*
 !examples/*.yaml
 !examples/*.png
 .idea
+.coverage
+.coverage.*

CLAUDE.md

@@ -17,11 +17,17 @@ uv sync                          # install deps + the mkcd2app entry point
 uv run ruff check .              # lint
 uv run ruff format --check .     # format check (drop --check to apply)
 uv run mypy .                    # strict type check
-uv run pytest                    # all tests
+uv run pytest                    # all tests, with a coverage report
+uv run pytest -m unit            # only the fast offline tests (tests/unit)
+uv run pytest -m e2e --no-cov    # only the e2e tests (tests/e2e)
+uv run pytest --no-cov           # skip coverage (e.g. when using a debugger)
 uv run pytest tests/e2e/test_statics.py::test_empty_game_build_to_statics_static
 ```
 
-CI (`lint.yaml`, `e2e.yaml`) runs exactly these lint/format/mypy/pytest commands.
+CI runs exactly these commands: `lint.yaml` the lint/format/mypy checks, and
+`unit.yaml` (`-m unit`) and `e2e.yaml` (`-m e2e --no-cov`) the tests, independently, on
+ubuntu/windows/macos. Every test must be under `tests/unit` or `tests/e2e`, or
+collection fails, since a test in neither would run in neither workflow.
 pre-commit runs ruff check, ruff format --check, and mypy on staged Python files.
 
 Running the tool end to end (installs must happen first, in this order):
@@ -32,6 +38,16 @@ uv run mkcd2app target install 4.0.14
 uv run mkcd2app --debug build "examples/Racers.yaml"
 ```
 
+Notes on the unit tests:
+- `tests/conftest.py` marks every test `unit` or `e2e` by its directory, and holds
+  the `mkcd2app_home` fixture (points `MKCD2APP_HOME` at a temp dir). Keep it the only
+  `conftest.py`: mypy fails on two modules both named `conftest`.
+- Shared helpers (`make_config_yaml`, `FakeRunCmd`, `capture_logger`, ...) are in
+  `tests/unit/helpers.py`, imported as `from helpers import ...`.
+- Call a redun task's plain function with `.func(...)`, and monkeypatch `run_cmd` in
+  the module under test, so no `npm`/`mkc` runs.
+- Our loggers set `propagate = False`, so `caplog` needs `capture_logger` to see them.
+
 Notes on the e2e tests:
 - Run pytest from the repo root: tests use `Path.cwd()` and repo-relative paths, and
   shell out to the installed `mkcd2app` executable rather than importing it.

docs/PROJECT_CONTEXT.md

@@ -4,7 +4,8 @@ _Verified against `toolchain` @ `d7ad64a` plus the `MKCD2APP_HOME` change, the
 small known-issue fixes, and the separate deps-install task, and `origin/main` @ `d31e2a5`, as of 2026-10-29.
 External program requirements audited against `toolchain` @ `4d518d9` on 2026-11-03, then
 updated for the Dulwich git replacement and the generic `git` code source the same day,
-and for the `file` code source (PNG/UF2 with embedded project, then ZIPs) the same day._
+and for the `file` code source (PNG/UF2 with embedded project, then ZIPs) the same day.
+Test suite section updated for the unit test expansion on 2026-11-03._
 
 ## Goal
 Rewrite `UnsignedArduino/MakeCode-Arcade-to-App` (converts MakeCode Arcade games into a
@@ -36,8 +37,8 @@ main (legacy)  ←  PR #6 `dag` (draft)  ←  PR #7 `toolchain` (draft, base = d
 | `utils/` | `run_cmd`, paths (platformdirs), `run_redun_task`, `rmtree_robust`, semver, `git` (Dulwich clone/checkout) |
 | `resources/` | Vite+React+TS template, `mkc` js_tools, empty fixture project |
 
-Other: `tests/e2e/test_statics.py`, `tests/unit/` (config code sources, `utils/git`, PNG/UF2/ZIP file source), `tests/fixtures/`, `examples/` (Racers), CI
-workflows `lint.yaml` (ubuntu) and `e2e.yaml` (ubuntu/windows/macos). Tooling: uv,
+Other: `tests/e2e/test_statics.py`, `tests/unit/` (config code sources, `utils/git`, PNG/UF2/ZIP file source, code/website tasks, redun patches, utils), `tests/fixtures/`, `examples/` (Racers), CI
+workflows `lint.yaml` (ubuntu), `unit.yaml` and `e2e.yaml` (both ubuntu/windows/macos). Tooling: uv,
 hatchling, Python ≥3.14, ruff + mypy strict, pre-commit.
 
 ## Architecture
@@ -57,8 +58,8 @@ hatchling, Python ≥3.14, ruff + mypy strict, pre-commit.
   `build_website*` task copies that `node_modules` into its own copy of the filled site
   and runs only `npm run build*`.
 - **redun patches** (`_redun_compat.py`): `get_proto` (Windows drive letters),
-  `ContentDir._calc_hash` and `Dir._calc_hash` (hash file contents, not mtimes). No
-  tests cover them yet.
+  `ContentDir._calc_hash` and `Dir._calc_hash` (hash file contents, not mtimes).
+  Covered by `tests/unit/test_redun_compat.py`.
 ## External programs
 `run_cmd` only ever runs `npm` and `mkc`:
 - **Node.js ≥ 22.12 + npm, on PATH. Always required.** The floor comes from
@@ -120,8 +121,10 @@ configs; `fetch_code` handles both in one `case`.
   logs to stderr and requires `DONE == 0` and `CACHED == TOTAL` on an unchanged
   rebuild; this depends on redun's log format, which is safe while redun is pinned.
   Nothing yet tests that changing an input does trigger a rebuild.
-- **CI** (`lint.yaml`, `e2e.yaml`) runs on pushes to `main` and on PRs into any branch,
-  so the stacked PRs get CI. Lint and all three e2e jobs are green on PR #7.
+- **CI** (`lint.yaml`, `unit.yaml`, `e2e.yaml`) runs on pushes to `main` and on PRs
+  into any branch, so the stacked PRs get CI. Lint and all three e2e jobs are green on
+  PR #7. `unit.yaml` runs `pytest -m unit` (no Node.js), and `e2e.yaml` runs
+  `pytest -m e2e --no-cov`; neither waits for the other.
 - `tests/unit` covers config parsing of the `git`/`github` sources and
   `clone_and_checkout` against a local repo built with Dulwich (default branch, branch,
   tag, short/full SHA, the same-tree-as-HEAD regression, unknown ref), so it runs
@@ -138,14 +141,26 @@ configs; `fetch_code` handles both in one `case`.
   `empty_game_file_source.yaml`, and a ZIP of `tests/fixtures/empty_game` made at test
   time with a GitHub-style `empty_game-master/` wrapper (checked byte for byte
   against the checkout, so CRLF on Windows is fine). The duck builds end to end on 4.1.27, and on 4.0.14
-  with the version-mismatch warning, but isn't in e2e. Nothing covers the redun
-  patches yet.
+  with the version-mismatch warning, but isn't in e2e.
+- `test_code.py` and `test_website.py` run every code-input and website task through
+  `.func` with `run_cmd` (and `clone_and_checkout`, `requests.get`,
+  `get_sim_html_path`) monkeypatched, against a copy of the real bundled template.
+  `test_redun_compat.py` checks the `get_proto` patch and that directory hashes
+  ignore mtimes but change with content, file adds/removes/renames, and path.
+  `utils/` (`paths`, `run`, `filesystem`, `resources`) and `extract_meta_comment` have
+  their own small tests. Untested: `cli.py`, `main.py`, the `build_project` DAG
+  wiring, toolchain/target install tasks, and `is_strict_semver`.
+- pytest config is in `pyproject.toml`: coverage on by default (`--no-cov` to skip),
+  `--strict-markers`, and `unit`/`e2e` markers applied by `tests/conftest.py`. Unit
+  tests cover about 66% of `src/mkcd2app` (run `uv run pytest -m unit`).
 ## Roadmap
 1. ~~Fix e2e + CI~~ (done: fixture `target:`, isolated install fixture, caching on,
    job-status cache assertion, CI on all PRs).
 2. ~~Rename `--clear-cache`~~ (done: now `--full-rebuild`, and its help text says to use
    it for out-of-band input changes). Still belongs in the README once that exists.
-3. More unit/integration tests, including the redun patches (`git` and `file` sources are covered).
+3. More unit/integration tests: `cli.py`/`main.py`, `build_project` wiring with
+   `run_cmd` stubbed, and the toolchain/target install tasks (redun patches, code and
+   website tasks, and utils are covered).
 4. Electron, then Tauri (reference: `main`).
 5. Save-state for Electron/Tauri (investigate `pxtsim` postMessage).
 6. Toolchain polish: `status`/`list`, `build` pre-flight, extension cache, version-pinning

pyproject.toml

@@ -31,6 +31,7 @@ dev = [
     "mypy>=2.3.0",
     "pre-commit>=4.6.1",
     "pytest>=9.1.1",
+    "pytest-cov>=7.1.0",
     "pytest-timeout>=2.4.0",
     "ruff>=0.16.0",
     "types-pyyaml>=6.0.12.20260724",
@@ -42,3 +43,16 @@ target-version = "py314"
 [tool.mypy]
 python_version = "3.14"
 strict = true
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+# Coverage is on by default; pass --no-cov to turn it off (e.g. when debugging)
+addopts = ["--strict-markers", "--cov", "--cov-report=term-missing"]
+# Applied by directory in tests/conftest.py, so tests don't need to mark themselves
+markers = [
+    "unit: fast, offline tests under tests/unit",
+    "e2e: slow tests under tests/e2e that install the toolchain and build real games",
+]
+
+[tool.coverage.run]
+source = ["mkcd2app"]

tests/conftest.py

@@ -0,0 +1,29 @@
+from pathlib import Path
+
+import pytest
+
+TESTS_DIR = Path(__file__).parent
+
+
+def pytest_collection_modifyitems(items: list[pytest.Item]) -> None:
+    """
+    Mark each test `unit` or `e2e` by its directory, so `-m unit` selects them.
+
+    CI runs `-m unit` and `-m e2e` as separate workflows, so a test outside both
+    directories would run in neither; fail collection instead of skipping it quietly.
+    """
+    for item in items:
+        parts = item.path.relative_to(TESTS_DIR).parts
+        if len(parts) < 2 or parts[0] not in ("unit", "e2e"):
+            raise pytest.UsageError(
+                f"{item.nodeid}: tests must live under tests/unit or tests/e2e"
+            )
+        item.add_marker(parts[0])
+
+
+@pytest.fixture
+def mkcd2app_home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
+    """Point MKCD2APP_HOME at a temp dir, so the test can't touch the real user dirs."""
+    home = tmp_path / "mkcd2app-home"
+    monkeypatch.setenv("MKCD2APP_HOME", str(home))
+    return home

tests/unit/helpers.py

@@ -0,0 +1,109 @@
+"""Shared helpers for unit tests (a plain module, since pytest won't collect it)."""
+
+import hashlib
+import logging
+import os
+from collections.abc import Callable, Iterator
+from contextlib import contextmanager
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+import pytest
+import yaml
+
+TARGET_VERSION = "4.0.14"
+
+
+def make_config_yaml(
+    build_dir: Path,
+    code: dict[str, Any] | None = None,
+    icon: dict[str, Any] | None = None,
+    project: dict[str, Any] | None = None,
+    target: str = TARGET_VERSION,
+) -> str:
+    """
+    Build a version 1 config as YAML text, the form the redun tasks take.
+
+    :param build_dir: The config's build_dir.
+    :param code: The inputs.code source. Defaults to a path source that doesn't exist,
+     for tasks that never read it.
+    :param icon: The inputs.assets.icon source, or None for no icon.
+    :param project: Keys to override in the project section.
+    :param target: The MakeCode Arcade target version.
+    """
+    assets: dict[str, Any] = {"icon": icon} if icon else {}
+    return yaml.safe_dump(
+        {
+            "version": 1,
+            "project": {
+                "name": "Test Game",
+                "path_friendly_name": "test",
+                "description": "A test game",
+                "author": "Tester",
+                "version": "1.2.3",
+                **(project or {}),
+            },
+            "inputs": {
+                "code": code or {"type": "path", "value": str(build_dir / "nowhere")},
+                "assets": assets,
+            },
+            "build_dir": str(build_dir),
+            "target": target,
+            "outputs": [{"type": "static"}],
+        }
+    )
+
+
+@dataclass
+class CmdCall:
+    command: list[str]
+    cwd: Path
+    which_path: str | os.PathLike[str] | None
+
+
+class FakeRunCmd:
+    """
+    Stands in for ``run_cmd``: records each call, then runs ``on_run`` (if given) so
+    the test can create the files the real command would have.
+    """
+
+    def __init__(self, on_run: Callable[[CmdCall], None] | None = None) -> None:
+        self.calls: list[CmdCall] = []
+        self.on_run = on_run
+
+    def __call__(
+        self,
+        command: list[str],
+        cwd: Path | str,
+        which_path: str | os.PathLike[str] | None = None,
+    ) -> str:
+        call = CmdCall(command, Path(cwd), which_path)
+        self.calls.append(call)
+        if self.on_run:
+            self.on_run(call)
+        return ""
+
+
+def tree_digest(root: Path) -> dict[str, str]:
+    """Map each file under ``root`` (relative POSIX path) to its SHA-256."""
+    return {
+        p.relative_to(root).as_posix(): hashlib.sha256(p.read_bytes()).hexdigest()
+        for p in sorted(root.rglob("*"))
+        if p.is_file()
+    }
+
+
+@contextmanager
+def capture_logger(
+    caplog: pytest.LogCaptureFixture, logger: logging.Logger
+) -> Iterator[pytest.LogCaptureFixture]:
+    """
+    Capture records from one of our loggers. ``create_logger`` sets
+    ``propagate = False``, so caplog's root handler never sees them otherwise.
+    """
+    logger.addHandler(caplog.handler)
+    try:
+        yield caplog
+    finally:
+        logger.removeHandler(caplog.handler)