diff --git a/.github/workflows/e2e.yaml b/.github/workflows/e2e.yaml index b978ab1..f468d7d 100644 --- a/.github/workflows/e2e.yaml +++ b/.github/workflows/e2e.yaml @@ -1,4 +1,4 @@ -name: CI +name: E2E tests on: push: @@ -35,5 +35,7 @@ jobs: - name: Install dependencies run: uv sync - - name: Run tests with pytest - run: uv run pytest + # Coverage is skipped: the tests run the mkcd2app executable in a subprocess, + # which coverage doesn't measure + - name: Run e2e tests + run: uv run pytest -m e2e --no-cov diff --git a/.github/workflows/unit.yaml b/.github/workflows/unit.yaml new file mode 100644 index 0000000..7a6b041 --- /dev/null +++ b/.github/workflows/unit.yaml @@ -0,0 +1,35 @@ +name: Unit tests + +on: + push: + branches: [ main ] + # Any base branch, so stacked PRs get CI too + pull_request: + +jobs: + unit: + strategy: + matrix: + os: [ ubuntu-latest, windows-latest, macos-latest ] + runs-on: ${{ matrix.os }} + permissions: + contents: read + steps: + - name: Check out repo + uses: actions/checkout@v4 + + - name: Install uv + uses: astral-sh/setup-uv@v5 + with: + enable-cache: true + version: "latest" + + - name: Set up Python + run: uv python install 3.14 + + - name: Install dependencies + run: uv sync + + # Offline and fast; no Node.js or toolchain install needed + - name: Run unit tests with coverage + run: uv run pytest -m unit diff --git a/.gitignore b/.gitignore index dbe7f58..dc19d7a 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,5 @@ examples/* !examples/*.yaml !examples/*.png .idea +.coverage +.coverage.* diff --git a/CLAUDE.md b/CLAUDE.md index f29a8f9..70a9a3c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,11 +17,17 @@ uv sync # install deps + the mkcd2app entry point uv run ruff check . # lint uv run ruff format --check . # format check (drop --check to apply) uv run mypy . # strict type check -uv run pytest # all tests +uv run pytest # all tests, with a coverage report +uv run pytest -m unit # only the fast offline tests (tests/unit) +uv run pytest -m e2e --no-cov # only the e2e tests (tests/e2e) +uv run pytest --no-cov # skip coverage (e.g. when using a debugger) uv run pytest tests/e2e/test_statics.py::test_empty_game_build_to_statics_static ``` -CI (`lint.yaml`, `e2e.yaml`) runs exactly these lint/format/mypy/pytest commands. +CI runs exactly these commands: `lint.yaml` the lint/format/mypy checks, and +`unit.yaml` (`-m unit`) and `e2e.yaml` (`-m e2e --no-cov`) the tests, independently, on +ubuntu/windows/macos. Every test must be under `tests/unit` or `tests/e2e`, or +collection fails, since a test in neither would run in neither workflow. pre-commit runs ruff check, ruff format --check, and mypy on staged Python files. Running the tool end to end (installs must happen first, in this order): @@ -32,6 +38,16 @@ uv run mkcd2app target install 4.0.14 uv run mkcd2app --debug build "examples/Racers.yaml" ``` +Notes on the unit tests: +- `tests/conftest.py` marks every test `unit` or `e2e` by its directory, and holds + the `mkcd2app_home` fixture (points `MKCD2APP_HOME` at a temp dir). Keep it the only + `conftest.py`: mypy fails on two modules both named `conftest`. +- Shared helpers (`make_config_yaml`, `FakeRunCmd`, `capture_logger`, ...) are in + `tests/unit/helpers.py`, imported as `from helpers import ...`. +- Call a redun task's plain function with `.func(...)`, and monkeypatch `run_cmd` in + the module under test, so no `npm`/`mkc` runs. +- Our loggers set `propagate = False`, so `caplog` needs `capture_logger` to see them. + Notes on the e2e tests: - Run pytest from the repo root: tests use `Path.cwd()` and repo-relative paths, and shell out to the installed `mkcd2app` executable rather than importing it. diff --git a/docs/PROJECT_CONTEXT.md b/docs/PROJECT_CONTEXT.md index 5725d8e..08ac0a8 100644 --- a/docs/PROJECT_CONTEXT.md +++ b/docs/PROJECT_CONTEXT.md @@ -4,7 +4,8 @@ _Verified against `toolchain` @ `d7ad64a` plus the `MKCD2APP_HOME` change, the small known-issue fixes, and the separate deps-install task, and `origin/main` @ `d31e2a5`, as of 2026-10-29. External program requirements audited against `toolchain` @ `4d518d9` on 2026-11-03, then updated for the Dulwich git replacement and the generic `git` code source the same day, -and for the `file` code source (PNG/UF2 with embedded project, then ZIPs) the same day._ +and for the `file` code source (PNG/UF2 with embedded project, then ZIPs) the same day. +Test suite section updated for the unit test expansion on 2026-11-03._ ## Goal Rewrite `UnsignedArduino/MakeCode-Arcade-to-App` (converts MakeCode Arcade games into a @@ -36,8 +37,8 @@ main (legacy) ← PR #6 `dag` (draft) ← PR #7 `toolchain` (draft, base = d | `utils/` | `run_cmd`, paths (platformdirs), `run_redun_task`, `rmtree_robust`, semver, `git` (Dulwich clone/checkout) | | `resources/` | Vite+React+TS template, `mkc` js_tools, empty fixture project | -Other: `tests/e2e/test_statics.py`, `tests/unit/` (config code sources, `utils/git`, PNG/UF2/ZIP file source), `tests/fixtures/`, `examples/` (Racers), CI -workflows `lint.yaml` (ubuntu) and `e2e.yaml` (ubuntu/windows/macos). Tooling: uv, +Other: `tests/e2e/test_statics.py`, `tests/unit/` (config code sources, `utils/git`, PNG/UF2/ZIP file source, code/website tasks, redun patches, utils), `tests/fixtures/`, `examples/` (Racers), CI +workflows `lint.yaml` (ubuntu), `unit.yaml` and `e2e.yaml` (both ubuntu/windows/macos). Tooling: uv, hatchling, Python ≥3.14, ruff + mypy strict, pre-commit. ## Architecture @@ -57,8 +58,8 @@ hatchling, Python ≥3.14, ruff + mypy strict, pre-commit. `build_website*` task copies that `node_modules` into its own copy of the filled site and runs only `npm run build*`. - **redun patches** (`_redun_compat.py`): `get_proto` (Windows drive letters), - `ContentDir._calc_hash` and `Dir._calc_hash` (hash file contents, not mtimes). No - tests cover them yet. + `ContentDir._calc_hash` and `Dir._calc_hash` (hash file contents, not mtimes). + Covered by `tests/unit/test_redun_compat.py`. ## External programs `run_cmd` only ever runs `npm` and `mkc`: - **Node.js ≥ 22.12 + npm, on PATH. Always required.** The floor comes from @@ -120,8 +121,10 @@ configs; `fetch_code` handles both in one `case`. logs to stderr and requires `DONE == 0` and `CACHED == TOTAL` on an unchanged rebuild; this depends on redun's log format, which is safe while redun is pinned. Nothing yet tests that changing an input does trigger a rebuild. -- **CI** (`lint.yaml`, `e2e.yaml`) runs on pushes to `main` and on PRs into any branch, - so the stacked PRs get CI. Lint and all three e2e jobs are green on PR #7. +- **CI** (`lint.yaml`, `unit.yaml`, `e2e.yaml`) runs on pushes to `main` and on PRs + into any branch, so the stacked PRs get CI. Lint and all three e2e jobs are green on + PR #7. `unit.yaml` runs `pytest -m unit` (no Node.js), and `e2e.yaml` runs + `pytest -m e2e --no-cov`; neither waits for the other. - `tests/unit` covers config parsing of the `git`/`github` sources and `clone_and_checkout` against a local repo built with Dulwich (default branch, branch, tag, short/full SHA, the same-tree-as-HEAD regression, unknown ref), so it runs @@ -138,14 +141,26 @@ configs; `fetch_code` handles both in one `case`. `empty_game_file_source.yaml`, and a ZIP of `tests/fixtures/empty_game` made at test time with a GitHub-style `empty_game-master/` wrapper (checked byte for byte against the checkout, so CRLF on Windows is fine). The duck builds end to end on 4.1.27, and on 4.0.14 - with the version-mismatch warning, but isn't in e2e. Nothing covers the redun - patches yet. + with the version-mismatch warning, but isn't in e2e. +- `test_code.py` and `test_website.py` run every code-input and website task through + `.func` with `run_cmd` (and `clone_and_checkout`, `requests.get`, + `get_sim_html_path`) monkeypatched, against a copy of the real bundled template. + `test_redun_compat.py` checks the `get_proto` patch and that directory hashes + ignore mtimes but change with content, file adds/removes/renames, and path. + `utils/` (`paths`, `run`, `filesystem`, `resources`) and `extract_meta_comment` have + their own small tests. Untested: `cli.py`, `main.py`, the `build_project` DAG + wiring, toolchain/target install tasks, and `is_strict_semver`. +- pytest config is in `pyproject.toml`: coverage on by default (`--no-cov` to skip), + `--strict-markers`, and `unit`/`e2e` markers applied by `tests/conftest.py`. Unit + tests cover about 66% of `src/mkcd2app` (run `uv run pytest -m unit`). ## Roadmap 1. ~~Fix e2e + CI~~ (done: fixture `target:`, isolated install fixture, caching on, job-status cache assertion, CI on all PRs). 2. ~~Rename `--clear-cache`~~ (done: now `--full-rebuild`, and its help text says to use it for out-of-band input changes). Still belongs in the README once that exists. -3. More unit/integration tests, including the redun patches (`git` and `file` sources are covered). +3. More unit/integration tests: `cli.py`/`main.py`, `build_project` wiring with + `run_cmd` stubbed, and the toolchain/target install tasks (redun patches, code and + website tasks, and utils are covered). 4. Electron, then Tauri (reference: `main`). 5. Save-state for Electron/Tauri (investigate `pxtsim` postMessage). 6. Toolchain polish: `status`/`list`, `build` pre-flight, extension cache, version-pinning diff --git a/pyproject.toml b/pyproject.toml index 3730fd5..eb911d3 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -31,6 +31,7 @@ dev = [ "mypy>=2.3.0", "pre-commit>=4.6.1", "pytest>=9.1.1", + "pytest-cov>=7.1.0", "pytest-timeout>=2.4.0", "ruff>=0.16.0", "types-pyyaml>=6.0.12.20260724", @@ -42,3 +43,16 @@ target-version = "py314" [tool.mypy] python_version = "3.14" strict = true + +[tool.pytest.ini_options] +testpaths = ["tests"] +# Coverage is on by default; pass --no-cov to turn it off (e.g. when debugging) +addopts = ["--strict-markers", "--cov", "--cov-report=term-missing"] +# Applied by directory in tests/conftest.py, so tests don't need to mark themselves +markers = [ + "unit: fast, offline tests under tests/unit", + "e2e: slow tests under tests/e2e that install the toolchain and build real games", +] + +[tool.coverage.run] +source = ["mkcd2app"] diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 0000000..c9422b2 --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,29 @@ +from pathlib import Path + +import pytest + +TESTS_DIR = Path(__file__).parent + + +def pytest_collection_modifyitems(items: list[pytest.Item]) -> None: + """ + Mark each test `unit` or `e2e` by its directory, so `-m unit` selects them. + + CI runs `-m unit` and `-m e2e` as separate workflows, so a test outside both + directories would run in neither; fail collection instead of skipping it quietly. + """ + for item in items: + parts = item.path.relative_to(TESTS_DIR).parts + if len(parts) < 2 or parts[0] not in ("unit", "e2e"): + raise pytest.UsageError( + f"{item.nodeid}: tests must live under tests/unit or tests/e2e" + ) + item.add_marker(parts[0]) + + +@pytest.fixture +def mkcd2app_home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Point MKCD2APP_HOME at a temp dir, so the test can't touch the real user dirs.""" + home = tmp_path / "mkcd2app-home" + monkeypatch.setenv("MKCD2APP_HOME", str(home)) + return home diff --git a/tests/unit/helpers.py b/tests/unit/helpers.py new file mode 100644 index 0000000..4d27e99 --- /dev/null +++ b/tests/unit/helpers.py @@ -0,0 +1,109 @@ +"""Shared helpers for unit tests (a plain module, since pytest won't collect it).""" + +import hashlib +import logging +import os +from collections.abc import Callable, Iterator +from contextlib import contextmanager +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +import pytest +import yaml + +TARGET_VERSION = "4.0.14" + + +def make_config_yaml( + build_dir: Path, + code: dict[str, Any] | None = None, + icon: dict[str, Any] | None = None, + project: dict[str, Any] | None = None, + target: str = TARGET_VERSION, +) -> str: + """ + Build a version 1 config as YAML text, the form the redun tasks take. + + :param build_dir: The config's build_dir. + :param code: The inputs.code source. Defaults to a path source that doesn't exist, + for tasks that never read it. + :param icon: The inputs.assets.icon source, or None for no icon. + :param project: Keys to override in the project section. + :param target: The MakeCode Arcade target version. + """ + assets: dict[str, Any] = {"icon": icon} if icon else {} + return yaml.safe_dump( + { + "version": 1, + "project": { + "name": "Test Game", + "path_friendly_name": "test", + "description": "A test game", + "author": "Tester", + "version": "1.2.3", + **(project or {}), + }, + "inputs": { + "code": code or {"type": "path", "value": str(build_dir / "nowhere")}, + "assets": assets, + }, + "build_dir": str(build_dir), + "target": target, + "outputs": [{"type": "static"}], + } + ) + + +@dataclass +class CmdCall: + command: list[str] + cwd: Path + which_path: str | os.PathLike[str] | None + + +class FakeRunCmd: + """ + Stands in for ``run_cmd``: records each call, then runs ``on_run`` (if given) so + the test can create the files the real command would have. + """ + + def __init__(self, on_run: Callable[[CmdCall], None] | None = None) -> None: + self.calls: list[CmdCall] = [] + self.on_run = on_run + + def __call__( + self, + command: list[str], + cwd: Path | str, + which_path: str | os.PathLike[str] | None = None, + ) -> str: + call = CmdCall(command, Path(cwd), which_path) + self.calls.append(call) + if self.on_run: + self.on_run(call) + return "" + + +def tree_digest(root: Path) -> dict[str, str]: + """Map each file under ``root`` (relative POSIX path) to its SHA-256.""" + return { + p.relative_to(root).as_posix(): hashlib.sha256(p.read_bytes()).hexdigest() + for p in sorted(root.rglob("*")) + if p.is_file() + } + + +@contextmanager +def capture_logger( + caplog: pytest.LogCaptureFixture, logger: logging.Logger +) -> Iterator[pytest.LogCaptureFixture]: + """ + Capture records from one of our loggers. ``create_logger`` sets + ``propagate = False``, so caplog's root handler never sees them otherwise. + """ + logger.addHandler(caplog.handler) + try: + yield caplog + finally: + logger.removeHandler(caplog.handler) diff --git a/tests/unit/test_code.py b/tests/unit/test_code.py new file mode 100644 index 0000000..7193288 --- /dev/null +++ b/tests/unit/test_code.py @@ -0,0 +1,324 @@ +import json +import logging +from collections.abc import Generator +from pathlib import Path +from typing import Any + +import pytest +import requests +from helpers import ( + TARGET_VERSION, + CmdCall, + FakeRunCmd, + capture_logger, + make_config_yaml, + tree_digest, +) +from PIL import Image +from redun.file import ContentDir, ContentFile + +from mkcd2app.build_project.inputs import code +from mkcd2app.build_project.inputs.code import ( + _warn_on_target_mismatch, + build_binary_js, + copy_support_files, + fetch_code, +) +from mkcd2app.utils.paths import get_js_tools_bin_dir + +FIXTURES = Path(__file__).parent.parent / "fixtures" +EMPTY_GAME_DIR = FIXTURES / "empty_game" +EMPTY_PNG = FIXTURES / "embedded" / "empty-game.png" # saved with 4.0.14 +DUCK_PNG = FIXTURES / "embedded" / "falling-duck.png" # saved with 4.1.27 +MKC_JSON = {"targetWebsite": f"https://arcade.makecode.com/v{TARGET_VERSION}"} + + +def read_mkc_json(code_path: Path) -> Any: + return json.loads((code_path / "mkc.json").read_text(encoding="utf-8")) + + +def run_fetch_code(build_dir: Path, source: dict[str, Any]) -> Path: + result: ContentDir = fetch_code.func(make_config_yaml(build_dir, code=source)) + return Path(result.path) + + +# ------------------------------------------------------------ fetch_code ------ + + +def test_path_source_copies_project(tmp_path: Path) -> None: + code_path = run_fetch_code( + tmp_path / "build", {"type": "path", "value": str(EMPTY_GAME_DIR)} + ) + assert code_path == tmp_path / "build" / "test-source" + copied = tree_digest(code_path) + assert copied.pop("mkc.json") + assert copied == tree_digest(EMPTY_GAME_DIR) + assert read_mkc_json(code_path) == MKC_JSON + + +def test_fetch_code_replaces_previous_source(tmp_path: Path) -> None: + stale = tmp_path / "build" / "test-source" / "stale.ts" + stale.parent.mkdir(parents=True) + stale.write_text("old", encoding="utf-8") + run_fetch_code(tmp_path / "build", {"type": "path", "value": str(EMPTY_GAME_DIR)}) + assert not stale.exists() + + +def test_mkc_json_uses_config_target(tmp_path: Path) -> None: + result: ContentDir = fetch_code.func( + make_config_yaml( + tmp_path / "build", + code={"type": "path", "value": str(EMPTY_GAME_DIR)}, + target="4.1.27", + ) + ) + assert read_mkc_json(Path(result.path)) == { + "targetWebsite": "https://arcade.makecode.com/v4.1.27" + } + + +def test_share_link_source_runs_mkc_download( + tmp_path: Path, mkcd2app_home: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + fake = FakeRunCmd() + monkeypatch.setattr(code, "run_cmd", fake) + url = "https://arcade.makecode.com/S12345-67890-12345-67890" + + code_path = run_fetch_code(tmp_path / "build", {"type": "share_link", "value": url}) + + assert len(fake.calls) == 1 + call = fake.calls[0] + assert call.command == ["mkc", "download", url] + assert call.cwd == code_path + assert call.which_path == get_js_tools_bin_dir() + assert read_mkc_json(code_path) == MKC_JSON + + +@pytest.mark.parametrize( + ("source", "expected_ref"), + [ + ({"type": "git", "url": "https://example.com/a/b.git", "ref": "v1"}, "v1"), + ({"type": "git", "url": "https://example.com/a/b.git"}, None), + ({"type": "github", "value": "https://github.com/a/b", "checkout": "dev"}, "dev"), + ({"type": "github", "value": "https://github.com/a/b"}, "master"), + ], +) # fmt: skip +def test_git_sources_clone( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + source: dict[str, str], + expected_ref: str | None, +) -> None: + # Cloning itself is covered by test_git.py; this checks fetch_code's dispatch + calls: list[tuple[str, Path, str | None]] = [] + + def fake_clone(url: str, path: Path, ref: str | None = None) -> None: + # The parent must exist, and the clone target must not + assert path.parent.is_dir() + assert not path.exists() + calls.append((url, path, ref)) + path.mkdir() + + monkeypatch.setattr(code, "clone_and_checkout", fake_clone) + code_path = run_fetch_code(tmp_path / "build", source) + + url = source.get("url") or source["value"] + assert calls == [(url, code_path, expected_ref)] + assert read_mkc_json(code_path) == MKC_JSON + + +def test_file_source_extracts_project(tmp_path: Path) -> None: + code_path = run_fetch_code( + tmp_path / "build", {"type": "file", "value": str(EMPTY_PNG)} + ) + assert (code_path / "pxt.json").is_file() + assert (code_path / "main.ts").is_file() + assert read_mkc_json(code_path) == MKC_JSON + + +def test_file_source_missing_file(tmp_path: Path) -> None: + with pytest.raises(FileNotFoundError, match="nope.png"): + run_fetch_code( + tmp_path / "build", {"type": "file", "value": str(tmp_path / "nope.png")} + ) + + +@pytest.mark.parametrize(("png", "warns"), [(EMPTY_PNG, False), (DUCK_PNG, True)]) +def test_file_source_warns_on_target_mismatch( + tmp_path: Path, caplog: pytest.LogCaptureFixture, png: Path, warns: bool +) -> None: + with capture_logger(caplog, code.logger): + run_fetch_code(tmp_path / "build", {"type": "file", "value": str(png)}) + warnings = [r for r in caplog.records if r.levelno == logging.WARNING] + if warns: + assert [r.getMessage() for r in warnings] == [ + f"Project was saved with Arcade 4.1.27, building against {TARGET_VERSION}" + ] + else: + assert warnings == [] + + +# ---------------------------------------------- _warn_on_target_mismatch ------ + + +@pytest.mark.parametrize( + "pxt_json", + [ + None, # file missing + "not json", + "[]", + "{}", + '{"targetVersions": []}', + '{"targetVersions": {"targetId": "arcade"}}', + f'{{"targetVersions": {{"target": "{TARGET_VERSION}"}}}}', + ], +) +def test_target_mismatch_no_warning( + tmp_path: Path, caplog: pytest.LogCaptureFixture, pxt_json: str | None +) -> None: + path = tmp_path / "pxt.json" + if pxt_json is not None: + path.write_text(pxt_json, encoding="utf-8") + with capture_logger(caplog, code.logger): + _warn_on_target_mismatch(path, TARGET_VERSION) + assert not [r for r in caplog.records if r.levelno >= logging.WARNING] + + +def test_target_mismatch_warns( + tmp_path: Path, caplog: pytest.LogCaptureFixture +) -> None: + path = tmp_path / "pxt.json" + path.write_text(json.dumps({"targetVersions": {"target": "2.0.48"}}), "utf-8") + with capture_logger(caplog, code.logger): + _warn_on_target_mismatch(path, TARGET_VERSION) + assert [(r.levelno, r.getMessage()) for r in caplog.records] == [ + ( + logging.WARNING, + f"Project was saved with Arcade 2.0.48, building against {TARGET_VERSION}", + ) + ] + + +# ------------------------------------------------------- build_binary_js ------ + + +def test_build_binary_js( + tmp_path: Path, mkcd2app_home: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + code_path = tmp_path / "build" / "test-source" + code_path.mkdir(parents=True) + (code_path / "main.ts").write_text("// game", encoding="utf-8") + before = tree_digest(code_path) + + def fake_mkc_build(call: CmdCall) -> None: + (call.cwd / "built").mkdir() + (call.cwd / "built" / "binary.js").write_text("// built", encoding="utf-8") + (call.cwd / "built" / "other.txt").write_text("x", encoding="utf-8") + + fake = FakeRunCmd(fake_mkc_build) + monkeypatch.setattr(code, "run_cmd", fake) + + result: ContentFile = build_binary_js.func( + make_config_yaml(tmp_path / "build"), ContentDir(str(code_path)) + ) + + assert [(c.command, c.cwd, c.which_path) for c in fake.calls] == [ + (["mkc", "build", "-j"], code_path, get_js_tools_bin_dir()) + ] + output = tmp_path / "build" / "test-binary-js" / "binary.js" + assert Path(result.path) == output + assert output.read_text(encoding="utf-8") == "// built" + # built/ is removed so fetch_code's ContentDir hash doesn't change + assert tree_digest(code_path) == before + + +# ---------------------------------------------------- copy_support_files ------ + + +@pytest.fixture +def sim_html(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Generator[Path]: + path = tmp_path / "target" / TARGET_VERSION / "---simulator.html" + path.parent.mkdir(parents=True) + path.write_text("sim", encoding="utf-8") + requested: list[str] = [] + + def fake_get_sim_html_path(v: str) -> Path: + requested.append(v) + return path + + monkeypatch.setattr(code, "get_sim_html_path", fake_get_sim_html_path) + yield path + assert requested == [TARGET_VERSION] + + +def make_png(path: Path, size: tuple[int, int] = (32, 32)) -> Path: + Image.new("RGBA", size, (255, 0, 0, 255)).save(path) + return path + + +def run_copy_support_files(build_dir: Path, icon: dict[str, str] | None) -> Path: + result: ContentDir = copy_support_files.func(make_config_yaml(build_dir, icon=icon)) + return Path(result.path) + + +def test_support_files_without_icon(tmp_path: Path, sim_html: Path) -> None: + support = run_copy_support_files(tmp_path / "build", None) + assert support == tmp_path / "build" / "test-binary-js-support" + assert sorted(p.name for p in support.iterdir()) == ["---simulator.html"] + assert (support / "---simulator.html").read_bytes() == sim_html.read_bytes() + + +def test_support_files_replaces_previous_output(tmp_path: Path, sim_html: Path) -> None: + stale = tmp_path / "build" / "test-binary-js-support" / "favicon.ico" + stale.parent.mkdir(parents=True) + stale.write_bytes(b"old icon") + support = run_copy_support_files(tmp_path / "build", None) + assert not (support / "favicon.ico").exists() + + +def test_support_files_icon_from_path(tmp_path: Path, sim_html: Path) -> None: + icon = make_png(tmp_path / "icon.png") + support = run_copy_support_files( + tmp_path / "build", {"type": "path", "value": str(icon)} + ) + with Image.open(support / "favicon.ico") as im: + assert im.format == "ICO" + + +class FakeResponse: + def __init__(self, content: bytes, status_code: int = 200) -> None: + self.content = content + self.status_code = status_code + + def raise_for_status(self) -> None: + if self.status_code >= 400: + raise requests.HTTPError(f"{self.status_code} error") + + +def test_support_files_icon_from_url( + tmp_path: Path, sim_html: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + png = make_png(tmp_path / "icon.png").read_bytes() + urls: list[str] = [] + + def fake_get(url: str) -> FakeResponse: + urls.append(url) + return FakeResponse(png) + + monkeypatch.setattr(requests, "get", fake_get) + support = run_copy_support_files( + tmp_path / "build", {"type": "url", "value": "https://example.com/icon.png"} + ) + assert urls == ["https://example.com/icon.png"] + with Image.open(support / "favicon.ico") as im: + assert im.format == "ICO" + + +def test_support_files_icon_url_http_error( + tmp_path: Path, sim_html: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setattr(requests, "get", lambda url: FakeResponse(b"", 404)) + with pytest.raises(requests.HTTPError): + run_copy_support_files( + tmp_path / "build", {"type": "url", "value": "https://example.com/x.png"} + ) diff --git a/tests/unit/test_filesystem.py b/tests/unit/test_filesystem.py new file mode 100644 index 0000000..21bc110 --- /dev/null +++ b/tests/unit/test_filesystem.py @@ -0,0 +1,28 @@ +import os +import stat +import sys +from pathlib import Path + +from mkcd2app.utils.filesystem import _remove_readonly, rmtree_robust + + +def make_readonly_tree(root: Path) -> Path: + (root / "objects" / "ab").mkdir(parents=True) + readonly = root / "objects" / "ab" / "cdef" + readonly.write_bytes(b"git object") + os.chmod(readonly, stat.S_IREAD) + return readonly + + +def test_removes_tree_with_readonly_files(tmp_path: Path) -> None: + root = tmp_path / ".git" + make_readonly_tree(root) + rmtree_robust(root) + assert not root.exists() + + +def test_remove_readonly_handler_clears_bit_and_retries(tmp_path: Path) -> None: + # Called directly, since only Windows refuses to unlink read-only files + readonly = make_readonly_tree(tmp_path) + _remove_readonly(os.unlink, str(readonly), sys.exc_info()) # type: ignore[arg-type] + assert not readonly.exists() diff --git a/tests/unit/test_meta_comment.py b/tests/unit/test_meta_comment.py new file mode 100644 index 0000000..d53c3b3 --- /dev/null +++ b/tests/unit/test_meta_comment.py @@ -0,0 +1,72 @@ +import json +from pathlib import Path +from typing import Any + +import pytest +from pydantic import ValidationError + +from mkcd2app.utils.text import extract_meta_comment + +# Copied from a real `mkc build -j` of the empty game on 4.0.14 +META: dict[str, Any] = { + "simUrl": "https://trg-arcade.userpxt.io/---simulator", + "cdnUrl": "https://cdn.makecode.com", + "version": "v0.0.0", + "target": "arcade", + "targetVersion": "4.0.14", +} + + +def write_binary_js(tmp_path: Path, *lines: str) -> Path: + path = tmp_path / "binary.js" + path.write_text("\n".join(lines) + "\n", encoding="utf-8") + return path + + +def test_extracts_metadata(tmp_path: Path) -> None: + path = write_binary_js( + tmp_path, "// start", f"// meta={json.dumps(META)}", "var x = 1;" + ) + metadata = extract_meta_comment(path) + assert str(metadata.simUrl) == META["simUrl"] + assert metadata.target == "arcade" + assert metadata.targetVersion == "4.0.14" + + +def test_indented_comment(tmp_path: Path) -> None: + path = write_binary_js(tmp_path, f" // meta={json.dumps(META)} ") + assert extract_meta_comment(path).targetVersion == "4.0.14" + + +def test_first_meta_comment_wins(tmp_path: Path) -> None: + other = {**META, "targetVersion": "9.9.9"} + path = write_binary_js( + tmp_path, f"// meta={json.dumps(META)}", f"// meta={json.dumps(other)}" + ) + assert extract_meta_comment(path).targetVersion == "4.0.14" + + +def test_missing_comment(tmp_path: Path) -> None: + path = write_binary_js(tmp_path, "// no metadata here", "var meta = {};") + with pytest.raises(ValueError, match="Meta comment not found"): + extract_meta_comment(path) + + +@pytest.mark.parametrize( + "meta", + [ + {**META, "unexpected": 1}, # extra="forbid" + {k: v for k, v in META.items() if k != "simUrl"}, + {**META, "cdnUrl": "not a url"}, + ], +) +def test_invalid_metadata(tmp_path: Path, meta: dict[str, Any]) -> None: + path = write_binary_js(tmp_path, f"// meta={json.dumps(meta)}") + with pytest.raises(ValidationError): + extract_meta_comment(path) + + +def test_invalid_json(tmp_path: Path) -> None: + path = write_binary_js(tmp_path, "// meta={not json") + with pytest.raises(ValidationError): + extract_meta_comment(path) diff --git a/tests/unit/test_paths.py b/tests/unit/test_paths.py new file mode 100644 index 0000000..2ff585e --- /dev/null +++ b/tests/unit/test_paths.py @@ -0,0 +1,60 @@ +from collections.abc import Callable +from importlib.metadata import version +from pathlib import Path + +import pytest + +from mkcd2app.utils import paths + +VERSION = version("mkcd2app") + +DATA_PATHS: list[tuple[Callable[[], Path], str]] = [ + (paths.get_toolchain_dir, "toolchain"), + (paths.get_js_tools_dir, "toolchain/js_tools"), + (paths.get_js_tools_bin_dir, "toolchain/js_tools/node_modules/.bin"), + (paths.get_templates_npm_cache_dir, "toolchain/templates_npm_cache"), + (paths.get_target_dir, "target"), + (lambda: paths.get_target_version_dir("4.0.14"), "target/4.0.14"), + (lambda: paths.get_sim_html_path("4.0.14"), "target/4.0.14/---simulator.html"), +] +STATE_PATHS: list[tuple[Callable[[], Path], str]] = [ + (paths.get_redun_db_for_toolchain_path, "redun_db_for_toolchain.sqlite3"), + (paths.get_redun_db_for_target_path, "redun_db_for_target.sqlite3"), +] + + +def test_version_matches_package() -> None: + assert paths.get_mkcd2app_version() == VERSION + + +def test_home_override(mkcd2app_home: Path) -> None: + assert paths.get_home_override_dir() == mkcd2app_home + assert paths.get_user_data_dir() == mkcd2app_home / "data" / VERSION + assert paths.get_user_state_dir() == mkcd2app_home / "state" / VERSION + + +@pytest.mark.parametrize(("get_path", "relative"), DATA_PATHS) +def test_data_paths_under_home( + mkcd2app_home: Path, get_path: Callable[[], Path], relative: str +) -> None: + assert get_path() == mkcd2app_home / "data" / VERSION / relative + + +@pytest.mark.parametrize(("get_path", "relative"), STATE_PATHS) +def test_state_paths_under_home( + mkcd2app_home: Path, get_path: Callable[[], Path], relative: str +) -> None: + assert get_path() == mkcd2app_home / "state" / VERSION / relative + + +@pytest.mark.parametrize("value", [None, ""]) +def test_no_override_uses_platformdirs( + monkeypatch: pytest.MonkeyPatch, value: str | None +) -> None: + if value is None: + monkeypatch.delenv(paths.HOME_ENV_VAR, raising=False) + else: + monkeypatch.setenv(paths.HOME_ENV_VAR, value) + assert paths.get_home_override_dir() is None + assert paths.get_user_data_dir() == Path(paths.DIRS.user_data_dir) / VERSION + assert paths.get_user_state_dir() == Path(paths.DIRS.user_state_dir) / VERSION diff --git a/tests/unit/test_redun_compat.py b/tests/unit/test_redun_compat.py new file mode 100644 index 0000000..9566d18 --- /dev/null +++ b/tests/unit/test_redun_compat.py @@ -0,0 +1,130 @@ +import os +from collections.abc import Callable +from pathlib import Path + +import pytest +import redun.file +from redun.file import ContentDir, Dir, File + +# Importing mkcd2app applies the patches +from mkcd2app._redun_compat import ( + _orig_get_proto, + _patched_get_proto, + assert_pinned_redun_version, +) + + +def test_pinned_redun_version() -> None: + assert_pinned_redun_version() + + +# ------------------------------------------------------------- get_proto ------ + + +def test_get_proto_is_patched() -> None: + assert redun.file.get_proto is _patched_get_proto + + +@pytest.mark.parametrize( + "url", ["C:\\Users\\me\\game", "c:\\x", "E:/build/x", "z:/"] +) # fmt: skip +def test_windows_drive_paths_are_local(url: str) -> None: + assert _orig_get_proto(url) != "local" # the bug the patch works around + assert redun.file.get_proto(url) == "local" + + +@pytest.mark.parametrize( + ("url", "proto"), + [ + ("/tmp/x", "local"), + ("relative/x", "local"), + ("", "local"), + (None, "local"), + ("s3://bucket/key", "s3"), + ("gs://bucket/key", "gs"), + ("http://example.com/x", "http"), + ("C:x", "c"), # drive-relative, not a drive path: left to urlparse + ("1:/x", "local"), # not a letter, so not patched; urlparse has no scheme + ], +) +def test_other_urls_unchanged(url: str | None, proto: str) -> None: + assert redun.file.get_proto(url) == _orig_get_proto(url) == proto + + +def test_file_with_drive_letter_uses_local_filesystem() -> None: + assert File("C:\\Users\\me\\game\\main.ts").filesystem.name == "local" + + +# --------------------------------------------------------- content hashing ---- + + +@pytest.fixture +def tree(tmp_path: Path) -> Path: + root = tmp_path / "tree" + (root / "sub").mkdir(parents=True) + (root / "a.txt").write_text("a", encoding="utf-8") + (root / "sub" / "b.txt").write_text("b", encoding="utf-8") + return root + + +def content_hash(path: Path) -> str: + # A new object each time: redun caches the hash on the instance + return ContentDir(str(path)).hash + + +def dir_hash(path: Path) -> str: + return Dir(str(path)).hash + + +def touch_all(root: Path, mtime: float) -> None: + for p in root.rglob("*"): + os.utime(p, (mtime, mtime)) + + +@pytest.mark.parametrize("hash_fn", [content_hash, dir_hash]) +def test_hash_ignores_mtime(tree: Path, hash_fn: Callable[[Path], str]) -> None: + touch_all(tree, 1_000_000_000) + before = hash_fn(tree) + touch_all(tree, 2_000_000_000) + assert hash_fn(tree) == before + + +def test_hash_is_stable(tree: Path) -> None: + assert content_hash(tree) == content_hash(tree) + + +def test_dir_hash_matches_content_dir(tree: Path) -> None: + assert dir_hash(tree) == content_hash(tree) + + +def test_hash_changes_with_content(tree: Path) -> None: + before = content_hash(tree) + (tree / "sub" / "b.txt").write_text("B", encoding="utf-8") + assert content_hash(tree) != before + + +def test_hash_changes_with_new_file(tree: Path) -> None: + before = content_hash(tree) + (tree / "c.txt").write_text("c", encoding="utf-8") + assert content_hash(tree) != before + + +def test_hash_changes_with_removed_file(tree: Path) -> None: + before = content_hash(tree) + (tree / "a.txt").unlink() + assert content_hash(tree) != before + + +def test_hash_changes_with_rename(tree: Path) -> None: + before = content_hash(tree) + (tree / "a.txt").rename(tree / "sub" / "a.txt") + assert content_hash(tree) != before + + +def test_hash_includes_path(tree: Path, tmp_path: Path) -> None: + # Same contents somewhere else is a different output dir, so a different value + copy = tmp_path / "copy" + (copy / "sub").mkdir(parents=True) + (copy / "a.txt").write_text("a", encoding="utf-8") + (copy / "sub" / "b.txt").write_text("b", encoding="utf-8") + assert content_hash(copy) != content_hash(tree) diff --git a/tests/unit/test_resources.py b/tests/unit/test_resources.py new file mode 100644 index 0000000..941ff67 --- /dev/null +++ b/tests/unit/test_resources.py @@ -0,0 +1,30 @@ +from mkcd2app.utils.resources import ( + get_resource_empty_project_path, + get_resource_js_tools_path, + get_resource_template_path, +) + + +def test_templates_dir() -> None: + with get_resource_template_path() as path: + assert (path / "vite-project").is_dir() + + +def test_vite_template() -> None: + with get_resource_template_path("vite-project") as path: + for name in ("package.json", "package-lock.json", "index.html"): + assert (path / name).is_file(), name + # fill_website_template copies binary.js and the simulator in here + assert (path / "src" / "assets").is_dir() + + +def test_js_tools() -> None: + with get_resource_js_tools_path() as path: + assert (path / "package.json").is_file() + assert (path / "package-lock.json").is_file() + + +def test_empty_project() -> None: + with get_resource_empty_project_path() as path: + assert (path / "pxt.json").is_file() + assert (path / "main.ts").is_file() diff --git a/tests/unit/test_run.py b/tests/unit/test_run.py new file mode 100644 index 0000000..2aa40cf --- /dev/null +++ b/tests/unit/test_run.py @@ -0,0 +1,61 @@ +import sys +from pathlib import Path + +import pytest + +from mkcd2app.utils.run import BuildError, run_cmd + + +def python(code: str) -> list[str]: + return [sys.executable, "-c", code] + + +def test_returns_stdout(tmp_path: Path) -> None: + assert run_cmd(python("print('hello')"), cwd=tmp_path).strip() == "hello" + + +def test_runs_in_cwd(tmp_path: Path) -> None: + out = run_cmd(python("import os; print(os.getcwd())"), cwd=tmp_path) + assert Path(out.strip()).samefile(tmp_path) + + +def test_accepts_str_cwd(tmp_path: Path) -> None: + assert run_cmd(python("print(1)"), cwd=str(tmp_path)).strip() == "1" + + +def test_failure_raises_build_error(tmp_path: Path) -> None: + command = python( + "import sys; print('to stdout'); print('to stderr', file=sys.stderr); " + "sys.exit(3)" + ) + with pytest.raises(BuildError) as exc_info: + run_cmd(command, cwd=tmp_path) + err = exc_info.value + assert err.returncode == 3 + assert err.command == command + assert err.cwd == tmp_path + assert "to stdout" in err.output + assert "to stderr" in err.output + assert "(3)" in str(err) + + +def test_missing_executable(tmp_path: Path) -> None: + with pytest.raises(FileNotFoundError, match="definitely-not-a-real-program"): + run_cmd(["definitely-not-a-real-program"], cwd=tmp_path) + + +def test_which_path_limits_search(tmp_path: Path) -> None: + # The interpreter's own name is on PATH, but not in an empty which_path dir + name = Path(sys.executable).name + empty = tmp_path / "empty" + empty.mkdir() + with pytest.raises(FileNotFoundError): + run_cmd([name, "-c", "pass"], cwd=tmp_path, which_path=empty) + + +def test_which_path_finds_executable(tmp_path: Path) -> None: + exe = Path(sys.executable) + out = run_cmd( + [exe.name, "-c", "print('found')"], cwd=tmp_path, which_path=exe.parent + ) + assert out.strip() == "found" diff --git a/tests/unit/test_website.py b/tests/unit/test_website.py new file mode 100644 index 0000000..db8a4fb --- /dev/null +++ b/tests/unit/test_website.py @@ -0,0 +1,276 @@ +import base64 +import json +import os +import shutil +from pathlib import Path + +import pytest +from helpers import CmdCall, FakeRunCmd, make_config_yaml, tree_digest +from redun import Task +from redun.file import ContentDir, ContentFile + +from mkcd2app.build_project import website +from mkcd2app.build_project.website import ( + build_website, + build_website_singlefile, + copy_website_template, + fill_website_template, + install_website_deps, +) +from mkcd2app.utils.paths import get_templates_npm_cache_dir +from mkcd2app.utils.resources import get_resource_template_path + +FAVICON_LINK = '' +BINARY_JS = "// fake binary.js\n" +SIM_HTML = "fake simulator\n" + + +@pytest.fixture +def template_dir(tmp_path: Path) -> Path: + """A copy of the real bundled Vite template, so template drift gets caught.""" + dst = tmp_path / "template" + with get_resource_template_path("vite-project") as src: + shutil.copytree(src, dst, ignore=shutil.ignore_patterns("node_modules", "dist")) + return dst + + +@pytest.fixture +def website_dir(tmp_path: Path, template_dir: Path) -> Path: + """The template as copy_website_template leaves it in the build dir.""" + dst = tmp_path / "build" / "test-website" + shutil.copytree(template_dir, dst) + return dst + + +def make_support_dir(tmp_path: Path, favicon: bytes | None = None) -> Path: + support = tmp_path / "build" / "test-binary-js-support" + support.mkdir(parents=True) + (support / "---simulator.html").write_text(SIM_HTML, encoding="utf-8") + if favicon is not None: + (support / "favicon.ico").write_bytes(favicon) + return support + + +def fill(tmp_path: Path, website_dir: Path, favicon: bytes | None = None) -> Path: + bin_js = tmp_path / "build" / "test-binary-js" / "binary.js" + bin_js.parent.mkdir(parents=True) + bin_js.write_text(BINARY_JS, encoding="utf-8") + result: ContentDir = fill_website_template.func( + make_config_yaml(tmp_path / "build"), + ContentDir(str(website_dir)), + ContentFile(str(bin_js)), + ContentDir(str(make_support_dir(tmp_path, favicon))), + ) + return Path(result.path) + + +# ------------------------------------------------- copy_website_template ------ + + +def test_copy_template_to_build_dir(tmp_path: Path, template_dir: Path) -> None: + build_dir = tmp_path / "build" + result: ContentDir = copy_website_template.func( + make_config_yaml(build_dir), ContentDir(str(template_dir)) + ) + assert Path(result.path) == build_dir / "test-website" + assert tree_digest(Path(result.path)) == tree_digest(template_dir) + + +def test_copy_template_replaces_previous_copy( + tmp_path: Path, template_dir: Path +) -> None: + stale = tmp_path / "build" / "test-website" / "stale.txt" + stale.parent.mkdir(parents=True) + stale.write_text("old", encoding="utf-8") + copy_website_template.func( + make_config_yaml(tmp_path / "build"), ContentDir(str(template_dir)) + ) + assert not stale.exists() + + +# ------------------------------------------------- fill_website_template ------ + + +def test_fill_writes_to_sibling_dir(tmp_path: Path, website_dir: Path) -> None: + assert fill(tmp_path, website_dir) == tmp_path / "build" / "test-website-filled" + + +def test_fill_copies_binary_and_simulator(tmp_path: Path, website_dir: Path) -> None: + assets = fill(tmp_path, website_dir) / "src" / "assets" + assert (assets / "binary.js").read_text(encoding="utf-8") == BINARY_JS + assert (assets / "---simulator.html").read_text(encoding="utf-8") == SIM_HTML + + +def test_fill_sets_default_title(tmp_path: Path, website_dir: Path) -> None: + index_html = (fill(tmp_path, website_dir) / "index.html").read_text("utf-8") + assert "