From f5ea3b8e2b2f48e014cd985c3a20412bbdce0956 Mon Sep 17 00:00:00 2001 From: Dmitry Zhuravlev Date: Wed, 21 Oct 2026 09:06:09 -0400 Subject: [PATCH 1/3] feat(smoke): add bounded opt-in Compose runtime checks --- README.md | 6 +- docs/compose-rendering.md | 2 +- docs/release-contract.md | 2 +- docs/roadmap.md | 2 +- docs/runtime-smoke.md | 72 +++++ pyproject.toml | 1 + scripts/check_package.py | 6 + src/container_platform/smoke.py | 311 ++++++++++++++++++++++ src/container_platform/smoke_transport.py | 166 ++++++++++++ tests/test_smoke.py | 303 +++++++++++++++++++++ 10 files changed, 867 insertions(+), 4 deletions(-) create mode 100644 docs/runtime-smoke.md create mode 100644 src/container_platform/smoke.py create mode 100644 src/container_platform/smoke_transport.py create mode 100644 tests/test_smoke.py diff --git a/README.md b/README.md index 3885522..bb22d43 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ Container Platform defines the contract for an **Online Shop** release composed The project does not build application code or operate a Kubernetes cluster. Its target workflow turns images published to Docker Hub into a versioned release bundle pinned by digests for review, validation and promotion to the SRE Platform. -> **Status:** Release contract v1 is defined; a digest-pinned baseline candidate and profile inventory are available for review. Offline and opt-in registry metadata checks, PR checks and offline Compose rendering are available; required-check rules are owner-configured. Runtime integration and SRE handoff remain planned. +> **Status:** Release contract v1 is defined; a digest-pinned baseline candidate and profile inventory are available for review. Offline and opt-in registry metadata checks, PR checks, Compose rendering and an opt-in runtime runner are available; required-check rules are owner-configured. Runtime evidence remains subject to recorded live results; SRE handoff remains planned. ## Why this platform exists @@ -120,3 +120,7 @@ Add `--registry` to inspect immutable manifest/config bytes and declared platfor ## Compose configuration [Offline Compose rendering](docs/compose-rendering.md) generates all 12 digest-pinned services from a candidate and the packaged AML profile. Legacy baseline rendering requires explicit `--inspection-only`; its validation remains negative. Static `docker compose config` checks do not establish runtime compatibility or release approval. + +## Optional runtime smoke + +[Bounded runtime smoke](docs/runtime-smoke.md) requires explicit `--execute`, consumes the rendered candidate snapshot and cleans up its unique Compose project. It checks service process state and a synthetic catalog/cart/checkout flow. Inspection-only baseline execution preserves legacy ineligibility. See the recorded live outcome and limitations; runtime success is separate from release approval and deployment authorization. diff --git a/docs/compose-rendering.md b/docs/compose-rendering.md index 86b1fb6..36b186d 100644 --- a/docs/compose-rendering.md +++ b/docs/compose-rendering.md @@ -36,7 +36,7 @@ Compose bytes go only to `--output`; diagnostics/report go to stdout. Invalid in Offline tests cover deterministic bytes, exact mapping and candidate digests, service addresses, loopback exposure, safe model fields, invalid inputs, inspection restrictions and output protection. Package checks exercise both rendering interfaces and profile/schema resources from a clean installed wheel outside checkout. -The actual baseline static Compose check result is recorded below. No image pull, container execution, build or publication is part of these checks. Registry availability remains a separate opt-in validator operation; runtime smoke tests, resource sizing, shutdown/cleanup, application compatibility and deployment authorization remain future work. +The actual baseline static Compose check result is recorded below. No image pull, container execution, build or publication is part of these checks. Registry availability remains a separate opt-in validator operation. The [separate runtime runner](runtime-smoke.md) requires explicit execution opt-in and records its own result; static evidence does not establish application compatibility or deployment authorization. ### Static observation: 2026-10-20 diff --git a/docs/release-contract.md b/docs/release-contract.md index 729eebe..a924506 100644 --- a/docs/release-contract.md +++ b/docs/release-contract.md @@ -50,7 +50,7 @@ The pull reference is `repository@digest`. Tags are not accepted selectors. For 1. **Structure:** parse strictly and validate against the pinned v1 schema. This establishes field shape only. 2. **Offline release policy and opt-in registry metadata (implemented):** resolve the pinned local inventory, validate its structure and exact membership/allowlists, identity uniqueness, declared platforms and traceability eligibility. Reject examples and unresolved references. The CLI does not establish inventory approval. With explicit registry opt-in, manifest/config availability, bytes and kind/platform metadata are checked after shared prerequisites pass, independently of traceability eligibility. Layer availability is not checked. -3. **Configuration and integration:** [offline Compose rendering](compose-rendering.md) generates digest-pinned configuration and supports static Compose checks. Explicit legacy inspection preserves negative release validation. Runtime integration remains planned; it must record evidence tied to the exact candidate revision and image set. Structure, static configuration and registry success do not establish service compatibility. +3. **Configuration and integration:** [offline Compose rendering](compose-rendering.md) generates digest-pinned configuration and supports static Compose checks. The [opt-in runtime runner](runtime-smoke.md) records bounded service/HTTP evidence for the exact candidate bytes and image set. Explicit legacy inspection preserves negative release validation. Runner availability is distinct from a successful live observation. Structure, static configuration and registry success do not establish service compatibility. 4. **Container-release review:** maintainers review composition, policy results and required integration evidence. Approval is external review evidence tied to the exact manifest/revision, not a self-asserted manifest flag. Only a release PR reviewed by authorized maintainers with all documented mandatory checks passing may use merge as the approval event. Git merge itself performs no validation. Until those checks and review gates are implemented, merging this contract or an example approves no deployable release. 5. **SRE promotion (planned):** SRE separately authorizes suitability for a specific environment through a reviewed GitOps change. It must record the full immutable Container Platform commit SHA and manifest path, and consume exactly that image set. A moving branch/tag is insufficient. Checking rendered SRE values against the manifest, making that check mandatory and protecting merges remain future implementation; bypasses are not blocked today. 6. **Deployment and runtime:** Argo CD applies environment configuration; Kubernetes pulls from Docker Hub. SRE evaluates rollout health, observability, data/state constraints and operational acceptance. diff --git a/docs/roadmap.md b/docs/roadmap.md index 9108b04..192824a 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -87,7 +87,7 @@ Invalid, incomplete, mutable, or unavailable releases cannot be presented as val **Goal:** Make release validation repeatable and prove that the published images compose into an application. -[Pull request checks](pull-request-validation.md) provide tests/package and new-candidate validation. [Offline Compose rendering](compose-rendering.md) provides deterministic configuration from digest-pinned candidates. Required-check settings are owner-configured; runtime integration remains unimplemented, so this milestone is not complete. +[Pull request checks](pull-request-validation.md) provide tests/package and new-candidate validation. [Offline Compose rendering](compose-rendering.md) provides deterministic configuration from digest-pinned candidates. An [opt-in bounded runtime runner](runtime-smoke.md) is available; its live result and limitations are recorded separately. Required-check settings are owner-configured; runner availability does not complete this milestone or prove baseline runtime compatibility. ### Scope diff --git a/docs/runtime-smoke.md b/docs/runtime-smoke.md new file mode 100644 index 0000000..e2ff6d9 --- /dev/null +++ b/docs/runtime-smoke.md @@ -0,0 +1,72 @@ +# Opt-in Online Shop runtime smoke + +`container-smoke` / `python -m container_platform.smoke` runs one temporary Compose project. This is integration evidence for a selected image set, not a long-lived environment or SRE deployment. The existing validator and renderer never start Docker. + +## Prerequisites and execution + +Install the package with Python 3.11+ and Git. Supply local inventory Git objects, a local Docker Engine with native Linux/amd64 runtime, and Docker Compose v2 supporting JSON config/ps and pull policy options. Only local Unix socket/named pipe transports are supported; SSH/TCP remote contexts are rejected before mutations. The runner does not install, start or reconfigure system tools. Loopback port 8080 must be free; a conflict never stops another project. A race after the port check is reported as startup failure. + +```text +python -m container_platform.smoke candidate.json --repository . --profile aml-online-shop-compose-v1 --execute +container-smoke releases/baseline-2026-10-18.json --repository . --profile aml-online-shop-compose-v1 --execute --inspection-only +``` + +Without `--execute`, no candidate read or Docker call occurs (exit 2). Ordinary execution requires successful offline validation. Inspection-only permits exclusively the renderer's existing legacy traceability failures; all other prerequisites must pass. It preserves the full negative offline result and grants no eligibility exception. Baseline eligibility remains negative even if runtime succeeds. + +The input is capped at 1 MiB and read once into a private snapshot. The existing renderer validates that snapshot and generates the only accepted Compose file. Report SHA-256 identifies its exact bytes. No external Compose file is accepted. All 12 required services use candidate `repository@sha256:...` references and `linux/amd64`. Pull downloads these images; up prohibits builds and further pulls. Images remain cached after the test. + +Each execution generates a unique `cp-smoke-` project and checks its resource labels before creating resources. Commands use an explicit Compose file, project directory, empty env file and project name. Inherited `COMPOSE_*` selectors and `DOCKER_DEFAULT_PLATFORM` are removed. Docker client context/transport configuration remains the operator's responsibility; no credentials or environment are emitted in reports. Frontend-only loopback exposure, ephemeral Redis storage and the AML-derived [profile](compose-rendering.md) are preserved. No foreign networks, named volumes, host binds, Docker socket, privileged containers or auxiliary images are requested. + +## Budgets and checks + +All retries share monotonic deadlines; they never restart a stage budget. + +| Operation | Limit | +|---|---:| +| Overall coordination | 1,200 seconds | +| Prerequisites | 30 seconds | +| Pull | 600 seconds | +| Startup | 120 seconds | +| Readiness | 180 seconds | +| Entire semantic smoke sequence | 60 seconds | +| Each HTTP request, including redirects/body | 10 seconds | +| Diagnostics | 20 seconds | +| Cleanup, including resource verification | 120 seconds | + +Work stops before the overall deadline to reserve diagnostics and cleanup time. Child process termination/reaping can add at most ten seconds per timed-out process; one HTTP socket read can add up to one second. These are bounded termination grace, not retry budgets. Process output is capped at 1 MiB per command; HTTP responses at 2 MiB. Raw process output, response bodies and cookies are not included in the report. Diagnostics collect only sanitized service states; no application logs or payment/session payloads are published. + +Readiness requires exactly 12 expected service records, the exact image references, every service running and a frontend catalog response. This establishes process state plus frontend availability, not independent health evidence for every backend. Host-side semantic assertions then exercise catalog browsing, a selected product, empty/add/remove/re-add cart operations and synthetic checkout. A final service-state check detects an exit during the sequence. Loadgenerator traffic is independent and cannot satisfy these assertions. + +Routes/forms and assertions derive from immutable AML revision [`9960484b5c7f1568688fa6fc8eb256976d6c18a4`](https://github.com/DimitryZH/application-modernization-lab/tree/9960484b5c7f1568688fa6fc8eb256976d6c18a4/workloads/online-shop/src): frontend routes/handlers, product/cart/order templates and loadgenerator. Payment service performs a pretend charge; email starts in dummy mode in this profile. The runner uses a standard test card number, reserved `.invalid` email and synthetic address, without payment/mail credentials. It checks selected product ID, cart quantity, order confirmation/tracking and post-checkout cart clearance. It does not fetch browser assets, execute scripts or follow redirects outside `http://127.0.0.1:8080`. This is a narrow demo flow, not verification of real payment/email systems or image provenance. + +## Report and exit codes + +Stdout contains a separate versioned JSON report: release ID and exact candidate-byte SHA-256, inventory reference, Compose profile, exact image set, inspection mode, original offline validation, check/reason codes, sanitized service states, smoke assertions, runtime outcome, cleanup outcome and overall exit code. Input or resource failures may leave unavailable identity fields null. No output paths, environment dump or raw Docker errors are included. + +| Exit | Meaning for this command | +|---|---| +| 0 | Runtime flow and verified cleanup succeeded | +| 1 | Release validation or application/semantic failure | +| 2 | Missing opt-in, prerequisite/execution error, or cleanup failure after runtime success | +| 3 | Deadline/request timeout | +| 130 | Handled interruption | + +These codes do not change validator/render codes. A report can have runtime success and exit 2 when cleanup fails. A cleanup failure never overwrites the original runtime failure; inspect both fields. No outcome grants release approval or deployment authorization. + +## Cleanup and interruptions + +After an attempted startup, a finally block runs project-scoped `down --volumes --remove-orphans --timeout 10` and checks containers, networks and volumes by the unique project label. No global prune or image deletion is performed. Before startup, cleanup is `not_required`: only read operations and possibly an image pull have occurred. Failed cleanup is explicit and cannot produce overall success. Ctrl+C and SIGTERM are handled; a second interruption can prevent completion and is reported where possible. + +Power loss, forced process termination or an unavailable Engine can prevent cleanup. Each report includes `project` and an exact `manualCleanupPowerShell` command using only that project's label, independent of temporary files. Preserve the project identifier if terminating externally. If no final report was emitted, locate only labels starting with `cp-smoke-`, identify the interrupted run, and substitute its exact identifier below; never substitute another project or omit the filter: + +```powershell +$smokeProject = 'cp-smoke-' +$smokeLabel = "label=com.docker.compose.project=$smokeProject" +docker ps -aq --filter $smokeLabel | ForEach-Object { docker rm -f -v $_ } +docker network ls -q --filter $smokeLabel | ForEach-Object { docker network rm $_ } +docker volume ls -q --filter $smokeLabel | ForEach-Object { docker volume rm $_ } +``` + +## Validation evidence + +Automated tests use mock Docker/HTTP adapters and isolated local Git fixtures; the ordinary suite requires no Docker or registry requests. Installed-wheel checks exercise both smoke interfaces outside checkout without runtime opt-in. No mandatory runtime PR job is added. Live results are recorded separately below; runtime integration cannot be claimed from mocks or static configuration alone. diff --git a/pyproject.toml b/pyproject.toml index 33ea7fc..3534b89 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -12,6 +12,7 @@ dependencies = ["jsonschema>=4.23,<5", "referencing>=0.35,<1"] [project.scripts] container-release = "container_platform.cli:main" container-compose = "container_platform.compose:main" +container-smoke = "container_platform.smoke:main" [tool.setuptools] packages = ["container_platform", "container_platform.schemas", "container_platform.profiles"] diff --git a/scripts/check_package.py b/scripts/check_package.py index 84cf457..804e7e7 100644 --- a/scripts/check_package.py +++ b/scripts/check_package.py @@ -65,6 +65,12 @@ def check(repository, wheel_directory): artifacts.append(destination.read_bytes()) if artifacts[0] != artifacts[1]: raise RuntimeError("Installed rendering interfaces differ.") + smoke_console = str(binaries / ("container-smoke.exe" if os.name == "nt" else "container-smoke")) + for command in ([python, "-m", "container_platform.smoke"], [smoke_console]): + report = json.loads(run([*command, str(repo / "releases/baseline-2026-10-18.json"), + "--repository", str(repo), "--profile", profile], outside, env, expected=2)) + if report["diagnostics"] != ["runtime_opt_in_required"] or report["runtimeOutcome"] != "not_started": + raise RuntimeError("Installed smoke interface must require explicit runtime opt-in.") def main(): diff --git a/src/container_platform/smoke.py b/src/container_platform/smoke.py new file mode 100644 index 0000000..93949c4 --- /dev/null +++ b/src/container_platform/smoke.py @@ -0,0 +1,311 @@ +"""Explicit, isolated and bounded Online Shop runtime smoke validation.""" + +from datetime import datetime, timezone +import hashlib +from html.parser import HTMLParser +import json +from pathlib import Path +import re +import signal +import socket +import tempfile +import time +import uuid + +from .cli import Parser +from .compose import IDS, PROFILE, render +from .smoke_transport import Docker, Session, SmokeError, remaining +from .validation import Failure, strict_json + + +LIMITS = {"total": 1200, "prerequisites": 30, "pull": 600, "startup": 120, + "readiness": 180, "smoke": 60, "diagnostics": 20, "cleanup": 120} +EXIT = {"success": 0, "validation_failed": 1, "application_failure": 1, + "execution_error": 2, "timeout": 3, "interrupted": 130} +PRODUCT = "0PUK6V6EV0" + + +def checked(result, reason): + code, output = result + if code: + raise SmokeError(reason) + return output + + +def decoded(data): + try: + return strict_json(data) + except Failure: + raise SmokeError("docker_output_invalid") from None + + +def resources(docker, deadline): + label = "label=com.docker.compose.project=" + docker.project + return {kind: checked(docker.raw(args + ["--filter", label, "--format", "{{.ID}}" if kind != "volumes" else "{{.Name}}"], deadline), + "project_resource_query_failed").strip() + for kind, args in (("containers", ["ps", "-a"]), ("networks", ["network", "ls"]), ("volumes", ["volume", "ls"]))} + + +def port_available(): + try: + with socket.socket() as connection: + connection.bind(("127.0.0.1", 8080)) + except OSError: + raise SmokeError("frontend_port_unavailable") from None + + +def prerequisites(docker, images, deadline): + server = decoded(checked(docker.raw(["version", "--format", "{{json .Server}}"], deadline), "docker_engine_unavailable")) + if not isinstance(server, dict) or server.get("Os") != "linux" or server.get("Arch") not in ("amd64", "x86_64"): + raise SmokeError("runtime_platform_unsupported") + endpoint = decoded(checked(docker.raw(["context", "inspect", "--format", "{{json .Endpoints.docker.Host}}"], deadline), "docker_context_unavailable")) + endpoints = [endpoint, getattr(docker, "env", {}).get("DOCKER_HOST", endpoint)] + if any(not isinstance(value, str) or not value.startswith(("unix:///", "npipe://")) for value in endpoints): + raise SmokeError("remote_docker_transport_unsupported") + checked(docker.raw(["compose", "version", "--short"], deadline), "compose_unavailable") + if any(resources(docker, deadline).values()): + raise SmokeError("project_name_collision") + config = decoded(checked(docker.compose(["config", "--format", "json"], deadline), "compose_config_failed")) + services = config.get("services", {}) if isinstance(config, dict) else {} + if not isinstance(services, dict) or set(services) != IDS: + raise SmokeError("runtime_model_mismatch") + for name, service in services.items(): + if (service.get("image") != images[name] or service.get("platform") != "linux/amd64" + or set(service) & {"build", "privileged", "network_mode", "container_name", "volumes", "env_file", "extends"} + or (name != "frontend" and service.get("ports"))): + raise SmokeError("runtime_model_mismatch") + ports = services["frontend"].get("ports", []) + if len(ports) != 1 or ports[0].get("host_ip") != "127.0.0.1" or str(ports[0].get("published")) != "8080" or ports[0].get("target") != 8080: + raise SmokeError("runtime_model_mismatch") + port_available() + + +def service_state(docker, images, deadline): + output = checked(docker.compose(["ps", "--all", "--format", "json"], deadline), "service_query_failed") + # Compose versions emit either one JSON array or one JSON object per line. + rows = decoded(output) if output.lstrip().startswith(b"[") else [decoded(line) for line in output.splitlines() if line.strip()] + if not isinstance(rows, list) or any(not isinstance(row, dict) for row in rows): + raise SmokeError("docker_output_invalid") + names = [r.get("Service") for r in rows] + if len(names) != len(set(names)) or set(names) - IDS: + raise SmokeError("unexpected_project_services", "application_failure") + states = [] + for row in rows: + name, state = row.get("Service"), row.get("State") + if row.get("Image") != images[name]: + raise SmokeError("running_image_mismatch", "application_failure") + if state in ("exited", "dead", "removing"): + raise SmokeError("service_not_running", "application_failure") + states.append({"service": name, "state": state if state in ("running", "created", "restarting", "paused") else "unknown"}) + return states, len(rows) == 12 and all(r.get("State") == "running" for r in rows) + + +class Page(HTMLParser): + def __init__(self, body): + super().__init__() + self.inputs, self.links, self.text = {}, [], [] + self.feed(body) + + def handle_starttag(self, tag, attrs): + attrs = dict(attrs) + if tag == "input" and "name" in attrs: + self.inputs[attrs["name"]] = attrs.get("value", "") + if tag == "a": + self.links.append(attrs.get("href", "")) + + def handle_data(self, data): + self.text.append(data) + + @property + def words(self): + return " ".join(" ".join(self.text).split()) + + +def require(condition, code): + if not condition: + raise SmokeError(code, "application_failure") + + +def smoke_flow(session, deadline, results): + def get(path, form=None): + return Page(session.request(path, deadline, form)) + + home = get("/") + require("/product/" + PRODUCT in home.links, "frontend_catalog_missing") + results.append("frontend_catalog_pass") + product = get("/product/" + PRODUCT) + require(product.inputs.get("product_id") == PRODUCT, "product_selection_failed") + results.append("product_browse_pass") + get("/cart/empty", {}) + require("Your shopping cart is empty!" in get("/cart").words, "cart_empty_failed") + get("/cart", {"product_id": PRODUCT, "quantity": "2"}) + cart = get("/cart") + require("SKU #" + PRODUCT in cart.words and "Quantity: 2" in cart.words and "Cart (2)" in cart.words, + "cart_add_semantics_failed") + results.append("cart_add_pass") + # Exercise removal and re-add in this cookie session before checkout. + get("/cart/empty", {}) + require("Your shopping cart is empty!" in get("/cart").words, "cart_remove_failed") + results.append("cart_remove_pass") + get("/cart", {"product_id": PRODUCT, "quantity": "1"}) + cart = get("/cart") + require("SKU #" + PRODUCT in cart.words and "Quantity: 1" in cart.words, "checkout_cart_missing") + order = get("/cart/checkout", { + "email": "smoke@example.invalid", "street_address": "123 Test Street", "zip_code": "12345", + "city": "Test City", "state": "CA", "country": "United States", + "credit_card_number": "4111111111111111", "credit_card_expiration_month": "12", + "credit_card_expiration_year": str(datetime.now(timezone.utc).year + 1), "credit_card_cvv": "123", + }) + require("Your order is complete!" in order.words and re.search(r"Confirmation # [a-zA-Z0-9-]{8,}", order.words) + and re.search(r"Tracking # [a-zA-Z0-9-]{4,}", order.words), "checkout_confirmation_missing") + require("Your shopping cart is empty!" in get("/cart").words, "checkout_cart_not_cleared") + results.append("synthetic_checkout_pass") + + +def run(candidate_path, repository, profile, *, execute=False, inspection_only=False, + docker_factory=Docker, session_factory=Session, clock=time.monotonic, sleep=time.sleep): + start = clock() + total_deadline = start + LIMITS["total"] + work_deadline = total_deadline - LIMITS["cleanup"] - LIMITS["diagnostics"] + report = {"smokeReportVersion": 1, "runtimeOutcome": "not_started", "exitCode": 2, + "inspectionOnly": inspection_only, "profile": profile if profile == PROFILE else None, "candidate": None, + "validation": None, "checks": [], "smokeAssertions": [], "services": [], + "diagnostics": [], "cleanup": {"outcome": "not_required"}, "limitsSeconds": dict(LIMITS), + "scope": "Temporary runtime smoke only; release eligibility is separate. No approval or deployment authorization."} + if not execute: + report["diagnostics"] = ["runtime_opt_in_required"] + return report + project = "cp-smoke-" + uuid.uuid4().hex + report["project"] = project + label = "label=com.docker.compose.project=" + project + report["manualCleanupPowerShell"] = ( + "docker ps -aq --filter " + label + " | ForEach-Object { docker rm -f -v $_ }; " + "docker network ls -q --filter " + label + " | ForEach-Object { docker network rm $_ }; " + "docker volume ls -q --filter " + label + " | ForEach-Object { docker volume rm $_ }") + mutated, docker = False, None + active = None + + def stage(name, operation): + nonlocal active + active = {"id": name, "status": "running"} + report["checks"].append(active) + deadline = min(work_deadline, clock() + LIMITS[name]) + operation(deadline) + remaining(deadline, clock) + active.update(status="pass") + + with tempfile.TemporaryDirectory(prefix="cp-smoke-") as temporary: + directory = Path(temporary) + try: + with Path(candidate_path).open("rb") as stream: + data = stream.read(1024 * 1024 + 1) + if len(data) > 1024 * 1024: + raise SmokeError("candidate_size_limit", "validation_failed") + report["candidate"] = {"releaseId": None, "sha256": hashlib.sha256(data).hexdigest(), + "inventory": None, "images": {}} + snapshot = directory / "candidate.json" + snapshot.write_bytes(data) + rendered = render(snapshot, repository, profile, directory / "compose.json", inspection_only=inspection_only) + report["validation"] = rendered["validation"] + if rendered["validation"] and all(c["status"] == "pass" for c in rendered["validation"]["checks"][:2]): + metadata = strict_json(data) + report["candidate"].update(releaseId=metadata["releaseId"], inventory=metadata["inventory"], + images=dict(sorted((c["id"], c["image"]["repository"] + "@" + c["image"]["digest"]) for c in metadata["components"]))) + if rendered["exitCode"]: + report["checks"].append({"id": "render", "status": "fail", "reason": "release_render_failed"}) + report["diagnostics"] = rendered["diagnostics"] + report.update(runtimeOutcome="validation_failed" if rendered["exitCode"] == 1 else "execution_error", + exitCode=rendered["exitCode"]) + return report + candidate = strict_json(data) + images = {c["id"]: c["image"]["repository"] + "@" + c["image"]["digest"] for c in candidate["components"]} + report["candidate"] = {"releaseId": candidate["releaseId"], "sha256": hashlib.sha256(data).hexdigest(), + "inventory": candidate["inventory"], "images": dict(sorted(images.items()))} + report["checks"].append({"id": "render", "status": "pass"}) + (directory / "empty.env").write_bytes(b"") + docker = docker_factory(project, directory, clock) + stage("prerequisites", lambda deadline: prerequisites(docker, images, deadline)) + stage("pull", lambda deadline: checked(docker.compose(["pull", "--policy", "always"], deadline), "image_pull_failed")) + mutated = True # A failed/interrupted up can already have created resources. + stage("startup", lambda deadline: checked(docker.compose(["up", "--detach", "--no-build", "--pull", "never"], deadline), "compose_startup_failed")) + session = session_factory(clock) + + def ready(deadline): + while True: + states, running = service_state(docker, images, deadline) + report["services"] = states + if running: + try: + body = session.request("/", deadline) + if "/product/" + PRODUCT in Page(body).links: + return + except SmokeError as failure: + if failure.outcome == "execution_error": + raise + sleep(min(2, remaining(deadline, clock))) + + stage("readiness", ready) + stage("smoke", lambda deadline: smoke_flow(session, deadline, report["smokeAssertions"])) + states, running = service_state(docker, images, min(work_deadline, clock() + 10)) + report["services"] = states + require(running, "services_not_running_after_smoke") + report.update(runtimeOutcome="success", exitCode=0) + except KeyboardInterrupt: + report.update(runtimeOutcome="interrupted", exitCode=130) + report["diagnostics"].append("runtime_interrupted") + except SmokeError as failure: + report.update(runtimeOutcome=failure.outcome, exitCode=EXIT[failure.outcome]) + report["diagnostics"].append(failure.code) + except (OSError, Failure): + report.update(runtimeOutcome="execution_error", exitCode=2) + report["diagnostics"].append("runtime_input_or_io_failed") + finally: + if active and active["status"] == "running": + active.update(status="fail", reason=report["diagnostics"][-1] if report["diagnostics"] else "runtime_failed") + if mutated: + # Preserve primary outcome; diagnostics/cleanup never erase it. + try: + deadline = min(total_deadline - LIMITS["cleanup"], clock() + LIMITS["diagnostics"]) + states, _ = service_state(docker, report["candidate"]["images"], deadline) + report["services"] = states + except (SmokeError, KeyboardInterrupt, OSError): + report["diagnostics"].append("diagnostics_unavailable") + try: + deadline = min(total_deadline, clock() + LIMITS["cleanup"]) + checked(docker.compose(["down", "--volumes", "--remove-orphans", "--timeout", "10"], deadline), "cleanup_command_failed") + if any(resources(docker, deadline).values()): + raise SmokeError("cleanup_resources_remaining") + report["cleanup"] = {"outcome": "success"} + except (SmokeError, KeyboardInterrupt, OSError) as failure: + report["cleanup"] = {"outcome": "failed", "reason": failure.code if isinstance(failure, SmokeError) else "cleanup_interrupted_or_io_failed"} + if report["exitCode"] == 0: + report["exitCode"] = 2 + report["elapsedSeconds"] = round(clock() - start, 3) + return report + + +def main(argv=None): + parser = Parser(description=__doc__, allow_abbrev=False) + parser.add_argument("candidate") + parser.add_argument("--repository", required=True) + parser.add_argument("--profile", required=True) + parser.add_argument("--execute", action="store_true") + parser.add_argument("--inspection-only", action="store_true") + previous = signal.getsignal(signal.SIGTERM) + def interrupted(*_): + raise KeyboardInterrupt() + + try: + signal.signal(signal.SIGTERM, interrupted) + args = parser.parse_args(argv) + report = run(args.candidate, args.repository, args.profile, execute=args.execute, inspection_only=args.inspection_only) + except Failure: + report = {"smokeReportVersion": 1, "runtimeOutcome": "execution_error", "exitCode": 2, "diagnostics": ["usage_error"]} + finally: + signal.signal(signal.SIGTERM, previous) + print(json.dumps(report, indent=2, sort_keys=True)) + return report["exitCode"] + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/container_platform/smoke_transport.py b/src/container_platform/smoke_transport.py new file mode 100644 index 0000000..a91fc00 --- /dev/null +++ b/src/container_platform/smoke_transport.py @@ -0,0 +1,166 @@ +"""Bounded process and loopback HTTP adapters for explicit runtime checks.""" + +import http.cookiejar +import os +import signal +import subprocess +import threading +import time +import urllib.error +import urllib.parse +import urllib.request + + +OUTPUT_LIMIT = 1024 * 1024 +HTTP_LIMIT = 2 * 1024 * 1024 + + +class SmokeError(Exception): + def __init__(self, code, outcome="execution_error"): + super().__init__(code) + self.code, self.outcome = code, outcome + + +def remaining(deadline, clock=time.monotonic): + value = deadline - clock() + if value <= 0: + raise SmokeError("deadline_exceeded", "timeout") + return value + + +def stop_process(process): + if os.name == "nt": + subprocess.run(["taskkill", "/PID", str(process.pid), "/T", "/F"], + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=5, check=False) + else: + try: + os.killpg(process.pid, signal.SIGKILL) + except ProcessLookupError: + pass + process.kill() if process.poll() is None else None + process.wait(timeout=5) + + +def process_output(args, cwd, env, deadline, clock=time.monotonic): + """Drain into capped memory and stop the child process group on every abort.""" + remaining(deadline, clock) + try: + process = subprocess.Popen(args, cwd=cwd, env=env, stdin=subprocess.DEVNULL, + stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, shell=False, bufsize=0, + start_new_session=os.name != "nt", + creationflags=subprocess.CREATE_NEW_PROCESS_GROUP if os.name == "nt" else 0) + except OSError: + raise SmokeError("docker_executable_unavailable") from None + output = bytearray() + overflow = threading.Event() + + def drain(): + while True: + try: + chunk = process.stdout.read(8192) + except (OSError, ValueError): + return + if not chunk: + return + if len(output) + len(chunk) > OUTPUT_LIMIT: + overflow.set() + return + output.extend(chunk) + + reader = threading.Thread(target=drain, daemon=True) + reader.start() + try: + while process.poll() is None: + if overflow.is_set(): + raise SmokeError("subprocess_output_limit") + time.sleep(min(0.05, remaining(deadline, clock))) + reader.join(timeout=min(1, remaining(deadline, clock))) + if reader.is_alive() or overflow.is_set(): + raise SmokeError("subprocess_output_limit") + return process.returncode, bytes(output) + except BaseException: + try: + stop_process(process) + except (OSError, subprocess.TimeoutExpired): + pass # Preserve the original error; project cleanup is still attempted. + raise + finally: + process.stdout.close() + + +class Docker: + def __init__(self, project, directory, clock=time.monotonic): + self.project, self.directory, self.clock = project, directory, clock + self.env = {k: v for k, v in os.environ.items() if not k.upper().startswith("COMPOSE_")} + self.env.pop("DOCKER_DEFAULT_PLATFORM", None) + self.env["COMPOSE_DISABLE_ENV_FILE"] = "1" + + def raw(self, args, deadline): + return process_output(["docker", *args], self.directory, self.env, deadline, self.clock) + + def compose(self, args, deadline): + return self.raw(["compose", "--ansi", "never", "--project-name", self.project, + "--project-directory", str(self.directory), "--env-file", "empty.env", + "-f", "compose.json", *args], deadline) + + +class NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self, req, fp, code, msg, headers, newurl): + return None + + +class Session: + """One private cookie jar; no proxies, external redirects, assets or scripts.""" + origin = "http://127.0.0.1:8080" + + def __init__(self, clock=time.monotonic): + self.clock = clock + self.opener = urllib.request.build_opener(urllib.request.ProxyHandler({}), NoRedirect(), + urllib.request.HTTPCookieProcessor(http.cookiejar.CookieJar())) + + def request(self, path, deadline, form=None): + # Per-request deadline spans redirects and response reads, not just connect. + deadline = min(deadline, self.clock() + 10) + method = "POST" if form is not None else "GET" + data = urllib.parse.urlencode(form).encode() if form is not None else None + url = urllib.parse.urljoin(self.origin, path) + for _ in range(4): + if urllib.parse.urlsplit(url)[:2] != ("http", "127.0.0.1:8080"): + raise SmokeError("http_redirect_outside_frontend", "application_failure") + request = urllib.request.Request(url, data=data, method=method) + try: + try: + response = self.opener.open(request, timeout=min(1, remaining(deadline, self.clock))) + except urllib.error.HTTPError as error: + response = error + with response: + if response.code in (301, 302, 303, 307, 308): + target = response.headers.get("Location") + if not target: + raise SmokeError("http_redirect_invalid", "application_failure") + url = urllib.parse.urljoin(url, target) + if response.code in (301, 302, 303): + method, data = "GET", None + continue + if response.code != 200: + raise SmokeError("http_status_failed", "application_failure") + result = bytearray() + while True: + remaining(deadline, self.clock) + chunk = response.read1(16384) + if not chunk: + return result.decode("utf-8", errors="replace") + result.extend(chunk) + if len(result) > HTTP_LIMIT: + raise SmokeError("http_response_limit", "application_failure") + except TimeoutError: + raise SmokeError("http_request_timeout", "timeout") from None + except urllib.error.URLError as error: + if isinstance(error.reason, TimeoutError): + raise SmokeError("http_request_timeout", "timeout") from None + remaining(deadline, self.clock) + raise SmokeError("frontend_connection_failed", "application_failure") from None + except OSError: + remaining(deadline, self.clock) + raise SmokeError("frontend_connection_failed", "application_failure") from None + raise SmokeError("http_redirect_limit", "application_failure") diff --git a/tests/test_smoke.py b/tests/test_smoke.py new file mode 100644 index 0000000..4dadc5b --- /dev/null +++ b/tests/test_smoke.py @@ -0,0 +1,303 @@ +"""Runtime orchestration tests never call Docker or the network.""" + +import hashlib +import io +import json +import os +from pathlib import Path +import sys +import tempfile +import time +import unittest +from unittest.mock import patch + +import test_compose as fixtures +from container_platform import smoke +from container_platform.smoke_transport import Docker, Session, SmokeError, process_output + + +class Clock: + def __init__(self): + self.now = 0 + + def __call__(self): + return self.now + + def sleep(self, delay): + self.now += delay + + +class FakeDocker: + def __init__(self, project, directory, clock): + self.project, self.clock = project, clock + self.document = json.loads((directory / "compose.json").read_bytes()) + self.calls, self.failure, self.states, self.cleanup_fail = [], None, "running", False + + def raw(self, args, deadline): + smoke.remaining(deadline, self.clock) + self.calls.append(args) + if args[0] == "version": + return 0, b'{"Os":"linux","Arch":"amd64"}' + if args[0] == "context": + return 0, b'"unix:///var/run/docker.sock"' + return 0, b"" + + def compose(self, args, deadline): + smoke.remaining(deadline, self.clock) + self.calls.append(args) + if self.failure and args[0] == self.failure[0]: + raise self.failure[1] + if args[0] == "down" and self.cleanup_fail: + return 1, b"private paths must never be echoed" + if args[0] == "config": + return 0, json.dumps(self.document).encode() + if args[0] == "ps": + return 0, json.dumps([{"Service": k, "State": self.states, "Image": v["image"]} + for k, v in self.document["services"].items()]).encode() + return 0, b"" + + +class FakeSession: + def __init__(self, clock): + self.quantity, self.bad_checkout = 0, False + + def request(self, path, deadline, form=None): + if path == "/": + return 'Catalog' + if path.startswith("/product/"): + return '' + if path == "/cart/empty": + self.quantity = 0 + if path == "/cart" and form: + self.quantity += int(form["quantity"]) + if path == "/cart/checkout": + assert form["email"].endswith(".invalid") + self.quantity = 0 + return "unrelated HTTP 200 page" if self.bad_checkout else "Your order is complete! Confirmation # 12345678-abcd Tracking # TRACK123" + if self.quantity: + return f"Cart ({self.quantity}) SKU #{smoke.PRODUCT} Quantity: {self.quantity}" + return "Your shopping cart is empty!" + + +class SmokeTests(unittest.TestCase): + def setUp(self): + self.fixture = fixtures.ComposeTests() + self.fixture.setUp() + self.addCleanup(self.fixture.doCleanups) + self.input, self.repo = self.fixture.input, self.fixture.repo + self.input.write_text(json.dumps(self.fixture.candidate)) + self.clock, self.docker, self.session = Clock(), None, None + self.configure = lambda d: None + self.configure_session = lambda s: None + self.port = patch.object(smoke, "port_available") + self.port.start() + self.addCleanup(self.port.stop) + + def docker_factory(self, *args): + self.docker = FakeDocker(*args) + self.configure(self.docker) + return self.docker + + def session_factory(self, clock): + self.session = FakeSession(clock) + self.configure_session(self.session) + return self.session + + def run_smoke(self, **kwargs): + options = dict(execute=True, inspection_only=True) + options.update(kwargs) + return smoke.run(self.input, self.repo, "aml-online-shop-compose-v1", docker_factory=self.docker_factory, + session_factory=self.session_factory, clock=self.clock, sleep=self.clock.sleep, **options) + + def test_no_opt_in_does_not_even_read_input_or_construct_docker(self): + self.input.unlink() + result = self.run_smoke(execute=False) + self.assertEqual(result["diagnostics"], ["runtime_opt_in_required"]) + self.assertEqual(result["exitCode"], 2) + self.assertIsNone(self.docker) + + def test_validation_and_inspection_restrictions_before_docker(self): + self.assertEqual(self.run_smoke(inspection_only=False)["exitCode"], 1) + self.assertIsNone(self.docker) + self.fixture.candidate["components"].pop() + self.input.write_text(json.dumps(self.fixture.candidate)) + result = self.run_smoke() + self.assertEqual(result["exitCode"], 1) + self.assertIsNone(self.docker) + self.assertEqual(result["candidate"]["sha256"], hashlib.sha256(self.input.read_bytes()).hexdigest()) + + def test_success_exact_images_snapshot_isolation_report_and_cleanup(self): + original = self.input.read_bytes() + def change_input(docker): + self.input.write_bytes(b"changed after validation") + self.configure = change_input + result = self.run_smoke() + self.assertEqual(result["exitCode"], 0) + self.assertEqual(result["runtimeOutcome"], "success") + self.assertEqual(result["cleanup"]["outcome"], "success") + self.assertEqual(result["validation"]["exitCode"], 1) + self.assertEqual(result["candidate"]["sha256"], hashlib.sha256(original).hexdigest()) + self.assertEqual(len(result["candidate"]["images"]), 12) + self.assertEqual(result["smokeAssertions"], ["frontend_catalog_pass", "product_browse_pass", "cart_add_pass", "cart_remove_pass", "synthetic_checkout_pass"]) + self.assertEqual(result["project"], self.docker.project) + self.assertRegex(result["project"], r"^cp-smoke-[a-f0-9]{32}$") + self.assertIn(["up", "--detach", "--no-build", "--pull", "never"], self.docker.calls) + self.assertIn(["down", "--volumes", "--remove-orphans", "--timeout", "10"], self.docker.calls) + self.assertNotIn(str(self.repo), json.dumps(result)) + self.assertNotIn("private paths", json.dumps(result)) + + def test_partial_startup_and_interruption_always_cleanup(self): + for failure, expected in [(SmokeError("partial_up_failed"), 2), (KeyboardInterrupt(), 130)]: + self.configure = lambda d: setattr(d, "failure", ("up", failure)) + result = self.run_smoke() + self.assertEqual(result["exitCode"], expected) + self.assertEqual(result["cleanup"]["outcome"], "success") + + def test_pull_timeout_has_no_project_resources(self): + self.configure = lambda d: setattr(d, "failure", ("pull", SmokeError("pull_timeout", "timeout"))) + result = self.run_smoke() + self.assertEqual(result["exitCode"], 3) + self.assertEqual(result["cleanup"]["outcome"], "not_required") + self.assertFalse(any(c[0] == "up" for c in self.docker.calls)) + + def test_readiness_retries_share_one_deadline(self): + self.configure = lambda d: setattr(d, "states", "created") + result = self.run_smoke() + self.assertEqual(result["exitCode"], 3) + self.assertEqual(result["elapsedSeconds"], smoke.LIMITS["readiness"]) + self.assertEqual(result["cleanup"]["outcome"], "success") + + def test_http_200_without_order_confirmation_fails(self): + self.configure_session = lambda s: setattr(s, "bad_checkout", True) + result = self.run_smoke() + self.assertEqual(result["exitCode"], 1) + self.assertIn("checkout_confirmation_missing", result["diagnostics"]) + self.assertEqual(result["cleanup"]["outcome"], "success") + + def test_cleanup_failure_never_gives_success_or_hides_primary(self): + self.configure = lambda d: setattr(d, "cleanup_fail", True) + result = self.run_smoke() + self.assertEqual(result["runtimeOutcome"], "success") + self.assertEqual(result["exitCode"], 2) + self.assertEqual(result["cleanup"]["outcome"], "failed") + self.configure_session = lambda s: setattr(s, "bad_checkout", True) + result = self.run_smoke() + self.assertEqual(result["exitCode"], 1) + self.assertEqual(result["diagnostics"][0], "checkout_confirmation_missing") + self.assertEqual(result["cleanup"]["outcome"], "failed") + + def test_missing_services_and_image_mismatch_are_not_ready(self): + def wrong_image(d): + d.document["services"]["frontend"]["image"] = "other" + self.configure = wrong_image + result = self.run_smoke() + self.assertEqual(result["diagnostics"], ["runtime_model_mismatch"]) + self.assertFalse(any(c[0] == "pull" for c in self.docker.calls)) + + def test_engine_or_busy_port_failure_has_no_mutations(self): + with patch.object(smoke, "port_available", side_effect=SmokeError("frontend_port_unavailable")): + result = self.run_smoke() + self.assertEqual(result["diagnostics"], ["frontend_port_unavailable"]) + self.assertFalse(any(c[0] in ("pull", "up", "down") for c in self.docker.calls)) + def unavailable(docker): + docker.raw = lambda args, deadline: (1, b"machine-specific connection error") + self.configure = unavailable + result = self.run_smoke() + self.assertEqual(result["diagnostics"], ["docker_engine_unavailable"]) + self.assertEqual(result["cleanup"]["outcome"], "not_required") + + def test_remote_engine_context_is_rejected_before_pull(self): + def remote(docker): + docker.env = {"DOCKER_HOST": "ssh://remote.invalid"} + self.configure = remote + result = self.run_smoke() + self.assertEqual(result["diagnostics"], ["remote_docker_transport_unsupported"]) + self.assertFalse(any(c[0] in ("pull", "up", "down") for c in self.docker.calls)) + + def test_failed_container_is_application_failure_and_cleanup_is_checked(self): + self.configure = lambda d: setattr(d, "states", "exited") + result = self.run_smoke() + self.assertEqual(result["runtimeOutcome"], "application_failure") + self.assertIn("service_not_running", result["diagnostics"]) + self.assertEqual(result["cleanup"]["outcome"], "success") + + def test_missing_service_records_cannot_satisfy_readiness(self): + def missing_service(docker): + original = docker.compose + def compose(args, deadline): + if args[0] == "ps": + code, output = original(args, deadline) + return code, json.dumps(json.loads(output)[:-1]).encode() + return original(args, deadline) + docker.compose = compose + self.configure = missing_service + result = self.run_smoke() + self.assertEqual(result["exitCode"], 3) + self.assertEqual(len(result["services"]), 11) + self.assertEqual(result["cleanup"]["outcome"], "success") + + def test_cleanup_verification_detects_resources_remaining(self): + def residue(docker): + original_raw = docker.raw + def raw(args, deadline): + if any(c[0] == "down" for c in docker.calls) and args[0] == "ps": + return 0, b"remaining-project-container" + return original_raw(args, deadline) + docker.raw = raw + self.configure = residue + result = self.run_smoke() + self.assertEqual(result["exitCode"], 2) + self.assertEqual(result["cleanup"]["reason"], "cleanup_resources_remaining") + + +class TransportTests(unittest.TestCase): + def test_compose_environment_and_explicit_arguments(self): + with tempfile.TemporaryDirectory() as temporary, patch.dict(os.environ, {"COMPOSE_FILE": "hostile", "COMPOSE_PROFILES": "other", "COMPOSE_PROJECT_NAME": "foreign", "DOCKER_DEFAULT_PLATFORM": "linux/arm64"}): + docker = Docker("cp-smoke-test", Path(temporary)) + self.assertEqual([k for k in docker.env if k.startswith("COMPOSE_")], ["COMPOSE_DISABLE_ENV_FILE"]) + self.assertNotIn("DOCKER_DEFAULT_PLATFORM", docker.env) + with patch.object(docker, "raw", return_value=(0, b"")) as call: + docker.compose(["up", "--no-build"], time.monotonic() + 1) + args = call.call_args.args[0] + self.assertIn("empty.env", args) + self.assertIn("compose.json", args) + self.assertIn("cp-smoke-test", args) + + def test_process_output_and_timeout_are_bounded(self): + with tempfile.TemporaryDirectory() as temporary: + with self.assertRaises(SmokeError) as error: + process_output([sys.executable, "-c", "import time; time.sleep(10)"], temporary, dict(os.environ), time.monotonic() + 0.2) + self.assertEqual(error.exception.outcome, "timeout") + with self.assertRaises(SmokeError) as error: + process_output([sys.executable, "-c", "print('x'*2000000)"], temporary, dict(os.environ), time.monotonic() + 10) + self.assertEqual(error.exception.code, "subprocess_output_limit") + + def test_http_cannot_redirect_to_external_origin(self): + import urllib.error + from email.message import Message + session = Session() + headers = Message() + headers["Location"] = "https://external.invalid/" + error = urllib.error.HTTPError(session.origin, 302, "redirect", headers, io.BytesIO()) + with patch.object(session.opener, "open", side_effect=error) as opened: + with self.assertRaises(SmokeError) as result: + session.request("/", time.monotonic() + 10) + self.assertEqual(result.exception.code, "http_redirect_outside_frontend") + self.assertEqual(opened.call_count, 1) + + def test_http_response_size_and_request_deadline(self): + response = io.BytesIO(b"x" * (2 * 1024 * 1024 + 1)) + response.code = 200 + session = Session() + with patch.object(session.opener, "open", return_value=response): + with self.assertRaises(SmokeError) as error: + session.request("/", time.monotonic() + 10) + self.assertEqual(error.exception.code, "http_response_limit") + with patch.object(session.opener, "open", side_effect=TimeoutError): + with self.assertRaises(SmokeError) as error: + session.request("/", time.monotonic() + 10) + self.assertEqual(error.exception.outcome, "timeout") + + +if __name__ == "__main__": + unittest.main() From 9f28d77df64106ddefa52c2934c132d9cb022bb0 Mon Sep 17 00:00:00 2001 From: Dmitry Zhuravlev Date: Wed, 21 Oct 2026 09:07:23 -0400 Subject: [PATCH 2/3] docs(smoke): record blocked baseline runtime attempt --- README.md | 2 +- docs/runtime-smoke.md | 12 +++++++++++- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index bb22d43..750a0fc 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ The two paths into the SRE Platform have different roles: - Docker Hub is the data path: Kubernetes nodes pull the approved images from `dmitryzhuravlev/online-shop-*`. - Container Platform is the release-control path: it identifies the exact image digests that are approved together as one Online Shop release. -The diagram describes target relationships. Offline, opt-in Docker Hub metadata, PR checks and Compose rendering are implemented; required-check configuration remains an owner action. Runtime integration and SRE handoff remain planned. +The diagram describes target relationships. Offline, opt-in Docker Hub metadata, PR checks and Compose rendering are implemented; required-check configuration remains an owner action. Runtime evidence is reported separately; SRE handoff remains planned. ## Responsibilities diff --git a/docs/runtime-smoke.md b/docs/runtime-smoke.md index e2ff6d9..40a595a 100644 --- a/docs/runtime-smoke.md +++ b/docs/runtime-smoke.md @@ -37,7 +37,7 @@ Work stops before the overall deadline to reserve diagnostics and cleanup time. Readiness requires exactly 12 expected service records, the exact image references, every service running and a frontend catalog response. This establishes process state plus frontend availability, not independent health evidence for every backend. Host-side semantic assertions then exercise catalog browsing, a selected product, empty/add/remove/re-add cart operations and synthetic checkout. A final service-state check detects an exit during the sequence. Loadgenerator traffic is independent and cannot satisfy these assertions. -Routes/forms and assertions derive from immutable AML revision [`9960484b5c7f1568688fa6fc8eb256976d6c18a4`](https://github.com/DimitryZH/application-modernization-lab/tree/9960484b5c7f1568688fa6fc8eb256976d6c18a4/workloads/online-shop/src): frontend routes/handlers, product/cart/order templates and loadgenerator. Payment service performs a pretend charge; email starts in dummy mode in this profile. The runner uses a standard test card number, reserved `.invalid` email and synthetic address, without payment/mail credentials. It checks selected product ID, cart quantity, order confirmation/tracking and post-checkout cart clearance. It does not fetch browser assets, execute scripts or follow redirects outside `http://127.0.0.1:8080`. This is a narrow demo flow, not verification of real payment/email systems or image provenance. +Routes/forms and assertions derive from immutable AML revision [`9960484b5c7f1568688fa6fc8eb256976d6c18a4`](https://github.com/DimitryZH/application-modernization-lab/tree/9960484b5c7f1568688fa6fc8eb256976d6c18a4/workloads/online-shop/src): frontend routes/handlers, product/cart/order templates and loadgenerator. Pinned AML source implements a pretend charge and dummy email default; legacy image runtime behavior remains unverified until live checks pass. The runner uses a standard test card number, reserved `.invalid` email and synthetic address, without payment/mail credentials. It checks selected product ID, cart quantity, order confirmation/tracking and post-checkout cart clearance. It does not fetch browser assets, execute scripts or follow redirects outside `http://127.0.0.1:8080`. This is a narrow demo flow, not verification of real payment/email systems or image provenance. ## Report and exit codes @@ -70,3 +70,13 @@ docker volume ls -q --filter $smokeLabel | ForEach-Object { docker volume rm $_ ## Validation evidence Automated tests use mock Docker/HTTP adapters and isolated local Git fixtures; the ordinary suite requires no Docker or registry requests. Installed-wheel checks exercise both smoke interfaces outside checkout without runtime opt-in. No mandatory runtime PR job is added. Live results are recorded separately below; runtime integration cannot be claimed from mocks or static configuration alone. + +### Live observation: 2026-10-21 + +- Implementation: `f5ea3b8e2b2f48e014cd985c3a20412bbdce0956`, installed-wheel code/profile/schema matched before execution. +- Input: `releases/baseline-2026-10-18.json`, byte SHA-256 `005293954afebcf1581116b2e7e6bc5ec6313a658496c88df3186aa84a46a99f`; profile `aml-online-shop-compose-v1`, explicit inspection-only. +- Render passed; prerequisites failed with `docker_engine_unavailable`. Runtime outcome: `execution_error`, exit 2. Pull/startup and HTTP smoke were not attempted. +- Cleanup: `not_required`; no project resources were created. +- Offline validation remained exit 1 with 12 `legacy_ineligible` failures. No approval or eligibility was granted. + +One bounded live attempt was made. Provide a running local Linux/amd64 Docker Engine and repeat the documented baseline command to verify readiness, semantic checkout and cleanup after actual startup. Those live behaviors remain unverified; mock tests do not replace them. From 4bfc981fc5d24befb52e5c6c5213092984c66916 Mon Sep 17 00:00:00 2001 From: Dmitry Zhuravlev Date: Wed, 21 Oct 2026 09:31:07 -0400 Subject: [PATCH 3/3] docs(smoke): record successful baseline runtime verification --- docs/runtime-smoke.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/runtime-smoke.md b/docs/runtime-smoke.md index 40a595a..716a916 100644 --- a/docs/runtime-smoke.md +++ b/docs/runtime-smoke.md @@ -80,3 +80,13 @@ Automated tests use mock Docker/HTTP adapters and isolated local Git fixtures; t - Offline validation remained exit 1 with 12 `legacy_ineligible` failures. No approval or eligibility was granted. One bounded live attempt was made. Provide a running local Linux/amd64 Docker Engine and repeat the documented baseline command to verify readiness, semantic checkout and cleanup after actual startup. Those live behaviors remain unverified; mock tests do not replace them. + +### Live observation: 2026-10-21 — successful follow-up + +- Implementation: `9f28d77df64106ddefa52c2934c132d9cb022bb0`; installed code, profile and schema matched the commit before execution. +- Input: `baseline-2026-10-18-001`, `releases/baseline-2026-10-18.json`, byte SHA-256 `005293954afebcf1581116b2e7e6bc5ec6313a658496c88df3186aa84a46a99f`; inventory `inventories/online-shop-sre-dev-v1.json` at `480bd0a5769ae61313c4bd0da846ba3a6fc0ce51`; profile `aml-online-shop-compose-v1`, inspection-only. +- One bounded run passed in 45.8 seconds: prerequisites, pull, startup, all 12 services running with exact candidate digest references, readiness and semantic catalog/product/cart/synthetic-checkout assertions. Runtime exit 0. +- Cleanup passed; an independent label-scoped query confirmed zero remaining project containers, networks and volumes. Cached images were retained. +- Offline validation remained exit 1 with 12 `legacy_ineligible` failures. The earlier negative observation is retained above. + +This confirms the selected demo flow for these exact inputs in one run. It does not establish production readiness, performance, full backend coverage, provenance, release approval or deployment authorization. No mandatory runtime CI job was added.