Skip to content

tests: the styling gate covers run, trace and metrics - #126

Merged
Shashankss1205 merged 1 commit into
mainfrom
test/issue-17-styled-command-coverage
Sep 26, 2026
Merged

Shashankss1205 merged 1 commit into
mainfrom
test/issue-17-styled-command-coverage

Conversation

@Shashankss1205

Copy link
Copy Markdown
Collaborator

Closes #17.

The contract — strip the escapes from what a terminal receives and it equals what a pipe receives, exactly — was gated for four commands: plan, models, models --check, demo stage0. The commands most likely to be piped were the uncovered ones: run --check-only is a linter in this repository's own CI, trace | grep and metrics in a script are the obvious uses, and run's REFUSED block tints the key and the value of its verdict line — a shape no covered case reached.

The gap, demonstrated rather than asserted

Painting trace's step column unconditionally (a one-line change at its call site, exactly the regression the issue describes):

result
the old four-case gate 7 passed — fully green
the new cases trace fails in both tests

That is the whole claim of the issue, reproduced and then closed.

What changed

Four cases added to STYLED/COMPARABLE: run --check-only (ADMITTED), run on a bad topology (REFUSED), trace, metrics.

A case is a builder rather than a literal argv now, because these commands take a file argument. The workspace fixture writes an admitted topology, a refused one carrying two objections so the per-objection row runs more than once, and a trace from one demo stage0 --trace run — the pattern test_viz_on_a_terminal_stays_pasteable_mermaid already uses.

The hermetic argv matters more than it looks. run resolves its registry and policy from the working directory, and this repository's root carries a registry.py and a grapharc.toml that are gitignored — dogfooding residue. A case leaning on those would read one registry locally and a different one in CI, then compare output differing for reasons unrelated to styling. So the registry is named explicitly and --config points at an empty file.

Existing judgement calls kept: exit codes stay unpinned, and models --check stays out of COMPARABLE.

One normaliser, added under protest

run's fingerprint is not stable across two loads of the same topology — Subgraph.proposal_id defaults to a fresh uuid4 and fingerprint() hashes the whole model, that field included. So the same topology file yields a different fingerprint every invocation, while graphrun.py prints it under the comment "the fingerprint is what a later run is compared against".

The issue predicted run's output would be deterministic apart from the mkdtemp trace path; that turned out to be wrong, and this is why. Normalising the fingerprint keeps the ADMITTED block's styling inside the byte-for-byte comparison rather than dropping run over one token.

Filed separately as #125, with the two ways it could be resolved and why the choice is a maintainer's. When it is fixed, _FINGERPRINT should be deleted and this comparison gets stricter for it.

Verified

2,205 selected, 13 deselected, ruff check . clean. Eight styled ids collect where there were four.

🤖 Generated with Claude Code

Closes #17. The contract — strip the escapes from what a terminal receives and
it equals what a pipe receives, exactly — was gated for four commands: `plan`,
`models`, `models --check`, `demo stage0`. The commands most likely to be piped
were the uncovered ones: `run --check-only` is a linter in this repository's own
CI, `trace | grep` and `metrics` in a script are the obvious uses, and `run`'s
REFUSED block tints the key *and* the value of its verdict line, a shape no
covered case reached.

Four cases added, and the gap is demonstrated rather than asserted. Painting
`trace`'s step column unconditionally leaves the old four-case gate **fully
green** while both new `trace` cases fail. That is the whole claim of the issue,
reproduced.

A case is a builder rather than a literal argv now, because these commands take
a file argument. The `workspace` fixture writes an admitted topology, a refused
one carrying two objections so the per-objection row runs more than once, and a
trace from one `demo stage0` run.

The hermetic argv matters more than it looks. `run` resolves its registry and
policy from the working directory, and this repository's root carries a
`registry.py` and a `grapharc.toml` that are **gitignored** — dogfooding
residue. A case leaning on those would read one registry locally and another in
CI and then compare output that differs for reasons unrelated to styling. So the
registry is named explicitly and `--config` points at an empty file.

One normaliser added, under protest. `run`'s fingerprint is not stable across
two loads of the same topology: `Subgraph.proposal_id` defaults to a fresh
`uuid4` and `fingerprint()` hashes the whole model, that field included. So the
same topology file yields a different fingerprint every invocation, while
`graphrun.py` prints it under the comment "the fingerprint is what a later run
is compared against". Normalising it keeps the ADMITTED block's styling in the
comparison instead of dropping `run` over one token; the underlying problem is
filed separately, and when it is fixed this normaliser should go and the
comparison gets stricter for it.

Verified: 2205 selected, 13 deselected, ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Shashankss1205
Shashankss1205 merged commit 2e4b836 into main Sep 26, 2026
7 checks passed
@Shashankss1205
Shashankss1205 deleted the test/issue-17-styled-command-coverage branch September 26, 2026 12:13
Shashankss1205 added a commit that referenced this pull request Sep 26, 2026
#127 and #126 merged after the release commit and neither is in the notes.
0.1.8 is still unpublished and untagged, so folding them in is correct for the
same reason #124 did it for #123.

The reclaim gets a full entry: it changes shipped behaviour and adds a public
method, and the entry is explicit that it closes a documented *limitation*
rather than a defect -- which comes to the same thing for whoever hit it. What
it refuses, and why nothing calls it automatically, are the load-bearing parts
and are stated.

The styling gate goes in the tooling paragraph, where it belongs: it changes no
shipped behaviour, it changes what a regression in shipped behaviour would be
caught by.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cli: the tty-vs-piped styling gate covers four commands; run, trace and metrics are styled but untested

1 participant