Skip to content

feat(uicheck): fail contrast below AA, resolve Tailwind theme tokens in design - #162

Merged
CodeWithJuber merged 7 commits into
masterfrom
fix/uicheck-exit-codes
Sep 24, 2026
Merged

CodeWithJuber merged 7 commits into
masterfrom
fix/uicheck-exit-codes

Conversation

@CodeWithJuber

Copy link
Copy Markdown
Owner

What & why

This PR fixes the MEDIUM finding "uicheck design passes token-based Tailwind by default; contrast exits 0 on failure and accepts hex only" from an evaluation run against the HostLelo site. That run measured:

  • forge uicheck design src/components/v2/primitives.tsx returned PASS with "1 radius level, 0 shadows, 0 colors". The file actually uses rounded-control / rounded-md / rounded-card, shadow-lift and about 20 semantic color classes (bg-brand-fill, text-fg, …).
  • An empty <div/> file also got PASS (slop distance 1).
  • forge uicheck contrast '#777' '#fff' printed FAILS AA and exited 0.
  • rgb(0 0 0 / 50%) and oklch(0.6 0.1 250) were rejected as "bad hex color", although oklch is Tailwind v4's default. There was no --large and no --json.

Contrast (src/uicheck.js, the contrast branch of src/cli.js):

  • forge uicheck contrast and the bare forge uicheck <fg> <bg> exit 1 when the pair fails AA.

  • --large applies the 3:1 bar. --json prints ratio, level, thresholds, the hexes actually compared, and notes.

  • New parseColor reads:

    • hex with 3, 4, 6 or 8 digits (the # is optional)
    • rgb() and hsl(), in both comma and / alpha syntax
    • oklch() and oklab()
    • black, white and transparent

    Anything else throws.

  • A translucent foreground is composited over the background, and a translucent background over white. Both are rounded to 8-bit before measuring, so the ratio always matches the hexes reported.

Design gate (src/uifingerprint.js, src/cli.js, src/uivisual.js):

  • fingerprint and design read theme tokens:
    • Tailwind v4: --color-*, --radius-* and --shadow-* in @theme blocks. var(), hsl(var(--x)) and calc() are resolved.
    • Tailwind v3: the colors, borderRadius and boxShadow objects of tailwind.config.*. The config is parsed statically, never executed.
  • Theme sources are discovered under the working directory, skipping node_modules, build output and dot-directories. --theme <file> (repeatable) names them explicitly. A named file that is missing is an error.
  • rounded-*, shadow-* and bg|text|border|ring|fill|stroke-* utilities resolve through those tokens. Arbitrary values are parsed too: rounded-[13px], shadow-[…], p-[…], text-[#…], bg-[oklch(…)].
  • An empty feature vector gets the verdict insufficient-signal and exits 1 in both design and visual. It never shows PASS.

Review follow-ups (commit 15f0666). Each issue was reproduced on 2f8651f before it was fixed.

  • Dark mode: overrides in .dark, [data-theme=dark] and @media (prefers-color-scheme: dark) no longer win var() resolution. A property declared only for dark mode still resolves, and :root:not(.dark) counts as light. On HostLelo, bg-canvas resolved to the dark #080f17. It now resolves to the default #f1f5f9.
  • Contrast quantizing: contrastReport now rounds the background to 8-bit before compositing, as contrastRatio does. For rgba(0,0,0,.5) on rgba(255,0,0,.3) it reported #805959 at 3.54 while contrastRatio said 3.50. Both now give #805a5a at 3.50.
  • Shadow colors: shadow-[#123456] and shadow-[color:…] set a shadow color, so they count in the palette, not as an elevation level.
  • Transparent colors: transparent, alpha 0 and transparent theme tokens no longer count as black.
  • Deep nesting: calc() evaluation and the tailwind.config reader cap their nesting depth. About 5000 nested parentheses threw RangeError and crashed uicheck design.
  • Empty mint: fingerprint --mint refuses an empty vector: it exits 1 and stores nothing. Before, it stored an empty "home" that failed every later design run.

Result on HostLelo now:

  • uicheck design src/components/v2/primitives.tsx finds src/app/globals.css (123 color, 8 radius and 9 shadow tokens).
  • It measures 20 colors, radii 10/14/16 and 1 shadow level.
  • It FAILs palette-size: 20 colors against the existing cap of 8. I left that cap unchanged.
  • The empty fixture prints INSUFFICIENT SIGNAL with exit 1, and contrast "#777" "#fff" exits 1.

Tests added (test/uicheck.test.js, test/uifingerprint.test.js, test/uivisual.test.js):

  • Contrast:
    • parseColor in every syntax, checked against CSS Color 4 reference points: oklch red gives #ff0000 and Tailwind v4 blue-500 gives #2b7fff.
    • compositeOver, toHex and contrastReport.
    • Agreement between contrastReport and contrastRatio when both colors are translucent.
    • CLI: #777 on #fff exits 1, plus --large, --json, usage errors and bad-color errors.
  • Design gate:
    • Theme parsing: themeFromCss (v4) and themeFromTailwindConfig (v3).
    • Theme sources: findThemeSources, themeSourcesFor and loadThemeTokens.
    • Fingerprinting: token-utility resolution and arbitrary values.
    • Verdicts: hasDesignSignal, overallVerdict, and insufficient-signal in uiGate, the design CLI and visualGate (via a fake browser).
    • CLI: --theme discovery and errors. Also, the minted vector includes theme tokens.
  • Review follow-ups: dark-scope resolution (including :root:not(.dark)), shadow colors, transparent colors, deep nesting and the empty-mint refusal. All 6 of these tests fail on 2f8651f and pass on HEAD.

Declined or left for follow-up:

  • ASSERTABLE_CHECKS still lists alt-text, form-labels and tap-target with no code behind them. That is part of the same finding, but it is outside this branch's scope.
  • The first two commits are typed fix:, and history was not rewritten. The CHANGELOG now has a ### Changed entry for the new non-zero exits. This PR is titled feat:, so a squash merge releases a minor version. A merge commit would release a patch.

Checklist

  • npm test passes (Node 18/20/22). It passes on Node 22.22.2 locally: 1474 tests, 1471 pass, 0 fail, 3 skipped. Node 18 and 20 were not run here; that is left to CI's matrix.
  • npm run check passes (Biome lint + format). Exit 0. Its 14 warnings and 2 infos are all in files this PR does not touch, and npx biome check on the changed files reports nothing.
  • New public functions have a test
  • Conventional commit message (feat:/fix:/docs: …)
  • CHANGELOG.md updated under ## [Unreleased] (### Changed for the new exit codes, ### Fixed for the rest)
  • No new runtime dependency (dev deps ok). package.json is untouched.
  • Substrate/docs updated if this changes forge substrate, forge impact, router/gate, or MCP substrate tools. Not applicable: none of these change. The uicheck docs were updated (docs/GUIDE.md, mintlify/cli/quality.mdx, mintlify/concepts/verification-gates.mdx, the help text in src/commands.js), and node src/cli.js docs check exits 0. Its one warning, that the ARCHITECTURE.md repo-map block is out of date, is also on master.

Risk & rollback

  • Risk level: medium
  • New non-zero exits. forge uicheck now exits 1 in three cases that used to exit 0: contrast below AA, insufficient-signal in design/visual, and an empty fingerprint --mint. A script that relied on exit 0 will now fail. No hook, workflow or MCP tool in this repo invokes uicheck. It appears only in a comment in static.yml and in skill prose that tells agents to run the gate, where a failing exit is intended.
  • Existing fingerprints. The vector now includes theme tokens. A project fingerprint minted before this change can make design report conformance drift for token-based UIs until it is re-minted. The CHANGELOG says so, and nothing re-mints automatically.
  • Known limits:
    • Only color, radius and shadow tokens are read. Font (--font-*) and spacing (--spacing-*) tokens are not.
    • tailwind.config values produced by code (spreads, require(), function calls) are skipped, never evaluated.
    • --color-*: initial resets are ignored.
    • insufficient-signal and FAIL share exit 1. They are told apart only by verdict in --json.
  • Rollback plan: git revert the merge commit, or revert 15f0666, 2f8651f and 2b3d480. There is no schema or data migration. Fingerprint claims minted by this version are ordinary ledger claims the old code can read. Re-mint after rolling back.

Extra checks (tick if applicable)

  • npm run typecheck passes
  • Input validated at boundaries; errors handled (no swallowing).
    • An unparseable color throws, and the CLI exits 1, with a JSON error under --json.
    • A valueless --theme, a missing --theme file and a wrong color count are all errors with exit 1.
    • Deliberate skips: unreadable input or theme files are skipped (as before), and non-color text in class strings is ignored while scanning.
  • Authorization/ownership checked (if it touches access). Not applicable.
  • Logs contain no secrets/PII. The new output prints colors, file paths and token counts only.
  • If AI-assisted: I understand it, verified the package APIs, and it has tests. This change was written by Claude. It uses Node built-ins only and has tests. The human submitter should tick this box after their own review.

Found by an evaluation run against the HostLelo site (CodeWithJuber/my-next-app).

🤖 Generated with Claude Code

https://claude.ai/code/session_01UUhB8JaPayd43w37dxiXrW


Generated by Claude Code

…alpha

`forge uicheck contrast '#777' '#fff'` printed FAILS AA and exited 0, so no
script or CI step could gate on it. It also rejected every color that was not
#rgb/#rrggbb, including oklch(), Tailwind v4's default palette syntax.

- contrast (and the bare legacy `uicheck <fg> <bg>`) exit 1 when AA fails.
- --large grades against the 3:1 large-text / UI bar; --json prints the
  full report (ratio, level, thresholds, the colors compared, notes).
- parseColor() reads hex with an optional alpha pair, rgb()/rgba(),
  hsl()/hsla(), oklch(), oklab() and black/white/transparent, in legacy
  comma and modern `/ alpha` syntax. It throws on anything else.
- A translucent foreground is composited over the background (and a
  translucent background over white) before measuring, quantized to the
  8-bit color that is actually painted. Notes say when that happened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UUhB8JaPayd43w37dxiXrW
Signed-off-by: Claude <noreply@anthropic.com>
… insufficient-signal

`forge uicheck design` returned PASS for token-based Tailwind because it saw
nothing. On HostLelo's src/components/v2/primitives.tsx (rounded-card,
shadow-lift, bg-brand-fill, ...) it counted 1 radius, 0 shadows and
0 colors. An empty `<div/>` file also printed PASS.

- fingerprint/design read theme tokens: Tailwind v4 `@theme { --color-*
  --radius-* --shadow-* }` (var(), hsl(var(--x)) and calc() resolved) and
  the colors/borderRadius/boxShadow objects of tailwind.config.* (parsed
  statically, never executed).
- Theme sources are discovered under the working directory, skipping
  node_modules, build output and dot-directories, or named with the
  repeatable --theme <file>. A named file that is missing is an error.
- rounded-*, shadow-* and (bg|text|border|ring|fill|stroke|...)-* utilities
  resolve through those keys. A theme key that redefines a default wins.
- Arbitrary values are parsed: rounded-[..], shadow-[..], p-/m-/gap-[..],
  text-[#..], bg-[oklch(..)]. oklch()/oklab() in plain CSS count as colors.
- An empty feature vector gets the verdict `insufficient-signal`
  (pass:false, exit 1) in design and visual, never PASS. --json reports
  `verdict` and the theme summary.
- --mint uses the same theme, so the minted home matches what design gates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UUhB8JaPayd43w37dxiXrW
Signed-off-by: Claude <noreply@anthropic.com>
…erprint

Follow-ups from the independent review of the two previous commits. Each
one was reproduced on 2f8651f before the fix.

- Dark-mode overrides no longer win var() resolution. `.dark {}`,
  `[data-theme=dark] {}` and `@media (prefers-color-scheme: dark) {}`
  declarations are applied only to properties with no default. Negations
  such as `:root:not(.dark)` count as light scope. On HostLelo's
  globals.css, bg-canvas resolved to the dark #080f17; it now resolves to
  the default #f1f5f9.
- contrastReport quantizes the background before compositing the
  foreground, exactly as contrastRatio does. For rgba(0,0,0,.5) on
  rgba(255,0,0,.3) it reported #805959 at 3.54 while contrastRatio gave
  3.50. Both now give #805a5a at 3.50.
- shadow-[#123456] and shadow-[color:...] are shadow colors: they count
  in the palette, not as an elevation level. Fully transparent colors
  (transparent, alpha 0 in hex/rgb/hsl/oklch, transparent theme tokens)
  are no longer palette entries counted as black.
- calc() evaluation and the tailwind.config object reader cap their
  nesting depth. About 5000 nested parens, or 20000 nested config
  objects, threw RangeError and crashed `uicheck design`.
- mintProjectFingerprint refuses an empty vector (ok:false,
  insufficient-signal) and writes nothing. `fingerprint --mint` then
  exits 1. Before, it stored an empty "home" that failed every later
  `design` run.

CHANGELOG gains a "Changed" entry for the three new non-zero exits
(contrast AA failure, insufficient-signal, empty --mint). GUIDE, the
Mintlify pages and the --mint help text describe the new rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UUhB8JaPayd43w37dxiXrW
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Claude <noreply@anthropic.com>

# Conflicts:
#	CHANGELOG.md
#	src/cli.js
@CodeWithJuber
CodeWithJuber marked this pull request as ready for review September 24, 2026 04:36
Signed-off-by: Claude <noreply@anthropic.com>

# Conflicts:
#	CHANGELOG.md
Signed-off-by: Claude <noreply@anthropic.com>

# Conflicts:
#	CHANGELOG.md
Signed-off-by: Claude <noreply@anthropic.com>

# Conflicts:
#	CHANGELOG.md
@CodeWithJuber
CodeWithJuber merged commit 24c9d0d into master Sep 24, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants