Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 35 additions & 2 deletions .github/workflows/claude-code-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,44 @@ jobs:
with:
fetch-depth: 1

# The review runs on z.ai (the owner, 2026-09-27: "замени ANTHROPIC_API_KEY
# на z.ai"). Every run before this failed two seconds in: the OAuth token
# was refused and ANTHROPIC_API_KEY was empty. z.ai serves the Anthropic
# Messages API at /api/anthropic, and Claude Code reaches it through
# ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN (docs.z.ai, the Claude Code
# guide). The key is the ZAI_API_KEY secret; a PR without it (a fork, or
# before the secret is set) is told so and not reviewed.
- name: Check for the z.ai key
id: zai
env:
ZAI_API_KEY: ${{ secrets.ZAI_API_KEY }}
run: |
if [ -n "$ZAI_API_KEY" ]; then
echo "ready=true" >> "$GITHUB_OUTPUT"
else
echo "ready=false" >> "$GITHUB_OUTPUT"
echo "::notice title=Review skipped::The ZAI_API_KEY secret is not set, so this PR is not reviewed. Add it under Settings > Secrets and variables > Actions."
fi

- name: Run Claude Code Review
id: claude-review
if: steps.zai.outputs.ready == 'true'
uses: anthropics/claude-code-action@v1
env:
ANTHROPIC_BASE_URL: https://api.z.ai/api/anthropic
ANTHROPIC_AUTH_TOKEN: ${{ secrets.ZAI_API_KEY }}
# z.ai's guide raises Claude Code's request timeout for GLM.
API_TIMEOUT_MS: "3000000"
# Background and subagent calls ask for a Haiku, Sonnet or Opus by
# name; these send them to GLM instead. ZAI_MODEL (a repository
# variable) picks the model; glm-4.6 is the default because it is the
# z.ai id the Queen's own model catalogue lists.
ANTHROPIC_DEFAULT_OPUS_MODEL: ${{ vars.ZAI_MODEL || 'glm-4.6' }}
ANTHROPIC_DEFAULT_SONNET_MODEL: ${{ vars.ZAI_MODEL || 'glm-4.6' }}
ANTHROPIC_DEFAULT_HAIKU_MODEL: glm-4.5-air
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# The action requires a key input; it is the same z.ai key.
anthropic_api_key: ${{ secrets.ZAI_API_KEY }}
# Note: the log suggests `show_full_output: true`, but that is not an
# input this action version accepts -- its inputs are trigger_phrase,
# prompt, settings, claude_args and the auth/provider set. Extra
Expand All @@ -60,5 +93,5 @@ jobs:

# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options
claude_args: '--verbose --allowed-tools "Bash(gh issue view:*),Bash(gh search:*),Bash(gh issue list:*),Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*)"'
claude_args: '--model ${{ vars.ZAI_MODEL || ''glm-4.6'' }} --verbose --allowed-tools "Bash(gh issue view:*),Bash(gh search:*),Bash(gh issue list:*),Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*)"'

6 changes: 6 additions & 0 deletions .github/workflows/website-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,12 @@ jobs:
- name: Queen WARS experiment ledger
run: npm run check:wars && npm run test:wars-spec

# LEVEL II on the ROADMAP tab: the raid, honey, bosses, cracks and the
# round's pulse. The raid list is pinned here and in gHashTag/t27's
# feed_roadmap.py, so the page and the feeder raid the same sector.
- name: Roadmap game rules
run: npm run check:roadmap-game

- name: Build
run: npx vite build

Expand Down
1 change: 1 addition & 0 deletions apps/website/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@
"check:onboarding": "node scripts/onboarding-from-spec.mjs --check",
"check:wars": "node scripts/queen-wars-from-spec.mjs --check && node --experimental-strip-types qa/queen-wars-contract.mjs",
"test:wars-spec": "node --test scripts/queen-wars-from-spec.test.mjs",
"check:roadmap-game": "node --experimental-strip-types qa/roadmap-game-contract.mjs",
"test:viewport-spec": "node --test scripts/viewport-from-spec.test.mjs",
"check:explorer-viewport": "node qa/explorer-viewport-contract.mjs",
"check:passport": "node --experimental-strip-types qa/passport-figures.mjs"
Expand Down
41 changes: 41 additions & 0 deletions apps/website/public/queen/runs/campaign-20260926/base-4613.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Queen WARS acceptance transcript: gHashTag/t27#4613
# worktree: /home/user/ghashtag/t27
# head: afe2186cfa9572c8c2d519506aa85a8c9447eae5
# judge: t27c 0.4.0; zig 0.16.0
# at: 2026-09-26T18:59:37Z
# diff --numstat: (no change)

## criterion 1: FAIL (expected 1)
$ t27c coverage specs/base/ternary_encoding.t27 2>&1 | grep -cE '^Untested: +0$'
0

## criterion 2: PASS (expected 13)
$ grep -cE '^[[:space:]]*(pub[[:space:]]+)?fn[[:space:]]' specs/base/ternary_encoding.t27
13

## criterion 3: FAIL (expected >= 11)
$ grep -cE '^[[:space:]]*test[[:space:]]+("|[A-Za-z_])' specs/base/ternary_encoding.t27
10

## criterion 4: PASS (expected IMPLEMENTED)
$ t27c spec-status specs/base/ternary_encoding.t27
IMPLEMENTED

## review FR-002 parse
$ t27c parse specs/base/ternary_encoding.t27 >/dev/null 2>/tmp/t27c_parse_err; echo exit=$?; head -3 /tmp/t27c_parse_err
exit=0

## review vacuity
$ t27c validate-vacuity --specs-dir specs/base --top 500 2>&1 | grep -F ternary_encoding.t27 || echo "ternary_encoding.t27: not listed as vacuous"
ternary_encoding.t27: not listed as vacuous

## review test-report
$ t27c test-report specs/base/ternary_encoding.t27 2>&1 | tail -15
--- test report: specs/base/ternary_encoding.t27 ---
BLOCKED does not compile: /tmp/t27c-test-report-ternary_encoding/spec.zig:8:9: error: assertion failed
/tmp/t27c-test-report-ternary_encoding/spec.zig:280:47: note: called at comptime here

A blocked spec is not a failing one. It never produced a
binary, so it has no per-test result to report.

# ACCEPTANCE: FAILED
46 changes: 46 additions & 0 deletions apps/website/public/queen/runs/campaign-20260926/base-4614.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Queen WARS acceptance transcript: gHashTag/t27#4614
# worktree: /home/user/ghashtag/t27
# head: afe2186cfa9572c8c2d519506aa85a8c9447eae5
# judge: t27c 0.4.0; zig 0.16.0
# at: 2026-09-26T18:51:51Z
# diff --numstat: (no change)

## criterion 1: FAIL (expected 1)
$ t27c coverage specs/boards/arty_a7.t27 2>&1 | grep -cE '^Untested: +0$'
0

## criterion 2: PASS (expected 5)
$ grep -cE '^[[:space:]]*(pub[[:space:]]+)?fn[[:space:]]' specs/boards/arty_a7.t27
5

## criterion 3: FAIL (expected >= 18)
$ grep -cE '^[[:space:]]*test[[:space:]]+("|[A-Za-z_])' specs/boards/arty_a7.t27
17

## criterion 4: PASS (expected IMPLEMENTED)
$ t27c spec-status specs/boards/arty_a7.t27
IMPLEMENTED

## criterion 5: PASS (expected 0)
$ t27c test-report specs/boards/arty_a7.t27 2>&1 | grep -c BLOCKED
0

## review FR-002 parse
$ t27c parse specs/boards/arty_a7.t27 >/dev/null 2>/tmp/t27c_parse_err; echo exit=$?; head -3 /tmp/t27c_parse_err
exit=0

## review vacuity
$ t27c validate-vacuity --specs-dir specs/boards --top 500 2>&1 | grep -F arty_a7.t27 || echo "arty_a7.t27: not listed as vacuous"
specs/boards/arty_a7.t27 0 0 0.0% 0 0

## review test-report
$ t27c test-report specs/boards/arty_a7.t27 2>&1 | tail -15
--- test report: specs/boards/arty_a7.t27 ---

tests 17
pass 17
FAIL 0
invariants 11 proved -- comptime, so compiling IS the check
rate 100.0%

# ACCEPTANCE: FAILED
46 changes: 46 additions & 0 deletions apps/website/public/queen/runs/campaign-20260926/base-4695.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Queen WARS acceptance transcript: gHashTag/t27#4695
# worktree: /home/user/ghashtag/t27
# head: afe2186cfa9572c8c2d519506aa85a8c9447eae5
# judge: t27c 0.4.0; zig 0.16.0
# at: 2026-09-26T18:59:36Z
# diff --numstat: (no change)

## criterion 1: FAIL (expected 1)
$ t27c coverage specs/fpga/testbench/simulator_tb.t27 2>&1 | grep -cE '^Untested: +0$'
0

## criterion 2: PASS (expected 4)
$ grep -cE '^[[:space:]]*(pub[[:space:]]+)?fn[[:space:]]' specs/fpga/testbench/simulator_tb.t27
4

## criterion 3: FAIL (expected >= 7)
$ grep -cE '^[[:space:]]*test[[:space:]]+("|[A-Za-z_])' specs/fpga/testbench/simulator_tb.t27
6

## criterion 4: PASS (expected IMPLEMENTED)
$ t27c spec-status specs/fpga/testbench/simulator_tb.t27
IMPLEMENTED

## criterion 5: PASS (expected 0)
$ t27c test-report specs/fpga/testbench/simulator_tb.t27 2>&1 | grep -c BLOCKED
0

## review FR-002 parse
$ t27c parse specs/fpga/testbench/simulator_tb.t27 >/dev/null 2>/tmp/t27c_parse_err; echo exit=$?; head -3 /tmp/t27c_parse_err
exit=0

## review vacuity
$ t27c validate-vacuity --specs-dir specs/fpga/testbench --top 500 2>&1 | grep -F simulator_tb.t27 || echo "simulator_tb.t27: not listed as vacuous"
specs/fpga/testbench/simulator_tb.t27 7 0 0.0% 1 0

## review test-report
$ t27c test-report specs/fpga/testbench/simulator_tb.t27 2>&1 | tail -15
--- test report: specs/fpga/testbench/simulator_tb.t27 ---

tests 6
pass 6
FAIL 0
invariants 1 proved -- comptime, so compiling IS the check
rate 100.0%

# ACCEPTANCE: FAILED
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
#!/usr/bin/env python3
"""Queen WARS acceptance judge: run an issue's own acceptance commands in a worktree.

Usage: accept.py <issue> <worktree> <transcript-out>

Runs every acceptance criterion exactly as the issue states it, from the
worktree root, with the judge t27c and zig 0.16.0 on PATH, and writes a
transcript: command, raw output, expected value, pass/fail. Then two review
checks the issue's requirements name (FR-002 parse, vacuity) and the spec's
own test-report summary. Exit 0 iff every acceptance criterion passes.
"""
import datetime
import os
import subprocess
import sys

JUDGE_PATH = "/home/user/wars/judge:/home/user/tools/bin"

SPECS = {
"4614": ("specs/boards/arty_a7.t27", 5, 18, True),
"4695": ("specs/fpga/testbench/simulator_tb.t27", 4, 7, True),
"4613": ("specs/base/ternary_encoding.t27", 13, 11, False),
}


def sh(cmd, cwd):
env = dict(os.environ, PATH=JUDGE_PATH + ":" + os.environ["PATH"])
p = subprocess.run(["bash", "-c", cmd], cwd=cwd, env=env,
capture_output=True, text=True, timeout=900)
return (p.stdout + p.stderr).strip()


def main():
issue, wt, out = sys.argv[1], sys.argv[2], sys.argv[3]
spec, n_fn, min_tests, has_c5 = SPECS[issue]
crit = [
("1", f"t27c coverage {spec} 2>&1 | grep -cE '^Untested: +0$'", lambda o: o == "1", "1"),
("2", f"grep -cE '^[[:space:]]*(pub[[:space:]]+)?fn[[:space:]]' {spec}",
lambda o: o == str(n_fn), str(n_fn)),
("3", f"grep -cE '^[[:space:]]*test[[:space:]]+(\"|[A-Za-z_])' {spec}",
lambda o: o.isdigit() and int(o) >= min_tests, f">= {min_tests}"),
("4", f"t27c spec-status {spec}", lambda o: "IMPLEMENTED" in o.split(), "IMPLEMENTED"),
]
if has_c5:
crit.append(("5", f"t27c test-report {spec} 2>&1 | grep -c BLOCKED",
lambda o: o == "0", "0"))
lines = [f"# Queen WARS acceptance transcript: gHashTag/t27#{issue}",
f"# worktree: {wt}",
f"# head: {sh('git rev-parse HEAD', wt)}",
f"# judge: {sh('t27c --version', wt)}; zig {sh('zig version', wt)}",
f"# at: {datetime.datetime.now(datetime.timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ')}",
f"# diff --numstat: {sh('git diff --numstat', wt) or '(no change)'}",
""]
ok_all = True
for n, cmd, ok, want in crit:
o = sh(cmd, wt)
passed = ok(o)
ok_all &= passed
lines += [f"## criterion {n}: {'PASS' if passed else 'FAIL'} (expected {want})",
f"$ {cmd}", o, ""]
for label, cmd in (("review FR-002 parse", f"t27c parse {spec} >/dev/null 2>/tmp/t27c_parse_err; echo exit=$?; head -3 /tmp/t27c_parse_err"),
("review vacuity", f"t27c validate-vacuity --specs-dir {os.path.dirname(spec)} --top 500 2>&1 | grep -F {os.path.basename(spec)} || echo \"{os.path.basename(spec)}: not listed as vacuous\""),
("review test-report", f"t27c test-report {spec} 2>&1 | tail -15")):
lines += [f"## {label}", f"$ {cmd}", sh(cmd, wt), ""]
lines.append(f"# ACCEPTANCE: {'PASSED' if ok_all else 'FAILED'}")
open(out, "w").write("\n".join(lines) + "\n")
print(f"#{issue}: {'PASSED' if ok_all else 'FAILED'} -> {out}")
return 0 if ok_all else 1


if __name__ == "__main__":
sys.exit(main())
114 changes: 114 additions & 0 deletions apps/website/public/queen/runs/campaign-20260926/judge/mutate.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
#!/usr/bin/env python3
"""Queen WARS review: does an arm's NEW test catch a fixed set of mutants?

Usage: mutate.py <issue> <worktree> <new-test-name> <transcript-out>

For each mutant of the function under test (fixed per issue, the same for
every arm), rewrite the arm's patched spec, run `t27c test-report --verbose`,
and record whether the arm's new test FAILs (killed), passes (survived) or
the spec does not build (blocked). The arm's file is restored byte-for-byte
after every mutant, and the restore is checked.
"""
import os
import subprocess
import sys

PATH = "/home/user/wars/judge:/home/user/tools/bin:" + os.environ["PATH"]

MUTANTS = {
"4614": ("specs/boards/arty_a7.t27", [
("return NUM_BUTTONS + 1", "fn count_buttons() -> usize {\n return NUM_BUTTONS;",
"fn count_buttons() -> usize {\n return NUM_BUTTONS + 1;"),
("return 0", "fn count_buttons() -> usize {\n return NUM_BUTTONS;",
"fn count_buttons() -> usize {\n return 0;"),
("return NUM_LEDS - 1", "fn count_buttons() -> usize {\n return NUM_BUTTONS;",
"fn count_buttons() -> usize {\n return NUM_LEDS - 1;"),
]),
"4695": ("specs/fpga/testbench/simulator_tb.t27", [
("cycle += 2", " sim_cycle = sim_cycle + 1;\n }\n\n fn reset",
" sim_cycle = sim_cycle + 2;\n }\n\n fn reset"),
("clock ends low", " clk = false;\n clk = true;\n sim_cycle = sim_cycle + 1;",
" clk = true;\n clk = false;\n sim_cycle = sim_cycle + 1;"),
("no cycle advance", " sim_cycle = sim_cycle + 1;\n }\n\n fn reset",
" sim_cycle = sim_cycle;\n }\n\n fn reset"),
("tick also counts an event", " sim_cycle = sim_cycle + 1;\n }\n\n fn reset",
" sim_cycle = sim_cycle + 1;\n events_processed = events_processed + 1;\n }\n\n fn reset"),
]),
"4613": ("specs/base/ternary_encoding.t27", [
("never rejects", " if (!is_valid_trit(trits[i])) {\n return false;",
" if (false) {\n return false;"),
("checks only the first slot", "fn validate_trits(trits: []i32, len: usize) → bool {\n var i : usize = 0;\n while (i < len) {",
"fn validate_trits(trits: []i32, len: usize) → bool {\n var i : usize = 0;\n while (i < 1) {"),
("skips the last slot", "fn validate_trits(trits: []i32, len: usize) → bool {\n var i : usize = 0;\n while (i < len) {",
"fn validate_trits(trits: []i32, len: usize) → bool {\n var i : usize = 0;\n while (i + 1 < len) {"),
("rejects everything", " i = i + 1;\n }\n return true;\n }\n\n // ═",
" i = i + 1;\n }\n return false;\n }\n\n // ═"),
]),
}

# Pre-existing blockers in ternary_encoding.t27 at afe2186c, outside #4613's
# boundary: two comptime invariants that fail (balanced encoders, unipolar
# decoders) and two tests that discard a return value. Removed only in the
# review copy, so an arm's new test can be run at all.
DEBLOCK = {"4613": ["invariant byte_trits_roundtrip", "invariant bits_trits_roundtrip",
"test byte_to_trits_roundtrip", "test char_encoding_roundtrip"]}


def deblock(text, names):
out, skip = [], False
for line in text.split("\n"):
head = line[4:] if line.startswith(" ") and not line.startswith(" ") else None
if head is not None or line.startswith("}"):
skip = head is not None and any(head == n or head.startswith(n + " ")
for n in names)
if not skip:
out.append(line)
return "\n".join(out)


def main():
issue, wt, test_name, out = sys.argv[1:5]
source = sys.argv[5] if len(sys.argv) > 5 else None # arm file to review in wt
spec, mutants = MUTANTS[issue]
path = os.path.join(wt, spec)
if source:
text = deblock(open(source).read(), DEBLOCK[issue])
open(path, "w").write(text)
original = open(path, "rb").read()
lines, killed = [f"# mutation review: gHashTag/t27#{issue}, new test `{test_name}`"
+ (f" (de-blocked copy of {source})" if source else ""), ""], 0
if source:
r = subprocess.run(["bash", "-c", f"t27c test-report --verbose {spec} 2>&1"],
cwd=wt, env=dict(os.environ, PATH=PATH),
capture_output=True, text=True, timeout=900)
lines += ["## unmutated de-blocked copy", r.stdout.strip(), ""]
for label, old, new in mutants:
text = original.decode()
if text.count(old) != 1:
lines.append(f"## {label}: SKIPPED (anchor not unique: {text.count(old)})")
continue
try:
open(path, "w").write(text.replace(old, new))
r = subprocess.run(["bash", "-c", f"t27c test-report --verbose {spec} 2>&1"],
cwd=wt, env=dict(os.environ, PATH=PATH),
capture_output=True, text=True, timeout=900)
rep = r.stdout
finally:
open(path, "wb").write(original)
assert open(path, "rb").read() == original, "restore failed"
if "BLOCKED" in rep:
verdict = "blocked"
elif f"FAIL {test_name}" in rep:
verdict, killed = "killed", killed + 1
elif f"pass {test_name}" in rep:
verdict = "survived"
else:
verdict = "not-run"
lines += [f"## {label}: {verdict}", rep.strip(), ""]
lines.append(f"# MUTANTS KILLED BY THE NEW TEST: {killed}/{len(mutants)}")
open(out, "w").write("\n".join(lines) + "\n")
print(f"#{issue} {os.path.basename(wt)}: {killed}/{len(mutants)} killed -> {out}")


if __name__ == "__main__":
main()
Loading
Loading