Repository navigation
Add a skill that runs an agent evaluation round of the benchup tools - #64
Merged
Merged
Conversation
Collaborator
Author
|
@dannycoulombe in this PR I propose to include an initial skill, in an .agents dir (so agent-agnostic), to help with reviews/tests. We could include more related to the dev process. |
furilo
force-pushed
the
agent-eval-skill
branch
2 times, most recently
from
November 14, 2026 12:21
3daddd7 to
55bc85f
Compare
.agents/skills/benchup-agent-eval tells Claude Code or Codex how to run a round of crates/benchup-cli/evals: the regression test first, one subagent per case with its prompt verbatim, an independent check of every result with check_model and a render, and results/round-N.md with the findings sorted into tool bugs, guide gaps, core limits and missing features. .claude/skills links to .agents/skills so Claude Code finds it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
furilo
force-pushed
the
agent-eval-skill
branch
from
November 14, 2026 12:35
55bc85f to
937e871
Compare
This was referenced Nov 14, 2026
Merged
Resolve the scene.rs conflict by keeping joint_geometry and main's joint_kind comment. Main added pocket joints (biscuit, Domino, groove, loose tongue): joint_geometry reports only their region and depth, since their pocket sizes are already in params and a tenon-style cut does not describe them, and get_estimate uses the app's default minutes for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Take main's handling of pocket joints in joint_geometry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #63: review that one first. Until #63 is merged this PR also shows its two commits; the change here is only the last commit.
Adds a skill that runs a round of the agent evaluation in
crates/benchup-cli/evals, added in #63:.agents/skills/benchup-agent-eval/SKILL.md..agents/skillsis a location shared by agents (Codex reads it), which fits a repository that usesAGENTS.md. The skill tells the agent to:setup.py;check_modeland a render, instead of trusting the reports;results/round-N.md, comparing with the previous round and sorting the findings into tool bugs, guide gaps, core limits and missing features;.claude/skillsis a symlink to../.agents/skills, so Claude Code finds the same skill without a copy.A round costs tokens: the skill says to run one only when asked, and to state how many agents it will start. Rounds 5 and 6 in #63 were run this way.
🤖 Generated with Claude Code