Skip to content

Rewrite the skills README around how the skill works - #552

Draft
Yuge Zhang (ultmaster) wants to merge 1 commit into
microsoft:mainfrom
ultmaster:docs/skills-readme-rewrite
Draft

Rewrite the skills README around how the skill works#552
Yuge Zhang (ultmaster) wants to merge 1 commit into
microsoft:mainfrom
ultmaster:docs/skills-readme-rewrite

Conversation

@ultmaster

Copy link
Copy Markdown
Contributor

Draft. This is an in-progress rewrite of skills/README.md, committed as-is from
the working tree — content unreviewed and unedited, opened for visibility while
it is still being worked on.

What changes

  • Leads with what the skill does for a coding agent instead of the Agent Skills
    format preamble.
  • Adds a How It Works section making the point that the harness is already a
    strong optimizer, with the baseline prompt it responds to, and positions the
    skill as boosting that.
  • Describes the experiment setup inline: which models drive the agents being
    optimized versus the optimizer harnesses, and that all non-coding-agent rows
    come from the SkillOpt paper.
  • Renames the last two table rows from "Agentic optimizer average" to
    "Coding Agent (Avg. of CC+Codex+GHCP)" / "Coding Agent (with Agent Lightning
    Skill)". Numbers are unchanged.
  • Replaces the two per-chart methodology sections with a single Performance
    Breakdowns
    section that explains the budget control and the finale-cost
    question.

Open items before this is ready

  • [agent-lightning] is used as a reference-style link but the file has no
    matching [agent-lightning]: ... definition, so it renders literally.
  • The finale-cost paragraph now says "Every point in the chart is an average of
    three runs". The text it replaced said each point is one of three runs. If
    the charts still plot individual replicates, this claim needs to go back.
  • Dropped explanations that the charts may still depend on: the ALFWorld $0
    finale-cost / jitter note, the dotted score-only ALFWorld baseline, and the
    pristine-baseline reference points.
  • Dropped the note that skills/agent-lightning/ is both the canonical Agent
    Skills package and the Claude Code plugin root.
  • The file now starts at ## with no # heading.
  • Typos: "The core files of the skill is in", "The model to to drive", "an more
    expensive model", "pervious experiment".

🤖 Generated with Claude Code

https://claude.ai/code/session_01SnuugZnz2GXy8F78zkSmcY

Lead with what the skill does for a coding agent, add a "How It Works"
section with the baseline prompt that harnesses already respond to, and
reframe the results as coding-agent rows rather than "agentic optimizer
average". Describe the budget-controlled setup and the finale-cost
question in the performance breakdowns instead of the per-chart
methodology notes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnuugZnz2GXy8F78zkSmcY
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant