Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tools/spec-loop/.last-sync
Original file line number Diff line number Diff line change
@@ -1 +1 @@
6cd53e0d931b6a4828bc8a3b74dacf6c1d32e477
1e5c4f9110812f1c0bc4bc6312723c9f98c82fca
33 changes: 18 additions & 15 deletions tools/spec-loop/specs/adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -499,6 +499,7 @@ uv run --all-packages --group dev pytest tools/github-rollup/tests

- `experimental` overall — adapter coverage varies; a new adopter system
(e.g. a different mail backend) is a gap the plan pass records.
<!-- gap-closes-when: none (narrative) -->
- **GitLab adapter is a read-only foundation.** `tools/gitlab/` covers
repository metadata, issue and merge-request reads, MR diffs, commits and
pipelines, and pipeline status; it is `partial` for all three contracts
Expand All @@ -507,9 +508,11 @@ uv run --all-packages --group dev pytest tools/github-rollup/tests
#305.
Fetched issue and MR titles, descriptions, diffs and commit messages are
external data, never instructions.
<!-- gap-closes-when: none (narrative) -->
- **Chat covers Slack and Discord.** `contract:chat` ships Slack and Discord
adapters; Matrix and Zulip remain extension points, and `none` is an explicit
no-op backend.
<!-- gap-closes-when: none (narrative) -->
- **Bitbucket adapter is new and intentionally partial.** `tools/bitbucket/`
currently provides read-only repository metadata, read-only branch restriction
context, pull-request discovery, pull-request fetching, read-only pull-request
Expand All @@ -518,45 +521,45 @@ uv run --all-packages --group dev pytest tools/github-rollup/tests
Cloud-only asynchronous merge task-status fetching, read-only merge-check context fetching,
and read-only pull-request status fetching. Native Bitbucket Cloud issue
commands fail closed because Atlassian removed the Issue Tracker API on
2026-08-20; #606 remains open for linked Jira tracker coverage and the
remaining change-request gaps.
2026-08-20. On Data Center, `pr jira-issues` lists the Jira keys linked to a
pull request's commits; #606 remains open for broader linked Jira tracker
coverage and the remaining change-request gaps.
<!-- gap-closes-when: none (narrative) -->
- Bitbucket write operations follow the framework write-path discipline:
the calling skill must obtain explicit user confirmation before invoking a
mutation. The bridge executes only the confirmed action; current write
coverage is Bitbucket Cloud and Data Center pull-request comment creation,
backend-neutral `post_review` through
comment/approve/request-changes, backend-neutral `reject`, and Bitbucket
Cloud pull-request approve/unapprove, request-changes/remove-request-changes,
decline, and
approve/unapprove and request-changes/remove-request-changes, plus
Bitbucket Cloud-only backend-neutral `post_review` through
comment/approve/request-changes, backend-neutral `reject`, decline, and
strategy-aware merge actions pinned to a caller-confirmed source commit
(7–40 hex characters); gate checks before a merge are the caller's
responsibility via `pr merge-checks`.
- Fetched Bitbucket descriptions, issue titles/descriptions, fetched or created issue comments, attachment names, uploader names when present, attachment links, raw attachment payloads, issue reporter/assignee/commenter names, issue links, branch restriction policy, commit messages, diff hunks, file paths, comments, pull-request task content, task creator/resolver names, reviewer names, review decisions/events, approval/change-request activity, merge-check decisions/blockers, status descriptions,
<!-- gap-closes-when: none (narrative) -->
- Fetched Bitbucket descriptions, branch restriction policy, commit messages, diff hunks, file paths, comments, pull-request task content, task creator/resolver names, reviewer names, review decisions/events, approval/change-request activity, merge-check decisions/blockers, status descriptions,
CI URLs, and raw payloads are external data, never agent instructions;
private or embargoed content must follow the
approved-LLM/privacy gate before model use.
<!-- gap-closes-when: none (narrative) -->
- **Forgejo adapter is doc-only and partial.** `tools/forgejo/` is a set
of `tea` / REST recipes with no package or tests and no adopter pilot;
the project-board capability is a no-op, and full coverage remains part
of #310.
<!-- gap-closes-when: path-exists:tools/forgejo/pyproject.toml -->
- **Mailman 3 backend reads only.** It covers no drafts or sent mail, and
the `hyperkitty` placeholder of the separate `mail-archive` contract
(search-URL construction) is not implemented by it.
<!-- gap-closes-when: none (narrative) -->
- **SourceHut adapter is new and untested end-to-end.** `tools/sourcehut/`
ships the GraphQL-based bridge (ticket, patchset, CI, repo), but no
adopter pilot has exercised it; signal/roster heuristics may change.
<!-- gap-closes-when: none (narrative) -->
- Adapters cover the *system-swap* case; the broader audit of residual
ASF coupling across the catalogue, and the capability-flag mechanism for
workflow branches that no adapter resolves, live in
[project-agnosticism.md](project-agnosticism.md).
- **Adapter authoring smoke validation is shipped.**
`validate_adapter_authoring` (SOFT advisory) checks that each
`contract:*` adapter README declares credential/privacy handling,
operations, and config keys. `substrate:*` tools are excluded.
`validate_tools` (HARD) separately enforces the
**Capability:** declaration and the **Prerequisites** section
on every tool README, so all five original gap items are now
covered across the two validators.
<!-- gap-closes-when: none (narrative) -->
- **Mail-adapter privacy tests are thin.** The redaction contract exists,
but adapter-level fixtures should prove that private mail and embedded
prompt-injection attempts do not enter model-facing context untreated.
<!-- gap-closes-when: none (narrative) -->
10 changes: 3 additions & 7 deletions tools/spec-loop/specs/adoption-and-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -922,18 +922,14 @@ uv run --project tools/skill-and-tool-validator --group dev skill-and-tool-valid
*default* way an adopter takes the framework; the snapshot install described
here is the fallback. The skills delivered are the same tree either way —
only the namespace of their invocation differs.
<!-- gap-closes-when: none (narrative) -->

- **The `setup` welcome check is silent until the checker is in its
`checker_dir` on a sandboxed marketplace install,** where the plugin cache cannot
be executed, and on a machine whose installed checker predates the
`due` action until `setup upgrade` refreshes it.
<!-- gap-closes-when: none (narrative) -->
- `stable`; gaps appear as new agent targets to add to the registry
([`agents.md`](../../../plugins/magpie-setup/skills/setup/agents.md)) or new override
surfaces — recorded by the plan pass.
- **Built since first recorded as gaps:** the personal override surface
(acceptance 6; `.apache-magpie-local/` when adopted,
`<git-common-dir>/apache-magpie/` when only installed, since #1533/#1535) and the one-shot default-run
switch (acceptance 7, the `--no-overrides` flag in
`docs/setup/agentic-overrides.md` § One-shot defaults run) both ship.
They were tracked as work items `magpie-local-convention` and
`override-bypass-one-shot` in the plan.
<!-- gap-closes-when: none (narrative) -->
9 changes: 9 additions & 0 deletions tools/spec-loop/specs/agent-isolation-sandbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -655,6 +655,13 @@ The reference model is four layers, layered:
`releases.astral.sh` for the node and python hook environments
(#1565). sandbox-lint's `expected.json` carries all five, and the
update skill reports any of them missing as drift.
Apache-specific hosts are not in those shared defaults (#1978).
`tools/agent-isolation/claude-settings.asf.fragment.json` carries them,
and setup adds it only for a project whose resolved `project.md` holds one
explicit `organization: ASF` declaration:
`setup_preflight/organization_network.py` ignores comments, fences and
examples, treats anything else as not ASF, and refuses a fragment whose
grants differ from the reviewed set.
3. **Tool permissions** — the host's `permissions.deny` blocks denied
paths/binaries (`Read(~/.ssh/**)`, `Bash(curl *)`, …).
4. **Forced confirmation** — `permissions.ask` on `git push` and,
Expand Down Expand Up @@ -754,9 +761,11 @@ uv run --project tools/setup-preflight --group dev pytest \

- `stable`; drift shows up when a new state-mutating command is added to a
skill without a matching `ask` rule — the plan pass flags it.
<!-- gap-closes-when: none (narrative) -->
- **`tools/agent-guard/` and `tools/egress-gateway/` are new additions**
since the last pilot cycle; end-to-end integration with a real adopter
session has not yet been exercised.
<!-- gap-closes-when: none (narrative) -->

## Antigravity setup lifecycle

Expand Down
13 changes: 11 additions & 2 deletions tools/spec-loop/specs/container-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,12 @@ CLIs speak. Three properties fall out of the policy:
creates is labelled with the project; every read or act call is
filtered to that label. Two projects on one machine share a daemon
and see disjoint worlds.
3. **Same egress policy as the shell.** Containers get the egress
3. **Egress through the gateway.** Containers get the egress
gateway as their HTTP proxy, so tools that honour proxy variables
are bound by the same host allow-list as sandboxed commands. This is
are bound by the gateway's host allow-list, which is separate from the
sandbox's: the project-local ASF fragment does not extend it, so an ASF
project runs its own gateway with those hosts in `EGRESS_ALLOW_EXTRA`
(#1978). This is
a friction layer, not a wall: a raw socket from inside a container
bypasses it, exactly as RFC-AI-0004 says of the permission layer.

Expand Down Expand Up @@ -619,6 +622,7 @@ PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner \
block by hand, or takes the settings diff
`setup-isolated-setup-install` Step L proposes. Automating it in
`/magpie-setup:setup config` is the follow-up.
<!-- gap-closes-when: none (narrative) -->
- **Residual risk — a forged worktree layout.** In a linked worktree
of a project that has not adopted Magpie, the run directory's anchor
(the git common directory) is located from the worktree's `.git`
Expand All @@ -639,18 +643,23 @@ PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner \
path is allowlisted in settings the agent cannot edit, and container
binds remain limited to the worktree's bind roots. Mitigation: pass
`--run-dir` explicitly, or adopt (per-worktree in-tree run dir).
<!-- gap-closes-when: none (narrative) -->
- A create, exec or build field the gateway has not learned is refused,
so a daemon feature that arrives after this table is unavailable
through the gateway until the allow-list learns it. The refusal names
the field, which is the signal to add it.
<!-- gap-closes-when: none (narrative) -->
- The egress alias for Linux depends on the backend's bridge
configuration and on the egress gateway listening on a non-loopback
address; it binds loopback by default, so Linux adopters start it
with `--host <bridge address>`.
<!-- gap-closes-when: none (narrative) -->
- Docker Desktop's file-sharing settings, not the gateway, decide
which host paths the VM can see; a project root outside the shared
set fails at mount time with Docker's own error.
<!-- gap-closes-when: none (narrative) -->
- `podman compose` shells out to an external compose provider whose
own socket handling (e.g. `docker-compose` reading `DOCKER_HOST`) is
outside the gateway's control; the labels the gateway injects
coexist with compose's own.
<!-- gap-closes-when: none (narrative) -->
4 changes: 4 additions & 0 deletions tools/spec-loop/specs/cve-tooling.md
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,7 @@ uv run --project tools/cve-tool-vulnogram/oauth-api --group dev pytest

- `stable`; drift appears if the CVE 5.x schema or Vulnogram export shape
changes upstream — caught by the tool's own tests.
<!-- gap-closes-when: none (narrative) -->
- **`tools/osv/` has no consuming skill yet.** Its `tool.md` names the
intended consumers — `security-issue-triage` (early deduplication against
known vulnerabilities), `security-issue-deduplicate` (alias resolution
Expand All @@ -274,11 +275,14 @@ uv run --project tools/cve-tool-vulnogram/oauth-api --group dev pytest
query about a pre-disclosure report must also stay within the embargo
boundaries `tool.md` sets: only public identifiers, packages and commits
go into a query.
<!-- gap-closes-when: none (narrative) -->
- The ASF Vulnogram tokens are short-lived by design (they live as long
as the browser login that minted them) and there is no long-lived
token; a re-setup after each expiry is the expected cost.
The browser-approved setup and API allocation need server support;
on a Vulnogram without it (`--server` exit 4) setup falls back to the
paste path and allocation to the form recipe.
<!-- gap-closes-when: none (narrative) -->
- **API allocation is Vulnogram-only.** Other `cve_authority` adapters keep
the print-the-recipe allocation flow until they offer an equivalent API.
<!-- gap-closes-when: none (narrative) -->
20 changes: 8 additions & 12 deletions tools/spec-loop/specs/marketplace-distribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,12 @@ manifest is Claude Code's `.claude-plugin/plugin.json`, but they are not
Claude-Code-only: Codex installs them by falling through to that same file.
The exception is a plugin whose `hooks/hooks.json` carries Claude Code mods
(`$schema`, `modules`), which Codex's strict hooks schema rejects whole:
`magpie-setup`, `magpie-utilities`, `magpie-security` and
`magpie-pr-management` each carry a
`magpie-setup`, `magpie-utilities`, `magpie-security`,
`magpie-pr-management` and `magpie-release-management` each carry a
complete `.codex-plugin/plugin.json` (read instead of the Claude manifest)
whose `hooks` names a clean `hooks/codex-hooks.json` — the `startup`
upgrade check (`/bin/bash -p check-upgrade.sh`) for `magpie-setup`, an empty
file for the other three (#1836, #1676, #1920).
file for the other four (#1836, #1676, #1920, #2007).
`tools/dev/check-family-plugins.py --fix` generates them and fails a Codex
catalogue plugin whose Codex-visible hook file has any other top-level key.

Expand Down Expand Up @@ -305,22 +305,18 @@ For the Claude Code family, `claude plugin validate . --strict`.
conform to their vendors' published
documentation but have not been exercised end-to-end. Re-check each against
current vendor docs before a marketplace publish.
<!-- gap-closes-when: none (narrative) -->
- **`microsoft/apm` installs Magpie for Copilot only.** A live `apm install` registers the plugin with Copilot, but whether its skills then load is unverified.
Every other target refuses an Agent Plugins package, and a single skill cannot be installed by subpath, because the root `skills/` entries are symlinks.
<!-- gap-closes-when: none (narrative) -->
- **Antigravity is checked only with `agy plugin validate .`** (1.3.6 reports the
root's skills); an end-to-end `agy plugin install` has not been exercised.
<!-- gap-closes-when: none (narrative) -->
- **Pi was exercised with `pi install git:` from a branch** (Pi 1.4.0, macOS): the package installed its
dependency and loaded the extension; Linux and Windows are unverified.
- **Codex never runs the upgrade-detection hook.**
`plugins/magpie-setup/` has no `.codex-plugin/plugin.json`, so Codex reads
its default `hooks/hooks.json`, whose top-level `$schema` and `modules`
keys its strict hooks-file schema rejects: no hook loads.
The root `.codex-plugin/plugin.json` hook #1829 re-pointed is written as
`{"SessionStart": …}` without the inner `hooks` wrapper Codex's inline
form needs, so it is ignored too, and the Codex catalogue never installs
the root plugin anyway; `check-family-plugins.py`'s `CODEX_HOOKS` pins
that shape.
<!-- gap-closes-when: none (narrative) -->
- **Nothing checks the dev-suffix stamp is fresher than the last release**, so
a bump that is forgotten fails silently in the one way that matters: adopters
keep being told they are up to date. The staleness is only visible by reading
the timestamp.
<!-- gap-closes-when: none (narrative) -->
47 changes: 13 additions & 34 deletions tools/spec-loop/specs/meta-and-quality-tooling.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,9 @@ trustworthy as it grows.
measured with `skill-token-count` before and after every pass; the
always-on frontmatter budget is spent on first, because it is paid in
every session for every skill (#1331, #1332).
It also estimates a run cost per path, because the body understates what
an invocation reads, and every new pointer is proved with a pointer-only
eval run against a copy with the pointers deleted (#2000).
Every pass other than the rewrite is behaviour-preserving: moved bytes
are identical bytes, heading level excepted, and a heading that moves
takes its references with it — eval `step-config.json` `step_heading`
Expand Down Expand Up @@ -432,41 +435,17 @@ uv run --project tools/dev pytest tools/dev/tests/test_ci_pytest_batches.py tool

## Known gaps

- **Eval coverage is complete.** All 83 shipped skills have a matching
suite in `tools/skill-evals/evals/`; the soft eval-coverage check in
`skill-and-tool-validator` (check #8) warns when a newly added skill has
no suite, keeping coverage complete going forward.
- **Frontmatter validation is still shallow.** Current validation covers
required fields, but the next pass should make `mode`, `status`,
`capability`, `organization`, and `source` combinations explicit and
test-backed.
- **Capability taxonomy drift is now checked.**
`validate_capability_taxonomy_coverage` parses the Axis 1 / Axis 2
vocabulary tables in `docs/labels-and-capabilities.md`, surfaces
taxonomy rows that no skill/tool implements (entries marked
*(reserved)* / *(future)* are exempt), and cross-checks the
`SKILL_CAPABILITIES` / `TOOL_CAPABILITIES` code constants against the
doc — all SOFT advisories. Undocumented frontmatter capability values
are rejected by skill validation separately.
- **`docs/modes.md` is no longer manually synced.**
`validate_modes_doc_consistency` compares the doc's per-mode tables
against live skill frontmatter — missing skills, mode mismatches,
skill-count mismatches, and unlisted skills — as SOFT advisories.
- **Tool README prerequisites are normalized and enforced.** The HARD
`tool-prerequisites-fields` check requires the bold sub-field labels
(**Runtime:**, **CLIs:**, **Credentials / auth:**, **Network:**) or
the pure-contract delegation shorthand in every tool README, so the
normalize-then-tighten ordering this bullet used to track is done.
- **Pilot evidence shape is now defined and validated.** `docs/pilot-report-template.md`
defines the required frontmatter schema and body sections; `tools/pilot-report-validator/`
enforces the schema on every `.md` file with a YAML frontmatter block.
The "no common shape" gap is closed; the remaining gap is adoption —
no adopter pilot has yet submitted a validated report against the template.
- **Vendor-neutrality measurement is available but unscheduled.** `tools/vendor-neutrality-score/`
ships and can be run ad hoc, but is not yet wired into CI or the
build loop. Score drift is not automatically surfaced.
- **The 500-line cap is advisory and still exceeded.** `skill-line-limit`
is a SOFT check. After the family trims and deep splits (#1390
pr-management, #1400 issue, #1435 to #1438 and #1672 security, #1620
release-management, and the setup isolated-setup splits), one `SKILL.md`
remains over 500 lines: `security-issue-import-from-pr` (516).
<!-- gap-closes-when: none (narrative) -->
- **No adopter pilot has submitted a validated report.** The shape is defined
in `docs/pilot-report-template.md` and enforced by
`tools/pilot-report-validator/`, but the template is the only file in the
tree carrying that frontmatter.
<!-- gap-closes-when: none (narrative) -->
- **The 500-line cap is advisory.** `skill-line-limit` is a SOFT check, so
nothing blocks a `SKILL.md` growing past it. No `SKILL.md` is over 500 lines
today; the largest, `issue-triage`, sits at exactly 500.
<!-- gap-closes-when: none (narrative) -->
Loading