Skip to content

Add an MCP server and optional agent add-ons (Agent connection, Build with MIRA) - #3

Open
aarvee11 wants to merge 17 commits into
BenchUp3D:mainfrom
aarvee11:feature/build-with-mira
Open

aarvee11 wants to merge 17 commits into
BenchUp3D:mainfrom
aarvee11:feature/build-with-mira

Conversation

@aarvee11

@aarvee11 aarvee11 commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Following the feedback here and the plan in #13, this PR now builds on an MCP-first design. MIRA, the in-app chat, is one optional client on top of it. Everything is opt-in and can be compiled out.

Refs #1, #13

What's in it

Piece What it does
crates/benchup-tools Shared operation layer: 27 woodworking tools with JSON Schemas, typed validation, limits, and the lock, visibility, edit-scope, read-only and lease checks, plus .bup encode/decode and an atomic save. It depends only on benchup-core, which is unchanged.
crates/benchup-mcp Stdio MCP server, with two modes. file <project.bup> edits a file on disk: one history entry per change, an atomic save, and it refuses to overwrite a file that changed on disk. live is the bridge from #13: it listens on 127.0.0.1 only, and the app (desktop or browser) connects out to it over WebSocket.
App: Agent connection Settings → Add-ons → Agent connection, off by default. It holds a per-device pairing token and copies the Claude Code command or MCP JSON. Each agent call is one undo step labelled Agent: ….
App: Build with MIRA The in-app chat (OpenAI-compatible endpoint; endlesstokens.ai is only the default), now running on benchup-tools. It's a separate add-on, on by default.
Shared host (agent_host.rs) Both add-ons run calls through the same code, so they follow the same rules. Agent calls that arrive during a MIRA turn wait and then run in order. Agent changes appear in the MIRA transcript as "Agent activity" cards with Undo.

Each add-on is a Cargo feature (assistant, agent-link, both on by default) with a stub module, so call sites have no cfg, plus a persisted run-time switch in Settings → Add-ons. Saved preferences survive builds that leave an add-on out.

Security (live mode)

  • The listener binds to loopback only. Browser Origins are limited to the hosted app and localhost dev servers.
  • Pairing is challenge-response HMAC-SHA256 in both directions, so the token is never sent, and the app ignores any program on the port that cannot prove it knows the token.
  • Calls carry deadlines and are cancelled on timeout. Calls queued for a project that is no longer open are refused.
  • Size limits apply before and after authentication, the auth window is 5 s, and unauthenticated connections are capped.
  • Shared projects keep their read-only and lease rules. A post-call check rolls back any change outside the lease. This also closes a gap that was in the first version of MIRA.

The threat model is in docs/mira/persistence-and-security.md.

Docs

  • docs/mcp.md: user guide (setup with Claude Code or any MCP client, both modes, troubleshooting).
  • docs/assistant.md: MIRA user guide.
  • docs/mira/: contributor docs (architecture, module reference, tool catalog, protocol, security, testing, roadmap, development history).

Testing

  • cargo test -p benchup-tools: 31 pass. cargo test -p benchup-mcp: 19 pass.
  • App tests:
    • agent: 35 pass, including 3 end-to-end tests of the real bridge against the real app.
    • assistant: 66 pass. mira: 10 pass.
  • The full benchup-app lib suite has 31 failures, the same set that already fails on main.
  • 0 warnings natively and on wasm32, for every feature combination.
  • CI (.github/workflows/ci.yml) runs all of the above.
  • Not yet tested by hand: a real Claude Code session against the desktop app, the browser build connecting to the local bridge, and MIRA's detached window on a second monitor.

The MCP layer, the Agent connection and MIRA can be split into separate PRs if that's easier to review.

🤖 Generated with Claude Code

aarvee11 and others added 3 commits November 8, 2026 21:20
A dockable inspector panel that lets designers describe changes in plain
language. An OpenAI-compatible model (default preset: endlesstokens.ai)
proposes tool calls and the editor applies them; each turn is one undo step.

- assistant/client: OpenAI-compatible transport (ureq thread on desktop,
  fetch with credentials omitted on web), URL guard, error mapping
- assistant/credentials: per-endpoint API key in a 0600 data-dir file on
  desktop or localStorage on web; never in preferences or projects
- assistant/scene + tools: compact mm-based scene summary and 27 validated
  woodworking tools (create_board, move/rotate/scale/resize, push_pull,
  materials, grain, groups, tags, joints, dimensions, text, selection)
- assistant/turn: bounded tool loop committed once via finish_edit; Stop
  commits the edits applied so far as one step
- panel: @ references, outliner drag-in, "Ask MIRA about this", include
  selection, model picker, change cards with Undo turn, connect modal
- window: latch into the dock or unlatch into an OS window (in-canvas
  window on web) with persisted placement
- busy gate blocks interleaving user edits during a turn
- docs/assistant.md user guide; preferences persisted with serde defaults

Relates to BenchUp3D#1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Checks the native workspace and the wasm32 browser build, and runs the
core, assistant and release-critical app tests. Release builds and
publishing remain in build.yml and pages.yml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
aarvee11 and others added 4 commits November 8, 2026 21:44
- Shared projects: mutating tools only touch entities in the confirmed
  edit lease (or created this turn); a remote replacement discards the
  turn instead of recording it against the old history; edit permission
  is re-checked before every round of tool calls
- Bound model-driven work: 32 calls per reply, 1000 polyline points,
  coordinate/size/text limits, scale factor in [1e-3, 1e3], and
  non-finite results are rolled back
- push_pull validates the face's owner chain and edit scope
- The JSON tool-call fallback only fires when the whole reply is the call
- Key file permissions enforced on every save; key saved only under the
  endpoint it was validated against
- Truncate server errors and tool results; share the scene budget with
  selection and references; stricter tools-unsupported detection;
  Content-Length check on web
- Assistant shortcut cannot shadow Ctrl/Cmd command letters

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removes the only warning in the wasm32 browser build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Include selection off hides the selection from the scene and get_selection
- A turn rolled back by lapsed shared access shows an error, not a change card
- The designer's edit scope is restored after a turn, on Stop/discard and by undo
- Edit access is re-checked before every mutating call in a batch
- Oversized tool results stay valid JSON; history roles always alternate
- Tag-name and scale schemas match what the executor enforces
- Shift+Enter inserts a newline; Enter alone sends
- A no-change turn keeps the previous Undo turn card
- A typed model id survives a model list that omits it
- A new base URL is saved only after a successful connect
- Bring to front works for the in-canvas window on the web
- Change cards count every hidden row in "+N more"
- select is limited to the edit lease in shared projects
- CI runs the 'assistant' and 'mira' filters (82 tests, was 68)
- Ignore .env.local

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/mira/ documents the assistant for anyone picking up the fork:
architecture and turn lifecycle, a reference for every type and function
with file:line citations, the full tool catalog, the exact LLM protocol
(system prompt, scene summary, budgets, error mapping), UI/UX, persistence
and the security model with the review hardening log, testing (inventory,
pre-existing failures, CI, local wasm check), contributing recipes,
development history and the roadmap. Linked from docs/README.md and the
user guide.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dannycoulombe

Copy link
Copy Markdown
Collaborator

I'm not sure it's the right approach, but I also wonder how we could implement this without having to create an addon/plugin layer on top of the application.

#13 is proposing a MCP which I think would open many doors such as this one.

I quickly checked the mira website and a link points to your Github profile so I assume its your project?

We could have a section on the Github page "Tools that use BenchUp" with your logo and link, or even within the app.

@Ladvien

Ladvien commented Nov 9, 2026 •

Copy link
Copy Markdown

#13 is proposing a MCP which I think would open many doors such as this one.

From my limited experience, MCP is the way to go. And trying to keep parity allowing all actions a user can take to also be performed through the MCP. The MCP pattern also allows users to bring their own harness, which they'll want to do.

Love this project, by the way.

@aarvee11

aarvee11 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Agree @dannycoulombe

Lemme drive this towards MCP first... The current one has MCP built in the PR and then a Model endpoint directly so users can chat within the App itself, without using any other harness and makes it stand alone.

Lemme steer this towards MCP and then a LLM Plugin separately.

@aarvee11

aarvee11 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @dannycoulombe, that's fair. Thanks to @Ladvien too.

Yes, MIRA is my project.

I agree that MCP is the better foundation. It lets people bring their own agent, and it avoids adding a provider-specific layer inside BenchUp. So I'll stop pushing this PR as it is and leave it as a draft for reference.

Instead, I've offered on #13 to contribute the part of this work that doesn't depend on the editor UI: the operation layer (28 tools with schemas, validation and tests). It would become a shared crate that the CLI and MCP server can build on, and I'll coordinate with @furilo on it.

Whether an in-app chat makes sense later, on top of the MCP, is entirely your call. I have no expectations there. And thank you for offering a "Tools that use BenchUp" listing. I'd be glad to be included once MIRA works through the MCP.

aarvee11 and others added 9 commits November 9, 2026 15:12
A self-contained library on top of benchup-core with the 28 woodworking
operations an agent can call: JSON Schemas, typed argument parsing, limits,
lock/visibility/edit-scope/lease checks and structured errors. Lengths are
millimetres at the boundary and metres in the document. It also carries the
.bup encode/decode and an atomic save, so tools outside the app can read and
write projects.

Wire adapters for OpenAI-compatible tool calling and MCP tools/list are
included. Nothing in the app uses it yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The assistant now runs its tool calls through the shared crate and keeps
only the LLM-specific pieces (OpenAI ToolCall conversion and the fenced-JSON
fallback). A fixture test proves the tool definitions sent to the server are
unchanged. Project encode/decode delegate to benchup_tools::files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A small stdio MCP server (JSON-RPC 2.0, no async runtime) that offers the
benchup-tools operations to any MCP client.

- `benchup-mcp file <project.bup>` edits a project on disk. Each change is one
  history entry labelled "Agent: <tool>", and the file is saved atomically.
  Shared projects are refused unless --allow-shared.
- `benchup-mcp live` is the bridge proposed in BenchUp3D#13: it listens on 127.0.0.1
  only, and BenchUp (desktop or browser) connects out to it over WebSocket.
  The app must present the pairing token, and browser origins are limited to
  the hosted app and localhost dev servers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BenchUp can now be driven by an outside MCP agent through `benchup-mcp live`.
Settings > Add-ons > Agent connection (off by default) connects to the bridge
on 127.0.0.1 with a per-device pairing token. Each external call is one undo
step labelled "Agent: ...", and it obeys the same read-only, lease, lock and
edit-scope rules as MIRA.

agent_host.rs holds the parts both add-ons share: building the tool
environment from app state, running one call, and selecting what changed.
Calls that arrive during a MIRA turn wait in a queue and run in order once
the turn commits. Agent changes appear in the MIRA transcript as "Agent
activity" cards with Undo.

The key-file code moved to a shared credentials.rs, which now holds both the
MIRA key and the pairing token.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each add-on is a Cargo feature (`assistant`, `agent-link`, both on by
default). When a feature is off, a small stub module takes its place, so call
sites need no cfg. Their preferences live in an always-compiled addons.rs, so
a build without an add-on still keeps the user's saved settings.

Settings > Add-ons switches each one on or off at run time: Build with MIRA
is on by default, Agent connection off. Turning MIRA off stops a running turn
(what it applied stays as one undo step) and hides its section, menu item,
context-menu item and shortcut, without touching the saved inspector layout.
The Agent connection keeps working on its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hand-rolled generator could produce a predictable token: all zeros in a
browser without Web Crypto, or a clock-based value on some desktops. The
token now comes from the OS or browser random source through the uuid crate.
If none is available, no token is created and Agent connection shows an
error instead of connecting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Shared projects: creating tools are refused inside a group the user does not
hold, enter_object checks the lease, and after every mutating call a check
rolls back any change to an entity outside the lease. This closes a gap that
was also present in MIRA.

Bridge protocol v2: the pairing token is never sent. The bridge sends a
challenge, and each side proves it knows the token with HMAC-SHA256, so the
app no longer trusts whatever program is listening on the port. Calls carry
a deadline and the bridge cancels them on timeout, so a call the agent was
told failed is never applied later. Size limits apply before and after
authentication, the auth deadline is checked on every frame, and
unauthenticated connections are capped.

App: queued calls are tied to the project that was open when they arrived,
and are refused if it changed. Results report the open document. The
suggested Claude Code command passes the token in an env var rather than
argv. Client names are cleaned of format characters. Credentials are written
atomically with 0600 from the start.

File mode refuses to save over a project that changed on disk. save_atomic
keeps symlinks and file permissions and fsyncs the directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three tests run the app's Agent connection against the real benchup-mcp
bridge on loopback. An MCP tools/call edits the open project as one undo
step. A call made during a MIRA turn waits for the turn to finish. A wrong
pairing token is refused and nothing is edited. benchup-mcp is a native-only
dev-dependency, so it is not part of the browser build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/mcp.md is a new user guide: file and live modes, pairing with Claude
Code or any MCP client, undo, working alongside MIRA, shared projects,
security and troubleshooting. The MIRA contributor docs now cover the shared
benchup-tools crate, the agent host, the Agent connection add-on, the Cargo
features and stubs, protocol v2, the threat model and the new tests.

CI runs the benchup-tools, benchup-mcp and agent tests, checks every add-on
feature combination, and checks the browser build without default features
and the shared crate on wasm32.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aarvee11 aarvee11 changed the title Add Build with MIRA, an optional LLM design assistant Add an MCP server and optional agent add-ons (Agent connection, Build with MIRA) Nov 10, 2026
@aarvee11
aarvee11 marked this pull request as ready for review November 10, 2026 00:49
testing.md now gives the browser-check commands for both rustup and
Homebrew Rust. It covers installing rustup alongside Homebrew without
changing PATH, removing it again without deleting the shared ~/.cargo
download cache (rustup self uninstall would), and how much disk space the
build caches take.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dannycoulombe pushed a commit that referenced this pull request Nov 13, 2026
Taken from #3 (commit c5c7099) without the bridge pairing module, which
belongs to the live bridge and is left for a later change.

Co-Authored-By: Álvaro Ortiz <alvaro@furilo.com>
dannycoulombe pushed a commit that referenced this pull request Nov 13, 2026
Built on benchup-tools: list tools and schemas, read the scene, run any
tool or an all-or-nothing batch as one undo step, undo/redo, and serve
the same tools over MCP on stdio (protocol code from #3). Saves are
atomic and refuse to overwrite a file changed on disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants