Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,14 @@ jobs:
working-directory: packages/app
run: bun run test:unit

- name: Run Computer Use packaging tests
working-directory: plugins/computer-use
run: bun test test

- name: Verify the pinned Computer Use catalog bundle
working-directory: plugins/computer-use
run: sh script/verify-catalog.sh

generated:
name: generated code
runs-on: ubuntu-latest
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,10 @@ Apexo reads JSON/JSONC config files:
| Project directory | `.apexo/` (agents, commands, plugins, themes, tools) |
| Environment variables | `APEXO_*` (for example `APEXO_CONFIG`, `APEXO_CONFIG_CONTENT`) |

When upgrading, close older Apexo or OpenCode processes before the first launch. Apexo moves an existing per-user `opencode` directory to `apexo` when the destination is absent or empty. The old path becomes a filesystem alias so saved absolute paths still resolve. A populated `apexo` directory takes precedence. Apexo continues to read legacy project configuration, and `APEXO_APP_DIR_NAME` can select a separate profile.

Apexo Desktop loads its interface from `apexo://renderer` and copies saved window routes and preferences on first launch. For older remote Apexo servers, pass `--cors apexo://renderer` to allow connections from the updated Desktop app.

Example `apexo.json`:

```jsonc
Expand Down
106 changes: 106 additions & 0 deletions docs/computer-use-plugin-acceptance.md

Large diffs are not rendered by default.

189 changes: 189 additions & 0 deletions docs/computer-use-plugin-development.md

Large diffs are not rendered by default.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
177 changes: 177 additions & 0 deletions docs/development/computer-use-acceptance.md

Large diffs are not rendered by default.

71 changes: 71 additions & 0 deletions docs/development/computer-use-permission-transitions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Computer Use OS permission follow-up

Date: 2026-11-16. This extends the [implementation acceptance record](computer-use-acceptance.md) for D01. The user performed the grant actions in System Settings. No existing CLI/Desktop permission was reset and no TCC database was edited.

## Artifacts and isolation

The plugin is the same arm64 0.1.0 artifact recorded in the main report, backed by upstream 1.3.1. The model is the local recording provider, `computer-fixture/model`; it exercises the actual Session, permission, MCP, and image paths without sending desktop content to an external model.

The packaged Desktop was copied to an ignored test output directory, assigned the display name **Apexo Permission Test** and bundle/signing identifier `com.apexolab.desktop.computer-use-permissions.947f1657`, and ad-hoc signed. Its `app.asar` and compiled CLI payloads were byte-identical to the previously tested package:

| Payload | SHA-256 |
| ------------ | ------------------------------------------------------------------ |
| `app.asar` | `505a96f01b4c249bdb193abc3883e5ec15fb166cd3461e6afe5f862eadf8ee2c` |
| Compiled CLI | `1381f03e776c71623ae3defc98076b01ddbd48e3b466283fef49c715e250ce04` |

It ran through Launch Services with the existing V2 sidecar and onboarding isolation. Its two dedicated permission entries were independently visible in System Settings. The copy was not a new production build or a claim about every signing/distribution identity.

A separate tiny AppKit launcher, **Apexo CLI Permission Test**, started the actual source CLI acceptance fixture and installed plugin. It pins the worker arguments and working directory in its bundle resources, so an OS relaunch does not start the ordinary product profile. The helper and all native outputs stay outside Git. Its code-signing identifier was `com.apexolab.computer-use.cli-permissions.8d2d131f2a0d42e5996b2d25fc46e476`; its launcher executable SHA-256 was `7df1c9c2a1a13d4bd7baba96fc9ebf9e458a3dc74214a8ac4485388b5bb9a434`.

The CLI helper's own name was not independently visible in the final System Settings lists, which also contained a generic `agent-desktop` entry. Its tool reports changed from denied to granted, but the precise OS grant attribution was not established. Do not treat a distinct bundle ID or a successful action as proof of independent TCC ownership.

## Observed stages

All document observations and screenshots below came through the real installed plugin. Documents contained only the synthetic marker `Apexo isolated permission fixture.`. Setup could open the fixture, but did not implement the measured observation or screenshot.

| Launch chain and state | Independent observation | Result |
| ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------- |
| Packaged Desktop, Accessibility denied / Screen Recording denied | Passive status returned both denied. Snapshot and screenshot returned `PERM_DENIED`, `delivery: not_delivered`, `retry: safe`. | pass |
| Packaged Desktop, Accessibility granted / Screen Recording denied | Snapshot contained the exact fixture text. Screenshot returned `PERM_DENIED` specifically naming Screen Recording. | pass |
| Packaged Desktop, both granted, first attempt | Status reported both granted and snapshot worked, but screenshot returned `INTERNAL`, `screencapture exited with error`, `could not create image from window`. | failed attempt retained |
| Packaged Desktop, both granted, after controlled relaunch | Fresh window observation, snapshot marker, and an inline PNG attachment all completed. | pass for this run |
| CLI helper, both denied | Real snapshot and screenshot calls returned `PERM_DENIED` with no image. | pass for observed state |
| CLI helper, requested Accessibility-only stage | The host exited during manual setup and the controller timed out. On relaunch both permissions were granted, so the Accessibility-only assertion correctly failed. | not accepted |
| CLI helper, both granted | Snapshot contained the marker; screenshot returned a validated PNG of 267,147 bytes. Final revised-harness rerun returned a validated PNG of 262,908 bytes. | pass for observed state; attribution limit remains |

The first post-grant Desktop screenshot failure has an unknown root cause. A successful controlled relaunch provides recovery evidence, not proof of a causal fix. Initial missing-app/stale-window attempts were discarded as fixture failures and were followed by fresh target selection; the original requests were not replayed as mutations.

The packaged Desktop's permission panel distinguished **Screen & System Audio Recording** from **System Audio Recording Only**. An enabled item in the latter did not make the plugin's Screen Recording check pass. The user then added the exact Desktop test bundle to the former and its switch was observed on.

## Cleanup and scope

Owned permission hosts, CLI daemons, and worker processes were stopped. The dedicated **Apexo Permission Test** entries were removed from both Accessibility and Screen Recording using System Settings, and their disappearance was observed. No separately named CLI-helper entry was visible to remove; generic/pre-existing entries were left untouched. The original launch environment continued to report both native grants as granted. The temporary app also registered the `apexo` URL scheme at startup; its previous handler, `com.apexolab.desktop.dev`, was restored.

**D01 remains partial overall.** The packaged Desktop's denied, Accessibility-only, and granted states now have direct evidence. The CLI helper's denied/granted outcomes are verified, but a separate CLI Accessibility-only stage and unambiguous grant attribution remain uncertified. The reverse combination (Screen Recording granted while Accessibility denied) was not separately tested. None of these results closes the independent new-document Save-dialog gap in D02.

## Follow-up repository verification

- Typecheck: 21 successful Turbo tasks, 20 cached.
- Full repository tests: six successful tasks, three cached; all 18 Apexo batches exited zero in 12m15.937s.
- Client and legacy SDK generated-source checks: passed without generated changes.
- HttpApi coverage/auth/effect: each passed 360 cases, zero failures/skips/missing.
- Plugin packaging: four tests / 68 assertions passed. The focused permission-profile and Responses tests passed three tests / 84 assertions on rerun; an earlier unnamed cleanup-hook timeout remains recorded under [BUG-20261116-017](../testing/test-stability/computer-use-repository-gates.md#bug-20261116-017-focused-follow-up-reported-an-unnamed-cleanup-timeout), without a claimed causal fix.
- Production source hashes, local links, script syntax, formatting, and Git whitespace checks passed. The plugin archive and production payloads were unchanged.

## Reproduce the CLI probe

From `packages/apexo`, with the assembled plugin at the usual path and Xcode Command Line Tools available:

```sh
bun script/computer-use-permissions.mjs
```

The command prints its new test app path and signing identifier. Run `denied` before granting anything; do not assume the identity is isolated unless the returned states are denied. Run `guide` only for explicit setup. Have the operator add the exact host identified by the permission guide and record what System Settings actually lists. After granting only Accessibility, run `ax-only`; after granting Screen Recording, run `granted`. The assertions require the exact state combination, the actual document marker, the expected permission error, or a decoded PNG signature as appropriate.

The controller writes private phase receipts atomically, preserves prior records on worker restart, and issues one explicit phase command at a time. It does not change OS grants, repeatedly request them, run an external model, or replay a prior phase after restart. If macOS quits the helper, reopen the printed test app, then reconnect the controller with:

```sh
bun script/computer-use-permissions.mjs --resume /absolute/printed-output-directory
```

Issue a new phase command after reconnecting. `finish` closes the owned fixture and host. Remove only the test identities' grants afterwards; never reset a user's existing application grants to obtain a missing-state test.
7 changes: 7 additions & 0 deletions docs/development/computer-use-source-identity.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"plugins/computer-use/bin/mcp": "ddaacff9ecefc4475a0063e8301f8d037384981d80e04824d5b2a27493077ee3",
"plugins/computer-use/script/build.mjs": "3dffbc58c7ab9af035df00b5b12d6d5fba4dc879435f07cca5a26e7538a00a51",
"plugins/computer-use/skills/computer-use/SKILL.md": "03da6ec2bc03b8f7397b1cdc186dce814159747615e324682df7c7c3b719e410",
"packages/desktop/src/main/background-cli.ts": "e79941d67946cd4db35f9374135dce709bfac257fcfd095650e958d082d43689",
"packages/llm/src/protocols/openai-responses.ts": "b7eb3064f425a84ddbc2f5c604ca68e6e6613f6d9dc4708233f1f0952717848e"
}
2 changes: 2 additions & 0 deletions docs/native-plugin-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Apexo's native runtime activates trusted installed bundle agents, commands, embedded skills, hook sources, and MCP tools from the shared global plugin store. Catalog installation and enablement describe stored configuration; `GET /api/plugins/runtime` reports what the selected Location has actually activated.

The [Computer Use plugin package](../plugins/computer-use/README.md) reuses this bundle lifecycle and upstream stdio MCP. Its [development plan](computer-use-plugin-development.md) and [acceptance gates](computer-use-plugin-acceptance.md) cover local, sequential native V2 desktop use. Assembly and integration tests exist; a platform support claim still requires the recorded graphical acceptance results.

The runtime response includes `generation`, `active` plugin IDs, and an optional failure. The status read is passive. `POST /api/plugins/reload` returns `true` on successful activation without starting provider work. Active work in the same Location produces a typed busy response. Other Locations remain independent.

Bundle files and schema validation are staged first. Agent, command, and skill state transforms prepare private drafts before publication. The runtime holds their writer permits and the Location admission gate until it synchronously publishes the prepared state and bundle generation. A validation, load, or unsupported-component failure keeps the previous effective generation and persists a failure for subsequent status reads. A busy attempt does not overwrite that failure. Interrupted preparation releases the gate and writer permits.
Expand Down
11 changes: 11 additions & 0 deletions docs/testing/build/computer-use-bundle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Computer Use bundle assembly

## BUG-20261116-001: Hash input rejected by Bun's Node crypto compatibility

- Scope: the standalone Computer Use bundle builder, Bun 1.4.4 on macOS arm64.
- Observed: assembling the downloaded pinned archive failed before extraction with `ERR_INVALID_ARG_TYPE`; `Hash.update` rejected an `ArrayBuffer`.
- Expected: verify archive/file SHA-256 values and publish an installable bundle.
- Reproduction: run `bun script/build.mjs --archive <verified-release.tar.gz>` from `plugins/computer-use` using the initial builder implementation.
- Root cause: the Node crypto API accepts byte arrays, not the `ArrayBuffer` returned by `Bun.file(...).arrayBuffer()` in this runtime.
- Fix: use `Bun.file(...).bytes()` for both archive and assembled-file hashes.
- Verification: offline assembly and the builder's own download path both succeeded with the pinned arm64 archive SHA-256 `9b7b7cfc74892eab46c2c85d6cb7dc4863bf8dab0a222cfcfb8ae4604c69c86c`. The corrupt-archive test rejects publication. The four plugin tests passed with 68 assertions. Real installed-bundle CLI and Electron-backend scenarios also completed successfully.
Loading
Loading