Skip to content

Add scoped Code Mode execution - #119

Closed
kortiene wants to merge 10 commits into
nuclia:mainfrom
kortiene:scoped-codemode
Closed

kortiene wants to merge 10 commits into
nuclia:mainfrom
kortiene:scoped-codemode

Conversation

@kortiene

@kortiene kortiene commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add an explicit capability-scoped Code Mode tool factory while preserving the existing compatibility tool.
  • Add result projection, source/result/output/nested-call limits, nested event ancestry, execution admission, and tool inheritance controls.
  • Harden local and remote sandbox execution with authenticated transport, bounded protocol handling, runtime and memory ceilings, cancellation cleanup, shared admission, and configurable socket and metrics exposure.
  • Document scoped SDK usage and production deployment guidance.

Security

  • Scoped Code Mode requires the remote sandbox by default and exposes only explicitly supplied capabilities.
  • Nested event payloads redact sensitive fields and replace opaque strings with byte-count metadata.
  • The sandbox protocol rejects oversized, deeply nested, malformed, non-finite, and unsupported serializer/dataclass values before remote transport.

Testing

  • make lint
  • uv run pytest tests/codemode/test_model.py tests/codemode/test_protocol.py tests/harness_sdk/test_codemode.py tests/harness_sdk/test_harness.py -q (137 passed, 1 skipped)
  • uv run pytest tests/ --ignore=tests/api --ignore=tests/standalone --ignore=tests/test_next.py -q (278 passed, 1 skipped)
  • uv run pytest tests/test_sandbox.py -q in agents/restricted (14 passed)
  • git diff --check

Known limitation

make all-tests was attempted, but the API/standalone fixtures could not complete locally because port 8000 is occupied by existing Docker Desktop and SSH listeners. Those unrelated processes were left untouched; the broad Hyperforge run above excludes only the affected API, standalone, and recorded-network test paths.

Behavior changes

  • Local Code Mode worker transport (pool and isolated-process modes) now uses the bounded JSON worker protocol instead of pickle. Values crossing the worker boundary must be JSON worker values; previously picklable types such as bytes, sets, and dataclasses are rejected before transport. The remote sandbox path already had this contract.

  • The sandbox metrics server is disabled by default and binds to 127.0.0.1 when enabled. Set SANDBOX_METRICS_ENABLED=true (and SANDBOX_METRICS_HOST/SANDBOX_METRICS_PORT as needed) to restore scraping.

  • Sandbox connections presenting an invalid token are closed during the handshake without a protocol-level error message; clients now see "Sandbox connection closed unexpectedly" instead of a token-specific error.

  • Rebased onto main after Make codemode sandbox token optional #114 (optional sandbox token): the generic sandbox service and SandboxRunner.remote keep main's token-optional behavior; scoped Code Mode (create_codemode_tool) still fails closed and requires both SANDBOX_SOCKET and SANDBOX_TOKEN.

@kortiene

kortiene commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Polaris did not reach code analysis. The job failed during action initialization because POLARIS_ACCESS_TOKEN is unavailable to this fork-originated pull_request workflow (required parameters for polaris is missing).

GitHub intentionally withholds repository secrets from fork PRs. Maintainer edits are enabled; this check needs an upstream maintainer to run the branch from a trusted nuclia/hyperforge ref, or a separate safe workflow change for fork PR handling. I have not added a feature-branch change that skips or weakens SAST.

Failed job: https://github.com/nuclia/hyperforge/actions/runs/33825876621/job/100878306192

@kortiene

kortiene commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

Consolidation commit (f3d0cd6) folds the platform-side hardening from nuclia/data-platform#1044 into this PR so the foundation ships once, upstream:

  • Runner injection seam: create_codemode_tool(runner=...) with a public CodeModeRunner protocol/CodeModeDispatch type; injected runners bypass socket/token fail-closed checks (deterministic tests, no sandbox). Default path unchanged, including orphan-callback slot retention.
  • Output contract: output(value) must be called exactly once with exactly one value; a missing/empty/repeated call now fails the invocation (scoped tool only; stock codemode unchanged).
  • Cumulative result cap: CodeModeLimits.max_cumulative_result_bytes (default 4 MiB, validated >= max_result_bytes).
  • Client timeout slack: remote connections are bounded by requested runtime (or SANDBOX_MAX_SESSION_RUNTIME_SECONDS when unset) plus SANDBOX_TIMEOUT_SLACK_SECONDS (default 10) — removes the exact-instant race with the server kill and the unbounded no-runtime path.
  • Misc: question passthrough to the worker, SANDBOX_TOKEN fail-closed at execute time, nested events carry duration_ms/result_bytes, sensitive-key list extended (passwd, private_key).

Local verification: 288 passed/1 skipped (full non-api suite), 14 passed in agents/restricted, ruff + ty + mypy clean. CI matches; Polaris still needs a trusted-ref rerun for the fork-PR secret limitation noted earlier.

@kortiene
kortiene marked this pull request as ready for review September 9, 2026 21:30
@kortiene
kortiene requested a review from a team September 9, 2026 21:30
Sekou Kone added 4 commits September 10, 2026 14:26
Rebased onto main after nuclia#123 (pluggable sandbox auth and ToolCallContext)
landed; the branch already absorbed that foundation, so conflicts were
resolved in favor of the combined final state.

- Capability-scoped Code Mode tool factory (create_codemode_tool) with
  result projection, source/result/output/nested-call limits, nested
  event ancestry, execution admission, and tool inheritance controls
- Hardened local/remote sandbox execution with authenticated transport,
  bounded protocol handling, runtime/memory ceilings, cancellation
  cleanup, and configurable socket/metrics exposure
- Fix test_sandbox_rejects_concurrent_session to read the socket path
  from SANDBOX_SOCKET env instead of the import-time settings singleton
- Apply ruff format to codemode/sandbox.py, tools/codemode.py,
  test_codemode.py

@vangheem vangheem left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove the current_tool_call context var? I don't think that's necessary right?

@vangheem vangheem left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're going to have to be careful about not breaking existing implementations right now so all changes should be backward compatible

For example, #114

@kortiene

Copy link
Copy Markdown
Contributor Author

Superseded by the split stack:

The hardening PR targets the foundation branch in the contributor fork because this account cannot create an upstream base branch. Once #137 merges, the hardening branch can be proposed directly against upstream main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants