Skip to content

feat(tool-search): add deferred tool visibility with exact loading - #25

Merged
boh5 merged 4 commits into
mainfrom
codex/tool-search-projection-hard-cut
Aug 30, 2026
Merged

feat(tool-search): add deferred tool visibility with exact loading#25
boh5 merged 4 commits into
mainfrom
codex/tool-search-projection-hard-cut

Conversation

@boh5

@boh5 boh5 commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Summary

  • Replace eager model-visible tool schemas with a deterministic projection: Core tools, state-activated tools, and Execution-loaded contracts remain visible, while authorized local long-tail and ready MCP tools appear in a compact deferred directory.
  • Add tool_search with select:<exact registryName> as the primary loading path and deterministic BM25/trigram ranking as an auxiliary natural-language path.
  • Persist immutable Execution authorization snapshots and loaded contract refs so HITL, resume, compaction, and supported cold recovery reconstruct the same visibility boundary.
  • Keep authorization, permissions, Tool Registry, MCP execution, Tool Batch, finalized-output, and audit ownership unchanged. Search can only reduce an already-authorized catalog; it cannot grant access.

Why

Eagerly sending every local and MCP schema increases initial provider context and makes long-tail selection noisier. This keeps common capabilities immediately available while loading less-frequent contracts only when needed, without depending on provider-native Tool Search APIs.

Compatibility and risks

  • Tool Search is deterministic and local. It does not use embeddings, translation, LLM query rewriting, or an eager/load-all fallback.
  • select:<exact registryName> is exact-only; an exact miss does not fall back to ranking. Natural-language recall still depends on metadata quality and language, and deferred use may require one additional model step.
  • New Execution records always write toolAuthorizationSnapshot and loadedToolRefs.
  • Older records missing those fields hydrate as { extraTools: [], toolProjection: null } and [], then re-persist normally. No data-format version or migration framework is introduced.
  • Present but malformed values remain invalid and continue through Runtime Data / Settings diagnostics and recovery.
  • The old eager tools.tools contract, namespace-summary compatibility path, and load-all fallback are intentionally removed.
  • Artifact and tombstone recovery metadata is hard-cut to version 2; version 1 recovery artifacts are rejected and removed during startup cleanup rather than migrated.

Validation

  • bun run typecheck — 5/5 workspace tasks passed
  • bun run test — 8/8 workspace tasks passed
  • bun run build — production Web build and 308-asset entrypoint passed
  • ./dist/archcode --versionarchcode 0.0.10
  • bun run tool-contract:measure — Lead initial wire 5,136 tokens; deferred directory 1,188 tokens; combined 6,324 vs. 11,845 baseline (-46.61%)
  • bun run tool-search:benchmark — 1,000 entries / 100 queries; p50 93.54 ms, p95 96.06 ms
  • git diff --check origin/main...HEAD
  • Legacy Session browser QA — 7 real Sessions / 14 legacy Executions loaded; Runtime Data reported zero issues; an old Session completed a real model round; the source runtime files remained byte-for-byte unchanged

Documentation and security

  • I updated user-facing documentation when behavior, setup, or configuration changed.
  • I added or updated tests for behavior changes.
  • I did not commit credentials, tokens, private configuration, or generated secrets.
  • Tool Search preserves existing authorization and permission boundaries; sensitive search queries are rejected/redacted before Store, audit, or logs.

Related issue

None.

Review in cubic

boh5 added 4 commits August 27, 2026 00:49
BREAKING CHANGE: Execution tool authorization and artifact metadata now use hard-cut schemas with no compatibility reader.
Expose a compact deferred tool directory and load known names with select:<name>. Keep BM25 as the auxiliary discovery path and preserve strict authorization and recovery boundaries.
Hydrate absent authorization snapshots and loaded refs with canonical empty values while keeping present malformed data invalid. Do not introduce a data-format version.
@coderabbitai

coderabbitai Bot commented Aug 30, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 128 files, which is 28 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 48d7f3d2-5cce-43e2-b319-7b89eda0d340

📥 Commits

Reviewing files that changed from the base of the PR and between 544aeae and 410aa68.

📒 Files selected for processing (128)
  • AGENTS.md
  • CHANGELOG.md
  • apps/server/src/app.test.ts
  • apps/server/src/server-host.test.ts
  • apps/web/src/components/composite/ExecutionWorkstream.interaction.tsx
  • apps/web/src/components/features/SessionComposerDock.interaction.tsx
  • apps/web/src/components/features/TodoProgressButton.interaction.tsx
  • apps/web/src/context/global-sse.test.tsx
  • apps/web/src/lib/execution-status-presentation.test.ts
  • apps/web/src/routes/session.test.tsx
  • apps/web/src/store/session-store.test.ts
  • docs/architecture.md
  • docs/goals/tool-search-projection-hard-cut-plan-goal.md
  • docs/goals/tool-search-projection-hard-cut-progress.md
  • package.json
  • packages/agent-core/package.json
  • packages/agent-core/scripts/tool-contract-measure.ts
  • packages/agent-core/scripts/tool-search-benchmark.ts
  • packages/agent-core/src/__arch__/goal-boundaries.test.ts
  • packages/agent-core/src/__arch__/shared-code-unification.test.ts
  • packages/agent-core/src/__arch__/tool-output-policy-matrix.test.ts
  • packages/agent-core/src/agent-tree/projection.test.ts
  • packages/agent-core/src/agents/configured-agent.test.ts
  • packages/agent-core/src/agents/configured-agent.ts
  • packages/agent-core/src/agents/definitions/analyst.ts
  • packages/agent-core/src/agents/definitions/build.ts
  • packages/agent-core/src/agents/definitions/definitions.test.ts
  • packages/agent-core/src/agents/definitions/discussion.ts
  • packages/agent-core/src/agents/definitions/explore.ts
  • packages/agent-core/src/agents/definitions/lead.ts
  • packages/agent-core/src/agents/definitions/librarian.ts
  • packages/agent-core/src/agents/factory-types.ts
  • packages/agent-core/src/agents/factory.test.ts
  • packages/agent-core/src/agents/factory.ts
  • packages/agent-core/src/agents/query/hooks/auto-compact.test.ts
  • packages/agent-core/src/agents/query/hooks/hybrid-compression.test.ts
  • packages/agent-core/src/agents/query/loop.test.ts
  • packages/agent-core/src/agents/query/loop.ts
  • packages/agent-core/src/agents/query/provider-secret-redaction.integration.test.ts
  • packages/agent-core/src/agents/query/recovery.test.ts
  • packages/agent-core/src/agents/query/types.ts
  • packages/agent-core/src/agents/session-agent-manager.test.ts
  • packages/agent-core/src/agents/session-agent-manager.ts
  • packages/agent-core/src/agents/tool-filter.test.ts
  • packages/agent-core/src/agents/tool-filter.ts
  • packages/agent-core/src/agents/tool-visibility/catalog.ts
  • packages/agent-core/src/agents/tool-visibility/deferred-tool-directory.ts
  • packages/agent-core/src/agents/tool-visibility/index.ts
  • packages/agent-core/src/agents/tool-visibility/projection.ts
  • packages/agent-core/src/agents/tool-visibility/search-eval-cases.ts
  • packages/agent-core/src/agents/tool-visibility/search.ts
  • packages/agent-core/src/agents/tool-visibility/tool-visibility.test.ts
  • packages/agent-core/src/agents/tool-visibility/types.ts
  • packages/agent-core/src/agents/types.ts
  • packages/agent-core/src/background/tasks/title-generation.test.ts
  • packages/agent-core/src/execution/session-execution-manager.test.ts
  • packages/agent-core/src/execution/session-execution-manager.ts
  • packages/agent-core/src/execution/session-tool-batch-scheduler.test.ts
  • packages/agent-core/src/execution/session-tool-batch-scheduler.ts
  • packages/agent-core/src/index.ts
  • packages/agent-core/src/lead-architecture-flows.integration.test.ts
  • packages/agent-core/src/main.test.ts
  • packages/agent-core/src/mcp/naming.test.ts
  • packages/agent-core/src/mcp/naming.ts
  • packages/agent-core/src/mcp/runtime-service.test.ts
  • packages/agent-core/src/mcp/runtime-service.ts
  • packages/agent-core/src/mcp/transports.integration.test.ts
  • packages/agent-core/src/memory/idle-coordinator.test.ts
  • packages/agent-core/src/memory/learning-input.test.ts
  • packages/agent-core/src/multi-agent-control-plane.integration.test.ts
  • packages/agent-core/src/prompt/compiler.test.ts
  • packages/agent-core/src/prompt/compiler.ts
  • packages/agent-core/src/prompt/lint.ts
  • packages/agent-core/src/prompt/live-eval.ts
  • packages/agent-core/src/prompt/types.ts
  • packages/agent-core/src/runtime-data/service.test.ts
  • packages/agent-core/src/runtime-mcp.test.ts
  • packages/agent-core/src/runtime.ts
  • packages/agent-core/src/session-input/service.test.ts
  • packages/agent-core/src/store/helpers.test.ts
  • packages/agent-core/src/store/helpers.ts
  • packages/agent-core/src/store/logical-execution.test.ts
  • packages/agent-core/src/store/memory-learning.test.ts
  • packages/agent-core/src/store/message-phase-hard-cut.test.ts
  • packages/agent-core/src/store/session-store-manager.test.ts
  • packages/agent-core/src/store/store.test.ts
  • packages/agent-core/src/store/types.ts
  • packages/agent-core/src/testing/test-execution-fixtures.ts
  • packages/agent-core/src/testing/test-mcp-runtime.test.ts
  • packages/agent-core/src/testing/test-mcp-runtime.ts
  • packages/agent-core/src/tool-output/access-service.test.ts
  • packages/agent-core/src/tool-output/access-service.ts
  • packages/agent-core/src/tool-output/artifact-lifecycle.test.ts
  • packages/agent-core/src/tool-output/artifact-store.test.ts
  • packages/agent-core/src/tool-output/artifact-store.ts
  • packages/agent-core/src/tool-output/artifact-types.ts
  • packages/agent-core/src/tool-output/capture.test.ts
  • packages/agent-core/src/tool-output/finalizer.ts
  • packages/agent-core/src/tool-output/live-bash.integration.test.ts
  • packages/agent-core/src/tool-output/types.ts
  • packages/agent-core/src/tools/builtins/background-output.ts
  • packages/agent-core/src/tools/builtins/cancel-session.ts
  • packages/agent-core/src/tools/builtins/compress.ts
  • packages/agent-core/src/tools/builtins/index.ts
  • packages/agent-core/src/tools/builtins/lsp/lsp-diagnostics.ts
  • packages/agent-core/src/tools/builtins/lsp/lsp-symbols.ts
  • packages/agent-core/src/tools/builtins/model-visible-contract.test.ts
  • packages/agent-core/src/tools/builtins/output-artifacts.test.ts
  • packages/agent-core/src/tools/builtins/output-artifacts.ts
  • packages/agent-core/src/tools/builtins/send-message.ts
  • packages/agent-core/src/tools/builtins/tool-search.test.ts
  • packages/agent-core/src/tools/builtins/tool-search.ts
  • packages/agent-core/src/tools/builtins/wait-for-reminder.ts
  • packages/agent-core/src/tools/builtins/web-fetch.ts
  • packages/agent-core/src/tools/names.ts
  • packages/agent-core/src/tools/registry.test.ts
  • packages/agent-core/src/tools/registry.ts
  • packages/agent-core/src/tools/types.ts
  • packages/protocol/src/execution.test.ts
  • packages/protocol/src/execution.ts
  • packages/protocol/src/guards.test.ts
  • packages/protocol/src/guards.ts
  • packages/protocol/src/message-phase-hard-cut.test.ts
  • packages/protocol/src/reduce.test.ts
  • packages/protocol/src/reduce.ts
  • packages/protocol/src/tools.test.ts
  • packages/protocol/src/tools.ts
  • packages/protocol/src/types.ts

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

boh5 commented Aug 30, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 30, 2026

Copy link
Copy Markdown
⚠️ Action not completed

Review skipped: 128 files exceed the limit of 100.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@boh5
boh5 merged commit b68f8c5 into main Aug 30, 2026
7 checks passed
@boh5
boh5 deleted the codex/tool-search-projection-hard-cut branch August 30, 2026 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant