Skip to content

feat(router): make search policy auditable and fail closed on capability conflict - #46

Merged
cryptoxdog merged 3 commits into
mainfrom
claude/router-search-policy-coajrt
Aug 18, 2026
Merged

feat(router): make search policy auditable and fail closed on capability conflict#46
cryptoxdog merged 3 commits into
mainfrom
claude/router-search-policy-coajrt

Conversation

@claude

@claude claude Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Type of Change

  • Bug fix
  • Feature / enhancement
  • Refactor (no behavior change)
  • Documentation
  • CI / governance change
  • Breaking change (see rollback plan below)

Governance Checklist

  • Governance setup verified — ran setup_workspace_symlinks.sh, symlinks resolve (§2)
  • Symlinks validatedls -la .cursor/rules .cursor/skills .cursor/commands all resolve
  • All CI gates green — no required checks failing or bypassed
  • Anti-patterns checked — reviewed CANONICAL_LAW.md §7 — none violated
  • CODEOWNERS notified — blast-radius files trigger auto-request; confirmed reviewers assigned
  • Workspace wiring intact§8 wiring requirements satisfied
  • TRACEABILITY_MAP.yaml updated — if this PR resolves an open unknown, mark as RESOLVED
  • Kernel ref discipline — thin callers use @v1, never @main or bare SHA

Breaking Change

  • This is a breaking change

If checked, describe the impact and migration path:

Rollback Plan


Related Issues

Closes #

Commits

  • feat(router): make search policy auditable and fail closed on capability conflict

Test plan

  • make pr-check (local changed-files gate) PASS before open
  • L4 kernels: Recursive Alignment + Validate & Repair (release authorized)
  • CI green; agent PR remediation subscribed after open

Generated by Claude Code

claude added 2 commits August 17, 2026 02:18
…ity conflict

`requiresSearch` was already authoritative over the TaskType default, but three
gaps let the routing decision misrepresent what actually happened.

1. Search + vision silently lost a capability. `resolveRoute` tested the search
   policy before the vision branch, so a visual task carrying images with
   `requiresSearch: true` routed to Perplexity — which has no multimodal
   transport — and the images were dropped with no signal to the caller. That
   combination now throws `UnsupportedCapabilityCombinationError` (code
   `UNSUPPORTED_CAPABILITY_COMBINATION`) from route resolution, before request
   identity, budget reservation, circuit permit, or provider dispatch, so
   neither capability can disappear and neither budget nor circuit state is
   touched.

2. The Perplexity config could contradict its own decision.
   `disableSearch: requiresSearch === false && !isSearchTask(type)` was
   unreachable on a resolved search route and, off-route, produced a
   search-provider config with web search turned off. A Perplexity config is
   only ever built for a search route, so `disableSearch` is now always false,
   and dispatch asserts both that the config has search on and that
   `decision.searchRequired` equals `provider === PERPLEXITY` in both
   directions.

3. A decision could not be audited. `RoutingResolution` now carries
   `searchRequired` and `searchPolicySource` (EXPLICIT | TASK_DEFAULT), so the
   call log proves whether search was chosen by the caller or by the TaskType
   default instead of leaving it to be inferred from model names.

The rule itself now has exactly one implementation: `resolveSearchPolicy()`
returns both the answer and its authority; `requiresSearchProvider()` is a
boolean view of it and `isSearchTask()` supplies only the TaskType default.

Behaviour that did not change: TaskType defaults when the flag is omitted, the
model matrices, vision model selection for a given image count, budget
reservation and reconciliation ordering, circuit classification, the bounded
fallback chain, image safety validation, and the deprecated 1.x provider
subpath exports. Consensus stays an execution modifier — it never pulls a task
onto the search plane.

Tests: 161 (was 100). Adds the full routing matrix as a table-driven suite,
audit-evidence assertions, provider-dispatch proofs with injected clients,
capability-conflict proofs, config/decision consistency proofs, and
non-regression guards for budget, circuits, image safety, and vision model
selection.

Version 1.2.0: additive public API (SearchPolicySource, SearchPolicyResolution,
resolveSearchPolicy, UnsupportedCapabilityCombinationError, two new
RoutingResolution fields) plus a fail-closed correction, within the 1.x line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018YFsaV8KzRmmAkBUHxfE9z
Removing the `!isSearchTask(task.type)` term from the `disableSearch`
predicate left the import in `perplexity-matrix.ts` serving nothing but the
backward-compatibility re-export. Collapse it to `export … from` and say in
the comment why nothing in this module calls it any more: the search decision
is settled before a Perplexity config is ever resolved.

No behaviour or export-surface change — `isSearchTask` remains importable from
both `./matrices/perplexity-matrix.js` and the package root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018YFsaV8KzRmmAkBUHxfE9z

Copy link
Copy Markdown
Collaborator

Campaign 7-ROUTER — explicit search policy + router integrity

Baseline: 56e2f48 on main, package 1.1.3, clean tree, 100 tests green.

requiresSearch was already authoritative over the TaskType default (landed in #45). This PR closes the three gaps that let the routing decision misrepresent what actually happened. Each was proven by an executable probe against the baseline before any code changed.

Verified defects

# Defect Pre-change evidence
A Search + vision silently dropped a capability. resolveRoute tested the search policy before the vision branch, so SCREENSHOT_ANALYSIS + images + requiresSearch: true resolved to Provider.PERPLEXITY. PerplexityClient.buildRequestBody builds text-only messages — the images were discarded with no signal. probe asserted decision.provider === PERPLEXITY for a task carrying images
B No audit evidence. RoutingDecision carried no field distinguishing "the caller asked for search" from "the task type implied it". probe asserted 'searchRequired' in decision === false
C Perplexity config could contradict its own decision. disableSearch: requiresSearch === false && !isSearchTask(type) is unreachable on a resolved search route; off-route it produced a search-provider config with web_search_options stripped. probe asserted disableSearch === true off-route, false on-route

Implementation

  • resolveSearchPolicy() in src/matrices/search-policy.ts is now the single implementation of the rule and returns both the answer and its authority. requiresSearchProvider() is a boolean view of it; isSearchTask() supplies only the TaskType default. One typeof task.requiresSearch === 'boolean' exists in src/.
  • RoutingResolution gains searchRequired: boolean and searchPolicySource: EXPLICIT | TASK_DEFAULT, inherited by RoutingDecision and the call log.
  • Search + vision + images throws UnsupportedCapabilityCombinationError (code: 'UNSUPPORTED_CAPABILITY_COMBINATION') from route resolution — before request identity, budget reservation, circuit permit, or dispatch.
  • disableSearch is now always false, plus two dispatch invariants: decision.searchRequired === (provider === PERPLEXITY) in both directions, and a search route may never dispatch a config with search off.

Routing matrix (tests/routing-matrix.test.ts, table-driven)

Case requiresSearch Expected Actual
A/B/C STRATEGIC_REASONING false / true / omitted NON_SEARCH / SEARCH / NON_SEARCH
D/E/F COMPETITOR_RESEARCH true / false / omitted SEARCH / NON_SEARCH / SEARCH
G/H FACT_VERIFICATION false / omitted NON_SEARCH / SEARCH
I/J CONTENT_GENERATION false / true NON_SEARCH / SEARCH
K/L SCREENSHOT_ANALYSIS + images false / omitted VISION
M SCREENSHOT_ANALYSIS + images true fail closed ✅ throws
N/O EXTRACTION, CLASSIFICATION true SEARCH
P/Q MARKET_RESEARCH false / omitted NON_SEARCH / SEARCH

Every non-fail-closed case also asserts searchPolicySource (EXPLICIT for a boolean, TASK_DEFAULT for omitted) and that searchRequired matches the plane actually selected.

Non-regression proved, not assumed

Budget reserved before dispatch on both planes and reconciled to actual cost; failures charged to the circuit of the provider the policy selected (search failure opens Perplexity only, general failure opens OpenRouter only); the capability refusal reserves nothing and opens nothing; image safety validation still precedes every routing outcome; consensus stays an execution modifier and never pulls a task onto the search plane; vision model selection is byte-identical for every (image count × complexity) pair; TaskDescriptor still exposes capability fields only — application-supplied provider/model keys are stripped and the router keeps its choice.

Tests: 100 → 161. The package export map is unchanged: no 1.x provider subpath removed, none added.

Consumer compatibility (read-only; neither repo modified)

The real policy modules from each consumer — Website-Bot@3eb3536 src/intelligence/improve-llm-policy.ts and SEO-Bot@9691264 src/services/improve-llm-policy.ts — were copied unmodified into disposable ephemeral projects, compiled against the packed 1.2.0 tarball under each consumer's own strict compilerOptions, and executed:

  • Website-Bot — PASS. assertWebsiteImprovePolicy() holds. All 5 operations stay off the search plane; VISUAL_PATTERN_ANALYSIS and VISUAL_DELTA_ANALYSIS still resolve to vision.
  • SEO-Bot — PASS. assertSeoImprovePolicy() holds. The 4 reasoning operations stay non-search; FRESH_WEB_EVIDENCEperplexity/sonar-pro. The inline requiresSearch: true helpers (COMPETITOR_RESEARCH, CITATION_CHECK, MARKET_RESEARCH) stay on search, and analyzeScreenshot (LAYOUT_VALIDATION + images, flag omitted) stays on vision with searchPolicySource = TASK_DEFAULT.

Neither consumer combines vision with requiresSearch: true, so the new fail-closed path is unreachable from their current call sites.

Version

1.1.31.2.0. Additive public API (SearchPolicySource, SearchPolicyResolution, resolveSearchPolicy, UnsupportedCapabilityCombinationError, two RoutingResolution fields) plus a fail-closed correction — a minor bump within 1.x. Both consumers pin 1.1.2 exactly, so neither auto-adopts; adoption is each repo's own decision.

Not proven here — please note before tagging

  • Real-provider smoke was not run. No PERPLEXITY_API_KEY / OPENROUTER_API_KEY is reachable from the execution sandbox. Status is REAL_PROVIDER_SMOKE_BLOCKED, not PASS.
  • Registry state for 1.2.0 was not verified from the sandbox (no read:packages scope). Confirm 1.2.0 is unpublished before pushing a v1.2.0 tag — published versions are immutable.

npm run verify:package — the isolated tarball install and export smoke I could not run locally — passed in CI on Node 20.19.0, 22.23.1, and 24.18.0.


Generated by Claude Code

The governed agent toolchain writes phase locks, memory receipts, L4
phase/release receipts, and PR handoffs under `.l9/`. They are machine- and
session-local — they carry session IDs and absolute paths — so they are not
source and must not be committed.

Every other repo in the constellation already ignores this tree
(Cursor-Governance and Website-Bot ignore `.l9/` wholesale; l9-ci-core ignores
`.l9/runtime/` and `.l9/autonomy/`). LLM-Router was the outlier, which left the
files showing up as untracked after any governed run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018YFsaV8KzRmmAkBUHxfE9z
@sonarqubecloud

Copy link
Copy Markdown

@cryptoxdog
cryptoxdog merged commit c47c024 into main Aug 18, 2026
17 checks passed
@cryptoxdog
cryptoxdog deleted the claude/router-search-policy-coajrt branch August 18, 2026 02:54
cryptoxdog added a commit that referenced this pull request Aug 18, 2026
Operator decision: publish current main as 1.1.3 (search-policy audit work
from #46 stays; version bump rolled back to keep the 1.1.x line). Fix #4
will advance to the next patch after this publishes.

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants