Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 31 additions & 8 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
```text
validated execution task
-> effective image set merged into task
-> pure route resolution
-> capability resolution + fail-closed validation
-> pure route resolution (single decision)
-> request identity and timestamp
-> atomic process-local budget reservation
-> provider-family-safe downgrade
Expand All @@ -19,9 +20,20 @@ validated execution task

Route resolution is pure. Request IDs and timestamps are added afterward and do not participate in routing equivalence.

## Search policy authority
## Capability authority

The application declares the capability; the router selects the provider. That split is enforced by a single resolver, `resolveSearchPolicy()` in `src/matrices/search-policy.ts`:
The application declares the capability; the router selects the provider. One authority chain turns a `TaskDescriptor` into one decision that both routing and dispatch consume:

```text
TaskDescriptor
-> resolveCapabilities (search, source, vision, images)
-> validateCapabilities (fail closed on unsupported combinations)
-> resolve provider/model
-> reserve budget
-> dispatch EXACT resolved capability
```

`resolveSearchPolicy()` in `src/matrices/search-policy.ts` is the single implementation of the search rule:

```text
typeof task.requiresSearch === 'boolean'
Expand All @@ -30,21 +42,32 @@ otherwise
-> { required: isSearchTask(task.type), source: TASK_DEFAULT }
```

There is exactly one implementation of this rule. `requiresSearchProvider()` is a boolean view of it and `isSearchTask()` supplies only the `TaskType` default. `resolveRoute()` consumes the resolution and copies `searchRequired` and `searchPolicySource` onto every `RoutingResolution`, so a decision is auditable without inferring intent from model names.
`requiresSearchProvider()` is a boolean view of it and `isSearchTask()` supplies only the `TaskType` default. `resolveCapabilities()` composes the search policy with the canonical vision-task inventory (`VISION_TASKS`, owned by `search-policy.ts`) and the effective image set. `resolveRoute()` consumes the validated capabilities and copies `searchRequired`, `searchPolicySource`, and `visionRequired` onto every `RoutingResolution`; `dispatchProvider()` branches on the same decision fields and asserts the provider contract each branch requires. Dispatch never re-derives a plane from the raw task.

Fail-closed validation refuses every combination the provider plane would silently drop, before any budget reservation or provider dispatch:

| Combination | Error code |
| --- | --- |
| Vision task without images | `VISION_INPUT_REQUIRED` |
| Search + vision together | `UNSUPPORTED_CAPABILITY_COMBINATION` |
| Images on a non-vision task | `IMAGES_NOT_SUPPORTED_FOR_TASK` |
| `recency` / `domainFilter` without search | `SEARCH_MODIFIER_WITHOUT_SEARCH` |
| `consensus` on a non-search route | `CONSENSUS_REQUIRES_SEARCH` |

Two invariants keep the audit honest:
Three invariants keep the audit honest:

- A resolved Perplexity config always has `disableSearch: false`. A search decision can never dispatch a config with web search turned off.
- A resolved Perplexity config always has `disableSearch: false`, and `resolvePerplexityConfig()` refuses non-search tasks. A search decision can never dispatch a config with web search turned off.
- Before dispatch, `decision.searchRequired` must equal `decision.provider === Provider.PERPLEXITY`. A disagreement in either direction is a hard error, not a downgrade.
- Failed routed calls are auditable: `RoutingDecision` records `outcome`, `failureKind`, and `errorCode` on failure, without ever logging prompts, keys, or image contents.

Search and vision have no combined provider contract. A vision `TaskType` carrying images with `requiresSearch: true` throws `UnsupportedCapabilityCombinationError` from route resolution — before request identity, budget reservation, circuit permit, or provider dispatch — so neither capability is silently discarded. Because the throw precedes reservation and permit acquisition, it cannot affect budget state or provider circuit health.
Because every refusal precedes reservation and permit acquisition, none of them can affect budget state or provider circuit health.

## Module ownership

```text
src/types.ts public legacy contracts
src/schemas.ts runtime validation for public legacy input
src/matrices/* deterministic model and search resolution
src/matrices/* deterministic model resolution plus capability authority
src/pricing.ts canonical OpenRouter price table
src/budget/* process-local admission and spend accounting
src/circuit-breaker.ts process-local provider health control
Expand Down
18 changes: 15 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ await router.execute(

### Auditing a routing decision

Every `RoutingDecision` — from `route()` and from `getCallLog()` — reports whether search was selected and on whose authority, alongside `taskType`, `complexity`, `provider`, `model`, `reason`, `estimatedCost`, `taskId`, `clientId`, `timestamp`, downgrade state, and (after execution) `actualCost` and `latencyMs`. No credentials or prompts are recorded.
Every `RoutingDecision` — from `route()` and from `getCallLog()` — reports whether search was selected and on whose authority (`searchRequired`, `searchPolicySource`), whether the route is vision-backed (`visionRequired`), and, after execution, the call outcome: `actualCost`, `latencyMs`, and for failed routed calls `outcome: 'FAILED'` with `failureKind` and `errorCode`. The same decision object drives dispatch, so the audit can never disagree with what actually executed. No credentials, prompts, or image contents are recorded.

```ts
const decision = router.route({ clientId: 'tenant-a', type: TaskType.MARKET_RESEARCH, complexity: TaskComplexity.HIGH, requiresSearch: false });
Expand All @@ -127,12 +127,24 @@ decision.provider; // Provider.OPENROUTER

### Unsupported capability combinations

No provider in this router serves search and vision together. A visual task that supplies images *and* sets `requiresSearch: true` is refused with `UnsupportedCapabilityCombinationError` (code `UNSUPPORTED_CAPABILITY_COMBINATION`) before any budget reservation, circuit permit, or provider dispatch — rather than silently dropping the images or silently skipping the search. Split such work into a vision task and a search task.
The router fails closed on every capability combination the provider plane cannot execute faithfully — nothing is silently dropped:

| Combination | Error code |
| --- | --- |
| Visual task without images | `VISION_INPUT_REQUIRED` |
| Search and vision together | `UNSUPPORTED_CAPABILITY_COMBINATION` |
| Images on a non-visual task | `IMAGES_NOT_SUPPORTED_FOR_TASK` |
| `recency` / `domainFilter` without search | `SEARCH_MODIFIER_WITHOUT_SEARCH` |
| `consensus` on a non-search route | `CONSENSUS_REQUIRES_SEARCH` |

All of these throw `UnsupportedCapabilityCombinationError` before any budget reservation, circuit permit, or provider dispatch, so an invalid request never half-executes and never affects budget state or circuit health. Split search+vision work into a vision task and a search task.

## Vision execution

Images supplied through execution options are merged into the validated task before routing. This ensures model selection and budget estimation use the same image count that reaches the provider.

A vision task without images fails closed with `VISION_INPUT_REQUIRED` instead of silently degrading to a text-only call, and images attached to a non-visual task fail with `IMAGES_NOT_SUPPORTED_FOR_TASK` instead of being ignored.

```ts
const result = await router.execute(
{
Expand All @@ -157,7 +169,7 @@ Only HTTPS public URLs and bounded `data:image/*;base64` payloads are accepted.

For eligible high-complexity Perplexity tasks, `{ consensus: true }` executes the configured variations in parallel. The returned content is selected from the successful responses, while token and cost fields represent the aggregate successful consensus execution so budget reconciliation does not undercount spend.

Consensus is an execution modifier, not search-policy authority. It applies only to a route that already resolved to the search plane; on a general or vision route it is inert and never pulls the task onto a search provider.
Consensus is an execution modifier, not search-policy authority. It applies only to a route that already resolved to the search plane; requesting it on a general or vision route throws `CONSENSUS_REQUIRES_SEARCH` before budget reservation instead of being silently ignored.

## Budget semantics

Expand Down
78 changes: 52 additions & 26 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import { CircuitBreaker, CircuitOpenError, type CircuitPermit } from './circuit-breaker.js';
import { resolveGeneralConfig, getFallbackChain } from './matrices/general-matrix.js';
import { resolvePerplexityConfig } from './matrices/perplexity-matrix.js';
import { resolveSearchPolicy, UnsupportedCapabilityCombinationError } from './matrices/search-policy.js';
import { resolveAndValidateCapabilities, UnsupportedCapabilityCombinationError } from './matrices/search-policy.js';
import { classifyProviderError, isCircuitFailure } from './provider-errors.js';
import { OpenRouterClient, validateImageUrl, type OpenRouterClientLike } from './providers/openrouter.js';
import { PerplexityClient, type PerplexityClientLike } from './providers/perplexity.js';
Expand All @@ -19,18 +19,16 @@
GeneralModel,
Provider,
SonarModel,
TaskType,
type BudgetConfig,
type LLMResponse,
type RouterConfig,
type RoutingDecision,
type RoutingResolution,
type TaskDescriptor,
type TaskType,
} from './types.js';
import { generateFullSiteQAPlan, resolveVisionConfig, VIEWPORTS, type FullSiteQAConfig, type VisualQATask } from './vision/index.js';

const VISION_TASKS = new Set<TaskType>([TaskType.VISUAL_QA, TaskType.SCREENSHOT_ANALYSIS, TaskType.LAYOUT_VALIDATION]);

export interface RouterDependencies {
clock?: () => Date;
idFactory?: () => string;
Expand All @@ -41,26 +39,26 @@
}

export function resolveRoute(task: TaskDescriptor): RoutingResolution {
const policy = resolveSearchPolicy(task);
const audit = { taskType: task.type, complexity: task.complexity, searchRequired: policy.required, searchPolicySource: policy.source };
const imageCount = task.images?.length ?? 0;

// Fail closed before either capability can be silently discarded. The search
// plane has no multimodal transport, so a visual task carrying images cannot
// also be answered by web search.
if (policy.required && VISION_TASKS.has(task.type) && imageCount > 0) {
throw new UnsupportedCapabilityCombinationError(
`Task[${task.type}] supplied ${imageCount} image(s) and requires search, but no provider in this router serves search and vision together. Split the work into a vision task and a search task.`,
{ taskType: task.type, searchRequired: true, imageCount },
);
}
// One capability decision for the whole call: routing and dispatch both
// consume this resolution, so a request can never be interpreted one way
// at routing time and another way at dispatch time. Validation refuses
// every combination the provider plane would silently drop.
const capabilities = resolveAndValidateCapabilities(task);
const audit = {
taskType: task.type,
complexity: task.complexity,
searchRequired: capabilities.searchRequired,
searchPolicySource: capabilities.searchPolicySource,
visionRequired: capabilities.visionRequired,
};

if (policy.required) {
if (capabilities.searchRequired) {
const config = resolvePerplexityConfig(task);
return { ...audit, provider: Provider.PERPLEXITY, model: config.model, estimatedCost: config.estimatedCostPerCall, reason: config.resolutionReason };
}
if (VISION_TASKS.has(task.type)) {
const config = resolveVisionConfig(task.type as TaskType.VISUAL_QA | TaskType.SCREENSHOT_ANALYSIS | TaskType.LAYOUT_VALIDATION, task.complexity, task.images?.length ?? 1);
if (capabilities.visionRequired) {
// Validation guarantees at least one image on a vision route.
const config = resolveVisionConfig(task.type as TaskType.VISUAL_QA | TaskType.SCREENSHOT_ANALYSIS | TaskType.LAYOUT_VALIDATION, task.complexity, task.images!.length);
return { ...audit, provider: Provider.OPENROUTER, model: config.model, estimatedCost: config.estimatedCostPerCall, reason: config.resolutionReason };
}
const config = resolveGeneralConfig(task);
Expand Down Expand Up @@ -125,6 +123,16 @@
const images = task.images;
if (images) for (const image of images) validateImageUrl(image);
const decision = this.route(task);
// Consensus is a search execution modifier, not hidden routing authority:
// a non-search route would silently ignore it, so refuse the combination
// before any budget reservation.
if (options?.consensus && !decision.searchRequired) {
throw new UnsupportedCapabilityCombinationError(
'Consensus requires a search-backed route',
undefined,
'CONSENSUS_REQUIRES_SEARCH',
);
}
const governedMemory = await hydrateRouterPrompt(this.memory, decision.clientId, task.type, userPrompt);
const effectiveSystemPrompt = governedMemory ? `${systemPrompt}${governedMemory}` : systemPrompt;

Expand All @@ -149,6 +157,7 @@
reservationId = undefined;
decision.actualCost = response.cost;
decision.latencyMs = response.latencyMs;
decision.outcome = 'SUCCESS';
this.callLog.push(decision);
return response;
} catch (error) {
Expand All @@ -160,12 +169,20 @@
if (isCircuitFailure(error, decision.provider)) this.circuitBreaker.recordFailure(permit, this.clock());
else this.circuitBreaker.release(permit, this.clock());
}
// Failed routed calls are auditable too: record the classified failure
// on the decision before rethrowing. Prompt, keys, and image contents
// never enter the call log.
const classified = classifyProviderError(error, decision.provider);
decision.outcome = 'FAILED';
decision.failureKind = classified.kind;
decision.errorCode = classified.code ?? (error instanceof Error ? error.name : undefined);
this.callLog.push(decision);
if (providerCompleted) throw error;
throw this.toExecutionError(error, task, decision);
}
}

private dispatchProvider(

Check failure on line 185 in src/index.ts

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Refactor this function to reduce its Cognitive Complexity from 16 to the 15 allowed.

See more on https://sonarcloud.io/project/issues?id=Quantum-L9_LLM-Router&issues=AaAU4074l0NUu4-iakH4&open=AaAU4074l0NUu4-iakH4&pullRequest=57
task: TaskDescriptor,
decision: RoutingDecision,
effectiveSystemPrompt: string,
Expand All @@ -173,14 +190,16 @@
images: string[] | undefined,
options: { images?: string[]; assistantContext?: string; consensus?: boolean; signal?: AbortSignal } | undefined,
): Promise<LLMResponse> {
// The audited decision and the plane about to be dispatched must agree in
// both directions: a search decision may not execute on the general plane,
// and a non-search decision may not execute web search. Perplexity is the
// router's only search-capable provider.
// Dispatch consumes the resolved decision — it never re-derives the plane
// from the raw task. The audited decision and the plane about to be
// dispatched must agree in both directions: a search decision may not
// execute on the general plane, and a non-search decision may not execute
// web search. Perplexity is the router's only search-capable provider.
if (decision.searchRequired !== (decision.provider === Provider.PERPLEXITY)) {
throw new Error(`Routing decision searchRequired=${decision.searchRequired} disagrees with provider ${decision.provider}`);
}
if (decision.provider === Provider.PERPLEXITY) {
if (decision.searchRequired) {
if (decision.provider !== Provider.PERPLEXITY) throw new Error('Search decision resolved a non-Perplexity provider');
const config = resolvePerplexityConfig(task);
if (!Object.values(SonarModel).includes(decision.model as SonarModel)) throw new Error('Perplexity route resolved a non-Sonar model');
// A search route may never dispatch a config that turns search off.
Expand All @@ -199,7 +218,9 @@
}
return this.perplexity.complete(config, effectiveSystemPrompt, userPrompt, options?.assistantContext, options?.signal);
}
if (VISION_TASKS.has(task.type) && images?.length) {
if (decision.visionRequired) {
if (decision.provider !== Provider.OPENROUTER) throw new Error('Vision decision resolved a non-OpenRouter provider');
if (!images || images.length === 0) throw new Error('Vision route dispatched without images');
const config = resolveVisionConfig(task.type as TaskType.VISUAL_QA | TaskType.SCREENSHOT_ANALYSIS | TaskType.LAYOUT_VALIDATION, task.complexity, images.length);
config.model = decision.model as GeneralModel;
return this.openrouter.completeWithVision(config, effectiveSystemPrompt, userPrompt, images, options?.signal);
Expand Down Expand Up @@ -274,8 +295,13 @@
isSearchTask,
requiresSearchProvider,
resolveSearchPolicy,
resolveCapabilities,
resolveAndValidateCapabilities,
validateCapabilities,
VISION_TASKS,
UnsupportedCapabilityCombinationError,
} from './matrices/search-policy.js';
export type { ResolvedCapabilities, CapabilityConflictCode } from './matrices/search-policy.js';

export { hydrateRouterPrompt } from './memory.js';
export type { RouterMemoryConfig } from './memory.js';
11 changes: 8 additions & 3 deletions src/matrices/perplexity-matrix.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,9 @@ import {
type TaskDescriptor,
} from '../types.js';
// Re-exported for backward compatibility: `isSearchTask` historically lived in
// this module. Its canonical home is now ./search-policy.ts, and nothing here
// calls it any more — the search decision is made before a Perplexity config
// is ever resolved.
// this module. Its canonical home is now ./search-policy.ts.
export { isSearchTask } from './search-policy.js';
import { resolveSearchPolicy } from './search-policy.js';

function selectSonarModel(complexity: TaskComplexity, rank: number): SonarModel {
if (complexity === TaskComplexity.CRITICAL) return SonarModel.SONAR_DEEP_RESEARCH;
Expand All @@ -40,6 +39,12 @@ function selectReasoningEffort(model: SonarModel, complexity: TaskComplexity): '
}

export function resolvePerplexityConfig(task: TaskDescriptor): PerplexityConfig {
// Provider config and routing authority must agree: a Perplexity config is
// only ever produced for a route that resolved to search. A non-search task
// reaching this resolver is a contract violation, not a configurable state.
if (!resolveSearchPolicy(task).required) {
throw new Error('resolvePerplexityConfig called for a non-search task');
}
const rank = complexityRank(task.complexity);
const model = selectSonarModel(task.complexity, rank);
const searchContextSize = selectSearchContextSize(rank);
Expand Down
Loading
Loading