How Routing Works

Back to home

Every routing decision runs the same deterministic pipeline, in the order below. Same inputs always produce the same decision, and every stage is visible in the output of offrouter route explain.

1. Policy Gates

Policy runs first. Every candidate provider/model pair is checked, and anything that fails is removed with a structured denial: a machine-readable code, a target, and a message. Denied candidates are never resurrected by later stages.

Profile and workspace gates

• Work profiles are denied by default (work_profile_denied): any profile ending in -work or containing -work-, plus anything matching denied_profile_patterns.
• The active profile must be listed in allowlisted_profiles (profile_not_allowlisted).
• An untrusted workspace stops routing entirely (project_untrusted).

Per-candidate hard filters

• denied_providers removes a provider outright (provider_denied).
• Per-provider allowed_models and denied_models restrict model ids (model_not_allowed). Patterns are exact matches or a single trailing * prefix glob such as "gpt-5.6-*". Deny beats allow, and the lists apply to every tier.
• Task constraints remove candidates that cannot satisfy them: localOnly (local_only_required), maxCostUsd (cost_cap_exceeded), and required capabilities such as tools or images (capability_missing).
• Subscription candidates reached through a third-party or unknown auth surface are denied (subscription_scope_denied) unless allow_third_party_subscription_adapters is set.
• Dead providers and exhausted, rate-limited, or expired subscriptions are removed (provider_denied).

Paid-spend gates (API-key candidates only)

• Spend caps: when estimated paid spend for the current accounting window (cumulative since the usage file was last reset, not daily or monthly) meets or exceeds spend_cap_usd, the API-key candidate is denied (spend_cap_exceeded). A per-provider spend_cap_usd under [providers.<id>] overrides the global cap. Subscription and local candidates are never blocked by spend caps.
• Approval: with require_api_spend_approval = true, every API-key candidate is denied (api_spend_approval_required) unless the evaluation carries an explicit approval — --approve-api-spend on the CLI or approve_api_spend on MCP tools.

Outbound secret scan

Before any provider call, outbound prompt text is scanned once for credential shapes. The outbound_secret_scan policy key picks the mode: "warn" (the default) proceeds and audits the finding, "block" fails the request before any network call (outbound_secret_blocked), and "off" skips the scan. Findings name the credential pattern only; matched values are never logged.

2. User-Defined Routing Rules

After policy gates, user-defined [[routing.rules]] are evaluated in order. The first rule whose match keys all hold selects a preference among the allowed candidates:

• model or provider pins filter the candidate list to matching allowed entries. If the pinned candidate is denied by policy, the rule is skipped and evaluation continues.
• tier preferences reorder allowed candidates so the named tier ranks first, but they never cross a policy gate or resurrect a denied candidate.

Rules match on caller-supplied role (exact equality) and/or the classifier's family signal. Proxy traffic has no role surface in V1, so proxy requests can only match family rules. If no rule matches, the pipeline continues in auto mode.

3. Subscription-First Tier Order

Surviving candidates are ranked, and the primary sort key is always the auth tier:

1. Healthy subscription (active)
2. Healthy subscription near its limit
3. Local runtime
4. Degraded subscription
5. API key

This order is a hard invariant: nothing below can cross it. A sick subscription still beats a healthy API key, and no health score or classifier signal ever promotes an API-key candidate over a subscription candidate. Policy reinforces the same rule separately: while a healthy subscription can satisfy the task, API-key candidates are denied outright (api_key_blocked_by_subscription_first_policy), and with allow_api_key_fallback = false they stay denied even without one.

When the winning subscription is near its limit and no healthier subscription alternative exists, the decision carries a near-limit warning telling you which account is affected.

4. Health Re-Ordering Within a Tier

Inside a single tier, recent outcomes reshuffle candidates before cost is considered. OffRouter keeps a small per-(provider, model) record of successes, failures, and an exponentially weighted latency average:

• Each recent failure subtracts 100 points from a neutral baseline of 0 — failures dominate the score.
• Each recent success adds 1 point, capped at 20 successes, so a recovering provider climbs back above its peers.
• Latency is a tiny tie-break only: a full second of average latency moves the score by about 0.01, never enough to override a failure.
• Anything older than 10 minutes stops counting. Scores decay back to neutral, so a provider that had a bad morning is not penalized forever.
• Unknown providers score exactly neutral — a brand-new candidate is never penalized for having no history.

Health tracking is automatic: there are no config keys for it, and it never crosses tier boundaries.

5. Classifier Signal Within a Tier

A deterministic task classifier inspects the prompt preview and its length — no network, no LLM, no config keys — and recommends a logical model family: fast-coder or frontier-coder. The rules are simple heuristics. Prompts over 2,000 characters, multiple code fences, and multi-step verbs like refactor or migrate push toward frontier; short prompts (280 characters or less) with mechanical verbs like rename or format push toward fast. Ambiguous input defaults to frontier-coder at low confidence, so complex work is never quietly under-served.

The recommendation is a signal, not authority. Only a high-confidence classification nudges matching-family candidates up within their tier, after health re-ordering. Medium and low confidence change nothing and are recorded for explain output only. The classifier never crosses tiers, never resurrects a policy-denied candidate, and policy always has the final say.

6. Cost and Id Tiebreaks

If candidates are still tied after tier, health, and classifier ordering, cheaper wins: lower estimated cost first (an unknown cost is treated as $0.05), then provider id, then model id, both alphabetical. The result is fully deterministic — the same inputs always produce the same decision.

Inspecting Decisions

offrouter route explain "summarize this file" --profile claude-personal
offrouter status --costs
offrouter doctor

• offrouter route explain runs the full pipeline locally — no provider is called — and prints the winner, a human-readable explanation, the ordered fallback chain, and every structured policy denial. Add --json for the raw decision object.
• offrouter status --costs prints the estimated paid-spend rollup for the current accounting window: per-provider totals, configured caps, and remaining headroom. When pricing for a model is unknown, the affected totals are flagged as incomplete instead of guessed.
• offrouter doctor reports Node version, config validity, harness detection, provider readiness, and the MCP serve path. Secret values are never printed.
• Over MCP, agents get the same read-only introspection: offrouter_models (models from the built-in catalog and configured providers), offrouter_usage (provider health and per-account quota usage), and offrouter_config_check (config validity, sources, allowlisted profiles, and warnings).