Model Routing with Jev

Most agent requests do not need the most expensive model. A lookup, an extraction, or a localized edit can run on something small; architecture and high-stakes decisions cannot. Model routing is the practice of sending each request to the cheapest model that can complete it — and Jev is a fast, cheap way to make that call.

Measure the router as part of the whole agent run. A faster classification step is not a useful optimization if it damages cache reuse, picks an insufficient model, or increases retries and review.

The Cache Constraint

Naive model routing destroys prompt caching. If you switch the main model between turns, the cached prefix is invalidated and every subsequent call pays full input cost. That can erase the routing savings entirely.

The design that works:

• Classify short-lived subagent work when it is useful. Cache loss is less likely to compound for isolated work.
• Avoid switching a cache-sensitive main model repeatedly. Decide once per stable session or measure whether the savings exceed cache misses.
• Let Jev adjust effort level mid-session. Effort and specialist selection can change without swapping the base model.

The details depend on the host's cache implementation, so validate this policy with real request and cache-hit telemetry.

The Routing Decision

Represent the model classes as a Choice and let the result drive selection:

const { answers } = await client.systemOne({
  state: {
    request,
    files_touched: files,
    recent_context: recentTurns,
  },
  questions: {
    model_class: choice('Which model class is least costly but still sufficient?', {
      fast: 'Direct lookups, extraction, formatting, localized single-file changes',
      powerful: 'Architecture, multi-file refactors, ambiguous requirements, high-stakes decisions',
      human: 'Sensitive, outside policy, or unrecoverably destructive',
    }),
    effort: score('How much reasoning does this request need?', [
      'No reasoning, retrieval or formatting only',
      'Some reasoning, a few connected steps',
      'Deep reasoning across the whole codebase',
    ]),
  },
})

const model =
  answers.model_class.choice === 'fast' ? 'small-model'
  : answers.model_class.choice === 'powerful' ? 'frontier-model'
  : null

if (model === null || answers.model_class.confidence < 0.55) {
  return escalateToHuman(request)
}

The instruction matters: "least costly but still sufficient," not "which model is best." Without the cost framing, the classifier biases toward the powerful option.

Do Not Confuse Routing with Permission

Routing decides which model runs. It does not decide what that model is allowed to do. A cheap subagent that can still push code, send email, or hit a customer API carries the same blast radius at a lower bill.

Stage the destructive capabilities after the router picks:

1. Route the request to a model class.
2. Then grant tools according to the same risk policy you would apply to any caller.
3. Keep write and irreversible actions gated regardless of which model was chosen.

The classifier narrows who runs; the permission layer decides what they may do. Keep them separate.

Audit the Policy

Model routing turns into silent quality drift the moment nobody can see it. Two requirements keep it honest:

• Log the chosen model, the effort level, and the answer that produced them. Turn optimization into something a team can review after the fact.
• Allow per-repo or per-project overrides. A team that disagrees with a classification should be able to pin a decision without editing the router.

Framework Support

The Vercel AI SDK exposes Jev as an evaluation model. Framework middleware changes quickly, so use the integration's current documentation and pin the package version before adopting an experimental router. See Integrations for the supported evaluation path.

Measure the Right Thing

Per-call cost is the wrong metric. What matters is cost per solved task, which includes the retries and human review a wrong route causes. A cheap decision that sends a request down the wrong branch costs more than the decision saved.

Track end-to-end task success and cache stability alongside the routing cost. If a router improves average latency but raises the escalation rate, it has moved cost rather than removed it.

Related

• Confidence Gating — the bands that decide when to act versus escalate
• Skill Routing — the same idea applied to skills instead of models
• Integrations — LangChain and Vercel SDK support