Send every prompt to the cheapest model that can actually do the job.

Overview

What it is

model-router-python is a small Python library that answers one question for every LLM call: which model should handle this? It returns a model id. You then call that model however you already do.

from model_router import Router

router = Router(
    openrouter_api_key="sk-or-...",              # or jev_api_key="jev_..."
    providers=["openai", "anthropic", "google"],   # every current model from these
)

router.route("Translate 'good morning' into French.")
# -> "openai/gpt-6-luna"

router.route("Design a lock-free hash map in Rust and prove it's linearizable.")
# -> "anthropic/claude-opus-5.5"

The problem

Why use it

Most apps and agents send every request to one flagship model, but most requests are routine: extract a field, rephrase a sentence, answer an FAQ. Flagship models cost up to 40× more per token than small ones.

Where to use it

Under the hood

How it works

Each route() call runs three steps. The first is free and runs locally. The second is skipped when only one model is left.

Get going in a minute

Quick start

Requires Python 3.12+. You pass every key in code, so no .env file or environment variable is required.

pip install model-router-python      # or: uv add model-router-python

from model_router import Router, Limits

router = Router(
    openrouter_api_key="sk-or-...",
    models=["anthropic/claude-opus-5.5", "openai/gpt-6-luna", "google/gemini-3.5-flash-lite"],
)

model = router.route("Summarize this paragraph: ...")

# Per-call limits: how much output you expect, and a hard cost cap in USD
model = router.route("Write a detailed design doc ...", Limits(output_tokens=4000, max_cost_usd=0.05))

Two setups

Configure

Pick one routing key, then tell the router which models it may choose from.

router = Router(
    jev_api_key="jev_...",                       # from jevai.org/agent/keys
    providers={
        "openai": "sk-...",
        "anthropic": "sk-ant-...",
    },
)

model = router.route(task)          # e.g. "openai/gpt-6-luna"
key = router.api_key_for(model)     # your OpenAI key

router = Router(
    openrouter_api_key="sk-or-...",              # from openrouter.ai/keys
    providers=["openai", "anthropic", "google"],   # which providers are allowed
)

Choosing candidate models

If you pass both jev_api_key and openrouter_api_key, routing goes through Jev directly.

Always up to date

Live model data

Every candidate is sent to Jev as one compact line of live data from OpenRouter's public model list:

anthropic/claude-opus-5.5 | $4.00/M in, $20.00/M out | context 1000000 | max output 128000 | est. $0.020500 for this task | Claude Opus 5.5 is Anthropic's flagship…

• Fetched once, not per request. The list is downloaded when the first Router is created, then shared by every router in the process for 24 hours. Call refresh_catalog() to download it again.
• No key needed for pricing. Providers' own APIs don't publish prices, so prices, context sizes and output limits come from OpenRouter's public list. This works with either option.
• Fits Jev's 32 KiB request limit. Descriptions shrink automatically when there are many candidates, but prices and limits are always kept. Three full providers (102 models) come to about 24 KB.

Reference

API reference

Errors

All errors are subclasses of RouterError, so one except clause covers every routing failure.

Patterns

Using it in agents

Route every LLM call separately. A planning step and a formatting step rarely need the same model. Keep a fallback so a routing outage never stalls the agent.

from model_router import Limits, NoModelFitsError, RouterError

FALLBACK = "openai/gpt-6-luna"

def pick(router, prompt, limits):
    try:
        return router.route(prompt, limits)
    except NoModelFitsError:
        raise                 # input too large for every model: shrink it
    except RouterError:
        return FALLBACK       # Jev down or rate limited: keep going

# Multi-turn chat: route on the whole transcript so the context check grows with it
transcript = "\n".join(f"{m['role']}: {m['content']}" for m in messages)
model = pick(router, transcript, Limits(output_tokens=2000))

The repo's examples/agent.py shows plan-and-execute with a total budget spread across the steps, and a chat that switches to a stronger model when the question gets hard.

Transparent pricing

Costs

The router itself

Routing cost grows with the number of candidates, because Jev reads each model's price and limits every time. For high-volume traffic, pass a short models=[...] list. Only the first 8,000 characters of a task are sent, so a single routing call costs at most about $0.0001 extra however long the prompt.

The models it picks between

OpenRouter prices, September 2026. The router always uses live prices, not this table.

Real-world estimates

Use cases

All-flagship vs. routed, using the prices above plus the cost of routing. The share of easy vs. hard tasks is an assumption, so measure it on your own traffic before relying on these numbers.

Where the savings come from

• Cheaper tokens, not fewer tokens. Your prompt is the same size either way. Easy prompts go to models that charge up to 40× less per token.
• No wasted calls. Prompts that won't fit a model's context window are rejected before any request is sent.
• No truncated answers. Models whose output cap is below your output_tokens are filtered out, so you don't pay for a cut-off answer and then again for the retry.
• Hard cost caps. max_cost_usd rules out any model whose estimated cost for this call is over budget.

Honest caveats

Limitations

• Token counts are estimated at about 4 characters per token, so the context and cost checks are approximate. Leave some headroom (#15).
• Returned ids use OpenRouter's naming. A provider's own name can differ, e.g. Anthropic's API uses claude-opus-5-5 (#16).
• route() returns only the model id, not Jev's confidence (#14).
• Routing adds one network call before each model call, except when only one model passes the limits.