When your app needs a model router, and when it doesn't

Somewhere in your app there is a line with a model name in it, next to a key and a URL. It was a five-second decision and it is now one of the most load-bearing things you own. When that model gets a retirement date, when its price moves, when the provider has a bad afternoon, that line is what you go and change, along with every prompt, parser and test that quietly assumed the thing behind it would keep behaving the way it did in March.

A model router is the standard answer. It is a service that sits between your code and the model vendors, takes one request in a single format, and decides which provider and which model answers it. The pitch is that you stop being married to one vendor. The part that gets less attention is that you have not removed a dependency, you have moved it, and the new one sits in the path of every AI feature you ship. This guide is for someone who calls models from code, in a product or an internal tool, at a volume where the bill is a real number. If your AI usage is a chat window and a weekly script, none of this applies to you and you should keep it that way.

A router is a switchboard, not an upgrade

The mechanics are less mysterious than the category name suggests. You send a request naming a model. The routing layer holds accounts with the providers that serve it, picks one, forwards the request, and returns the answer in a shape your code already understands.

The interesting part is the picking. OpenRouter’s default strategy load-balances between providers rather than always choosing the cheapest, weighting each provider inversely to the square of its price, so a provider three times cheaper than another is nine times more likely to be tried first [1]. It skips providers that have seen significant outages in the last 30 seconds, and if a provider returns an error it automatically falls back to the next one [1][2]. You can override all of that: sort set to price, throughput or latency turns off load balancing and walks the list in order, with :floor as the shortcut for price and :nitro for throughput, and allow_fallbacks set to false pins you to the providers you named [1].

The commercial model is worth knowing before you assume the layer is expensive. OpenRouter states that it passes through the pricing of the underlying providers with no markup on inference, and takes its cut at the point where you buy credits, at 5.5% with a $0.80 minimum for card purchases and 5% for crypto [2]. So the fee is real but small, and it is not the reason to hesitate. The reasons to hesitate are further down.

Your model has a retirement date whether you route or not

The strongest argument for a routing layer has nothing to do with saving money. It is that the thing you built on is scheduled to stop existing, and the schedule is published.

Anthropic commits to at least 60 days’ notice before retiring a publicly released model, and states plainly that requests to models past the retirement date will fail [3]. Its current table gives Claude Haiku 4.5 a tentative retirement date of not sooner than 15 October 2026, with Sonnet 5 not sooner than 30 June 2027, Opus 5 not sooner than 24 July 2027 and Fable 5.1 not sooner than 1 September 2027 [3]. Models further back have already gone: claude-opus-4-1-20250805 retired on 5 August 2026, and claude-opus-4-20250514 and claude-sonnet-4-20250514 both on 15 June 2026, two months after they were deprecated [3].

OpenAI publishes the same commitment on a different clock. Generally available models get at least six months’ notice, specialised variants at least three months, and preview models can be retired with as little as two weeks [4]. Its shutdown list has gpt-3.5-turbo-0125, gpt-4-0613 and o1-2024-12-17 going dark on 23 October 2026, and gpt-5-2025-08-07, o3-2025-04-16 and o3-pro-2025-06-10 on 11 December 2026, with GPT-5.6 Terra and GPT-5.6 Sol named as the replacements [4].

Read those two policies next to each other and the planning problem is obvious. Sixty days is enough time to change a model name and not enough to discover that your prompts, your JSON parsing and your tool-calling conventions were tuned to a model that no longer exists. The migration is not the API call. It is everything downstream of it, and you will do this on someone else’s date, repeatedly, for as long as the product lives.

Prices move on a calendar you don’t control

The second recurring event is repricing, and vendors now publish the increases in advance, which makes them easy to plan for and easy to miss.

Google lists Gemini 3.8 Flash at $0.75 per million input tokens and $3.75 per million output on the standard paid tier through 31 December 2026, then $1.50 and $7.50 from 1 January 2027 [7]. That is a doubling with a date on it, sitting in a table most people read once. Within a single vendor the ladders are steep enough that the same question comes up constantly. Anthropic prices Fable 5.1 at $10 and $50 per million tokens, Opus 5 at $5 and $25, Sonnet 5 at $2 and $10, and Haiku 4.5 at $1 and $5 [5]. OpenAI prices GPT-5.6 Sol at $4 and $20 per million tokens on its short-context tier, Terra at $2 and $12, and Luna at $0.20 and $1.20 [6], which puts Sol’s input at 20 times Luna’s. The same table charges Sol $8 and $30 once a request crosses into long context [6], so the tier your prompts land in is part of the price too.

None of that is an argument for a router by itself. It is an argument for being able to answer, cheaply, the question of what a given task costs on the rung below the one it currently runs on. If answering that question requires a branch and a deploy, you will not ask it often enough to benefit from the answer.

The part that pays is the boundary, not the product

Almost everything above is solved by a boundary in your own code rather than by a vendor. One function that takes a task name and a payload, decides which model handles it, and returns text. The model name lives in configuration, not in the call site. Nothing else in the app knows which vendor answered.

That boundary is what makes a swap cheap, and you can build it in an afternoon. The compatibility layers help: Anthropic documents using the OpenAI SDK against the Claude API by changing three things, the base URL to https://api.anthropic.com/v1/, the key, and the model name [8]. What matters more is the warning attached to it. Anthropic states that the compatibility layer “is primarily intended to test and compare model capabilities, and is not considered a long-term or production-ready solution for most use cases”, and lists what falls off, including prompt caching, audio input and the strict parameter on function calling [8]. Treat those layers as a way to price an alternative in an hour, not as the architecture.

The other half of the boundary is a fixed test set. Twenty real inputs from your own product, with the outputs you were happy with, stored in a folder. A model swap you cannot check is not a swap, it is a bet, and the failure is silent: the cheaper model keeps producing plausible output and the errors are individually too small to report.

What a router costs that is not money

Three costs, none of which appear on the pricing page.

The first is the data path. Every prompt and every response now passes through a third party, because that is how routing and billing work. OpenRouter says prompts and completions are not logged by default, and offers a 1% discount on usage for opting into logging [2]. Its routing controls let you set data_collection to deny, which uses only providers that do not collect user data, and zdr: true, which routes only to endpoints with a zero-data-retention policy [1]. Those settings exist because the default behaviour of the pool underneath you is not uniform. Decide the retention posture before the first production request, and check that it matches what your customers were told, because it is the setting nobody reads and the one a security review finds.

The second is that you have concentrated failure rather than removed it. Provider failover is genuine value, and it is the clearest thing a router buys [2]. It is also true that when the router is unavailable, every model is unavailable to you at once, and a direct integration with one vendor would have been down only when that vendor was. Whether that trade is good depends on whether you actually configure the failover, or leave the defaults and hope.

The third is the quiet one. Interchangeability is bought by giving up the vendor-specific features that do the most for your bill. Cached input on OpenAI costs $0.40 per million on short-context Sol against $4.00 uncached [6], and Anthropic’s cache reads cost $0.25 per million on Fable 5.1 against $10 for ordinary input [5]. Caching is held by a provider for your traffic, so a layer whose job is to spread requests across providers is working against the mechanism that saves you the most, and the OpenAI compatibility layer does not support prompt caching at all [8]. Batch processing has the same shape, at 50% off inputs and outputs on both vendors [5][6]. Before adopting a router to cut costs, check that you are not giving up a larger discount to chase a smaller one.

Volume and variety decide this, not architecture taste

The honest test has two parts. Volume: is your monthly model spend large enough that moving part of it down a rung is worth an afternoon and an ongoing maintenance job. Variety: do you run several distinct tasks with genuinely different cost and quality needs, or one task at one quality bar. A router earns its place when both answers are yes. With one task and a small bill, the boundary in your own code gets you most of the benefit and none of the new dependency.

If that number is negative, you have your answer, and it will stay negative until either the spend grows or the maintenance shrinks. If it is positive but small, the boundary is still worth building and the router is not, because the boundary is what lets you take the same saving by hand on the two tasks that account for most of the bill.

What still goes wrong

The savings numbers that sell routing come from continuously moving work to whatever is cheapest at an acceptable quality, and the phrase doing the work is “at an acceptable quality”. Two models that look equivalent on a public benchmark can behave differently on your formatting, your tool-calling conventions and your edge cases, and a layer that swaps models to save money can change your product’s behaviour without anything failing. The test set is the only defence, and a test set of twenty inputs is a smoke alarm, not a guarantee.

Retirement dates are published but not fixed. Anthropic’s own table describes its future dates as tentative and as “not sooner than” a given day [3], which is a floor rather than a promise, and preview models at OpenAI can go with two weeks’ notice [4]. Any plan that assumes you will get a comfortable window is a plan that will eventually be executed in a hurry. The useful habit is a calendar reminder each quarter to open the deprecation pages of the vendors you name and check whether anything you depend on has moved.

Finally, the layer is only as good as the attention you give it. A router configured once and forgotten routes to whatever its defaults prefer, retains whatever its defaults retain, and fails over to providers you never evaluated. That is not worse than a hard-coded model name in every way, but it is worse in the way that matters most, which is that you can no longer say with confidence which model answered your users this week.