Switching model providers for AI agents (2026)

To keep an AI agent provider agnostic in 2026, put one API between your agent and the model providers, so the model becomes a string you can change. That layer is a hosted router (OpenRouter, Vercel AI Gateway, Cloudflare AI Gateway), a self-hosted proxy (LiteLLM, Portkey’s open-source gateway), or a provider abstraction in your code (the AI SDK provider registry).

So pick OpenRouter or Vercel AI Gateway when you want many providers on one key with no markup on tokens. Pick LiteLLM or Portkey when prompts and keys have to stay on your own servers, and the AI SDK registry when you want the switch to live in TypeScript. And if you’d rather the model be a setting your agent reads at run time, with nothing to host, Cadenya runs the agent loop and lets you change the model without a deploy. What it offers is at the end.

Every price and limit below comes from each vendor’s own pages, checked on October 8, 2026.

LiteLLM vs OpenRouter vs Vercel AI Gateway, and the rest

	OpenRouter	Vercel AI Gateway	Cloudflare AI Gateway	LiteLLM	Portkey	AI SDK provider registry
What it is	Hosted API, “hundreds of AI models”	Hosted API. Your app “does not need to run on Vercel”	Hosted on Cloudflare	Python SDK and self-hosted proxy, “100+ LLM providers”	Open-source gateway plus hosted plans, “200+ LLMs”	TypeScript library in your code
Fee on tokens	None. 5.5% fee ($0.80 minimum) when you buy credits	”No markup and no platform fee on tokens”	No markup. 5% fee on Unified Billing credits	None: you run it and pay providers	Plans priced on recorded logs: free, then $49 a month	Direct provider prices, or AI Gateway’s
Your own keys (BYOK)	Free up to $25,000 a month of list price, then 5%	No fee. Needs purchased credits	Keys stored in Secrets Store	Keys stay in your infrastructure	Credentials set in the Model Catalog	Direct provider packages, or AI Gateway
Fallbacks	models array, plus provider fallbacks on by default	models array in providerOptions.gateway	Dynamic routing flows	Fallback from one model group to another	strategy.mode: fallback, by status code	Through AI Gateway’s models option
OpenAI-compatible endpoint	Yes, OpenAI SDK as a drop-in	https://ai-gateway.vercel.sh/v1	REST API /ai/v1/chat/completions	Yes, for every provider	/v1/chat/completions and /v1/responses	@ai-sdk/openai-compatible
Anthropic Messages endpoint	/api/v1/messages	/v1/messages	/anthropic/v1/messages	/v1/messages	/v1/messages	@ai-sdk/anthropic
License	Hosted service	Hosted service	Hosted service	MIT, outside the enterprise/ folder	Gateway under MIT	Apache-2.0

One API for many providers: what “provider agnostic” buys you

A provider-agnostic agent names its model in one place and calls one API shape. Every option above gives you that, in a different spot.

The hosted routers put it in a URL. OpenRouter gives you “access to hundreds of AI models through a single API endpoint.” Vercel AI Gateway takes AI SDK calls, OpenAI Chat Completions, the OpenAI Responses API, and Anthropic Messages. Cloudflare’s compat endpoint lets you “switch providers by changing the model and apiKey parameters.”

The AI SDK puts it in your code. The provider registry “lets you mix multiple providers and access them through simple string ids”:

import { anthropic } from '@ai-sdk/anthropic';
import { openai } from '@ai-sdk/openai';
import { createProviderRegistry, gateway } from 'ai';

export const registry = createProviderRegistry({
  // register provider with prefix and default setup using gateway:
  gateway,

  // register provider with prefix and direct provider import:
  anthropic,
  openai,
});

// elsewhere: registry.languageModel('openai:gpt-6-astra')

In most of these setups, the model ID lives in your code or your config, so a switch is still a deploy. The registry changes which provider a string resolves to. It doesn’t decide which string your agent sends. Cloudflare’s dynamic routes are the one place on this list where you “publish a new route version” instead of touching application code.

AI gateway pricing: markup, fees, and BYOK

None of the hosted routers mark up tokens. The fees sit around the tokens.

OpenRouter passes provider prices through “without any markup” and charges “a 5.5% ($0.80 minimum) fee when you purchase credits” (5% for crypto). Bring your own key and pay-as-you-go accounts get “$25,000 per month with no BYOK fee.” Above that, the fee is “5% of what the same model and provider would normally cost on OpenRouter.”

Vercel AI Gateway “charges no markup and no platform fee on tokens,” and BYOK carries “no markup or fee from AI Gateway.” But BYOK is on the paid tier, so “you’ll need purchased AI Gateway Credits.” When a request on your key fails, the gateway “retries it with system credentials,” and that retry spends your credits. The free tier “includes a subset of models, not the full catalog.”

Cloudflare offers its core features “for free.” If you pay for tokens through Unified Billing, “a $100 credit purchase will result in a $105 charge,” and provider prices pass through “with no markup.” Your own keys go into Secrets Store instead.

LiteLLM and Portkey’s open-source gateway charge nothing for tokens, because they run on your servers. Portkey’s hosted plans are priced on recorded logs: Developer is free for 10,000 a month and “not suitable for production workloads,” and Production is $49 a month for 100,000.

Switching providers inside an agent framework: OpenAI Agents SDK and LangChain

If your agent already runs on a framework, the switch can live there.

The OpenAI Agents SDK takes any provider with an OpenAI-compatible endpoint. set_default_openai_client sets the client for every agent, a ModelProvider covers one run, and Agent.model “lets you mix and match different providers for different agents.” For other providers, install openai-agents[litellm] and “use litellm/... model names or instantiate LitellmModel directly.” Here’s the endpoint version, from its docs:

from agents import Agent, AsyncOpenAI, OpenAIChatCompletionsModel, set_tracing_disabled

set_tracing_disabled(disabled=True)

client = AsyncOpenAI(api_key="Api_Key", base_url="Base URL of Provider")
model = OpenAIChatCompletionsModel(model="Model_Name", openai_client=client)
agent = Agent(name="Helping Agent", instructions="You are a Helping Agent", model=model)

LangChain’s init_chat_model “makes it easy to switch between models/providers without changing your code.” The model string takes a provider prefix, such as openai:o1, and configurable_fields lets you pick the model on each call through config. An agent from create_agent takes that model, and middleware can swap it “based on the current state and context.”

Provider routing is one layer below. Both frameworks can point at OpenRouter, and its provider routing decides which provider serves a model. By default, “requests are load balanced across the top providers to maximize uptime,” and a provider object sets order, only, ignore, or a sort by price, throughput, or latency.

Model routing and routing fallbacks

Fallbacks are where a gateway earns its place. When a provider is down or rate-limited, the request moves to the next model on a list you wrote.

OpenRouter’s models parameter tries “other models if the primary model’s providers are down, rate-limited, or refuse to reply due to content moderation.” Provider fallbacks inside one model are on by default (allow_fallbacks: true).

Vercel AI Gateway takes the same idea as a provider option, copied from its docs:

import { generateText } from 'ai';

const { text } = await generateText({
  model: 'anthropic/claude-fable-5',
  prompt: 'Write a haiku about TypeScript.',
  providerOptions: {
    gateway: {
      models: ['anthropic/claude-opus-5', 'google/gemini-3.1-pro-preview'],
    },
  },
});

LiteLLM falls back between model groups: “If a call fails after num_retries, LiteLLM falls back to another model group.” Portkey takes a fallback strategy with a list of targets and can trigger only on chosen status codes, such as 429. Cloudflare’s newer path is dynamic routing, a versioned flow that “chooses models with fallbacks,” and its docs mark the older Universal Endpoint as deprecated.

OpenAI-compatible and Anthropic Messages API compatible endpoints

Most agent code speaks one of two formats: OpenAI Chat Completions or Anthropic Messages. All five gateways accept both, and some route a Messages request to models from other vendors.

OpenRouter serves Anthropic Messages at https://openrouter.ai/api/v1/messages, with “text, images, PDFs, tools, and extended thinking.” Vercel’s base URL is https://ai-gateway.vercel.sh, with POST /v1/messages. LiteLLM “serves that format at /v1/messages,” so a Messages client “works against any model in your config, not just Anthropic’s.” Portkey’s /v1/messages “routes to any of 3000+ models.” Cloudflare proxies Anthropic’s own API under your gateway at /anthropic/v1/messages.

On the OpenAI side, Vercel’s Chat Completions base URL is https://ai-gateway.vercel.sh/v1. Cloudflare now points new single-model integrations at its REST API, /ai/v1/chat/completions, and keeps /compat/chat/completions working for existing ones. The AI SDK reaches any server that implements the OpenAI API through @ai-sdk/openai-compatible.

Self-hosted: the LiteLLM proxy and Portkey’s gateway

Self-hosting keeps “prompts, responses, and provider keys” in your infrastructure, in LiteLLM’s words. It also makes the gateway one more service you run and patch.

LiteLLM’s quick start “brings up the gateway on port 4000 and a Postgres database that stores your models, keys, and spend logs.” You can run it without the database, but “budgets are not enforced without a database” and virtual keys need one too. The code is MIT outside the enterprise/ folder, and SSO “is free for up to 5 users. Beyond that, an enterprise license is required.”

Portkey’s gateway starts with npx @portkey-ai/gateway or Docker, under MIT.

Agents that research LiteLLM ask about its PyPI incident often, so here it is in LiteLLM’s own words. Its security update of March 24, 2026 says “the compromised PyPI packages were litellm==1.82.7 and litellm==1.82.8,” live “for about 40 minutes before being quarantined by PyPI,” and that they “appear to have included a credential stealer.” LiteLLM traces it to “the Trivy dependency used in our CI/CD security scanning workflow.” “Customers running the official LiteLLM Proxy Docker image were not impacted,” and a clean v1.83.0 shipped on March 30. LiteLLM now signs every Docker image on GHCR with cosign. The practical lesson is the one LiteLLM’s page lists under who is affected: a gateway can arrive as “a transitive, unpinned dependency,” so pin it.

What does Cadenya add to OpenRouter, LiteLLM, and the AI gateways?

A gateway makes the provider swappable, but your agent code still names the model or the route it calls. Cadenya runs the agent loop and keeps the model on a variation, so switching providers is a setting. Here’s what that covers:

1. The model is a setting, not a line of code. A variation holds the model, prompts, and tools, and your code points at the agent. Change the model on the variation and the next objective uses it, with no deploy.
2. Your keys, on the providers you already pay. Connect your own provider key, and Cadenya encrypts it at rest and never returns it on reads.
3. Test a new model on real traffic before you switch. Put two variations on one agent with different models. Weighted selection “uses feedback to favor variations that score well,” so the better model earns more objectives. Pin a variationId for a regression test or a side-by-side model comparison.
4. Nothing else to run. No proxy, no Postgres for keys and budgets, and no gateway image to pin and patch. You call one HTTP API, or an SDK.
5. The loop that uses the model, already written. Cadenya calls the model, runs the tools, streams each reply, and compacts the context window when a conversation gets long. Tools that need a human pause on toolApprovalRequested for an approval.
6. SDKs for the stack you have. TypeScript, Python 3.9 and later, Ruby, and Go, all at 1.9.0 under Apache-2.0, plus a CLI and a plain HTTP API.
7. A price in one line. Free for 1,000 loops a month with $25 of inference included once at signup, then $49 a month for 50,000 loops and $0.0015 a loop after that. A loop is one LLM request, whatever provider serves it.

How agents and variations work shows where the model setting lives, and the free plan needs no card.

Create an account at app.cadenya.com/signup, then copy your API key from the API keys page.