OpenRouter alternatives for production workloads: what Cloudflare, Bedrock, Vertex, Groq, LiteLLM and Portkey actually document

There is no ranked list that answers this question, because the products people offer as “OpenRouter alternatives” are three different shapes: cloud catalogs (AWS Bedrock, Google’s Agent Platform on Vertex), a single inference vendor with an OpenAI-shaped door (Groq), and gateways or proxies sitting in front of provider keys you already hold (Cloudflare AI Gateway, LiteLLM, Portkey). What a production team needs to know about any of them fits in six columns — failover, spend caps, protocols, retention, rate-limit units, logs — and the matrix below reads the seven products’ own documentation on all six, as the pages stood on 2026-08-29, writing “not stated” where the docs are silent. The last section says which column to read first for which workload.

Three shapes of “alternative”, and why a ranked list cannot hold them

Groq is in the table because AI answers keep citing its docs here; it runs open-weight models fast, and is not a router.

The matrix: six columns, seven products, read from the docs on 2026-08-29

Legend: ✓ documented, standard path · ⚠ documented but plan-gated, narrower, or carrying a caveat quoted below · ✗ not offered per the vendor’s own scope statement, or the docs say nothing — the cell says which.

Six sections follow, one per column, with the sentences behind each cell. Abbreviations, as the vendors use them: ZDR zero data retention; TPM/RPM/RPD/TPD tokens or requests per minute or day; mantle Bedrock’s OpenAI-compatible endpoint, distinct from bedrock-runtime.

Failover: who moves traffic across vendors, and who only moves it across regions

The gateways and OpenRouter document failover the way I dissected in Automatic failover in LiteLLM, Portkey and OpenRouter: a trigger list, a retry policy, a cooldown, unchanged on the pages I re-read since 2026-08-25. LiteLLM: “Fallbacks are how LiteLLM does automatic failover. If a call fails after num_retries, LiteLLM falls back to another model group” (Proxy — Fallbacks, Load Balancing, Retries). Portkey: “By default, fallback triggers on any non-2xx status code” (Fallbacks). OpenRouter ranks providers that “have not seen significant outages in the last 30 seconds” first and publishes no retry count, cooldown or timeout (Provider Routing); its Limits page adds that on an upstream 429, “fallback routing retries other providers for the same model automatically before the error reaches you.”

Cloudflare’s cell is ⚠ for a documentation reason: the Fallbacks page — “By default, Cloudflare triggers your fallback if a model request returns an error” — is written against the Universal Endpoint, whose own page now opens with “The Universal Endpoint is deprecated. Use the OpenAI-compatible endpoint … for new integrations, and Dynamic Routing for fallbacks, retries, and conditional routing” (Universal Endpoint (Deprecated)). Dynamic routing is a visual or JSON flow of Model, Conditional, Rate Limit and Budget Limit nodes, each Model node naming a provider — so cross-vendor failover exists, but it is marked Beta and “is invoked through this /compat/chat/completions endpoint. The REST API does not currently cover dynamic routing” (Unified API (OpenAI compat)). Per-request retries are precise: “a maximum of five retry attempts,” delay up to 5 seconds, constant, linear or exponential (Request handling).

Bedrock has two features that read like failover and are not. Intelligent prompt routing routes “between different foundational models within the same model family”; you “must choose exactly two models within the same family,” and its “fallback model” is a quality anchor, not an error handler (Understanding intelligent prompt routing in Amazon Bedrock). Cross-Region inference “automatically selects a commercial AWS Region within that geography to process your inference request”; the page covers residency, a roughly 10% saving on the Global profile, and CloudTrail’s inferenceRegion field, and does not state which errors, if any, move a request between Regions (Route model inference requests across AWS Regions). Retries are yours: “Use retry logic with exponential backoff to handle transient throttling” (Quotas for the bedrock-mantle endpoint).

Vertex and Groq have no failover row to fill. Google’s advice on 429 is “Use the global endpoint instead of a regional endpoint whenever possible” and “truncated exponential backoff” (Error code 429). Groq’s error page is candid on one point: “You will not be charged for requests that return server error codes” (API Error Codes and Responses). A refund is not a second provider.

Spend caps: per key, per metadata dimension, per organization, or a tag on the bill

I mapped OpenRouter, LiteLLM and Portkey cell by cell in Per-key spending limits: OpenRouter caps a key with limit + limit_reset and answers 402; LiteLLM caps a virtual key with max_budget, and needs a database; Portkey caps an API key in USD or tokens and answers 412 Precondition Failed, with the feature gated — “Available on Enterprise plan and select Pro customers” (Enforce Budget Limits and Rate Limits). Where the shared wallet sits relative to the caps is its own article.

Cloudflare’s version is Beta and scoped differently from all three: spend-limit rules on the gateway, “to any combination of model, provider, or custom metadata dimensions like user ID, team, or application” — a per-user budget is a metadata key set to “Split by value,” not a property of a credential. Its caveats are stated next to the feature: “Spend limits are eventually consistent. The current request’s cost is recorded after completion, so a burst of concurrent requests can briefly exceed the limit”; “A maximum of 20 spend limit rules can be configured per gateway”; cost tracking is “a best-effort estimation” (Spend limits). A hard number per key this is not; a soft number per dimension it is.

Groq has one cap and it is the organization’s: “The limit applies organization-wide across all API keys … API calls from any key in your organization will return a 400 with code blocked_api_access.” Monthly only, “Limits reset on the 1st of each month,” tracking “updates every 10-15 minutes,” and “you might exceed your limit by a small amount during high usage periods.” Per key? “A: No” (Spend Limits).

Bedrock’s cost pages are about attribution, not caps: the native methods “deliver aggregated billed dollars to AWS Cost Explorer and CUR 2.0. The finest grain is per usage type per day … they do not produce a per-request row” (Track usage and costs in Amazon Bedrock); Projects on bedrock-mantle add tags for Cost Explorer (Projects (OpenAI-compatible)). Nothing on these pages blocks a request at a number. Google’s generative-AI pages I read say nothing about a spend cap at all; both clouds’ account-level budget tools live outside these docs.

Protocols: OpenAI-compatible is table stakes; Messages and Gemini-native split the column

All seven speak Chat Completions. The other three formats are where migrations succeed or stall.

Bedrock now serves three protocols on two endpoints: the endpoints page lists bedrock-runtime as “InvokeModel / Converse / Chat Completions / Responses API / Messages API” and bedrock-mantle as “Responses API / Chat Completions API / Messages API,” both reachable “by changing only the base URL and API key” (Endpoints supported by Amazon Bedrock). The surfaces differ at the edges: on mantle, Messages requests with output_config.format “are rejected with a 400 error”; on bedrock-runtime, Responses requests with background=true are rejected the same way. No Gemini path — no Google model in the catalog.

Cloudflare exposes each provider’s native path under a prefix — /anthropic/v1/messages (Anthropic) and /google-ai-studio/v1/models/{model}:generateContent (Google AI Studio) — plus the OpenAI-compatible REST API, whose supported-provider list runs to 14 names. LiteLLM’s /v1/messages works with “All LiteLLM supported providers” (/v1/messages), and its Gemini pass-through is one substitution: “Just replace https://generativelanguage.googleapis.com with LITELLM_PROXY_BASE_URL/gemini” (Google AI Studio SDK). Portkey lists three formats and says “Each works with all providers — Portkey handles translation automatically” (Universal API); its Messages page names the production case — tools “like Claude Code and the Claude Agent SDK — work with any backend model through Portkey without modification” (Messages). OpenRouter carries POST /messages, “the Anthropic Messages API format. Supports text, images, PDFs, tools, and extended thinking” (Create a message); I found no Gemini-native endpoint in its index.

Google’s compatibility layer is narrower than its name. “The Chat Completions API works as an Open AI-compatible endpoint,” and it “supports both Gemini models and select self-deployed models from Model Garden” — Gemini plus containers running Gemma, Llama, Mistral (Using OpenAI libraries with Gemini Enterprise Agent Platform). Claude on Vertex is a separate door — “send a request directly to the Agent Platform API endpoint” (Anthropic’s Claude on Google Cloud models), identified elsewhere as the partner models “that use rawPredict or streamrawPredict.” Porting an OpenAI client, mind two parameters: tools.parameters follows “the OpenAPI specification. This differs from the OpenAI parameters field,” and tool_choice: validated is “Google-specific.”

Groq is “mostly compatible with OpenAI’s client libraries” at https://api.groq.com/openai/v1, also serves the Responses API, and names its gaps: logprobs, logit_bias, top_logprobs and messages[].name “will result in a 400 error,” n must be 1, temperature 0 becomes 1e-8 (OpenAI Compatibility). If your codebase sends logit_bias, that is a one-line grep before the migration, not after.

Data retention: “ZDR” means at least four different mechanisms

Every vendor here uses the phrase zero data retention or a close cousin, for different objects.

OpenRouter’s ZDR is an endpoint attribute you route on: privacy settings “only allow you to route to endpoints that have a Zero Data Retention policy. You can enforce ZDR globally, per model group, per guardrail, or per request” — per request is provider: { zdr: true }. When a provider’s policy is unclear, OpenRouter assumes “that the endpoint both retains and trains on data and mark[s] it as such”; in-memory prompt caching is “not considered ‘retaining’ data”; the list is machine-readable at /api/v1/endpoints/zdr (Zero Data Retention). Its own stance: “your prompts are not retained unless you specifically opt in to prompt logging” (Data Collection).

Cloudflare’s ZDR is a billing-path setting: it “routes Unified Billing traffic through provider endpoints that do not retain prompts or responses … only applies to Unified Billing requests that use Cloudflare-managed credentials. It does not apply to BYOK,” is “currently supported for: OpenAI, Anthropic,” and — underline this — “ZDR does not control AI Gateway logging” (Unified Billing). The gateway’s own logs, “which include metrics as well as request and response data, are enabled by default”; turning them off is a settings switch or per-request cf-aig-collect-log: false (Logging). Nothing stored anywhere means two switches on two pages.

Bedrock’s ZDR is a retention mode that gates which models you can call. Default: “by default, Amazon Bedrock does not store model inputs or outputs,” with named exceptions — “For Anthropic Claude Fable 5, inputs and outputs will be retained for up to 30 days,” and using it requires opting in to sharing retained traffic with Anthropic (Amazon Bedrock abuse detection). The control is data_retention_mode per account or project; under none, “No request or response data is written to durable storage by AWS or shared with the model provider,” and a model whose allowed_modes excludes your mode “will appear as status: "unavailable" … and requests to it will be blocked” (Data retention). Strictest control in the table; the price is losing models that require retention. Also: “At launch, there is no console UI for configuring data retention.”

Groq’s ZDR is a toggle over a no-retention default: “By default, Groq does not retain customer data for inference requests”; reliability and abuse logs are the exception, “up to 30 days”; “All customers may enable Zero Data Retention (ZDR) in Data Controls settings” — globally or per feature — which disables batch and fine-tuning; retained data sits “in Google Cloud Platform (GCP) buckets located in the United States” (Your Data in GroqCloud).

Google’s is a checklist, not a switch. Training is excluded by contract, but retention has a list: abuse-monitoring prompt logging (“you can request an exception”); “Zero data retention may not be possible when using some Advanced AI features”; Grounding with Google Search keeps query logs “for up to three (3) days” with “no way to disable the storage”; the Interactions API defaults store to true; Gemini’s in-memory cache has “a 24-hour TTL” and “can be disabled at the project level” (Gemini Enterprise Agent Platform and zero data retention).

LiteLLM’s answer is architectural — “No data or telemetry is stored on LiteLLM Servers when you self-host” (Data Privacy and Security) — plus redaction switches (turn_off_message_logging, per-request headers) on the logging page. Portkey’s retention is a plan attribute — Developer “3 day Log Retention,” Production 30 days, Enterprise unlimited (Logs) — with x-portkey-debug:false per request and an org-wide “Metrics Only (Privacy Mode)” that is Enterprise and “will not retroactively remove previously logged data” (Configure Request Logging). Behind a BYOK proxy, what the provider keeps is the provider’s policy; neither vendor claims otherwise.

Rate limits: the unit of the quota is the whole story

Comparing numbers across these seven is meaningless; comparing what they count is not.

Bedrock counts tokens per model per Region, twice. On bedrock-runtime, quotas are “tokens per minute (input + output, combined),” a daily cap defaulting to the per-minute quota × 24 × 60, RPM “enforced for some models … and not others,” and “shared across every inference API you call it with” (Quotas for the bedrock-runtime endpoint). On bedrock-mantle, input and output TPM are separate quotas, “does not enforce requests-per-minute (RPM) quotas,” admission counts request tokens “plus the value of max_tokens,” cached input tokens are free, and the one published default at reading time was Claude Opus 4.7 at 20,000,000 in / 4,000,000 out (Quotas for the bedrock-mantle endpoint). Two endpoints, two pools, “even when calling the same underlying model.”

Google counts an organization’s 30-day spend: Standard PayGo tiers set a TPM baseline — Gemini Pro from 500,000 TPM at Tier 1 ($10–$250) to 2,000,000 at Tier 3 (over $2,000), Flash up to 10,000,000 — with “no separate requests-per-minute (RPM) limit” and traffic allowed to “burst beyond this limit on a best-effort basis” (Standard PayGo). Guaranteed capacity is the Provisioned Throughput subscription, and the status code moves with it: on standard PT, while under the throughput you bought, “errors that might otherwise be 429 are returned as 5XX and count toward the SLA error rate” — Single Zone PT is the documented exception, where they do not count (Error code 429). A vendor telling you which status code you pay for.

Groq counts the organization: “Rate limits apply at the organization level, not individual users,” in RPM, RPD, TPM, TPD and audio-seconds units, cached tokens excluded. Its headers are precise about referents: x-ratelimit-limit-requests “Always refers to Requests Per Day (RPD),” x-ratelimit-limit-tokens to TPM, and retry-after appears only on a 429. Published Developer-plan base at reading time: openai/gpt-oss-120b at 30 RPM / 1K RPD / 8K TPM / 200K TPD (Rate Limits).

OpenRouter counts globally — “Making additional accounts or API keys will not affect your rate limits, as we govern capacity globally” — and publishes numbers only for free variants: 20 requests per minute, 50 or 1,000 per day by credit history (Limits). Cloudflare counts requests per window per gateway, fixed or sliding, “uniformly applied to all requests for that gateway” (Rate limiting); per-user limits are a Dynamic Route node. LiteLLM counts per credential — tpm_limit and rpm_limit on keys, teams, users, per-team per-model limits “inherited by keys unless overridden” (Budgets, Rate Limits). Portkey counts per API key or provider, requests or tokens per minute, hour or day, behind the same Enterprise gate as its budgets; once set, provider-level limits “cannot be edited by any organization member,” and “Setting the limit to 0 disables the provider” (Rate Limits).

Observability: where the per-request log row lives

Ask where one request’s tokens, cost and outcome can be found afterwards, and whether it is on by default.

OpenRouter puts it in the response: “Full usage details are now always included automatically in every response,” including cost, cost_details.upstream_inference_cost and cached and reasoning token counts (Usage Accounting); Broadcast forwards traces to 19 documented destinations — Datadog, Langfuse, OTel, S3, BigQuery among them — with per-destination key filters, sampling and a Privacy Mode that strips prompt and completion content (Broadcast).

Cloudflare stores it by default — “user prompt, model response, provider, timestamp, request status, token usage, cost, duration” — capped at 100,000 logs per free account against 10 million per paid gateway, two orders of magnitude apart, exportable via Logpush on Workers Paid (Logging · Limits). Its cost column is “an estimation … refer to your provider’s dashboard for the most accurate cost details” (Costs).

Bedrock’s row is off until enabled, and only on one door: “Model invocation logging is disabled by default.” It writes bodies up to 100 KB with token counts and identity.arn to CloudWatch or S3, and “is only supported for calls made through the bedrock-runtime endpoint … the same APIs on bedrock-mantle are not currently captured” (Monitor model invocation using CloudWatch Logs and Amazon S3). Behind a gateway, “Amazon Bedrock records the gateway’s IAM role as the caller’s identity”; per-user rows need request metadata or role sessions (Track usage and costs).

Google’s row is a Preview that samples: request-response logging writes “samples of requests and responses” to BigQuery with a sampling_rate; “For Anthropic models, only REST is supported for logging configuration” (Log and share requests and responses). LiteLLM’s row goes wherever a callback points, and every response carries x-litellm-call-id and x-litellm-response-cost (Logging). Portkey’s log records tokens, thinking tokens and cost plus a Status column (Retry Success on {x} Tries, “Fallback Active”) (Logs); its OTel export comes in two grades, Analytics Stable, Complete Logs Experimental (OpenTelemetry (OTel) Export). Groq’s docs mention a console logs page; the pages I read document no export.

Which column to read first, by workload

Pick the column your next incident comes from; read its seven cells first.

• A fleet of agents on many keys. The spend-cap column. OpenRouter and LiteLLM cap the key; Cloudflare caps a metadata dimension, eventually consistently; Groq caps the organization once a month; Portkey caps the key on Enterprise; the clouds tag the bill and do not block. Getting this wrong costs one intern’s weekend, which is why it is first.
• Regulated data. The retention column, then the vendor’s page, not this one. Bedrock’s data_retention_mode: none is the only control here that refuses models that would retain; OpenRouter’s ZDR is the only one filtering endpoints across vendors; Cloudflare’s needs a second switch for its own logs.
• Tooling on the Anthropic Messages format, Claude Code included. The protocol column. Portkey, LiteLLM, Cloudflare, OpenRouter and Bedrock accept it; Groq and Vertex’s compat layer do not.
• Bursty batch work. The rate-limit column, unit before number: per-model TPM pools (Bedrock), org-level spend tiers with best-effort burst (Google), org-level headers (Groq), global capacity with no paid figures published (OpenRouter).
• Open-weight models, lowest latency, one vendor. Groq, eyes open: three ✗ or ⚠ cells on failover, per-key caps and Messages, plus a monthly org-wide cap with a 10–15 minute lag. A gateway from the third row in front of it buys back the first columns.

Three lines to grep in whatever you run: the retry and cooldown values, the credential-level budget field or its absence, and the log-collection default. Two questions for any vendor whose cell says “not stated”: which error classes move a request off a provider, and whether a burst can overshoot a cap before the meter catches up — three of the seven admit the overshoot in their own words. We are building a router ourselves, so this matrix doubles as the checklist we hold our own docs to (current state). Everything above is what the vendors wrote on 2026-08-29; the links are there so you can check whether it still is.