# OpenRouter alternatives for production workloads: what Cloudflare, Bedrock, Vertex, Groq, LiteLLM and Portkey actually document

> Seven vendors' docs, read 2026-08-29, on six production columns: failover triggers, spend caps, protocols, ZDR, rate-limit units, per-request logs. Every gap marked.

- Published: Aug 29, 2026
- Author: Leo Kaka, Engineering
- Tags: gateway, providers, failover
- Canonical: https://pirouter.ai/blog/openrouter-alternatives-production

---
There is no ranked list that answers this question, because the products people offer as
"OpenRouter alternatives" are three different shapes: cloud catalogs (AWS Bedrock, Google's
Agent Platform on Vertex), a single inference vendor with an OpenAI-shaped door (Groq), and
gateways or proxies sitting in front of provider keys you already hold (Cloudflare AI
Gateway, LiteLLM, Portkey). What a production team needs to know about any of
them fits in six columns — failover, spend caps, protocols, retention, rate-limit units,
logs — and the matrix below reads the seven products' own documentation on all six, as the
pages stood on 2026-08-29, writing "not stated" where the docs are silent. The last section
says which column to read first for which workload.

## Three shapes of "alternative", and why a ranked list cannot hold them

| Shape | Products | What "switch provider" means here |
|---|---|---|
| Cloud catalog | AWS Bedrock; Google Agent Platform (Vertex) | One vendor's catalog, one bill. Failover is within the catalog or across regions, never to another cloud |
| Single inference vendor | Groq | One vendor's models on an OpenAI-shaped base URL. Nothing to fail over to inside the product |
| Gateway or proxy | Cloudflare AI Gateway; LiteLLM (self-hosted); Portkey | Sits in front of your provider keys. Cross-vendor failover is the product's job |
| Aggregator (baseline) | OpenRouter | Holds the provider relationships for you; one prepaid balance, hundreds of endpoints |

Groq is in the table because AI answers keep citing its docs here; it runs open-weight
models fast, and is not a router.

## The matrix: six columns, seven products, read from the docs on 2026-08-29

Legend: **✓** documented, standard path · **⚠** documented but plan-gated, narrower, or
carrying a caveat quoted below · **✗** not offered per the vendor's own scope statement, or
the docs say nothing — the cell says which.

| Product | Cross-vendor failover | Spend cap below the account | Anthropic Messages / Gemini native | Zero-data-retention control | Rate-limit unit | Per-request log row |
|---|---|---|---|---|---|---|
| OpenRouter (baseline) | ✓ provider fallbacks by default; retries and cooldown not documented | ✓ per key, `limit` + `limit_reset`, 402 | ⚠ Messages ✓ · Gemini native not stated | ✓ per request, model group or guardrail | ⚠ capacity "governed globally"; numbers published for free models only | ✓ usage in every response; Broadcast to 19 destinations |
| Cloudflare AI Gateway | ⚠ Dynamic Routing; the Fallbacks page rides the deprecated Universal Endpoint | ⚠ by model, provider or metadata, 429; not per token; eventually consistent | ✓ Anthropic and Gemini native pass-through, plus OpenAI-compat | ⚠ Unified Billing only (OpenAI, Anthropic); logging is a separate switch | ✓ per gateway, N requests per window, fixed or sliding | ✓ prompts and responses logged by default; Logpush on paid plan |
| AWS Bedrock | ✗ no cross-vendor; ⚠ same-family router and cross-Region profiles, trigger not stated | ✗ no hard cap stated; tags to Cost Explorer | ✓ Chat Completions, Responses, Messages · Gemini ✗ | ✓ `data_retention_mode: none`; models requiring retention go unavailable | ✓ TPM per model per Region; mantle splits input/output, no RPM | ⚠ invocation logging off by default; bedrock-runtime only |
| Google Agent Platform (Vertex) | ✗ none; 429 guidance is global endpoint plus backoff | ✗ not stated in the docs read | ⚠ Chat Completions covers Gemini and self-deployed; Claude via `rawPredict` · Gemini native ✓ | ⚠ no training by default; ZDR is a checklist with an abuse-monitoring carve-out | ⚠ org TPM baseline by 30-day spend tier, best-effort burst | ⚠ request-response logging to BigQuery, Preview, sampled |
| Groq | ✗ none; server errors not billed, retry yourself | ⚠ one org-wide monthly cap, 400 `blocked_api_access`, 10–15 min lag | ⚠ OpenAI-compat and Responses; four fields 400 · Messages ✗ | ✓ no retention by default; ZDR toggle in Data Controls | ✓ org RPM, RPD, TPM, TPD with `x-ratelimit-*` headers | ⚠ console logs page; export not documented on pages read |
| LiteLLM (self-hosted) | ✓ fallbacks after `num_retries`; 5 s cooldown default | ✓ `max_budget` + `budget_duration` per key, team, user; needs a DB | ✓ `/v1/messages` all providers; `/gemini` pass-through | ✓ nothing leaves your infrastructure; redaction switches | ✓ per key and team `tpm_limit`, `rpm_limit`, 429 | ✓ callbacks: Langfuse, OTel, S3, Datadog, more |
| Portkey | ✓ any non-2xx by default; `on_status_codes` | ⚠ per key, cost or tokens, 412 — Enterprise and select Pro | ✓ Chat Completions, Responses, Messages · Gemini native not stated | ⚠ retention by plan (3 d / 30 d / unlimited); Metrics Only is Enterprise | ⚠ per key or provider — Enterprise and select Pro | ✓ logs with retry and fallback status; OTel export |

Six sections follow, one per column, with the sentences behind each cell. Abbreviations, as the vendors use them: **ZDR** zero data retention; **TPM/RPM/RPD/TPD** tokens or requests per minute or day; **mantle** Bedrock's OpenAI-compatible endpoint, distinct from `bedrock-runtime`.

## Failover: who moves traffic across vendors, and who only moves it across regions

The gateways and OpenRouter document failover the way I dissected in
[Automatic failover in LiteLLM, Portkey and OpenRouter](/blog/failover-signals): a trigger
list, a retry policy, a cooldown, unchanged on the pages I re-read since 2026-08-25.
LiteLLM: "Fallbacks are how LiteLLM does automatic failover. If a call fails after
`num_retries`, LiteLLM falls back to another model group"
([Proxy — Fallbacks, Load Balancing, Retries](https://docs.litellm.ai/docs/proxy/reliability)).
Portkey: "By default, fallback triggers on any non-2xx status code"
([Fallbacks](https://portkey.ai/docs/product/ai-gateway/fallbacks)). OpenRouter ranks providers that
"have not seen significant outages in the last 30 seconds" first and publishes no retry
count, cooldown or timeout
([Provider Routing](https://openrouter.ai/docs/guides/routing/provider-selection)); its
[Limits](https://openrouter.ai/docs/api_reference/limits) page adds that on an upstream 429,
"fallback routing retries other providers for the same model automatically before the error
reaches you."

Cloudflare's cell is ⚠ for a documentation reason: the
[Fallbacks](https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/) page —
"By default, Cloudflare triggers your fallback if a model request returns an error" — is
written against the Universal Endpoint, whose own page now opens with "The Universal
Endpoint is deprecated. Use the OpenAI-compatible endpoint … for new integrations, and
Dynamic Routing for fallbacks, retries, and conditional routing"
([Universal Endpoint (Deprecated)](https://developers.cloudflare.com/ai-gateway/usage/universal/)).
[Dynamic routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) is
a visual or JSON flow of Model, Conditional, Rate Limit and Budget Limit nodes, each Model
node naming a provider — so cross-vendor failover exists, but it is marked Beta and "is
invoked through this `/compat/chat/completions` endpoint. The REST API does not currently
cover dynamic routing"
([Unified API (OpenAI compat)](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/)).
Per-request retries are precise: "a maximum of five retry attempts," delay up to 5 seconds,
constant, linear or exponential
([Request handling](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/)).

Bedrock has two features that read like failover and are not. Intelligent prompt routing
routes "between different foundational models within the same model family"; you "must
choose exactly two models within the same family," and its "fallback model" is a quality
anchor, not an error handler
([Understanding intelligent prompt routing in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-routing.html)).
Cross-Region inference "automatically selects a commercial AWS Region within that geography
to process your inference request"; the page covers residency, a roughly 10% saving on the
Global profile, and CloudTrail's `inferenceRegion` field, and does not state which errors,
if any, move a request between Regions
([Route model inference requests across AWS Regions](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html)).
Retries are yours: "Use retry logic with exponential backoff to handle transient throttling"
([Quotas for the bedrock-mantle endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-mantle.html)).

![AWS documentation page "Understanding intelligent prompt routing in Amazon Bedrock", showing the introduction about routing between models within the same model family and the topic list including "Prompt router criteria and fallback model"](/blog/images/openrouter-alternatives-production-aws-prompt-routing.png "Bedrock's routing page: same model family, two models, a quality threshold. The word fallback here means a baseline model, not a failed-request handler. Source: [Understanding intelligent prompt routing in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-routing.html).")

Vertex and Groq have no failover row to fill. Google's advice on 429 is "Use the global
endpoint instead of a regional endpoint whenever possible" and "truncated exponential
backoff" ([Error code 429](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/error-code-429?hl=en)).
Groq's error page is candid on one point: "You will not be charged for requests that return
server error codes" ([API Error Codes and Responses](https://console.groq.com/docs/errors)).
A refund is not a second provider.

## Spend caps: per key, per metadata dimension, per organization, or a tag on the bill

I mapped OpenRouter, LiteLLM and Portkey cell by cell in
[Per-key spending limits](/blog/per-key-spending-limits): OpenRouter caps a key with
`limit` + `limit_reset` and answers 402; LiteLLM caps a virtual key with `max_budget`, and
needs a database; Portkey caps an API key in USD or tokens and answers
[412 Precondition Failed](https://docs.portkey.ai/docs/guides/use-cases/enforcing-limits-and-budgets),
with the feature gated — "Available on Enterprise plan and select Pro customers"
([Enforce Budget Limits and Rate Limits](https://docs.portkey.ai/docs/product/administration/enforce-budget-and-rate-limit)).
Where the shared wallet sits relative to the caps is
[its own article](/blog/shared-team-wallet-llm-api).

Cloudflare's version is Beta and scoped differently from all three: spend-limit rules on the
gateway, "to any combination of model, provider, or custom metadata dimensions like user ID,
team, or application" — a per-user budget is a metadata key set to "Split by value," not a
property of a credential. Its caveats are stated next to the feature: "Spend limits are
eventually consistent. The current request's cost is recorded after completion, so a burst
of concurrent requests can briefly exceed the limit"; "A maximum of 20 spend limit rules can
be configured per gateway"; cost tracking is "a best-effort estimation"
([Spend limits](https://developers.cloudflare.com/ai-gateway/features/spend-limits/)). A
hard number per key this is not; a soft number per dimension it is.

Groq has one cap and it is the organization's: "The limit applies organization-wide across
all API keys … API calls from any key in your organization will return a 400 with code
`blocked_api_access`." Monthly only, "Limits reset on the 1st of each month," tracking
"updates every 10-15 minutes," and "you might exceed your limit by a small amount during
high usage periods." Per key? "A: No"
([Spend Limits](https://console.groq.com/docs/spend-limits)).

Bedrock's cost pages are about attribution, not caps: the native methods "deliver aggregated
billed dollars to AWS Cost Explorer and CUR 2.0. The finest grain is per usage type per day
… they do not produce a per-request row"
([Track usage and costs in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/cost-management.html));
Projects on bedrock-mantle add tags for Cost Explorer
([Projects (OpenAI-compatible)](https://docs.aws.amazon.com/bedrock/latest/userguide/projects.html)).
Nothing on these pages blocks a request at a number. Google's generative-AI pages I read say
nothing about a spend cap at all; both clouds' account-level budget tools live outside these
docs.

| Product | Where the cap lives | On breach | Reset |
|---|---|---|---|
| OpenRouter | key (`limit`); workspace budgets are Enterprise | 402 / 403 | daily, weekly, monthly, none |
| LiteLLM | key, team, user; per-model is Enterprise | 400, 401 or 429 by path | any `budget_duration` |
| Portkey | API key, workspace, provider — Enterprise, select Pro | 412 | none, weekly, monthly, custom days |
| Cloudflare AI Gateway | gateway rule by model, provider, metadata | 429 | rolling or fixed window |
| Groq | organization, all keys | 400 `blocked_api_access` | 1st of month |
| AWS Bedrock | not stated as a cap; tags to Cost Explorer | — | — |
| Google Agent Platform | not stated in the docs read | — | — |

## Protocols: OpenAI-compatible is table stakes; Messages and Gemini-native split the column

All seven speak Chat Completions. The other three formats are where migrations succeed or
stall.

Bedrock now serves three protocols on two endpoints: the endpoints page lists
bedrock-runtime as "InvokeModel / Converse / Chat Completions / Responses API / Messages
API" and bedrock-mantle as "Responses API / Chat Completions API / Messages API," both
reachable "by changing only the base URL and API key"
([Endpoints supported by Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/endpoints.html)).
The surfaces differ at the edges: on mantle, Messages requests with `output_config.format`
"are rejected with a 400 error"; on bedrock-runtime, Responses requests with
`background=true` are rejected the same way. No Gemini path — no Google model in the
catalog.

Cloudflare exposes each provider's native path under a prefix — `/anthropic/v1/messages`
([Anthropic](https://developers.cloudflare.com/ai-gateway/usage/providers/anthropic/)) and
`/google-ai-studio/v1/models/{model}:generateContent`
([Google AI Studio](https://developers.cloudflare.com/ai-gateway/usage/providers/google-ai-studio/))
— plus the OpenAI-compatible REST API, whose supported-provider list runs to 14 names. LiteLLM's
`/v1/messages` works with "All LiteLLM supported providers"
([/v1/messages](https://docs.litellm.ai/docs/anthropic_unified)), and its Gemini pass-through
is one substitution: "Just replace `https://generativelanguage.googleapis.com` with
`LITELLM_PROXY_BASE_URL/gemini`"
([Google AI Studio SDK](https://docs.litellm.ai/docs/pass_through/google_ai_studio)). Portkey
lists three formats and says "Each works with all providers — Portkey handles translation
automatically" ([Universal API](https://docs.portkey.ai/docs/product/ai-gateway/universal-api));
its Messages page names the production case — tools "like Claude Code and the Claude Agent
SDK — work with any backend model through Portkey without modification"
([Messages](https://docs.portkey.ai/docs/product/ai-gateway/messages-api)). OpenRouter carries
`POST /messages`, "the Anthropic Messages API format. Supports text, images, PDFs, tools, and
extended thinking"
([Create a message](https://openrouter.ai/docs/api/api-reference/anthropic-messages/create-a-message));
I found no Gemini-native endpoint in its index.

Google's compatibility layer is narrower than its name. "The Chat Completions API works as
an Open AI-compatible endpoint," and it "supports both Gemini models and select
self-deployed models from Model Garden" — Gemini plus containers running Gemma, Llama,
Mistral ([Using OpenAI libraries with Gemini Enterprise Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate/openai/overview?hl=en)).
Claude on Vertex is a separate door — "send a request directly to the Agent Platform API
endpoint" ([Anthropic's Claude on Google Cloud models](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude?hl=en)),
identified elsewhere as the partner models "that use `rawPredict` or `streamrawPredict`."
Porting an OpenAI client, mind two parameters: `tools.parameters` follows "the OpenAPI
specification. This differs from the OpenAI parameters field," and `tool_choice: validated`
is "Google-specific."

![Google Cloud documentation page "Using OpenAI libraries with Gemini Enterprise Agent Platform", noting the Chat Completions API is an OpenAI-compatible endpoint for Gemini and recommending the Google Gen AI SDK otherwise](/blog/images/openrouter-alternatives-production-google-openai-compat.png "Google's OpenAI-compatible layer as documented: Gemini plus select self-deployed Model Garden models; Claude is reached through a different endpoint. Source: [Using OpenAI libraries with Gemini Enterprise Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate/openai/overview?hl=en).")

Groq is "mostly compatible with OpenAI's client libraries" at
`https://api.groq.com/openai/v1`, also serves the Responses API, and names its gaps:
`logprobs`, `logit_bias`, `top_logprobs` and `messages[].name` "will result in a 400 error,"
`n` must be 1, temperature 0 becomes `1e-8`
([OpenAI Compatibility](https://console.groq.com/docs/openai)). If your codebase sends
`logit_bias`, that is a one-line grep before the migration, not after.

## Data retention: "ZDR" means at least four different mechanisms

Every vendor here uses the phrase zero data retention or a close cousin, for different
objects.

OpenRouter's ZDR is **an endpoint attribute you route on**: privacy settings "only allow you
to route to endpoints that have a Zero Data Retention policy. You can enforce ZDR globally,
per model group, per guardrail, or per request" — per request is `provider: { zdr: true }`.
When a provider's policy is unclear, OpenRouter assumes "that the endpoint both retains and
trains on data and mark[s] it as such"; in-memory prompt caching is "not considered
'retaining' data"; the list is machine-readable at `/api/v1/endpoints/zdr`
([Zero Data Retention](https://openrouter.ai/docs/guides/features/zdr)). Its own stance:
"your prompts are not retained unless you specifically opt in to prompt logging"
([Data Collection](https://openrouter.ai/docs/guides/privacy/data-collection)).

Cloudflare's ZDR is **a billing-path setting**: it "routes Unified Billing traffic through
provider endpoints that do not retain prompts or responses … only applies to Unified Billing
requests that use Cloudflare-managed credentials. It does not apply to BYOK," is "currently
supported for: OpenAI, Anthropic," and — underline this — "ZDR does not control AI Gateway
logging" ([Unified Billing](https://developers.cloudflare.com/ai-gateway/features/unified-billing/)).
The gateway's own logs, "which include metrics as well as request and response data, are
enabled by default"; turning them off is a settings switch or per-request
`cf-aig-collect-log: false`
([Logging](https://developers.cloudflare.com/ai-gateway/observability/logging/)). Nothing
stored anywhere means two switches on two pages.

![Cloudflare docs overview page for AI Gateway, headed "Observe and control your AI applications", listing analytics, logging, caching, rate limiting, request retries and model fallback, with "Available on all plans" shown](/blog/images/openrouter-alternatives-production-cloudflare-overview.png "Cloudflare's AI Gateway overview. Request and response logging is on by default; the ZDR toggle documented under Unified Billing covers provider-side retention for two vendors and, per the docs, does not control the gateway's own logs. Source: [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/).")

Bedrock's ZDR is **a retention mode that gates which models you can call**. Default: "by
default, Amazon Bedrock does not store model inputs or outputs," with named exceptions —
"For Anthropic Claude Fable 5, inputs and outputs will be retained for up to 30 days," and
using it requires opting in to sharing retained traffic with Anthropic
([Amazon Bedrock abuse detection](https://docs.aws.amazon.com/bedrock/latest/userguide/abuse-detection.html)).
The control is `data_retention_mode` per account or project; under `none`, "No request or
response data is written to durable storage by AWS or shared with the model provider," and a
model whose `allowed_modes` excludes your mode "will appear as `status: "unavailable"` …
and requests to it will be blocked"
([Data retention](https://docs.aws.amazon.com/bedrock/latest/userguide/data-retention.html)).
Strictest control in the table; the price is losing models that require retention. Also:
"At launch, there is no console UI for configuring data retention."

Groq's ZDR is **a toggle over a no-retention default**: "By default, Groq does not retain
customer data for inference requests"; reliability and abuse logs are the exception, "up to
30 days"; "All customers may enable Zero Data Retention (ZDR) in Data Controls settings"
— globally or per feature — which disables batch and fine-tuning; retained data sits "in
Google Cloud Platform (GCP) buckets located in the United States"
([Your Data in GroqCloud](https://console.groq.com/docs/your-data)).

Google's is **a checklist, not a switch**. Training is excluded by contract, but retention
has a list: abuse-monitoring prompt logging ("you can request an exception"); "Zero data
retention may not be possible when using some Advanced AI features"; Grounding with Google
Search keeps query logs "for up to three (3) days" with "no way to disable the storage"; the
Interactions API defaults `store` to true; Gemini's in-memory cache has "a 24-hour TTL" and
"can be disabled at the project level"
([Gemini Enterprise Agent Platform and zero data retention](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/data-governance?hl=en)).

LiteLLM's answer is architectural — "No data or telemetry is stored on LiteLLM Servers when
you self-host" ([Data Privacy and Security](https://docs.litellm.ai/docs/data_security)) —
plus redaction switches (`turn_off_message_logging`, per-request headers) on the
[logging page](https://docs.litellm.ai/docs/proxy/logging). Portkey's retention is a plan
attribute — Developer "3 day Log Retention," Production 30 days, Enterprise unlimited
([Logs](https://docs.portkey.ai/docs/product/observability/logs)) — with `x-portkey-debug:false`
per request and an org-wide "Metrics Only (Privacy Mode)" that is Enterprise and "will not
retroactively remove previously logged data"
([Configure Request Logging](https://docs.portkey.ai/docs/product/administration/configuring-request-logging)).
Behind a BYOK proxy, what the provider keeps is the provider's policy; neither vendor claims
otherwise.

## Rate limits: the unit of the quota is the whole story

Comparing numbers across these seven is meaningless; comparing what they count is not.

Bedrock counts **tokens per model per Region**, twice. On bedrock-runtime, quotas are
"tokens per minute (input + output, combined)," a daily cap defaulting to the per-minute
quota × 24 × 60, RPM "enforced for some models … and not others," and "shared across every
inference API you call it with"
([Quotas for the bedrock-runtime endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-runtime.html)).
On bedrock-mantle, input and output TPM are separate quotas, "does not enforce
requests-per-minute (RPM) quotas," admission counts request tokens "plus the value of
`max_tokens`," cached input tokens are free, and the one published default at reading time
was Claude Opus 4.7 at 20,000,000 in / 4,000,000 out
([Quotas for the bedrock-mantle endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-mantle.html)).
Two endpoints, two pools, "even when calling the same underlying model."

Google counts **an organization's 30-day spend**: Standard PayGo tiers set a TPM baseline —
Gemini Pro from 500,000 TPM at Tier 1 ($10–$250) to 2,000,000 at Tier 3 (over $2,000),
Flash up to 10,000,000 — with "no separate requests-per-minute (RPM) limit" and traffic
allowed to "burst beyond this limit on a best-effort basis"
([Standard PayGo](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/dynamic-shared-quota?hl=en)).
Guaranteed capacity is the Provisioned Throughput subscription, and the status code moves
with it: on standard PT, while under the throughput you bought,
"errors that might otherwise be 429 are returned as 5XX and count toward the SLA error rate"
— Single Zone PT is the documented exception, where they do not count
([Error code 429](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/error-code-429?hl=en)).
A vendor telling you which status code you pay for.

Groq counts **the organization**: "Rate limits apply at the organization level, not
individual users," in RPM, RPD, TPM, TPD and audio-seconds units, cached tokens excluded.
Its headers are precise about referents: `x-ratelimit-limit-requests` "Always refers to
Requests Per Day (RPD)," `x-ratelimit-limit-tokens` to TPM, and `retry-after` appears only
on a 429. Published Developer-plan base at reading time: `openai/gpt-oss-120b` at 30 RPM /
1K RPD / 8K TPM / 200K TPD ([Rate Limits](https://console.groq.com/docs/rate-limits)).

OpenRouter counts **globally** — "Making additional accounts or API keys will not affect
your rate limits, as we govern capacity globally" — and publishes numbers only for free
variants: 20 requests per minute, 50 or 1,000 per day by credit history
([Limits](https://openrouter.ai/docs/api_reference/limits)). Cloudflare counts **requests
per window per gateway**, fixed or sliding, "uniformly applied to all requests for that
gateway" ([Rate limiting](https://developers.cloudflare.com/ai-gateway/features/rate-limiting/));
per-user limits are a Dynamic Route node. LiteLLM counts **per credential** — `tpm_limit`
and `rpm_limit` on keys, teams, users, per-team per-model limits "inherited by keys unless
overridden" ([Budgets, Rate Limits](https://docs.litellm.ai/docs/proxy/users)). Portkey
counts per API key or provider, requests or tokens per minute, hour or day, behind the same
Enterprise gate as its budgets; once set, provider-level limits "cannot be edited by any
organization member," and "Setting the limit to 0 disables the provider"
([Rate Limits](https://docs.portkey.ai/docs/product/ai-gateway/virtual-keys/rate-limits)).

## Observability: where the per-request log row lives

Ask where one request's tokens, cost and outcome can be found afterwards, and whether it is
on by default.

OpenRouter puts it in the response: "Full usage details are now always included
automatically in every response," including `cost`, `cost_details.upstream_inference_cost`
and cached and reasoning token counts
([Usage Accounting](https://openrouter.ai/docs/cookbook/administration/usage-accounting));
Broadcast forwards traces to 19 documented destinations — Datadog, Langfuse, OTel, S3,
BigQuery among them — with per-destination key filters, sampling and a Privacy Mode that
strips prompt and completion content
([Broadcast](https://openrouter.ai/docs/guides/features/broadcast)).

Cloudflare stores it by default — "user prompt, model response, provider, timestamp, request
status, token usage, cost, duration" — capped at 100,000 logs per free *account* against
10 million per paid *gateway*, two orders of magnitude apart, exportable via Logpush on
Workers Paid
([Logging](https://developers.cloudflare.com/ai-gateway/observability/logging/) ·
[Limits](https://developers.cloudflare.com/ai-gateway/reference/limits/)). Its cost column is
"an estimation … refer to your provider's dashboard for the most accurate cost details"
([Costs](https://developers.cloudflare.com/ai-gateway/observability/costs/)).

Bedrock's row is off until enabled, and only on one door: "Model invocation logging is
disabled by default." It writes bodies up to 100 KB with token counts and `identity.arn` to
CloudWatch or S3, and "is only supported for calls made through the bedrock-runtime endpoint
… the same APIs on bedrock-mantle are not currently captured"
([Monitor model invocation using CloudWatch Logs and Amazon S3](https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html)).
Behind a gateway, "Amazon Bedrock records the gateway's IAM role as the caller's identity";
per-user rows need request metadata or role sessions
([Track usage and costs](https://docs.aws.amazon.com/bedrock/latest/userguide/cost-management.html)).

Google's row is a Preview that samples: request-response logging writes "samples of requests
and responses" to BigQuery with a `sampling_rate`; "For Anthropic models, only REST is
supported for logging configuration"
([Log and share requests and responses](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging?hl=en)).
LiteLLM's row goes wherever a callback points, and every response carries
`x-litellm-call-id` and `x-litellm-response-cost`
([Logging](https://docs.litellm.ai/docs/proxy/logging)). Portkey's log records tokens,
thinking tokens and cost plus a Status column (`Retry Success on {x} Tries`, "Fallback
Active") ([Logs](https://docs.portkey.ai/docs/product/observability/logs)); its OTel export
comes in two grades, Analytics Stable, Complete Logs Experimental
([OpenTelemetry (OTel) Export](https://docs.portkey.ai/docs/product/enterprise-offering/otel/otel)).
Groq's docs mention a console logs page; the pages I read document no export.

## Which column to read first, by workload

Pick the column your next incident comes from; read its seven cells first.

- **A fleet of agents on many keys.** The spend-cap column. OpenRouter and LiteLLM cap the key;
  Cloudflare caps a metadata dimension, eventually consistently; Groq caps the organization
  once a month; Portkey caps the key on Enterprise; the clouds tag the bill and do not
  block. Getting this wrong costs one intern's weekend, which is why it is first.
- **Regulated data.** The retention column, then the vendor's page, not this one. Bedrock's
  `data_retention_mode: none` is the only control here that refuses models that would
  retain; OpenRouter's ZDR is the only one filtering endpoints across vendors; Cloudflare's
  needs a second switch for its own logs.
- **Tooling on the Anthropic Messages format, Claude Code included.** The protocol column. Portkey,
  LiteLLM, Cloudflare, OpenRouter and Bedrock accept it; Groq and Vertex's compat layer
  do not.
- **Bursty batch work.** The rate-limit column, unit before number: per-model TPM pools (Bedrock),
  org-level spend tiers with best-effort burst (Google), org-level headers (Groq), global
  capacity with no paid figures published (OpenRouter).
- **Open-weight models, lowest latency, one vendor.** Groq, eyes open: three ✗ or ⚠ cells on
  failover, per-key caps and Messages, plus a monthly org-wide cap with a 10–15 minute lag.
  A gateway from the third row in front of it buys back the first columns.

Three lines to grep in whatever you run: the retry and cooldown values, the
credential-level budget field or its absence, and the log-collection default. Two questions
for any vendor whose cell says "not stated": which error classes move a request off a
provider, and whether a burst can overshoot a cap before the meter catches up — three of the
seven admit the overshoot in their own words. We are building a router ourselves, so this
matrix doubles as the checklist we hold our own docs to
([current state](https://pirouter.ai/)). Everything above is what the vendors wrote on
2026-08-29; the links are there so you can check whether it still is.

---

## Sources

### docs.litellm.ai

- [Proxy — Fallbacks, Load Balancing, Retries](https://docs.litellm.ai/docs/proxy/reliability)
- [/v1/messages](https://docs.litellm.ai/docs/anthropic_unified)
- [Google AI Studio SDK](https://docs.litellm.ai/docs/pass_through/google_ai_studio)
- [Data Privacy and Security](https://docs.litellm.ai/docs/data_security)
- [logging page](https://docs.litellm.ai/docs/proxy/logging)
- [Budgets, Rate Limits](https://docs.litellm.ai/docs/proxy/users)

### portkey.ai

- [Fallbacks](https://portkey.ai/docs/product/ai-gateway/fallbacks)

### openrouter.ai

- [Provider Routing](https://openrouter.ai/docs/guides/routing/provider-selection)
- [Limits](https://openrouter.ai/docs/api_reference/limits)
- [Create a message](https://openrouter.ai/docs/api/api-reference/anthropic-messages/create-a-message)
- [Zero Data Retention](https://openrouter.ai/docs/guides/features/zdr)
- [Data Collection](https://openrouter.ai/docs/guides/privacy/data-collection)
- [Usage Accounting](https://openrouter.ai/docs/cookbook/administration/usage-accounting)
- [Broadcast](https://openrouter.ai/docs/guides/features/broadcast)

### developers.cloudflare.com

- [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/)

### docs.aws.amazon.com

- [Understanding intelligent prompt routing in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-routing.html)
- [Route model inference requests across AWS Regions](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html)
- [Quotas for the bedrock-mantle endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-mantle.html)
- [Track usage and costs in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/cost-management.html)
- [Projects (OpenAI-compatible)](https://docs.aws.amazon.com/bedrock/latest/userguide/projects.html)
- [Endpoints supported by Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/endpoints.html)
- [Amazon Bedrock abuse detection](https://docs.aws.amazon.com/bedrock/latest/userguide/abuse-detection.html)
- [Data retention](https://docs.aws.amazon.com/bedrock/latest/userguide/data-retention.html)
- [Quotas for the bedrock-runtime endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-runtime.html)
- [Monitor model invocation using CloudWatch Logs and Amazon S3](https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html)

### docs.cloud.google.com

- [Error code 429](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/error-code-429?hl=en)
- [Using OpenAI libraries with Gemini Enterprise Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate/openai/overview?hl=en)
- [Anthropic's Claude on Google Cloud models](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude?hl=en)
- [Gemini Enterprise Agent Platform and zero data retention](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/data-governance?hl=en)
- [Standard PayGo](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/dynamic-shared-quota?hl=en)
- [Log and share requests and responses](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/request-response-logging?hl=en)

### console.groq.com

- [API Error Codes and Responses](https://console.groq.com/docs/errors)
- [Spend Limits](https://console.groq.com/docs/spend-limits)
- [OpenAI Compatibility](https://console.groq.com/docs/openai)
- [Your Data in GroqCloud](https://console.groq.com/docs/your-data)
- [Rate Limits](https://console.groq.com/docs/rate-limits)

### docs.portkey.ai

- [412 Precondition Failed](https://docs.portkey.ai/docs/guides/use-cases/enforcing-limits-and-budgets)
- [Enforce Budget Limits and Rate Limits](https://docs.portkey.ai/docs/product/administration/enforce-budget-and-rate-limit)
- [Universal API](https://docs.portkey.ai/docs/product/ai-gateway/universal-api)
- [Messages](https://docs.portkey.ai/docs/product/ai-gateway/messages-api)
- [Logs](https://docs.portkey.ai/docs/product/observability/logs)
- [Configure Request Logging](https://docs.portkey.ai/docs/product/administration/configuring-request-logging)
- [Rate Limits](https://docs.portkey.ai/docs/product/ai-gateway/virtual-keys/rate-limits)
- [OpenTelemetry (OTel) Export](https://docs.portkey.ai/docs/product/enterprise-offering/otel/otel)