# Automatic failover in LiteLLM, Portkey and OpenRouter: what actually triggers it, and the 429 no gateway tracks

> Seven LLM gateways' failover docs, read verbatim: the trigger lists agree, every cooldown is a timer, and no routing policy records a 429 lasting eight hours.

- Published: Aug 25, 2026
- Author: Leo Kaka, Engineering
- Tags: reliability, failover, gateway
- Canonical: https://pirouter.ai/blog/failover-signals

---
Seven LLM gateways — LiteLLM, Portkey and OpenRouter from the title, plus Cloudflare AI
Gateway, Kong AI Proxy Advanced, Helicone and LLMGateway — support automatic failover across
providers, and every one of them documents it the same way: a list of conditions that count
as a failure, a retry policy, and a cooldown after which the failed provider is put back in
rotation. The lists agree more than the marketing suggests — 5xx, timeouts and connection
errors everywhere; 429 in most; 404 and 400 rarely, and in LLMGateway not at all, because
its policy is "server-side failures only." What no list has is a row for the failure that
returns a well-formed response: a rate-limit error that is actually a paused account, a
not-found that is actually a retired model, a 200 from a model you did not pin. This post
reads the seven products' fallback and retry documentation verbatim, puts the trigger
conditions side by side, and then works through what falls between the rows. The seven are
simply the ones whose documentation I read end to end.

It is the third post in a series on where LLM failure domains actually live. The
[first](/blog/billing-failure-domain) was about failures that arrive through the billing
layer; the [second](/blog/retry-storms) was about how retries make failures bigger. This one
is about what the layer in the middle — the gateway — is and is not configured to notice.

## "Automatic failover" is a list, so read the list

Each vendor defines the feature in one sentence, and the sentence is always about errors.
LiteLLM: ["Fallbacks are how LiteLLM does automatic failover. If a call fails after
`num_retries`, LiteLLM falls back to another model
group."](https://docs.litellm.ai/docs/proxy/reliability) Portkey: ["By default, fallback
triggers on any non-2xx status
code."](https://portkey.ai/docs/product/ai-gateway/fallbacks) OpenRouter, for its `models`
array: ["any error can trigger the use of a fallback model, including: Context length
validation errors, Moderation flags for filtered models, Rate-limiting,
Downtime."](https://openrouter.ai/docs/guides/routing/model-fallbacks) Cloudflare AI
Gateway: ["Cloudflare can trigger your fallback provider in response to request errors or
predetermined request
timeouts."](https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/) Kong's
AI Proxy Advanced is the most precise, because it inherits nginx's vocabulary: failover
happens on whatever is in `config.balancer.failover_criteria`, and ["Client errors don't
trigger failover."](https://developer.konghq.com/plugins/ai-proxy-advanced/#retry-and-fallback)

![Portkey documentation diagram, two panels: on the left, "Without Fallbacks", an application calls one AI provider and gets a 4xx back; on the right, "With Fallbacks", the application calls an AI Gateway which sends the request to AI Provider 1 (first target), and on failure routes to AI Provider 2 (fallback, second target), returning a 2xx to the application](/blog/images/failover-signals-portkey-fallback.png "Portkey's own picture of the feature: the trigger is a status code on the response, and the gateway's move is to try the next target. Source: [Portkey Docs — Fallbacks](https://portkey.ai/docs/product/ai-gateway/fallbacks).")

Read as a group, the feature has four moving parts in every product: a **trigger set** (which
responses count as failure), a **retry policy** (how many times, how spaced, whether the
provider's `Retry-After` is honored), a **cooldown or breaker** (how long a failed target is
kept out of rotation — "breaker" as in circuit breaker, which *opens* to stop traffic and
*closes* to let it through again), and a **re-admission rule** (what brings it back). The
differences between vendors are real, and they are all inside those four parts. Nothing
outside them exists in any of the seven documents, with one partial exception noted under
the matrix.

One split matters before the tables: where the policy runs. LiteLLM and Kong are software
you run, so the numbers below sit in a file you can grep; OpenRouter and Cloudflare AI
Gateway run the policy on their side; Helicone and LLMGateway are open source and come in
both forms. On a hosted router, every *not documented* cell below is a question for the
vendor.

## The trigger matrix: seven gateways, side by side

Both tables were built from the linked pages as they read on 2026-08-25. Start with the one
that matters — which response classes make each product move traffic. Legend: **✓** fails
over by default · **⚠** opt-in, or with a documented side effect · **—** the documentation
does not say · **✗** not a trigger, stated or *inferred* from the vendor's own scope
statement.

| Response class | LiteLLM | Portkey | OpenRouter | Cloudflare | Kong | Helicone | LLMGateway |
|---|---|---|---|---|---|---|---|
| 5xx | ✓ | ✓ default | ✓ ("downtime") | ✓ ("an error") | ⚠ `http_5xx` opt-in; default set is `error`/`timeout` | ✓ | ✓ |
| Timeout / connection error | ✓ | ✓ (`request_timeout`) | ✓ | ✓ (first-byte timeout) | ✓ default | ✓ (408) | ✓ |
| 429 | ✓ + immediate cooldown | ✓ default | ✓ ("rate-limiting") | — "an error", codes unspecified | ⚠ `http_429` opt-in | ✓ | ✗ *inferred*: "server-side failures only"; 429 not named |
| 404 (model not found) | ✓ fails over, and cools the deployment for 5 s | ✓ non-2xx | — | — | ⚠ `http_404` opt-in for failover; never counts toward `max_fails` | — not listed | ✗ *inferred*, as above |
| 400 context length | ✓ own fallback list | ✓ non-2xx | ✓ | — | ✗ not in the enum | ✓ | ✗ on the not-triggered list |
| Content filter | ✓ own fallback list | ✓ non-2xx | ✓ ("moderation flags") | — | ✗ not in the enum | — not listed | ✗ named explicitly |
| 200, but the provider's account or model state changed | ✗ | ⚠ output guardrails only: a failed check turns the 200 into a 446, which fallback can act on (non-streaming) | ✗ | ✗ | ✗ | ✗ | ✗ |

One footnote on the last row: ✗ means no product documents a trigger on a 200. The partial
exception is Portkey's [guardrails](https://portkey.ai/docs/product/guardrails), which check
the response *content* and fail over on a 446 — a content check, not state tracking, and "no
action is taken for output guardrails on streaming."

Three things fall out of this table. First, 5xx and timeouts are a universal language; if
that is all you need, every product on the list will do and the decision is about everything
else. Second, 429 is where the products split: LiteLLM cools a deployment down the instant
it sees one, LLMGateway's list is server-side failures only and leaves 429 out, and Kong
makes you ask for it by name. Both positions are defensible — switching immediately gets
this user's request answered now, while a 429 is also the provider telling you to slow
down, and rerouting a burst to the next provider is how you export your congestion to
someone else's rate limit. Third, and this is the row I care about: 4xx is treated as the
caller's problem almost everywhere, which is correct for the malformed request that produced
most 4xx you have ever seen and wrong for the two 4xx classes this series has been
documenting — the paused-account 429 and the retired-model 404.

For anyone who wants to check the cells, the parameters behind them. *Not documented* means
I could not find the parameter on the vendor's fallback, retry, breaker or timeout pages —
not that the product lacks the behavior.

| Gateway | Default trigger set | Configurable trigger codes | Retries (default → max) | Honors `Retry-After` | Cooldown / breaker | Re-admission |
|---|---|---|---|---|---|---|
| [LiteLLM](https://docs.litellm.ai/docs/proxy/reliability) | `fallbacks` after `num_retries`, "covers all errors (429, 500, etc.)"; separate `context_window_fallbacks` and `content_policy_fallbacks` | per exception class via [`RetryPolicy`](https://docs.litellm.ai/docs/routing#advanced-custom-retries-cooldowns-based-on-error-type) | `num_retries` (example: 3); exponential backoff on 429, immediate on other errors | not documented on the routing page; the [router source](https://github.com/BerriAI/litellm/blob/main/litellm/utils.py) reads `retry-after` and honors it up to 60 s (`retry_after` in config is a *minimum* wait) | [`allowed_fails: 3`, `cooldown_time: 5s`](https://docs.litellm.ai/docs/routing#cooldowns); immediate on 429, on >50 % failures in the current minute, or on 401/404/408 | timer: "Automatically re-enable deployments when cooldown expires"; opt-in health-check routing swaps the timer for a probe |
| [Portkey](https://portkey.ai/docs/product/ai-gateway/fallbacks) | any non-2xx | `on_status_codes` on the strategy | [up to 5](https://portkey.ai/docs/product/ai-gateway/automatic-retries); default codes `[429, 500, 502, 503, 504, 529]`; backoff 1/2/4/8/16 s | ✓ with `use_retry_after_headers: true`; cumulative wait capped at 60 s | [circuit breaker](https://portkey.ai/docs/product/ai-gateway/circuit-breaker): `failure_threshold`, `failure_status_codes` default >500, `cooldown_interval` min 30 s | timer: "Circuit closes (CLOSED) automatically after `cooldown_interval` passes" |
| [OpenRouter](https://openrouter.ai/docs/guides/routing/provider-selection) | provider "unavailable"; `models` fallback on context length, moderation, rate limit, downtime | none — `order`, `allow_fallbacks` (default `true`), `ignore`, `only`, `sort` shape the candidate list, not the trigger | not documented | not documented | no cooldown documented; default ranking step 1: ["Prioritize providers that have not seen significant outages in the last 30 seconds"](https://openrouter.ai/docs/guides/routing/provider-selection#price-based-load-balancing-default-strategy) | a ranking window, not a removal; no re-admission semantics documented |
| [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/) | "if a model request returns an error", or the request timeout | not documented | [`cf-aig-max-attempts` ≤ 5, `cf-aig-retry-delay` ≤ 5000 ms, `cf-aig-backoff` constant/linear/exponential](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/) | not documented | none documented; Dynamic Routing's [Rate Limit and Budget Limit nodes](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/) fail over on *your* quota, not the provider's | n/a |
| [Kong AI Proxy Advanced](https://developer.konghq.com/plugins/ai-proxy-advanced/reference/#schema--config-balancer-failover-criteria) (breaker: Kong Gateway 3.13+) | `failover_criteria` default `["error", "timeout"]` | enum `error, http_403, http_404, http_429, http_500, http_502, http_503, http_504, invalid_header, non_idempotent, timeout` | `retries` default 5 | not documented | `max_fails` default 0 ("The zero value disables the circuit breaker"); `fail_timeout` default 10 000 ms; `http_403`/`http_404` "never considered unsuccessful attempts" for the count | timer (`fail_timeout`), only once `max_fails` ≥ 1 |
| [Helicone](https://docs.helicone.ai/gateway/provider-routing#failover-triggers) | 429, 401, 400 (context length), 408, 500+ | not documented | opt-in via `Helicone-Retry-Enabled: true`; [`Helicone-Retry-Num` default 5, factor 2, 1–10 s backoff](https://docs.helicone.ai/features/advanced-usage/retries) | not documented | not documented | n/a |
| [LLMGateway](https://docs.llmgateway.io/features/routing#what-triggers-a-retry) | "server-side failures only": 5xx, timeouts, connection failures; *not* 400/401/403/422 or content filters | not documented | "Up to 2 retries" | not documented | no fixed cooldown: an uptime score over a [60-minute rolling window, time-decayed](https://docs.llmgateway.io/features/routing#low-uptime-protection); reroute below 90 %, penalty below 95 % | score recovery |

## Every cooldown expiry is a retry on behalf of the whole fleet

Look at the re-admission column again. Every product with a cooldown brings a failed
provider back on a timer: LiteLLM after `cooldown_time`, [5 seconds by
default](https://docs.litellm.ai/docs/routing#cooldowns); Kong after `fail_timeout`, [10
seconds by
default](https://developer.konghq.com/plugins/ai-proxy-advanced/reference/#schema--config-balancer-fail-timeout)
— once `max_fails` is at least 1, since the default 0 disables the breaker and the failed
target is never taken out at all; Portkey after `cooldown_interval`, [30 seconds at
minimum](https://portkey.ai/docs/product/ai-gateway/circuit-breaker). OpenRouter's 30-second
window is a ranking rule, with no removal or re-admission semantics documented. None of the
three probes before re-admitting by default — LiteLLM's opt-in health-check routing is the
exception, and it comes up below. The clock runs out and the next real user request is the
probe.

![Bar chart: re-admission attempts over an eight-hour provider pause, from documented default cooldowns — LiteLLM 5 s: 5,760; Kong 10 s with max_fails ≥ 1: 2,880 (the default max_fails of 0 disables the breaker); Portkey 30 s minimum: at most 960; OpenRouter: 30 s ranking window, no count; LLMGateway: no fixed cooldown, 60-minute decayed window](/blog/images/failover-signals-cooldown-clocks.png "Fig. 1 — how many times each documented default cooldown would put a paused provider back in rotation over eight hours. Arithmetic on published defaults, not a measurement; per router process where cooldown state is in memory (LiteLLM shares it across replicas only through Redis). Kong assumes max_fails ≥ 1; OpenRouter's 30 s is a ranking window, so no count is shown. Source: [LiteLLM](https://docs.litellm.ai/docs/routing#cooldowns), [Kong](https://developer.konghq.com/plugins/ai-proxy-advanced/reference/), [Portkey](https://portkey.ai/docs/product/ai-gateway/circuit-breaker), [OpenRouter](https://openrouter.ai/docs/guides/routing/provider-selection).")

For a thirty-second blip that is exactly right. For the eight-hour pause one user reported
in the [Google tier-downgrade
thread](https://discuss.ai.google.dev/t/urgent-billing-account-downgraded-from-tier-3-to-tier-1-unexpectedly-service-paused/142555)
the arithmetic is different: 28,800 seconds divided by a five-second cooldown is 5,760
re-admissions, and each one costs at least one real request a failed hop before the
deployment — LiteLLM's word for one provider-plus-model entry — is cooled down again. That
is a retry storm moved up one level. The per-request retry has a budget, or at least a
count; the fleet-level retry — "put it back and see" — has neither. Its interval is a
constant, it does not back off, and it does not read `Retry-After`, because by the time the
timer fires the response that carried the header is gone.

LiteLLM deserves credit for doing this multiplication once. It [pins the provider SDK to
`max_retries: 0`](https://docs.litellm.ai/docs/routing#where-num_retries-can-be-set-and-which-one-wins)
and says why: "it is what stops a deployment `num_retries: N` from being applied twice and
turning one request into `(1 + N) ** 2` upstream calls." That is the right instinct,
enforced at one boundary. Nobody enforces it at the cooldown boundary, in any product,
because cooldown expiry is not modelled as a retry. It should be.

## The row none of them has: a 429 that is not congestion

Every product in the matrix treats 429 as one thing. Providers do not. Anthropic's rate-limit
documentation describes what happens when an organization hits its monthly spend cap: ["API
usage pauses until 00:00 UTC on the first day of the next month … API requests return HTTP
429"](https://platform.claude.com/docs/en/api/rate-limits#reaching-your-spend-cap), and then
the sentence that matters for every gateway above: "The error type is `rate_limit_error`, the
same as for a rate limit, but the response has no `retry-after` header. Retrying, including
the SDKs' automatic retries, fails until access resumes." The only distinguishing mark is a
nested field, `error.details.error_code: "enforced_spend_limit_reached"`. OpenAI's
equivalent is the `insufficient_quota` 429 whose message — ["You exceeded your current quota,
please check your plan and billing
details"](https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb)
— is reproduced in their own cookbook. Same status code as a busy afternoon. A rate limit
lasts seconds; a spend cap lasts until the first of next month.

Run the three incident classes from the [first post](/blog/billing-failure-domain) through
the matrix and the pattern holds:

| What happened at the provider | What the gateway sees | Per-request failover rescues the call? | Does the routing policy record that the state changed? |
|---|---|---|---|
| Billing tier downgraded, service paused for hours; or monthly spend cap reached | 429, well-formed, sometimes without `Retry-After` | ✓ in six of seven — the request lands on the fallback (LLMGateway leaves 429 out; Kong only with `http_429` set) | ✗ the deployment cools down for 5–30 s and is re-admitted on the timer, all day |
| Model retired from the catalog | 404 / 400 with a model-lifecycle error code | ⚠ depends on the trigger set: Portkey and LiteLLM fail over; Kong only with `http_404` set; Helicone does not list it; LLMGateway excludes it | ✗ LiteLLM cools the deployment for 5 s, then sends the next user to the retired model again |
| Model silently substituted or degraded after a migration | 200 | ✗ nothing to fail over from; Portkey's output guardrails check content, not identity | ✗ latency-sorted routers may drift traffic away; none records why |

The first row is the one worth staring at. Per-request failover *works* — six of the seven
products will get the individual request to a second provider. What none of them keeps,
inside the routing policy, is the fact. A 429 that has persisted for eight hours while your
request rate sits well below your known limit is not a rate limit, it is a state change in
the relationship between you and the provider, and the correct responses — open the billing
console, switch to prepaid credit where the provider offers it, move the traffic
deliberately, stop paying for the probe — are not things a five-second timer can do. Nobody
looks because the gateway itself is healthy: the fallback is serving, error rate is flat, and
the only symptom is a bill from the wrong provider at the end of the month. Your monitoring
can be taught to page on this — the last section says how — but this post is about why the
routing policy does not do it for you.

The 404 row has a sharper edge. Kong's schema is explicit that ["the cases of `http_403` and
`http_404` are never considered unsuccessful
attempts"](https://developer.konghq.com/plugins/ai-proxy-advanced/reference/#schema--config-balancer-max-fails)
for the purpose of the breaker count, and its default trigger set leaves `http_404` out;
LLMGateway's ["Retries are triggered by server-side failures
only"](https://docs.llmgateway.io/features/routing#what-triggers-a-retry) is a policy
statement. They are right for the general case — a 404 is your typo far more often than
their retirement — and the cost of being right in general is that a retired model produces
one failed real request per cooldown cycle until a human edits a config file.

### Why the documentation is shaped this way

The gateway sees HTTP. The account is on the other side of the wire. The two products in the
matrix with quota-aware routing — Cloudflare's Dynamic Routing, with [rate and budget limits
"per your key, per period"](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/),
and LiteLLM's [Budget Routing](https://docs.litellm.ai/docs/proxy/provider_budget_routing),
with per-provider, per-model and per-tag budgets — both enforce limits against *your*
ledger, which the gateway can read, not the provider's, which it cannot. LLMGateway's
[uptime score](https://docs.llmgateway.io/features/routing#low-uptime-protection) is the
closest thing in the table to state with a duration in it — a 60-minute rolling window,
weighted toward the last few minutes — and it is fed exclusively by 5xx, timeouts and
connection failures, so a paused account with a valid key scores as perfectly healthy.
LiteLLM's ">50% failures in the current minute" is a state, too, and it resets on the
minute. Nobody is doing anything wrong here. The abstraction is HTTP, and HTTP does not have
a status code for "your relationship with this vendor changed at 09:14."

![Cloudflare AI Gateway's visual Dynamic Routing editor: a START node feeds an If/Else split on user_plan == paid; the True branch goes to an o4-mini-high model node, the False branch goes to a RATE LIMIT node set to 100 requests per day on metadata.userId, whose red Fallback edge and whose normal edge both end at End, with a gpt-4.1 model node on the normal path](/blog/images/failover-signals-cf-dynamic-routing.png "Quota-aware failover in Cloudflare's route editor: the RATE LIMIT node counts against your own key (100 requests/day on metadata.userId), and its red Fallback edge fires when that budget is spent — not when the provider's state changes. Source: [Cloudflare AI Gateway Docs — Dynamic routing](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/).")

## What a gateway could watch instead

Three signals exist today that a gateway could consume. None is exotic; what is missing is
the wiring. Scope, stated once: this is the design we are building toward in our own
gateway, not a shipped feature — every claim below reads "a gateway can," and the links go
to provider documentation. It does not touch the 200 row; a model substituted behind a 200 is
a content-layer problem none of these three signals sees.

**A classification with a duration in it.** The first post proposed splitting faults into
`transient` and `structural`, where a 429 becomes structural once it has persisted past a
recovery window. The half the matrix is missing is what happens next: a structural fault
should not be re-admitted by a timer. It should be re-admitted by a probe — a single request
on a dedicated key, whose success is the only event that closes the state. Cost: you keep a
small stream of failing probes running against a paused provider, on a line item you can
see, instead of 5,760 failing user requests you cannot.

**Quota headers, where the provider sends them.** Anthropic returns
[`anthropic-ratelimit-requests-remaining`, `anthropic-ratelimit-tokens-remaining` and their
`-reset` timestamps](https://platform.claude.com/docs/en/api/rate-limits#response-headers)
in its responses; OpenAI returns
[`x-ratelimit-remaining-requests`, `x-ratelimit-remaining-tokens` and `x-ratelimit-reset-*`](https://platform.openai.com/docs/guides/rate-limits).
A gateway that records these per key sees the tier downgrade *before* the first 429: the
`limit` header steps down while your traffic does not. Two caveats, stated next to the
recommendation: not every provider sends them — the providers in this series' incidents
mostly did not — and a spend-cap pause may arrive without any of them (Anthropic's spend-cap
429 carries no `retry-after`, and the docs do not promise the ratelimit headers on it
either). Headers are a leading indicator where they exist, not a replacement for the
duration rule.

**Synthetic probes that take the real path and do not ignore 429.** LiteLLM already ships
the mechanism: [`background_health_checks: true` with
`enable_health_check_routing`](https://docs.litellm.ai/docs/proxy/health_check_routing)
pings every deployment on an interval and removes failures from the pool proactively — their
docs say it plainly, "The cooldown system is reactive." The same page then offers
`health_check_ignore_transient_errors: true`, under which "429 and 408 never affect routing"
and only "hard failures (401, 404, 5xx)" count. For congestion that is the right default. For
a paused account it is a switch labelled *do not notice this*. The design change is small:
a probe 429 at a time when your production request rate is a fraction of your known limit is
not transient, and the probe should say so.

The illustrative shape:

```yaml
# Provider state tracking — illustrative shape, not a product config.
providers:
  provider-a:
    triggers:                      # the usual list; nothing new here
      status: [429, 500, 502, 503, 504]
      timeout: true
      connection_error: true
    structural:                    # the row the matrix is missing
      persist_429_for: 15m         # 429 past this window at low utilization = state change
      model_lifecycle_4xx: true    # 404/400 with model_not_found|deprecated|retired codes
      quota_headers:
        track: [x-ratelimit-limit-requests, anthropic-ratelimit-requests-limit]
        alert_on_step_down: true   # the limit fell; your traffic did not
    readmission:
      transient: timer             # cooldown expiry, as today
      structural: probe            # only a successful probe on a dedicated key closes it
    probe:
      path: real_completion        # not /health; the completion path is what pauses
      key: probe-key-a             # its spend is its own line item
      treat_429_as: structural     # do NOT ignore transient errors on the probe path
```

Two costs, because there are always two. A structural state that only a probe can close keeps
a provider out longer than a timer would when the incident really was a thirty-second blip at
low utilization. How often the structural case happens I cannot tell you — this series'
evidence is a handful of forum threads, not a distribution — which is why `persist_429_for`
should come from your own 429 history, not from this file. And the probe key spends money
against a provider that is, by hypothesis, not serving you: at LiteLLM's default
`health_check_interval` of 300 seconds, 288 minimal completions a day. Small enough to
ignore, large enough to exist. That is the price of knowing.

## Three lines to grep, five questions to ask

1. **Your cooldown.** `cooldown_time` in LiteLLM; `fail_timeout` and `max_fails` in Kong;
   `cooldown_interval` in Portkey. Then ask what the documentation does not: what happens on
   the hundredth expiry? If the answer is "the same thing as the first," you have a
   fleet-level retry loop with no budget.
2. **Whether 404 is in your trigger set**, and decide on purpose. Kong's default leaves it
   out and never lets it trip the breaker; Portkey's default fails over on it; LiteLLM fails
   over, then sends the next user back in five seconds. Each is defensible. Not knowing
   which one you have is not.
3. **Whether your health checks ignore 429.** In LiteLLM that is
   `health_check_ignore_transient_errors`; if it is on, a paused account is invisible to the
   probe by configuration.

On a hosted router the same three become questions for the vendor, plus two: is cooldown
state shared across your instances or kept per process, and is there a metric or webhook
when a target is removed or re-admitted?

And one thing that needs no gateway change: alert, in whatever you already use for
dashboards, on fallback share — "provider B has carried more than X % of traffic for longer
than Y minutes" — or on the wall-clock duration of the primary's 429s. That is the counter
this post says nobody owns. In all seven documents the trigger lists are stateless, the
cooldowns are timers, and the one duration-shaped concept — a per-minute failure rate —
resets before your coffee is cold. The providers' billing systems are in your failure domain;
your gateway's retry policy is in theirs; and the state that connects the two currently lives
in a forum thread, eight hours after the fact. Until a gateway owns that counter, your
dashboard can. That is the row to add.

---

## Sources

### docs.litellm.ai

- [Fallbacks are how LiteLLM does automatic failover. If a call fails after…](https://docs.litellm.ai/docs/proxy/reliability)
- [`RetryPolicy`](https://docs.litellm.ai/docs/routing#advanced-custom-retries-cooldowns-based-on-error-type)
- [`allowed_fails: 3`, `cooldown_time: 5s`](https://docs.litellm.ai/docs/routing#cooldowns)
- [pins the provider SDK to `max_retries: 0`](https://docs.litellm.ai/docs/routing#where-num_retries-can-be-set-and-which-one-wins)
- [Budget Routing](https://docs.litellm.ai/docs/proxy/provider_budget_routing)
- [`background_health_checks: true` with `enable_health_check_routing`](https://docs.litellm.ai/docs/proxy/health_check_routing)

### portkey.ai

- [By default, fallback triggers on any non-2xx status code.](https://portkey.ai/docs/product/ai-gateway/fallbacks)
- [guardrails](https://portkey.ai/docs/product/guardrails)
- [up to 5](https://portkey.ai/docs/product/ai-gateway/automatic-retries)
- [circuit breaker](https://portkey.ai/docs/product/ai-gateway/circuit-breaker)

### openrouter.ai

- [any error can trigger the use of a fallback model, including: Context length…](https://openrouter.ai/docs/guides/routing/model-fallbacks)
- [OpenRouter](https://openrouter.ai/docs/guides/routing/provider-selection)
- [Prioritize providers that have not seen significant outages in the last 30…](https://openrouter.ai/docs/guides/routing/provider-selection#price-based-load-balancing-default-strategy)

### developers.cloudflare.com

- [Cloudflare can trigger your fallback provider in response to request errors or…](https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/)
- [`cf-aig-max-attempts` ≤ 5, `cf-aig-retry-delay` ≤ 5000 ms, `cf-aig-backoff`…](https://developers.cloudflare.com/ai-gateway/configuration/request-handling/)
- [Rate Limit and Budget Limit nodes](https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/)

### developer.konghq.com

- [Client errors don't trigger failover.](https://developer.konghq.com/plugins/ai-proxy-advanced/#retry-and-fallback)
- [Kong](https://developer.konghq.com/plugins/ai-proxy-advanced/reference/)

### github.com

- [router source](https://github.com/BerriAI/litellm/blob/main/litellm/utils.py)
- [You exceeded your current quota, please check your plan and billing details](https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb)

### docs.helicone.ai

- [Helicone](https://docs.helicone.ai/gateway/provider-routing#failover-triggers)
- [`Helicone-Retry-Num` default 5, factor 2, 1–10 s backoff](https://docs.helicone.ai/features/advanced-usage/retries)

### docs.llmgateway.io

- [LLMGateway](https://docs.llmgateway.io/features/routing#what-triggers-a-retry)
- [60-minute rolling window, time-decayed](https://docs.llmgateway.io/features/routing#low-uptime-protection)

### discuss.ai.google.dev

- [Google tier-downgrade thread](https://discuss.ai.google.dev/t/urgent-billing-account-downgraded-from-tier-3-to-tier-1-unexpectedly-service-paused/142555)

### platform.claude.com

- [API usage pauses until 00:00 UTC on the first day of the next month … API…](https://platform.claude.com/docs/en/api/rate-limits#reaching-your-spend-cap)
- [`anthropic-ratelimit-requests-remaining`,…](https://platform.claude.com/docs/en/api/rate-limits#response-headers)

### platform.openai.com

- [`x-ratelimit-remaining-requests`, `x-ratelimit-remaining-tokens` and…](https://platform.openai.com/docs/guides/rate-limits)