---
title: "Provider routing"
description: "pirouter spreads requests across the providers that serve a model by weight and fails over to the next provider on errors; workspace settings and the request's provider object change the order and the set."
canonical: "https://pirouter.ai/docs/guides/routing"
last-updated: "2026-10-17"
lang: "en"
source: "content/docs/en/guides/routing.mdx"
---

# Provider routing

pirouter spreads requests across the providers that serve a model by weight and fails over to the next provider on errors; workspace settings and the request's provider object change the order and the set.

With nothing set, pirouter picks one of the providers that serve the model by weight, and price plays no part in the choice. When that provider fails, the request moves to the next provider (failover). To change the order or the set of providers, set a workspace default in the console or add a `provider` object to a single request.

## The default

With nothing set, pirouter spreads requests across the providers that serve the model in proportion to their weight. Price plays no part in the default order.

A provider currently marked as down is never picked.

To see which providers serve a model, open its page or call `GET /v1/models/{vendor}/{model}/endpoints`. The `provider` of each entry is the name used everywhere below.

## Set the workspace default

Open **Routing** in the [console](https://app.pirouter.ai). Only workspace owners can change it; other members can view it.

| Setting               | What it does                                                                                                                |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| **Sort**              | Which provider goes first when several serve the same model: **Balanced**, **Lowest price** or **Custom order**             |
| **Allowed providers** | Only send requests to these providers                                                                                       |
| **Ignored providers** | Never send requests to these providers                                                                                      |
| **Fallback models**   | Tried in order when the requested model is unavailable; see [Model fallbacks](https://pirouter.ai/docs/guides/fallbacks.md) |

![The Routing page in the console: three fallback models, providers in a custom order, Allowed and Ignored providers off](/docs/shots/routing-page.en.webp)

A saved change applies from the next request.

## Override per request

Add a `provider` object at the top level of the request body. pirouter accepts the `provider` object on requests of all three protocols: OpenAI, Anthropic and Gemini.

```bash
curl https://api.pirouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $PIROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-opus-5.5",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "provider": { "only": ["azure"] }
  }'
```

A successful request returns `200`, and this one can only go to `azure`.

| Field             | Type                    | What it does                                                                             |
| ----------------- | ----------------------- | ---------------------------------------------------------------------------------------- |
| `order`           | array of strings        | Try these first, in this order. Providers not listed come after them and are still tried |
| `allow_fallbacks` | boolean, default `true` | With `false` and an `order`, only the providers in `order` may serve                     |
| `sort`            | `"price"`               | Rank by input price plus output price, lowest first                                      |
| `only`            | array of strings        | Use only these providers                                                                 |
| `ignore`          | array of strings        | Never use these providers. Wins over `only`                                              |
| `data_collection` | `"allow"` or `"deny"`   | `"deny"` excludes providers that train on your data or have not said whether they do     |
| `zdr`             | boolean                 | `true` keeps only providers with zero data retention                                     |
| `quantizations`   | array of strings        | Use only deployments at these precisions                                                 |

### Provider names

Names are case-insensitive. Besides the names in the catalog, three aliases are accepted: `google-vertex` (for `gcp`), `google-ai-studio` (for `gemini`) and `amazon-bedrock` (for `aws`).

An unknown name returns `422`, and the message names the field and the position. A name followed by `/` and a region is not supported and also returns `422`.

An empty array and `null` both mean the field is not set. `"only": []` does not mean no provider is allowed.

### Order and price

`order` is a preference, not an allowlist. To confine a request to the providers you list, add `"allow_fallbacks": false`:

```json
{ "provider": { "order": ["anthropic", "azure"], "allow_fallbacks": false } }
```

`"sort": "price"` compares input price plus output price, at the price tier this request would be billed at. Used with `order`, price only ranks providers within the same `order` position. Sorting by `"throughput"` or `"latency"` is not supported and returns `422`.

### Data policy and quantization

`"data_collection": "deny"` excludes two kinds of provider: those that train on prompts or have not declared, and those whose data retention is neither zero nor limited. `"zdr": true` is stricter and keeps only providers with zero data retention (ZDR).

`quantizations` takes `fp4`, `int4`, `int8`, `fp8`, `fp16`, `bf16`, `fp32` and `unknown`. Once it is set, a deployment that has not declared its precision is excluded too.

## How the two layers combine

When the workspace settings and a request's `provider` both apply:

- **Limits only tighten.** `only` and `quantizations` are intersected across the two layers; `ignore` is the union. If the workspace denies data collection, requires zero data retention or turns fallbacks off, a request cannot relax it.
- **The sort mode is replaced as a whole.** A request that sets either `order` or `sort` replaces the workspace sort mode entirely. If the workspace has a custom order and the request sends `"sort": "price"`, that request is ranked by price alone.

## When no provider is left

When the settings exclude every provider, the request returns `404` with the error code `no_eligible_provider`. Nothing was forwarded to a provider and nothing is billed; see [Requests that fail](https://pirouter.ai/docs/guides/billing.md#requests-that-fail).

```json
{
  "error": {
    "message": "No allowed provider can serve this model: the request's provider preferences, together with the workspace routing settings, exclude every provider that serves it. Providers serving it: anthropic, azure.",
    "type": "invalid_request_error",
    "code": "no_eligible_provider",
    "metadata": {
      "available_providers": ["anthropic", "azure"],
      "routing_funnel": [
        { "step": "initial", "providers": 2 },
        { "step": "data_policy", "providers": 2 },
        { "step": "quantization", "providers": 2 },
        { "step": "provider_ignore", "providers": 2 },
        { "step": "provider_only", "providers": 0 }
      ],
      "failed_step": "provider_only"
    }
  }
}
```

`metadata` tells you which step emptied the set:

- `available_providers`: the providers that serve the model before any setting applies.
- `routing_funnel`: how many are left after each step.
- `failed_step`: the step that left none.

| `failed_step`       | Setting behind it                                  |
| ------------------- | -------------------------------------------------- |
| `data_policy`       | `data_collection`, `zdr`                           |
| `quantization`      | `quantizations`                                    |
| `provider_ignore`   | `ignore`, plus the workspace **Ignored providers** |
| `provider_only`     | `only`, plus the workspace **Allowed providers**   |
| `fallback_disabled` | `allow_fallbacks` set to `false`                   |

`404 no_eligible_provider` is not the same as `404 model_not_found`: `model_not_found` means the model name is wrong; `no_eligible_provider` means the model exists and the settings left it no provider.

Two checks run before a provider is chosen, and a request that fails them is not forwarded or billed either: the model is not served on the endpoint you called (`422 validation_failed`), or the estimated input is more than 1.25 times the model's context limit (`400 context_length_exceeded`). See [Errors](https://pirouter.ai/docs/api/errors.md).

## Failover

When the chosen provider fails before the first byte of the response, pirouter retries on the next provider (failover). You do not need to retry yourself.

- **Retried**: the provider returns `429`, `500`, `502` or `503`, the connection fails, or the provider refuses pirouter's own credentials (`401`, `403`; this is not about your key).
- **Not retried**: any other `4xx` (a bad parameter, a context that is too long), a timeout, and any error after the response has started. Neither is a `429` that pirouter returns for a deployment's own quota: it comes before anything is forwarded; see [Rate limits](https://pirouter.ai/docs/api/rate-limits.md). A request whose provider is silent for 90 seconds starts its response then, so it is not retried after that point. See [Long requests](https://pirouter.ai/docs/guides/timeouts.md#long-requests).
- A request is tried at most 5 times, streaming or not.
- When every attempt fails, you get the error of the last attempt, with two exceptions. If the last provider refused pirouter's credentials (`401`, `403`), you get `503` with the error code [`provider_unavailable`](https://pirouter.ai/docs/api/errors.md#provider_unavailable). If pirouter's rate limit refused the next attempt, you get `429` with the error code `rate_limited` and a `Retry-After` header.

With `"allow_fallbacks": false` and no `order`, retries stay on the provider that was picked first.

## Which provider served a request

Read the response headers. A response returned by a provider carries these; add `-i` to curl to see them:

| Response header            | What it holds                                                                                             |
| -------------------------- | --------------------------------------------------------------------------------------------------------- |
| `X-Pi-Provider`            | The provider that served this request                                                                     |
| `X-Pi-Model`               | The ID of the model that served it, the same as `model` in the response body                              |
| `X-Pi-Upstream-Model`      | The model name the provider itself reported                                                               |
| `X-Pi-Upstream-Request-Id` | The provider's own request ID, for raising an issue with the provider; absent when the provider sent none |
| `X-Pi-Trace-Id`            | The trace ID, on every response; quote it when reporting a problem                                        |

An error the gateway raised itself (authentication, balance, rate limit, validation, no provider) was never served by a provider and carries `X-Pi-Trace-Id` only. So does a request that waited more than 90 seconds for the provider to start answering: its headers were sent before the provider answered. See [Long requests](https://pirouter.ai/docs/guides/timeouts.md#long-requests).

The `503 provider_unavailable` returned when a provider refuses pirouter's credentials is the exception: it carries `X-Pi-Provider` and `X-Pi-Model`, naming the provider that refused, but not `X-Pi-Upstream-Model` or `X-Pi-Upstream-Request-Id`.

After a failover, the headers describe the attempt that returned the response. A request that was forwarded to a provider can be found in **Logs** in the console by its `X-Pi-Trace-Id`. A request the gateway refused itself is not logged; see [Request logs](https://pirouter.ai/docs/guides/logs.md#requests-that-are-not-in-the-logs).

## What a mistake returns

A mistake inside `provider` always returns `422` with the error code `validation_failed`. The `message` starts with `Invalid provider preferences:` and names the field:

| Case                                    | What the `message` says                     |
| --------------------------------------- | ------------------------------------------- |
| An unknown key                          | `provider: unrecognized key "foo"`          |
| A known key that is not implemented yet | `provider.max_price is not supported yet`   |
| An unknown provider                     | `provider.only[0]: unknown provider "nope"` |
| A wrong type                            | `provider.zdr: must be a boolean`           |

## Not available today

- **Reading or changing the workspace routing settings through the API.** They are set in the console only; what the API offers is `provider` on each request.
- **Sorting by throughput or latency.** `sort` accepts `"price"` only.
- **Price ceilings and the remaining fields.** `max_price`, `require_parameters`, `preferred_min_throughput`, `preferred_max_latency`, `enforce_distillable_text` and `options` return `422`.

## Next steps

- [Model fallbacks](https://pirouter.ai/docs/guides/fallbacks.md): Switch to another model when the one you asked for is unavailable.
- [API reference](https://pirouter.ai/docs/api/chat-completions.md#body): The full definition of the `provider` object in a request body.