# Is a self-hosted LLM proxy better than a managed LLM gateway? Five things that change hands, and what each costs you

> No general answer exists. Self-hosted versus managed moves five things between you and a vendor: the provider key and contract, the data path, the failure domain, breaking changes, and operations. Each read from the vendors' own docs on 2026-09-03, with the price of holding it either way.

- Published: Sep 3, 2026
- Author: Leo Kaka, Engineering
- Tags: gateway, providers, reliability
- Canonical: https://pirouter.ai/blog/self-hosted-proxy-vs-managed-gateway

---
There is no general answer, and anyone who gives you one is selling one of the two. "Self-hosted
proxy or managed gateway" is a question about who holds five things: the provider key and the
contract behind it, the data path, the failure domain, the breaking change when a model ships, and
the operations. Going managed moves each to a vendor, and each move has a price. This post reads the
docs of LiteLLM, Portkey, Cloudflare AI Gateway, OpenRouter and Kong as they stood on 2026-09-03.
Where a page does not answer, it says so. No benchmarks.

![Matrix of five things that change hands between a self-hosted proxy and a managed gateway: key and contract, data path and logs, failure domain, breaking changes, and operations — shown for a self-hosted proxy, a managed gateway with your keys, and a managed gateway with theirs](/blog/images/self-hosted-proxy-vs-managed-gateway-handover-matrix.png "The five handovers, read from the vendors' own docs on 2026-09-03.")

## Four shapes, not two

The question assumes two products. The docs describe four:

| Shape | Who runs the process | Whose key | Examples, from the docs |
|---|---|---|---|
| Self-hosted proxy | You | Yours | LiteLLM OSS on Docker; Portkey's MIT gateway; Kong on-prem |
| Managed, your keys | Vendor | Yours, stored with them | Cloudflare BYOK; Portkey hosted; OpenRouter BYOK |
| Managed, their keys | Vendor | Theirs | OpenRouter credits; Cloudflare Unified Billing |
| Hybrid | Data plane you, control plane vendor | Yours | Portkey Enterprise; Kong Konnect |

The hybrid row is the one people forget.
[Portkey Enterprise](https://portkey.ai/docs/product/enterprise-offering/private-cloud-deployments)
runs the data plane "within your VPC" and the control plane "hosted by Portkey", with a one-minute
heartbeat between them. [Kong](https://developer.konghq.com/ai-gateway/) says data plane nodes "run
in your environment" and "connect to Konnect for configuration and observability". The prompt never
leaves your network; the routing configs and, in Portkey's words, "API keys" arrive from the
vendor's plane on each sync.

![Portkey's hybrid deployment diagram: a Data Plane box labelled Your VPC containing the AI app, the AI Gateway, Redis and a blob store, calling out to LLM providers; a Private Link to a separate Control Plane box containing the Portkey dashboard, a backend service and its own data stores](/blog/images/self-hosted-proxy-vs-managed-gateway-portkey-hybrid.png "The hybrid shape as the vendor draws it: traffic and logs stay in your VPC, configuration and keys sync from a control plane you do not run. Source: [Portkey — Enterprise Architecture](https://portkey.ai/docs/product/enterprise-offering/private-cloud-deployments)")

Everything below walks the five things across these columns, which matter more than the product
names: every product here sells more than one.

## Thing one: the provider key, and the contract behind it

Self-hosted, the key is an environment variable. LiteLLM's
[quickstart](https://docs.litellm.ai/docs/proxy/docker_quick_start) has you reference
`os.environ/OPENAI_API_KEY` and encrypts stored provider keys with `LITELLM_SALT_KEY`, which you set
once and "never change afterwards" — credentials encrypted under the old value cannot be read under
a new one. The contract is the one you signed. Cost of holding it: one account, one invoice, one
rate-limit tier to climb and one set of terms, per provider.

Managed with your keys, the key sits in the vendor's store. Cloudflare keeps it in
[Secrets Store](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/) and
tells you to "delete any hardcoded API keys or environment variables". Its
[credential precedence](https://developers.cloudflare.com/ai-gateway/features/unified-billing/) runs
in three steps: a provider key on the request, then the BYOK key under the `default` alias, then
Cloudflare-managed credentials billed to your Cloudflare balance. Read the note twice: "only the
BYOK key stored under the default alias prevents fall-through", so a key stored under `production`
is not consulted and the request "will fall through to Unified Billing". An alias name moves a
request from your contract to Cloudflare's — documented, and easy to do by accident.

Managed with their keys, the contract is with the intermediary, and the model provider is a party
you never met. OpenRouter's [terms](https://openrouter.ai/terms) state that "Each Model Provider
retains sole control over access to its Model", that OpenRouter "cannot guarantee that we will be
able to provide advance notice" of a suspension, and that it "may assign these Terms at any time
without notice or consent" while you may not. [BYOK](https://openrouter.ai/pricing) is free to
$25,000 of list-price inference a month, 5% after.

The intermediary itself can change hands. Palo Alto Networks
[announced](https://www.paloaltonetworks.com/company/press/2026/palo-alto-networks-to-acquire-portkey-to-secure-the-rise-of-ai-agents)
its intent to acquire Portkey on 30 April 2026, saying it "will continue to support existing and new
Portkey customers", and
[closed](https://www.paloaltonetworks.com/company/press/2026/palo-alto-networks-completes-acquisition-of-portkey-to-secure-ai-agents)
on 29 May 2026; the docs now carry a banner reading "Portkey is now PRISMA AIRS AI Gateway". No
judgement on the deal. It is the chain from the
[change-of-control post](/blog/model-access-change-of-control), one layer down: if your key sits
inside an intermediary, its ownership is a variable in your supply.

## Thing two: the data path, whose logs, and for how long

Four defaults, from four docs:

| | Logged by default | Retention | The off switch |
|---|---|---|---|
| LiteLLM, self-hosted | Spend logs in your Postgres; prompts only with `store_prompts_in_spend_logs` | Yours to set | [`disable_spend_logs`](https://docs.litellm.ai/docs/proxy/db_info); [`turn_off_message_logging`](https://docs.litellm.ai/docs/proxy/logging) |
| [Cloudflare](https://developers.cloudflare.com/ai-gateway/observability/logging/) | Prompt and response, "enabled by default" | [100,000 free, 10 million per paid gateway](https://developers.cloudflare.com/ai-gateway/reference/limits/) | `cf-aig-collect-log: false` |
| [Portkey, hosted](https://docs.portkey.ai/docs/product/observability/logs) | Full request and response | Developer 3 days, Production 30 days, Enterprise custom | [`x-portkey-debug: false`](https://portkey.ai/docs/product/administration/configuring-request-logging); Metrics Only is Enterprise |
| [OpenRouter](https://openrouter.ai/docs/guides/privacy/data-collection) | Metadata only; prompts "unless you opt in" | Not stated for metadata | Off unless you enable it |

One line deserves a second read. LiteLLM's
[security page](https://docs.litellm.ai/docs/data_security) says "No data or telemetry is stored on
LiteLLM Servers when you self-host" — true, but not the same as "no data is stored". Its
[production checklist](https://docs.litellm.ai/docs/proxy/prod) traces the 4Gi memory floor to spend
logging with `store_prompts_in_spend_logs` enabled, "because each row then carries a full prompt and
response". Self-hosting makes you the controller of a Postgres full of prompts, and the retention
policy is yours to write and enforce.

Cloudflare's ZDR toggle "does not apply to BYOK" and "does not control AI Gateway logging":
provider-side retention and gateway-side logging are two switches on two pages. Portkey adds that
switching to Metrics Only "will not retroactively remove previously logged data".

Region is already priced at the provider. Anthropic's
[`inference_geo: "us"`](https://platform.claude.com/docs/en/manage-claude/data-residency) costs 1.1x
"across all token pricing categories", and the parameter "is also not available through the OpenAI
SDK compatibility endpoint". A gateway that normalises Anthropic calls to Chat Completions has to
carry that field for you; whether it does is a question for its docs. A managed hop adds one more
retention policy and one more jurisdiction, and where it runs lives in a data processing agreement
none of these pages quote.

## Thing three: the failure domain, or a second account that can say no

The [billing-failure-domain post](/blog/billing-failure-domain) argued that your provider's billing
tier is inside your failure domain even while every status page is green. A managed gateway adds a
second account with the same power. Cloudflare warns that "your credit balance may go negative" and
it will charge the card on file; its limits page caps Unified Billing at 200 requests per 60 seconds
per gateway and notes the limit "does not apply to requests that use your own provider keys".
OpenRouter [sells the inverse](https://openrouter.ai/enterprise), "Fail over into our capacity when
your limits are hit": their capacity is your fallback, under terms permitting suspension "with or
without notice".

Self-hosted, the failure domain is one you already own, and LiteLLM's production checklist lists its
contents without flinching. The Helm chart's default `maxReplicas` of 100 "asks for roughly 1000
connections", "far past what a stock Postgres accepts". Below 4Gi, "a single large write is enough"
to have the kernel OOM-kill the pod. Without Redis, "each instance enforces limits independently".
Without a database, `max_budget: 100` "keeps serving requests past $100", and a startup warning "is
the only signal you get".

The hybrid shape shrinks this without removing it: Portkey's gateway "operates independently between
syncs" on a seven-day cache, but outbound "access to Control Plane endpoints" is required.

Cost: managed, a failure domain you cannot page; self-hosted, one you must.

## Thing four: breaking changes, and what they change on the way through

This is the thing the docs are most candid about and readers skip most. When Claude Fable 5.1
shipped, Anthropic's
[what's-new page](https://platform.claude.com/docs/en/models/fable-5-1/whats-new-fable-5-1) listed
forced tool use as gone: `tool_choice` of type `any` or `tool` "returns a 400
invalid_request_error". Thinking blocks became one-directional: when "a router or fallback that
switches models mid-conversation" carries a block the target cannot read, the API drops it, and
without the beta header "the drop is silent".

Now read what a hosted gateway did with the first one. The llmgateway.io
[changelog](https://llmgateway.io/changelog/claude-fable-5-1) for that model, dated 1 September 2026,
says: "Anthropic rejects forced tool use … with a 400. The gateway forwards these requests with
`auto` instead, so existing code keeps working." It also strips "sampling parameters such as
`temperature` and `top_p` before forwarding, so a request that sets them still succeeds". Two
rewrites, each turning a 400 into a 200.

This is not a complaint about that gateway. Absorbing breaking changes is why a managed layer
exists. But "existing code keeps working" is doing a lot of work: a forced tool call became optional,
and the JSON your parser expected may now arrive as prose, from a request that returned 200. Same
shape as decision four in the [failover post](/blog/multi-provider-failover-setup): a well-formed
response that is not the one you wanted.

A second kind of rewrite lands on policy, not parameters. Cloudflare: "If ZDR is enabled for a
provider that does not support it, AI Gateway falls back to the standard (non-ZDR) Unified Billing
configuration." llmgateway.io, on the same model: "A provider key from a zero-data-retention
Anthropic organization is rejected upstream." One downgrades the policy, one lets the refusal
through. Only one is what you meant.

Self-hosted, the breaking change lands on you, on a schedule. LiteLLM's
[release cycle](https://docs.litellm.ai/docs/proxy/release_cycle) is weekly — "weekly scheduled
releases bump the MINOR component" — and the enterprise page
[supports](https://docs.litellm.ai/docs/enterprise) "the four most recent stable minor lines";
older ones reach "end of life". Four weeks of releases is the whole support window.

Cost: managed, you read their changelog. Self-hosted, you keep up weekly or fall out of the
support window in a month.

## Thing five: operations, the invoice you do not receive

LiteLLM's [deployment pages](https://docs.litellm.ai/docs/proxy/deploy) are the most complete public
description of what "free to self-host" costs in hours. Postgres is "required for the proxy's auth
and tracking features"; Redis is "required once you run more than one instance"; a migrations job
runs once per upgrade while proxy pods set `DISABLE_SCHEMA_UPDATE=true`. Each worker wants 1 vCPU
and 4Gi, one per pod on Kubernetes. SSO "is free for up to 5 users", then an Enterprise licence;
audit logs, secret-manager integration and key rotation sit on the Enterprise side. The gateway is
free. The governance around it is a purchase.

Portkey's open-source gateway starts with one `npx` command, and its
[README](https://github.com/Portkey-AI/gateway) marks prompt management and observability with an
asterisk, "Available in hosted and enterprise versions". The Enterprise page lists Kubernetes, Helm,
1–2 cores and 2–4GB per instance, a Redis cache and an S3-compatible or MongoDB log store. Its "No
database management overhead" refers to the control-plane database; the cache and the log store are
yours.

Managed pricing is the invoice for the operations you did not do. Cloudflare's
[core features](https://developers.cloudflare.com/ai-gateway/reference/pricing/) are free, with 5%
on Unified Billing credits and Logpush on the paid plan. OpenRouter takes 5.5% on credits.
Portkey's [Production plan](https://portkey.ai/pricing) is $49 a month for 100,000 logs. Whether
that is cheap depends on your traffic and your on-call roster. I have neither, so I am not doing
that arithmetic.

## Rewrite the question

Put the five things against the three columns and the answer stops being a product name:

| | Self-hosted proxy | Managed, your keys | Managed, their keys |
|---|---|---|---|
| Key and contract | You | You, stored with them | Them |
| Data path and logs | You | Split: their logs, your policy | Them |
| Failure domain | Yours to page | Yours plus theirs | Theirs |
| Breaking changes | You, weekly | Them, per their changelog | Them, likewise |
| Operations | You | Them | Them |

"Is a self-hosted LLM proxy better than a managed LLM gateway" becomes "which of these five do I
want to hold". Three answers recur:

- **Regulated data.** Hold the data path. A hybrid data plane in your VPC is built for this; its
  cost is an outbound dependency on a control plane.
- **A small team with no on-call rotation.** Give away the failure domain and operations, keep the
  key through BYOK, and read the credential-precedence note.
- **Agent workloads heavy on tool calls.** Whoever absorbs breaking changes must tell you what they
  rewrite.

We are building a gateway designed to leave keys and contracts in the customer's hands; that is a
design position, not a shipping claim, and nothing above is ranked. Every quote comes from a page
linked here, read on 2026-09-03. Read the pages. They are more honest than any comparison chart,
including this one.

---

## Sources

### portkey.ai

- [Portkey Enterprise](https://portkey.ai/docs/product/enterprise-offering/private-cloud-deployments)
- [`x-portkey-debug: false`](https://portkey.ai/docs/product/administration/configuring-request-logging)
- [Production plan](https://portkey.ai/pricing)

### developer.konghq.com

- [Kong](https://developer.konghq.com/ai-gateway/)

### docs.litellm.ai

- [quickstart](https://docs.litellm.ai/docs/proxy/docker_quick_start)
- [`disable_spend_logs`](https://docs.litellm.ai/docs/proxy/db_info)
- [`turn_off_message_logging`](https://docs.litellm.ai/docs/proxy/logging)
- [security page](https://docs.litellm.ai/docs/data_security)
- [production checklist](https://docs.litellm.ai/docs/proxy/prod)
- [release cycle](https://docs.litellm.ai/docs/proxy/release_cycle)
- [supports](https://docs.litellm.ai/docs/enterprise)
- [deployment pages](https://docs.litellm.ai/docs/proxy/deploy)

### developers.cloudflare.com

- [Secrets Store](https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/)
- [credential precedence](https://developers.cloudflare.com/ai-gateway/features/unified-billing/)
- [Cloudflare](https://developers.cloudflare.com/ai-gateway/observability/logging/)
- [100,000 free, 10 million per paid gateway](https://developers.cloudflare.com/ai-gateway/reference/limits/)
- [core features](https://developers.cloudflare.com/ai-gateway/reference/pricing/)

### openrouter.ai

- [terms](https://openrouter.ai/terms)
- [BYOK](https://openrouter.ai/pricing)
- [OpenRouter](https://openrouter.ai/docs/guides/privacy/data-collection)
- [sells the inverse](https://openrouter.ai/enterprise)

### paloaltonetworks.com

- [announced](https://www.paloaltonetworks.com/company/press/2026/palo-alto-networks-to-acquire-portkey-to-secure-the-rise-of-ai-agents)
- [closed](https://www.paloaltonetworks.com/company/press/2026/palo-alto-networks-completes-acquisition-of-portkey-to-secure-ai-agents)

### docs.portkey.ai

- [Portkey, hosted](https://docs.portkey.ai/docs/product/observability/logs)

### platform.claude.com

- [`inference_geo: "us"`](https://platform.claude.com/docs/en/manage-claude/data-residency)
- [what's-new page](https://platform.claude.com/docs/en/models/fable-5-1/whats-new-fable-5-1)

### llmgateway.io

- [changelog](https://llmgateway.io/changelog/claude-fable-5-1)

### github.com

- [README](https://github.com/Portkey-AI/gateway)