Is a self-hosted LLM proxy better than a managed LLM gateway? Five things that change hands, and what each costs you

There is no general answer, and anyone who gives you one is selling one of the two. “Self-hosted proxy or managed gateway” is a question about who holds five things: the provider key and the contract behind it, the data path, the failure domain, the breaking change when a model ships, and the operations. Going managed moves each to a vendor, and each move has a price. This post reads the docs of LiteLLM, Portkey, Cloudflare AI Gateway, OpenRouter and Kong as they stood on 2026-09-03. Where a page does not answer, it says so. No benchmarks.

Four shapes, not two

The question assumes two products. The docs describe four:

The hybrid row is the one people forget. Portkey Enterprise runs the data plane “within your VPC” and the control plane “hosted by Portkey”, with a one-minute heartbeat between them. Kong says data plane nodes “run in your environment” and “connect to Konnect for configuration and observability”. The prompt never leaves your network; the routing configs and, in Portkey’s words, “API keys” arrive from the vendor’s plane on each sync.

Everything below walks the five things across these columns, which matter more than the product names: every product here sells more than one.

Thing one: the provider key, and the contract behind it

Self-hosted, the key is an environment variable. LiteLLM’s quickstart has you reference os.environ/OPENAI_API_KEY and encrypts stored provider keys with LITELLM_SALT_KEY, which you set once and “never change afterwards” — credentials encrypted under the old value cannot be read under a new one. The contract is the one you signed. Cost of holding it: one account, one invoice, one rate-limit tier to climb and one set of terms, per provider.

Managed with your keys, the key sits in the vendor’s store. Cloudflare keeps it in Secrets Store and tells you to “delete any hardcoded API keys or environment variables”. Its credential precedence runs in three steps: a provider key on the request, then the BYOK key under the default alias, then Cloudflare-managed credentials billed to your Cloudflare balance. Read the note twice: “only the BYOK key stored under the default alias prevents fall-through”, so a key stored under production is not consulted and the request “will fall through to Unified Billing”. An alias name moves a request from your contract to Cloudflare’s — documented, and easy to do by accident.

Managed with their keys, the contract is with the intermediary, and the model provider is a party you never met. OpenRouter’s terms state that “Each Model Provider retains sole control over access to its Model”, that OpenRouter “cannot guarantee that we will be able to provide advance notice” of a suspension, and that it “may assign these Terms at any time without notice or consent” while you may not. BYOK is free to $25,000 of list-price inference a month, 5% after.

The intermediary itself can change hands. Palo Alto Networks announced its intent to acquire Portkey on 30 April 2026, saying it “will continue to support existing and new Portkey customers”, and closed on 29 May 2026; the docs now carry a banner reading “Portkey is now PRISMA AIRS AI Gateway”. No judgement on the deal. It is the chain from the change-of-control post, one layer down: if your key sits inside an intermediary, its ownership is a variable in your supply.

Thing two: the data path, whose logs, and for how long

Four defaults, from four docs:

One line deserves a second read. LiteLLM’s security page says “No data or telemetry is stored on LiteLLM Servers when you self-host” — true, but not the same as “no data is stored”. Its production checklist traces the 4Gi memory floor to spend logging with store_prompts_in_spend_logs enabled, “because each row then carries a full prompt and response”. Self-hosting makes you the controller of a Postgres full of prompts, and the retention policy is yours to write and enforce.

Cloudflare’s ZDR toggle “does not apply to BYOK” and “does not control AI Gateway logging”: provider-side retention and gateway-side logging are two switches on two pages. Portkey adds that switching to Metrics Only “will not retroactively remove previously logged data”.

Region is already priced at the provider. Anthropic’s inference_geo: "us" costs 1.1x “across all token pricing categories”, and the parameter “is also not available through the OpenAI SDK compatibility endpoint”. A gateway that normalises Anthropic calls to Chat Completions has to carry that field for you; whether it does is a question for its docs. A managed hop adds one more retention policy and one more jurisdiction, and where it runs lives in a data processing agreement none of these pages quote.

Thing three: the failure domain, or a second account that can say no

The billing-failure-domain post argued that your provider’s billing tier is inside your failure domain even while every status page is green. A managed gateway adds a second account with the same power. Cloudflare warns that “your credit balance may go negative” and it will charge the card on file; its limits page caps Unified Billing at 200 requests per 60 seconds per gateway and notes the limit “does not apply to requests that use your own provider keys”. OpenRouter sells the inverse, “Fail over into our capacity when your limits are hit”: their capacity is your fallback, under terms permitting suspension “with or without notice”.

Self-hosted, the failure domain is one you already own, and LiteLLM’s production checklist lists its contents without flinching. The Helm chart’s default maxReplicas of 100 “asks for roughly 1000 connections”, “far past what a stock Postgres accepts”. Below 4Gi, “a single large write is enough” to have the kernel OOM-kill the pod. Without Redis, “each instance enforces limits independently”. Without a database, max_budget: 100 “keeps serving requests past $100”, and a startup warning “is the only signal you get”.

The hybrid shape shrinks this without removing it: Portkey’s gateway “operates independently between syncs” on a seven-day cache, but outbound “access to Control Plane endpoints” is required.

Cost: managed, a failure domain you cannot page; self-hosted, one you must.

Thing four: breaking changes, and what they change on the way through

This is the thing the docs are most candid about and readers skip most. When Claude Fable 5.1 shipped, Anthropic’s what’s-new page listed forced tool use as gone: tool_choice of type any or tool “returns a 400 invalid_request_error”. Thinking blocks became one-directional: when “a router or fallback that switches models mid-conversation” carries a block the target cannot read, the API drops it, and without the beta header “the drop is silent”.

Now read what a hosted gateway did with the first one. The llmgateway.io changelog for that model, dated 1 September 2026, says: “Anthropic rejects forced tool use … with a 400. The gateway forwards these requests with auto instead, so existing code keeps working.” It also strips “sampling parameters such as temperature and top_p before forwarding, so a request that sets them still succeeds”. Two rewrites, each turning a 400 into a 200.

This is not a complaint about that gateway. Absorbing breaking changes is why a managed layer exists. But “existing code keeps working” is doing a lot of work: a forced tool call became optional, and the JSON your parser expected may now arrive as prose, from a request that returned 200. Same shape as decision four in the failover post: a well-formed response that is not the one you wanted.

A second kind of rewrite lands on policy, not parameters. Cloudflare: “If ZDR is enabled for a provider that does not support it, AI Gateway falls back to the standard (non-ZDR) Unified Billing configuration.” llmgateway.io, on the same model: “A provider key from a zero-data-retention Anthropic organization is rejected upstream.” One downgrades the policy, one lets the refusal through. Only one is what you meant.

Self-hosted, the breaking change lands on you, on a schedule. LiteLLM’s release cycle is weekly — “weekly scheduled releases bump the MINOR component” — and the enterprise page supports “the four most recent stable minor lines”; older ones reach “end of life”. Four weeks of releases is the whole support window.

Cost: managed, you read their changelog. Self-hosted, you keep up weekly or fall out of the support window in a month.

Thing five: operations, the invoice you do not receive

LiteLLM’s deployment pages are the most complete public description of what “free to self-host” costs in hours. Postgres is “required for the proxy’s auth and tracking features”; Redis is “required once you run more than one instance”; a migrations job runs once per upgrade while proxy pods set DISABLE_SCHEMA_UPDATE=true. Each worker wants 1 vCPU and 4Gi, one per pod on Kubernetes. SSO “is free for up to 5 users”, then an Enterprise licence; audit logs, secret-manager integration and key rotation sit on the Enterprise side. The gateway is free. The governance around it is a purchase.

Portkey’s open-source gateway starts with one npx command, and its README marks prompt management and observability with an asterisk, “Available in hosted and enterprise versions”. The Enterprise page lists Kubernetes, Helm, 1–2 cores and 2–4GB per instance, a Redis cache and an S3-compatible or MongoDB log store. Its “No database management overhead” refers to the control-plane database; the cache and the log store are yours.

Managed pricing is the invoice for the operations you did not do. Cloudflare’s core features are free, with 5% on Unified Billing credits and Logpush on the paid plan. OpenRouter takes 5.5% on credits. Portkey’s Production plan is $49 a month for 100,000 logs. Whether that is cheap depends on your traffic and your on-call roster. I have neither, so I am not doing that arithmetic.

Rewrite the question

Put the five things against the three columns and the answer stops being a product name:

“Is a self-hosted LLM proxy better than a managed LLM gateway” becomes “which of these five do I want to hold”. Three answers recur:

• Regulated data. Hold the data path. A hybrid data plane in your VPC is built for this; its cost is an outbound dependency on a control plane.
• A small team with no on-call rotation. Give away the failure domain and operations, keep the key through BYOK, and read the credential-precedence note.
• Agent workloads heavy on tool calls. Whoever absorbs breaking changes must tell you what they rewrite.

We are building a gateway designed to leave keys and contracts in the customer’s hands; that is a design position, not a shipping claim, and nothing above is ranked. Every quote comes from a page linked here, read on 2026-09-03. Read the pages. They are more honest than any comparison chart, including this one.