Self-Hosted LLMs vs API Providers in 2026: The Real Break-Even
Running your own model looks cheap until you price the GPU idle time. Here is the arithmetic nobody puts in the pitch deck.
"We'll just run an open model, it's free" is the most expensive sentence in AI engineering. Open weights are free; GPUs, on-call rotations, and 4am OOM errors are not.
The break-even calculation
A single A100-class GPU rented continuously costs roughly $1,100–$1,800 a month. That is the floor, and it is charged whether you serve one request or a million. API providers charge per token with zero idle cost.
| Monthly tokens | API cost (mid-tier model) | Self-hosted (1 GPU) | Winner |
|---|---|---|---|
| 20M | ~$40 | ~$1,400 | API by 35x |
| 500M | ~$900 | ~$1,400 | API |
| 2B | ~$3,600 | ~$1,800 (2 GPUs) | Self-host |
| 10B+ | ~$18,000 | ~$5,500 | Self-host clearly |
The crossover in 2026 sits around 1–2 billion tokens a month of steady traffic. Below it, self-hosting is a hobby subsidised by your runway.
The costs the table omits
- Engineering time. Serving stack, batching config, autoscaling, and evaluation harness is 2–4 weeks up front and ongoing maintenance forever.
- Idle waste. Bursty traffic means your GPU sits at 8% utilisation most of the day. Utilisation, not price per hour, decides the economics.
- Model upgrades. API providers ship better models for you. Self-hosting means you re-benchmark, re-tune, and re-deploy each time.
- Capability gap. A good 8B–70B open model handles classification, extraction, and summarisation well. Complex multi-step reasoning is still where frontier APIs earn their premium.
When self-hosting is right regardless of cost
- Contractual data residency or air-gapped deployment.
- Healthcare, defence, or government workloads where no third-party processing is permitted.
- Latency floors below ~150ms where a network hop to a provider is unacceptable.
- Extremely high, steady volume on a narrow task — the classic fine-tuned small-model case.
The hybrid architecture most teams should run
- Put a gateway in front of every call so the model is a configuration value, not a code change.
- Route high-volume, low-complexity tasks — classification, extraction, routing, moderation — to a small self-hosted or cheap hosted model.
- Route reasoning-heavy and customer-facing tasks to a frontier API.
- Run an eval suite against both continuously so you can move traffic on evidence.
- Keep a fallback provider wired up. Outages are the top cause of AI-feature downtime.
Decision shortcut
Under a billion tokens a month with no compliance mandate: use APIs, spend the saved engineering time on product. Above that, or under a residency requirement: build the hybrid, and self-host only the narrow tasks where a small model measurably matches the big one.
FAQ
Frequently asked questions
At what volume does self-hosting an LLM become cheaper?+
Roughly one to two billion tokens a month of steady traffic in 2026. Below that, idle GPU cost dominates and APIs are cheaper.
Are open models good enough for production?+
For classification, extraction, summarisation, and routing, yes. For complex multi-step reasoning, frontier API models still hold a meaningful lead.
What is the hidden cost of self-hosting?+
Low GPU utilisation on bursty traffic, plus the ongoing engineering time for serving, scaling, evaluation, and model upgrades.
Can I mix self-hosted and API models?+
Yes, and it is usually the right answer. Put a gateway in front, route cheap tasks to a small model and reasoning tasks to a frontier API, and compare with a shared eval suite.
Building something similar?
Let's talk in 30 minutes.