Open-source vs closed LLMs: how to decide

Should you run an open model on your own infrastructure or pay for a closed model through an API? The answer depends on licenses, volume, data sensitivity and how much engineering you can take on. This guide untangles what "open" actually means, flags the license terms that catch teams out, works through a real cost comparison and ends with a decision table.

Open weights vs open source

Most models described as open source are more precisely open-weight: the trained parameters are published for download, usually under a license with conditions. The training data, the full training code and the recipe usually stay private.

The Open Source Initiative (OSI) published its Open Source AI Definition 1.0 in October 2024. It requires the freedom to use, study, modify and share a system for any purpose, and three components that make those freedoms real.

• Data information: enough detail about the training data, including its sources and how it was filtered, for a skilled person to build a substantially equivalent system. The definition doesn't require releasing the data itself when it can't legally be shared.
• Code: the complete code for data processing, training and inference, under OSI-approved licenses.
• Parameters: the model weights, under OSI-approved terms.

By that standard, very few popular models qualify. A small number of research releases, such as the Allen Institute for AI's OLMo models, publish training data, code and intermediate checkpoints, while most commercial labs publish only weights and a technical report. For buyers, the label matters less than the license: it decides what you may legally do with the model.

Closed models sit at the other end. You can't download GPT-6 Astra, Claude Opus 5.5 or Gemini 3.8 Flash; you send requests to the provider's servers and pay per token, under the provider's terms.

Common licenses and their gotchas

License terms differ more than the word "open" suggests. The table summarizes the licenses you'll meet most often, as of October 2026; it's general guidance, not legal advice, so read the actual license for any model you deploy.

Three gotchas deserve emphasis. First, licenses change between versions of the same family: Gemma 4 moved to Apache 2.0, while earlier Gemma models remain under Google's custom terms, so check the license for the exact version you download.

Second, custom licenses can reach your downstream work. The Llama 4 naming rule applies when you use Llama or its outputs to create or improve a model you distribute, and Gemma's terms treat models distilled from Gemma outputs as derivatives.

Third, use policies flow downstream. Under licenses that incorporate an acceptable use policy, you're responsible for making sure your own users follow it, which belongs in your terms of service.

The trade-offs, criterion by criterion

Neither option wins everywhere. This table compares a typical self-hosted open-weight deployment with a typical closed API.

The quality gap is measured in months

The capability gap between open and closed models has narrowed to a matter of months. Epoch AI's analysis in October 2025 found open-weight models trailing the best closed models by about three months on average on its capabilities index, a figure that will drift as each new frontier model lands.

By October 2026, the largest open releases have reached a scale that once belonged only to closed labs. Moonshot's Kimi K3 lists 2.8 trillion total parameters, Alibaba's Qwen3.8 family includes a 2.4-trillion-parameter model, and DeepSeek's V4.1-Flash supports a one-million-token context under an MIT license.

Closed models still tend to lead on the hardest agentic and reasoning tasks, and a lag of a few months matters if those tasks are your product. For most routine workloads, the gap is smaller than the variation between prompts.

The hardware reality

"Open" doesn't mean "runs on a laptop". OpenAI's gpt-oss-20b runs in about 16 GB of memory, and gpt-oss-120b fits on a single 80 GB GPU. DeepSeek's own deployment example for V4-Pro uses a node with four GB300 GPUs. Use the VRAM calculator to size hardware before you commit, and remember that a mixture-of-experts model needs memory for all of its parameters, not just the active ones.

Total cost of ownership: a worked example

Consider a document-processing service handling 100,000 requests a day, or about 3 million a month. Each request uses 2,000 input tokens and 400 output tokens, so the month totals 6 billion input and 1.2 billion output tokens.

For the self-hosted option, assume a mid-sized open model on three H100 GPUs running around the clock. At Lambda's on-demand list price of $3.99 per GPU-hour as of October 2026, that's 3 × $3.99 × 730 hours, or $8,738 a month. Add a quarter of an engineer's time at a $200,000 fully loaded annual cost, about $4,167 a month, and $500 for monitoring, storage and networking, and the total is about $13,400 a month.

Self-hosting beats the mid-tier API by about $10,600 a month, but only $2,000 once prompt caching is in play. Against a budget closed model or an open model on a hosted API, it costs several times more, and about 11 times the budget API.

The break-even point against the uncached mid-tier API, at $0.008 per request, is about 1.7 million requests a month, or 56,000 a day. Below that volume, the fixed costs dominate.

The example also assumes three GPUs can carry your peak load and that a quarter of an engineer is enough; both deserve testing, because underestimating either erases the savings. GPU prices vary widely across providers and commitment terms, and the self-host vs API calculator lets you plug in your own numbers. The real lesson is that model tier and caching move the bill more than the open-or-closed choice does, so the strongest financial case for self-hosting is steady, high volume.

When to choose which

Choose open weights when

• Data must stay inside your network or a specific jurisdiction, for contractual or regulatory reasons.
• Volume is high and steady enough to keep GPUs busy.
• You need deep customization, such as full fine-tuning or a distilled small model for one task.
• You need offline or edge deployment, or exact version control for audits.
• You have, or can hire, people to run inference infrastructure.

Choose a closed API when

• You need the strongest available model for hard reasoning, coding or agentic work.
• Traffic is low, spiky or unpredictable.
• Time to market matters more than unit cost.
• You want built-in features such as web search, computer use, file handling and managed caching.
• Your team is small and has no one to run GPUs.

Both options often satisfy privacy requirements. Business tiers of closed APIs offer contractual data protections, and some offer zero data retention, so self-hosting is a requirement only when data can't reach a third party at all.

Hybrid routing strategies

You don't have to pick one side. Many production systems use several models behind a single internal interface and send each request to the cheapest model that can handle it.

• Tiered routing: a small classifier or rules engine sends routine requests to an open model or budget API and escalates complex ones to a frontier model.
• Cascade with fallback: try the cheap model first and escalate when a validation check fails or the model reports low confidence.
• Sensitivity routing: requests containing personal or regulated data go to a self-hosted model, and everything else may use a closed API.
• Distillation: a frontier model labels examples, and a small open model is fine-tuned on them to handle the bulk of traffic, after you confirm the frontier provider's terms allow it.
• Provider failover: a second model takes over automatically during an outage or rate limiting.

Routing adds work: each route needs its own prompts, evaluations and monitoring. Start with two tiers and add more only when your data shows clear savings.

Decision table

Find the row that best describes your situation. If two rows apply, the one with the hardest constraint usually wins.

Common mistakes

These are the errors that most often turn an open-versus-closed decision into an expensive one. Each is avoidable with a little homework.

• Calling a model open source without reading its license. The license, not the label, decides what you may do.
• Counting only GPU costs. Engineering time, monitoring, idle capacity and upgrades often equal or exceed the hardware bill.
• Benchmarking a full-precision model and deploying a heavily quantized one. Evaluate the exact configuration you'll run.
• Assuming self-hosting is automatically secure. You now own patching, access control, logging and defenses against prompt injection.
• Treating the decision as permanent. Prices and quality shift monthly, so revisit the choice every quarter with fresh evaluation runs.

Next steps

Run your own numbers in the self-host vs API calculator, then read how to run LLMs locally to try an open model on your own machine. For the broader selection process, including scoring matrices and evaluation, see how to choose an LLM. If you'd like help planning a self-hosted or hybrid deployment, you can get expert help.