LLM API pricing explained

LLM APIs charge by the token, and the final bill depends on much more than the headline price. This guide explains how billing works, which tokens cost extra and which are discounted, what prices look like as of October 2026 and how to estimate costs, with three worked examples and the tactics that save the most. It covers pay-as-you-go API pricing; chat subscriptions charge a flat monthly fee per seat instead.

How LLM API billing works

What a token is

A token is a chunk of text the model reads or writes: a short word, part of a longer word, a number or a punctuation mark. A common rule of thumb for English is about four characters, or three-quarters of a word, per token, so 1,000 words come to roughly 1,300 tokens. Code, numbers and many other languages use more tokens per word, and each provider's tokenizer splits text differently. Paste your own text into the token counter for a real estimate.

The billing formula

Prices are quoted per million tokens, with separate rates for input and output:

cost = (input tokens / 1,000,000) x input price
     + (output tokens / 1,000,000) x output price

Input is everything you send: the system prompt, tool definitions, retrieved documents, images and the conversation so far. Most chat APIs are stateless, so every turn resends the full history and you pay for it again. Message 20 of a conversation costs far more than message one.

Example: 2,000 input tokens and 500 output tokens on a model charging $2 and $10 per million cost $0.004 + $0.005 = $0.009. That looks tiny, but a million such requests cost $9,000.

Why output costs more

Output tokens typically cost four to six times as much as input because of how inference works. The model processes all input tokens in one parallel pass, which keeps GPUs efficiently busy. Output is generated one token at a time, and each new token needs another pass through the model, so generation occupies expensive hardware far longer per token. Verbose answers therefore cost more than long prompts of the same length.

Reasoning tokens: the hidden output

Reasoning models work through a problem before answering, generating internal reasoning tokens. Providers bill these at the output rate even though you usually see only a summary or nothing at all. OpenAI's documentation notes that they also take up context window space, and the count appears in each response's usage data.

Example: a 300-token answer preceded by 3,000 reasoning tokens bills 3,300 output tokens. At $10 per million, that is $0.033 instead of $0.003, eleven times more. Most providers offer a reasoning-effort setting or thinking budget; keep it low for simple tasks and raise it only where accuracy improves.

What prices look like in October 2026

List prices in US dollars per million tokens, checked against official pricing pages on October 10, 2026. Prices change often, so confirm them before you budget.

Three patterns stand out: input prices span 100x, from $0.10 to $10.00 per million; output costs five to six times input; and cached input costs 5-10 percent of the normal input price. The comparison tool and provider directory track more models.

Discounts: prompt caching and batch processing

Prompt caching

Prompt caching stores the processed form of a prompt's opening section. When a later request starts with exactly the same prefix, the provider reads it from cache at a steep discount: $0.10 instead of $2.00 per million on GPT-6.1 Sol and Claude Sonnet 5.5.

• Write premium. Writing to the cache costs 1.25x the input price on OpenAI's newer models and Anthropic's default 5-minute cache, and 2x for Anthropic's 1-hour cache. Google also charges hourly storage for explicit caches.
• Setup. OpenAI caches eligible prompts automatically. Anthropic requires a cache_control setting in the request.
• Minimums and lifetimes. Prefixes must reach a minimum length: 1,024 tokens on OpenAI's current models and 512 to 4,096 tokens on Claude models, depending on the model. Caches expire after minutes of inactivity, so savings depend on steady traffic.
• Prompt order. Put stable content first (system prompt, tool definitions, reference documents, examples) and variable content last. A timestamp at the top breaks the cache on every request.

Caching can also stretch rate limits: Anthropic does not count cache reads toward input-token limits on most models. Check the cached token count in each response's usage data to confirm hits.

Batch processing

A batch API accepts a file of requests and returns results asynchronously. OpenAI, Anthropic and Google all list batch prices at 50 percent off, with results within 24 hours and often much sooner. OpenAI also gives batch jobs a separate, higher rate-limit pool. Use batch for evaluations, backfills, classification, summarization and embeddings, anything a person is not waiting on.

Some providers also sell speed tiers. Google's priority tier for Gemini 2.5 Flash lists $0.54 per million input tokens versus $0.30 standard, and Anthropic's fast mode for Claude Opus 5.5 doubles its rates to $8 input and $40 output.

Long context, images, audio and tools

Long-context price tiers

Some models charge more once a prompt crosses a size threshold, and the higher rate usually applies to the entire request, not just the overflow. A 300,000-token prompt on GPT-6.1 Sol costs $1.20 in input instead of $0.60, because it crosses the 272K-token line.

Gemini 3.1 Pro Preview doubles its input price above 200K tokens, and Claude Haiku 5.5 charges five times more above 100K. Other current Claude models bill their full 1M-token context window at standard rates. Regional processing can add a premium too: Anthropic charges 1.1x for US-only inference.

Images, audio and video

Multimodal inputs are converted to tokens. Anthropic's formula divides an image into 28-pixel squares, so a 1,000 x 1,000 pixel image costs 1,296 tokens, about $0.0026 at $2 per million, or $25.92 per 10,000 images. Newer Claude models accept larger images, so a full-resolution 1920 x 1080 screenshot costs 2,691 tokens; resize when fine detail does not matter. Image generation is usually priced per image: Google lists Gemini 3.1 Flash Image at about $0.067 per 1K-resolution image and $0.151 at 4K.

Audio input often costs more than text. Gemini 3.1 Flash-Lite charges $0.50 per million audio tokens versus $0.25 for text, and real-time voice models carry separate, higher rates.

Tool calls

Tool definitions are input tokens on every request, and Anthropic adds a tool-use system prompt of a few hundred tokens (286 on Claude Opus 5.5). Tool results you send back are input too. Built-in server tools add per-call fees:

• Web search costs $10 per 1,000 searches on both OpenAI and Anthropic, plus the retrieved content billed as tokens.
• Google's search grounding for Gemini 3 models costs $14 per 1,000 requests after 5,000 free each month.
• Anthropic's code execution tool includes 1,550 free hours per organization each month, then $0.05 per container-hour.

Rate limits and usage tiers

Rate limits cap how much you can use per minute or per day: requests per minute (RPM), tokens per minute (TPM) and sometimes daily totals. Anthropic splits input and output tokens per minute. Whichever limit you hit first applies, and exceeding it returns an HTTP 429 error, usually with a retry-after header.

Limits rise with spending history. OpenAI's paid tiers, Build, Launch and Grow, unlock after $5, $100 and $500 in total credit purchases; for GPT-6.1 Sol they allow 1 million, 4 million and 40 million TPM. Anthropic's Start, Build, Scale and Custom tiers advance automatically with usage, and the Start tier allows Claude Opus 5.5 1,000 RPM, 2 million input TPM and 400,000 output TPM. Tiers also carry monthly spending caps, such as $500 on OpenAI's Build and Anthropic's Start tiers.

Check capacity before launch. A chatbot peaking at 600 requests per minute with 5,300 tokens each needs about 3.2 million TPM, well beyond a 1 million TPM tier. Retry with exponential backoff and jitter, and move offline work to batch, which has separate limits.

Three worked cost examples

These examples use clearly labeled example prices, rounded from real October 2026 list prices: the small tier resembles GPT-6 Luna or Claude Haiku 5.5, the mid tier GPT-6.1 Sol or Claude Sonnet 5.5 and the frontier tier GPT-6 Astra.

Example 1: customer support chatbot

Assume 30,000 conversations a month with four model calls each, or 120,000 calls. Each call sends a 3,000-token fixed prefix (instructions, policies, tool definitions), about 2,000 tokens of history and retrieved text, and receives 300 output tokens.

• Mid tier, no caching: input 120,000 x 5,000 = 600M tokens x $2 = $1,200. Output 120,000 x 300 = 36M x $10 = $360. Total: $1,560 a month.
• With caching, assuming 2 percent of calls write the prefix and 98 percent read it: writes 7.2M x $2.50 = $18; reads 352.8M x $0.10 = $35.28; uncached input 240M x $2 = $480; output $360. Total: about $893, 43 percent less.
• Plus routing 70 percent of calls (FAQ-style questions) to the small tier: small share about $32.50, mid share about $267.98. Total: about $300, 81 percent below the starting point.

Example 2: document summarization pipeline

Assume 10,000 documents a month, each 12,000 tokens plus a 500-token instruction, producing a 600-token summary.

• Mid tier: input 10,000 x 12,500 = 125M x $2 = $250. Output 6M x $10 = $60. Total: $310.
• Mid tier through the batch API: $155. Caching does not help here, because the only shared text is the 500-token instruction, below the 1,024-token minimum.
• Small tier through batch: ($12.50 + $3.00) / 2 = $7.75, if its summaries pass your quality checks.
• Watch outliers: one 300,000-token contract on a model with a 272K threshold is billed at the higher tier for the whole request. Splitting it into two 150K sections keeps each request at standard rates.

Example 3: coding agent

An agent completes a task in 40 model calls. Its context starts at 20,000 tokens (instructions, tool definitions, repository map) and grows by 2,500 tokens per step as outputs and tool results accumulate. Each step outputs 2,000 tokens, including reasoning.

• Total input: 40 x 20,000 + 2,500 x (0 + 1 + ... + 39) = 800,000 + 1,950,000 = 2.75M tokens. Output: 80,000 tokens.
• Mid tier, no caching: 2.75M x $2 = $5.50, plus 80,000 x $10 per million = $0.80. Total: $6.30 per task.
• Mid tier with caching: each step reads the previous context from cache and writes only new tokens, so writes total 117,500 tokens ($0.29) and reads 2,632,500 ($0.26). With output, the total is about $1.36 per task, 78 percent less.
• Frontier tier with caching: $1.47 + $2.63 + $4.00 = about $8.10 per task ($31.50 uncached). If higher reasoning effort tripled output to 6,000 tokens per step, the output line alone would rise from $4.00 to $12.00.

At 10 tasks a day over 21 workdays, one developer's agent costs about $285 a month on the cached mid tier and about $1,700 on the cached frontier tier. Run your own numbers in the LLM cost calculator.

How to cut your LLM API costs

1. Route by difficulty. Send routine requests to a small model and escalate hard ones. In Example 1, routing cut a cached bill from $893 to about $300.
2. Cache stable prefixes. Keep the start of every prompt identical and verify hits in the usage data.
3. Batch anything that can wait for an immediate 50 percent saving.
4. Shorten prompts. Trim instructions, summarize old conversation turns and retrieve fewer, better passages. Every 1,000 tokens removed saves $2 per 1,000 requests at the mid-tier input rate.
5. Control output. Set a maximum output length, ask for concise answers and use structured outputs so you get compact JSON instead of prose and fewer failed parses to retry. Lower reasoning effort for simple tasks.
6. Use smaller models. In the example sheet, the small tier costs one-twentieth of the mid tier. A small language model, possibly fine-tuned or distilled, often handles narrow tasks well.
7. Stay under long-context thresholds by splitting documents or retrieving only relevant sections.
8. Monitor spending. Log tokens per request and per feature, set budget alerts and use provider spend limits.

When self-hosting breaks even

Self-hosting an open-weight model replaces per-token fees with a fixed hardware cost. Take an example GPU server rented at $2.50 an hour around the clock: $2.50 x 730 hours = $1,825 a month. At a typical 3:1 ratio of input to output tokens, the example mid tier averages (3 x $2 + $10) / 4 = $4.00 per million, so break-even is $1,825 / $4.00 = about 456 million tokens a month, or 15 million a day. Against the small tier ($0.20 per million blended), break-even jumps to about 9.1 billion tokens a month.

Those figures assume the self-hosted model matches the API model's quality, which you must test, and they exclude engineering time, idle capacity and redundancy. The self-host vs API calculator handles the full comparison, and running LLMs locally covers hardware sizing.

Common mistakes

• Budgeting on input price alone. Output and reasoning tokens often dominate the bill.
• Forgetting that history is resent. Long chats and agent loops grow input with every turn.
• Breaking the cache. Dates, user names or shuffled tool lists at the top of a prompt prevent cache hits.
• Running offline jobs synchronously instead of through a batch API.
• Ignoring thresholds and extras. Long-context tiers, image sizes and per-search fees add up quietly.
• Choosing on price without testing quality. A cheaper model that needs retries or human fixes can cost more overall.

Next steps

Estimate your monthly bill with the LLM cost calculator. If your volume is large and steady, test the economics in the self-host vs API calculator. To weigh price against quality and speed, read how to choose an LLM. For a cost review of an existing application, you can hire an LLM expert.