Tokens are the row count and the context window is a scan, so LLM cost behaves like a query plan: where does the work happen, what gets read twice, and what can you avoid reading at all. This calculator turns the levers from What an LLM costs at campus scale into something you can run against your own numbers.

The four levers it models:

  • Tokens. You pay per input and output token, and output usually costs several times more. Retrieval-heavy prompts are input-dominated, so trimming the prompt is often the biggest win.
  • Caching. A stable prompt prefix can be reused at roughly a tenth of the input price. Keep the stable material first so the cached prefix stays large.
  • Batching. Work nobody is waiting on can run on an asynchronous batch path at around half price.
  • Routing. Send simple questions to a cheaper model and escalate only the hard ones, instead of paying flagship rates for every request.

How much does an LLM cost per request?

A request costs input tokens times the input price plus output tokens times the output price, both billed per million tokens. Output tokens usually cost several times more than input, but retrieval-heavy prompts are input-dominated, so most spend is often on the read, not the write.

How much does prompt caching save?

Caching a stable prompt prefix bills repeated tokens at roughly a tenth of the input price. For a high-traffic endpoint whose system prompt, tool definitions, and guardrail preamble do not change, that can cut effective input cost substantially. Keep the stable parts first so the cached prefix stays large; a per-request token near the top shrinks it to nothing.

What is batch pricing?

Asynchronous batch APIs typically run about 50% cheaper than real-time calls in exchange for slower, best-effort turnaround. Move any work nobody is waiting on, such as indexing, summaries, metadata extraction, and evals, to the batch path.

How do I estimate monthly LLM cost?

Multiply the effective per-request cost by requests per day and by 30. Lower it by trimming the prompt, raising the cache hit rate, batching offline work, and routing simple questions to a cheaper model before escalating the hard ones.

Why is my LLM bill higher than expected?

Usually a bloated prompt, a cache prefix broken by a variable token near the top, or everything routed to the flagship model. Track cost per request bucketed by feature so the offender points itself out, like a slow query log.