How to Estimate LLM API Costs Before You Build
Teams usually discover their LLM bill after launch. It is predictable in advance with four numbers and a spreadsheet. This is the method we use in every proposal.
- Published
The formula
Monthly cost ≈ tasks per month × (input tokens × input price + output tokens × output price), adjusted for caching. Everything else is estimating those inputs honestly.
- Count tasks: conversations, documents, tickets — whatever one unit of work is.
- Measure tokens per task: run 20 realistic examples through a prototype and log input and output tokens. Do not guess.
- Apply caching: the stable part of your prompt (instructions, tool definitions) can be cached, which cuts the cost of those tokens sharply on providers that support it.
- Choose the model mix: route easy tasks to a smaller model and hard ones to a larger model.
A worked example
A support agent handles 20,000 conversations a month. Measured on a prototype: 6,000 input tokens and 400 output tokens per conversation across all turns, of which 4,000 input tokens are the stable system prompt and tool definitions.
- Without caching, every turn re-sends the full 6,000 tokens at the normal input price.
- With caching, 4,000 of those tokens are billed at the discounted cached rate, so the effective input cost drops by more than half.
- Routing the 60% of simple questions to a smaller model cuts the remaining cost again.
Plug in your provider’s current per-token prices and you have a monthly figure that is usually within 20–30% of reality — close enough to make a build-or-don’t decision.
Where estimates go wrong
- Conversation history: every turn re-reads earlier turns, so long chats cost far more than short ones. Summarise or trim history.
- Retrieval bloat: stuffing ten documents into context when two would do.
- Retries and loops: an agent that retries tool calls can multiply cost. Cap steps per task.
- Output length: verbose answers cost more and read worse. Ask for concise output.
After launch
Log tokens and cost per task from day one and chart them weekly. Cost per resolved task is the number to watch — not total spend, which should grow with usage.
Frequently asked questions
Should we always use the cheapest model?
No. A cheaper model that fails more often costs more in retries, escalations and lost customers. Choose per task based on eval results.
How accurate is this method?
With measured token counts from a prototype, typically within 20–30%. With guessed counts, it can be off by several times.
Can you estimate ours?
Yes — send us a description of the task and expected volume and we will build the estimate with you.
