What Is Prompt Caching and How Much It Saves
Most agent prompts repeat the same thousands of tokens on every call: instructions, tool definitions, reference material. Prompt caching stops you paying full price for that repetition.
- Published
How it works
When the beginning of a prompt is identical to a recent request, providers that support caching can reuse their internal processing of that prefix. You pay a reduced rate for the cached part, and the response starts faster.
Structure prompts for caching
- Put stable content first: system instructions, tool definitions, fixed reference documents.
- Put variable content last: the user’s message, retrieved passages, the current date.
- Avoid tiny changes at the start — a timestamp at the top breaks the cache for everything after it.
- Keep tool definitions in a fixed order.
What it saves
For agents with long system prompts and many turns, caching commonly cuts input-token cost by half or more and noticeably reduces time to first token. The exact discount and cache lifetime depend on the provider, so check their current documentation when modelling costs.
Limits
- Caches expire after a period of inactivity, so very low-traffic apps benefit less.
- Only exact prefix matches count.
- Some providers charge a small premium to write the cache, repaid on reuse.
Frequently asked questions
Does caching change the model’s answers?
No. It changes how the provider processes the input, not the output.
Is my data stored longer?
Caches are short-lived and scoped to your account per provider terms; check them for your compliance needs.
Should every project use it?
Any project with a long, stable prompt prefix and repeat traffic should.
