Blog
Prompt caching
Part of the Unbiased glossary: the vocabulary of LLM pricing and infrastructure, defined with receipts instead of adjectives.
Prompt caching lets an LLM API reuse previously processed context instead of recomputing it, billing cached input at a fraction of the normal rate: about 10% on Anthropic, and a 90% read discount with a 1.25x write premium on OpenAI. For context-heavy workloads it is the largest single cost lever available.
Why it matters
Most production traffic resends the same system prompt, tool definitions, and shared context on every request. Without caching you pay full input price for identical tokens thousands of times a day; with it, those tokens bill at roughly a tenth. Agentic workloads amplify this: coding agents reread files constantly.
Measured: what caching did to a real agentic session
We ran a real Claude Code bug-fix session on Fable 5 and published the full ledger: 5 API calls, 114,419 cached-read tokens against just 1,859 written. The session billed $0.5887 with caching; the identical session without it computes to $1.5422, a 2.6x difference. At Fable rates, caching is not an optimization, it is the difference between viable and not. Full ledger in the Fable 5 breakdown.
Common questions
How much does prompt caching save?
Cache reads bill at roughly 10% of the input rate on Anthropic ($1/M on Fable 5 instead of $10/M) and at a 90% discount on OpenAI, which charges 1.25x for cache writes. Our measured agentic session saved 2.6x overall.
When does caching hurt instead of help?
When your context churns: every change to cached content triggers a write at full or premium price. Stable system prompts and shared documents cache well; per-request context does not.
Do I need to change my code to use caching?
On Anthropic you mark cacheable blocks explicitly; on OpenAI caching applies automatically to repeated prefixes. Either way, structuring prompts so stable content comes first is what makes it work.