You turned on prompt caching, and your input bill went up. That’s not a bug. Prompt caching has a discount on reads, but it also has a surcharge on writes, and if nothing ever reads the cache back, you’re just paying 25% extra for the same request.
If your app sends a long system prompt, a big tool list, or a stack of reference docs with every call, you’re paying for tens of thousands of input tokens even when the user types one line. Anthropic and OpenAI both offer prompt caching to cut that repeated cost. Whoever makes the API calls decides whether to cache, where the cached part ends, and how long it lives, so this piece is written for developers calling the API directly. Based on the two companies’ own docs, here’s when caching pays off and how to structure prompts so the cache actually hits.
All multipliers, minimum lengths, and model names here come from the two companies’ docs as checked on November 8, 2026. Pricing and model lists change often, so double-check the docs before you build on these numbers.
It caches computation, not answers
“Caching” usually means response caching: store the output for a given input and hand it back next time. Prompt caching isn’t that.
When a Transformer reads your prompt, it computes key and value vectors for every token so attention can work. That’s the KV cache. Prompt caching keeps those computed states around for a while, so when the next request starts with the same prefix, the model can skip recomputing it and pick up from there. Anthropic’s docs say it doesn’t store the raw text of your prompts; it holds KV cache representations and cryptographic hashes in memory for the cache lifetime.
That’s why the matching rule is strict. The whole prefix, from the start of the request up to the cache breakpoint, has to be identical. OpenAI’s guide puts it as “cache reuse requires the entire rendered prefix to match.” Switch models and you start from a cold cache.
Skipping that recomputation saves time as well as money. Both docs say caching cuts the time before the first token arrives on long inputs. Neither gives a number, and OpenAI notes that a live cache doesn’t guarantee every request actually reuses it or comes back faster.
Writes cost more, reads cost much less
Here are Anthropic’s multipliers, relative to the normal input price:
| Operation | Multiplier |
|---|---|
| Cache write, 5-minute lifetime | 1.25x |
| Cache write, 1-hour lifetime | 2x |
| Cache read, most models | 0.1x |
| Cache read, Claude Opus 5.5 and Sonnet 5.5 | 0.05x |
| Cache read, Claude Fable 5.1 and Mythos 5.1 | 0.025x |
OpenAI splits by model generation. On GPT-5.6 and later, a cache write with the default 30-minute lifetime costs 1.25x, a 12-hour write costs 2.5x, and reads cost 0.1x on most models (0.05x on GPT-6.1 Sol). Earlier models have no separate cache-write charge. OpenAI’s guide says cached input can be discounted by up to 95%.
The line that matters most is the first one. Send a 10,000-token prompt with caching on, get no matching request within five minutes, and you’ve paid for 12,500 tokens instead of 10,000.
One read is enough to flip that. Two uncached requests cost 2.0x. One write plus one read costs 1.25 + 0.10 = 1.35x, which is 32.5% less. Stretch it to ten requests and one write plus nine reads comes to 2.15x instead of 10x, a 78.5% cut. OpenAI’s guide uses these exact numbers as its own example.
Newer models with cheaper reads do even better. By our math, one write plus nine reads costs 1.70x at a 0.05x read rate (83% less) and 1.475x at 0.025x (about 85% less). The write surcharge doesn’t change, though, so a prefix that’s never read again costs 25% more on every model.
The 5-minute timer resets on every read
Anthropic’s default cache lifetime is five minutes, but every read refreshes it at no extra charge. A support bot that gets a follow-up every few minutes pays the 1.25x write once, then keeps the cache warm on reads alone for as long as the conversation keeps moving.
The opposite case is where people lose money. If a background job calls the API every seven minutes, each call lands after the cache has already expired. You pay the write surcharge every time and never get a single discounted read. That’s where longer lifetimes come in. Anthropic offers a 1-hour cache (2x to write), and OpenAI’s GPT-5.6 and later models offer 12 hours (2.5x to write).
A bigger write surcharge pushes the break-even point further out. Here’s our math at a 0.1x read rate:
| Lifetime (write cost) | Read back once | Read back twice |
|---|---|---|
| 5 or 30 minutes (1.25x) | 1.35x vs 2x uncached, ahead | 1.45x vs 3x, ahead |
| 1 hour (2x) | 2.1x vs 2x, behind | 2.2x vs 3x, ahead |
| 12 hours (2.5x) | 2.6x vs 2x, behind | 2.7x vs 3x, ahead |
So a longer lifetime only pays when you’re confident the prefix gets read at least twice within it. Anthropic’s own suggestions for the 1-hour cache line up with that: prompts used less often than every five minutes but more often than every hour, like a side agent that takes longer than five minutes, or a long chat where the user might take a while to reply. Q&A over a long contract or manual fits here too: if users read for a few minutes between questions, check whether you’ll get at least two questions within the hour and pick the 1-hour option if so. If a prefix doesn’t come back even once an hour, turn caching off.
Structuring prompts so the cache hits
It all starts with one rule: stack the prompt by how often each part changes. On Anthropic, it looks like this:
[tool definitions] rarely change
[system instructions, docs] breakpoint 1 at the end
[conversation so far] breakpoint 2 (or automatic caching)
[current time, new question] not cached
In an actual request, you can follow the pattern from Anthropic’s docs: put cache_control on the system block and add one more cache_control at the top level. The explicit breakpoint pins the system prompt, and automatic caching follows the growing conversation. You get up to four breakpoints per request.
{
"model": "claude-opus-5-5",
"max_tokens": 1024,
"cache_control": { "type": "ephemeral" },
"system": [
{
"type": "text",
"text": "Long instructions and reference docs...",
"cache_control": { "type": "ephemeral" }
}
],
"messages": [{ "role": "user", "content": "What's the refund policy?" }]
}
Keep anything that changes after the breakpoint
If a timestamp or request ID sits before your breakpoint, the prefix hash changes on every call, even when the 40,000 tokens after it are identical. The example Anthropic’s docs walk through is a close cousin: putting the breakpoint itself on a block that changes every request, like one holding a timestamp or the incoming message. Same result either way. You pay for a fresh write every time and never get a read. The docs’ advice is to put the breakpoint on the last block that stays identical across requests, which is the end of your static content.
Lock down tool definitions and key order
Anthropic builds the cache in a fixed order: tools, then system, then messages. A change at one level invalidates that level and everything after it, so touching the tool list also throws away your cached system prompt. The docs also warn that some languages, Swift and Go for example, randomize key order during JSON conversion, so check that the keys in your tool_use blocks come out in a stable order.
Even when you want a tool unavailable for one request, don’t remove it. OpenAI’s guide suggests leaving definitions in place and setting tool_choice to "none", or narrowing what’s callable with allowed_tools. Don’t copy that to Anthropic, though: its docs say changing tool_choice keeps the tools and system caches but invalidates the cached conversation.
Check the minimum length first
There’s a minimum length below which nothing gets cached. On Anthropic it depends on the model: 512 tokens for Claude Fable 5.1, Opus 5.5, Opus 5, Sonnet 5.5, and Haiku 5.5; 1,024 for Opus 4.8, Sonnet 5, and Sonnet 4.6; 2,048 for Opus 4.7; and 4,096 for Opus 4.6, Opus 4.5, and Haiku 4.5. OpenAI’s minimum is 1,024 tokens on GPT-5.6 and later.
Fall short and you won’t get an error. The request just runs uncached. The only way to know is to check cache_creation_input_tokens and cache_read_input_tokens in the response’s usage fields. If both are zero, nothing was cached.
When conversations get long
For multi-turn chatbots, Anthropic recommends starting with automatic caching, since it moves the breakpoint forward as the chat grows. The key is to leave earlier turns alone. OpenAI’s guide says to append new messages rather than rewrite old ones, and warns that summarization or compaction can change the prefix and reset cache reuse, though it adds that compaction can still come out cheaper overall if it cuts enough input.
Agents that add lots of blocks per turn have one more problem. Anthropic looks back at most 20 blocks from a breakpoint to find an earlier cache entry, so if the last write gets pushed outside that window, you miss the cache. The docs suggest adding a second breakpoint near that spot ahead of time.
Bunch up rare jobs, and warm up the first request
For jobs that only run a few times a day, send related requests close together instead of spreading them out. By our math, twenty requests that share a prefix and arrive within five minutes cost 1.25 + 19 × 0.1 = 3.15x for that shared prefix, versus 20x uncached, about 84% off. That only covers the shared part, so your total bill drops by less.
If the first request is the slow one, Anthropic documents cache pre-warming: set max_tokens to 0, put the breakpoint on your system prompt, add a short placeholder user message, and send it before real users arrive. The API writes the cache and returns right away without generating anything. You pay no output tokens, but the cache write is billed as usual.
On OpenAI, consider choosing your own breakpoints
On GPT-5.6 and later, OpenAI places a breakpoint at the end of the latest eligible message by default, which means each new question gets written at 1.25x. Set prompt_cache_options.mode to explicit and only the breakpoints you mark get written. Anything after the last one is billed at the normal input rate with no write charge, so you stop paying a surcharge on questions that will never be reused.
Is your cache actually saving money?
It all comes down to one question: does the same prefix get read again before the cache expires? If you’re not sure, leave caching on for a day, then add up cache_creation_input_tokens (writes) and cache_read_input_tokens (reads) from the usage fields of your responses. The ratio between those two numbers tells you whether you’re ahead.
Here’s the logic. Every token you write to the cache costs 0.25 more than normal input, and every token you read from it costs 0.9 less. To pay back the 25 extra units from 100 written tokens, you need 25 / 0.9, or about 28 read tokens. So with a 5-minute cache and 0.1x reads, your read tokens need to be more than roughly 28% of your write tokens before caching saves anything. (That’s our own math.)
Real prices make it clearer. Using Anthropic’s listed rates for Claude Sonnet 4.6 ($3 per million input tokens, $3.75 for 5-minute cache writes, $0.30 for cache reads), here’s a service that writes 1 million cached tokens a day, under three different read volumes:
| Daily usage | With caching | Without caching | Result |
|---|---|---|---|
| 1M written, 200K read | $3.75 + $0.06 = $3.81 | 1.2M × $3 = $3.60 | $0.21 worse |
| 1M written, 500K read | $3.75 + $0.15 = $3.90 | 1.5M × $3 = $4.50 | $0.60 better |
| 1M written, 9M read | $3.75 + $2.70 = $6.45 | 10M × $3 = $30.00 | $23.55 better |
In the first row, reads are 20% of writes, below the 28% line, so you’d have been better off with caching off, even though the cache did get used. The second row clears the line at 50%. And in the third, where each cached prefix gets read nine times, your input cost drops to about a fifth.
Even with a good setup, some things stay out of your hands. There’s no way to manually clear the cache (per Anthropic’s docs), and a live cache only helps if your request reaches a machine that holds it (per OpenAI’s guide). That’s why it’s worth checking the numbers instead of trusting the design. Anthropic also has a cache diagnostics feature that compares consecutive requests and tells you where the prefix diverged.
So if your daily reads come in under 28% of your writes, caching is costing you money, not saving it. Go back to how your prompt is stacked and check it piece by piece.