Quick answer
Prompt caching cuts AI API costs by charging much less for the part of your prompt that repeats between requests. Put the fixed parts (instructions, examples, reference documents) at the start and keep them identical every time, then add the changing part (the new question or customer message) at the end. With OpenAI’s GPT-6.2 Astra, cached input costs $0.40 per million tokens instead of $8.
Key takeaways
- Caching rewards repeated prompt starts. The matching part has to be exactly the same.
- Fixed first, changing last: system instructions, examples and documents at the top; the new request at the bottom.
- For chats, append new instructions to the conversation rather than editing earlier messages.
- Small edits at the top, such as a timestamp or a changing name, can break the cache for everything after them.
- Check your usage data for cached tokens to confirm it is working.
This guide supports our main article, GPT-6.2 Astra review: cheaper and efficient, but is it worth switching? When OpenAI launched GPT-6.2 Astra and Sol, it cut the cached input price by 60% compared with the previous Astra and recommended keeping prompt beginnings unchanged. If you run AI automations, chatbots or content tools through an API, this is one of the simplest ways to lower your bill.
📘 Main article: For Astra’s full pricing, benchmarks and tester reactions, read our GPT-6.2 Astra review.
What is prompt caching?
Every time you send a request to an AI model through its API, you pay for the input tokens (your prompt) and the output tokens (the answer). Many automations send the same long instructions every time, such as a 2,000-word style guide or a product catalogue, with only a short new question at the end.
Prompt caching means the provider remembers the start of a prompt it has seen recently. When the next request starts with exactly the same text, that repeated part is billed at a much lower cached input rate. Providers such as OpenAI, Anthropic and Google all offer a form of this, though the details and discounts differ.
How much can prompt caching save?
Here are OpenAI’s published GPT-6.2 rates per million tokens:
| Model | Normal input | Cached input | Output |
|---|---|---|---|
| GPT-6.2 Astra | $8 | $0.40 | $40 |
| GPT-6.2 Sol | $2 | $0.10 | $10 |
A simple illustration with made-up numbers: an automation sends a 10,000-token fixed brief plus a 500-token question, 1,000 times a month on Astra. The fixed brief alone is 10 million input tokens. Uncached, that part costs about $80. If it is cached, it costs about $4. Your real saving depends on how often the cache is hit and on output costs, which caching does not reduce.
💡 Tip: Output tokens are often the bigger cost. Pair caching with clear length limits on answers to keep both sides of the bill down.
How do I use prompt caching to cut AI API costs?
- List what repeats. Write down the parts of your prompt that are the same every time: role and rules, tone of voice, examples, reference text.
- Put the fixed parts first. Order your prompt as: instructions → examples → reference documents → the new request.
- Freeze the fixed text. Store it in one file or template and do not tweak it per request. Even changing a space or a date can break the match.
- Put changing details last. Customer names, today’s date, the new question and any per-request data go at the end.
- Append in conversations. In multi-turn chats, add new instructions as a new message instead of rewriting earlier ones, as OpenAI recommends for GPT-6.2.
- Batch similar jobs together. Caches usually expire after a period of inactivity, so running similar requests close together helps.
- Check the provider’s docs for minimum prompt lengths and how long caches last, because these vary.
Not sure which model to pay for? Before optimising costs, make sure you are on the right model. Our guide on routing simple tasks to cheaper AI models shows how to split work sensibly.
What breaks the cache?
| Mistake | Why it breaks caching | Fix |
|---|---|---|
| Date or time at the top of the prompt | The start changes every request | Move it to the end |
| Customer name in the system instructions | Every customer gets a different prefix | Put personal details in the final message |
| Editing the instructions often | Each version is a new prefix | Version your template and change it deliberately |
| Shuffling examples or documents | Order matters for an exact match | Keep a fixed order |
| Rewriting earlier chat messages | The conversation start no longer matches | Append new instructions instead |
⚠️ Warning: Do not put private customer data in the fixed, cached part of a prompt just to save money. Keep personal details in the changing section and follow your provider’s data policies.
How do I check caching is working?
Most APIs report cached tokens in each response’s usage details and in the billing dashboard. Run the same prompt twice within a few minutes and compare: the second request should show a large number of cached input tokens. If it shows none, look for anything changing near the top of your prompt.
If you are also choosing between models, our main article on GPT-6.2 Astra shows why cost per task matters more than price per token, and our guide to comparing AI models on token cost helps you work it out.
Frequently asked questions
Does prompt caching change the answers?
No. It only changes how the repeated input is billed and how quickly it is processed. The model still reads the full prompt.
Do I need to switch it on?
It depends on the provider. Some apply caching automatically for long enough prompts; others need you to mark which parts to cache. Check your provider’s documentation.
Does caching work in the ChatGPT app?
Caching discounts apply to API billing. In the ChatGPT app you pay a subscription, though efficient prompts can still help your usage limits.
How much is cached input on GPT-6.2 Astra?
OpenAI lists $0.40 per million cached input tokens for Astra, against $8 for normal input. GPT-6.2 Sol is $0.10 against $2.
Related posts
- GPT-6.2 Astra Review: Cheaper and Efficient, but Is It Worth Switching?
- How to Compare AI Models on Token Cost, Not Just Output Quality
- How to Route Simple Tasks to Cheaper AI Models Without Losing Quality
- How to Make Your AI Coding Plan’s Usage Limits Last All Week
