Concept · Building with AI
Prefix cache
Also called: prompt caching, prompt cache, context caching, caching prompts, prefix caching
A provider feature that remembers the opening of a request it has just handled. If your next request starts with the same text, the provider skips reworking it, bills that part at a discount and replies faster. Keeping the start of each request the same, such as the system prompt, is what makes it pay off.
For example: A support bot begins every request with the same long policy manual. With caching, it pays full price for the manual once and a discounted rate on later requests.
Connects to 7
- Cache tokens Concept
- Input tokens Concept
- System prompt Concept
- Model provider Concept
- Model provider request Concept
- Usage-based pricing Concept
- Startup ecosystem Concept