Q
What should I put in cached prefix?
System prompt, project conventions, stable reference docs, anything identical across turns. Keep it byte-stable; edits bust the cache.
Glossary · Prompt Caching
Prompt Caching.
Pay full price once for stable context.
Direct answer
Prompt caching stores the stable prefix of a context (system prompt, conventions, reference docs) so repeat turns serve it at ~10% of fresh-token price on Claude-class APIs. It never shrinks a single request, but across long sessions the 90% saving compounds hard, since stable layers re-bill every turn. Rule: keep unchanging context byte-stable and first, so cache hits; engineer the dynamic layers (retrieval, memory) separately.
In Anvaya
How We Implement It.
01Prompt CachingAnvaya's SystemPromptCache plus Mind-served stable context is designed cache-first: conventions and standing memory hit cache, per-turn packed memory stays small and fresh.
Questions
Asked About Prompt Caching.
Q
Does caching fix context rot?
No, cached noise still dilutes attention. Caching cuts price; curation protects quality. Do both.
Q
How big are the savings?
Cached input ~10% of fresh price. In sessions with 20+ turns and large stable prefixes, totals drop dramatically.
Run Agents That Fit On Your Laptop.
25.6 MB median RSS. 25 agents ran in parallel on a Core 2 Duo with 4 GB RAM. Hundreds on your machine. Zero cloud required on the Ollama path.
Requires Rust/cargo to build from source. Linux and macOS today, Windows not yet supported. Pre-1.0, public beta. Pricing TBD.