Q
What should I put in cached prefix?
System prompt, project conventions, stable reference docs — anything identical across turns. Keep it byte-stable; edits bust the cache.
Glossary · Prompt Caching
Prompt Caching.
Pay full price once for stable context.
Direct answer
Prompt caching stores the stable prefix of a context (system prompt, conventions, reference docs) so repeat turns serve it at ~10% of fresh-token price on Claude-class APIs. It never shrinks a single request — but across long sessions the 90% saving compounds hard, since stable layers re-bill every turn. Rule: keep unchanging context byte-stable and first, so cache hits; engineer the dynamic layers (retrieval, memory) separately.
In Anvaya
How We Implement It.
01Prompt CachingAnvaya's SystemPromptCache plus Mind-served stable context is designed cache-first: conventions and standing memory hit cache, per-turn packed memory stays small and fresh.
Questions
Asked About Prompt Caching.
Q
Does caching fix context rot?
No — cached noise still dilutes attention. Caching cuts price; curation protects quality. Do both.
Q
How big are the savings?
Cached input ~10% of fresh price. In sessions with 20+ turns and large stable prefixes, totals drop dramatically.
Stop Starting From Zero.
One binary. 11+9 Rust crates. 545 tests. Hand-written HNSW index. Three transport modes. Four providers, Ollama, Anthropic, OpenAI, Siemens. Zero API keys required to start. Mind remembers everything after the first session.