Pillar · Token Economics
One observed session burned 8.59M tokens: 90% re-reading files. Memory is how you stop paying that tax every session.
Direct answer
To reduce AI coding token usage, attack the three costs that dominate every session: stale conversation history (re-billed every turn), whole-file reads (tens of thousands of tokens for one-line answers), and re-learning project knowledge from zero each session. The canonical waste case is an observed 8.59M-token session: 97.2% cache, ~90% re-reading, 175:1 input-to-output. Anvaya’s answer is compounding memory: session outcomes become typed nodes (files ~20x, decisions ~16x, bugfixes ~24x, threads ~25x, per-node design estimates), and turns target 2–4K packed tokens instead of ~50K raw. What is measured today: micro-fold cut median uncached tokens −11.0% with verification 19/19 (n=5), and headless fix tasks landed −11%…−21% across 24/24 verified pairs. Prompt caching covers the stable layers at ~10% price; Phase 2 measures session-to-session compounding.
Per-Node Ratios
Design estimates per memory node type from Anvaya's memory design work, not benchmark results. The measured end-to-end lever is listed under Benchmarks.
Three Levers
Works with any agent, including Claude Code and Cursor, no new tools required.
Questions
Three levers dominate: split work into focused sessions (stale history is re-paid every turn), retrieve just-in-time instead of dumping files (spans, not whole files), and persist durable knowledge in a memory layer so sessions stop re-learning. Anvaya adds the compounding lever: calibrated memory that shrinks per-turn context from ~50K raw to 2–4K packed.
Per-node design estimates from Anvaya Mind's design work: each memory node type stores a fraction of the raw tokens that produced it (files ~20x, decisions ~16x, bugfixes ~24x, patterns ~24x, threads ~25x). A turn targets 2–4K curated tokens instead of ~50K raw, the 12–25x range. These are engineering estimates, not benchmark results. The measured end-to-end lever today is micro-fold at −11.0% median uncached tokens (n=5, verification 19/19); Phase 2 of the benchmark program measures session-level compounding.
Caching cuts the price of repeated tokens ~90% but doesn't shrink what the model must attend to, noise still dilutes attention and degrades answers. Cache the stable layers (system prompt, conventions) and engineer the dynamic layers (retrieval, memory). Anvaya does both: cacheable system context plus packed per-turn memory.
Every token loaded is re-billed on every subsequent turn until compaction. A 10K file read at turn 3 costs another 10K at turns 4, 5, 6… A 30-message thread carries 29 dead messages into every call. Early-session discipline compounds across the whole session.
At 2026 Claude pricing (~$2–5 per million input tokens), an 8.59M-token session is roughly $17–43 of input, ~90% of it re-reading files. Multiply by sessions per week per developer and the memory layer pays for itself in attention alone, before counting accuracy.
25.6 MB median RSS. 25 agents ran in parallel on a Core 2 Duo with 4 GB RAM. Hundreds on your machine. Zero cloud required on the Ollama path.
Requires Rust/cargo to build from source. Linux and macOS today, Windows not yet supported. Pre-1.0, public beta. Pricing TBD.