Claude Code Costs, Act III — The ecosystem of options for spending less

There is a whole open-source ecosystem aimed at cutting LLM cost. The trick to evaluating any of it is to ask which of the three cost lines it attacks — cached input (0.1×), uncached/written input (1×/2×), or output (the priciest, never cached) — and then whether it does so without forfeiting the model-scoped prompt cache.

That last clause is the recurring catch, and it’s the throughline of this entire guide:

Anything that rewrites the request prefix per turn forfeits the model-scoped prompt cache. A tool nets real savings only if it (a) preserves the cached prefix byte-for-byte, or (b) attacks a cost line the cache doesn’t cover — namely output.

We’ll walk the ecosystem in cost-line order: input/cache compressors, then the output compressor, then the model routers, then what routing actually costs inside Claude Code.

Input / cache-line compressors

Headroom — compressing the input / cache-write line

Headroom (headroomlabs-ai/headroom, formerly chopratejas/headroom; Apache-2.0) is a local-first compression library + proxy + MCP for coding agents, and the most deeply-engineered tool in this category. (It’s also brand-new and fast-moving — created 2026-01, dozens of commits a day — so treat specifics as a moving target.) Three things make it notable:

1. Content-type-aware routing — it picks a compressor per content type (table below) rather than compressing blindly.
2. Reversibility — it stores the originals, and the model retrieves the full text on demand, so compression doesn’t lose information.
3. A CacheAligner — it pulls volatile content (dates, IDs) out of the prefix to keep it stable across calls (one half of composing compression with the cache; see below).

Self-reported results: 47–92% token savings with accuracy held (GSM8K ±0, SQuAD 97% at 19% compression, BFCL tools 97% at 32%) [vendor self-reported]. Independent measurement is far lower: the one rigorous third-party benchmark found ~10% on-wire savings, matching Headroom’s own published fleet median of 4.8% — an order of magnitude below the headline. [independent benchmark, 2026-06]

Headroom’s content-type routing [med]:

Content Compressor Savings Preserves JSON arrays SmartCrusher 70–90% keys, UUIDs, booleans Search results SearchCompressor 80–95% matched lines Build/test logs LogCompressor 85–95% errors, stack traces Source code CodeCompressor (AST) 40–70% signatures, imports Diffs / HTML / text Diff/HTML/Text 50–80% high-entropy tokens Images trained router (MiniLM/SigLIP) 40–90% —

The image path routes through a **trained MiniLM/SigLIP classifier* (chopratejas/technique-router); HTML is handled by extraction (trafilatura — it drops irrelevant blocks, not tokens), not a lossy crusher.*

Two surfaces — where it compresses

Headroom works on two surfaces, and they stand in opposite relationships to the cache — so it’s worth separating them (the table’s ratios apply to only one of them). They’re two insertion points for the same compressors, picked by how you deploy Headroom — not two passes that both run. Integrate it (MCP/SDK/hooks) and tool outputs shrink at the source; just proxy it (ANTHROPIC_BASE_URL) and they’re caught later, on the wire. You normally get one, not both.

• The tool-result surface — via the MCP server, SDK, or hooks (this is what the table measures). When a tool produces a big blob — a 60 KB search result, a noisy build log, a giant JSON array — Headroom compresses that blob before it is ever appended to the transcript. The agent keeps the original and re-sends only the shrunk form, so the model sees the small version and the prompt cache forms around it. Safe by construction: the bytes shrink before any cache exists, so there’s no cached prefix to bust. This is where Headroom’s own docs scope the headline — “70–90% **on tool outputs.”
• The proxy surface — the /v1/messages body, when you point ANTHROPIC_BASE_URL at Headroom. Here the design is more delicate, and it’s the catch-point when there was no source-level integration. Headroom parses each request and finds compressible content in the live tail (the newest messages that aren’t cached yet — typically a large tool result that nothing shrank at the source, e.g. a Bash, WebFetch, or MCP output), runs it through the same compressors, and swaps in the shrunk form plus a retrievable marker (CCR — see the machinery section). The one thing it is built not to do is rewrite the frozen, already-cached prefix: those bytes have to stay byte-identical or the 0.1× cache discount is lost. So the intent is simple — shrink the fresh tail, never the cached prefix.

When the proxy surface actually pays off — and why that’s rarely Claude Code

The proxy path pays off only when there genuinely is a large, compressible blob sitting in the live tail. That’s common for raw-API clients and other agents that dump big tool outputs straight into the request — point them at Headroom and it trims the tail on the way out. A real Claude Code session is the case where it pays off least — measured live, it compressed 0% of the conversation — for two structural reasons:

1. Claude Code freezes its tool results into the cached prefix. CC places its cache_control breakpoints over recent activity, so by the time a tool result could be a compression candidate it’s already inside the frozen prefix — and Headroom, by design, won’t touch frozen bytes (doing so would bust the cache, as the next section shows). The block is forwarded byte-identical.
2. Claude Code already offloads big outputs itself. CC 2.1.150 saves any large tool output to a disk file and sends only a ~2 KB preview into the body (its <persisted-output> mechanism), so there’s rarely a big blob left in the conversation for a proxy to compress at all.

So behind Claude Code the conversation body is forwarded effectively unchanged, and Headroom’s real savings come from the tool-result surface above: route verbose tool output through its MCP server / hooks and you get the table’s ratios; point ANTHROPIC_BASE_URL at it and expect little from the conversation itself. (A separate standalone Rust proxy in the same repo goes further — it defaults to true byte-faithful passthrough, compression_mode = Off — but the headroom CLI never launches it.) The modest ~10% / 4.8% independent numbers above fit this picture: most of the real ratio lives on tool outputs, and the cache-busting moves that would chase more were deliberately fenced off — because, as the next section shows, getting proxy compression wrong costs far more than it saves. [measured, 2026-06]

Composing compression with the cache

Compression and caching are two separate discounts on the token bill, and the question is whether they stack or cancel. Caching is a 90%-off coupon on bytes the server has already seen — a cached prefix is billed at 0.1× (a “cache read”) instead of full price. Compression just means send fewer bytes. Whether they compose (both apply) or fight (one voids the other) depends entirely on which bytes you compress:

What you do Cost of a 100K-token cached prefix Fight compress the whole prefix 100K → 20K, rewriting the cached bytes those 20K are now new bytes → cache miss → billed as a 2× write ≈ 40K-equiv. You shrank the text but voided the coupon, and the bill went up (it was 10K-equiv as a read). Compose leave the cached span byte-identical, compress only the fresh tail the 100K stays a 0.1× read ≈ 10K-equiv and the new tail is smaller — both discounts apply.

So “composing compression with the cache” means arranging things so compression shrinks tokens and the cache discount survives. It takes two separate things, and only one of them is the CacheAligner:

1. A stable prefix — no volatile content (dates, IDs) drifting the cached bytes each call. This is the CacheAligner‘s entire job: it moves volatile content to the message tail. Claude Code already does exactly this — the date sits after the system breakpoints and the billing-header token is excluded from the cache key (Act I) — so the CacheAligner adds little behind CC. It earns its keep for hand-rolled API clients that would otherwise leave a timestamp or session ID at the top of the system prompt. (In current Headroom the CacheAligner actually ships disabled by default and reduced to a detector — its rewrite path was removed after it was found to mutate the cache hot zone, the very thing it was meant to protect.)
2. Cache-safe compression — leave the cached span byte-identical and shrink only the fresh tail; rewrite cached bytes and you trade a 0.1× read for a 2× write (the Act II flip). This is a different mechanism from the CacheAligner, and behind Claude Code it’s the half that actually matters: get it wrong and the compressor costs more than it saves — usually more than running no compressor at all, since a multi-turn CC session’s cached prefix dominates the bill. It’s exactly the bug list below.

The cache-killer trap — serialization drift

The cache is a byte-exact longest-prefix match (Act I): the API extends a cached entry only while the new request’s leading bytes are identical, and re-processes everything from the first differing byte onward. Headroom runs as a local proxy in that path, between Claude Code and the API — and the hazard of sitting there is that it can change those bytes without changing the meaning, which is enough to miss.

To see how, follow one request through it. There are three nodes:

[A] Claude Code  ──raw bytes──▶  [B] Headroom (proxy)  ──re-emitted bytes──▶  [C] Anthropic API
   (the client)                  (logs / compresses)                          (server + cache)

Enter fullscreen mode Exit fullscreen mode

A — Claude Code builds the request. It serializes the whole request to JSON bytes (model, tools, system, messages), writing compact JSON — no space after : or , — and raw UTF-8 for non-ASCII. A content field reaches Headroom as these literal bytes:

{"role":"user","content":"café"}

Enter fullscreen mode Exit fullscreen mode

B — a naive proxy re-emits the request. To compress a tool output (or merely to log the body), the natural code parses the incoming bytes into an object, edits its one field, and serializes them back — and a stock encoder is not Claude Code’s. Python’s default json.dumps() re-emits the same object as:

{"role": "user", "content": "caf\u00e9"}

Enter fullscreen mode Exit fullscreen mode

Two unrequested changes: a space after every : and ,, and é becomes the 6-character escape \u00e9. Same object, two valid byte encodings — the stock encoder just picked a different one than Claude Code did. Headroom calls this serialization drift, and its REALIGNMENT/01-bug-list.md is a 72-item P0–P6 self-audit of every way it can occur — each entry matching a real Anthropic prefix-cache rule. (Headroom has since fixed this in the default path — see What Headroom actually ships below.)

C — Anthropic looks up the cache. Get this step wrong and the proxy flips from saving money to costing it — a net expense, not a saving. The cache keys on the bytes the server receives — Headroom’s output — and never sees Claude Code’s originals at all. So each turn the server matches this request’s prefix against the bytes Headroom itself sent, and the server cached, on the previous turn — not against anything Claude Code produced. If Headroom re-serializes byte-identically every turn, those match and you still hit; the cache simply keys on Headroom’s encoding instead of Claude Code’s. The bill blows up only when Headroom’s output is not byte-stable from one turn to the next — and naive re-serialization makes that the default: json.dumps() without sort_keys, set/dict iteration order, drifting float formatting, or byte-copying some messages while re-serializing others. Any of those changes a byte near the top of the request, which re-keys the whole prefix after it → a cold write at 2× instead of a cache_read at 0.1×. (You also pay a one-time cold rebuild the moment Headroom is dropped in front of a cache Claude Code had already warmed with compact bytes — its spaced output matches none of those entries.)

What that costs:

• Proxied traffic collapses toward 100% cold writes (0% hit) — and not just the message that was touched: the drifted byte sits near the top, so it re-keys everything after it.
• 10–20× the price for the prefix on every turn (0.1× read → 1–2× write), for content the server already had cached. A proxy meant to save money can cost more than it saves.
• It is silent — the request is valid, so HTTP 200 and correct responses, no error. The only evidence is usage: cache_read stuck near 0 and cache_creation high every turn. You find it in the bill, not a stack trace.

Each row is a way a re-serializing proxy’s output drifts across turns — and each is a real entry in Headroom’s own 72-item audit, so you can look it up. The Headroom bug column gives the audit ID (P0 = its “cache-killer smoking guns — every customer affected” class, the label from when the audit was written):

Drift cause What the proxy did What changed in the bytes Headroom bug Separator / escaping drift re-encoded with default json.dumps() ,→,, :→:, raw UTF-8 → \uXXXX P0-2 (also P0-1 — system prompt .strip() + memory append) Key-order drift json.dumps() without sort_keys=True, or iterated a set/dict object keys come out in a different order request-to-request P3-28 (tool array), P3-29 (schema keys) Numeric-precision drift round-tripped a number through a generic JSON value 1.0→1; integers above 2⁵³ lose low digits P0-5 (also P4-46 — missing serde_json arbitrary_precision/raw_value) Re-emitting unchanged history re-serialized retained messages during compression instead of byte-copying them even “untouched” history shifts byte-for-byte P1-13 (also P0-3 ignored cache_control; P0-4 ICM dropped hot-zone messages)

All four are the silent invalidators from Act I, now seen from the proxy author’s chair. They collapse to one rule: forward the original request bytes verbatim. If you must mutate, do surgical byte-fragment replacement on the messages tail only — never parse-and-re-emit the whole envelope — and confirm cache_read is unchanged on the next request. A compressor that ignores this loses far more to cold rewrites than it ever saves in tokens.

What Headroom actually ships (v0.27.0). It now does exactly what the ✅ Fix below prescribes — which is why the live measurement earlier showed it forwarding Claude Code’s bytes unchanged. Its forwarder defaults to byte-faithful: an unmutated body is forwarded verbatim (the original bytes), and a mutated one is re-serialized once, canonically, to compact UTF-8 that matches Claude Code’s own encoding (no added spaces; non-ASCII stays raw, not escaped) — so the headline drift (the café case, audit item P0-2) is gone by default. The naive json.dumps path survives only as an explicit emergency rollback (HEADROOM_PROXY_PYTHON_FORWARDER_MODE=legacy_json_kwarg), pinned by a regression test (test_proxy_byte_faithful_forwarding.py). So read the 72-item audit as a catalog of ways a re-serializing proxy bites itself: the worst (P0-2) is fixed by the byte-faithful default; the rest are assigned fixes and worked through release by release. It’s the cleanest proof the hazard is real and worth engineering against.

⚠️ Mistake — adding a “compression” or “logging” proxy that re-serializes JSON. A re-encode stays cache-safe only if it reproduces the exact same bytes on every turn — and naive code never does (separators, escaping, key order, float formatting, or mixing byte-copied and re-serialized messages all drift). In practice the hit rate falls to ~0 for all proxied traffic. This is the most expensive proxy mistake there is.

✅ Fix — Forward the original request bytes verbatim. Mutate only by surgical byte-fragment replacement on the messages tail, never by parse-and-re-emit of the whole envelope. Verify with cache_read after the proxy is inserted.

Two more ways a proxy quietly costs you

Independent of compression, just from pointing ANTHROPIC_BASE_URL at a custom host:

• Tool schemas get inlined. Claude Code can’t assume a custom endpoint supports tool-search deferral, so it eagerly materializes every tool schema into the context window — the proxy you added to save tokens silently adds thousands. Fix: ENABLE_TOOL_SEARCH=true (or headroom wrap claude, which sets it). [Headroom troubleshooting docs + issue #753]
• The 1M-context beta header gets dropped. Behind a custom base URL, Claude Code can drop the context-1m-2025-08-07 beta and silently cap you at 200K context; the workaround is the [1m] model suffix (e.g. claude-opus-4-8[1m]). [issue #1158]

ReadLifecycleManager — collapsing stale and superseded reads

The biggest single source of waste in a coding session is old file reads piling up in replayed history, and this is the transform built to attack it. It’s worth a closer look — partly because it clears up a common misconception.

A block’s bytes never change — its relevance does. A tool_result is immutable once produced: the 400 lines of worker.py you read at step 1 are those bytes forever. What ReadLifecycleManager reacts to is a later message that makes the earlier read obsolete:

• STALE — a later Edit/Write modified the same file after it was read, so the old read no longer matches what’s on disk.
• SUPERSEDED — a later Read of the same file, so the earlier read is redundant; only the newest copy matters.

So it isn’t “a block’s content changed.” It’s “block #3 became dead weight because block #8 came along” — the manager rewrites the old block based on newer blocks further down the history. (Its own notes put roughly two-thirds of Read-output bytes going stale and another eighth superseded — about three-quarters dead weight that’s still re-sent every turn.)

Why this happens constantly: Claude sends many tool calls per turn. One thing you type is not one API request. Claude Code runs an agentic loop — call a tool, read the result, call another — often 10–50 round trips before the final answer, and each tool_use → tool_result round trip is a separate POST /v1/messages that replays the entire conversation so far. By request #20 the message array holds all 19 prior reads, edits, and bash outputs stacked up — which is exactly where stale and superseded reads accumulate. (When the sections above said “the live tail,” this loop is what fills it.)

Worked example. You type one prompt:

“Rename process() to handle() in src/worker.py and fix all the callers.”

Claude’s loop — each row is a separate API request that replays everything above it:

Step Tool call What lands in history 1 Read src/worker.py block A: full 400-line file (~1,500 tok) 2 Grep "process\(" 12 call sites 3 Edit src/worker.py (the def) edit confirmation — file on disk now changed 4 Edit src/worker.py (a caller) another edit 5 Read src/worker.py (re-read to verify) block E: updated 400-line file (~1,500 tok) 6 Bash: pytest test output 7 … final answer …

By the request for step 6, the replayed history contains both block A and block E — the same file, read twice. Block A is now STALE (the file was edited at steps 3–4, after it was read) — and also SUPERSEDED (re-read at step 5). Either way it’s dead weight, since block E is the current truth. The manager collapses block A to a one-liner via the stale path and stores the original for retrieval:

[Read content stale: src/worker.py was modified after this read — re-read the file for current content. Retrieve original: hash=ab12…]

Enter fullscreen mode Exit fullscreen mode

~1,500 tokens → ~20 — and because block A would otherwise ride along in every remaining request of the loop (steps 6, 7, …), you don’t save it once, you save it on every remaining round trip.

Why “stale,” not “superseded”? Headroom ships stale collapse on by default but superseded collapse off — collapsing a pure re-read (one with no intervening edit) is riskier, because the earlier copy usually sits in the cached prefix and rewriting it would bust the cache. So a re-read alone is left in place by default; it’s the edit that makes block A provably safe to drop. (One more clever lever shipped deliberately conservative for cache safety — see the pattern below.)

When the collapse actually fires (the cache tie-in). It can only rewrite block A while A is both obsolete and still in the mutable tail. Because the cached prefix grows as the loop runs, two regimes appear — and this is why so little of it shows up behind Claude Code:

• A is still in the live tail → collapsed on the spot.
• A is already in the frozen prefix → skipped, to protect the cache (rewriting it would trade a 0.1× read for a 2× write). It’s reclaimed only once that part of the cache turns over — and since Claude Code rides a 1-hour TTL that refreshes on every read (Act I), in practice that means a new session or a long idle gap, not mid-loop.

So the canonical trigger is the everyday read → edit → re-read loop Claude runs constantly — but, like every proxy-surface lever here, behind Claude Code it’s gated by the same frozen-prefix reality from the disclaimer above, so it mostly fires across turns rather than within one.

Inside the proxy — the rest of the machinery

You don’t need any of this to use Headroom, but it explains what you’re running and where the cost levers really are. Most of it is built for cases other than a live Claude Code conversation (one-shot calls, MCP tool output, other agents) — which is exactly why so little of it fires on the wire behind CC.

• proxy vs wrap — same compression, different wiring. Both run the same server with the same compression defaults. headroom proxy is just the compression layer — you point Claude Code at it. headroom wrap claude adds the integration: it sets ENABLE_TOOL_SEARCH=true, writes .claude/settings.local.json, downloads and wires RTK, adds a project header, and gives you a clean unwrap. A common misconception is that “wrap compresses harder.” It doesn’t — wrap is convenience + RTK; the compression aggressiveness is identical, and the aggressive agent-90 savings profile is opt-in in both. So the headline “60–95%” is a ceiling you choose, not out-of-the-box CC behavior.
• Subscription vs PAYG — what “saving” even means. Headroom classifies the client by User-Agent: Claude Code (claude-code/, claude-cli/) is treated as a subscription and gets the conservative policy; an API/PAYG client gets the aggressive one. This matters because on a subscription (Pro/Max) the bill is flat — token savings don’t reduce a per-token charge, they buy context headroom (fewer auto-compactions, longer sessions) and help you stay under the 5-hour / weekly caps. On PAYG / Bedrock / Vertex, the same savings are real dollars. Same tool, two different definitions of “win.”
• The break-even math — net_mutation_gain. Headroom has a formula for when it’s worth rewriting cached content: gain = ΔT·(w + r·(R−1)) − P_alive·(w−r)·(S+ΔT), mutate only if gain > 0 (ΔT = tokens removed, S = cached suffix below the edit, R = expected remaining reads, P_alive = chance the cache is still warm, w = 1.25 write, r = 0.1 read). The intuition is the whole Act II flip in one line: editing a message re-writes the entire cached suffix beneath it, so shaving 2K off a message sitting on a 50K cached suffix needs ~288 future reads to break even (never worth it), while shaving 50K above a 10K suffix breaks even in ~2.3 reads (worth it now). Crucially, this gate ships off by default and — together with a frozen-prefix guard — is why Headroom leaves your cached prefix alone.
• A real gap: the 1-hour-TTL write price. That formula hardcodes the write multiplier at 1.25 — the 5-minute TTL price. There is no branch that reads a request’s cache_control.ttl and switches it to 2.0 for a 1-hour cache. If a session uses the 1h tier, the model understates the re-write penalty, biasing any cache-break decision toward “too eager.” Mostly latent today (the gate is off), but a genuine correctness gap worth knowing if you ever enable aggressive mode.
• What’s protected from compression. Exact-output tools (Bash, WebFetch) are kept verbatim — lossy compression there would corrupt meaning — and the structural tools (Read/Glob/Grep/Write/Edit) are excluded from blind crushing. Only “safe” content types (JSON arrays, logs, search results) get lossy treatment.
• Reversibility (CCR), concretely. When a compressor drops content it caches the original locally, keyed by a SHA-256 hash (24 hex chars), and leaves a marker like [201 items compressed to 20. Retrieve more: hash=fa88…]. If the model needs the detail it calls the headroom_retrieve MCP tool (BM25-filterable). Default TTL ~30 min; the retrieve/compress endpoints are loopback-only, so dropped data can’t be reached off-box. This is why “same answers, fraction of the tokens” is defensible — the data is offloaded, not gone.
• Levers that cut turns/output, not just bytes. headroom learn mines ~/.claude/projects/*.jsonl for recurring error→recovery patterns (wrong path → right path) and writes a correction into CLAUDE.local.md so the next session gets it right the first time — fewer wasted turns. A verbosity shaper appends a deterministic, idempotent terseness instruction (the same output line Caveman attacks, done at the proxy). RTK (rtk-ai/rtk) filters/compacts verbose shell-command output at the source — wrap-only, installed as a Claude Code PreToolUse hook, downloaded on demand (not bundled; details under Other compressors below). And a cross-agent shared-context store dedups the handoff payload when work moves Claude → Codex → Gemini.
• How to check it’s actually saving (don’t trust, verify). The proxy logs a PERF line per request (tokens_before / tokens_after / tokens_saved + cache metrics); the analyzer prices saved tokens at the cache-read rate when the cache was hit, so it won’t overstate savings on a cached prefix. headroom perf / headroom gain report measured % against the target. That the accounting is itself cache-aware is the clearest sign the tool respects the one rule that governs this whole act.

Other input/cache & hosted compressors

Same cost line as Headroom (input / cache-write), shipped as point tools:

• Microsoft LLMLingua (microsoft/LLMLingua) — prompt compression up to 20× at ~1.5pt GSM8K loss; LongLLMLingua improves RAG retrieval up to 21.4% using ¼ the tokens. ⚠️ Degrades past ~20–25× and busts the prefix cache unless the prefix is re-stabilized afterward. Best for one-shot / RAG context, not a cached conversational prefix. [high]
• RTK (“Rust Token Killer”, rtk-ai/rtk) — a standalone Rust CLI that wraps a command and filters/compacts its output before it reaches the model (rtk ls / git / test / diff / grep / pytest …, 50+ subcommands). Savings are real but uneven — measured ~60% on ls, ~85% bytes on git log (one line per commit), but 0% when no filter matches (it passes through). Headroom downloads it on demand (pinned v0.42.4, over TLS, no checksum) and installs it wrap-only as a Claude Code PreToolUse hook (rtk init), so it sits outside the /v1/messages proxy path — it shrinks tool output at the source, the productized version of the preprocessing-hook technique. (It’s the same rtk you’d install standalone; savings are tracked via rtk gain, and it only auto-applies once the hook is installed.) Good when tool outputs are the bloat. [med — savings partly measured 2026-06]
• lean-ctx (yvgude/lean-ctx) — a CLI/MCP context trimmer. [med]

Hosted & provider-native (text leaves your box; non-reversible):

• OpenAI Compaction — provider-native conversation-history summarization. [med]
• Compresr.ai, The Token Company — hosted compression APIs. [low]

⚠️ Mistake — running an aggressive input compressor over a cached conversational prefix. Past ~20× compression accuracy degrades, and any rewrite of already-cached bytes flips reads to writes. You can spend more on rebuilds than you save on tokens.

✅ Fix — Use input compressors where there is no cache to lose (one-shot calls, RAG context assembly) or only on the fresh tail. On a cached prefix, prefer a cache-aware approach: compress only the uncached tail and keep the cached span byte-identical (the discipline Headroom’s frozen-prefix handling is built around).

Output-line compressor

Output is the most expensive class and is never cached at generation, so compressing it never busts the prefix cache — and it actually helps it: because each response is cache-written as input on the next turn (see What the cache stores), shrinking the output also shrinks every downstream cache-write and cache-read it would have become. So the win compounds — 5× saved on the output now, plus smaller writes/reads on every later turn — with no risk to the cached prefix. This makes it, alongside sticky routing, one of the two safest levers in the entire ecosystem.

Caveman — compressing the priciest line

Caveman (JuliusBrussee/caveman, dlepold/caveman-distillate, and forks) is a Claude Code skill that instructs the model to answer in telegraphic, low-grammar prose — “why use many token when few token do trick”: short sentences, no filler, dropped articles. Community-reported ~65% output-token reduction, corroborated across multiple independent repos (repo-benchmarked at ~65%, range 22–87% across 10 prompts). Best for internal/agent-facing output where terseness is fine; the trade-off is human readability. [community-reported]

Caveman’s savings come from prompt injection — the same context-shaping that governs caching — and its central engineering problem (keeping an instruction alive as context compression prunes it) is the flip side of the prefix-stability story from Acts I–II. Because the tool is a master class in how a Claude Code plugin actually works, the full anatomy is worth studying.

Caveman anatomy — what it actually is

Caveman is not a model or a service — it’s a prompt-injection + distribution system. One idea: inject a compression ruleset into a coding agent’s context so it writes terse “caveman-style” prose (drop articles, filler, hedging; keep every piece of technical substance), cutting ~65% of output tokens at full accuracy.

The repo is two codebases:

• The behavior — ~400 lines of Markdown prompt. This is the actual IP.
• The delivery machinery — ~5,800 lines of Node that detects 30+ installed agents and wires the behavior into each one’s incompatible config.

The npm package is caveman-installer; the shipped artifact is the installer, not a model. Maintainers edit only the source-of-truth files (skills/*/SKILL.md, agents/cavecrew-*.md, src/rules/*.md); CI mirrors them into the plugin distribution and rebuilds the release ZIP.

The behavior layer (the prompt)

skills/caveman/SKILL.md is the single source of truth for behavior:

• The frontmatter description doubles as the trigger surface — phrases like “talk like caveman”, “less tokens”, “be brief” are listed so semantic skill-matching auto-activates the skill.
• Persistence fights the real failure mode: models drift back to verbose after several turns or after context compression prunes the instruction. The rules repeatedly assert “ACTIVE EVERY RESPONSE.”
• Six intensity levels: lite, full (default), ultra, plus three classical-Chinese wenyan-* variants, each with worked examples.
• Auto-clarity safety valve: it drops back to normal prose for security warnings, irreversible-action confirmations, and any case where dropping articles/conjunctions would make a multi-step instruction ambiguous — then resumes.
• Language preservation: compress the style, not the language (Portuguese in → Portuguese caveman out); code, API names, commit-type keywords, and error strings stay verbatim.

cavecrew is a separate play: three subagent presets (cavecrew-investigator/builder/reviewer) that emit caveman output. The win here is main-context longevity, not output cost — a subagent’s result is injected verbatim into the main thread, so a 2k-token prose result costs 2k of main-context budget; the caveman version returns ~700. Each has a strict, greppable output contract (e.g. investigator: path:line — symbol — note).

The runtime layer — three stateless hooks, one flag file

Claude Code spawns a fresh process per hook event and keeps no plugin code resident, so the hooks share no memory — they coordinate through one ~12-byte file, $CLAUDE_CONFIG_DIR/.caveman-active (contents = the active mode string):

 SessionStart            UserPromptSubmit          Statusline
 caveman-activate.js     caveman-mode-tracker.js   caveman-statusline.sh/.ps1
       │                        │                         │
       └──── writes mode ───►  .caveman-active  ◄── reads ┘ (read-only)

Enter fullscreen mode Exit fullscreen mode

• SessionStart → caveman-activate.js (fires once): resolves + persists the mode; injects the ruleset as hidden system context (SessionStart stdout is system context the model sees but the user doesn’t); reads SKILL.md at runtime and filters the 6-level table down to the active level only (saves injected tokens, avoids confusing the model with inactive levels); and appends a statusline nudge if none is configured (how a plugin, which can’t write settings.json, bootstraps its badge).
• UserPromptSubmit → caveman-mode-tracker.js (every message): natural-language activation (below); /caveman-stats interception (blocks the prompt and returns stats via {decision:"block"} — a zero-token slash command); slash-command parsing; NL deactivation (unlink the flag); and per-turn reinforcement — a one-line re-anchor every turn, because SessionStart’s full ruleset gets pruned by context compression and competing plugins inject their own style each turn.
• Statusline → caveman-statusline.sh/.ps1 (read-only, frequent): never shells out to node; reads the flag (≤64 bytes), lowercases, strips anything outside [a-z0-9-], whitelist-matches a known mode, and renders [CAVEMAN:MODE] plus a savings suffix.

The security spine — caveman-config.js. All hook filesystem I/O funnels through one module because the flag path is predictable and user-writable — the canonical local-attacker setup. safeWriteFlag uses O_NOFOLLOW + atomic temp-write + rename + 0600, and refuses if the flag itself is a symlink (the clobber vector) while still tolerating a legitimately symlinked parent dir (resolve, then verify uid ownership). readFlag is symmetric (symlink-refuse, 64-byte cap, VALID_MODES whitelist → returns null on any anomaly). Without this, an attacker could symlink the flag to ~/.ssh/id_rsa, or fill it with ANSI/OSC escapes the statusline would render on every keystroke; the whitelist guarantees the only renderable outputs are the ~11 known mode strings. src/hooks/package.json pins {"type":"commonjs"} so the hooks load even when an ancestor package.json declares "type":"module".

“Off” is represented by file absence: deactivation unlinks the flag, and readers treat “missing” and “unreadable” identically to inactive — failing safe toward off, never toward leaking content.

Natural-language activation

The matcher in caveman-mode-tracker.js has three OR’d branches: (A) verb-before-noun (activate|enable|turn on|start|talk like … caveman), (B) noun-before-keyword (caveman … mode|activate|enable|…), and (C) brevity requests with no “caveman” at all (less tokens|fewer tokens|be brief|be terse|shorter answers). A negative guard (!/stop|disable|turn off|deactivate/) is load-bearing: “turn off caveman mode” matches Branch B, but the guard suppresses the write so a deactivation request can’t accidentally re-arm the mode.

Two deliberate asymmetries: activation never carries a level (it always resolves the default — “talk like caveman ultra” does not give ultra; in an ultra-pinned repo, “be brief” silently activates ultra), and it respects off as a configured default (even “talk like caveman” writes nothing if the configured default is off — configured intent beats an ad-hoc phrase). Firing order is fixed — NL activation → stats → slash → NL deactivation → reinforcement — and because deactivation runs last and unconditionally deletes, it always wins on conflict (fail-safe toward off).

Verified gap: the opencode port (src/plugins/opencode/plugin.js) reimplements the logic but has no Branch C — “less tokens” / “be brief” do not activate caveman under opencode, a genuine inconsistency with the Claude Code hook and the README promise. A clear fix-or-document candidate.

Why a SessionStart hook, not --append-system-prompt?

This is a useful design lesson about Claude Code’s injection paths. --append-system-prompt fails Caveman’s constraints; a SessionStart hook satisfies them:

1. A plugin cannot set it. It’s a CLI/SDK launch flag; plugin.json has no systemPrompt/appendSystemPrompt field (the keys are name/description/author/hooks). SessionStart stdout-as-context is the only plugin-declarable always-on injection path.
2. The content is dynamic. caveman-activate.js computes the injection each session (mode resolution, SKILL.md read, level filtering, conditional statusline nudge); a static launch string can’t.
3. Always-on, zero per-launch friction. --append-system-prompt is per-invocation; a SessionStart hook fires forever after one install.
4. Mid-session switching needs a hook anyway. A launch-time string is frozen for the process; Caveman already needs UserPromptSubmit for switching + reinforcement, so SessionStart is the natural sibling on one flag-file mechanism.
5. Single source of truth. The hook reads SKILL.md live; a static append would drift.

The honest counterpoint: --append-system-prompt sits higher in precedence (it’s appended to the real system prompt; SessionStart stdout is a lower-precedence system reminder), so a static append would be stickier in principle. Caveman gives that up and compensates with per-turn reinforcement — which arguably survives context compression better because it’s continuously re-injected. Net: the SessionStart hook wins not as a better injection primitive in the abstract, but as the only one that is plugin-distributable, dynamic, persistent, and switchable.

Cross-cutting lessons from Caveman

• Deliberately lopsided — trivial core behavior, heavily over-engineered distribution. That’s the correct trade: the hard problem isn’t making the model terse (one paragraph does that), it’s reliably injecting that paragraph into 30 agents with incompatible configs, idempotently, on every OS, without clobbering user config or opening a symlink attack, and keeping it loaded as context compresses.
• Issue-driven design — nearly every non-obvious line traces to a numbered GitHub issue (Windows quoting, hook double-firing, orphaned hooks, opencode hook keys), not speculation.
• Honest measurement — the eval harness compares skill-output vs plain-terse output (not vs baseline, which would conflate the skill with generic terseness), and stats report savings only for the benchmarked full mode.

Model routers — attacking the model-tier line

A model router looks at each turn and sends it to a different model based on how hard the turn looks — easy turns to a cheap model, hard turns to a strong one — to lower the bill.

• RouteLLM (UC Berkeley LMSYS, lm-sys/RouteLLM) — a trained router; reports ~85% cost reduction at ~95% of GPT-4 quality on MT Bench (45% on MMLU, 35% on GSM8K). The figures are GPT-4-era and benchmark-specific. [high]
• vLLM Semantic Router (vllm-project/semantic-router) — routes across local/private/frontier models by cost, latency, privacy, and safety. [high]

The idea is sound for stateless, one-shot calls. Inside a stateful coding agent like Claude Code, switching models mid-conversation (“dynamic” or per-request routing) has three costs that don’t show up on the per-token price tag — you only see them when you measure:

1. You lose the cache. The prompt cache is model-scoped (Act II) — Sonnet can’t read Opus’s cached prefix. The first turn you bounce to a cheaper model, that model has no warm cache and re-reads the entire conversation from scratch. On a long session the prefix dominates the bill, so that one cold read can wipe out everything the cheaper model saved you.
2. The reasoning is carried and billed — but you can’t verify the new model uses it. Earlier turns contain thinking blocks the original model produced. Across the Opus/Sonnet/Haiku family those blocks do cross the switch — re-rendered into the new model’s prompt and billed as input, so you pay to carry them. What you can’t see, because they’re encrypted, is whether the target model actually reused that reasoning or quietly ignored it — you pay for the carry with no guarantee of the benefit. The only symptom of a miss is the new model doing a worse job.
3. Coming back can miss too. A cache breakpoint only looks back ~20 content blocks (Act I). If you route away from your main model and stay away for more than ~20 blocks before returning, the main model’s cached prefix has aged out of that window — so even coming home is a cold miss, not the warm hit you expected.

Because of #2, judge routing by outcomes, not by the token price. A router can make every turn cheaper and still be a net loss if the weaker reasoning makes you take more human turns to get the same result — three cheap turns plus your time to re-steer the agent can cost more than one good expensive turn. So put evals in place (task success rate, turns-to-done, human-correction rate) and watch them the moment you enable routing. If the human experience gets worse, the token savings are a mirage.

⚠️ Mistake — routing per request inside one conversation. It forfeits the model-scoped cache, makes you carry-and-bill the original model’s (encrypted, unverifiable) reasoning on the new model, and can miss the cache even on the return trip. Measured below: bouncing 20% of turns to Sonnet actually loses money.

✅ Fix — Route sticky (per conversation or per sub-agent) so each model reads its own warm cache, or use routers only for stateless single-shot traffic where there’s no cache to lose. Either way, gate it behind evals so a cheaper bill can’t hide a worse experience.

How routing actually behaves inside Claude Code

The routers above assume you can swap models freely. Inside Claude Code you can’t: the request body is co-designed for one target model, and routing it elsewhere fails in escalating ways before you even reach the cache economics.

The parameter-failure ladder [measured / doc-confirmed]

A naive proxy that rewrites only the model field hits a wall fast. req 0001 → Sonnet (200); req 0002 → rerouted to Haiku → 400: "adaptive thinking is not supported on this model" → Claude Code aborts. Claude Code sends thinking:{type:"adaptive"} (valid for Sonnet); Haiku 4.5 rejects it.

Three parameters must be normalized to route to Haiku, each surfacing as a distinct 400:

1. output_config.effort — Haiku 4.5 rejects effort; drop it. (Relatedly, xhigh is Opus-4.7+-only: replaying an Opus request to Sonnet 4.6 gives 400: "does not support effort level 'xhigh'. Supported: high, low, max, medium.")
2. thinking:{type:"adaptive"} → must become {type:"enabled","budget_tokens":N}. You can’t set {type:"disabled"}, because Claude Code also sends context_management:{clear_thinking_20251015}, which returns 400: "clear_thinking_20251015 strategy requires thinking to be enabled or adaptive."
3. budget_tokens must be ≥1024 and < max_tokens.

With those fixed (a “smart” proxy), both models complete: feature built, all 87 tests pass, every request 200; rerouted-to-Haiku requests carrying Sonnet thinking blocks all succeeded. The lesson: the request body is interlocked — model name, thinking parameter, effort, and context-management strategy are co-designed. Naive per-request rewriting fails loudly (400), and even when fixed, forfeits the model-scoped cache.

Claude Code already routes correctly — and you should leave it alone. It calls Haiku itself for background/auxiliary work while the main loop stays on your primary model (observed as orig_model=haiku requests the proxy never touched). The right answer for almost everyone is to let the built-in routing do its job and add no router at all. Don’t insert your own routing unless you really know what you’re doing — you’d have to reproduce the interlocked parameter normalization above and still eat the model-scoped cache penalty, which usually costs more than it saves. If you have a genuine reason to route, route sticky (per task or per sub-agent), never mid-conversation — and treat that as an advanced move you measure yourself, not a default.

The rate card and the per-turn cost formula

Per 1M tokens; read 0.1×, write 2× (1-hour TTL):

component Opus 4.8 Sonnet 4.6 Haiku 4.5 cache read $0.50 $0.30 $0.10 cache write (2×) $10.00 $6.00 $2.00 output $25.00 $15.00 $5.00 (base input) $5.00 $3.00 $1.00

The per-turn cost is:

turn $ = read × read_rate + write × write_rate + output × output_rate

Enter fullscreen mode Exit fullscreen mode

With rates baked in (output ×0.775 tokenizer factor for Haiku/Sonnet):

Opus $   = read × 0.00000050 + write × 0.00001000 + output × 0.000025
Sonnet $ = read × 0.00000030 + write × 0.00000600 + (output × 0.775) × 0.000015
Haiku $  = read × 0.00000010 + write × 0.00000200 + (output × 0.775) × 0.000005

Enter fullscreen mode Exit fullscreen mode

The two regimes, and break-even

• Sticky / per-conversation routing → the cheaper model reads its own warm cache → a flat rate multiple of Opus → clean savings.
• Per-request bouncing → the 1st call cold-writes the full prefix; later calls catch-up-write the gap. Write-heavy.
• Break-even (per turn): a cold bounce only saves when the turn’s output is large relative to the prefix — empirically around 2,500 output tokens. Below that, the cold prefix write dwarfs whatever you saved on the cheaper output.

Three measured experiments

Experiment A — tiny task (linked-list fix, 6 turns, Opus 4.8). Prefix ~41K, all warm. All-Opus: $0.199 (~$0.033/turn). A 10%-Haiku per-request bounce: +16% (the cold write dwarfs the tiny outputs). A 10%-Haiku sticky: −8%. Break-even output ≈ 2,500 tokens. (Claude Code self-reported $0.182 — its 1.25× write estimate.)

Experiment B — feature build (16 Opus turns, MCP stabilized). Usage: cache_read 1,067,378 · cache_write 53,449 · output 35,579 · input 2,502. Acceptance 27/27, 24 unit tests green.

Turn read write output Opus $ 1 29,874 11,301 168 0.1321 2 41,175 1,544 13,466 0.3850 ✱ 3 42,719 16,024 127 0.1848 4 58,743 517 5,802 0.1796 5 59,260 5,858 8,347 0.2969 6 65,118 8,611 446 0.1298 7 73,729 561 194 0.0473 8 74,290 283 500 0.0525 9 74,573 591 316 0.0511 10 75,164 405 466 0.0533 11 75,569 562 96 0.0458 12 76,131 477 3,232 0.1236 13 76,608 3,468 123 0.0761 14 80,076 1,269 1,574 0.0921 15 81,345 1,659 227 0.0629 16 83,004 319 495 0.0571 Total 1.970

✱ Turn 2 includes a one-off uncached-input term (2,472 × $5/M = $0.0124). The cross-check is instructive: the documented 2× rate totals $1.970, while Claude Code displayed $1.77 (its 1.25× under-count). The $0.20 gap = 53,449 × ($10−$6.25)/M ≈ $0.20 — exactly the extra write cost. This is the displayed-cost gotcha made concrete.

Experiment D — interleaved 25-turn routing (Opus / Haiku / Sonnet). 20% of turns (3, 9, 14, 19, 23) routed to a cheaper model in a single session; turn 3 is the first bounce (cold), the rest are catch-up writes. One table answers the question directly — what a 25-turn session costs when 20% of its turns go to a different model, versus staying put or going fully sticky:

Routing strategy Session cost vs all-Opus Verdict All-Opus (no routing) $2.574 — baseline 20% of turns → Haiku, per-request $2.278 −11.5% modest win 20% of turns → Sonnet, per-request $2.628 +2.1% ❌ loses money All-Haiku, sticky ~$0.41 −85% biggest win All-Sonnet, sticky ~$1.21 −53% big win

Read it slowly — it’s the punchline of the whole guide. Routing 20% of turns to Sonnet per-request actually loses money (+2.1%): Sonnet’s write rate is 3× Haiku’s, so on the cold and catch-up turns the parallel-prefix write costs more than the cheaper output saves (turn 3 alone runs +$0.09 versus just staying on Opus). Yet the same model run sticky saves 53%. That’s the lesson: where you switch matters more than whether — bouncing per-request can cost you; keeping one cheaper model for the whole conversation is the real win.

Choosing a tool: the decision matrix

Tool / class Attacks Net win when… Watch-out RouteLLM, vLLM Semantic Router model tier routing is sticky per conversation/sub-agent per-request bouncing forfeits the model-scoped cache Headroom input + cache the prefix is stabilized so cache reads survive retrieval round-trips cost some tokens LLMLingua input one-shot / RAG context, not a cached prefix busts the cache; degrades past ~20× RTK, lean-ctx tool-output input tool outputs are verbose (logs, installers, git) — Caveman output output volume dominates and terse prose is acceptable human readability OpenAI Compaction, Compresr, Token Co. history / input provider or hosted flows hosted ones send data off-box; non-reversible

The single most reliable lever is protecting the cached prefix (Act I). Compression and routing help only insofar as they don’t undo that. Output compression (Caveman) and sticky routing are the two that never touch the prefix — which is why they’re the safest wins in the entire ecosystem.