For Claude Code, Codex and other coding agents, the best token compression tool compresses old session history re-read every turn, without breaking the prompt cache. For one-shot prompts and RAG, a token pruner like LLMLingua-2 fits better. Judge every tool on cost per solved task, not tokens removed. We build condense, one of the tools below, and every measured number here comes from our open-source harness, minmax-bench.
Token compression tools compared
Each tool is described by its own docs. Results are quoted only where someone published them.
| Tool | Compresses | Runs | Lossy? | Published result |
|---|---|---|---|---|
| Built-in compaction (/compact, auto-compact) | Conversation history | Inside the agent | Yes, summary of the whole session | None we know of on cost |
| Prompt caching | Nothing; discounts repeated prefixes | Provider | No | Provider pricing pages |
| LLMLingua / LLMLingua-2 | Prompt text | Python library | Yes, token pruning | Microsoft research papers, mostly QA and RAG |
| rtk | Shell command output | Local CLI hook | Yes, filtering and grouping | Up to 90% less bash output (README); +15.2% cost per pass in our Terminal-Bench run |
| Headroom | Tool outputs and context | Local proxy | Yes, filtering plus model-based kompress | 13.8% of the bill in our replay; kompress 1.8% |
| condense (ours) | Session history and the newest turn | Hosted proxy via the dense CLI | Yes, verbatim at the edge, summaries for finished work | 37.3% of the bill in the same replay; 72% on a 938-turn session |
Bill removed, same session
One real Claude Code session, replayed live through each tool on claude-haiku-4-5 and priced from provider usage with cache tiers.
Savings depend on length
Below about 20 turns condense costs slightly more than it saves. Past a few hundred turns the saving climbs.
Less output is not a lower bill
A tool that shrinks one slice of input can still lose on cost per solved task. Single run, one suite.
| Terminal-Bench, same model | No tool | rtk |
|---|---|---|
| Cost per suite pass | $25.46 | $29.33 |
| Tasks solved | 81.5% | 75% |
Details in the rtk Terminal-Bench run.
How to choose
Find your biggest input first. In real coding sessions, cache reads of old context were 67.7% of spend and fresh input 0.2% (the anatomy of a token bill).
- 1RAG or long documents in one call
Start with LLMLingua-2. This is the workload it was built for.
- 2Short agent sessions
Prompt caching plus one session per task. Trim output with these commands.
- 3Noisy build and test output
A shell filter like rtk. Check that the failing line survives.
- 4Long Claude Code, Codex or OpenCode sessions
condense:
curl -fsSL https://cli.condense.chat/unix | sh, thendense claude. It stacks with rtk. - 5Measure on your own tasks
Same tasks, same model, several runs, spend from the provider bill divided by tasks solved. See cost per task vs cost per token.
More on the methods behind these tools in what is context compression and /compact vs /clear.
Frequently asked questions
What is the best token compression tool for Claude Code?
It depends on where your tokens go: rtk targets shell output, while long sessions need a cache-aware history compressor such as /compact or condense. Measure cost per solved task on your own sessions before you commit.
Is LLMLingua good for coding agents?
It was built and benchmarked mainly on QA, summarization and RAG, and you wire the Python library into your agent yourself. Token-level dropping can damage identifiers and diffs in code.
Does prompt caching count as token compression?
No. Caching sends every token and bills repeated prefixes at a discount; it stacks with compression when the compressor keeps the cached prefix stable.
Why do token compression tools advertise 80-90% savings but my bill barely moves?
Headline numbers usually measure one slice of input, such as bash output, while your bill also includes cache writes, cache reads and output. Rewriting history can also break the cache and cost more than it saves.
Is token compression lossy?
Almost always. The real question is whether the agent still finishes the task, which only a with-and-without comparison answers.