We ran RTK on Terminal-Bench and compared it with running no tool at all. RTK compresses shell output before a coding agent reads it. In this run, cost per suite pass came in 15.2% higher with RTK, and fewer tasks were solved.
| Measure | No tool | RTK |
|---|---|---|
| Cost per suite pass | $25.46 | $29.33 |
| Tasks solved | 81.5% | 75% |
Both sides used the same suite and the same model. These figures don't show why the cost went up. That's still an open question.
RTK's claim
RTK intercepts an agent's shell commands and compresses what they return. Test runs come back as failures only, and grep results get grouped by file. Its README advertises up to 90% less bash output.
The same README says that isn't the same as cutting the bill by that much. Bash output is one part of the input, and input is one part of the bill. So this run doesn't contradict RTK's claim. The 90% is about output. Our figures are about cost and tasks solved.
The limits
This was a single run. Our public benchmark, minmax-bench, won't give a verdict on fewer than two finished runs per arm, so read this as one data point. Even with no tool, runs on Terminal-Bench flip tasks between solved and unsolved.
Terminal-Bench is also one kind of work. When we looked at recorded sessions from this suite in August, they ran bespoke scripts more than the package managers and test runners many of RTK's filters target. Projects with heavy build and test output could look different.
We're also not a neutral party. condense sells context management. We've said before that RTK and condense go after different parts of the bill and can run together. This run didn't test them together.