Test LLM API savings on your own workload before expanding context optimization to every conversation. Condense now lets organizations split new conversations between optimized and uncompressed paths. Compare cost, quality, and latency through one endpoint, with no second integration to maintain.
Choose how much traffic to optimize, keep a control group, and increase the share when the results meet your requirements.
One endpoint, two paths
Set a percentage for new conversations. At 25%, roughly one quarter use Condense’s context optimization; the rest pass through without it. Both paths use the same endpoint and the model your application requests.
A conversation is assigned when it starts and keeps that path for its lifetime. Changing the split affects new conversations, so an ongoing task stays on the same path.
The percentage applies to conversations, not tokens. Long and short conversations count equally, and small samples can land above or below the selected share. The transparent path still uses the Condense proxy; it forwards context without optimization.
Measure the work, not just the tokens
Pick a model and a representative set of tasks. Start with a share such as 25%, keep an uncompressed control, and let enough tasks finish to make the comparison useful.
- Cost per completed task. Include all turns, cache reads, and cache writes.
- Task quality. Use the same success criteria for both groups.
- Latency and retries. Check time to completion and repeated work.
The split assigns each conversation to one path; it does not run the same task twice. For a matched comparison, replay the same tasks through both paths. The agent quality evaluation and token bill breakdown show what to measure.
Expand when the results hold
Increase the optimized share when cost and quality meet your requirements. Reduce it when you need a larger uncompressed control to investigate a difference.
At 100%, all new conversations use context optimization. At 0%, all use the transparent path. The default is 100% until you choose a split. Changes require no application code updates.
Get started
API traffic splits are available to organizations. Connect your application with an organization API key using the API setup guide. Already connected? An organization admin can set the split now:
- Open the dashboard and select your organization.
- Go to Settings → Organization → Traffic through Condense.
- Choose the percentage with the slider or input, then click Save.
Only organization admins can see and change the control. If it is missing, check the selected organization and your role. Personal accounts do not have traffic splitting.
Follow the A/B testing and traffic splitting guide for setup, conversation assignment, and comparing cost and quality.