Docs/API

A/B testing with API traffic splits

Test context optimization on a share of your organization’s API conversations. Keep the rest as an uncompressed control, compare the results, and increase the optimized share when it meets your requirements. Both paths use the same Condense endpoint and the model you request.

Before you start

• An organization in Condense. Traffic splitting is not available on personal accounts.
• An organization admin to view and change the split. API access alone does not grant permission to change this setting.
• An application sending requests through Condense with an organization API key. Follow the API setup guide and quickstart if you are not connected yet.

The split is an organization setting. Choose a test workload and success criteria before changing it for live traffic.

Set the traffic split

1. Open the dashboard and select your organization.
2. Go to Settings → Organization → Traffic through Condense.
3. Choose the percentage of new conversations to optimize using the slider or number input. Click Save.
4. Start new conversations to use the updated split. Your application’s endpoint, credentials, and request format stay the same.

How conversations are assigned

Condense assigns a path when a conversation starts. The conversation keeps that path for its lifetime, including later requests. Updating the split only affects conversations that start after the change.

• Optimized: the conversation uses Condense’s context optimization before requests reach your model provider.
• Uncompressed: the transparent proxy forwards the context without optimization. Requests still pass through Condense; this is not a direct connection to the provider.

The percentage describes a share of conversations, not tokens, requests, or spend. A short conversation and a long conversation each count once. At 25%, the optimized group can account for more or less than 25% of your tokens, and a small sample may not match the selected ratio exactly.

Compare cost and quality

Start with a share such as 25% and let enough representative tasks finish in both groups. Keep the model, agent configuration, and task mix comparable. Record each task’s outcome and usage so differences in workload do not look like savings.

• Cost per completed task: include every model turn, input and output, cache reads and writes, and retries. Compare total cost with the number of successfully completed tasks.
• Task quality: use the same tests, acceptance criteria, or review process for both groups. Track failures and incomplete tasks alongside successes.
• Latency: measure time to complete the task, as well as slow requests and repeated work.

For a matched comparison, use a separate test organization and replay the same task set in fresh conversations at 0% and 100%. Keep the model and evaluation conditions consistent. Changing the split midway through an existing conversation will not move it to the other path.

See evaluating agent quality for task evaluation and the anatomy of a token bill for cost accounting, including cache behavior.

Increase or reduce the optimized share

Increase the percentage when cost and quality meet your criteria. Keep an uncompressed control while validating the result on a wider workload. At 100%, every new conversation uses the optimized path.

To stop assigning new conversations to optimization, set the share to 0% and save. Conversations already assigned to the optimized path stay there; start a new conversation to use the new setting.

Troubleshooting

The traffic control is missing

Check that an organization is selected and that you are an organization admin. Personal accounts and other organization roles do not see this control. Ask your organization’s admin to set the split.

An existing conversation did not switch paths

This is expected. Save the new percentage, then start a fresh conversation using an API key for the organization you updated.

The token share does not match the percentage

The setting applies to conversations. Different conversation lengths and small sample sizes can produce different shares of tokens and cost. Compare completed tasks across a representative sample.

Read the API traffic splits announcement or check the release notes.