Anthropic Launches Message Threads: 35% Faster Response Start in Long Sessions

Anyone who has built an agent on long context knows this pain: the session grows, and every new step starts with a pause. The model hasn't said anything yet, but you have already sent hundreds of thousands of tokens of history over the network. Anthropic released Message Threads to eliminate exactly this pause.

What Changed

Previously, the scheme was simple and inconvenient. At every step the client assembled the entire dialogue: the system prompt, tool results, earlier turns, and sent it all at once. The larger the history, the longer the delay before the first token.

Now the API request includes a thread parameter. The full data array is sent once. After that, the client sends only new messages and the ID of the previous response. Between requests, Anthropic keeps the context in RAM in encrypted form. The thread lives for five minutes after the last call, after which the history must be sent again.

According to the company, for agents and sessions of 500,000 to 1 million tokens, the wait for the response to begin is reduced by 35% on average. The public beta is available on the Claude platform, in AWS, Microsoft Foundry and Google Cloud.

How It Looks in Practice

Imagine a coding agent working through a large repository. It has read 40 files, run tests, received logs, and now the context takes up 600,000 tokens. Previously, every subsequent tool call meant resending all 600,000. Now the agent sends a short message like "here is the result of the last test" plus the ID of the previous response. The network and request preparation are offloaded, and the user sees a response sooner.

An important caveat: billing is unaffected. The model still processes the entire volume, so tokens are counted as before. This is a latency optimization, not a way to save budget. If cost is your pain point, look toward prompt caching and context compression, not Threads.

Limitations and What to Do

The main catch is the five-minute window. For an interactive agent that works continuously, that is enough. For a chat where the user has gone to lunch, the thread will expire, and the first request after the pause will be heavy again. That means client code must be able to rebuild the full history: store it on your side, and use thread as an accelerator, not as the source of truth.

Another point: the feature is in beta. Don't make it the only path until you have verified behavior on stream interruption and errors. A sensible plan: enable thread on one agent scenario, measure time to first token on real sessions, and compare it with the promised 35%. The figure is an average; on short dialogues the gain will be more modest.

What Else Is Nearby

Other news came out the same day that is worth keeping in mind. According to the FT, Nvidia is unofficially requiring data center operators to ensure that customers from China occupy no more than 49% of leased capacity under new contracts. If you rent GPUs from smaller clouds, expect questions about your customer mix and possible shifts in availability.

OpenAI has raised the base payout for critical vulnerabilities in ChatGPT and Codex to $30,000, and the maximum with bonuses reaches $172,000. For teams building products on top of these services, this is a signal: the attack surface is being tested seriously, and you should do the same.