Developer moves agent out of OpenWebUI after a 1.67-million-token request

If your agent reads a large file, its contents may be sent to the model again with every subsequent message. That is exactly how a small team's internal assistant running on OpenWebUI ended up with a 1,674,813-token request and an API rejection. The author of a breakdown published on Habr first cleaned up the history with a filter, then moved task execution to a separate service. The practical lesson: before increasing the context window, find out which data is being sent to the model repeatedly and which component holds the conversation state.

One file stays in the conversation for a long time

According to the author, a single read_file call returned 66,871 characters. Eleven tool calls produced almost 130 thousand characters—roughly 32 thousand tokens by the author's estimate. Those results kept traveling with the conversation history. The agent requested more checks, the context grew, and the user eventually canceled the task.

Imagine asking an agent to fix a service configuration. It reads the file, changes a few lines, and runs a check. At the next step, it may need the check result and a small configuration snippet. But the chat attaches the entire old file again, along with previous commands and their output. A few such cycles turn a working conversation into a warehouse of repeated data.

The first filter removed tool results from previous turns while leaving the current turn untouched. That is an important boundary: removing the result of a tool that has just been called would deprive the model of the data it needs to continue the task. The author then added a summary of the older conversation while retaining the most recent messages.

A synthetic test looked convincing: the estimated size dropped from 3637 to 103 tokens. But a day's logs told a different story: across 128 filter activations, it removed just two messages with the tool role. Much of the clutter remained inside ordinary assistant response text, including tool display blocks. Those had to be cleaned up too.

The agent's memory moved to a separate service

Summarization also ran into input-size limits. According to the author's report, compression requests reached 161–170 thousand tokens, while the selected model had a limit of 128 thousand. The mechanism failed on the longest conversations, where it was needed most.

There was also a timeout when the filter called OpenWebUI's own API in a single-worker configuration. The author attributed it to the handling of a nested request and replaced that route with a direct call to the gateway. This was an observation from one specific installation; the explanation should not be extended to every single-worker configuration without diagnosis.

By July, the author had changed the architecture. OpenWebUI retained user management, file uploads, and response display. A Pipe function began forwarding the latest message, system rules, and attachments to an external Agent Runtime. Conversation state is now held by the execution service itself, which uses the Agents SDK or Codex SDK.

For the team, the key is clear ownership. In engineering mode, one service manages history, tools, and task continuation. It no longer has to reconstruct working memory from the interface's conversation history. A separate Runtime does not, however, guarantee a short context on its own: retention and compression rules are still necessary.

Streaming responses had to be checked event by event

The switch to the Responses API exposed another trap. A tool call appeared in intermediate stream events, but output was empty in the final object. The installed SDK version took the turn result from that final object, causing the agent to repeat calls until it exhausted the step limit.

A separate probe that collected completed items from the stream allowed the workflow to continue. But the author did not establish where the data was lost: there was no comparison of the stream before and after the proxy. This case does not justify attributing the problem to the API itself.

After updating dependencies, the author reported six successful API scenarios without a hidden fallback to Chat Completions. Browser testing, sustained parallel load, and some state-recovery scenarios remained outside the scope of acceptance testing.

For your own system, it is worth repeating three checks: inspect the actual request history, test the summarizer at maximum input size, and confirm the protocol actually used through logs. This case shows how many errors can hide behind an agent response that appears successful.