Habr Deep Dive: Why r.Context() Doesn't Stop an LLM, and How to Build AI Tasks in Go

Closing a browser tab cancels the HTTP request, but it does not have to stop the model on the provider's side. The author of a recent Habr article shows how this gap turns into costs and bugs in an ordinary Go backend. If a user starts a document analysis and leaves, the service may end up with no result and still receive a bill for tokens. For long-running AI operations, the author proposes background tasks on PostgreSQL, strict idempotency and a deliberate retry policy.

Where the familiar model breaks

Go developers are used to a simple rule: work lives as long as `r.Context()` lives. The client drops the connection, the context is canceled, goroutines finish and resources are freed. For a database query or a call to an internal microservice, this is convenient and correct.

LLMs are different. A call can last tens of seconds or even minutes, especially when parsing large documents or chaining several model calls. Billing is based on tokens processed and generated, and canceling on your side does not guarantee that the provider stops working immediately. That depends on the specific API and on how the connection is closed.

This creates two problems at once:

• the work is canceled and there is no result, but the money is spent;
• the user comes back, presses the button again, and the paid operation runs a second time.

A background task as a standalone entity

The core idea of the article is that an AI operation should not belong to the HTTP request. The handler accepts the input, creates a task and immediately responds with an identifier. From there the task lives its own life, and the client learns about it through polling, SSE or WebSocket.

The author uses PostgreSQL for storage. For many teams this is a reasonable choice: the database is already in the stack, transactions are well understood, and a separate message broker is not needed. A typical set of components for such a queue looks like this:

• a tasks table with a status (created, running, completed, failed, canceled);
• task claiming by workers via `FOR UPDATE SKIP LOCKED`, so that two workers do not pick up the same task;
• a last-heartbeat timestamp, used to return stuck tasks to the queue;
• an attempt counter and the time of the next attempt;
• the stored result and data on tokens used.

The main gain is that state survives a service restart. During a deploy or a pod crash, the task does not vanish along with the goroutine. Another worker picks it up.

Idempotency and money

The most expensive spot in such a system is where retries meet billing. Repeating an LLM request costs money, so it cannot be done blindly, the way you might retry a cache lookup.

Practical principles that follow from the logic of the article:

• an idempotency key is created at the level of the user action and protects against a double click on the button;
• each task has a unique identifier, and the result is written once against it;
• retry is allowed only for errors where a repeat makes sense (timeout, 429, 5xx), but not for invalid input;
• charging the user is tied to the task, not to an individual attempt;
• cancellation by the user becomes an explicit status, not a side effect of a broken connection.

Where to start

If your service runs the LLM call directly in the handler, check three things. First: what happens to a paid generation if the client disconnects midway. Second: whether a repeated request can create a duplicate of a paid operation. Third: whether the task survives a process restart.

If the answer to even one of these questions is uncomfortable, you need a separate task model. The minimal step is a table with statuses and a single worker; `SKIP LOCKED`, heartbeat and a retry policy can be added as load grows. It makes sense to bring in a separate broker once PostgreSQL stops coping with the queue. Until then, it usually only complicates the system.

The article is not about a new model or a release, but about engineering discipline around LLMs. That is where a noticeable share of AI product budgets is being lost right now: the errors occur in the plumbing, not in the prompts.