Modal pins each session to 1 container and preserves context on reconnection

A voice bot is talking to a user, the connection drops, and the user reconnects—only to reach a process that knows nothing about them. For the server infrastructure, this may be normal load balancing. For the product, it is a broken conversation. Modal has introduced Sticky Sessions: a mechanism that routes requests within a session to the same container and scales infrastructure based on the number of active sessions.

In practice, developers can keep temporary state alongside the running code: a voice agent's context, the state of a speech recognition decoder or a shared game room. On reconnection, the request returns to where that state already lives, as long as the session remains alive.

Why a WebSocket alone is not enough

Imagine an interview practice service powered by voice AI. The agent and audio processing run in a container, which also holds the current conversation's state. The user switches from Wi-Fi to mobile data. The connection has to be reestablished, but the product needs to resume the interview where it left off.

As Modal explains, a WebSocket in its Web Functions is tied to an individual function invocation. After a disconnect, a new invocation is created and may land in a different container. A WebSocket alone therefore does not solve the problem of returning to the original process.

One workaround is to limit the application to a single container. Requests then have nowhere else to go, but solving the routing problem this way also eliminates horizontal scaling. Modal Servers also support routing through the Modal-Affinity-Routing-Key header, but this works on a best-effort basis: the container may be stopped or replaced.

Sticky Sessions introduce a stronger binding for the duration of a session. This is particularly useful for Gradio and Streamlit applications, where user state is tied to a specific server process. Developers no longer have to keep the entire service running as a single instance to preserve continuity for individual user workflows.

How a request finds the right process

To enable the mechanism, developers annotate the Modal Server class with the @modal.sessioned() decorator, then create a session through the SDK or HTTP API. In the company's example, calling sessions.start(idle_timeout=600) returns an object containing a token. The client passes this token in the Modal-Authorization header on subsequent requests.

The token is signed and contains the information needed for routing. The proxy validates it and forwards the request to the container that owns the session. This means the proxy does not need to maintain a separate, consistent session-to-container mapping across all its instances.

A single token can be shared with multiple participants in a shared room. For example, two users can connect to the same chat and work with state held in a single process. Reconnecting returns a participant to the same room within its lifetime. The token grants access to the session, so the application should treat it as a credential.

The unit of scaling changes as well. A container accepts a configured number of concurrent sessions. When no slots are available, new sessions wait for existing ones to end or for an additional container to become ready. Autoscaling accounts for active sessions and the queue, because a conversation already in progress cannot simply be moved to a neighboring process.

What to check before adoption

The main limitation: pinning a route does not mean data will survive a container failure. The announcement describes continuity of requests within a session, but does not promise to restore its in-memory state after a crash. Conversation histories, work results and other important data still need to be persisted separately if the product must survive such events.

Load distribution is another consideration. Two sessions can consume different amounts of resources: a short text conversation and a continuous audio stream impose very different loads. The number of sessions per container therefore needs to be chosen for the actual use case. The mechanism does not automatically move an existing resource-intensive session to balance the load.

For teams building voice agents or interfaces that depend on process memory, this is a reason to test behavior during connection drops and assess the cost of long-running sessions. Sticky Sessions address a specific infrastructure gap: they combine automatic container provisioning with the ability to return users to an already running process. A failure recovery plan remains part of the application architecture.