OpenAI explains how it rebuilt WebRTC for low-latency voice AI

OpenAI has revealed details of how it reworked its WebRTC infrastructure for ChatGPT voice features and the Realtime API. The goal was simple: keep conversations with AI from falling apart because of network delays, even under global load and with hundreds of millions of users.

Why the old approach didn't work

For voice AI, it is not enough to recognize speech and generate a reply quickly. A conversation has to move at the pace of human speech: no awkward pauses, no clipped interruptions, no one-second wait before the model reacts. OpenAI writes that at its scale this comes down to three requirements at once: global reach for more than 900 million weekly active users, fast connection setup, and low, stable audio round-trip time.

"Voice AI feels natural only when the conversation moves at the speed

of speech."

The problem was that the classic WebRTC approach fits poorly with OpenAI's cloud infrastructure. If every session needs its own public UDP port, high concurrency forces you to open and secure huge port ranges. That is awkward for Kubernetes, complicates load balancing, makes autoscaling more fragile and enlarges the attack surface. Meanwhile, ICE and DTLS sessions remain stateful: packets must land on exactly the process that owns a given connection.

Relay plus transceiver

After comparing several options, OpenAI rejected the design in which the model acts as an ordinary WebRTC participant through an SFU. Its typical workload is one-to-one: one user talks to one model, or one client talks to one real-time agent. So the company chose a transceiver model: an edge service terminates the client's WebRTC connection, then translates media and events into internal protocols for inference, transcription, speech synthesis, tool calling and orchestration.

The key idea of the new architecture is to separate packet routing from protocol termination. The relay became a lightweight UDP layer at the entrance with a small public network footprint, while the transceiver remained the stateful component that holds ICE, DTLS, SRTP keys and the whole session lifecycle. The relay does not decrypt media, does not negotiate codecs and does not try to pose as a WebRTC peer. It only reads a minimal amount of metadata from the packet and forwards traffic to wherever the right session lives.

The most interesting trick involves the first packet. OpenAI uses the ICE username fragment, or ufrag, and embeds in it exactly as much routing information as the relay needs to pick a cluster and a specific transceiver. During signaling, the client receives a shared relay VIP and a fixed UDP port, and the first STUN packet gives the system enough data to send the flow down the right path immediately, without a separate call to an external lookup service. Once the route is established, the address mapping is also stored in Redis so the relay can recover quickly after a restart.

How latency was reduced

Once the public UDP surface was reduced to a small number of stable addresses and ports, OpenAI scaled the same design globally. This produced Global Relay, a distributed set of entry points that accept packets closer to the user and bring them into OpenAI's network without an unnecessary first hop through a distant region. For signaling, the company uses Cloudflare's geographic and proximity routing, so both the initial HTTP/WebSocket request and the first ICE check arrive at the nearest suitable cluster.

• A small, fixed public UDP layer instead of thousands of open ports
• First-packet routing through data already embedded in the ICE ufrag
• A shared UDP socket on the transceiver side instead of one socket per session
• Short-lived in-memory state plus a Redis cache for fast route recovery
• `SO_REUSEPORT`, pinning threads to OS threads and minimizing allocations for high throughput

OpenAI wrote the relay in Go and deliberately kept its responsibilities narrow: it does not terminate the WebRTC session, but only quickly parses the needed headers, updates minimal flow state and forwards packets onward. The company specifically stresses that it did not need kernel bypass: careful optimization at the level of `SO_REUSEPORT`, thread pinning and fewer unnecessary copies was enough to serve global real-time media traffic with a relatively small relay layer and without abandoning standard WebRTC behavior on clients.

What this means

For users, all of this looks like "voice mode got more responsive," but for the market something else matters more: OpenAI showed how to build mass-market voice AI on top of standard WebRTC, without custom clients and without painful sprawl of network infrastructure. It is a good reference point for developers of real-time assistants, voice agents and products where half a second of latency already ruins the whole user experience.