OpenAI launches GPT-Realtime-2 and two more voice models in its API

OpenAI has announced expanded voice capabilities in its API. Developers now get access to an updated GPT-Realtime-2 model and two new voice models for speech recognition, speech synthesis and translation.

Three new voice models in the API

The API now includes three models: an updated GPT-Realtime-2 (an improved version of the existing model) and two entirely new ones. They target different tasks: recognizing the user's speech, synthesizing a spoken reply, and translating conversations between languages in real time.

This means developers can now build voice interaction directly into their apps without external speech recognition and synthesis services. Until now, they had to integrate several providers: one for recognition, another for synthesis, a third for translation. Now everything is in one place.

What the new models can do

• Speech recognition (speech-to-text) with support for many languages
• Speech synthesis (text-to-speech) with natural sound and intonation
• Real-time conversation translation that preserves context
• Low latency for interactive applications (streaming)
• Deep integration with GPT-4 for understanding meaning

The models are trained on large volumes of audio data and perform well in English as well as other languages. GPT-Realtime-2 has been updated with better handling of natural speech, better context understanding and faster responses.

Developers get tools to build applications that hear the user, understand what they say and reply by voice. That matters for voice assistants, call centers, learning apps and interactive services.

How it works in practice

Imagine a language-learning app. A student speaks in a foreign language. The API hears it (speech-to-text), sends the text to GPT-4 for checking and correction, then reads the result aloud in a natural voice (text-to-speech). All of this happens in real time.

Or a translation app: a tourist speaks Russian, and the API translates in real time and speaks in English. No delays, unlike Google Translate.

Availability and competition

For now, the models are available only to developers through the API. They will not appear in ChatGPT or other OpenAI consumer apps (at least in the near term). This lets OpenAI put the new capabilities in the hands of specialists, refine them in real applications and then, if needed, integrate them into consumer products.

API prices will be higher than for text models but lower than competitors' (for example, Google Cloud Speech-to-Text). OpenAI competes with Google, Amazon Polly, Microsoft Azure Speech Services and other cloud platforms. Voice APIs are a competitive field where every millisecond of latency and every percentage point of accuracy matters.

The voice interface is ceasing to be exotic; it is becoming the

standard for modern applications.

What this means

Voice interfaces are becoming more accessible. Any developer can now add voice conversation with AI to their app without costly integration of third-party services. This will speed up the arrival of voice AI applications on the market and make interacting with services more natural.