OpenAI's Chief Scientist Proposes an AI Speed Limit: GPT-4 Perplexity at 0.91, Now Below 0.8
To limit how fast AI develops, the industry needs a shared speedometer. Jakub Pachocki of OpenAI proposes perplexity, which measures how accurately a model predicts the next token (a piece of a word). The lower the number, the stronger the model. GPT-4 scored about 0.91, and models are now below 0.8, a level at which, by his account, strong cybersecurity and math capabilities emerged. Companies could agree not to cross that threshold until they learn to control their models.
AI-processed from TechCrunch; edited by Hamidun News
Imagine a speed limit on a road, but cars have no speedometers: each driver judges "fast" in their own way. That is roughly the situation the conversation about slowing AI development has found itself in. OpenAI Chief Scientist Jakub Pachocki has proposed such a speedometer: perplexity. The metric shows how accurately a language model predicts the next token (a piece of a word) in ordinary text. The lower it is, the better the model. GPT-4 scored about 0.91, and OpenAI's models have now dropped below 0.8. Below, we look at why this particular metric matters and what follows from it.
Why a "speedometer" is needed at all
The topic of slowing AI has stopped being the domain of activists alone. A former OpenAI and Anthropic researcher said that companies are "playing with our lives." Anthropic's IPO filing states plainly that the technology may carry "catastrophic or existential risks for society." Anthropic CEO Dario Amodei published a plan to "set the pace of the frontier," which, judging by the reaction, was backed by Sam Altman and Elon Musk. Amodei proposed that companies agree on shared safety standards and limits on the speed of uncontrolled progress.
But to set a speed limit, you first have to agree on how to measure it. That is the question Pachocki's post addresses. Altman called it "the most important thing OpenAI has published in a while."
Why perplexity instead of familiar benchmarks
Tests of applied skills (solving problems, writing code, olympiad math) go stale quickly: models hit a ceiling, and the test stops distinguishing a new version from the previous one. Perplexity, according to Pachocki, remains "informative" while other benchmarks come and go. Put simply, it measures how rarely a model is "surprised" by the actual continuation of a sentence. It has no connection to the startup Perplexity, which simply shares the name.
The post includes a chart showing how perplexity fell across OpenAI's models starting with GPT-3.5. Pachocki points to two markers. GPT-4, at around 0.91, could already reason productively through chain-of-thought. That prompted the team to deliberately develop this skill and build modern reasoning models. Now, with the prediction loss (called "loss" in the post) below 0.8, he says, "transformative" capabilities have appeared in cybersecurity and mathematics.
From there comes the idea: perplexity could become the basis of a "common intelligence benchmark," and on that basis companies would set "shared limits." They would pledge not to cross a certain threshold until the ability to understand and monitor AI behavior reaches a specific mark. The pause is tied not to the calendar but to how well people have learned to see what is happening inside models.
What remains off the table
For now, this is one company's proposal, not an industry agreement. The account from TechCrunch names no specific threshold, no mechanism for verifying that a competitor is not circumventing the limit, and no answer to who would calculate the numbers.
The practical questions are obvious. Perplexity is computed on a specific set of texts and with a specific tokenizer, so an honest comparison of models from different companies requires a shared test corpus. Closed labs would have to share at least this measurement. Moreover, the metric relates to real skills only indirectly. The cybersecurity and mathematics example shows a correlation, not a guarantee: which exact capabilities will appear at the next drop is unknown in advance.
What this means for developers and product teams
For now, the takeaway is mostly strategic. If the idea becomes an industry standard, the pace of releasing new frontier models could depend on a publicly agreed threshold, not only on hardware readiness and budget. That would affect the roadmaps of API providers and how quickly teams can plan around the next generation of models. Three things are worth watching: whether OpenAI names a specific number, whether Anthropic and Google join the scheme, and whether an independent body emerges to verify those figures.
- Nov 22, 2026OpenAI Reveals Its Models' Next-Word Prediction Error: From 0.99 for GPT-3.5 to Below 0.8
- Nov 22, 2026Ahrefs: A Brand With 10x the Web Mentions Gets Only 3.7x More in AI Answers
- Nov 22, 2026Axiom Math's AI Writes 43.1 Million Lines of Lean Proof in 24 Days
- Nov 22, 2026Yandex Grant Funds an LLM Engine: 35B Model on RTX 2080 Ti Runs Three Times Faster Than llama.cpp
Key Questions Answered
Why a "speedometer" is needed at all?
The topic of slowing AI has stopped being the domain of activists alone. A former OpenAI and Anthropic researcher said that companies are "playing with our lives." Anthropic's IPO filing states plainly that the technology may carry "catastrophic or existential risks for society." Anthropic CEO Dario Amodei published a plan to "set the pace of the frontier," which, judging by the reaction, was backed by Sam Altman and Elon Musk.
Why perplexity instead of familiar benchmarks?
Tests of applied skills (solving problems, writing code, olympiad math) go stale quickly: models hit a ceiling, and the test stops distinguishing a new version from the previous one. Perplexity, according to Pachocki, remains "informative" while other benchmarks come and go. Put simply, it measures how rarely a model is "surprised" by the actual continuation of a sentence. It has no connection to the startup Perplexity, which simply shares the name.
What remains off the table?
For now, this is one company's proposal, not an industry agreement. The account from TechCrunch names no specific threshold, no mechanism for verifying that a competitor is not circumventing the limit, and no answer to who would calculate the numbers. The practical questions are obvious. Perplexity is computed on a specific set of texts and with a specific tokenizer, so an honest comparison of models from different companies requires a shared test corpus. Closed labs would have to share at least this measurement.
Build & Ship Production AI Agents Without Boilerplate Code
Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.