GPT-6 Astra beats Claude Opus 5.5 at “Mafia”: a rating of 1859 versus 1549
Dwarkesh Patel spent $20,000 on API calls and put 11 models through Blood on the Clocktower—a more complex version of Mafia. One model controlled all the good players, another controlled the evil players, and then they switched sides, allowing deduction and deception to be measured separately. After 386 rounds, GPT-6 Astra finished first with a rating of 1859; Claude Opus 5.5 scored 1549.
AI-processed from TG: @ai_machinelearning_big_data (Machinelearning); edited by Hamidun News
Most benchmarks test whether a model can solve a problem. Almost none test whether it can avoid taking someone at their word. For agents that negotiate, read other people's reports or interact with other agents, that is a much more practical skill. Podcaster Dwarkesh Patel decided to measure it directly: he spent $20,000 on API calls and staged a Blood on the Clocktower tournament featuring 11 models.
How the game works
Blood on the Clocktower is a more complex counterpart to Mafia. A moderator assigns secret roles and plays for neither side. The evil players are a minority: a demon and two minions. They know who everyone on their side is from the start. The good players outnumber them but know nothing, not even who is on their own team.
The good players have abilities that secretly provide truthful clues at night. For example, one player learns another's role, while another discovers how many evil players are sitting next to them. During the day, everyone talks, and the good players must piece the clues together and identify the demon. Meanwhile, the evil players pass off fabricated clues as real ones. At the end of the day, a vote determines whether one suspect is executed or no one is. Good wins if the demon is executed; evil wins if only two players remain alive.
To keep the experiment controlled, one model played the entire good team while another played the entire evil team, and then they switched sides. This made it possible to measure two skills separately: detecting lies and deceiving others.
What the tournament showed
After 386 rounds, GPT-6 Astra took first place with a rating of 1859. Claude Opus 5.5 scored 1549. The gap appeared in both skills. Their lie-detection ratings were 1940 and 1500, respectively; their deception ratings were 1806 and 1585.
Some games look almost human. Playing evil, Astra coolly sent one of its own allies to execution to gain trust and lead its side to victory. Playing good, Opus 5.5 took an opponent at their word in a situation where GPT suspected a trick, double-checked the evidence and identified the deceiver.
Looking at models released this year, the picture is this: OpenAI is progressing faster at detecting disinformation, while Anthropic's gains in that area are more modest. Both companies are advancing at roughly the same pace in deception.
Limitations and what to do about them
First, the caveats. This is one game, one creator and one set of rules. A Clocktower rating is not a measure of a model's overall “reliability,” and a difference of hundreds of points does not guarantee an equivalent gap in your product. Moreover, each model controlled an entire side at once, something that rarely happens in real life.
The practical takeaway for agent builders: if your agent accepts information from other agents, users or external sources, test how easily it can be deceived. Taking someone at their word cost Opus 5.5 games here; in production, that behavior will translate into incorrect actions. A model's ability to double-check facts deserves its own evaluation, alongside the quality of its answers to static questions.
The second takeaway concerns security. Models that lie well already exist, and both companies are improving at this skill at roughly the same rate. Multi-agent systems therefore need independent checks rather than reliance on another agent's claims.
A third positive: the project is open. The code is on GitHub, and there is a benchmark website and an online version where you can play against AI yourself. For teams choosing a model for agent workflows, this offers an inexpensive way to add tests for manipulation and resistance to deception to their evaluation suite.
- Nov 12, 2026Jensen Huang steps down as Nvidia CEO: company loses $1.4 trillion in two weeks
- Nov 12, 2026Claude in Chrome requires approval for every click on category 3 sites
- Nov 12, 2026AWS Quick Gets Six Agents for Product Managers: From Customer Complaints to a Spec
- Nov 12, 2026Anthropic to Cut Off Claude for Russian Companies From November 12, Even Those With Foreign Legal Entities
Key Questions Answered
How the game works?
Blood on the Clocktower is a more complex counterpart to Mafia. A moderator assigns secret roles and plays for neither side. The evil players are a minority: a demon and two minions. They know who everyone on their side is from the start. The good players outnumber them but know nothing, not even who is on their own team. The good players have abilities that secretly provide truthful clues at night.
What the tournament showed?
After 386 rounds, GPT-6 Astra took first place with a rating of 1859. Claude Opus 5.5 scored 1549. The gap appeared in both skills. Their lie-detection ratings were 1940 and 1500, respectively; their deception ratings were 1806 and 1585. Some games look almost human. Playing evil, Astra coolly sent one of its own allies to execution to gain trust and lead its side to victory.
What are the key details on Limitations and what to do about them?
First, the caveats. This is one game, one creator and one set of rules. A Clocktower rating is not a measure of a model's overall “reliability,” and a difference of hundreds of points does not guarantee an equivalent gap in your product. Moreover, each model controlled an entire side at once, something that rarely happens in real life. The practical takeaway for agent builders: if your agent accepts information from other agents, users or external sources, test how easily it can be deceived.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.
