Thinking-in-Bets: Agents Lock In Claims Before Criticism and Sycophancy Drops

Agents built on language models have an unpleasant habit: the moment a user questions their work, they abandon a correct solution. A simple instruction to "stand your ground" doesn't help, because the agent then starts ignoring genuine mistakes too. Researcher Nghi D. Q. Bui proposes a fix that requires no fine-tuning. The Thinking-in-Bets method requires the agent to write a brief private report on its claims before it ever sees the criticism. Testing on five Gemini models and eight benchmarks showed that false rollbacks during patch review almost disappear.

Why agents cave after the first complaint

The author describes the mechanics this way: after reading an objection, the model devalues its own prior checks in order to agree with the interlocutor. Its reasoning adapts to the user's tone rather than the facts. That's why a blanket ban on yielding works poorly. The agent loses the ability to tell a fair remark from a mistaken one and starts waving off both.

Thinking-in-Bets changes the order of operations. First, the agent records three things in a private note: its main claims, the checks those claims rest on, and its confidence in each one. The author calls this note a Claim Checklist. When an objection arrives, the agent opens the list and acts selectively. It leaves confirmed parts untouched and re-examines only the uncertain ones and those challenged for the first time. The name nods to the idea of betting: a claim with a recorded confidence level is harder to give up without arguments.

Eight benchmarks and five models

The evaluation covers five Gemini models:

• Gemini 3.5 Flash-Lite;
• Gemini 3.5 Flash;
• Gemini 3.6 Flash;
• Gemini 3.8 Flash;
• Gemini 4.0 Argon.

The test suite consists of eight benchmarks:

• SWE-Together, SWE-Interact, and DeepSWE for software development;
• Agents' Last Exam for professional reports;
• Humanity's Last Exam and SycEval for academic reasoning;
• Vals Finance for financial analysis;
• Terminal-Bench for command-line tasks.

The main finding concerns timing. If claims are recorded before criticism arrives, false rollbacks during patch review almost disappear, and finished reports that already passed review remain intact under mistaken pressure. When the same note is written after reading the complaint, the gain nearly vanishes. That means the benefit doesn't come from the checklist itself, but from the fact that it was created before social pressure kicked in.

Coding, token savings, and work without criticism

In multi-step coding, notes written before a response noticeably reduce unnecessary file edits made in reaction to mistaken complaints. There's a practical stake here: an unnecessary diff in a repository costs the reviewer time and can break working code.

The method adds tokens, but the author offers a throttle: the note is written only when the agent itself is uncertain about the result. In that configuration, token spend is lower, while more genuine bugs get fixed than in the baseline. In other words, the savings don't come at the cost of the agent's ability to accept fair criticism.

The effect isn't limited to dialogue. In tasks with no user feedback, it's enough to list the key claims in a draft and recheck the weakest one before submission. This improves both answer accuracy and confidence calibration. The result held across all tested models.

What can be deployed in an agent right now

The method requires no fine-tuning and no access to model weights, so it can be built at the prompt and orchestration level:

• Before submitting any piece of work, the agent saves a hidden list of claims, checks, and confidence levels. The user never sees it.
• The agent reads the user's message only after the list has been saved.
• Upon an objection, the agent first consults the list and revises only the weak or newly challenged items.
• The note-writing step is triggered by low confidence, to avoid inflating the token bill.
• For autonomous tasks, a final recheck of the weakest claim in the draft is useful.

For teams building coding and analytical agents, this is a cheap way to reduce sycophancy without stripping the agent of its ability to correct itself. The project's code and materials are published at thinking-in-bets.github.io.