Schmidhuber and Pentland Explain Why Temperature Doesn't Make an Agent Curious
Raising the temperature or rewriting the prompt only makes an agent's actions more varied. It doesn't teach the agent to explore. The authors of the paper, who include Schmidhuber, define curiosity as something else: a prediction about the environment, actions taken to test that prediction, and memory that lives outside the context window. Only architectures with explicit hypotheses or a learned world model meet four of the five requirements. So when evaluating agents, the thing to check is not how varied their steps are, but whether their predictions change after experience.
AI-processed from TG: @gonzo_ML (gonzo-обзоры ML статей); edited by Hamidun News
Many teams building autonomous agents fight the same problem: the agent spins its wheels. It repeats similar actions, tries nothing new, and learns nothing about its environment beyond what was already in the prompt. The usual remedy is to raise the temperature, add randomness to sampling, or write "be curious" in the system prompt. A group of authors led by Alfonso Amayuelas, including Jürgen Schmidhuber and Alex Pentland, argues that this remedy doesn't work. The agent becomes more varied, but not more curious.
How curiosity differs from randomness
The authors take the classic idea of curiosity from reinforcement learning and apply it to LLM agents. A simple everyday analogy helps. A tourist in an unfamiliar bazaar can wander the aisles at random, and that is sampling. Or the tourist can hold a hunch ("vegetables are probably cheaper in the back rows"), go check exactly that, and remember the result. Curiosity is the second scenario.
Three things follow from this, and the authors consider them mandatory. First, the agent needs a predictive model of the environment: what will happen if I do this. Second, actions are chosen to improve that model, not merely to earn a reward. Third, conclusions are stored outside the context window. If knowledge lives only in the prompt, it vanishes with the session, and the agent starts from scratch every time. The paper lists five architectural requirements in total; here we describe the main ideas behind them.
What's wrong with popular agents
The authors then review existing classes of agents: search agents, agents with long-term memory, and "AI scientists." The result is disappointing. Only architectures with explicit hypothesis tracking or a learned world model meet four of the five requirements. And none of the systems reviewed sets exploration goals for itself.
The others fail in different ways. Some agents essentially optimize internal token perplexity, that is, the language model's "surprise" at its own text, rather than uncertainty about the real world. Others accumulate episodes in memory but never check their predictions against what actually happened, so memory grows while understanding of the environment does not. Still others do search for solutions, but only within goals that a human set in advance. That makes them good optimizers, not explorers.
What developers can do with this
The paper's practical value is that it offers a checklist and the idea of black-box behavioral tests. These help distinguish real exploration from random wandering. If you have an agent that needs to learn an unfamiliar API, codebase, or interface, you can check whether it formulates testable hypotheses, whether it picks actions that distinguish between them, and whether its behavior changes after a prediction is refuted. If the answer is "no," raising the temperature won't help you, and tokens spent on varied but pointless attempts will go to waste.
The takeaway for product managers is just as concrete. A claim like "our agent explores on its own" deserves the same scrutiny as any other: where is the model of the environment stored, what exactly gets updated, and does it survive the session? External memory with explicit records of hypotheses and test results looks like a far more reliable engineering foundation than hoping the prompt does the job.
Limitations
This is a position paper, not a benchmark: it contains no ready-made figures for quality gains, and its criteria are so far more a guideline than a standard. The authors themselves caveat that on transformers with implicit state, ablations that separate genuine exploration from random behavior don't work: you can't look inside and see whether the model is building hypotheses. Only external behavioral tests remain, and they are less precise. In addition, no one yet meets the requirement that an agent set its own exploration goals, and it is unclear whether that is even needed in applied tasks where a human defines the goal.
Still, the text is valuable because it gives people a language for the conversation. Instead of a vague "the agent isn't creative enough," you can now ask: does it have a world model, is that model updated, and is it preserved beyond the context?
- Nov 20, 2026Standard Bank cuts AI app review from two weeks to two hours
- Nov 20, 2026Firecrawl Local: an agent scraper that is 37x faster and uses 230x less memory
- Nov 20, 2026AI Agent Calls Patients Before Appointments: 10% More Completed Screenings Than With IVR
- Nov 20, 2026In six weeks, Claude derived an exact c/9.1266 bound in black hole physics
Key Questions Answered
How curiosity differs from randomness?
The authors take the classic idea of curiosity from reinforcement learning and apply it to LLM agents. A simple everyday analogy helps. A tourist in an unfamiliar bazaar can wander the aisles at random, and that is sampling. Or the tourist can hold a hunch ("vegetables are probably cheaper in the back rows"), go check exactly that, and remember the result. Curiosity is the second scenario. Three things follow from this, and the authors consider them mandatory.
What's wrong with popular agents?
The authors then review existing classes of agents: search agents, agents with long-term memory, and "AI scientists." The result is disappointing. Only architectures with explicit hypothesis tracking or a learned world model meet four of the five requirements. And none of the systems reviewed sets exploration goals for itself. The others fail in different ways.
What developers can do with this?
The paper's practical value is that it offers a checklist and the idea of black-box behavioral tests. These help distinguish real exploration from random wandering. If you have an agent that needs to learn an unfamiliar API, codebase, or interface, you can check whether it formulates testable hypotheses, whether it picks actions that distinguish between them, and whether its behavior changes after a prediction is refuted.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.