HiSTEVE 1.0 Plays Minecraft With a Mouse and Keyboard: 85.0% on MineStudio Bench

Researchers publishing under the Tencent brand on Hugging Face have introduced HiSTEVE 1.0, an agent that completes long Minecraft tasks the way a human player does: it looks at the screen, presses keys, and moves the mouse. On MineStudio Bench the system scored 85.0%, and on OpenHA it reached 37.4% average success. Beyond atomic tasks, the agent carries out long chains end to end, such as crafting an iron pickaxe from scratch. The paper came out on 22 October 2026 (arXiv 2610.31162) and immediately took second place in the daily Hugging Face Daily Papers ranking.

The gap the authors are closing

Until now, Minecraft agents have split into two camps. One camp offers native visuomotor control, meaning it works with pixels and real button presses, but cannot sustain long tasks without intervention. The other plans well at a high level, but executes the plan through an environment API or a set of predefined skills, and in effect relies on "cheating" privileges such as access to the game's internal state. HiSTEVE tries to join the two halves: autonomous planning, with execution through the same interface a human uses.

The architecture has two levels. At the top sits what the authors call a capability-aware agent harness, an orchestrator that knows what the lower model can do. Below it, a single VLA policy executes all subtasks using vision alone.

An orchestrator that knows the executor's limits

The orchestrator has three working components. Structured memory tracks progress on the task. A VLM planner selects subtasks and restricts itself to the VLA's capabilities: it does not ask the policy to do anything it cannot do. The third component is a policy-coupled task completion detector. It signals a candidate completion, and the agent uses that signal to decide whether to move on, recover from a failure, or replan. The description also mentions evidence-driven execution recovery.

One VLA for gameplay and menus

The main engineering problem at the lower level is that first-person gameplay and GUI work (inventory, crafting) are different interaction modes, and they interfere with each other inside a single model. The authors separate them with two techniques: dedicated RMSNorm branches for each mode and separate token embeddings for mouse movements.

Training runs in three stages: world knowledge injection, imitation learning, and reinforcement learning on environment feedback. According to the authors, the last stage improves reliability and the ability to recover from errors.

System components and metrics

• Agent type: hierarchical Vision-Language-Action, vision-only perception, native keyboard and mouse actions • Orchestration: structured memory, a VLM planner aware of VLA capabilities, a subtask completion detector tied to the policy context • Control: one VLA policy for all subtasks, including GUI • Architectural changes: mode-specific RMSNorm branches, separate embeddings for mouse movement tokens • Training: world knowledge injection, imitation learning, RL with environment feedback • MineStudio Bench: 85.0%, claimed state-of-the-art for atomic tasks • OpenHA: 37.4% average success • Long task: iron pickaxe from scratch • Authors: Zecong Tang, Zian Guan, Yuncheng Guo, Zhanqiu Zhang, Yiwen Guo, Bo Dai

Where this could be useful

Minimal environment privilege matters beyond Minecraft. An agent that needs only a screen and standard input can carry over to any game or desktop application without integrating with an internal API. This is the same logic behind computer-use agents, only on a tougher testbed with procedurally generated worlds and a long horizon.

Two takeaways are useful for agent developers. First, tie the planner to the executor's real set of skills rather than to an abstract "all-powerful" model. Second, separate heterogeneous input modes, such as menus and the game scene, at the level of normalization and embeddings, rather than hoping that one shared policy will cope on its own. The figures of 85.0% and 37.4% so far apply only to specific benchmarks, and independent reproduction will have to verify them.