Microsoft and Hugging Face trained Qwen3.8-27B on database-state rewards: from 38 to 107 tasks

Microsoft and Hugging Face have published ThinkingBox-Training, a follow-up to the benchmark that catches agents that report success while writing the wrong data to the database. The idea is simple: if a final-state DB check returns 1 or 0, that is already a ready-made reward function. After 50 steps of GRPO, Qwen3.8-27B raised its average success rate (pass@1) from 51.70% to 60.78%. The number of tasks solved in all twenty attempts grew from 38 to 107, or from 7.5% to 21.1% of the 507 ThinkingBox-Bench tasks.

A reward from database state, not from the text of the answer

Each ThinkingBox task spins up a real backend and gives the agent a set of MCP tools and a simulated user with a closed context. At the end, executable checks inspect which records are in the database. The sequence of calls is not scored, so a model can find a path that is absent from the reference annotation. Most rewards are deterministic, and a small share of narrow rubrics is handed to a GPT-5 Chat judge.

A corporate hotel booking is a good illustration. An employee needs a room downgrade, but policy requires manager approval. The base model changed the booking and issued refunds without approval in 15 of 20 attempts. After RL, the booking and payment state stayed untouched in all 20 attempts, and all expected records matched in 19.

Why task selection matters more than the optimizer

GRPO has an arithmetic limitation. If all 8 rollouts fail ([0,0,0,0,0,0,0,0]) or all succeed, within-group centering zeroes out the advantage and there is no gradient. Mixed groups such as [0,0,0,1,1,1,1,1] are needed. For a stateful agent, such empty groups are expensive: every rollout requires an isolated backend, a live MCP session and a separate model to play the user.

The authors introduced offline policy-relative difficulty. The success probability of each candidate task is estimated once on the base policy, and the pool is then frozen for the entire run. They deliberately did not re-probe difficulty during training. No task from ThinkingBox-Bench made it into training. The domains, databases, policies and most tools in the test set are new, although the shared framework, user simulator and task style match, which limits how strongly the results can be read as evidence of transfer.

The reference run configuration: full-parameter RL on 187 tasks, token-mean GRPO, 18 groups of 8 rollouts (144 episodes per step), on 24 H100s. Training is synchronous with parameter and optimizer offload, AdamW, lr 1e-6, and zero weight decay. Zero-variance groups were replaced following the DAPO scheme. System errors such as a backend timeout were not counted as agent failures; the group was regenerated instead. The code that interprets Qwen's outputs is shared between training and evaluation (trainer/core/tb_interpret.py), since otherwise the reward would be measuring a different agent.

Table: average success versus repeatability

Pass@1 across all 507 tasks and the number of tasks with 20/20:

• Claude Opus 5.5 — 67.16%, 241 tasks (47.53%) • Claude Opus 5 — 66.50%, 241 (47.53%) • GPT-5.4 — 65.36%, 128 (25.25%) • GPT-5.6 Sol — 61.91%, 82 (16.17%) • Qwen3.8-27B after RL — 60.78%, 107 (21.10%) • Claude Sonnet 4.6 — 59.19%, 102 (20.12%) • GPT-6 Astra — 58.31%, 231 (45.56%) • Kimi-K3 — 57.37%, 68 (13.41%) • Qwen3.8-27B base — 51.70%, 38 (7.50%)

GPT-5.6 Sol scores higher on pass@1 but reliably solves 25 fewer tasks. Opus 5.5 beat Opus 5 on the average by 0.66 percentage points, but the two are equal on reliability. Pass@1 and 20/20 grew in all five domains, and the leader in 20/20 gain is retail (+24.5 pp). Pass@20 fell from 453 to 446 tasks: it grew in neobanking and consulting and declined in retail, travel and auto insurance (where it fell by 7.0 pp). The authors describe this as sharpening: the distribution shifts toward trajectories that always work, and rare paths drop away.

The economics of a reliable task

The cost of the 507×20 campaign fell from $925.80 to $855.58 (−7.6%), and the price of a successful attempt fell from $0.177 to $0.139 (−21.4%). The "cost per reliably solved task" metric (campaign cost divided by the number of tasks with 20/20) dropped from $24.36 to $8.00 (−67.2%). For comparison: GPT-5.4 — $6.80, GPT-6 Astra — $7.45, Claude Opus 5.5 — $7.80. This is a comparative serving index, not a cloud bill. It includes inference but does not account for training and task probing, and the deployed model size stays the same.

For teams building agents on top of production databases, the takeaway is practical. If you have an executable state checker, it works both as a metric and as a reward. Start with task selection: tasks the model always or never solves burn GPU-hours and backends for nothing. Evaluate by 20/20, not pass@1. If coverage of rare scenarios matters to you, factor in the cost: repeatability came at the price of part of pass@20.