WorkFactory: 112,000 office tasks lift Qwen from 879 to 1565 Elo

A group of researchers from Renmin University of China, the Shenzhen Institute of Advanced Technology (SIAT) and Shanghai Jiao Tong University has built a factory for training tasks for AI agents that do office work. They assembled 112,686 tasks for 612 professions, ran them through a strong teacher model and fine-tuned Qwen on its recorded trajectories. The best result: the rating of the 35B-parameter version (3B of them active) on GDPval-AA v2.1 rose from 879 to 1565 Elo. GDPval-AA is a set of real work assignments from a range of professions. Models complete them, the finished documents are compared pairwise, and an Elo rating, as in chess, is calculated from the wins and losses.

Why a bad task ruins training

The most common way to teach an agent to work is to show it successful task runs. A strong model solves a task, its actions are recorded, and a smaller model is trained on those recordings. That means training quality is capped by the quality of the tasks themselves.

An example from the paper: an agent has to choose a supplier. Demand forecasts sit in one file and vendor price lists in another. But nowhere does it say whether delivery is included in one supplier's price. Without that, the total cost cannot be compared fairly. The teacher either invents an assumption or abandons the work halfway. Either way, the recording teaches poorly.

Fixing this head-on does not work: if you simply write finished calculations into the task, there is nothing left to learn. You need a boundary. All the starting conditions are in place, while the calculations and conclusions are left to the agent.

How the factory works

First, ideas for future tasks are collected from three sources:

• descriptions of professions and work situations; • skill documentation, from which chains of operations are assembled; • real work documents, from which the team reconstructs what task a client might have set.

Another model filters out ideas that are illogical or unlike real work. Then an information-support graph is built for each idea. A planner works backward from the result: which calculations and judgments the final document needs, and which data each of them requires. Each piece of data is assigned a location: in the text of the request, in a reference file, in an external source or in the future output. Everything the agent has to compute itself stays hidden.

Files are generated from the graph. Automated checks and checks by another model verify whether the output matches the plan. Tasks that still fail after several attempts are discarded.

Trial run and repair

Even after that, gaps remain in the files. So a trial agent attempts the task, seeing only the request and the files, with no graph and no plan. It reports obstacles and separates them from its own mistakes and from legitimate choices. Only what blocks completion gets fixed.

The repair model edits the request and files but does not change their number, types or structure. In the supplier example, it adds the missing condition: delivery from supplier C costs 0.15 dollars per box. It does not do the calculation for the agent. The edit is then checked again to see whether the meaning of the original work survives. If not, the task is discarded. If so, a new trial run starts. The cycle ends when there are no blockers or the attempt limit is exhausted.

Tasks that pass go to an independent teacher, GLM-5.3, which also sees neither the graph nor the plan. Its recordings are filtered for repetitive reasoning, looped tool calls and empty results. For the longest 5% of trajectories, documents are additionally compared pairwise and the best ones are kept.

What the experiments showed

• Three WorkPilot models were trained on top of Qwen: 4B, 27B and 35B-A3B. All three beat their base versions on four professional-work benchmarks. • WorkPilot-35B-A3B rose from 879 to 1565 Elo on GDPval-AA v2.1. • By the authors' local evaluation, WorkPilot-27B reached 1612 Elo. That is within the range of the frontier models it was compared against. The authors produced this estimate themselves, and there is no independent verification yet. • Gains from adding more data (up to 110 thousand tasks) slow down, and differently for different models. Trajectory length first grows, then shrinks. • In controlled comparisons, both planning and repair reduced the number of task defects found by the trial agent and improved final quality.

What this means in practice

For teams that train agents, the takeaway is simple: task volume does not guarantee quality, and you need to check that a task can be solved at all from the materials provided. A trial run in a sandboxed environment works here like a pre-release test. For business, this is a step toward small models that handle office tasks at the level of much larger ones. The authors have published some of the tasks, the data and the weights on GitHub (IQuestLab/WorkFactory). One caveat: the quality estimates rely largely on the authors' own tests and on document comparison by model judges, so testing on other people's tasks will be decisive.