Anthropic's Fable 5.2 Completes 41% of Real Freelance Jobs — a Year Ago the Leader Managed 2.5%

A year ago, the best AI agent completed 2.5% of real freelance jobs. Now Anthropic's Fable 5.2 reaches 41.3%. These are not classroom exercises or isolated skills, but genuine Upwork projects that a client is ready to accept and pay for. Let's look at what exactly was measured and what follows from it in practice.

How the test works

The Remote Labor Index (RLI) was created by Scale Labs and the Center for AI Safety. It is based on 240 completed projects from 358 vetted Upwork professionals. Together they are worth about $144,000. The average job costs $632.60 and takes a human freelancer almost 29 hours. The tasks vary: video editing, CAD design, game development, data analysis.

The agent gets a virtual workstation with familiar professional software such as Blender, GIMP and Audacity, plus AI tools for generating images, audio and video. An example job: draw a bathroom floor plan, pick furnishing options and produce photorealistic renders.

The work is evaluated by trained experts, and they do it manually. They open the delivered files in professional programs and compare the result with what the human delivered. Attempts to hand the evaluation to another model have so far failed. AI judges can barely operate this software, so they are fooled by work that looks tidy on the outside but is technically broken inside. As a result, they significantly overrate it.

The numbers: a gap over everyone else

The year's trajectory looks like this:

• about a year ago, the leader, Manus, completed 2.5% of jobs;
• in July, Fable 5 was ahead with roughly 16%, more than a fourfold increase in eight months;
• Fable 5.1 scores a little under 18%;
• Fable 5.2 takes 41.3%, about 2.5 times the July result of Fable 5.

Next come Anthropic's Opus 5.5 (21.25%) and OpenAI's GPT 6 Astra (20.83%). Fable 5.2 is almost twice as successful as its closest competitors. Google's best model, Gemini Argon 4, is in fifth place with 17.08%. The older GPT-5.2 and Gemini 3 Pro remain below 3%.

Almost 50 of Fable 5.2's successful deliveries are tasks that no agent in the test had solved before.

Where the model is strong and where it is still weak

According to Scale Labs, Fable 5.2's biggest gains are in audio and music editing, architecture and interior design, and 3D modeling. In addition, the agent copes with long, complex projects on which previous leaders regularly failed.

Industrial design and animation remain nearly out of reach. When agents fail, they usually deliver amateur-quality work, incomplete results, broken files or inconsistent designs.

What to do with this

First, the limits of the figure. 41.3% is the share of jobs in a sample of 240 projects. The agent still fails to complete more than half of the jobs at a professional level. Moreover, the results were published by Scale Labs and have not yet been independently reproduced.

The practical takeaways are as follows:

• Product managers and founders. Evaluate agents on complete "brief to file" tasks rather than on isolated skills. Build in human acceptance review: the agent's output must be opened in working software. Automated checking by another model on visual and file artifacts is still unreliable, as RLI showed.
• Developers of agent systems. The main losses come from broken files and contradictions within a project, not from weak ideas. Add integrity checks for output files and a comparison of the result against the original brief.
• Freelancers. Pressure is strongest in audio, 3D and interiors. Value is shifting toward task framing and quality control, where the agent is still blind.

The main signal is not the number itself but the pace. Growth from 2.5% to 41.3% in a year means that the line at which "the agent can handle it on its own" will have to be revisited more often than once a year.