Windsurf Blog→ original

Claude Haiku 5.5 Arrives in Devin: 58.4% on FrontierCode at One-Eighth the Cost of Sonnet 5

Claude Haiku 5.5 is now available in Devin. On FrontierCode 1.1, a proprietary benchmark that scores the quality and "mergeability" of solutions to real engineering tasks, the model scored 58.4%. That beats Sonnet 5 (56.2%) at roughly one-eighth the cost per task. Haiku 5.5 has also been added to Devin Fusion as a sidekick option. Paired with Opus 5.5 as the lead model, the combination scores 66.2. • Haiku 5.5: 58.4% on FrontierCode 1.1 • Cost per task is roughly 8 times lower than Sonnet 5 • Fusion with Haiku 5.5 as sidekick: 66.2 points

AI-processed from Windsurf Blog; edited by Hamidun News
Claude Haiku 5.5 Arrives in Devin: 58.4% on FrontierCode at One-Eighth the Cost of Sonnet 5
Source: Windsurf Blog. Collage: Hamidun News.
◐ Listen to article

Claude Haiku 5.5 has arrived in Devin, and the headline number of the release is 58.4% on FrontierCode 1.1. This is the Devin team's proprietary benchmark, which evaluates models on real engineering tasks by solution quality and by whether the result can be merged into the main branch. On this scale, the smallest model in the lineup outperforms Sonnet 5 (56.2%), and according to Windsurf, the cost of a single task is roughly eight times lower.

What FrontierCode 1.1 Showed

The leaderboard has eleven entries, and Haiku 5.5 lands in the middle. Above it are GPT-6.1 Sol, GPT-6 Sol, SWE-2, Claude Fable 5.1, Sonnet 5.5 and Opus 5.5. Below it are Kimi K3, Gemini 3.7 Flash, Sonnet 5 and GPT-6 Luna. The gap to the top remains noticeable: Haiku trails Sonnet 5.5 (64.4%) by 6 points and Opus 5.5 (65.3%) by almost 7. But its 2.2-point gap over Sonnet 5 runs in Haiku's favor.

Key figures from the publication:

  • Claude Haiku 5.5: 58.4%
  • Claude Sonnet 5: 56.2%
  • Kimi K3: 58.2%
  • Gemini 3.7 Flash: 56.3%
  • GPT-6 Luna: 56.1%
  • Claude Sonnet 5.5: 64.4%
  • Claude Opus 5.5: 65.3%
  • Cost per task for Haiku 5.5: roughly 1/8 of Sonnet 5

The spread deserves a caveat. Haiku 5.5 and Kimi K3 are separated by just 0.2 points, so on this benchmark they are effectively in the same league. On the cost-per-run charts, the axis runs from $0.02 to $20, and the models appear to differ in price by orders of magnitude. The text gives no exact values for each model, so the only figure that can be relied on is the stated ratio of "roughly eight times."

Fusion: A Weaker Model as Sidekick

The second part of the announcement concerns Devin Fusion. Haiku 5.5 joined the lineup of sidekick models, meaning helpers that work alongside a lead model. With Haiku 5.5 in that role, Fusion scores 66.2 points on FrontierCode. That is higher than Opus 5.5 alone (65.3%). According to the authors, cost and latency also drop.

The logic is straightforward. The expensive model plans and makes the key decisions, while the cheaper one handles the bulk of the work. The 0.9-point advantage for the combination is small, and the benchmark does not disclose confidence intervals. The main argument lies elsewhere: flagship-level quality at lower cost.

You can try it now in Devin CLI, where Opus 5.5 acts as the lead model. Devin Desktop is also available.

Where This Is Useful in Practice

First, the caveats. FrontierCode belongs to Devin itself, we cannot see its methodology or its task set, and the results were published on the company's own blog. That is not enough for an independent comparison. But for choosing a model inside a specific product, such data is useful: the team is measuring what actually happens in its own agent.

What you can do:

  • If your tasks are mostly routine (ticket fixes, small refactorings, tests), it is worth testing Haiku 5.5 instead of Sonnet 5. By the stated numbers, it is no worse in quality and noticeably cheaper.
  • For complex multi-step changes, keep Opus 5.5 as the lead model and add Haiku as the sidekick via Fusion.
  • Before switching your entire workflow, run 20–30 of your own tasks from your repository and compare the share of changes that passed review. That is exactly what the benchmark tries to measure.
  • Calculate cost per task, not per token: that makes the comparison with Sonnet 5 fairer.

If the stated eightfold ratio holds on your code, the savings at high volumes of agentic work will become a noticeable line in your budget.

ZK
Hamidun News
AI news without noise. Daily editorial selection from 50+ sources. A product by Zhemal Khamidun, Head of AI at Alpina Digital.
✦ Hands-On AI Engineering

Build & Ship Production AI Apps with Vibe Coding

Join Zhemal Khamidun's hands-on Vibe Coding Intensive at hamidun.com/vibecoding — master Claude Code, Cursor, and autonomous agentic workflows.

Explore Vibe Coding →Vibe Academy Platform
What do you think?
Loading comments…