Live Nov 18 · Cohort 3Nov 18, 19:00 MSK — free hands-on AI agents live workshop · Live Cohort 3 starts Nov 23Join Nov 18 Live →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableArtificial Analysis→ original

Meta Muse Ember Matches Claude Haiku 5.5 on Score at One-Third the Cost per Task

Meta has released Muse Ember, a compact model that scores 36 on the Artificial Analysis index, the same as Claude Haiku 5.5 at maximum settings. But a task costs $0.14 instead of $0.41. Per-token prices for the two models are identical; the savings come from cheap caching and roughly one-third as many output tokens. The weak spot is knowledge: the model often refuses to answer.

AI-processed from Artificial Analysis; edited by Hamidun News
Meta Muse Ember Matches Claude Haiku 5.5 on Score at One-Third the Cost per Task
Source: Artificial Analysis. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceArtificial Analysis ↗
Editor-in-Chief
Terminal-Bench 4.040.9% versus 32.8% for Haiku 5.5
Key metric #2In Muse Code it scores 46 versus 37 for Haiku in Claude Code
Key metric #3Refuses to answer 64% of questions, with a hallucination rate of 17.4%

Cheap models usually lose to expensive ones not on flashy benchmarks, but on the monthly bill: an agent takes dozens of steps, and each one drags its context along. Meta decided to strike right there. Its new model Muse Ember scores 36 on the Artificial Analysis Intelligence Index, the same as Claude Haiku 5.5 at maximum settings, but costs $0.14 per task versus $0.41.

Where the savings come from

Ember's per-token price is the same as Haiku 5.5 and GPT-6 Luna: $0.10 per million input tokens and $0.50 per million output tokens. The difference lies in two places. First, cached input costs $0.004 per million, which is 60% cheaper than competitors, and in agentic scenarios cached input makes up the bulk of the bill. Second, at maximum mode Ember generates roughly one-third as many output tokens as Haiku.

Imagine an agent that re-reads the same repository twenty times in a row. With expensive caching, those repeats eat the budget; with cheap caching, they turn into pennies. That is why comparing models by price per token is pointless here: what you need to count is the cost of a completed task.

In xhigh mode, Ember scores 34 for $0.12. The larger Muse Wonder delivers 44 points, but at $6.95 per task. That is roughly 50 times more expensive for eight extra points. For high-volume scenarios, such a trade is rarely justified.

Where the model is strong and where it is not

Its best result is AutomationBench-AA, a benchmark for workflow automation: 68.1%, on par with Muse Wonder. On GDP.pdf Ember beats Haiku (22.8% versus 20.8%), while on AA-Briefcase it falls behind (1521 versus 1574 Elo).

The coding picture is mixed. On Terminal-Bench 4.0 Ember leads: 40.9% versus 32.8%. On FrontierCode 2.0 Haiku leads: 34.9% versus 27.2%. In its own agent, Muse Code, however, the model scores 46 on the Coding Agent Index, while Haiku in Claude Code gets 37 and GPT-6 Luna in Codex gets 31. SWE-Atlas-QnA-AA and Terminal-Bench tip the balance.

A caveat: this compares model-plus-harness pairings, not pure models. Part of Ember's win may be explained by Muse Code being tuned for it.

Knowledge: a weak spot with an unexpected upside

Ember's accuracy on AA-Omniscience is only 21.1%. That is the lowest figure among models of comparable intelligence: Haiku 5.5 gets 34.7%, and Gemini 3.8 Flash (high) gets 50.5%. But the reason is not that the model makes things up. It simply refuses to answer 64% of questions. As a result, its hallucination rate is 17.4%, half that of Haiku.

In practice this means Ember is more honest about admitting ignorance than confidently lying. For a product where the cost of an error is high, this behavior can be more useful than higher accuracy with more fabrications.

What to do with this

The model is available only through Meta's own API and in Muse Code, so lock-in to a single platform remains. All the figures are measurements from one independent lab, and results may differ on your tasks.

The practical takeaway is simple. If you are building agents with long context where cached input dominates, run Ember on your own traces and compare the cost of a task, not of a token. For reference questions without connected search or RAG, it is better not to use it: it lacks the knowledge. And if your agent is oriented toward the terminal and workflow automation, Ember is worth trying as a cheap replacement for Haiku.

Story

Key Questions Answered

Where the savings come from?

Ember's per-token price is the same as Haiku 5.5 and GPT-6 Luna: $0.10 per million input tokens and $0.50 per million output tokens. The difference lies in two places. First, cached input costs $0.004 per million, which is 60% cheaper than competitors, and in agentic scenarios cached input makes up the bulk of the bill. Second, at maximum mode Ember generates roughly one-third as many output tokens as Haiku. Imagine an agent that re-reads the same repository twenty times in a row.

Where the model is strong and where it is not?

Its best result is AutomationBench-AA, a benchmark for workflow automation: 68.1%, on par with Muse Wonder. On GDP.pdf Ember beats Haiku (22.8% versus 20.8%), while on AA-Briefcase it falls behind (1521 versus 1574 Elo). The coding picture is mixed. On Terminal-Bench 4.0 Ember leads: 40.9% versus 32.8%. On FrontierCode 2.0 Haiku leads: 34.9% versus 27.2%. In its own agent, Muse Code, however, the model scores 46 on the Coding Agent Index, while Haiku in Claude Code gets 37 and GPT-6 Luna in Codex gets 31.

What are the key details on Knowledge: a weak spot with an unexpected upside?

Ember's accuracy on AA-Omniscience is only 21.1%. That is the lowest figure among models of comparable intelligence: Haiku 5.5 gets 34.7%, and Gemini 3.8 Flash (high) gets 50.5%. But the reason is not that the model makes things up. It simply refuses to answer 64% of questions. As a result, its hallucination rate is 17.4%, half that of Haiku. In practice this means Ember is more honest about admitting ignorance than confidently lying.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Hands-On AI Engineering · Hamidun Academy

Build & Ship Production AI Agents Without Boilerplate Code

Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.

Start 72h Free Trial →Vibe-Coding Intensive
What do you think?
Loading comments…