Local models lose to Opus and Codex: 0.21 on a MacBook versus 1.00 for Codex

Can you run a "doomsday agent system" that works without a subscription and without someone else's rules? The author of a Habr study, a developer of the open-daimon project, tested this in practice. He gave the same task to cloud models, a MacBook Pro with an Apple M5, a gaming laptop with an RTX 5080 and an NVIDIA DGX Spark. The cloud stayed far ahead: Codex scored 1.00, Opus 5.5 scored 0.96, and the best MacBook result was 0.21. One important caveat: the published part of the article cuts off at the description of the last run of the dense model on the Spark, so we do not see the author's final conclusions.

Why local models, and how the task works

The author states his motives plainly. Anthropic's consumer terms for residents of the EEA and Switzerland prohibit commercial use of Claude.ai, Pro and Max, so working in Claude Code on a Max subscription formally violates the rules. He also cites cases where Claude refused to help with rooting the user's own LG TV, and where a researcher had his account blocked after analyzing firmware through ChatGPT (the latter is only his claim; OpenAI has not commented). He adds the disappearance of "uncensored" weights from Hugging Face and passport checks through Persona since April.

The test is built on the minibank repository: about 700 lines of Python, a multi-currency ledger, and a DECISIONS.md file with the team's decisions. The prompt reads like a message from a manager: the statement does not match the balance, figure out why, fix it, verify, and check whether the same thing breaks anywhere else. There are seven traps: a decoy converter whose docstring says "use everywhere", an exchange rate corrected retroactively, a month boundary in UTC, a fee on a separate line, a CSV export with ISO timestamps, an old converter that is forbidden to edit, and a test that locks in wrong numbers.

Scoring was hidden from the agent. 20 hidden tests account for half, seven "subtle" conditions for 15%, nine deliberately wrong fixes that the agent's own tests should catch for a quarter, and rules for 10%. The scale is rebased so that 0 means "nothing changed" and a negative score means "broke more than it fixed". The limit is 20 minutes.

What the cloud, the laptops and the DGX Spark showed

Cloud baseline: Codex (GPT-6.1 Sol, medium) finished in 101 seconds at ≈$0.14 per run, Opus 5.5 in 91 seconds at $0.30, and Claude Fable 5.1 in 288 seconds at $1.79. All three read DECISIONS.md, avoided the decoy and updated the traitor test. The only difference was in their own tests: Codex caught 9 of 9 wrong fixes, while Opus and Fable caught 8 each. Repeat runs showed this is noise: Opus and Fable also scored 0.92 in some runs.

MacBook Pro M5 with 32 GB: Qwen3 Coder 30B A3B (MoE, 4-bit) produces 52 tok/s with a prefill of 743 tok/s, but it replaced the converter and kept recalculating the transfer, scoring 0. Qwen3.8 27B (5.4 tok/s) read DECISIONS.md but made no edits at all in 20 minutes. Gemma 4 12B did best: 0.21 in 575 seconds. It used the posted amount instead of recalculating, but missed UTC, ISO time and the fee. That result came in only three of ten runs; in another it scored −0.17. Sampling settings did not reduce the spread.

Gaming laptop with an RTX 5080 (16 GB VRAM) and Qwen3.6-35B-A3B: speed at a 42 thousand-token context rose from 9.5 to 94 tok/s after manual layer placement in LM Studio. All 40 layers sit on the GPU, expert weights for 40% of layers are on the CPU, and the KV-cache uses q8_0. The author attributes the earlier slowdown to LM Studio moving whole layers, attention included, to the CPU. Speed did not improve quality, however: the lightweight pi harness scored 0.31 in 285 seconds, while OpenCode scored 0.14 and missed the time limit. OpenCode sends about 15 thousand tokens on the first step versus 2 200 for pi, and its model produced 55 thousand reasoning tokens versus 11 thousand.

On the DGX Spark (GB10 chip, 128 GB of unified memory, vLLM, NVFP4, a 4-bit format with hardware support in the chip), Qwen3.6-35B-A3B distilled for Opus scored 0.25: 12 of 20 hidden tests and 3 of 7 subtle conditions. This model missed UTC months, ISO time and the fee in all five runs across two machines, two servers and two harnesses. The dense ThinkingCap-Qwen3.8-27B, under a 10-minute limit, left an empty diff even though its reasoning correctly named the main bug. Without a limit it spent 987 seconds and scored 0.92.

Practical takeaway

The author notes that this is a single task, in Python, with usually one or two runs per setup, so the test cannot tell similar local models apart. But noise cannot explain the gap between the cloud and fast local models. Two things stand out in practice. A lightweight harness and manual offload tuning can speed up a local agent several times over, but model size determines quality. And the only local result at cloud level, the dense 27B on the Spark, took almost 16.5 minutes instead of a minute and a half.