Nanbeige4.3-33B matches Qwen3.8 27B on the index and beats it on Terminal-Bench: 30% vs 6%

Choosing an open model usually comes down to a single number from a leaderboard, and that number can be badly misleading. Say you are building an agent that runs scripts in a console by itself, and you pick a model with a high overall score that turns out to be weak at exactly that: the command line. A fresh Artificial Analysis breakdown shows such a case.

Nanbeige4.3-33B and Qwen3.8 27B both score 20 on the composite Intelligence Index v5 (which combines nine tests).

But on Terminal-Bench 4.0, where a model solves tasks in a terminal, Nanbeige scores 30% and Qwen only 6%. On expert-level questions the picture is reversed.

The practical takeaway: don't look at the overall score, look at the test closest to your task.

What the model is

Nanbeige is the research team at BOSS Zhipin, a Chinese job-search platform run by the company Kanzhun. Nanbeige4.3-33B is a dense reasoning model with 33B parameters. It is released under the Apache 2.0 license, takes and produces text only, and has a context window of 524K tokens. According to Artificial Analysis, the model is compact enough to run locally on a laptop, although in practice you will most likely need weight compression (quantization) and enough memory.

For comparison, Muse Glimmer (high, 30B) scored just 6 on the same index. Nanbeige4.3-33B and Qwen3.8 27B, both dense, set a new Pareto-optimal point on the "index score vs. parameter count" chart: at this size, there is no better balance yet. But MoE models (where only a fraction of parameters is active for each token) pull ahead: DeepSeek V4.1 Flash 1017 (max) scored 33 with 552B total parameters and 16B active per token.

Same score, different strengths

The key finding in the report is the spread across individual tests. Nanbeige leads here:

• Terminal-Bench 4.0 (command-line tasks): 30% vs 6% for Qwen;
• CritPt v2, hard part (physics problems): 16% vs 4%;
• AutomationBench-AA (workflow automation): 53% vs 48%.

Qwen leads where knowledge and careful work with code and documents matter:

• AA-Briefcase v1.1: 1391 Elo for Qwen vs 1162 for Nanbeige;
• FrontierCode 2.0: 15% vs 11%;
• HLE-Diamond (expert-level questions): 15% vs 7%.

In short, Nanbeige looks more like an executor for the terminal and automation, while Qwen is the better-read conversation partner.

A cautious model and the price in tokens

On AA-Omniscience v1.1, which measures knowledge and the tendency to make things up, Nanbeige scored −4 and Qwen −6. Nanbeige hallucinates noticeably less often: 18% vs 29%. The reason is simple: the model attempts to answer only 29% of questions (Qwen: 43%) and declines the other 71%. As a result, it gives fewer correct answers: 12% vs 17%.

If a wrong answer costs more than silence in your product (support, search over internal documents, legal drafts), this behavior can be an advantage. If users expect some kind of answer, it is more likely a drawback.

There is also a cost drawback. Per index task, Nanbeige spends an average of 158K output tokens vs 110K for Qwen — 44% more at the same overall score. The count includes both the answer and the reasoning, including agent steps with tool calls. For local runs, that means a longer wait; with pay-per-token pricing, a higher bill.

What to do with this

First, the model has no public API yet, either from the team itself or from third-party providers. You can only try it on your own hardware, so the audience is enthusiasts and teams with their own infrastructure.

Second, if you are building an agent for the console, scripts, and routine automation, Nanbeige4.3-33B is worth running on your own tasks. If you need an assistant for complex code or for questions where breadth of knowledge matters, Qwen3.8 27B looks more reliable.

Third, estimate your token budget and latency in advance, and plan what happens when the model declines to answer: re-ask, pass the question to another model, or hand it to a human. The Apache 2.0 license, for its part, allows fine-tuning and commercial use without special conditions.