Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableX: @ArtificialAnlys→ original

Tencent Hy4 Fixes 76% of Real Code Vulnerabilities and Beats DeepSeek by 6 Points

Tencent has released Hy4, and the Artificial Analysis lab tested how well it finds and fixes real vulnerabilities in code. On the CWE-Bench-AA test, it solved 76% of tasks, 6 points more than DeepSeek V4.1 Flash. On the composite Cyber Index, it scores 58%, placing third behind Kimi K3.1 Pro (64%) and GPT-6 Sol (69%). For automated code review, it is a strong option at about $0.92 per task.

AI-processed from X: @ArtificialAnlys; edited by Hamidun News
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceX: @ArtificialAnlys ↗
Editor-in-Chief
CWE-Bench-AAthe agent searches for and fixes real vulnerabilities from the CWE catalog
Key metric #22 points ahead of Grok 4.7 (xhigh) and roughly 11 times cheaper
Key metric #3Each task uses 64K output tokens, 43% fewer than DeepSeek

Imagine a classic flaw hidden in your repository: a database query is built from user input, and through it, someone can pull other users' data. You want an AI agent to find that spot on its own, understand the root cause, and propose a fix. More and more, tasks like this are handed to models, and two things matter here: how often the model succeeds and how much it costs to run it across a large number of tasks. The independent lab Artificial Analysis tested the new Tencent Hy4 model and got a solid answer on both counts: 76% of tasks solved on the vulnerability find-and-fix test, and about $0.92 per task.

What the numbers mean in the test

Artificial Analysis maintains its own Cyber Index, a composite ranking of how well models handle cybersecurity tasks. Hy4 scored 58% there. That puts it third: one point ahead of DeepSeek V4.1 Flash 1017 (max), but behind Kimi K3.1 Pro (64%) and GPT-6 Sol (Daybreak Blue, max) at 69%. GPT-6 Sol carries a trusted-access label in this configuration: access is not granted to everyone, only to vetted users.

Hy4's most notable result came on a separate test, CWE-Bench-AA. This is Artificial Analysis's implementation of the CWE-bench v1 benchmark. The agent is given real vulnerabilities in software and must find and fix them. CWE is a widely used catalog of software weakness types, from buffer overflows to SQL injection. Hy4 solved 76% of the tasks, the top result. It leads DeepSeek V4.1 Flash 1017 (max) by 6 points.

Cost and tokens

Hy4's API pricing is $0.834 per million input tokens and $2.501 per million output tokens. But an agent does a lot of "thinking" and writing, so the cost per complete task matters more for it. Artificial Analysis puts that figure at $0.92. At this price, Hy4 sits on the Pareto frontier of capability versus cost: no model in the comparison is both stronger and cheaper.

Here is how its neighbors in the ranking compare:

  • Grok 4.7 (xhigh) scores 2 points lower and costs roughly 11 times more;
  • DeepSeek V4.1 Flash 1017 (max) trails by 1 point on the overall index and costs about half as much as Hy4.

So Hy4 is not the cheapest model in the comparison, but it is clearly stronger at finding and fixing vulnerabilities.

There is also data on verbosity. Hy4 uses about 64 thousand output tokens per task. That is 43% fewer than DeepSeek V4.1 Flash 1017 (max) and 37% fewer than Grok 4.7 (xhigh). But it writes 49% more than Kimi K3.1 Pro and 56% more than GPT-6 Sol. Fewer tokens mean a faster and cheaper agent, so on this metric Hy4 lands in the middle of the pack.

What to do with this

If you are integrating AI into code review, dependency scanning, or automated patching, Hy4 is worth adding to your shortlist for a trial run, especially when the budget is tight and you need more accuracy than the cheapest models deliver.

The caveats:

  • These results come from one independent lab and its own implementation of the benchmark. On your repositories, languages, and typical bugs, the outcome may differ.
  • On the overall index, Hy4 trails Kimi K3.1 Pro and GPT-6 Sol. If you need the best across all cybersecurity tasks, not just finding and fixing vulnerabilities, your choice may differ.
  • GPT-6 Sol has restricted access, so a direct comparison is not always realistic for an ordinary team.

The practical approach is simple. Take 20 to 50 real bugs or CVEs from your own codebase, run them through Hy4 and a couple of competitors, and compare the share of correct fixes and the final cost per task. That cost will show whether you can afford to run the agent on every commit or only on critical modules.

The news is also telling for the market. Chinese labs are taking an increasingly strong position at the top of applied rankings, and competition is now about the cost of each solved task, not just accuracy.

Story

Key Questions Answered

What the numbers mean in the test?

Artificial Analysis maintains its own Cyber Index, a composite ranking of how well models handle cybersecurity tasks. Hy4 scored 58% there. That puts it third: one point ahead of DeepSeek V4.1 Flash 1017 (max), but behind Kimi K3.1 Pro (64%) and GPT-6 Sol (Daybreak Blue, max) at 69%. GPT-6 Sol carries a trusted-access label in this configuration: access is not granted to everyone, only to vetted users. Hy4's most notable result came on a separate test, CWE-Bench-AA.

What are the key details on Cost and tokens?

Hy4's API pricing is $0.834 per million input tokens and $2.501 per million output tokens. But an agent does a lot of "thinking" and writing, so the cost per complete task matters more for it. Artificial Analysis puts that figure at $0.92. At this price, Hy4 sits on the Pareto frontier of capability versus cost: no model in the comparison is both stronger and cheaper.

What to do with this?

If you are integrating AI into code review, dependency scanning, or automated patching, Hy4 is worth adding to your shortlist for a trial run, especially when the budget is tight and you need more accuracy than the cheapest models deliver. The caveats: These results come from one independent lab and its own implementation of the benchmark. On your repositories, languages, and typical bugs, the outcome may differ. On the overall index, Hy4 trails Kimi K3.1 Pro and GPT-6 Sol.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…