Artificial Analysis launches HealthBench: GPT-6 Astra leads with 66.0%

Artificial Analysis has added two OpenAI medical benchmarks to its rankings: HealthBench and HealthBench Professional. Both test how models respond in realistic health conversations, and both grade answers against rubrics written by physicians. On the professional set, GPT-6 Astra (max) leads with 66.0% on the length-adjusted score. On the general HealthBench, Claude Sonnet 5.5 (max with fallback) is ahead with 62.1%. The gap between raw and adjusted scores reshuffles the table considerably.

How the tests work

HealthBench includes 5000 conversations with both ordinary users and healthcare professionals. They are split into seven themes, ranging from emergency referrals to global health. Each rubric criterion belongs to one of five axes: accuracy, completeness, communication quality, context awareness, and instruction following. GPT-4.1 grades the answers.

HealthBench Professional is smaller: 525 conversations, all of them with clinicians. It covers consultations, writing tasks, and research work, as well as deliberately difficult and adversarial cases. GPT-5.4 (low) acts as the judge.

Artificial Analysis publishes two metrics: the raw share of points earned and a length-adjusted score based on OpenAI's methodology. The adjustment lowers the score of answers longer than 2000 characters and raises it for shorter ones.

HealthBench Professional: OpenAI leads, length is penalized

OpenAI models take the top spots on the professional set. The entire Pareto frontier of score versus cost per task consists of them as well.

• GPT-6 Astra (max): 66.0%, $0.27 per task
• GPT-6.1 Sol (max): 65.1%, $0.047
• GPT-6 Sol (max): 62.3%, $0.036
• GPT-6 Luna (max): 61.9%, $0.004
• Claude Sonnet 5.5 (max with fallback): 63.0%

GPT-6 Luna trails Astra by 4 points at less than 2% of its cost.

Response length changes the picture. Claude Opus 5.5 and Claude Fable 5.1 write about 5900 characters on average, nearly three times the threshold, and lose more than 11 points after adjustment. Opus 5.5 (max with fallback) has the highest raw score, 71.4%, but drops to sixth place in the adjusted ranking with 59.9%. GPT-6 Sol (max) is the only model that stays within the threshold (about 1600 characters), and its score is the only one that rises after adjustment.

HealthBench: cheap models nearly catch up with expensive ones

On the general set, Claude Sonnet 5.5 takes first place (62.1%), followed by DeepSeek V4.1 Flash (max, 60.9%) and GPT-6.1 Sol (max, 58.5%). The Pareto frontier here consists of three models: GPT-6 Luna (54.6% at $0.002 per task), DeepSeek V4.1 Flash (60.9% at $0.005), and Claude Sonnet 5.5. The latter costs about $0.08 per task on average, roughly 16 times more than DeepSeek V4.1 Flash for a gain of about 1 point.

The breakdown by axis shows the price of brevity. GPT-6 Sol (max) writes the shortest answers, about 1000 characters, and gets the best raw communication-quality score (81%). But it has the worst completeness result, 34%. Claude Sonnet 5.5 leads on completeness with 63%.

How to read these numbers when choosing a model

First: look at both metrics. The raw score shows how much clinically relevant information the model produced. The adjusted score reflects how well the answer suits a real conversation, where a long wall of text does harm. If you are building an assistant for physicians who need completeness and reference depth, you can treat the length penalty as soft. For a patient-facing chat, it matters more.

Second: for high-volume use cases with heavy request traffic, economics decides. At $0.002–0.005 per task, GPT-6 Luna and DeepSeek V4.1 Flash deliver results close to those of expensive models. The costly Astra is justified where every point on difficult clinical cases matters.

Third: OpenAI created the benchmark, and OpenAI models serve as the judges in both tests. This does not invalidate the results, but before deployment you should run your candidates on your own data and with physicians involved. The ranking helps narrow the choice; it does not replace clinical validation.