Artificial Analysis launches AA-PersonalityBench: 36 models, 30 traits, 900 questions

Artificial Analysis has released AA-PersonalityBench, a benchmark that compares models by communication style rather than accuracy. Each of the 36 models answers 900 questions: 30 personality traits, 30 questions per trait. A panel of LLM judges evaluates the answers pairwise, and the results roll up into 8 indices: Expressiveness, Organization, Informativeness, Transparency, Warmth, Conviction, Imagination and Slop. In parallel, 21 formatting characteristics are measured programmatically: length, structure, punctuation and emphasis.

How the test works

The idea is simple. Two models can give equally correct answers, yet people will use them differently. The authors measure how a model writes by default: each question goes in as a single user message, with no system prompt.

Traits are grouped into indices. For example, Expressiveness includes Lexically Varied, Eloquent, Literate, Natural Sounding and Metaphorical. Warmth includes Friendly, Humorous, Sincere and Emotionally Intelligent. Conviction includes Direct, Confident, Opinionated and Constructive Pushback. The separate Slop index consists of a single trait, Slop. Transparency includes Sycophantic, inverted: the more sycophantic the model, the lower this index.

The judge sees the trait, its definition, the question and two anonymous answers. The order is random, to rule out position bias. The verdict is A, B or a tie. Each comparison is scored by one judge from the panel, and the load across models is balanced. Elo is computed from the verdicts with an anchor model set at 1000, and 95% confidence intervals come from resampling. If intervals overlap, the difference cannot be considered proven. An index is the mean of the trait z-scores, converted to a percentile from 1 to 100.

The full question set is closed to guard against contamination. Only two examples per trait are public.

What the first numbers show

The authors stress that a trait can be a plus, a minus or neutral depending on taste, so the benchmark measures how strongly a trait shows, not quality.

On the Friendly trait, Claude Opus 5.5 leads with an Elo of 1077 (interval 1049–1107, percentile 69). Next come Gemini 4 Argon (1059), Claude Fable 5.1 (1047), Claude Sonnet 5.5 (1041) and GPT-6 Astra (1001). The intervals for Opus and Gemini overlap, so there is no confident gap between them.

On the Warmth index, Claude Fable 5.1 is first (94), followed by Opus 5.5 (80), Gemini 4 Argon (71), GPT-6 Astra (68) and Sonnet 5.5 (65). The profiles differ: Gemini scores high on Humorous (z +1.8) but below zero on Sincere (−0.6). Fable's strongest trait is Sincere (+1.8).

The most striking differences appear in the formatting metrics. Average values per answer across all 900 questions:

• Length in words: Gemini 4 Argon 790, Claude Fable 5.1 576, Opus 5.5 571, Sonnet 5.5 450, GPT-6 Astra 352. • Lists: Gemini 3.9, Opus 2.9, Sonnet 2.8, Fable 1.7, Astra 1.5. • Colons: Gemini 14.5, Opus 7.4, Fable 7.1, Sonnet 5.4, Astra 5.2. • Bold text: Gemini 17.2, Opus 11.8, Sonnet 10.0, Astra 8.8, Fable 8.7.

Gemini 4 Argon writes twice as much as GPT-6 Astra and marks up its text more heavily than any other model. GPT-6 Astra is the most concise.

Model families by character

A separate section groups the models by hierarchical clustering on the correlation of 30 trait-Elo z-scores. The result is two large clusters. The first contains GPT-6 Luna, GPT-6.1 Sol and GPT-6 Astra along with Grok 4.7, with Claude, GLM-5.3 and Kimi K3 nearby. The second contains Muse Spark Open, Muse Wonder, Gemini 3.8 Flash, Gemini 4 Argon, Mistral Large 4 Preview, the Qwen3.8 family, MiMo-V2.6-Pro and others.

The closest pairs are predictable: Opus 5.5 and Sonnet 5.5, GPT-6.1 Sol and GPT-6 Astra, GLM-5.3 and Kimi K3. More curious is that Grok 4.7 is closer in character to the GPT-6 line than to anything else. Claude Fable 5.1 landed on the same branch as GLM and Kimi.

How to use the results

The benchmark is useful for choosing a model for a specific product. For customer support or a coaching app, look at Warmth and Sycophantic; for analytical reports, at Informativeness and Transparency. If you need a short answer without markup, the length and list metrics will tell you more than any intelligence ranking.

Keep the limitations in mind. The judges are themselves LLMs, so their preferences may influence the outcome. Some intervals overlap, and the closed question set cannot be re-verified. The test runs without a system prompt, so in a product with a tuned tone the picture will be different. Artificial Analysis publishes current tables on the benchmark page.