LM Arena (lmarena.ai)

LM Arena is a crowd-sourced AI evaluation platform where users compare two anonymous model responses side-by-side and vote on the better one, generating an Elo-based leaderboard of language model quality across open-ended, real-world prompts.

LM Arena is a crowd-sourced AI evaluation platform at lmarena.ai where users compare two anonymous model responses side-by-side and vote on the better one, generating an Elo-based leaderboard of language model quality across open-ended, real-world prompts.

Rankings are computed using a Bradley-Terry or Elo rating model, the same statistical framework used in chess rankings. Because each vote updates ratings relative to the strength of the models involved, the system is robust to the wide variation in prompt difficulty submitted by different users. By 2025 the platform had accumulated tens of millions of human preference votes across text, vision, and coding arenas, constituting one of the largest publicly available human-preference datasets for LLM evaluation.

LM Arena addresses a gap left by static benchmarks: it captures open-ended human judgment across diverse, naturally occurring prompts rather than predefined test sets that developers might optimize for. Its leaderboard has become a standard reference in model release announcements from OpenAI, Google, Anthropic, Meta, and Mistral, and it demonstrably influences which models practitioners select for production deployments. Known limitations include an English-language and technically literate user bias, and over-representation of coding and factual Q&A relative to the full spectrum of real-world applications.

By 2026, the platform has extended beyond text to dedicated arenas for vision understanding and coding tasks, and academic work has analyzed how well Arena Elo scores predict performance on specialized downstream benchmarks, finding moderate but imperfect correlation depending on domain.