Relay-Bench: даже GPT-5.5 набрала лишь 43,3% на мультидоменном тесте рассуждений

Researchers published the Relay-Bench benchmark on arXiv in July 2026, testing language models' ability to solve linked tasks from different subject domains within a single prompt. The best-performing model tested, GPT-5.5 in xHigh mode, scored just 43.3% — meaning even the strongest system failed to solve even half of the tasks.

What Relay-Bench Measures

Relay-Bench evaluates whether a model can hold multiple reasoning domains together within a single prompt. The test set consists entirely of composite tasks: several single-domain subtasks are chained together into a sequence that cannot be solved by reasoning within just one domain. Each task contains between 2 and 13 subtasks and remains fully text-based — no multimodal input or output is required, which makes the test easier to reproduce on any model.

The authors call the benchmark "unsaturated": current models still have enormous room to grow, the ceiling is far off, and it cannot be closed with a single update. This sets Relay-Bench apart from many older tests, where top models have long been stuck at 90–100% and stopped differentiating from one another.

• The top performer is GPT-5.5 (xHigh) with a score of 43.3%
• Each task is assembled from 2–13 linked subtasks
• Domains: visual reasoning, code, math, information retrieval (web search), problem-solving, general knowledge, data analysis
• The format is text-only — no images on input or output
• Models are allowed to execute code, search the web, and use any available tools

Why Even GPT-5.5 Scored Only 43%

The tasks are deliberately made difficult, so scores are low even for the strongest models. According to the arXiv description, layers of complexity are added to the base chains through prompt encoding and deliberate context bloat — the model must filter out noise without losing the thread between domains. There are no restrictions outside the model harness: the authors explicitly encourage the use of code execution, web search, and all available tools, but even so the 43.3% bar remains unconquered.

The key problem is switching between domains. A model can confidently solve a math subtask and a coding subtask separately, but stumbles when the result of one subtask must be carried over into the next within a single reasoning chain. It is precisely this "relay" that gives the benchmark its name.

"An unsaturated, holistic, text-only benchmark measuring an LLM's ability to complete a set of tasks from different domains within a single prompt," — from the

Relay-Bench description on arXiv.

How This Differs From Ordinary Tests

Relay-Bench does not test individual skills but their combination within a single task. Classic benchmarks measure code, math, or general knowledge separately — here, seven domains are woven into a single chain, and failure at any link breaks the entire task. This format is closer to real-world work scenarios, where an agent has to search for data on the web, do calculations, and write code all within a single assignment.

What This Means

Relay-Bench reveals the gap between models' narrow skills and end-to-end reasoning: LLMs can handle each domain piece by piece, but linking them into a long, multi-domain chain — not yet. The room for growth is enormous: there's still a long way from 43.3% to the ceiling, and it is precisely such "unsaturated" tests that will set the bar for the next generations of models and agentic systems.

Frequently Asked Questions

What is Relay-Bench?

Relay-Bench is a text-only benchmark introduced on arXiv in July 2026. It tests how language models solve composite tasks across seven domains (code, math, web search, data analysis, and others) assembled into a single chain within one prompt.

Which model achieved the best result?

The best result belongs to GPT-5.5 in xHigh mode: 43.3%. This means that even the strongest model tested solved less than half of the tasks, and the benchmark remains "unsaturated" and far from its ceiling.