DirCast directs and commentates esports on its own, beating GPT-5

A live esports broadcast depends on two people: a director, who decides every second whose screen goes on air, and a caster, who talks over that choice without knowing what happens next. DirCast tries to replace both with a single streaming video-language model. It receives the raw cameras of every player, picks the angle itself, and commentates on the action itself, relying only on the past. In tests on professional Dota 2, the model directs better than rule-based approaches and zero-shot models, GPT-5 included, and its commentary wins 29.2% of comparisons against GPT-4o.

A director who watches everything at once

Until now, video-language models have learned to commentate, but almost always over a broadcast that a human has already edited. The director's job stays off-screen: someone has already picked the right camera, and the model only has to describe the picture. The DirCast authors removed that crutch. They set a harder task: the input is every player's camera, the output is a finished broadcast with a voice, and all of it happens in real time with no access to future frames.

This setup matters because the director's and the caster's decisions are linked. The commentary follows wherever the broadcast looks. Split them into two systems, and the commentary risks drifting away from the picture. In DirCast they form a single loop, and the camera choice directly determines what the model sees and says.

A cheap overview and an expensive look

The main engineering idea is how the model allocates compute. Every second, it scans all cameras at low resolution, which is enough to spot where something is happening. Then the model decides whether to switch and to which angle. Only the chosen camera is read at high resolution, before the model speaks.

The second part of the design concerns memory. The context stores what went on air and what was said. The other angles drop out of it. As a result, processing cost stays bounded and a stream can run for as long as needed: the context does not balloon with match time. For live broadcasting this is a fundamental requirement, because Dota 2 tournaments last for hours.

Data and results

For training and evaluation, the authors assembled a dataset of 27 professional Dota 2 matches. It includes a camera for every hero, the real decisions of a human director, and the casters' commentary. This is rare material: usually only the final broadcast is publicly available, not all the raw angles.

Key figures from the paper:

• Task: direct and commentate a stream at the same time, using only past context
• Data: 27 professional Dota 2 matches, one camera per hero, plus the human director's choices and the casters' speech
• Operating mode: a per-second overview of all cameras at low resolution, a switching decision, and a high-resolution read of the chosen angle
• Memory: the broadcast and the speech are kept, the other views are discarded, so cost stays bounded for a stream of any length
• Directing: DirCast outperforms rule-based approaches and zero-shot models, including GPT-5
• Commentary: 29.2% of comparisons won against GPT-4o
• Baseline: the strongest of eleven streaming commentators that watched the human-directed broadcast scored 19.4%

The last comparison is telling. The competitors saw a broadcast already assembled by a professional director, so they got a ready-made hint about what matters. DirCast had no such hint and still came out ahead by ten percentage points.

Where this could be useful

The most direct application is niche broadcasts. Most esports titles and amateur tournaments have no budget for a director and casters, and the stream often comes from a single player or has no commentary at all. A model that takes in the participants' cameras and produces a full broadcast lowers the barrier to entry for such events. For large organizers, it is a potential source of additional streams: alternative broadcasts with automatic commentary in different languages or tailored to a specific player.

The principle itself is useful for developers. A cheap overview of all inputs, an expensive read of the one selected, and a context that forgets the excess suit any multi-camera task: sports broadcasts, video surveillance, robots with a set of sensors. The caveat is obvious: the evaluation covered a single game, Dota 2, and relied on comparisons with models, not on live viewers. The next question is whether the approach carries over to titles with different dynamics, and whether a real audience would accept such a broadcast.