FilmFlow: Four Agents Build 3D Camera Trajectories from Text and Video

The FilmFlow team has proposed an agent framework that translates a director's brief into an explicit camera trajectory for a dynamic 3D environment. The input can be either a natural-language phrase or a reference video. The output is an executable camera-motion plan that can be rendered. Four agents work inside the system, and the authors tested it on a new benchmark, CinemaBench: 150 Blender scenes, 750 text tasks, and 60 reference videos. In comparison with a set of baseline methods, FilmFlow achieved the best results and received the highest score in user preferences.

Four Roles Instead of One Model

The core idea of the work is that the task is not handed to a single model in a single query. Instead, the cinematographic work is split into roles, as on a film set.

Director is responsible for the outcome: it assembles the executable camera plan. Three specialists assist it. Interpreter extracts the director's intent from the instruction or reference and ties it to the specific scene: what matters most in the frame, where to look, what rhythm is needed. Scout picks the starting viewpoint. Critic evaluates the rendered result and returns visual feedback, after which Director revises the plan.

The last step matters most. A camera trajectory can easily look reasonable as numbers and still break the shot: the subject drifts out of frame, the camera passes through geometry, the movement misses the intended moment. The plan–render–critique–revise loop catches these errors in the image rather than on paper.

Camera Motion as a Composition of Atomic Operations

Director does not output the trajectory in one piece. It builds complex movements from parallel atomic operations that simultaneously control three things: translation, rotation, and framing. For example, an orbit around a character with a gradual approach and a shift of the subject within the frame breaks down into several simple operations running in sync.

This approach gives the plan an explicit, readable structure. You can review it, edit it by hand, or partially reuse it. For tasks that require spatial logic (where objects are, how they move over time, how to hold the composition), this is noticeably more convenient than implicit trajectory generation by a neural network.

CinemaBench: 750 Tasks and 60 References

The benchmark is a separate part of the work. The authors assembled 150 Blender scenes from their own and licensed sources. Each scene has five camera instructions, for a total of 750 text tasks. For some of the scenes, 60 reference videos are added, and the system must reproduce the character of the camera motion in them.

CinemaBench tests two modes: generation from text and generation from a video reference. Previously, such evaluation meant building datasets yourself or relying on general video-quality metrics, which barely distinguish camera work.

Key parameters of the work:

• Inputs: a text instruction or a reference video
• Output: an explicit camera trajectory in a dynamic 3D scene
• Agents: Director, Interpreter, Scout, Critic
• Controlled parameters: translation, rotation, framing
• Benchmark: 150 Blender scenes, 750 text tasks, 60 reference videos
• Result: the best scores among the evaluated baseline methods and the highest user rating

Where This Is Useful in Practice

The most immediate users of the approach are those who need a camera in a three-dimensional scene but have no camera operator or no time to place keyframes by hand. That includes previsualization in film and advertising, animation studios, game cutscenes, and architectural visualization. If a text phrase like "a slow push-in on the hero with a slight drift to the right" can be turned into a trajectory that passes render-based critique, rough work gets faster.

The pattern itself is also useful for developers of agent systems. Splitting the work into an interpreter, an executor, a scout, and a critic that gives feedback on the visual result carries over to other tasks where the output must not only be generated but also seen: lighting setup, editing, scene generation.

Researchers will find CinemaBench useful. The community gets a shared set for comparing methods on text and on references, in Blender format, which means a reproducible render. What matters now is how well the approach holds up on scenes outside the benchmark and how costly the iterative render loop will be in real pipelines. That is where it will become clear whether FilmFlow turns from a research prototype into a working tool.