Scene-Designer Trained on 35K Design Examples and 15K Critique Examples for 3D Scenes

Researchers have introduced Scene-Designer, an agentic system that builds three-dimensional interiors from a text prompt and evaluates its own output. Training relies on two new datasets: Scene-OTA-35K, with 35 thousand examples of design and error correction, and Scene-OTC-15K, with 15 thousand critique examples. On top of these, the authors added the Scene-GRPO algorithm, in which a trained critic drives the training. The work is published in Hugging Face Daily Papers.

The problem: generating a scene is not enough, it has to be "seen"

Generating 3D rooms from text is limited not by the ability to place a sofa at given coordinates but by common sense. Furniture must not intersect with walls. A dining table needs space where you can walk around the chairs. A bed does not belong in a doorway. According to the authors, existing approaches work in one of two ways. Either they use general-purpose models with no specialized design experience, or they learn from static, ready-made scenes that contain neither a process of searching for a solution nor a process of fixing mistakes. As a result, the model sees the outcome but not how it is reached.

Scene-Designer tries to close this gap with two complementary kinds of experience: how to design and how to evaluate a design.

How the system works

The agent follows an "observation, thought, action" loop. It looks at the current state of the scene, reasons about it, and makes an edit. A separate part of the training teaches critique: the same "observation, thought" loop, but the output is a verdict on quality.

• Scene-OTA-35K — 35K examples in the Observation–Thought–Action format. They teach the model to design a scene and to correct errors efficiently.
• Scene-OTC-15K — 15K examples in the Observation–Thought–Critic format. They teach the model to evaluate a scene on three criteria: physical validity, functional correctness, and layout coherence.
• Scene-Critic — a trained critic that evaluates scenes and is compared against general-purpose models.
• Scene-GRPO — a critic-driven variant of GRPO. It improves design decisions and cuts down on unnecessary trial-and-error cycles.

The last point matters in practice. Agentic pipelines for 3D often spend many steps placing objects, detecting a collision, rearranging, and checking again. If the model makes fewer mistakes on the first try, both the number of calls and the generation time drop.

What the experiments showed

The abstract gives no specific metric values, so it is too early to compare the numbers with other systems. The authors state their qualitative findings as follows. Scene-Designer outperforms baseline approaches on all three axes: physical validity, functional correctness, and layout coherence. Scene-Critic clearly outperforms general-purpose models at evaluating scenes. Critic hints at inference time further improve generation, so the critic is useful not only as a training tool.

One more observation: according to the analysis, design and critique abilities grow together. The better the model designs, the more accurately it evaluates, and vice versa. For agent training this is a familiar idea: the evaluator and the executor share a common understanding of the domain.

Who this is useful for

The nearest applications are generating synthetic environments for training robots and embodied agents, which need thousands of physically plausible rooms. It could also suit game prototypes, virtual interior staging, and layout visualization. For teams building their own agentic pipelines, the paper offers a recipe: do not stop at action trajectories, but train a critic separately and plug it in at inference as a cheap filter.

The limitations are also clear. Only the abstract is available so far, and without full tables, scene descriptions, and data on generation cost, it is hard to judge the real gain. It is worth waiting for the code and datasets to check how well the critic transfers to scenes outside the training distribution.