Researchers release a dataset of 100,000 illustrated articles and a seven-metric benchmark

When an AI model gets the request "explain how to cook risotto," a good answer is more than text. It needs steps and photos showing what the rice looks like at each stage. Models that can both write and draw are called unified multimodal models. But almost nobody has systematically tested how well they alternate text and images within a single response (the paper calls this interleaved generation). Researchers from Nanjing University, Meituan, Imperial College London and Nanyang Technological University have released IntReal. It is a dataset of 100 000 real examples with 538 452 images, plus a benchmark with seven metrics that show exactly where a model is weak.

Where the data came from and why 90% was thrown out

The authors collected more than a million raw examples from seven sources that permit public collection and redistribution. They sorted the material into four categories by the role that text and images play:

• planning: decisions tied to specific goals and constraints, for example a travel route with maps and photos; • knowledge: encyclopedic articles in which an image anchors an abstract concept to a real object; • instructions: cooking and other tutorials in which images show intermediate states; • stories: narratives in which images create characters and scenes (the idea was inspired by the SEED-Story project).

Cleaning then ran in four stages. First, the authors dropped images with absurd aspect ratios, low resolution or unsupported formats. Next, the RALI quality-assessment model removed bad images, while fine-tuned SigLIP2-based classifiers flagged watermarks and faces. The DINOv3 model removed near-duplicate images within the same example. In the third stage, the language model Qwen3.5-397B-A17B wrote captions for the images and edited the surrounding text. The original text takes priority over generated text, so that facts are not lost. In the fourth stage, the same model wrote a prompt and a reasoning trace for each example, so the data can be used for fine-tuning right away.

A single defect in a long example causes the whole example to be discarded. As a result, roughly 10% of the collected material remained, and the categories were balanced by volume.

Seven metrics instead of one overall score

Usually a long answer with images is handed to a single multimodal judge model. The authors argue that this makes it impossible to see where a model goes wrong, and that a generalist judge is not always competent in the details. In IntReal, each skill has its own expert.

The benchmark is built on 600 examples from the dataset, 150 per category. For each example, a model runs in two modes. In the first, it generates the entire answer from scratch. In the second, it is given the real context of text and images and asked to draw the next image. Each model needs more than 6 000 generated images embedded in text, which is why the authors deliberately kept the set compact.

The seven metrics:

• text richness: a language model compares the answer with the real text pairwise; • factual accuracy: the same model looks for errors, contradictions and logical gaps; • image aesthetics: an image quality assessment model; • repetition: DINO feature similarity between adjacent images, where similarity that is too high means the model drew the same thing twice; • image count: the model must produce as many illustrations as the real sample has; • context consistency: a multimodal model checks whether visual requirements, derived in advance from the preceding text and images, are met; • object consistency: similarity to a reference image via DINO, so that a story's hero or a recipe's ingredient does not change from frame to frame. This is computed only for stories and instructions.

The authors also validated the benchmark itself. The maximum pairwise Spearman correlation between metrics is below 0.42, which means they measure different things. Agreement with human judgments is 83.3%. When the expert models are swapped out, the results stay consistent, with a correlation above 0.9.

What the experiments showed

The authors compared four families of models. For one, the open checkpoint cannot interleave text and images on its own, so it was evaluated only after fine-tuning on IntReal-100K. Another can interleave but has no released fine-tuning methods, so it was used as is. The third, SenseNova-U1, supports both scenarios and comes in two variants: 8B-MoT and A3B-MoT. In both, the generation branch has 8 billion parameters, while the understanding block has either 8 billion parameters or 30 billion with a mixture-of-experts architecture. The fourth model is closed and available through an API.

Standard supervised fine-tuning on IntReal-100K improved SenseNova-U1's text generation but worsened some image metrics. This is a long-standing problem for such models: text and images pull the weights in different directions.

The fix turned out to be unexpected. The authors applied Flow-GRPO, a reinforcement learning method for generative models, only to the task of drawing the next image from context. The IntReal metrics served as the reward. This restored the lost image quality, and the model achieved the best overall score among the open models in the test.

The practical takeaway for engineers is that you don't have to train text and images together at every stage. You can optimize one modality and still gain on mixed responses. For teams building AI assistants for tutorials, travel guides and educational materials, IntReal offers a ready-made dataset and a diagnostic test to see exactly where a model breaks.