Halcyon beats LlamaGen and JiT-T2I with 18–275x less pretraining compute

A team of researchers has shown that a strong text-to-image model does not require internet-scale language pretraining. The Halcyon family is trained entirely from scratch on GPIC, a dataset of 100M image-text pairs. The smaller model, Halcyon-0.52B, scored a gFD of 48.0 against 63.3 for LlamaGen-XL/8 and used 18.2x less pretraining compute. The larger one, Halcyon-1.6B, beat JiT-T2I-XXL/8++ with 35.3 against 41.0, using 275x less compute. In both pairs, lower is better.

What the researchers tested

Modern T2I systems usually start from a language model pretrained on huge text corpora. The assumption is that such a model understands the prompt better and therefore draws more accurately. The authors challenge this premise and ask whether language pretraining is needed for strong image generation at all.

Halcyon is built as an early-fusion multimodal autoregressive model. Text and visual tokens are processed together from the very start, with no separate pretrained language tower. All of the training signal comes from 100M image-text pairs, a modest volume by the standards of internet-scale corpora.

The numbers and the control experiments

The main result comes down to the ratio of quality to cost:

• Halcyon-0.52B: 48.0 gFD against 63.3 for LlamaGen-XL/8, with 18.2x less pretraining compute.
• Halcyon-1.6B: 35.3 gFD against 41.0 for JiT-T2I-XXL/8++, with 275x less pretraining compute.
• Training data: GPIC, 100M image-text pairs.
• Architecture: early-fusion multimodal autoregression, trained from scratch.

Comparing against competitors proves nothing about causes on its own. The more important point is therefore a different one: the authors ran controlled experiments, and text-only pretraining did not improve T2I generation. This is an empirical observation, not just a good benchmark result. It suggests that the textual knowledge accumulated on language corpora contributes less to the "draw from a description" task than is commonly assumed. There is one caveat: the conclusion holds for a specific setup, model sizes and dataset. It cannot be carried over to arbitrary scales without testing.

Captioning comes for free

One separate detail concerns captioning. In T2I training, a share of captions is usually dropped at random (caption dropout) so that the model learns both conditional and unconditional generation. Halcyon replaces this technique with I2T supervision: instead of an empty condition, the model gets the task of describing the image. As a result, it learns to caption images and keeps competitive T2I quality at the same pretraining budget.

One model in one training run gains two capabilities. For teams that need both image generation and image labeling, this means not having to maintain two separate pipelines.

Practical takeaways

For researchers and small teams, the main signal is that entering the training of a custom T2I model is cheaper than it seemed. A dataset of 100M pairs is noticeably smaller than typical web corpora, and the difference in pretraining compute is measured in tens and hundreds of times. If the results reproduce on other data, labs without access to giant clusters will have a workable path.

You can check this yourself: the code and models are published on the project page at cs.stanford.edu/~keshik/halcyon. It is worth running Halcyon on your own prompts and domains, because gFD describes a distribution, not fitness for a specific product. It is also useful to compare the I2T-supervision variant with the classic caption-dropout scheme on your own data.

The question of scale remains. The work shows results for models with 0.52B and 1.6B parameters. Whether the advantage holds an order of magnitude higher is still unclear. The next round of experiments will answer it.