Plato's Hypothesis Proven for JEPA: A 109M Text Model Beats Qwen2.5-72B

The Platonic Representation Hypothesis (PRH) began as an observation: the larger the models, the more the internal representations of vision and language models resemble each other, as if all of them were converging on a single statistical model of reality. There was no proof. A new paper turns the hypothesis into a theorem for a class of architectures called JEPA (Joint-Embedding Predictive Architectures).

These models learn to predict the embedding of one "view" of an object from the embedding of another, rather than reconstructing pixels or words. The paper's headline number: a text JEPA with 109M parameters scores .184 on similarity to the DINOv2 vision model.

Every language model tested, up to Qwen2.5-72B (.154), scores lower.

For comparison, the CLIP text encoder gets .124.

What exactly is proven

The authors start from an idealized world: a finite number of states, with each modality (images, text, video, audio, actions) observing it without losing information. A JEPA with whitened anti-collapse, meaning a requirement that embeddings do not collapse into a single point, learns the spectral representation of the graph of its positive pairs. A positive pair is two inputs that the model treats as views of the same thing.

If, in two modalities, the positive pairs link the same world states, the learning problems coincide and differ only in the coordinate system. The solutions then match up to a rotation. This is the Platonic representation theorem. If the pairs differ, the misalignment is bounded above by a measure of that difference and by the size of the spectral gap. The authors tested the bound on 300 random worlds: the error stayed below the bound in every case, reaching at most 18% of it.

A caveat: the theorem holds in the idealized setting. It does not cover JEPA with a predictor, as in action-conditioned world models. Uniqueness of the solution under a full distributional constraint (SIGReg in LeJEPA) remains an assumption.

Good and bad "views"

The main practical conclusion concerns how the pairs are built.

• Good views: two different observations of one scene, such as two captions for the same photo or two frames of a video. They share only the scene, so the representation reflects the world. • Bad views: distortions of a single observation, such as two crops of a photo or two maskings of a sentence. They also preserve the modality's "noise": word choice, camera angle, lighting. The model spends part of the embedding dimensions on that noise, and no rotation will align it with another modality.

By this logic, next-token prediction is also a JEPA with bad views: the pair is a prefix and its continuation within the same document. This explains why language models match vision only partially, and mostly through scale. DINOv2 and the authors' own encoders also fall into the bad-view category, since their pairs are crops of one photograph.

What the experiments showed

The authors trained text JEPAs on image captions, where good views arise naturally, and compared them with language models.

• A 33M-parameter model trained on the texts of 50 million web images scores .125 on the benchmark from the PRH paper. On 145 thousand Flickr captions it scores .

034. That is on par with Qwen2.5-3B and 7B.

• On the same corpus, good views reach with 21.6 million examples the similarity that token prediction reaches with 900 million. That is a 42-fold difference.

• The 109M version, trained on the texts of 940 million images, read 116 billion tokens. It beats models 660 times larger that were trained on more than 100 times as much data. With the CLIP ViT-H vision encoder it scores .

230 versus .210 for the best LLM. • Misalignment follows a scaling law in the share of good views and the number of examples seen.

The law predicted 85 new runs before they were trained, as accurately as a rerun with a different random seed does.

There are applied results as well. A single linear map with orthonormal rows, the only part trained on pairs, connects frozen DINOv2 and DINOv3 to a language model fine-tuned on text alone. This setup gives CIDEr .838 on COCO. The map needs 0.22M parameters for DINOv2 ViT-B and 1.9M for the best captioner. With no pairs at all, the rotation is found from the geometry of unmatched images and captions. The 109M text JEPA beats all zero-shot results of the predecessor benchmark across 21 model pairs and five settings. It outperforms Qwen3-8B by 17 points of zero-shot accuracy on ImageNet-100.

Limitations and what engineers should take away

There are limits. An image JEPA trained on good views from 7 thousand MSR-VTT clips is still noticeably weaker than DINOv2. The authors leave scaling good views to images for later. Captions describe only part of a photograph, and the theory accounts for this only through the measure of pair mismatch.

For practitioners, the conclusion is this: if you need a shared multimodal embedding, how the positive pairs are built matters more than model size. Literal descriptions of what is visible, such as LLaVA recaptions, work better than alt-texts. A tiny text encoder with the right pairs can replace a huge LLM wherever all you need is alignment with a visual space.