Meta introduced Sapiens2 — a unified computer vision model for pose, segmentation, and 3D

Meta Reality Labs has released Sapiens2 — the next generation of human-centric vision models, which attempts to replace a fragmented set of specialized networks with a single unified foundation. The company has assembled in one lineup tasks that typically exist separately: human pose estimation, body part segmentation, surface normal reconstruction, pointmap for 3D geometry, and albedo estimation. For the market, this is an important signal: Meta continues to bet not only on generative AI but also on practical computer vision, which is necessary for AR devices, digital avatars, virtual try-on, motion capture, and video analysis.

The main idea behind Sapiens2 is that a single base architecture can work across multiple levels of understanding human presence in a frame. The system no longer requires a separate network for the skeleton, another for body markup, and a third for surface geometry. Meta claims that a single backbone, after fine-tuning, covers all these scenarios.

In practical terms, this simplifies the production pipeline: fewer components, less desynchronization between models, and lower maintenance costs. In the published checkpoints, there is, in particular, a model for top-down pose estimation on 308 keypoints, including detailed points of the face, hands, and feet, as well as segmentation into 29 classes of body parts. The key update is not only in the set of tasks but also in how the model was trained.

Sapiens2 was pretrained on a curated dataset of 1 billion high-quality images of people. In pretraining, Meta combined masked image reconstruction with self-distilled contrastive objective, so that the model simultaneously maintained low-level details for dense prediction and high-level semantics for zero-shot and few-label scenarios. The architecture also employed techniques from more recent frontier models to sustain longer training cycles without loss of stability.

The lineup scales from 0.4 to 5 billion parameters, works in native 1K resolution, and hierarchical variants support 4K and use windowed attention for longer spatial context. Compared to the first generation of Sapiens, Meta claims a notable improvement across nearly all key metrics.

On the pose estimation task, the new version adds 4 points of mAP, on body-part segmentation — 24.3 points of mIoU, and in surface normal evaluation it reduces angular error by 45.6%.

Separately important is that Sapiens2 goes beyond the typical tasks of the first release. Now the family can build pointmaps, that is, predict 3D coordinates in the camera system for each pixel, and work with albedo — the base color of the surface without the influence of lighting. For avatars, AR, and digital try-on, these representations are particularly useful: they help more accurately reconstruct human shape, transfer lighting, and build more plausible 3D scenes from a regular photograph.

The practical value of the release is that Meta did not limit itself to a research publication. The company has already posted the Sapiens2 family on Hugging Face and code on GitHub, with individual variants for pose, segmentation, normals, and pointmap available in the collection. This lowers the entry barrier for teams building products around computer vision for humans: from fitness apps and video analytics systems to XR interfaces and virtual characters.

At the same time, it is important to remember that Sapiens2 is not a universal model for any computer vision, but rather a strong stack for human-centric imagery. That is, its main zone of strength is frames where the human, his pose, surface, clothing, and body geometry remain the central object. What this means in practice: Meta is taking another step toward a unified visual backbone for everything related to the human in the frame.

If the stated results are confirmed in real production scenarios, the company will have a strong foundation for its own XR products and simultaneously set a new standard for open research in human-centric vision. For the market, this is a good example of how foundation models are beginning to bring benefits not only in text and generation, but also in precise, engineering tasks of computer vision.