Meta DINOv3 и SAM: University of Pittsburgh строит ассистивного робота RAMMP за $41,5 млн

On July 27, 2026, Meta announced that its vision models DINOv3, SAM, and RF-DETR form the foundation of RAMMP — an assistive robotic platform for people with limited mobility being developed by the Human Engineering Research Laboratories (HERL) at the University of Pittsburgh with an ARPA-H grant of up to $41.5 million.

What RAMMP is

RAMMP (Robotic Assistive Mobility and Manipulation Platform) is an assistive platform with a manipulator arm being led by the HERL lab at the University of Pittsburgh together with partner ATDev and a consortium of universities. The project targets ~5.5 million wheelchair users in the US, for whom everyday actions — opening a door, picking up a cup, rolling off a curb — remain a barrier.

• ARPA-H grant — up to $41.5 million
• Lead lab — HERL, University of Pittsburgh
• Partners: ATDev, Kinova Robotics, LUCI Mobility, Carnegie Mellon, Cornell, Northeastern, Purdue
• Meta models: DINOv3, SAM (including SAM 3.1), RF-DETR
• In the US, more than 100,000 wheelchair-related injuries land in emergency rooms every year

How Meta's models help the robot see

Meta's computer vision models give the RAMMP robot the ability to recognize the scene and objects around it. DINOv3 (a self-supervised transformer trained on unlabeled data) handles general image understanding, SAM and SAM 3.1 segment and highlight objects, and the lightweight RF-DETR detection model, fine-tuned on DINOv2 embeddings, is deployed directly on the robot for fast on-the-spot recognition.

"Meta's computer vision models are a critically important technology that lets the robotic platform 'see.'

They help the robot understand the scene, navigate space, and pick up the right objects, significantly reducing the cognitive load on the user and increasing their independence," — Mansoor Khan, ARPA-H program manager.

Why it all runs on the device

RAMMP runs the models directly on the robot, on edge compute, rather than in the cloud — this removes the dependency on connectivity and preserves real-time response. The team also uses a digital twin (a virtual environment for simulation), voice control, and 360-degree awareness. The first prototype already detects automatic door buttons, cups, and curbs; the project has moved into real-world testing and is working on voice and touch input.

"Running models like DINOv3 and SAM directly on the device delivers real-time perception the user can trust — without dependency on connectivity and without compromises on safety," —

Sivashankar Sivakanthan, RAMMP project hardware lead.

What it means

Meta's open vision models are moving beyond research into a physical product: they become the "eyes" of an assistive robot meant to restore independence to millions of people with limited mobility. As ARPA-H Director Alicia Jackson emphasized, technology should adapt to the person, not force the person to adapt to it. RAMMP shows how a single set of open-source models can advance both robotics and accessibility at once.

Frequently asked questions

Which Meta models are used in RAMMP?

The platform uses three Meta computer vision models: DINOv3 for scene understanding, SAM and SAM 3.1 for object segmentation, and RF-DETR — a lightweight detector fine-tuned on DINOv2 embeddings for running directly on the robot.

How much funding has been allocated to the project?

The development is funded by the ARPA-H agency with a grant of up to $41.5 million. The lead organization is the HERL lab at the University of Pittsburgh, with partners including ATDev, Kinova Robotics, LUCI Mobility, and the universities Carnegie Mellon, Cornell, Northeastern, and Purdue.

Who will RAMMP help?

The platform is designed for people with limited mobility: in the US alone, about 5.5 million people use wheelchairs, and more than 100,000 wheelchair-related injuries land in emergency rooms every year.

*Meta is recognized as an extremist organization and is banned in the Russian Federation.