Meta DINOv3 and SAM: University of Pittsburgh Builds $41.5M RAMMP Assistive Robot

On July 27, 2026, Meta said its DINOv3, SAM and RF-DETR vision models form the foundation of RAMMP, an assistive robot platform for people with limited mobility. The Human Engineering Research Laboratories (HERL) at the University of Pittsburgh is developing it with an ARPA-H grant of up to $41.5M.

What is RAMMP

RAMMP (Robotic Assistive Mobility and Manipulation Platform) is an assistive platform with a manipulator arm. The HERL lab at the University of Pittsburgh leads the project together with partner ATDev and a consortium of universities. It targets ~5.5M wheelchair users in the US, for whom everyday actions such as opening a door, picking up a cup or going down a curb remain a barrier.

• ARPA-H grant: up to $41.5M
• Lead lab: HERL, University of Pittsburgh
• Partners: ATDev, Kinova Robotics, LUCI Mobility, Carnegie Mellon, Cornell, Northeastern, Purdue
• Meta models: DINOv3, SAM (including SAM 3.1), RF-DETR
• In the US, more than 100,000 wheelchair-related injuries are treated in emergency departments every year

How Meta's models help the robot see

Meta's computer vision models give RAMMP the ability to recognize the scene and the objects around it. DINOv3, a self-supervised transformer trained on unlabeled data, handles general image understanding. SAM and SAM 3.1 segment and isolate objects. RF-DETR, a lightweight detection model fine-tuned on DINOv2 embeddings, runs directly on the robot for fast on-site recognition.

"Meta's computer vision models are a critical technology that allows the robotic platform to 'see.'

They help the robot understand the scene, navigate the space and pick up the right objects, noticeably reducing the user's cognitive load and increasing their independence," said Mansur Khan, program manager at ARPA-H.

Why everything runs on the device

RAMMP runs the models directly on the robot using edge computing rather than in the cloud. This removes the dependence on connectivity and keeps responses real-time. The team also uses a digital twin (a virtual environment for simulation), voice control and a 360-degree view. The first prototype already detects automatic door buttons, cups and curbs. The project has moved into real-world testing and is working on voice and touch input.

"Running models like DINOv3 and SAM directly on the device gives real-time perception that the user can trust, without depending on connectivity and without compromising on safety," said

Sivashankar Sivakanthan, chief of staff of the RAMMP project.

What this means

Meta's open vision models are moving beyond research into a physical product. They become the "eyes" of an assistive robot meant to restore independence to millions of people with limited mobility. As ARPA-H Director Alicia Jackson emphasized, technology should adapt to the person rather than force the person to adapt to it. RAMMP shows how a single set of open-source models advances both robotics and accessibility.

Frequently asked questions

Which Meta models does RAMMP use?

The platform uses three Meta computer vision models: DINOv3 for scene understanding, SAM and SAM 3.1 for object segmentation, and RF-DETR, a lightweight detector fine-tuned on DINOv2 embeddings to run directly on the robot.

How much funding has the project received?

The ARPA-H agency is funding development with a grant of up to $41.5M. The lead organization is the HERL lab at the University of Pittsburgh. Partners include ATDev, Kinova Robotics, LUCI Mobility, and the universities Carnegie Mellon, Cornell, Northeastern and Purdue.

Who will RAMMP help?

The platform is designed for people with limited mobility. In the US alone there are about 5.5M wheelchair users, and more than 100,000 wheelchair-related injuries are treated in emergency departments every year.

*Meta has been designated an extremist organization and banned in Russia.