A drone with the 1B OpenFlyV2 model flies to its target on its own in 48% of real flights, without training on them

Researchers at Shanghai AI Laboratory, together with Northwestern Polytechnical University, Shanghai Jiao Tong University, USTC and TeleAI, have introduced OpenFlyV2, a vision-language-action model for drones. It understands what the camera sees, finds objects from text descriptions, estimates distances and generates a flight trajectory at the same time. All of this runs on board, on an NVIDIA Jetson Orin NX module, with no computation offloaded to a ground server. In real-world tests, a DJI M30T quadcopter reached its target in 48% of flights, even though no real flight data was used for training.

One model instead of a set of specialists

Aerial robots are usually assembled from separate blocks: one handles scene understanding, another navigation, a third planning, and the heavy computation goes to an external server. That makes flight dependent on connection quality and latency. In OpenFlyV2, a shared vision-language backbone (0.8B parameters, based on Qwen3.5-VL) processes frames, the instruction and the flight history. When the drone needs to act, a compact 0.2B-parameter "action expert" plugs into it. The expert turns the multimodal context into 12 waypoints, which a low-level controller then executes. The update rate is 10 Hz.

To fit within the on-board hardware budget, the authors added two techniques. Motion-Adaptive Memory (MAM) compresses the flight history into a limited set of "event" tokens: motion segments plus keyframes. This way, the model does not have to carry dense video in its context. Motion-Structured Prior (MSP) relies on the observation that short drone trajectories reduce to a small set of recurring patterns. Generation starts not from pure noise but from a distribution around a suitable motion prototype, so the flow generator needs fewer denoising steps.

Data from simulation only

The OpenFlyV2-130K dataset is built from 13 environments: four AirSim scenes, two Unreal Engine city levels and seven 3D Gaussian Splatting reconstructions (five university campuses and two destroyed districts). In total, it contains 72 352 navigation trajectories and 60 006 question–answer pairs covering color, object descriptions, distances, spatial relations and visual grounding. The annotations were generated automatically and checked by humans. Four environments were held out of training entirely to test generalization.

What the measurements show

Across five aerial perception tasks, the model with the 0.8B backbone is comparable to GPT-5.5 and outperforms Qwen3.5-9B and Gemini 3 Flash. In object grounding, it scores 67.1% [email protected] versus 61.6% for GPT-5.5 (the metric is the share of answers where the predicted box matches the reference with an IoU of at least 0.5). Mean absolute distance estimation error is 47.32 m versus 52.62 m, and relative error is 0.27 versus 0.34. Description quality is practically equal: 6.38 versus 6.36 out of 10.

In simulated navigation, over 800 episodes in familiar environments and 300 in unfamiliar ones, success rates were 62.88% and 52.67%. Only an explicit stop signal within 25 m of the target counted. Four open baseline methods, including AeroVLA and the navigation model from OpenUAV, were retrained on the same data. They were compared only in simulation, because none of them was run on the Jetson Orin NX. In the physical test, the same checkpoint flew four real locations and achieved a 48% success rate.

Limitations

A 48% rate means that more than half of the real flights ended in failure, so industrial-grade reliability is a long way off. The 25 m tolerance in the success criterion is fairly generous for precision-approach tasks. The comparison with GPT-5.5 and other models was run on the authors' own benchmark, without independent verification. The text also indicates that some results came after additional reinforcement fine-tuning, and the details of the comparison with one of the baseline methods are truncated in the available description.

The practical value lies elsewhere: the work shows that a 1B-parameter model can fit scene understanding, spatial reasoning and continuous control into an on-board loop running at 10 Hz. That reduces the drone's dependence on a communication link. The team promises to publish the code and data in the AeroBrain repository on GitHub, but for now the claims cannot be verified independently.