Radar-Based Forecaster UFMF Hits 122 FPS and 0.45 mIoU on Fast-Moving nuScenes Objects

The authors of an arXiv paper have introduced UFMF (Ultra-Fast Multi-modal Forecaster), a model that forecasts in real time which areas around an autonomous car will be occupied over the next few seconds. On a single RTX 4090, it processes 122 frames per second. On the nuScenes dataset, it reaches 0.542 overall mIoU and 0.45 mIoU on the "fast bin," a group of objects with large displacement. On that bin, the model beats recent occupancy world models by 4.8 percentage points (mIoU is a metric of overlap between the predicted and actual area).

Where lightweight forecasters break down

An autonomous vehicle's planner relies on future occupancy maps to anticipate collisions. If a fast car blurs or disappears from the map at a long horizon, the planner loses the geometry it needs, and the forecast starts to do harm. The authors call this failure dynamic blank-collapse, an "empty" collapse of the dynamics.

Existing approaches attack the problem from opposite ends. Occupancy world models such as OccWorld, Drive-OccWorld and COME build dense 3D voxels with generative decoders and hold up well at long horizons, but most of them run slower than 25 FPS, and the strongest run at 1–2 FPS. Lightweight BEV forecasters (BEV is a top-down view of the scene), such as FlashOcc, fit within the latency budget: they compute in 2D and lift the result to 3D only at the output. The trade-off is that fast objects collapse when their motion diverges from the background, which is compensated for the ego vehicle's own motion.

Three add-ons to a cheap core

Instead of scaling the architecture uniformly, UFMF keeps a cheap temporal core, a diagonal state-space model without attention, and adds structure where lightweight models usually fail.

The first add-on is the Velocity-Aware Dynamic Prior (VADP), a radar-based dynamic prior that accounts for velocity. It combines differences in the semantic history with dense radar Doppler data and predicts a displacement field for each grid cell. The future displacement is split into a linear extrapolation and a learnable, horizon-dependent correction. The resulting flows carry a compact dynamic subspace of the hidden state forward during forecasting. The prior is applied only to classes with noticeable motion: cars, trucks, buses, bicycles, motorcycles and pedestrians. The base predictor still outputs all classes, including background and static objects.

Radar enters the model as a dense map, not as raw returns: for each BEV cell, the radial Doppler of all radar points that fall into it is averaged, and cells with no returns stay at zero. This is a parameter-free grid fusion. The same radar stream serves both as the temporal condition for the main forecasting path and as the input to VADP.

The second add-on is dynamics supervision in the world coordinate frame. Predicted and target occupancy are compared in a shared frame, so the model is penalized for trajectories that look plausible at each horizon but are inconsistent over time. Future ego poses are needed only during training; they are not available at inference.

The third add-on is class-dependent supervision of the vertical profile. It steers a lightweight head that lifts from BEV to 3D toward semantically plausible height distributions, and it needs no heavy 3D decoder. Both auxiliary add-ons work only during training, while VADP is built in alongside the fast inference path, so real-time performance is preserved.

What to keep in mind

The authors stress that the contribution is systemic: the paper introduces no new base operator. They claim that, under a strict 8 ms latency budget, UFMF is the first to eliminate dynamic blank-collapse, which previously only heavy generative world models managed to fix, and that it runs more than 100 times faster than the strongest baseline.

The figures come from the authors themselves and were obtained on a single dataset, nuScenes. Speed was measured on a desktop RTX 4090, not on an in-vehicle computer. In addition, the model uses radar Doppler, so for systems without such data, the gain would need to be verified separately.

For engineers building an autonomous driving stack, the takeaway is practical: the dynamics that a lightweight forecaster lacks can be supplied by physically meaningful priors, such as linear motion extrapolation and radar Doppler, instead of increasing model size. If the results reproduce on other datasets, this approach could become a cheap alternative to heavy world models.