Wearable Stereo Camera Estimates Food Volume With 9.1% Error, No Reference Object Needed

The authors of arXiv preprint 2610.12735 propose estimating food volume from a single stereo clip recorded by a wearable camera. They need no phone, no series of photos and no coin next to the plate for scale. Across five recorded dishes, the mean absolute percentage error of the volume estimate was 9.1%. The work is pitched as a step toward passive diet tracking, where a person simply eats and the system calculates portions on its own.

Where the scale comes from

Current stereo-based portion estimation methods depend on user effort. The user has to photograph the dish from several angles with a phone or place an object of known size in the frame. Otherwise the reconstruction has the right shape but an unknown size. For regular diet monitoring, this workflow is inconvenient: every meal turns into a small procedure.

The authors use what a wearable device already has by default. First, the stereo baseline is factory-calibrated, meaning the distance between the lenses is known precisely, so depth computed from disparity is immediately expressed in meters. Second, the device estimates camera poses on its own using visual-inertial odometry (VIO). Together, these give metric scale with no reference object in the scene and no manual setup.

How the pipeline works

The whole system is assembled from off-the-shelf components, and none of them was trained specifically for food:

• FoundationStereo — zero-shot stereo matching that produces depth maps from a pair of frames;
• SAM 2 — segmentation of the dish, separating the food from the plate and the table;
• TSDF fusion over a set of keyframes — merging the depth maps into a single 3D mesh.

The output is a mesh of any dish on the plate at true physical scale, and the volume is computed from it. The authors stress that the reconstruction is fully automatic and built from a single stereo recording.

Key parameters of the work:

• input: a single stereo recording from an egocentric wearable device;
• scale: factory stereo baseline plus camera poses from VIO, with no reference object;
• output: a visually and metrically accurate 3D mesh of the dish;
• sample: five dishes;
• mean absolute percentage error of volume: 9.1%.

What has not been shown yet

Five dishes make a feasibility demonstration, not a benchmark. This is a first-version preprint and has not been peer reviewed. The abstract includes no comparison with phone-based methods or reference-object approaches, so there is no way to tell whether 9.1% is better or worse than the alternatives. It is also unknown how the pipeline would handle translucent sauces, soups, dishes with occlusions, poor lighting, or head movement while eating.

What to do with the volume afterward is a separate question. Calories and nutrients require the dish type and its density, which is a different problem that this method does not solve. Volume only removes one of the most inconvenient problems: portion uncertainty.

Where this could be useful

The most obvious niche is wearables with stereo cameras and built-in VIO, such as smart glasses. If the factory calibration and poses are exposed through an API, a developer of nutrition apps could get metric geometry of a dish without imposing any requirements on the user. For dietary researchers, the approach promises less intrusive data collection than photo diaries.

Three things are worth watching as the work develops: whether a sample orders of magnitude larger than five dishes appears, whether direct comparisons with methods that require a reference show up, and whether the authors demonstrate robustness to real-world conditions rather than staged recordings.