Mask size predicts segmentation quality better than sample count

The model carefully outlines a car door but fails to notice the scratch on it. Yet the average AP looks tolerable: large parts appear frequently, are recognized well, and conceal the failure in the very area the system was built to address.

A comparison of Mask R-CNN and YOLO11s-seg on 1,812 anonymized production photographs of damaged vehicles exposed exactly this trap. The dataset contained 29 classes and nearly 25,000 annotated objects, ranging from body parts to dents, cracks, and corrosion. The main practical conclusion was simple: before changing the architecture or commissioning more annotation, calculate how many pixels the target object occupies after the image is downscaled to the network's input size.

The average metric hides small defects

The original dataset is severely imbalanced: the most common class appeared more than 3,200 times, while the rarest appeared fewer than 80 times. Yet sample count explained almost none of the performance differences between individual classes: its correlation with AP was about −0.23. By contrast, the relationship with the logarithm of median mask area was substantial, ranging from 0.68 to 0.74.

The reason is clear without complicated mathematics. After a photograph is resized to 512×384, a fine scratch turns into a strip only a few pixels wide. About 41% of defects occupy fewer than eight pixels. Standard binary cross-entropy averages the error across the region, so the signal from a tiny mask is lost among thousands of other pixels.

This creates an unpleasant product-level consequence. You can double the number of random photographs and mostly obtain more doors and bumpers that the model already knows how to find. In the experiment, expanding the dataset from 100 to 200 images added 0.096 mIoU, while going from 200 to 400 added only 0.007. Annotating everything indiscriminately proved far less cost-effective than deliberately collecting rare and small defects.

What actually changed the result

The cheapest useful adjustment was replacing the loss function. Combining BCE and Dice increased overall AP from 0.190 to 0.200. The gain was 14.7% for small objects and 15.8% for defect classes. AP on cracks became consistently greater than zero for the first time. The architecture and inference speed remained unchanged.

Focal Loss without Dice, by contrast, reduced AP to 0.177. In this task, accounting for the intersection relative to mask area mattered more than simply increasing the contribution of difficult pixels.

The training step size was also critical. For Mask R-CNN with AdamW, a value of 1e-4 produced an mIoU of 0.252, 5e-4 produced 0.125, and 1e-3 produced just 0.029. Excessively large updates quickly erased useful features learned during COCO pretraining. Training from scratch on 400 images effectively failed: mIoU stopped at around 0.001.

Short runs also require caution. Two epochs preserved the ranking of successful configurations but substantially underestimated the quality ceiling. After 15 epochs, test mIoU rose from about 0.286 to 0.373, while AP increased from 0.165 to 0.190. Such a protocol is suitable for quickly screening configurations, but not for publishing the final metric.

Architecture cannot recover an invisible object

Using the same evaluation code, YOLO11s-seg achieved an AP of 0.193 versus 0.189 for Mask R-CNN. The gap was more pronounced on small objects: 0.180 versus 0.127. However, YOLO used a 640×640 input, so every defect remained slightly larger.

The engineering difference is more significant than the quality difference. YOLO has 10.1 million parameters versus 46 million, its weights file occupies 19.6 MB versus 184 MB, and processing one image takes about 3.5–9.5 ms instead of 61–112 ms. That is a strong argument for an edge device. Yet the mean AP specifically for defects remained almost equally low: 0.023 versus 0.020. Switching models did not restore information lost when the frame was downscaled.

There are limitations. This is one primary production dataset, one family of models, and a limited run budget. The resolution comparison does not yet establish a universal "law of detectability," and some experiments on a second dataset, CarDD, remain incomplete. Differences of a few hundredths without confidence intervals may turn out to be noise.

The practical sequence is as follows: first measure object sizes after input transformation; then choose the camera and network resolution; tune the threshold to reflect the cost of misses and false positives; verify a small training step and an appropriate loss function. Only then should additional data be collected—not just any data, but examples containing genuinely small and rare cases.