Image Tiling Raised Small-Defect AP by 9.4% but Slowed Analysis Threefold

A small chip in the original photo is still visible to the eye. After the image is downscaled to the network's input size, only a few pixels remain, and on the feature map it occupies one or two cells. For a detector, it is barely an object at all, more like noise. A simple idea — cut the frame into tiles and enlarge each one — really does help. But it has an unpleasant flip side: while small defects become more noticeable, large ones fall apart into pieces.

The author of the experiment tested this trade-off on 1,812 production photos of vehicle damage. At an input size of 512, about 41% of defects became smaller than eight pixels. For car body parts, such objects made up only 1.6%. As a result, the single aggregate metric looked decent: large and simple objects masked near-zero quality on the smallest ones.

What tiling delivered

The cheapest test required no new training. An already trained Mask R-CNN was run separately on image tiles, then the detected masks were mapped back to full-frame coordinates and duplicates were removed. AP on small objects rose by 9.4%, and mIoU by 8.8%. The result is consistent with the SAHI idea: enlarging local regions gives the detector back details that disappear during ordinary resizing.

The cost turned out to be very real. Each image took 3–4 times longer to process because the model passed through every tile separately. For offline photo review this may be acceptable. For a video stream or a production line it is not, until the team optimizes tile size, overlap and the number of model runs.

Next, the author fine-tuned the network on tiled data. Quality improved on tiles, but on the full frame the model became worse than the original. The reason is simple: training and evaluation took place at different scales. The network got used to enlarged crops and stopped working confidently on ordinary images.

An attempt to train almost exclusively on heavily enlarged tiles ended even worse: overall AP on the full frame fell by roughly a third, and recall by half. When a mix of full frames and tiles at different scales was returned to training, the collapse almost disappeared. Small defects gained noticeably, but quality on large ones dropped by about half.

Why overall AP is misleading

On a single tile, a large dent looks not like a whole object but like a fragment without natural boundaries. Annotation is cut off at the tile edge. At inference, one large defect can turn into several masks that standard NMS or NMM will not merge: neighboring parts barely overlap, so their IoU is too low.

That is exactly why a rise in the overall metric should not be treated as a win. If the test set contains many small body-part details, their improvement offsets the regression on large damage. Without breaking AP down by size, a team can ship a model with an attractive average figure that is worse at an important part of the real task.

There is also a statistical limitation. Some classes are represented by only a handful of examples, and the variance between independent runs on the smallest objects is comparable to the improvement itself. Mean AP across the eight defect classes has not yet exceeded the baseline model. This is a useful engineering experiment, but not proof of a universal advantage of tiling.

What to check on your own data

Before training, calculate object sizes after the same resize used in production. If only a few percent become smaller than a few pixels, tiling will most likely add computation and boundary errors without noticeable benefit.

If the problem is real, start with tiling at inference only on top of a finished model. That quickly shows the potential gain without an expensive training cycle. Then compare the full frame and tiles separately, calculate AP for small, medium and large objects, and measure recall near tile boundaries.

Do not drop full frames from training. Mix scales and test the model in exactly the modes in which it will operate. Separately measure how many large objects split into two or more masks. The practical takeaway is not that frames should always be cut, but that scale is part of the model's contract. It must be tested as carefully as architecture and annotation.