FDSML Fractal Method Lifts Recall@1 on Stanford Cars from 56.2% to 78.4%

A group of researchers from Japan's National Institute of Advanced Industrial Science and Technology (AIST) and the Faculty of Science and Engineering at Waseda University has proposed a way to pretrain computer vision models without any real images and still get better embeddings. The method is called FDSML (Formula-Driven Supervised Metric Learning). On ResNet-50, it raises Recall@1 on Stanford Cars from 56.2% to 78.4%, and on Stanford Online Products from 65.6% to 74.0%, compared with classic formula-driven pretraining (FDSL).

Why fractal classification hits a ceiling

FDSL is an approach in which images and labels are generated from mathematical rules. The best-known example is FractalDB: fractals are built with IFS (iterated function systems, sets of contractive affine transformations), and a class is defined by a set of parameters. This kind of synthetic data requires no manual labeling, avoids licensing and privacy problems, and does not inherit the social biases of real-world datasets.

The weak point is that training boils down to classification. The continuous parameter space is split into discrete classes, and information about which fractals are more similar to each other is lost. In the authors' view, the model then overfits to local, low-level geometric textures. The symptoms follow: the representations differ from those learned on real data, the models are sensitive to blur, and in some target domains they lose to training from scratch. This is especially noticeable in image retrieval, where the geometry of the feature space matters more than class boundaries.

How FDSML works

The method has two parts. The first is the FineFractalDB dataset. In the original FractalDB-1k, each class has 1000 instances, but they come from only 25 parametric perturbations, plus reflections and rendering variations, so there is little diversity within a class. In addition, the transformations were not always strictly contractive, and some images "collapsed" into uninformative blobs. FineFractalDB raises the number of independent perturbations to 1000 per class, imposes a hard constraint on the contractivity of the transformations, and adds filtering: samples in which fewer than 0.05% of pixels are occupied are discarded.

The second part is Parameter-Aware Triplet (PA-Triplet) Loss. It is a standard triplet loss with an anchor, a positive and a negative, but roles are assigned not by class labels but by distance in IFS parameter space. For each image, the method computes a compact descriptor: a weighted expectation of the affine parameters. Of two candidates, the one whose descriptor is closer to the anchor becomes the positive. The authors stress that the distance is used only ordinally: it decides which candidate is closer and does not claim to be a perceptual metric.

Discrete classes still play a role as a coarse filter. Preliminary experiments showed that when the parameter distance is too large, the fractals differ so much that the comparison becomes trivial and teaches the model nothing. So candidates are restricted to one of four modes: unrestricted; one from the same class and one from a different class; both from the anchor's class; or both from other classes. In every case, the distance between parameters makes the final decision on the positive and the negative.

What the experiments showed and where the limits are

Beyond image retrieval, FDSML stays competitive on downstream classification and improves robustness to a number of distortions. For ResNet-50 the robustness gain is confirmed, but for ViT the authors state plainly that there are architecture-dependent trade-offs: there is no clear win there. They also describe qualitative differences in activation and attention maps, although the available text gives no details on them.

Open questions remain. The reported figures are for comparisons against FDSL pretraining, not against models trained on real data such as ImageNet. The method has been tested on synthetic fractals, and it is unknown how well the trick of using an ordinal signal from parameters will carry over to other formula-driven generators. The takeaway for practitioners: if you need synthetic pretraining for retrieval and ranking tasks, continuous generation parameters provide a richer signal than class labels. The project page is available at shineiarakawa.github.io/fdsml.