A Rules-Based Pilot Builds Its Own Labels and Cuts the Wait for a Model from a Year to Zero

Collecting labels for an anomaly detector as a separate phase means waiting six months, or even a year, before the system does anything at all. Alexey, an ML team lead, described on Habr a way to remove that pause: a rules-based pilot goes into production right away and, in parallel, produces the labels the model later learns from.

Labels do not exist until someone starts creating them

Reference data and data sources can be put in order in advance. That work does not depend on the model, and the author covered it in a separate article. Labels are different. Nobody has ever gone back and marked where an anomaly occurred and where things were normal. That history does not exist until someone starts issuing verdicts on a live stream of events.

This is the dead end of the classic plan: first collect labeled data, then train, then launch. If labels only appear while the system is running, the plan runs into its own precondition.

Why rules and not a model

The example is procurement control: finding inflated prices, purchases outside their intended purpose, and purchases made outside tenders. The author lists what breaks in each of the obvious approaches:

• a supervised model has nothing to train on, because there are no labeled examples;
• an unsupervised approach on rare events produces too many false positives;
• an LLM without a reference price confidently calls an inflated price a market one.

The conclusion in the announcement is blunt: a rules layer first, with the model on top of it, later. Rules deliver a working result from day one. Every alert that an expert confirms or rejects becomes a building block of the future labeled dataset. The pilot is useful in the meantime, rather than just waiting for data to accumulate.

In the previous article, the author left the market-price reference itself as an open question. Here he argues that you can launch without waiting for that reference to appear.

Key points

• Duration of a separate label-collection phase: six months to a year.
• Areas of application: inflated prices, purchases outside their intended purpose, purchases made outside tenders.
• The technique has been tested on various tasks; procurement was chosen as an illustration.
• Architecture: the rules layer comes first, and the ML model is built on top later.
• Main risk of the LLM approach: confident judgments about the market without an external reference.

One limitation is worth stating. The published excerpt gives no details of the mechanics: how exactly verdicts flow back into the dataset, who issues them, how much data is needed for training, and what quality metrics result. That is in the full version of the article, and it is too early to judge the approach's effectiveness from the announcement.

What you can take away for your own work

The logic applies well beyond procurement. If you are building an anomaly detector in finance, logistics, anti-fraud, or industrial monitoring, it helps to first ask where the labels will come from and how long you will have to wait. If the answer is "never" or "a year," the project should be repackaged.

The practical sequence looks like this. Start with simple, verifiable rules that experts understand and are willing to confirm. Build into the process a way to record the human decision on every alert. Treat those decisions as a main product of the pilot, on par with the findings themselves. Bring in the model once enough confirmed and rejected cases have accumulated, and use an LLM only where there is an external reference against which its claims can be checked.

This path is less flashy than "we'll train a neural network on everything." But the business gets results in weeks rather than a year, and the team ends up with its own labeled dataset instead of hoping for someone else's.