How ADWIN and online learning help models handle data drift

A machine learning model can show excellent metrics on tests and quickly lose accuracy in production when user behavior, demand, or market conditions change — this phenomenon is called concept drift, and an article on Habr explores how to combat it using a combination of online learning and the ADWIN algorithm.

What is concept drift

Concept drift is a situation where the statistical patterns on which the model was trained no longer correspond to reality over time. A model that showed high accuracy on historical data begins to make mistakes not because it was poorly trained, but because the world it describes has changed: users behave differently, demand shifts, the market responds to new factors.

• Problem — the model loses accuracy in production while test metrics remain unchanged
• Cause — changes in user behavior, demand, or market conditions over time
• Weakness of the classical approach — retraining on schedule often lags behind the moment when drift has already started
• Proposed solution — a combination of online learning with the ADWIN algorithm

Why scheduled retraining doesn't save

The standard approach is to periodically retrain the model from scratch on fresh data according to a fixed schedule. The problem is that the retraining schedule is determined in advance and is not tied to the actual moment when the data started changing: a model can operate on "outdated knowledge" from one retraining cycle to the next, losing accuracy precisely when changes are happening faster than the schedule provides for.

How ADWIN combination helps

ADWIN (ADaptive WINdowing) is an algorithm that monitors statistics of the incoming data stream in a sliding window and signals the moment when the data distribution has changed statistically significantly. Combined with online learning, where the model is fine-tuned incrementally on new examples rather than retrained from scratch, this allows adapting the model to new data without completely resetting accumulated knowledge — that is, responding to drift almost immediately after detection rather than waiting for the next scheduled retraining cycle.

How this differs from classical retraining

The difference between the two approaches is in response speed and amount of lost knowledge. In retraining from scratch, the model essentially starts over on a new data slice, and everything it learned on older but still relevant patterns is lost. Incremental online learning with a drift detector like ADWIN, on the other hand, preserves accumulated knowledge and only corrects the model in response to confirmed statistical change — this reduces computational costs of constant retraining and simultaneously shortens the time the model spends in a state of "outdated" knowledge.

What this means

For teams keeping ML models in production, concept drift is a constant source of quiet quality degradation that test metrics won't show. The combination of online learning with drift detection through ADWIN is a practical way to narrow the gap between when data changed and when the model accounted for it, without needing to maintain a separate full retraining cycle for every change in production.