Как объяснить непрерывное обучение нейросетей: attention rollout и Grad-CAM на временных рядах

A group of researchers published a paper on arXiv in July 2026 in which explainable AI methods — attention rollout and Grad-CAM — were applied to continual learning of neural networks for time series forecasting, testing the PatchMixer, PatchTST, and DLinear architectures on real-world groundwater level monitoring data.

What the study examined

The authors of the study focused on explainability as a tool that helps understand what happens inside a model when it is fine-tuned on a stream of new data. Time series forecasting in real-world conditions — for example, environmental monitoring — is complicated by non-stationarity: data statistics change over time, and abrupt regime shifts occur. To keep the model from "forgetting" old knowledge while learning new data, researchers apply the Experience Replay strategy — replaying selected past examples. The authors strengthened the selection of these examples with an attention-based mechanism: the model itself "suggests" which fragments of history are more important to preserve for adaptation.

• Task type — continual learning for time series forecasting
• Architectures — PatchMixer, PatchTST, DLinear
• Adaptation strategy — Experience Replay with attention-based example selection
• Explainability methods — attention rollout and Grad-CAM (gradient attribution)
• Data — real piezometric series (groundwater level) with regime shifts

Why explainability matters here

In this work, explainability is not decoration but a way to see the logic behind data selection and model adaptation. Deep learning models have long shown strong results on time series, but their adoption in real-world monitoring is held back precisely by opacity and sensitivity to non-stationary dynamics, the authors note. The attention rollout and Grad-CAM methods make it possible to see which parts of the input series the model relies on when forecasting and which examples it selects for replay. For practice, this is essential: if it is visible which data and features influence the forecast, an engineer can purposefully manage fine-tuning rather than relying on the model as a black box.

What the experiments showed

Experiments on piezometric series showed that analyzing model behavior and the example-selection mechanism yields valuable insight into the dynamics of continual learning. The key observation: feature attribution maps are not static — they evolve over time as regimes in the data change. Attention rollout traces how attention propagates through the model's layers, while Grad-CAM uses gradients to show the contribution of individual segments of the series — together they provide two independent projections of the same forecasting logic. According to the authors, this opens a path toward using explainability for more deliberate data selection and adaptation strategies in non-stationary scenarios.

"Our results highlight the challenges and opportunities of using

explainability to understand continual learning behavior," the paper's abstract on arXiv states.

What this means

The work is more methodological than applied: the authors do not provide specific accuracy metrics in the abstract. But the direction is important — explainability is turning from a post-hoc check into a working tool for managing model fine-tuning on live, changing data. A separate value lies in linking explainability with data selection strategy: attribution suggests which historical examples to bring back into training when a regime shift occurs. For environmental monitoring and other scenarios with data drift, this is a step toward more reliable and transparent forecasts.

Frequently asked questions

Which models were tested in the study?

Three neural network architectures for time series forecasting: PatchMixer, PatchTST, and DLinear. They were augmented with an attention-based example-selection mechanism to support the model's adaptation over time.

What data was used to test the approach?

Real piezometric time series — groundwater level monitoring data with heterogeneous patterns and regime shifts, where the signal's statistics change over time.