C3L Breaks the Explanation Drift Loop in Continual Learning: 5 Benchmarks, 20+ Baselines

Catastrophic forgetting in continual learning is usually measured as a drop in accuracy on old tasks. Tel Aviv University researchers Yehonatan Elisha, Oren Barkan and Noam Koenigstein argue that this shows only part of the problem. A model can keep its earlier metrics while no longer looking at a bird's beak, and instead looking at a branch or the background. They call this explanation drift and propose a method, C3L, tested on five benchmarks against more than 20 baseline approaches.

A loop that feeds itself

Existing explanation-aware continual learning methods work like this. They take the saliency map of a model trained on the previous task. Then, while training on the new task, they require the network to stay close to that map. It sounds reasonable, but the authors point to a structural flaw: the regularization target is computed from the model itself. If attention to the background leaks into the map on the first task, it gets frozen and becomes the reference for the next one. At every step the error is copied and amplified. The researchers call this a self-referential drift loop.

The problem is subtle. Accuracy does not fall, and the broken explanation shows up nowhere. For medical diagnostics, industrial inspection, or any system where it matters why the model gave an answer, this is a dangerous scenario.

Concepts instead of pixels

C3L (Concept-Consistent Continual Learning) breaks the loop by taking its regularization targets from outside the model. Instead of the network's own saliency maps, the method uses semantic concepts that are consistent across an entire class. Examples from the paper are "curved beak" and "striped pattern". The concepts are discovered offline and do not depend on how the model changed during training. This gives two properties:

• the target is independent of model drift: the reference does not degrade, no matter how many tasks the network goes through;
• semantic consistency across instances: the same concept applies to all objects in a class, rather than each object getting its own interpretation.

One important detail: C3L does not require human-labeled concepts. Concept discovery and the spatial grounding of concepts to image regions are fully automated. This removes the main practical objection to concept-based methods, where the feature vocabulary is usually compiled by hand.

The crowding-out effect

The authors describe a side mechanism that they themselves consider one of the main results. The method is combined with approximate relevance conservation in LRP (layer-wise relevance propagation). The total relevance attributed to a given prediction is bounded. Concept regions and the background share the same "budget". If the network is forced to concentrate high relevance on concept regions, less is left for the background. Spurious correlations are suppressed without a separate penalty on them. This is the crowding-out effect.

The practical takeaway: you do not need to know in advance which spurious features will appear. It is enough to set the right positive target.

What was measured

The authors claim state-of-the-art results on five axes at once:

• accuracy on sequential learning tasks;
• degree of forgetting;
• explanation quality;
• mitigation of spurious correlations;
• robustness to out-of-distribution data.

The comparison covered five benchmarks and more than 20 baseline methods. For concrete numerical gains, see the paper itself (arXiv 2610.33877). But the choice of metrics shows the intent: to compare not only "how much the model remembers" but also "for what reasons it decides".

Where this is useful

Teams that fine-tune vision models on a stream of new data get a technique they can verify. Examples include manufacturing inspection, medical imaging and monitoring. A simple test: after each round of fine-tuning, compare not only accuracy but also the distribution of relevance between the object's regions and the background. If the background gains weight, that signals explanation drift, even when the metrics are stable.

The limitations are also clear. The approach relies on the quality of automatic concept discovery and its grounding in the image. The work is described on image classification tasks, and transfer to other modalities will have to be checked separately. But the idea of moving the regularization anchor outside the model applies more broadly than continual learning.