SUM: A Server-Side Fix for Forgetting in Federated Continual Learning, Up to 22% Gain

A team of researchers posted SUM (Unified Geometric Surgery on Spatio-Temporal Adaptation Vectors) to arXiv in July 2026. It is a server-side framework for federated continual learning that reduces "catastrophic forgetting" and improves quality by up to 22% over earlier FCIL methods.

What problem does SUM solve

SUM targets "spatial-temporal catastrophic forgetting" (ST-CF): knowledge lost where two training paradigms meet. Federated Class Incremental Learning (FCIL) combines federated learning (FL), in which a model learns from the isolated data of different clients, with continual learning (CL), in which tasks arrive sequentially.

According to the arXiv paper, this combination creates two sources of interference at once. The first is spatial interference from heterogeneous clients whose data differ. The second is temporal interference from sequential tasks, where new knowledge displaces old. Earlier methods handled these problems with separate mechanisms, often burdened clients with extra computation or data exchange, and did not regulate directional interactions between updates during aggregation.

How does the "geometric surgery" work

SUM runs entirely on the server and corrects "adaptation vectors" while updates are being aggregated. The authors propose treating FCIL as a single multi-task learning problem, in which updates from both clients and tasks are vectors in a shared parameter space.

• SUM is a server-side framework with no extra computation, communication or memory on the client
• Spatial SUM removes inter-client interference within each training round
• Temporal SUM (causal online) removes inter-task interference over time
• Quality gain of up to 22% over previous FCIL methods
• Tested on computer vision and language benchmarks, with robustness to unreliable clients

How much better is it than earlier approaches

SUM outperforms previous FCIL methods by up to 22% across a range of vision and language benchmarks, while remaining computationally efficient. The key difference is that all the "repair" happens during aggregation on the server, so clients do not have to compute anything extra or send additional data.

"Spatial SUM reduces client-level interference within each round, while the causal online

Temporal SUM removes inter-task interference over time without extra computation, communication or memory on the client beyond standard federated learning," the paper's abstract on arXiv states.

The authors separately point to the method's robustness against unreliable clients, meaning those that return noisy or incomplete updates. This matters for real-world distributed systems, where some devices are always unstable.

What this means

Federated continual learning is a scenario for systems that must both learn from scattered private data and adapt to new tasks without retraining from scratch. SUM shows that forgetting can be handled on the server without shifting the load onto client devices, which makes the approach applicable to phones, IoT and other hardware with limited resources.

Frequently asked questions

What is Federated Class Incremental Learning?

FCIL combines federated learning (a model learns from the isolated data of many clients without collecting it in one place) with class-incremental learning (new classes and tasks arrive over time). The main difficulty is "catastrophic forgetting," where the model loses old knowledge as it learns new material.

How does SUM differ from previous FCIL methods?

SUM works only on the server and combines the fight against spatial and temporal interference in a single mechanism: "geometric surgery" on adaptation vectors. Earlier approaches used separate mechanisms and often burdened clients, whereas SUM requires no extra computation, communication or memory from them.