Subliminal Learning: Do Neural Networks Remember What They Were Made to Forget?

In earlier articles we touched on subliminal learning in neural networks, raising more questions than answers. Now it is time to dig into the phenomenon, drawing on new experiments and code analysis. One of the key questions in AI alignment and large language model (LLM) safety is this: is fine-tuning or reinforcement learning from human feedback (RLHF) a reliable way to remove unwanted or dangerous information originally embedded in a model?

Experiments show that the known effect of mode connectivity makes it practically impossible to fully erase information acquired during pre-training with standard fine-tuning. The point is that a structural "imprint" persists in the topology of the network's weights and can be read out through a kind of "subliminal" channel. Even with the weights fully unfrozen (that is, with every parameter of the network free to change) and aggressive L2 regularization aimed at actively "forgetting" old knowledge, the topology of the latent space formed during pre-training survives and continues to exert a significant influence on how the new task is solved. Recall of old knowledge that was seemingly removed can reach 88-99%.

The mode connectivity effect can be explained as follows: a neural network's loss landscape (that is, the function it tries to minimize during training) has a complex structure with many local minima. Each minimum corresponds to a particular "mode," or way of solving the task. Mode connectivity means these minima are linked by "paths" of relatively low loss, which lets the model switch between different operating regimes while keeping the overall structure of its knowledge.

The implications of this finding for LLM safety and reliability are huge. If unwanted information cannot be fully removed, it may "surface" at the worst possible moment, for example when generating text, interacting with a user or making decisions. This is especially dangerous for models used in critical areas such as healthcare, finance or the justice system.

Moreover, the results call into question the effectiveness of existing AI alignment methods designed to control and steer LLM behavior. If a model retains hidden knowledge that cannot be directly controlled, new and better methods are needed that account for this subliminal learning effect.

One possible direction is to design neural network architectures that are more resistant to retaining unwanted information. Another is to develop more effective fine-tuning methods that not only adapt the model to a new task but also actively "forget" old knowledge without destroying its overall structure.

In conclusion, the study of subliminal learning in neural networks shows that fine-tuning and RLHF are not a panacea for unwanted information. The structural imprint in the weight topology persists and can be activated. That calls for new approaches to AI alignment that account for this effect and aim to build safer, more reliable LLMs.