Forgetful neural networks shed light on how humans learn
How the brain learns is one of the biggest unsolved mysteries in science. Researchers in Artificial Intelligence from the University of Groningen (the Netherlands) and KU Leuven (Belgium) hope that a specific type of forgetting in artificial neural networks will help to capture what is happening in the human brain. The study is published in the journal Trends in Cognitive Sciences.
FSE Science Newsroom | René Fransen
Artificial neural networks are computer systems that are modeled on the human brain: similar to the brain, they consist of a network of units (the neurons) that signal to each other. Such an artificial neural network can learn a new task through a stepwise process, strengthening or weakening the connections between the units to approach the desired outcome as best as possible. This is called gradient descent learning.
Lasting and transient forgetting
Once the system is fully trained on a task (for example, to recognize the image of a boat), training for a second task can begin (such as training to recognize a car). However, computer scientists have known for a while that when learning this second task, the neural network rapidly and lastingly ‘forgets’ how to perform the first task. This is called ‘catastrophic forgetting’ (see figure).
The usual fix to reduce this kind of forgetting is an approach called replay. Take a student who has learned French and then switches to learning Spanish. To keep their French in shape, they might repeat some French lessons during the Spanish course. A network with replay does something similar: while it learns the new task, it keeps practicing on examples from the old task. This often works well, so that by the end of training, the network can do both tasks.

But researchers noticed something surprising when they checked the network’s performance throughout training, and not just at the end. They found that even when the network keeps practicing the old task, its performance on that task still worsens as it starts learning the new task, before recovering later. The researchers called this form of transient forgetting the ‘stability gap’ (see figure).
Presence of a stability gap in human learning
Scientists from the University of Groningen and KU Leuven asked whether a stability gap also occurs in human learning. They performed a survey of the cognitive science literature and identified different types of transient forgetting in humans, but none directly matched the stability gap. Importantly, they found no evidence against a stability gap in human learning either, which prompted them to suggest how future studies could test for it.
The scientists reasoned that the stability gap can provide important insight about how the human brain learns. An important open question is whether the human brain learns by a process that approximates gradient descent. In their paper, they show that the stability gap is a kind of fingerprint left behind by gradient descent learning. This means that if scientists find a stability gap in human learning in future experiments, it would be a striking sign that our brains learn more like machines than we might think. If they look very carefully and it’s not there, that would be just as revealing, suggesting the brain has its own, quite different way of learning.
Reference: Milan Van Maldegem, Hans Op de Beeck, Niels A. Taatgen, Tinne Tuytelaars, Gido M. van de Ven: Lost, then found: transient forgetting in humans and machines. Trends in Cognitive Sciences, 4 November 2026
More news
-
17 November 2026
Insects react differently to light
-
13 November 2026
Royal Honour for Jan Post