Complementary Learning Systems (CLS) theory, developed by James McClelland, Bruce McNaughton, and Randall O'Reilly (1995), provides a computational rationale for why the brain uses two distinct memory systems with different learning rates. The theory addresses the fundamental stability-plasticity dilemma: a system that learns quickly risks catastrophically overwriting previous knowledge, while a system that learns slowly cannot capture individual episodes.
The Catastrophic Interference Problem
Standard neural network models that learn through gradient descent suffer from catastrophic interference: training on new patterns rapidly destroys memory for previously learned patterns. McCloskey and Cohen (1989) and Ratcliff (1990) demonstrated that this problem is inherent in networks that use overlapping distributed representations and modify shared weights. CLS theory proposes that the brain solves this problem architecturally, through two complementary systems:
The Hippocampal System
The hippocampus uses sparse, pattern-separated representations and a high learning rate. Sparse coding (implemented through competitive inhibition in the dentate gyrus) ensures that even similar experiences are represented by non-overlapping neural populations, minimizing interference between episodes. The high learning rate allows one-shot encoding of individual experiences. However, the sparseness means that the hippocampus cannot discover shared statistical structure across experiences.
The Neocortical System
The neocortex uses distributed, overlapping representations and a slow learning rate. The distributed coding means that similar items share representational features, enabling generalization and extraction of statistical regularities (categories, prototypes, schemas). The slow learning rate prevents individual experiences from catastrophically disrupting the accumulated structure. However, the neocortex cannot rapidly encode individual episodes without disrupting existing knowledge.
Interplay and Consolidation
The two systems interact through a process of memory consolidation. New episodes are rapidly encoded in the hippocampus and then gradually "replayed" to the neocortex during offline periods (particularly sleep). Each replay is equivalent to a small training step for the neocortex, allowing it to integrate new information with existing knowledge without catastrophic interference:
This replay-based consolidation explains why hippocampal damage causes temporally graded retrograde amnesia (recent memories that have not yet been consolidated are lost, while remote memories that have been transferred to the neocortex are preserved) and why sleep deprivation impairs memory consolidation.
The catastrophic interference problem that motivated CLS theory has re-emerged as a central challenge in modern deep learning under the name "continual learning" or "lifelong learning." Techniques such as elastic weight consolidation (Kirkpatrick et al., 2017) and experience replay buffers are directly inspired by CLS principles, demonstrating the enduring relevance of this psychological theory for artificial intelligence.