Learning Without Supervision

How Biological Brains Learn Without Labels

The brain uses several learning mechanisms that operate without external supervision, each suited to different types of information and different timescales.

Hebbian learning strengthens synaptic connections between neurons that are consistently active together. When neuron A repeatedly fires just before neuron B, the synapse from A to B is strengthened. This simple rule, formalized as "neurons that fire together wire together," allows neural circuits to detect and encode statistical regularities in their inputs without any external signal about what those regularities mean. Hebbian learning is believed to underlie the formation of receptive fields in sensory cortex, where neurons become tuned to specific features (edges, colors, spatial frequencies) that are statistically prevalent in natural sensory input.

Spike-timing-dependent plasticity (STDP) refines Hebbian learning by incorporating the precise timing of pre-synaptic and post-synaptic spikes. If the pre-synaptic neuron fires just before the post-synaptic neuron (suggesting a causal relationship), the synapse is strengthened. If the post-synaptic neuron fires first (suggesting no causal relationship), the synapse is weakened. STDP implements a form of causal learning that allows neural circuits to learn temporal sequences and predictive relationships, all without external supervision.

Predictive coding operates at the circuit level, with each layer of the cortical hierarchy generating predictions about its inputs and learning from the prediction errors. Neurons that accurately predict their inputs become suppressed (their prediction errors are small), while neurons that encounter unexpected inputs become activated (their prediction errors are large). This mechanism allows the brain to continuously refine its internal model of the world based on the ongoing stream of sensory experience, without requiring any labeled training data.

Sleep replay and consolidation. During sleep, the hippocampus replays recent experiences to the cortex, allowing cortical networks to gradually extract statistical regularities from episodic memories. This process transforms specific, context-dependent memories into general, abstract knowledge, effectively performing a form of unsupervised statistical learning on the brain own memory stores. The brain literally teaches itself by reviewing its own experiences during sleep.

Unsupervised Learning in AI

Classical unsupervised learning methods in AI focus on discovering structure in unlabeled data. Clustering algorithms (k-means, hierarchical clustering) group similar data points together. Dimensionality reduction methods (principal component analysis, autoencoders) find compact representations that preserve the essential structure of high-dimensional data. Generative models (variational autoencoders, generative adversarial networks) learn the underlying probability distribution of the data, enabling the generation of new samples that resemble the training data.

These methods capture some aspects of biological unsupervised learning, but they lack the temporal dynamics, the multi-scale organization, and the integration with other cognitive processes that characterize brain-based learning. A brain does not simply cluster its inputs or reduce their dimensionality; it builds a predictive model of the world that supports perception, action, and planning across multiple timescales.

Self-Supervised Learning: The AI Breakthrough

Self-supervised learning has emerged as the most successful AI paradigm for learning from unlabeled data. The key insight is to create supervisory signals from the data itself, by withholding part of the input and training the system to predict the missing part from the rest.

In natural language processing, masked language modeling (as used in BERT) randomly hides words in a text and trains the model to predict them from context. Autoregressive language modeling (as used in GPT) trains the model to predict the next word given all previous words. Both approaches allow models to learn rich linguistic representations from raw text without any labeled examples.

In computer vision, contrastive learning methods like SimCLR and DINO learn visual representations by training the model to recognize that different augmented views of the same image should have similar representations, while views of different images should have dissimilar representations. Masked image modeling (as in MAE) extends the masked prediction approach from language to vision, training models to reconstruct masked patches of images from the visible patches.

These methods have produced representations that rival or exceed supervised learning on downstream tasks, and they scale effectively with data and model size. More remarkably, the representations they learn increasingly resemble those found in the biological brain. Self-supervised visual models develop representations that correlate with neural activity in the primate visual hierarchy, suggesting that the brain and these AI systems are converging on similar solutions to the problem of learning useful representations from raw sensory input.

Connections to Neuroscience

The success of self-supervised learning in AI has reinvigorated interest in unsupervised learning theories in neuroscience. Predictive coding, which has been proposed as a unifying theory of cortical computation, is essentially a form of self-supervised learning where the supervisory signal is the prediction error between expected and actual inputs. The brain is constantly predicting its own sensory input, and it learns by correcting its predictions when they are wrong.

Contrastive learning has parallels in the brain as well. The hippocampal memory system appears to implement a form of contrastive learning by creating pattern-separated representations of similar experiences (making similar things distinguishable) while also performing pattern completion (recognizing the same thing across different contexts). This dual process of separation and completion is functionally similar to the contrastive objectives used in self-supervised visual learning.

The forward-forward algorithm proposed by Geoffrey Hinton in 2022 is explicitly designed as a more biologically plausible alternative to backpropagation. Rather than propagating errors backward through the network, the forward-forward algorithm trains each layer locally using two forward passes: one with real data (positive pass) and one with corrupted data (negative pass). Each layer learns to have high activation for real data and low activation for corrupted data, using only locally available information. While still in early stages of development, this approach points toward training methods that could operate in biological neural tissue without requiring the biologically implausible machinery of backpropagation.

The Gap That Remains

Despite impressive progress, current self-supervised AI systems still fall short of biological unsupervised learning in several important ways. Biological brains learn continuously from a single, ongoing stream of experience, while AI systems typically learn from large static datasets processed in multiple epochs. Biological brains integrate unsupervised learning with reinforcement learning, supervised learning from social feedback, and active exploration driven by curiosity, while AI systems typically use these learning paradigms in isolation. And biological brains learn to support an open-ended range of downstream tasks without knowing in advance what those tasks will be, while AI representations are typically evaluated on a fixed set of benchmarks.

Closing this gap is essential for building artificial brains that can learn with the flexibility, efficiency, and generality of biological organisms. The convergence of neuroscience insights and AI engineering is accelerating progress, and the coming years are likely to produce increasingly brain-like self-supervised learning systems that narrow the gap between artificial and biological unsupervised learning.