How AI Learns from Mistakes

The Error Signal: Loss Functions

A loss function is a mathematical formula that converts the difference between a prediction and the correct answer into a single number. This number, the loss, is the model's quantified mistake. The entire training process exists to minimize this number.

Different tasks require different loss functions. For classification (is this email spam or not?), cross-entropy loss is standard. It heavily penalizes confident wrong predictions. If the model says "99% spam" and the email is legitimate, the loss is very high. If the model says "51% spam" and the email is legitimate, the loss is much smaller. This asymmetry teaches the model to be cautious: being wrong with high confidence is far more costly than being wrong with low confidence.

For regression (predict the house price), mean squared error (MSE) is the default. MSE squares the difference between the predicted price and the actual price, which means large errors are penalized quadratically more than small ones. Predicting $200,000 when the true price is $300,000 produces a loss 100 times larger than predicting $290,000 when the true price is $300,000. This pushes the model to eliminate large errors first, even at the expense of small ones.

For text generation, the loss is typically the negative log probability of the correct next token. If the correct next word is "Paris" and the model assigned a probability of 0.85 to "Paris," the loss for that token is small. If the model assigned only 0.01 to "Paris," the loss is very large. Summed across all tokens in the training data, this loss pushes the model toward assigning high probability to the correct next word in every context.

Tracing the Blame: Backpropagation

Once the loss is computed, the model needs to figure out which of its billions of parameters contributed to the error and by how much. This is the job of backpropagation, short for "backward propagation of errors."

Backpropagation uses the chain rule of calculus to compute the gradient (the rate of change) of the loss with respect to every parameter in the network. The gradient tells you two things about each parameter: the direction to change it to reduce the loss (increase or decrease), and how sensitive the loss is to that parameter (how much to change it).

The computation flows backward through the network, from the output layer to the input layer. At the output layer, the gradient of the loss with respect to the output is straightforward to compute. At each preceding layer, the gradient is computed using the chain rule, multiplying the downstream gradient by the local gradient of each operation. By the time the computation reaches the first layer, every parameter in the network has a gradient.

This backward pass takes roughly the same amount of computation as the forward pass (making the prediction), which makes it efficient enough to repeat billions of times. Without backpropagation, you would have to estimate gradients numerically by individually perturbing each parameter and measuring the change in loss, which would take billions of forward passes per gradient computation instead of one backward pass. Backpropagation made training deep networks practical.

Making the Correction: Gradient Descent

With gradients in hand, the optimizer adjusts every parameter in the direction that reduces the loss. The simplest version is stochastic gradient descent (SGD): subtract a fraction of the gradient from each parameter. The fraction is the learning rate, a hyperparameter that controls how large each correction step is.

If a parameter's gradient is positive (increasing the parameter would increase the loss), the parameter is decreased. If the gradient is negative (increasing the parameter would decrease the loss), the parameter is increased. The magnitude of the gradient determines how much the parameter changes: parameters that contributed more to the error get larger adjustments.

Modern optimizers like Adam improve on basic SGD by adapting the learning rate for each parameter individually. Parameters with consistently large gradients (indicating the loss is sensitive to them) get smaller learning rates to prevent overshooting. Parameters with small, noisy gradients get larger learning rates to make meaningful progress. Adam also incorporates momentum, using a running average of past gradients to smooth out noise and accelerate convergence through flat regions of the loss landscape.

One Mistake at a Time, Millions of Times

A single gradient descent step produces a tiny improvement. The model might reduce its loss by 0.001% on the current batch of examples. But training consists of millions of these steps, each making a small correction. Over time, the corrections accumulate into a model that has learned complex, generalizable patterns.

The process is not a straight line. Early in training, the loss drops rapidly as the model learns the most obvious patterns: common words in text, prominent edges in images, strong correlations in tabular data. As training progresses, the easy gains are exhausted and the model works on subtler patterns. The loss decreases more slowly, and individual gradient steps become smaller as the model nears a good configuration.

Training is typically organized into epochs (one full pass through the training data) and batches (subsets of data processed together). Instead of computing the gradient over the entire dataset (which would be slow), stochastic gradient descent computes it over a small batch (typically 16 to 256 examples). Each batch gives a noisy estimate of the true gradient, but the noise is actually beneficial. It helps the model escape shallow local minima (poor solutions that look good locally) and find broader, more generalizable solutions.

What Makes Error Correction Effective

Several properties of the training process make error correction work well in practice, even though the optimization landscape has billions of dimensions.

Error signals are specific. Backpropagation does not just say "you were wrong." It says precisely how wrong, and which parameters contributed most. A misclassified cat image produces gradients that are largest in the neurons responsible for cat-related features, leaving unrelated features relatively untouched. This specificity means the model can fix its cat recognition without degrading its dog recognition.

Errors from different examples reinforce consistent patterns. If 100 different cat images all produce gradients that strengthen the same set of cat-related features, those features converge quickly and reliably. Random noise in individual examples averages out, while consistent patterns accumulate. This is why more data leads to better models: more examples produce cleaner, more reliable error signals.

The loss landscape is surprisingly navigable. Despite having billions of dimensions, the loss landscapes of neural networks have favorable geometric properties. Research has shown that local minima in high-dimensional networks tend to have similar loss values, meaning you do not need to find the global minimum, just any good local minimum. Saddle points (flat regions that are not minima) are more common obstacles than local minima, and momentum-based optimizers are effective at crossing them.

When Error Correction Goes Wrong

Vanishing gradients. In very deep networks, gradients can become extremely small as they propagate backward through many layers. When gradients vanish, early layers stop learning because their error signal is too weak to produce meaningful parameter updates. Techniques like residual connections (skip connections), careful weight initialization, and batch normalization address this by keeping gradients at a reasonable magnitude throughout the network.

Exploding gradients. The opposite problem: gradients grow exponentially as they propagate backward, causing parameter updates so large that the model's weights diverge to infinity. Gradient clipping (capping the gradient magnitude at a threshold) is the standard fix.

Catastrophic forgetting. When a trained model is retrained on new data, the error correction for new examples can overwrite the knowledge learned from old examples. The model corrects its new mistakes at the expense of reintroducing old ones. This is particularly problematic in continual learning scenarios where the model must learn new tasks without losing performance on previous tasks.