Backpropagation Step-by-Step
Backpropagation is the algorithm that makes neural networks trainable: it applies the chain rule backwards through the network to find how much each weight contributed to the error. This playground draws a fixed 2→2→1 network large enough to label every node and edge with its actual value. Type in a training example (x₁, x₂ and a target y), pick sigmoid or tanh, and step through the whole pass with Next/Previous: weighted sums, activations, output, loss, and then every gradient edge by edge, with the chain-rule factors written out as multiplication chains in a side panel with the real numbers substituted in. Active nodes and edges glow at each step. Set the learning rate, apply the update, and a mini-chart tracks the loss falling with every pass you commit.
Runs 100% in your browser — models are trained and computed locally on your device.
Read the full guide to this tool
Notes
- Chain rule per weight: ∂L/∂w = ∂L/∂out · ∂out/∂z · ∂z/∂w — each factor is a local derivative, and backprop just multiplies them along the path from the loss to the weight.
- The sigmoid derivative σ′(z) = σ(z)(1 − σ(z)) never exceeds 0.25, and tanh′(z) = 1 − tanh²(z) never exceeds 1 — every layer a gradient passes through can shrink it.
- Gradient descent update: w ← w − η·∂L/∂w. Too small a learning rate η crawls; too large a one overshoots the minimum and the loss can rise instead of fall.
- Backprop computes all gradients in one backward sweep at roughly the cost of two forward passes — far cheaper than nudging each weight separately.
- Runs 100% in your browser — models are trained and computed locally on your device.
Related tools
Handwritten Digit Recognizer
Draw a digit and watch a neural network trained on MNIST recognize it live in your browser.
Neural Network Playground
Train a tiny neural network live in your browser and watch its decision boundary learn four classic datasets.
Activation Function Explorer
Compare sigmoid, tanh, ReLU and friends with their derivatives, and see why deep gradients vanish.
Convolution Filter Visualizer
Slide a 3×3 kernel over an image and watch convolution, ReLU and max-pooling build the output pixel by pixel.
Hopfield Network
Store pixel patterns in a Hopfield memory, corrupt them, and watch the network recall the original.
Self-Attention Visualizer
See how Transformer tokens attend to each other with arcs, heatmaps, heads and causal masks.