Regularization Demo (L1/L2/Dropout)
A network with more capacity than the data deserves will happily memorize noise — regularization is the set of tricks that stops it. Here a small 2-8-8-2 network trains on a noisy two-class dataset (two moons, rings or XOR blobs, with a noise slider), split into training and test points. Three independent controls fight overfitting: an L2 penalty λ that pulls all weights towards zero, an L1 penalty that drives many exactly to zero, and a dropout rate that randomly silences hidden units during training. Press Play and watch the decision heatmap tighten or smooth around the points, the weight histogram contract under L2 or spike at zero under L1 (the near-zero percentage is reported live), and the train and test accuracy curves separate when the network overfits — then close up again as you add regularization.
Runs 100% in your browser — models are trained and computed locally on your device.
Read the full guide to this tool
Notes
- L2 adds λΣw² to the loss, i.e. weight decay w ← w(1 − ηλ) each step: weights shrink proportionally but rarely reach exactly zero.
- L1 adds λΣ|w|, a constant-magnitude pull ηλ·sign(w) — small weights get dragged exactly to zero, so L1 performs feature selection and produces sparse networks.
- Dropout with rate p silences each hidden unit independently with probability p at train time; with inverted scaling by 1/(1 − p) the test-time network needs no change and behaves like an averaged ensemble.
- The telltale of overfitting is the gap between the curves: training accuracy keeps climbing while test accuracy stalls or falls.
- Runs 100% in your browser — models are trained and computed locally on your device.
Related tools
Backpropagation Step-by-Step
Walk through one forward and backward pass of a tiny neural network, number by number.
Handwritten Digit Recognizer
Draw a digit and watch a neural network trained on MNIST recognize it live in your browser.
Neural Network Playground
Train a tiny neural network live in your browser and watch its decision boundary learn four classic datasets.
Activation Function Explorer
Compare sigmoid, tanh, ReLU and friends with their derivatives, and see why deep gradients vanish.
Convolution Filter Visualizer
Slide a 3×3 kernel over an image and watch convolution, ReLU and max-pooling build the output pixel by pixel.
Hopfield Network
Store pixel patterns in a Hopfield memory, corrupt them, and watch the network recall the original.