Multi-Armed Bandit Lab
The multi-armed bandit is the cleanest setting for the exploration–exploitation dilemma: k arms pay out with unknown probabilities, and you want to find and exploit the best one while wasting as few pulls as possible. This lab runs four strategies side by side on identical reward streams — pure greedy, ε-greedy, UCB1 and Thompson sampling — and plots their cumulative regret, the payoff lost compared with always playing the best arm. Set the number of arms, randomize the hidden means, tune ε and UCB’s exploration constant, and watch the regret curves separate: greedy plateaus on a wrong arm, ε-greedy pays a constant exploration tax, while UCB1 and Thompson bend toward flat. Inspect per-arm pull counts and Thompson’s Beta posterior distributions to see what each strategy believes.
Runs 100% in your browser — models are trained and computed locally on your device.
Read the full guide to this tool
Notes
- Regret is the gap between the reward earned by always pulling the best arm and what the strategy actually earned; good algorithms achieve regret that grows only logarithmically with time.
- UCB1 pulls the arm maximising mean + c·√(2·ln t / n): the bonus shrinks as an arm gets pulled — optimism in the face of uncertainty.
- Thompson sampling keeps a Beta(successes+1, failures+1) posterior per arm, samples one value from each and pulls the argmax — randomness in belief does the exploring.
- Pure greedy has linear regret: with no exploration bonus it can settle forever on a suboptimal arm after a few unlucky first draws.
- Runs 100% in your browser — models are trained and computed locally on your device.
Related tools
AI Learns to Drive (Neuroevolution)
Watch a population of neural-network cars evolve from crashing into corners to lapping a race track.
Q-Learning Gridworld
Paint a gridworld and watch a Q-learning agent discover the best path by trial and error.
CartPole Balancing AI
Balance the pole yourself, then watch the cross-entropy method evolve a policy that never drops it.
Backpropagation Step-by-Step
Walk through one forward and backward pass of a tiny neural network, number by number.
Decision Tree Builder
Grow a CART decision tree split by split and watch the partition map and tree diagram build together.
Diffusion Model Toy
Train a tiny denoising diffusion model and watch it turn pure noise back into a 2D shape.