CartPole Balancing AI
CartPole is the fruit fly of reinforcement learning: a pole hinged on a cart must be kept upright by pushing the cart left or right, and an episode ends when the pole leans past 12° or the cart runs off the track. Try it yourself in Manual mode with the arrow keys and chase your personal best, then switch to Train and watch the cross-entropy method teach a linear policy to balance: each generation samples 30 weight vectors, keeps the 6 best by episode length, refits the sampling distribution around them and then shows the current champion attempting a live run. The chart tracks best and mean episode length per generation — most runs solve the task (500 steps) within a few dozen generations. Sliders for pole length and push force let you make the problem harder.
Runs 100% in your browser — models are trained and computed locally on your device.
Read the full guide to this tool
Notes
- The physics follow Barto, Sutton and Anderson’s 1983 formulation: cart mass 1 kg, pole mass 0.1 kg, gravity 9.8 m/s², a fixed-magnitude push and Euler integration at dt = 0.02 s.
- A linear policy is enough — push right when w·(x, ẋ, θ, ω) + b ≥ 0. CartPole is solvable without any neural network.
- The cross-entropy method is derivative-free: sample policies from a Gaussian, evaluate them, refit the Gaussian to the elite fraction and repeat — an evolution strategy rather than gradient descent.
- Balancing manually is hard because the pole is an inverted pendulum: deviations grow exponentially, so corrections must anticipate the fall rather than react to it.
- Runs 100% in your browser — models are trained and computed locally on your device.
Related tools
AI Learns to Drive (Neuroevolution)
Watch a population of neural-network cars evolve from crashing into corners to lapping a race track.
Q-Learning Gridworld
Paint a gridworld and watch a Q-learning agent discover the best path by trial and error.
Multi-Armed Bandit Lab
Race ε-greedy, UCB1 and Thompson sampling on the same bandit and compare their regret.
Backpropagation Step-by-Step
Walk through one forward and backward pass of a tiny neural network, number by number.
Decision Tree Builder
Grow a CART decision tree split by split and watch the partition map and tree diagram build together.
Diffusion Model Toy
Train a tiny denoising diffusion model and watch it turn pure noise back into a 2D shape.