ABC Tools

t-SNE, Explained

After reading this you will know how t-SNE turns distances into neighbour probabilities, why it separates clusters so cleanly, and which features of a t-SNE plot you are allowed to trust.

t-SNE (t-distributed stochastic neighbour embedding) is a way to place high-dimensional points on a flat plane so that points which were close together stay close together. It does not try to preserve distances. It tries to preserve neighbourhoods: who is near whom. That single choice explains almost everything about how the plots look and how they can mislead you.

Here is the hook. Take four Gaussian blobs living in 10 dimensions. Your eyes cannot see 10 dimensions, and a plain projection onto two axes often smears the blobs on top of each other. Feed the same 300 points to t-SNE and after a few hundred gradient steps you see four tight, well-separated islands, coloured by their true labels. The algorithm never saw the labels. It found the structure from distances alone.

What t-SNE does, in one idea

t-SNE builds two probability distributions over pairs of points and then makes them agree.

  1. In the original high-dimensional space, it asks: if you stood at point i and picked a neighbour at random with probability falling off like a Gaussian, how likely would you pick point j? Call that p_{ij}.
  2. In the 2D map, it defines a similar probability q_{ij}, but using a heavy-tailed Student-t kernel instead of a Gaussian.
  3. It moves the 2D points by gradient descent until the q_{ij} pattern looks as much like the p_{ij} pattern as possible.

The measure of "look as much like" is the Kullback-Leibler divergence, a number that is zero when the two distributions match and grows as they diverge.

When to use it, and when not to

Reach for t-SNE when you have vectors in tens or hundreds of dimensions and you want a picture that shows whether the data forms distinct groups. It is a visualisation method. It shines at revealing local clusters that linear projections hide.

Do not use a t-SNE plot to measure anything. The distance between two clusters on screen is not proportional to their true separation. The size of a blob does not reflect the spread of that group. If you need those quantities, use a method that preserves them.

t-SNE is not a clustering algorithm and it is not a feature transform you feed downstream. It gives no rule for placing a new point, and re-running it gives a different (equally valid) layout. If you want assignments, run k-Means Clustering or DBSCAN Clustering on the original data, not on the 2D coordinates.

For a fast linear baseline, compare against the PCA Toy. PCA keeps global variance and distances but cannot bend a curved manifold flat. t-SNE can, at the cost of throwing distances away. The demo lets you toggle PCA ghosts so you can see both on the same data.

The formulas and what each symbol means

Start with the high-dimensional similarities. For each ordered pair, define a conditional probability with a Gaussian centred on point i:

p_{j|i} = \frac{\exp(-\|x_i - x_j\|^2 / 2\sigma_i^2)}{\sum_{k \ne i} \exp(-\|x_i - x_k\|^2 / 2\sigma_i^2)}

Here \|x_i - x_j\| is the ordinary distance between the two original vectors, and \sigma_i is the Gaussian bandwidth for point i. Close points get a large numerator, far points get a tiny one. The denominator normalises so the probabilities from i sum to 1.

Each point gets its own \sigma_i, chosen so the distribution has a target perplexity. Perplexity is roughly the effective number of neighbours. It is defined as 2^{H} where H is the Shannon entropy of the p_{j|i} distribution in bits. The algorithm binary-searches \sigma_i until the perplexity matches your setting.

The pair probabilities are then symmetrised: p_{ij} = (p_{j|i} + p_{i|j}) / (2n) for n points.

In the map, similarities use a Student-t kernel with one degree of freedom:

q_{ij} = \frac{(1 + \|y_i - y_j\|^2)^{-1}}{\sum_{k \ne l} (1 + \|y_k - y_l\|^2)^{-1}}

The y_i are the 2D coordinates you are solving for. The (1 + d^2)^{-1} shape has a fat tail: it decays slowly, so moderately distant map points still carry non-trivial similarity. This is the fix for the crowding problem. A Gaussian in 2D would force everything into a cramped ball; the t kernel gives dissimilar points room to spread out.

The cost function is the KL divergence between the two distributions:

C = \sum_{i \ne j} p_{ij} \log \frac{p_{ij}}{q_{ij}}

Because the log ratio is weighted by p_{ij}, the penalty is largest when true neighbours (large p_{ij}) are placed far apart in the map (small q_{ij}). Getting near-pairs right matters far more than getting far-pairs right, which is exactly why local structure survives and global structure does not.

Gradient descent on C gives the update shown in the tool footnote:

\frac{\partial C}{\partial y_i} = 4 \sum_j (p_{ij} - q_{ij})(y_i - y_j)(1 + \|y_i - y_j\|^2)^{-1}

Read the sign. When p_{ij} \gt q_{ij} the points are truer neighbours than the map shows, so the term pulls them together. When p_{ij} \lt q_{ij} it pushes them apart. The layout settles where every pair balances.

A Gaussian kernel with bandwidth 1 assigns similarity 0.607 at distance 1 and 0.135 at distance 2. A Student-t kernel assigns 0.5 at distance 1 and 0.2 at distance 2. The t kernel keeps more weight at large distances, which lets the map spread clusters apart.

A worked example on four 10D Gaussians

Reproducing the demo defaults

The demo button loads four Gaussian clusters in 10 dimensions, 300 points total, at the default perplexity. You can follow the first few numbers by hand for a tiny slice.

Suppose point i has three candidate neighbours at squared distances 0.5, 2.0, and 8.0, and pick 2\sigma_i^2 = 2.

  1. Unnormalised weights: e^{-0.5/2} = 0.7788, e^{-2/2} = 0.3679, e^{-8/2} = 0.01832.
  2. Sum: 0.7788 + 0.3679 + 0.01832 = 1.165.
  3. Conditional probabilities: 0.6685, 0.3158, 0.01573. The nearest candidate holds two-thirds of the mass; the far one is almost ignored.

Now put the same three points in the map at squared distances 0.5, 2.0, 8.0 and apply the t kernel:

  1. Weights: (1+0.5)^{-1} = 0.6667, (1+2)^{-1} = 0.3333, (1+8)^{-1} = 0.1111.
  2. Sum: 1.111. Probabilities: 0.6000, 0.3000, 0.1000.

Compare the far point: the Gaussian gave it 0.01573, the t kernel gives it 0.1000, about six times more. That extra tail weight is what lets the map place dissimilar points far away without paying a large cost.

Points: the Gaussian input weights. Line: the Student-t map weights. At squared distance 8 the t kernel keeps far more weight, which is the crowding fix.

Reading the plot without fooling yourself

The trustworthy signal in a t-SNE plot is which points share a blob. If the four Gaussians land as four clean groups, the algorithm confirms that neighbourhoods separate in the original space. That is real.

Three things on the plot are not trustworthy:

Between-cluster distance
Two blobs 3 cm apart are not "closer" than two blobs 6 cm apart. The KL cost barely constrains far-pair distances, so the gap is mostly an accident of initialisation.
Cluster size
A tight true cluster and a loose one can end up the same size on screen. t-SNE inflates dense regions and shrinks sparse ones to equalise local density.
Absolute position and rotation
The whole picture can rotate, flip, or translate between runs with no change in meaning.

Run the demo three times with a fresh random init each time. The colours stay grouped, but the arrangement of the groups changes. Trust what is stable across runs (the grouping) and distrust what moves (the layout).

Common mistakes

Reading one perplexity as the truth. Perplexity near 5 exposes fine local structure and can shatter one real cluster into fragments. Perplexity near 50 favours global shape and can merge distinct clusters. For 300 points, try 5, 30, and 50 and keep only the features that persist.

Over-interpreting the swiss roll. On the 3D swiss roll, t-SNE unrolls the sheet into a strip, which looks impressive. But the strip's length is not the roll's arc length. It is a neighbourhood map, not a ruler.

Too few iterations. Early exaggeration multiplies the attractive p_{ij} terms during the opening iterations so clusters pull apart cleanly. If you stop before the exaggeration ends and the layout settles, you see half-formed clumps. Let it run to convergence.

Learning rate mismatch. Too low and the points barely move in the iterations you watch; too high and clusters fly apart into a diffuse cloud. If the embedding looks like uniform noise, lower the learning rate and re-run.

Expecting it to scale. This demo is O(n^2) per iteration, fine for 300 points. Real datasets use Barnes-Hut or FFT approximations that bring the cost to O(n \log n).

Related tools on this site

t-SNE only draws a picture. To act on structure, pair it with a method that assigns points. Fit soft elliptical clusters with Gaussian Mixture & EM, or build a tree of merges with Hierarchical Clustering & Dendrogram. For a different take on laying data out on a grid that preserves topology, see the Self-Organizing Map. And once you can see the groups, an Anomaly Detection Playground helps you reason about the points that sit in no group at all.

Frequently asked questions

Why do clusters look separated even when they are not?

The heavy-tailed t kernel plus early exaggeration actively push different neighbourhoods apart. Given a target perplexity, t-SNE will often display gaps even between groups that overlap slightly in the original space. Check the PCA ghosts and try a smaller perplexity before you trust a gap.

What perplexity should I pick?

There is no single right value. It should be smaller than the number of points and roughly the size of the neighbourhoods you care about. For 300 points, values between 5 and 50 are reasonable. Run several and keep the features that appear in all of them.

Why does every run look different?

t-SNE starts from a random initialisation and descends a non-convex cost. Different starts reach different local minima, all with similar KL cost. Rotation and reflection are free. The grouping is stable; the exact layout is not.

Can I use the 2D coordinates as features for a model?

No. t-SNE has no way to map a new point into the existing embedding, and the coordinates encode neighbourhood rank, not distance. Use the original high-dimensional vectors for any downstream model.

How is t-SNE different from PCA?

PCA is a linear projection onto directions of maximum variance; it preserves large distances and global geometry but cannot flatten a curved manifold. t-SNE is non-linear and preserves local neighbourhoods while discarding distances. Use PCA when magnitudes matter, t-SNE when you only need to see the groups.

Try it yourself in the tool

Related tools

Distribution Sketcher

Sketch the shape of a probability density — or paste raw data — and search 16 distributions for the ones that fit it best.

PCA Toy

Find the axes of maximum variance and flatten a point cloud onto them.

k-Means Clustering

Step through Lloyd’s algorithm and watch centroids chase the middles of your point clouds.

Anomaly Detection Playground

Flag outliers three different ways and see where each score breaks down.

DBSCAN Clustering

Grow density-based clusters point by point and watch noise get left behind.

Gaussian Mixture & EM

Fit soft, elliptical clusters with the expectation–maximization algorithm.