Correlation and Linear Regression, Explained

After reading this you can measure how strongly two variables move together, fit the straight line that best predicts one from the other, and tell an honest signal from a misleading one.

What correlation and regression tell you

Correlation answers one question: as one variable rises, does the other tend to rise, fall, or drift with no pattern? The Pearson correlation coefficient r puts a number on that tendency, from -1 (a perfect downhill line) through 0 (no linear link) to +1 (a perfect uphill line).

Linear regression goes one step further. It draws the single straight line that comes closest to your points and gives you an equation, \hat{y} = a + bx, so you can predict y from x. Correlation describes the strength of the association. Regression gives you a formula to use.

Here is the hook. Suppose you record eight days of study hours and test scores and get r = 0.997. That single number says the points sit almost exactly on a line. The regression then tells you the line rises by about 4.4 score points per study hour. Both facts come from the same eight pairs.

When to use it, and when not to

Use Pearson r and the least-squares line when three things hold: the relationship looks roughly straight on a scatter plot, both variables are numeric, and no single point sits far from the crowd. If any of those fail, the numbers still compute, but they mislead.

A curved relationship is the classic trap. Take y = x^2 for x from -3 to 3. The link is perfect and deterministic, yet Pearson r = 0 because the uphill and downhill halves cancel. Correlation only sees the straight-line part.

When the data are ranks or the relationship is monotone but bent, reach for Spearman \rho instead. Spearman is just Pearson r computed on the ranks of the data, so it measures whether y consistently increases with x without demanding a straight line.

The formulas and the intuition

Pearson correlation is the average product of the two variables measured in standard-deviation units:

Here \bar{x} and \bar{y} are the means, and each sum runs over all n pairs. When a point is above average on both x and y, the product (x_i - \bar{x})(y_i - \bar{y}) is positive and pushes r up. When a point is high on one and low on the other, the product is negative and pulls r down. The denominator scales the result into the range from -1 to +1.

The regression slope reuses the same top sum, divided by the spread of x alone:

The slope b is the predicted change in y for a one-unit rise in x. The intercept a forces the line through the point (\bar{x}, \bar{y}). The line found this way minimizes the sum of squared vertical gaps between the points and the line, which is why it is called least squares.

Finally, R^2 is simply r^2 in simple regression. It is the fraction of the variance in y that the line explains. An R^2 of 0.99 means the line accounts for 99 percent of the up-and-down variation in y.

A worked example with the demo data

Reading and interpreting the results

Start with the scatter plot, always. A number cannot tell you whether the shape is straight, curved, or clumped. Then read the coefficients.

Pearson r

Strength and direction of the linear link. Rough guide: 0.1 weak, 0.3 moderate, 0.5 or more strong, but always judge against your field.

Spearman rho

The same idea on ranks. If \rho is much higher than r, the relationship is monotone but bent.

Slope b

Expected change in y per unit of x, in the original units. Here 4.405 score points per study hour.

Intercept a

Predicted y when x = 0. Meaningful only if x = 0 is realistic.

p-value

The probability of seeing a correlation this large if the true correlation were zero. Small (below 0.05) suggests the link is unlikely to be pure chance.

Watch the interplay between r and sample size in the p-value. With only n = 5 points, an r of 0.7 gives a p-value near 0.19, not significant. With n = 30, that same r = 0.7 yields a p-value below 0.0001. More data makes the same correlation far more convincing.

Common mistakes

Extrapolating past the data. The demo line fits days 1 to 8. Using it to predict day 20 gives 47.18 + 4.405 \times 20 = 135, a score above any real test. Nothing guarantees the line stays straight outside the observed range.

Trusting r without the plot. Anscombe's quartet is four datasets with the same r = 0.816 and the same fitted line, yet one is a clean line, one a curve, and one is dominated by a single outlier. Identical numbers, very different truths.

Confusing R squared with accuracy. A high R^2 means the line explains variance in your sample, not that predictions are correct on new data. Check residuals and, when you can, test on held-out points.

Fitting a line to a curve. If the residuals show a smile or frown shape, a straight line is the wrong model. Try the Polynomial Regression Calculator for a degree-2 or higher fit.

Related tools

Once you have predictions, judge their error with the Regression Metrics Calculator, which reports MSE, RMSE, MAE, MAPE and R^2 from actual versus predicted values. For classification problems where the target is a category rather than a number, correlation does not apply; use the Confusion Matrix Metrics Calculator instead.

Related tools

Confusion Matrix Metrics Calculator

Accuracy, precision, recall, F1, MCC and more from TP/FP/FN/TN.

Polynomial Regression Calculator

Fit a polynomial of degree 2–8 with R² and residuals.

Regression Metrics Calculator

MSE, RMSE, MAE, MAPE and R² from actual vs. predicted values.

Binomial Distribution Calculator

Probabilities of k successes in n independent trials.

Chi-Square Test of Independence

Test association between two categorical variables from a contingency table.

Combinations & Permutations Calculator

nCr and nPr, with and without repetition.