ABC Tools

Polynomial Regression, Explained

After reading this you will be able to fit a curve to paired data, read the R² and adjusted R² it reports, and choose a degree that captures the trend without chasing noise.

What polynomial regression does

Polynomial regression fits a curve of the form \hat{y} = b_0 + b_1 x + b_2 x^2 + \dots + b_d x^d to your data by least squares. It is still linear regression under the hood: the model is linear in the coefficients b_0, b_1, \dots, b_d, even though the curve it draws bends.

Here is the hook. Take the demo data x = 1, 2, 3, 4, 5, 6 and y = 2.1, 4.3, 9.2, 15.8, 25.3, 35.9. A straight line misses the upward curl at the right end. A degree-2 fit, \hat{y} = 1.06 - 0.30 x + 1.06 x^2, tracks it closely and reaches an R² of about 0.9997. The data was built to curve, so a quadratic reads it almost exactly.

When to use it, and when not to

Reach for a polynomial when a scatter plot shows a smooth bend that a straight line cannot follow: acceleration, diminishing returns, a single hump, or a valley. Degree 2 handles one bend, degree 3 handles an S-shape with two bends, and so on. Each extra degree adds one more possible turn.

Avoid it when the true relationship is not polynomial. Growth that levels off toward a ceiling is better described by a logistic or exponential curve; forcing a degree-4 polynomial onto it will wave up and down between your points. If a straight line already fits well, stay with it. The Correlation and Linear Regression Calculator covers that case and reports Pearson r alongside the line.

A polynomial fit is only trustworthy inside the range of your x values. Extrapolating beyond them is dangerous: the quadratic above predicts y = 22.3 at x = 0 going backward, which is nonsense for data that started at 2.1. High-degree terms explode fastest outside the data.

The math and the intuition

The calculator minimizes the sum of squared residuals. Write the fitted value as

\hat{y}_i = \sum_{j=0}^{d} b_j x_i^{\,j}

Here x_i is the i-th x value, d is the degree, and b_j is the coefficient on the j-th power. The residual for point i is e_i = y_i - \hat{y}_i, and the fit chooses the coefficients that make

\text{SSE} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2

as small as possible. The solution comes from treating each power of x as its own predictor column, then solving the normal equations. You do not have to do that by hand; the point is that the curve is the unique least-squares choice for the degree you picked.

Intuition: a degree-d polynomial has d + 1 coefficients, so it can pass exactly through any d + 1 points. With 6 points, a degree-5 curve hits every point and SSE drops to zero, but that curve is describing the noise, not the trend.

Worked example with the demo data

Fitting a quadratic to the six points

Use x = 1, 2, 3, 4, 5, 6, y = 2.1, 4.3, 9.2, 15.8, 25.3, 35.9, and degree 2.

  1. The least-squares fit gives \hat{y} = 1.057 - 0.296 x + 1.061 x^2.
  2. Predict each point. At x = 3: 1.057 - 0.296(3) + 1.061(9) = 9.72, against the observed 9.2, a residual of -0.52.
  3. At x = 6: 1.057 - 0.296(6) + 1.061(36) = 37.48, against 35.9, a residual of -1.58.
  4. Square and sum all six residuals to get SSE. Compare it to the total variation in y around its mean \bar{y} = 15.43 to form R².

The reported R² is about 0.9997, adjusted R² about 0.9995. The residuals stay small and show no obvious pattern, which is the sign of a good fit.

The dots are the six data points; the smooth curve is the degree-2 fit. It threads the points closely and bends upward as x grows.

Reading the results

Equation
The fitted coefficients from highest power down. Read the sign and size of each term to see how the curve bends.
R²
The fraction of variation in y explained by the fit, from 0 to 1. An R² of 0.9997 means the curve leaves only 0.03% of the variation unexplained.
Adjusted R²
R² penalized for the number of coefficients. It can go down when you add a degree that does not earn its keep.
Residuals
The gaps y_i - \hat{y}_i. Scan them for structure. Random scatter around zero is good; a smooth wave means the degree is wrong.

R² almost always rises as you add degrees, because a more flexible curve can only fit the training points better or equally. Adjusted R² is the honest number to watch. It is defined as

R^2_{\text{adj}} = 1 - (1 - R^2)\,\frac{n - 1}{n - p}

where n is the number of points and p is the number of coefficients (p = d + 1). When you go from degree 2 to degree 4 on the demo data, plain R² inches from 0.9997 toward 1, but the shrinking denominator n - p means adjusted R² stops improving and can fall.

With the six demo points, degree 2 gives R² 0.9997 and adjusted R² 0.9995. Degree 3 nudges R² up but leaves adjusted R² about the same. Degree 5 forces R² to exactly 1.0000 because the curve passes through every point, yet the curve wiggles between points and predicts poorly off the data.

Common mistakes

Choosing degree by R² alone. R² will keep climbing. On the demo data a degree-5 fit reaches R² 1.0000 because six points and six coefficients let the curve pass through everything. That is memorization, not a trend.

Too few points for the degree. You need more points than coefficients, and comfortably more. Fitting degree 4 (five coefficients) to six points leaves only one degree of freedom, and adjusted R² becomes unstable.

Ignoring the residual pattern. If the residuals curve smoothly instead of scattering randomly, the degree is too low. If they swing wildly at the ends, the degree is too high.

Extrapolating. The fit describes the data range only. Predicting far outside it lets the leading term dominate and the estimate diverges fast.

Start at degree 2 and raise it one step at a time. Stop at the first degree where adjusted R² stops improving and the residuals look patternless. That is usually the lowest degree that captures the trend.

Plain R² climbs with every degree, while adjusted R² levels off after degree 2 and collapses at degree 5, where there are no residual degrees of freedom.

Related tools

If a straight line is enough, the Correlation and Linear Regression Calculator fits it and reports Pearson r and Spearman ρ. To compare predictions against actuals with several error measures, use the Regression Metrics Calculator for MSE, RMSE, MAE and MAPE. For classification rather than regression, the Confusion Matrix Metrics Calculator gives accuracy, precision, recall and F1.

Frequently asked questions

What degree should I use?

The lowest one that captures the shape. Try degree 2 first. Raise it only if the residuals still show a smooth pattern and adjusted R² keeps rising. On the demo data degree 2 is enough: R² is already 0.9997.

Why does R² keep going up with higher degrees?

More coefficients give the curve more freedom, so it can only fit the same points better or equally. That is why you compare adjusted R², which subtracts a penalty for each coefficient, and why you inspect residuals instead of chasing R² to 1.

How many data points do I need?

Strictly more than d + 1, and preferably several times that. With six points, keep the degree at 2 or 3. A degree-5 fit on six points passes through every point and tells you nothing about the trend.

Is polynomial regression the same as linear regression?

Yes, in the sense that it is linear in the coefficients and solved the same way. The x-powers x, x^2, \dots, x^d act as separate predictors, so the standard least-squares machinery applies.

Can I trust predictions outside my data range?

No. The fit is reliable only between your smallest and largest x. Outside that window the highest-power term takes over and the curve can shoot off in either direction.

Try it yourself in the tool

Related tools

Correlation & Linear Regression Calculator

Pearson r, Spearman ρ, R² and the least-squares line.

Confusion Matrix Metrics Calculator

Accuracy, precision, recall, F1, MCC and more from TP/FP/FN/TN.

Regression Metrics Calculator

MSE, RMSE, MAE, MAPE and R² from actual vs. predicted values.

Binomial Distribution Calculator

Probabilities of k successes in n independent trials.

Chi-Square Test of Independence

Test association between two categorical variables from a contingency table.

Combinations & Permutations Calculator

nCr and nPr, with and without repetition.