Linear Regression & Line of Best Fit Calculator

Linear Regression Guide

What's the key thing to understand about Least-Squares Regression?

The least-squares regression line minimises the sum of squared vertical distances from each data point to the line (residuals). Gradient m = Σ(xᵢ−x̄)(yᵢ−ȳ) / Σ(xᵢ−x̄)². Y-intercept c = ȳ − m×x̄. The line always passes through the point (x̄, ȳ) — the mean of both variables. This is the LSRL or line of best fit. It minimises Σ(yᵢ − ŷᵢ)² where ŷᵢ = predicted values. There is only one least-squares line for a given dataset (unlike other fitting methods).

What should I know about R-squared Interpretation?

R² = square of the correlation coefficient r. It represents the proportion of variance in Y explained by X. R² = 0.85 means 85% of the variation in Y is explained by the linear relationship with X. The remaining 15% is due to other factors (lurking variables, random variation). R² = 1: perfect fit. R² = 0: no linear relationship. Important: a high R² does not mean causation, and does not mean the linear model is appropriate — always plot the data. R² can be high even when the relationship is clearly non-linear, which is exactly why plotting the raw data alongside any regression statistic is essential rather than trusting the number alone.

What's the difference between Interpolation and Extrapolation?

Interpolation: using the regression line to predict Y for X values within the range of the data — generally reliable. Extrapolation: predicting outside the data range — unreliable and potentially very wrong. The linear relationship that holds between x=10 and x=100 may not hold at x=500. Example: a linear model relating height and weight in children aged 5-10 is reasonable for predicting weight at age 7. Predicting weight at age 40 from the same model would be nonsense. Always state when predictions move beyond the range of the original data, since the linear relationship observed within that range is not guaranteed to hold outside it.

What's the key thing to understand about Assumptions of Linear Regression?

Linearity: the relationship between X and Y is actually linear — check with a scatterplot. Independence: each observation is independent (no repeated measurements). Homoscedasticity: variance of residuals is constant across all X values — if residuals are larger at high X, this assumption is violated. Normality of residuals: for hypothesis testing, residuals should be approximately normally distributed (less critical with large samples). Residual plot: plot residuals against fitted values. Random scatter around zero in a residual plot supports the model's validity; clear patterns instead suggest the linear model isn't the right fit for the data.

What do I need to know about Calculating the Regression Line by Hand?

The least-squares line y = mx + c has slope m = Σ[(x−x̄)(y−ȳ)] / Σ(x−x̄)², and intercept c = ȳ − m×x̄. For points (1,2), (2,4), (3,5), (4,8): x̄=2.5, ȳ=4.75. Deviations: (−1.5,−2.75), (−0.5,−0.75), (0.5,0.25), (1.5,3.25). Numerator: (−1.5×−2.75)+(−0.5×−0.75)+(0.5×0.25)+(1.5×3.25) = 4.125+0.375+0.125+4.875 = 9.5. Denominator: 2.25+0.25+0.25+2.25=5. So m=9.5/5=1.9, and c=4.75−1.9×2.5=0. The line is y=1.9x. This line minimises the sum of squared vertical distances between each actual point and the line — not the perpendicular distance — which is why it's called 'least squares' and why it treats x as the exactly-known variable and y as the one with error, an asymmetry worth remembering when both variables actually carry measurement uncertainty.

What should I know about Residuals and Checking Model Fit?

A residual is the difference between an actual observed y-value and the value predicted by the regression line: residual = y_actual − y_predicted. Plotting residuals against x reveals whether a straight line was really the right model — residuals scattered randomly around zero with no pattern indicate a good linear fit, while a clear curve, funnel shape, or trend in the residuals signals that the relationship isn't actually linear, or that variance isn't constant across the range (heteroscedasticity). This residual check matters more than R² alone, because R² can look deceptively high even when a straight line is the wrong model for curved data — famously demonstrated by Anscombe's quartet, four very different-looking datasets that all produce nearly identical regression lines and R² values. Always sketch the scatter plot and residuals, not just the summary statistics, before trusting a linear regression result.

What's the difference between Correlation and Regression — What Each One Tells You?

Correlation (measured by Pearson's r) and regression are closely related but answer different questions. Correlation measures the strength and direction of a linear relationship between two variables, symmetrically — it doesn't matter which variable you call x and which you call y, and r is the same either way. Regression goes further: it fits a specific line, y = mx + c, that lets you predict one variable (the dependent variable, y) from the other (the independent variable, x), and this line does change depending on which variable you're predicting. Two variables can have a strong correlation without one meaningfully "causing" or predicting the other in a useful direction, which is why regression requires choosing — based on the actual question being asked — which variable is being explained and which is doing the explaining, a decision correlation alone never requires.

What should I know about a Worked Example with Correlation Strength?

As a fully worked example including correlation strength, consider exam scores plotted against hours studied: (2,65), (4,70), (6,75), (8,85), (10,95). The means are x̄=6 and ȳ=78. Deviations from the mean: x: −4,−2,0,2,4; y: −13,−8,−3,7,17. The slope's numerator, Σ(x−x̄)(y−ȳ) = (−4×−13)+(−2×−8)+(0×−3)+(2×7)+(4×17) = 52+16+0+14+68 = 150. The denominator, Σ(x−x̄)² = 16+4+0+4+16 = 40. So m=150/40=3.75, and c=78−3.75×6=55.5, giving y=3.75x+55.5. Predicting the score for 5 hours of study: y=3.75×5+55.5=74.25. To find R², also compute Σ(y−ȳ)² = 169+64+9+49+289 = 580, then Pearson's r = 150/√(40×580) = 150/√23,200 ≈ 0.985, so R²≈0.970 — about 97% of the variation in exam scores is explained by hours studied.

What should I know about the Standard Error of the Estimate?

R² tells you what proportion of variance is explained, but it doesn't directly say how far typical predictions might be off in the original units. That's what the standard error of the estimate measures: roughly, the typical size of a residual, calculated as s = √(Σ(y−ŷ)²/(n−2)) for a simple linear regression (the n−2 divisor accounts for the two estimated parameters, slope and intercept). A smaller standard error means predictions from the line tend to land closer to the actual observed values, in real units. This is a more directly interpretable companion to R² — a model could have a respectable R² of 0.75 but still carry a standard error large enough to make individual predictions practically useless, which is why both R² (relative fit) and standard error (absolute prediction accuracy) are worth reporting together.

What should I know about Interpreting the Strength of R-squared?

These bands are only general guidance, not a strict mathematical rule — what counts as a 'strong' R² varies hugely by field. In controlled physics experiments, an R² below 0.95 might signal a measurement problem, while in social science or biological research, an R² of 0.3 can represent a genuinely important and statistically significant relationship, because human behaviour and biological systems are influenced by many more uncontrolled factors than a physics experiment. Always interpret R² relative to what's typical and achievable within the specific field being studied, not against the same fixed scale across every domain.

What should I know about Multiple Regression (Beyond a Single Predictor)?

This calculator fits a line with a single predictor variable (simple linear regression), but many real relationships depend on more than one input. Multiple regression extends the same least-squares idea to a model like y = b0 + b1x1 + b2x2 + ..., finding the combination of coefficients that minimises the sum of squared residuals across all predictors simultaneously, rather than considering just one input variable in isolation. For example, predicting a house's sale price might use both its floor area and its number of bedrooms as predictors rather than either alone. The coefficients in a multiple regression represent the effect of each predictor while holding the others constant — a subtly different interpretation from the simple regression slope, which doesn't control for any other variable at all. Multiple regression requires matrix algebra to solve efficiently for more than one or two predictors, which is why it's almost always computed using statistical software rather than by hand, unlike the simple two-variable case this calculator handles directly. This single-predictor simplicity is precisely what makes simple linear regression the right starting point before reaching for multiple regression on a genuinely multi-factor problem.