Simple Linear Regression

Simple linear regression explains the variation in one variable using one other variable. You learn how least squares fits the line, how to read the coefficients, which assumptions must hold, how to judge the fit with ANOVA, \(R^2\), the F-test and the standard error of the estimate, how to test coefficients, how to build a prediction interval, and how log transformations handle curved relationships.

1. 1. The regression model and least squaresSimple linear regression explains the variation in Y with one variable X by fitting the straight line that makes the sum of squared vertical misses as small as possible.Free14 min
2. 2. Interpreting coefficients, fitted values and residualsThe slope tells you how much Y moves per one-unit move in X, the intercept tells you Y when X is zero, and each residual tells you how far an observation sits above or below the line.12 min
3. 3. The four assumptions and residual plotsOLS conclusions are valid only if the relationship is linear and the residuals have constant variance, are independent and are normally distributed. Residual plots are how you spot violations.Video · 7 min13 min
4. 4. Sums of squares, \(R^2\), the ANOVA table and the SEETotal variation in Y splits into the part the line explains (SSR) and the part it misses (SSE). The ANOVA table organises that split and gives $R^2$, the F-statistic and the standard error of the estimate.Video · 6 min14 min
5. 5. Testing the slope: t-test, correlation test and F-testTo test a slope, divide (estimate − hypothesised value) by its standard error and compare with a t critical value with n − 2 degrees of freedom; the F-test and the correlation test give the same verdict on a zero slope.Video · 7 min15 min
6. 6. Intercept tests, indicator variables and p-valuesThe intercept is tested just like the slope; a 0/1 indicator variable turns the regression into a test of a difference in means; and the p-value tells you the smallest significance level at which you would reject.13 min
7. 7. Forecasts and prediction intervalsA forecast is a point on the fitted line; the prediction interval around it widens with the SEE, with a small sample and with how far the forecast X is from the mean of X.13 min
8. 8. Functional forms: log-lin, lin-log and log-logWhen the relationship is curved, transform Y, X or both with natural logs so that a straight line fits, then read the slope as a relative change and convert forecasts back.Video · 7 min13 min
9. 9. Integrated cases: reading a full regression printoutA regression printout holds every number you need for fit, significance, forecasting and risk; the case is to find the right cell, convert between them, and avoid the labels that look alike.16 min