Estimation and Hypothesis Testing

From a sample to the population: the central limit theorem, confidence intervals and sampling, the hypothesis-testing process, parametric tests of means, variances and correlations, and non-parametric tests including tests of independence.

1. 1. The central limit theoremWhatever the shape of the population, the mean of a large random sample is approximately normally distributed around the true mean, with variance $\sigma^2/n$.Free13 min
2. 2. The standard error of the sample meanThe standard error measures how precisely a sample mean estimates the population mean: $\sigma/\sqrt{n}$, or $s/\sqrt{n}$ when $\sigma$ is unknown.Video · 6 min13 min
3. 3. Confidence intervals for the meanA confidence interval turns a point estimate into a range: point estimate ± reliability factor × standard error.15 min
4. 4. Value at risk: a one-sided confidence intervalValue at risk turns a one-sided confidence interval into a risk number: the loss a position should not exceed over a set horizon at a chosen confidence level.13 min
5. 5. Populations, samples and sampling errorWe study a sample because examining the whole population is impossible or too costly, and the price we pay is sampling error.Free13 min
6. 6. Stratified and cluster samplingStratified sampling draws randomly from every subgroup to guarantee representation and precision; cluster sampling picks whole subgroups to save time and money.14 min
7. 7. Good estimators, and the CLT across sampling designsA good estimator is unbiased, efficient, consistent and, when the data are messy, robust; and how fast the sample mean becomes normal depends on the population's shape and on how the sample was drawn.13 min
8. 8. Convenience and judgmental sampling, and choosing a methodNon-probability samples are chosen by convenience or expert judgment: quick and cheap, but at real risk of not representing the population.12 min
9. 9. The four steps and stating the hypothesesA hypothesis test asks whether a sample result lies so far from a claimed population value that chance alone is a poor explanation.Free13 min
10. 10. Type I and Type II errors, significance and powerA test can go wrong in two ways, rejecting a true null (Type I) or missing a false one (Type II); the significance level caps the first, and power measures how well the test avoids the second.Video · 5 min12 min
11. 11. Critical values, p-values and the decision ruleReject the null when the test statistic lands beyond the critical value or, equivalently, when the p-value is smaller than the significance level.Video · 7 min13 min
12. 12. p-values and p-hackingA p-value is only honest if the test was fixed before looking at the results; re-running variations until p dips below 5% (p-hacking) manufactures false positives.12 min
13. 13. Testing a single meanTo test a claim about a population mean with unknown variance, measure how many standard errors the sample mean lies from the claim, using a t-statistic with n − 1 degrees of freedom.13 min
14. 14. Two means: independent vs paired samplesBefore comparing two means, decide whether the samples are independent (pooled t-test on the difference in means) or related (paired comparisons test on the mean of the differences).Video · 6 min14 min
15. 15. Testing variances: chi-square and FA claim about one variance is tested with chi-square; a comparison of two variances is tested with the F-ratio of the sample variances.Video · 7 min14 min
16. 16. The t-test for a Pearson correlationA sample correlation is never exactly zero, so convert it into a t-statistic with n − 2 degrees of freedom and ask whether it is too large to be sampling noise.13 min
17. 17. Sample size, p-values and many correlations at onceWhether a correlation counts as significant depends heavily on how many observations you have; with enough data even a tiny correlation passes the test.13 min
18. 18. Parametric vs nonparametric testsParametric tests are about parameters and lean on distribution assumptions; when those assumptions fail, the data are ranks, or the question is not about a parameter, use a nonparametric test.12 min
19. 19. The Spearman rank correlation testWhen the data are not normal, contain outliers or are already ranks, replace each value by its rank and correlate the ranks instead.Video · 7 min14 min
20. 20. Contingency tables and the chi-square test of independenceFor categorical data, compare the count in each cell with the count you would expect if the two classifications had nothing to do with each other.Video · 6 min14 min
21. 21. Interpreting the result: standardized residuals and mosaicsRejecting independence tells you the classifications are related; standardized residuals tell you which cells are responsible and in which direction.12 min
22. 22. Choosing the test: four steps and a map of every testEvery test in this module runs the same four steps; what changes is the question (known value, two unknowns, association), and that question picks the statistic.14 min
23. 23. Integrated cases: testing whether a new strategy beats the old oneA strategy-comparison case runs the whole testing chain on one data set: hypotheses, the right statistic for unequal samples, the decision at two levels, a check on the variance assumption, and an honest reading of what significance does and does not show.16 min