Lesson 23 of 23 · All lessons in this module
  1. The central limit theorem
  2. The standard error of the sample meanVideo lesson
  3. Confidence intervals for the mean
  4. Value at risk: a one-sided confidence interval
  5. Populations, samples and sampling error
  6. Stratified and cluster sampling
  7. Good estimators, and the CLT across sampling designs
  8. Convenience and judgmental sampling, and choosing a method
  9. The four steps and stating the hypotheses
  10. Type I and Type II errors, significance and powerVideo lesson
  11. Critical values, p-values and the decision ruleVideo lesson
  12. p-values and p-hacking
  13. Testing a single mean
  14. Two means: independent vs paired samplesVideo lesson
  15. Testing variances: chi-square and FVideo lesson
  16. The t-test for a Pearson correlation
  17. Sample size, p-values and many correlations at once
  18. Parametric vs nonparametric tests
  19. The Spearman rank correlation testVideo lesson
  20. Contingency tables and the chi-square test of independenceVideo lesson
  21. Interpreting the result: standardized residuals and mosaics
  22. Choosing the test: four steps and a map of every test
  23. Integrated cases: testing whether a new strategy beats the old one

This lesson is part of the 2027 curriculum. You are following the 2026 curriculum, where it is not taught in this form. Switch if you are sitting the exam under the 2027 curriculum.

Lesson 23 of 23 · 16 min

Integrated cases: testing whether a new strategy beats the old one

A strategy-comparison case runs the whole testing chain on one data set: hypotheses, the right statistic for unequal samples, the decision at two levels, a check on the variance assumption, and an honest reading of what significance does and does not show.

In short

  • Put the claim to be proven in HAH_A: to show that the new strategy has the higher mean, H0:μnew≤μoldH_0: \mu_{\text{new}} \le \mu_{\text{old}} and HA:μnew>μoldH_A: \mu_{\text{new}} > \mu_{\text{old}}.
  • Two independent samples with different variances and sizes: t=(xˉ1−xˉ2)/s12/n1+s22/n2t = (\bar x_1 - \bar x_2)/\sqrt{s_1^2/n_1 + s_2^2/n_2}, with n1+n2−2n_1 + n_2 - 2 degrees of freedom in the 2027 reading.
  • The decision follows the tail of the alternative: one-sided here, so the critical value is the one-tailed 5% (or 1%) value. A one-sided rejection can coexist with a two-sided non-rejection.
  • An F-test on the ratio of the sample variances checks whether pooling is justified. Pooling when the variances differ and the sizes differ gives a wrong standard error.
  • Significance is not profit. Costs, and the number of strategies tried before this one, decide what a rejection is worth.

Unlock this lesson free for 7 days

Create a free account to get 7 days free: every lesson, video and flashcard, and the question bank. No card needed.

Start your 7-day free trialLog in