The comeback illusion
Retest unusually high or low scores and separate real change from chance.
Retest the extremes
1 question · No time limit
Example
Example
Without an intervention, how does the selected group’s expected retest mean compare with its first-test mean?
Stable scores are equally likely to be 47, 51, 55. On each measurement, an independent temporary offset is equally likely to be −4, 0, 4. The first score is stable score plus offset. Selection rule: keep first scores of at most 47. Stable scores do not change. The retest gets a fresh independent offset with mean zero.
1. Predict
| Stable score | −4 | 0 | +4 |
|---|---|---|---|
| 47 | 43 | 47 | 51 |
| 51 | 47 | 51 | 55 |
| 55 | 51 | 55 | 59 |
Highlighted cells meet the selection rule.
Make a prediction first. It is not scored.
2. Test your idea
Retest a cohort to watch selected first scores and new scores diverge.
Each group has 12 people per cohort. Thin lines show the latest people; the bold line shows their mean. Table averages include all simulated cohorts.
Try the controls, then check your solution.
Why?
The selection keeps 3 equally likely stable-score/first-offset combinations. Their first-test mean is 137/3.
Their stable-score mean is 145/3. The fresh temporary offset averages zero, so the expected untreated retest mean is 145/3, a change of 8/3.
Regression toward the mean is a statement about averages after selection, not a rule that every person must move toward the population mean. In real studies, attributing a difference to treatment requires a valid comparable control design.
Revisit your prediction
Their stable-score mean is 145/3. The fresh temporary offset averages zero, so the expected untreated retest mean is 145/3, a change of 8/3.
Keep exploring the experiment.
3. Change the rules
A linked challenge. This answer is not scored.
Continue your unfinished round
Your answers are saved in this tab. Continue from where you left off.
Continuing uses no additional practice start.
Discard the unfinished round?
Your answers from this round will be discarded. You can choose your settings before starting a new round.
Starting the new round uses one practice start.
- Answered
- 0 / 1
- Active time
- 0:00
Round complete.
Optional and off by default. Share starts, completions, replays and error categories. For visits from ChatGPT, starts and completions can also include the first QuickenBrain page and broad country. No answers, drawings or names.
How this data is usedHelp
Try the controls, then check your solution.
Your first checked solution is recorded. You can keep experimenting afterwards.
About this practice
Learn a method
Separate change from selectionRetest unusually high or low scores and separate real change from chance.
For a broader mix of practices, explore the statistics practice test. Membership is required to start a test.
Report a problem
Optional and off by default. Share starts, completions, replays and error categories. For visits from ChatGPT, starts and completions can also include the first QuickenBrain page and broad country. No answers, drawings or names.
How this data is used