Distinguish a real effect from the change expected after selecting unusual first results.
This is the first lesson in its subject. A free account reads it in full.
You are reading the opening. Members read every lesson and guide in full. A free account reads the first lesson in each subject in full.
You are reading the opening. Your free account reads the first lesson in each subject in full. Members read every lesson and guide in full.
First lesson in this subject: Read data before calculating
We couldn’t check your membership. Please reload and try again.
Members read every lesson and guide in full and practice without limits. View membership
In this small model, everyone has a stable score of 50. On each occasion, independent noise is equally likely to be −10, 0 or +10. The first scores are 40, 50 or 60. Selecting only people who scored 60 selects a favorable first noise value, not a stable score of 60.
You are reading the opening. Members read every lesson and guide in full. A free account reads the first lesson in each subject in full.
You are reading the opening. Your free account reads the first lesson in each subject in full. Members read every lesson and guide in full.
First lesson in this subject: Read data before calculating
We couldn’t check your membership. Please reload and try again.
Practice · Retest unusually high or low scores and separate real change from chance.
Book · Read distributions, question samples, and explore how groups and outliers change a conclusion.
Previous lesson · Understand what confidence intervals cover, what more data changes and what bias leaves behind.
Browser storage is unavailable. Changes last only on this page.