When Results Do Not Repeat
Written by Studio AM.
Your result
- Words per minute
- Score
- Pace
Words per minute mean something only with the score beside them. They describe this passage on this read. With fewer than 3 right, try the next slower pace.
Choose a pace and select Start. The passage is paced and timed, then the questions check what you understood. How pace mode works
Reading time 0:00
In 2015, the Open Science Collaboration reported attempts to repeat one hundred studies from three psychology journals. The teams used higher-powered designs and original materials when available. Thirty-six percent of the replications reached statistical significance. Across the project, the average replication effect was roughly half the average size originally reported. That comparison was not limited to results that remained statistically significant.
The findings raised questions about how published evidence is selected and tested, and which incentives influence researchers. A journal's preference for surprising positive results can leave unsuccessful studies unseen. Flexible analysis choices can also make a result look stronger than it is, even when researchers believe their decisions are sensible. Sincerity does not by itself prevent bias. A replication difference can also arise from sampling variation or differences in methods and settings; it is not automatic proof of fraud or of a nonexistent effect.
Reforms aim to make the process easier to examine. Preregistering hypotheses and analysis plans distinguishes planned tests from later exploration. Registered Reports offer publication decisions based on reviewed study plans before the results are known, subject to carrying out and reporting the work properly. Sharing materials, code, and data where privacy permits helps others check the work. Appropriate sample sizes improve precision.
Publishing a result that differs from an earlier study can be valuable. It makes uncertainty visible and gives other researchers a clearer basis for further tests. These practices improve the conditions for checking claims; they do not guarantee that every result will repeat.
The passage is hidden while you answer. It comes back with your result.
Questions
Choose an answer. The explanation appears after you answer.
-
Question 1 of 4
Which statement best captures the argument of the passage?
The answer is C: Research incentives helped produce results that did not repeat, and reforms began to address them.
The passage links replication findings to selection and analysis practices, then describes reforms and their limits.
-
Question 2 of 4
How did average effect sizes compare across the replication project?
The answer is A: The replication average was roughly half the original average.
The opening compares averages across the project, not only statistically significant replications.
-
Question 3 of 4
Why can publishing a result that differs from an earlier study be useful?
The answer is B: It makes uncertainty available for examination and guides further tests.
The final paragraph values an inspectable record of uncertainty without claiming that publication guarantees correctness.
-
Question 4 of 4
What does “sincerity” mean here?
The answer is C: honest belief, with no intention to mislead
Researchers may believe their choices are sensible without intending to mislead, yet still introduce bias.
Score: none answered yet.