All level 5 passages

When Results Do Not Repeat

In 2015, the Open Science Collaboration reported attempts to repeat one hundred studies from three psychology journals. The teams used higher-powered designs and original materials when available. Thirty-six percent of the replications reached statistical significance. Across the project, the average replication effect was roughly half the average size originally reported. That comparison was not limited to results that remained statistically significant.

The findings raised questions about how published evidence is selected and tested, and which incentives influence researchers. A journal's preference for surprising positive results can leave unsuccessful studies unseen. Flexible analysis choices can also make a result look stronger than it is, even when researchers believe their decisions are sensible. Sincerity does not by itself prevent bias. A replication difference can also arise from sampling variation or differences in methods and settings; it is not automatic proof of fraud or of a nonexistent effect.

Reforms aim to make the process easier to examine. Preregistering hypotheses and analysis plans distinguishes planned tests from later exploration. Registered Reports offer publication decisions based on reviewed study plans before the results are known, subject to carrying out and reporting the work properly. Sharing materials, code, and data where privacy permits helps others check the work. Appropriate sample sizes improve precision.

Publishing a result that differs from an earlier study can be valuable. It makes uncertainty visible and gives other researchers a clearer basis for further tests. These practices improve the conditions for checking claims; they do not guarantee that every result will repeat.

Check your understanding

Answer 4 questions about this passage and see why each answer is right

Questions and pace mode are free with sign-in.

Sign in free to answer

About pace mode

Signed in, you can choose a pace from 150 to 400 words per minute. A highlight moves through the passage, then the questions check what you understood. Choose Off to read at your own speed.

Pace mode measures this reading session. It is not a measure of your overall reading ability.

Practice reading