Check how the sample was taken before you read the p-value

Ask about the sampling, the sampling fraction and the model condition first; only when all three hold does the p-value decide

Conditions, then the number

Three questions, in order

1. How was the sample taken?

2. What share of the population is it?

3. Does the model's condition hold?

Then: is p below 0.05?

The p-value comes last.

Make the inference describes a study, its sample and a test, and shows the p-value some software reported. You judge whether the software's conditions hold, check its statistic, and choose the conclusion or the next step.

More

Before you read the p-value, ask three questions in order. Was the sample random, or the first observations that were easy to get? Is it under a tenth of the population? Does the model's condition hold: expected counts of at least 10 each for a proportion, a roughly normal population for a small sample mean?

Only when all three hold does the p-value decide: below 0.05 is evidence for the alternative, above it is not enough evidence, and neither proves a hypothesis. When one fails, the next step is its fix, a random sample, an exact test or a finite-population correction, whatever the p-value says.

1Random sample?

2Under a tenth of the population?

3Model condition met? Only then read the p-value.

Lesson help

Check the sampling and the model's conditions first, and read the p-value only when they hold.

About this lesson

Try the method in a practice when you are ready.

Report

Conditions, then the number

Make the inference describes a study, its sample and a test, and shows the p-value some software reported. You judge whether the software's conditions hold, check its statistic, and choose the conclusion or the next step.

Before you read the p-value, ask three questions in order. Was the sample random, or the first observations that were easy to get? Is it under a tenth of the population? Does the model's condition hold: expected counts of at least 10 each for a proportion, a roughly normal population for a small sample mean?

Only when all three hold does the p-value decide: below 0.05 is evidence for the alternative, above it is not enough evidence, and neither proves a hypothesis. When one fails, the next step is its fix, a random sample, an exact test or a finite-population correction, whatever the p-value says.

1Random sample?

2Under a tenth of the population?

3Model condition met? Only then read the p-value.

Three questions, in order

1. How was the sample taken?

2. What share of the population is it?

3. Does the model's condition hold?

Then: is p below 0.05?

The p-value comes last.

A random sample

A simple random sample of 100 learners from a population of 10000: 65 completed the course. The test is H₀: p = 0.5 against p ≠ 0.5, and the software reports p ≈ 0.0027.

Random: yes. Under a tenth: 100 of 10000, yes. Expected counts under H₀: 50 and 50, both at least 10. The conditions hold, and 0.0027 is below 0.05, so the output supports p ≠ 0.5 at the 5% level.

Study

Simple random sample of 100 from 10000

65 of 100 completed

H₀: p = 0.5, H₁: p ≠ 0.5

Software: p ≈ 0.0027

Checks

Random: yes

Share: 100 of 10000, under a tenth

Expected counts: 50 and 50

So read p: 0.0027 < 0.05

All three conditions hold before the p-value is read.

The first 400

A convenience sample takes the first 400 readily available learners from the same population of 10000: 170 completed. The same test, and again the software reports p ≈ 0.0027.

Study

Convenience sample: the first 400 readily available, from 10000

170 of 400 completed

H₀: p = 0.5, H₁: p ≠ 0.5

Software: p ≈ 0.0027

Ask the three questions first.

Try it

Which conclusion or next step fits this output?

  1. The output supports p ≠ 0.5 at the 5% level.
  2. Collect a random sample before inferring.
Show the answer

Collect a random sample before inferring.

Right. A convenience sample leaves the estimate open to selection bias, so its p-value is not evidence about the population. The fix is a random sample.

Enable JavaScript to answer interactively and save your place.

Practice Make the inference

Sign in free

Practice without limits and keep your progress.