Conditions, then the number
Make the inference describes a study, its sample and a test, and shows the p-value some software reported. You judge whether the software's conditions hold, check its statistic, and choose the conclusion or the next step.
Before you read the p-value, ask three questions in order. Was the sample random, or the first observations that were easy to get? Is it under a tenth of the population? Does the model's condition hold: expected counts of at least 10 each for a proportion, a roughly normal population for a small sample mean?
Only when all three hold does the p-value decide: below 0.05 is evidence for the alternative, above it is not enough evidence, and neither proves a hypothesis. When one fails, the next step is its fix, a random sample, an exact test or a finite-population correction, whatever the p-value says.
1Random sample?
2Under a tenth of the population?
3Model condition met? Only then read the p-value.
Three questions, in order
1. How was the sample taken?
2. What share of the population is it?
3. Does the model's condition hold?
Then: is p below 0.05?
A random sample
A simple random sample of 100 learners from a population of 10000: 65 completed the course. The test is H₀: p = 0.5 against p ≠ 0.5, and the software reports p ≈ 0.0027.
Random: yes. Under a tenth: 100 of 10000, yes. Expected counts under H₀: 50 and 50, both at least 10. The conditions hold, and 0.0027 is below 0.05, so the output supports p ≠ 0.5 at the 5% level.
Study
Simple random sample of 100 from 10000
65 of 100 completed
H₀: p = 0.5, H₁: p ≠ 0.5
Software: p ≈ 0.0027
Checks
Random: yes
Share: 100 of 10000, under a tenth
Expected counts: 50 and 50
So read p: 0.0027 < 0.05
The first 400
A convenience sample takes the first 400 readily available learners from the same population of 10000: 170 completed. The same test, and again the software reports p ≈ 0.0027.
Study
Convenience sample: the first 400 readily available, from 10000
170 of 400 completed
H₀: p = 0.5, H₁: p ≠ 0.5
Software: p ≈ 0.0027
Try it
Which conclusion or next step fits this output?
- The output supports p ≠ 0.5 at the 5% level.
- Collect a random sample before inferring.
Show the answer
Collect a random sample before inferring.
Right. A convenience sample leaves the estimate open to selection bias, so its p-value is not evidence about the population. The fix is a random sample.
Enable JavaScript to answer interactively and save your place.
Practice Make the inference