Sampling, margin of error and statistical claims
Judge what conclusions data supports: random samples, margins of error, bias, and experiments versus observational studies.
Also known as: margin of error, random sampling, survey bias, experiments vs observational studies.
Key ideas
- Results from a random sample can be generalised only to the population it was drawn from.
- An estimate ± margin of error gives a plausible range for the population value.
- Larger random samples (all else equal) give smaller margins of error.
- Only a randomised experiment supports cause-and-effect conclusions.
Samples and populations
If randomly selected students at one school are surveyed, conclusions apply to that school's students, not all students in the country. Samples chosen for convenience (e.g. volunteers) may be biased.
Margin of error
" of sampled residents support the plan, with a margin of error of " means the true population proportion is plausibly between and . A value outside that interval is unlikely, but values inside it are all plausible; we can't say the true value is exactly .
Cause and effect
To conclude that a treatment causes an outcome, subjects must be randomly assigned to groups. Observational studies can show association only.
Worked examples
Example 1 · Using a margin of error
A random sample estimates that 45% of a town's adults cycle weekly, with a margin of error of 3%. Is it plausible that exactly half the adults cycle weekly?
- Step 1.Plausible range: 42% to 48%.
- Step 2.50% is outside this range.
Answer: No, 50% is not within the plausible range.
Example 2 · Scope of conclusions
Volunteers at a gym tried a new drink and reported more energy. Can we conclude the drink increases energy for all adults?
- Step 1.Volunteers aren't a random sample, and there's no random assignment or control group.
Answer: No, neither generalisation nor causation is supported.
Common mistakes
- Generalising a sample to a wider population than the one sampled.
- Treating the point estimate as the exact population value.
- Claiming causation from an observational study.
Try it yourself
Original practice questions. Your answer is checked instantly.
A random sample estimates that 45% of a town's adults cycle weekly, with a margin of error of 3%. Which is a plausible value for the percentage of all adults in the town who cycle weekly?
A researcher randomly selected 200 students from one university and found that 30% work part-time. To which group can this result be generalised?