Bias is anything in the way data is collected that pushes the result one way. It can come from who is asked, how they are asked, or how things are measured, and a bigger sample does not fix it.
A survey at a footy match finds that 90% of people love football. Is that what the whole town thinks?
Bias is anything in how data is collected that pushes the result away from the truth, always in the same direction. A biased result is not just a little off by chance: it is off for a reason.
Predict: Read the problem and decide what you think before you try anything. A wrong guess is fine: it shows you what to look for.
Try: Try it on the board and compare the result with your prediction.
Investigate: Work out why it happens, one step at a time.
Modify: Change the problem on purpose and see what happens.
Practise: A few on your own, to check you have it.
Make: Make your own problem using what you have learned. Expect a few tries; that is normal.
Make your prediction first. Help opens once you've locked it in.
PredictRead the problem and decide what you think before you try anything. A wrong guess is fine: it shows you what to look for.
The library survey
TryTry it on the board and compare the result with your prediction.
Who you ask
Filled dots play sport. The dashed line is the whole school's proportion.
Try it yourselfYour edits are saved in this browser.
Take a convenience sample (the 20 nearest the door) and look where it lands. Then take a simple random sample of 20. Which is closer to the whole school? Was your prediction's reasoning the same?
Use the board. This ticks when you get there.
InvestigateWork out why it happens, one step at a time.
Kinds of bias
Here is a routine for spotting bias.
Worked exampleBefore each step, say what you think comes next.
Survey: people leaving a gym, ’How often do you exercise?’Read the plan.
Who? People at a gym exercise moreWho was asked: sampling bias.
How? People may say more than they doHow they were asked: response bias.
Result: too high. Fix: random people, anonymous answersSay which way it pushes, and a fix.
Read the radio station's poll. Who chose to be in the sample?
Which kind of bias is each?
Surveying football fans about the town's favourite sport
Asking 'Don't you agree litter is a disgrace?'A leading question pushes people to agree.
Weighing parcels on scales that read 200 g too heavy
Asking students in front of their teacher how much homework they really doPeople give the answer they think is wanted.
Only people who chose to reply to an online poll (they selected themselves)Self-selection: the people in the sample chose themselves, and they often feel strongly. Some books call it voluntary response bias.
A radio station runs a poll. Click the line where the reasoning goes wrong.
ModifyChange the problem on purpose and see what happens.
Does a bigger sample help?
Need a hint?
Make n bigger, then take a sample. Which year groups are in it?
Try it yourselfYour edits are saved in this browser.
Make the convenience sample 80 or bigger. Is it still too high?
Use the board. This ticks when you get there.
PractiseA few on your own, to check you have it.
Spotting bias
MakeMake your own problem using what you have learned. Expect a few tries; that is normal.
Fix a biased survey
The canteen wants to know if students would buy a healthier lunch. It plans to ask the students in the canteen queue: "Wouldn't you love a fresh, delicious salad bar?" with their names on the form. Name every bias, say which way it pushes the result, and write a fairer plan.
Need a hint?
Check who is asked, how they are asked, and how it is measured.
Another hint
Random choice, a neutral question and anonymous answers fix most problems.
For teachers
Students see a convenience sample miss the population on a simulated school, then name and fix sampling, response and measurement bias in real plans (AC9M8ST01, node 2). About 40 minutes.
Do the Run board yourself first. Students need census and sample (last lesson).
The one idea: bias comes from who is asked (sampling), how they are asked (response) or how it is measured (measurement), and it pushes the result one way. The common mistake is thinking a big sample cancels bias (the radio phone-in).
The board's convenience sample is the students nearest the door: the Year 7 rows. Year 7s play more sport and sleep more, so the result is pushed up.
Wider: the 1936 Literary Digest poll (over 2 million replies, wrong winner) is the classic large biased sample.
Running the lesson
Predict: ask who is in the library at lunch.
Run: ask which year groups the convenience sample holds.
Investigate: ask who chooses to phone a radio station.
Modify: ask whether growing the sample changed who is in it.
Make: ask for one fix for each bias.
The hard part (on purpose)
Saying which way the bias pushes. Ask: compared with everyone, would this group answer higher or lower?
When students get stuck
A student says the sample is fine because it is big. Ask: Who is in it? Would more of the same people change that? Fix: Size reduces chance error, not bias.
A student can name bias but not its direction. Ask: Would this group answer higher or lower than everyone? Fix: Say the direction: 'too high because...'.
A student calls any small sample biased. Ask: Was anyone more likely to be chosen than others? Fix: Small random samples vary, but they aren't biased; biased means pushed one way.
Stretch and support
Stretch: Research the 1936 Literary Digest poll. How could 2.4 million replies get the winner wrong?
Support: A card with the three kinds of bias and a question to ask for each: who? how asked? how measured?
Where it leads
This completes AC9M8ST01.
Where it leads: sampling methods (next), and investigations with samples.
What each step is for
the library survey: Committing to how a library survey answers 'do we need a bigger library?' shows whether they see who was asked.
who you ask: A convenience sample from one corner of the school gives a result pushed one way; a random sample does not.
kinds of bias: Bias comes from who is asked, how they are asked, or how things are measured; a big biased sample is still biased (the radio poll).
does a bigger sample help?: Growing a convenience sample doesn't remove its bias, while leading questions and faulty tools bias any sample.
spotting bias: Naming the source and direction of bias in new plans, mixed; thinking a big sample cancels bias is the planted slip; techniques are the review.
fix a biased survey: Rewriting a biased plan to remove each kind of bias shows they can design fairly.