--- title: "Bias in collecting data" subtitle: "Data and chance · Samples · Lesson 16" --- # Bias in collecting data **Name:** {{field-24}} **Date:** {{field-15}} **Class:** {{field-15}} > **Key question:** A survey at a footy match finds that 90% of people love football. Is that what the whole town thinks? :::teacher-note **In one sentence:** Students see a convenience sample miss the population on a simulated school, then name and fix sampling, response and measurement bias in real plans (AC9M8ST01, node 2). **Students should be able to say:** I can spot bias in the way data is collected. I can say which way a bias pushes the result. I can change a plan to make it fairer. **Width:** the unmarked parts are the floor every student does (predict, run, investigate and make). Typical adds the modify practice; wider adds the real-world contexts. They add contexts, not difficulty. Students work in the online tutorial alongside this sheet. ::: ## Predict :::keep-together **Students in the library at lunch are asked: 'Does the school need a bigger library?' How will the result compare with what the whole school thinks?** ☐ Too few yes answers ☐ Too many yes answers ☐ About right My prediction, and why: {{lines-3}} ::: :::teacher-note **What happens:** Too many yes answers, most likely. Students who choose the library at lunch are more likely to want a bigger one. ::: ## Run Try it on the board in the tutorial. Was your prediction right? What was different? {{lines-2}} ## Kinds of bias :::keep-together ### Kinds of bias Here is a routine for spotting bias. 1. $\text{Survey: people leaving a gym, 'How often do you exercise?'}$ — Read the plan. 2. $\text{Who? People at a gym exercise more}$ — Who was asked: sampling bias. 3. $\text{How? People may say more than they do}$ — How they were asked: response bias. 4. $\text{Result: too high. Fix: random people, anonymous answers}$ — Say which way it pushes, and a fix. **Which kind of bias is each?** Groups: Who is asked (sampling bias), How they are asked (response bias), How it is measured (measurement bias) - Surveying football fans about the town's favourite sport: {{field-15}} - Asking 'Don't you agree litter is a disgrace?': {{field-15}} - Weighing parcels on scales that read 200 g too heavy: {{field-15}} - Asking students in front of their teacher how much homework they really do: {{field-15}} - Only people who chose to reply to an online poll (they selected themselves): {{field-15}} ::: :::teacher-note **Answer:** Surveying football fans about the town's favourite sport → Who is asked (sampling bias); Asking 'Don't you agree litter is a disgrace?' → How they are asked (response bias); Weighing parcels on scales that read 200 g too heavy → How it is measured (measurement bias); Asking students in front of their teacher how much homework they really do → How they are asked (response bias); Only people who chose to reply to an online poll (they selected themselves) → Who is asked (sampling bias) Sampling bias: the sample isn't like the population. Response bias: the question or setting pushes answers. Measurement bias: the tool or method is off. ::: :::keep-together ### Find the mistake **A radio station runs a poll. Click the line where the reasoning goes wrong.** ``` 1 A radio station asks listeners to phone in: "Should the speed limit be raised?" 2 1200 people call, and 80% of them say yes. 3 With so many people, the result must show what most drivers think. ``` Line: {{field-15}} Why is it wrong? {{lines-2}} ::: :::teacher-note **Answer:** line 3 Line 3: the callers chose themselves, and people who want a change are more likely to call. 1200 biased answers are still biased: a bigger sample doesn't fix bias. ::: :::keep-together ### A fairer library survey **How could the library survey be made less biased?** ☐ Ask the librarians ☐ Ask students chosen at random from the whole school roll ☐ Ask only Year 7s ☐ Ask more students in the library ::: :::teacher-note **Answer:** Ask students chosen at random from the whole school roll Random choice from the whole population gives every student the same chance of being asked. ::: :::typical ## Does a bigger sample help? **Bigger convenience sample.** **Leading question.** **Faulty tape.** **Honest answers.** Work in the tutorial. Plan first: what will you change, and how will you test it? {{box-4}} ::: ## Spotting bias :::keep-together ### Screen time **To find how much screen time teenagers have, a researcher surveys students at a gaming expo. The result is likely to be:** ☐ Exactly right, if enough are asked ☐ Too low ☐ Too high ☐ About right ::: :::teacher-note **Answer:** Too high People at a gaming expo probably have more screen time than teenagers in general: sampling bias. ::: :::keep-together ### Online poll **An online poll gets 50 000 votes. Why might it still be biased?** ☐ Online polls can't count ☐ 50 000 is too few ☐ It isn't biased: it is so big ☐ Voters choose themselves, so people who care a lot are overrepresented ::: :::teacher-note **Answer:** Voters choose themselves, so people who care a lot are overrepresented Self-selected samples are biased however big they are. ::: :::keep-together ### A fair sample **Which is a fair sample of 30 from a school's 600 students?** ☐ One Year 9 class of 30 ☐ 30 students picked by a random number generator from the roll ☐ 30 volunteers ☐ The first 30 students at the gate ::: :::teacher-note **Answer:** 30 students picked by a random number generator from the roll Random choice from the whole roll gives every student the same chance. ::: :::keep-together ### Review: technique **Review: scientists give half the tomato plants a new plant food and compare their growth. What technique is this?** ☐ Observation ☐ Sample survey ☐ Experiment ☐ Census ::: :::teacher-note **Answer:** Experiment Changing something on purpose and comparing is an experiment. ::: ## Make: fix a biased survey The canteen wants to know if students would buy a healthier lunch. It plans to ask the students in the canteen queue: "Wouldn't you love a fresh, delicious salad bar?" with their names on the form. Name every bias, say which way it pushes the result, and write a fairer plan. Work in the tutorial. Plan first: what will you change, and how will you test it? {{box-4}} ## Exit ticket :::keep-together **A gym surveys its members on how often people exercise. Name the bias and which way it pushes the result.** {{lines-3}} ::: :::teacher-note **Answer:** Sampling bias: gym members exercise more than most people, so the result is too high. :::