Evaluating Methods of Sampling from a Set of Data

Introduction

Many surveys are conducted in order to describe how large groups of people feel about a particular issue.

clipboard with a checklist

Pollsters may want to know which issues are important to voters in an upcoming election.

 

principal standing behind desk

A school principal may want to know how parents, teachers, or students feel about an issue so that he may make a good decision for the school. Principals must make decisions about bus routes and times, school lunches, and extracurricular activities. Knowing how parents, teachers, or students feel about those things helps the principal make a better decision.

 

woman standing between shelves in a library

Community leaders may also use surveys to make decisions about how to allocate resources. Parks, libraries, and museums are all important features of any community. Making sure that people who live in the community have access to these features is important for decision making.

 

President Truman holding a copy of the newspaper with a false headline

In this resource, you will investigate different ways to survey a group of people and describe the attitudes and beliefs of that group. You will also look for ways to make sure that the results of the survey accurately reflect the attitudes and beliefs of the group. This accuracy is very important. Otherwise, you may end up with a situation like what President Harry S. Truman faced in 1948, when even though he was re-elected, a newspaper printed a headline proclaiming "Dewey Defeats Truman."

 

Source for images use in this section:
President Truman at Union Station in St. Louis, Missouri with "Dewey Defeats Truman headline,” Harry S Truman Library and Museum
All other images, Clipart

 

Methods of Sampling

Texas flag placed on Texas on a globe

 

One common purpose for surveys is to find out how people feel about candidates for a political office. Every four years, Texas elects statewide officeholders. In 2012, there were about 26,400,000 people living in Texas. Of these, approximately 14,000,000 were eligible to vote.

 

People in line

The entire group that a survey seeks to describe is called the population. 14,000,000 people is a lot of people to ask about how they feel about particular candidates. So, instead of surveying every member of the population, a political pollster might ask a small part of the population, or a sample of the population.

Survey samples must be carefully constructed, otherwise there could be problems when interpreting the survey results.

 

Important Note!
For a survey to lead to accurate conclusions, the sample must be representative of the population. Random samples are the best way to ensure that the sample represents the entire population.

Sources of Survey Bias

woman with clipboardRandomization is one important way to make sure that survey data can be used to reach valid conclusions.

Another factor that may influence the conclusions reached by survey data is bias. A survey has bias if the survey data has a tendency to overestimate or underestimate a particular characteristic that is being surveyed. In other words, the results of the survey are tilted toward one particular viewpoint.

Bias can result from a variety of sources. One type of bias, selection bias, results from the selection of a sample that is not representative of the population being described.

Another type of bias, response bias, can come from the way that a survey instrument itself is constructed.

Selection bias example: wanting to know how all 8th grade students feel about the upcoming basketball game but only asking basketball players. Response bias example: asking leading questions, such as, "You don't want school to start any earlier do you?"

Practice

For the following questions, determine if the sample is representative of the population or not.

  1. A neighborhood organization wants to know how many people in the neighborhood would support a new bus route, so they survey every fourth person exiting a bus on an existing route.
  2. Margie wants to know how many people are likely to take a new Computer Technology course in her school, so she surveys every fifth person entering school on Tuesday.
  3. Ruben wants to know which city is a favorite vacation spot for people in his city, so he surveys every third person in line to board an airplane to Las Vegas at the city airport.
  4. Aleshia wants to know how students at Robertson Middle School feel about the temperature of their classrooms, so she surveys all students in her English class.
  5. Spiral Review Question: Using a randomly selected survey that is representative of 8th grade students at Rodriguez Middle School, Brandon has determined that 65% of students plan to vote for Selena as the 8th Grade Prom Queen. If there are 440 students in the 8th grade at Rodriguez Middle School, and all of them vote, how many will NOT vote for Selena?

Summary

As you have seen in this resource, it is not always possible to survey every member of a group in order to determine the attitudes or beliefs of the members of the group. Therefore, it is important to select a sample of the population for the survey that is

  • representative of the population, and
  • randomly selected.

picture of a teacher in the classroomIt is also important that the survey be asked in such a way that the questions do not skew the results of the survey one way or the other. Bias in a survey can influence the validity of the conclusions that can be drawn from the survey data.

For example, suppose that three students want to know which teacher is the favorite among all students at Lakeview Junior High School.

 

Sandra surveys 120 students randomly selected using their student ID numbers. She concludes that Mr. Martinez is the favorite teacher at Lakeview Junior High School.

Hunter surveys 35 of his classmates in 7th grade, and determines that Mrs. Stephenson is the favorite teacher at Lakeview Junior High School.

Lakeshia surveys every third student in line at C lunch, and determines that Ms. Nguyen is the favorite teacher at Lakeview Junior High School.

 

Sandra’s conclusion is valid because her sample is representative of all students at Lakeview Junior High School (every student had an equal chance of being selected for the sample).

Hunter's conclusion may not be valid because he only surveyed 7th grade students, and he wanted to determine the favorite teacher of all the students.

Lakeshia’s conclusion may not be valid because she only surveyed students at C lunch, so not every student in the school had an equal chance of being selected.