To critically analyze the role of probability and chance in real-world situations.
Go to http://news.gallup.com
and choose a recent poll that interests you which also has a Survey
Methods section that includes a margin of sampling error %.
Survey Information and Methods:
- What is the total sample size for this poll, if Gallup lists it? ____________________
- What is the overall margin of sampling error Gallup gives? (choose another poll if you can't find one) ______________________
What is the confidence level for the margin of error? (choose another poll if you can't find one) __________________
- On what date(s) did Gallup conduct this poll? _______________________
- Who did Gallup survey for this poll?
method did Gallup use to collect the data (face-to-face interviews,
- What is the headline or title of the poll? Write it here and on the front board (or put a check next to it if someone else has written it already).
- What is the publication date?
- What do you think is the larger "population" this poll is meant to represent?
- Given the way Gallup chose its sample and collected its data for this poll, do you think they actually managed to get a reasonably representative sample of this population of interest? If yes, why do you think so? If not, what subgroups do you think might not be well-represented?
If Gallup had taken a completely unbiased simple random sample from the population for this poll, what would be the overall 95% confident margin of error, given the poll's sample size above in 1. and using the conservative estimate? You may wish to review
these slides on confidence intervals. Show the computation and then convert to an approximate percentage.
- Is Gallup's margin of error in 2. larger, smaller, or the same as the computed one in 11?
- If Gallup's margin of error is different than your computation, does it generate a wider
interval or narrower interval than yours would?
- Does the title of your poll make any definitive-sounding statements about
changes in opinions over time,
majority opinion, or differences between groups (e.g.,
"Record-High Support for Legalizing Marijuana Use in US,"
"Slim Majority against Government Pushing Traditional Values," or "Women Lead Men on Key Workplace Engagement Measures")? If so, list them. You may also find these kinds of statements
inside the article. If you can't find any inside the article, what are the claims of the poll?
- Assume for this question that Gallup's sample is indeed representative of the population (e.g., no bias). If you take into account Gallup's margin of error when you interpret the poll results, is it statistically valid for Gallup to make the statements it did
in the title or elsewhere about the population?
In other words, are the primary conclusions supported by the data if you take into account Gallup's margin of error?
Take any overlaps into consideration--show them and explain why or why not. For
instance, arguments might look something like:
54%-4% = 50% but the headline makes it sound like a majority, or
45%-5%=40% may not be highest since 1969 because... You may wish to review the first three slides of clickers on confidence intervals, which
has an analysis similar to what you will do. Show your intervals and analysis here and why the conclusions are supported or why they are not.
- Review these slides and then explain what 95% confidence interval means.
- Explain what margin of error means.
- Assume little to no bias and truly a random sample. If Gallup conducted 100 such polls with a 95% confidence interval, then about how many polls are likely to
include the true population percentage (use the definition of confidence interval)?
- Is there any way to know which intervals contain the true percentage and which ones don't?
- Is there any way to know for sure if it is a representative sample?
- How should we interpret a margin of error if the sample is very biased?
- Name at least one item from your article that you found interesting or surprising, or that you had a question on.