In opinion polls, what does “margin of error” mean?
[14 Jan 2008]
Don’t ever believe surveys. They can never be ‘accurate’, even to within a margin of error of ±3% (as many of them claim).
Most importantly, people tend to tell the pollster what they think the pollster wants to hear. Or those that have something to hide, will deliberately distort their view.
And then there is the convenience issue. Face it, how many of us are irritated when some pollster asks us for “just a few minutes” of our time (and we know that it will take way more time than that…)?
Anyway, it is always better to observe what people do in a certain situation, rather than ask them what they might do. However, that’s not always possible or practical. So we are stuck with opinion polls for a while yet.
Here’s an article from the highly experienced Harris Poll people titled “Margin of Error”, When Used by Pollsters, Is Widely Misunderstood and Confuses Most People.
After outlining the various sources of polling error, the article reveals the large number of misconceptions that people have with the idea of “margin of error”. Only a small percentage of people got the following one right:
Only a very small 12 percent of the public agrees that the words “margin of error” should only address one specific source of error, sampling error â€” as they almost always do
Under the question “So what?” the article points out:
- The use of words such as “margin of error” is controversial because they are often used when reporting telephone polls even though it is not possible to calculate a real margin of error.
- Pollsters need to do a much better job of explaining all the possible sources of error in their polls not just a theoretical sampling error, which does not take into account of other, potentially substantial, sources of
- The accuracy of opinion polls should be judged empirically by the accuracy and reliability of their findings, not on a theoretical basis when there is no way to calculate a real margin of error;
- Pre-election polls should continue to be trusted only so long as their final forecasts are reasonably accurate, not because they are theoretically “scientific” (since there is no means to establish that they are);
- The words “margin of error” should probably not be used at all in conjunction with polling results.
And how about the Harris Poll? What do they do with the phrase “margin of error”?
The Harris Poll has not used the phrase “margin of error” for many years.
People should understand how polls work, and how to interpret the findings, especially as we head into another US election.
Check out the full Harris Poll article.