Reading a poll, in seven short lessons.

What the margin covers

A poll asks a sample, not everyone. The margin of error says how far the sample's answer could plausibly be from what everyone would have said, because of which people happened to be picked. At the usual 95% confidence, a ±3 margin means that if the same survey were run many times, about 95 in 100 results would land within 3 points of the true value.

That is all it covers. It does not cover how the question was worded, people the poll could not reach, people who did not answer honestly, or how “likely voters” were picked. Those errors are real and usually not reported, so treat the margin as the smallest uncertainty a poll has, not the whole of it.

The margin of the lead is about twice as wide

Reports print the margin for one answer. A lead is the gap between two answers, and both of them move with the sample: when one goes up, the other tends to go down. So the gap's own margin is roughly double.

Example (made-up): 1,000 people, 46% one way, 50% the other. Each share has a margin of about ±3.1, but the 4-point lead has a margin of ±6.1. This poll does not show who is ahead.

Changes between polls

When a number moves from one poll to the next, both polls carry sampling error, so the change has a wider margin than either poll alone. Two polls of 800 people, 44% then 48%: the margins are ±3.4 and ±3.5, but the margin of the 4-point change is ±4.9.

A run of several polls moving the same way says more than any one pair. That is what the Poll Book is for.

Subgroups have bigger margins

“Among younger people...” is a smaller poll inside the poll. In a sample of 1,067 (±3.0), a subgroup of about 160 has a margin near ±7.7, and a lead inside that subgroup roughly twice that. Big swings in small subgroups are often noise.

Sample size matters; population size hardly does

1,000 people measure a town of 100,000 (±3.1) about as well as a country of 300 million (±3.1). The population only starts to matter when the sample is a large part of it, like 300 of a school's 800 students (±4.5 instead of ±5.7). The tools apply that correction when you give a population.

Weighting widens the margin

Pollsters weight their samples so groups that answered too often or too rarely count in their true proportion. That reduces bias but adds variability; statisticians call the increase the design effect. A design effect of 1.5 turns a ±3.1 margin into ±3.8. Careful pollsters include it in the margin they report; the tools let you enter it when a report gives it.

Opt-in polls and “credibility intervals”

The margin of sampling error comes from probability sampling, where everyone had a known chance of being picked. Many online polls use opt-in panels instead and may report a “credibility interval”. The American Association for Public Opinion Research says that is a model-based measure, not the same thing as a margin of sampling error, and that the public should not rely on it in the same way. When you check such a poll here, read the result as “if this were a probability sample”.

Where these points come from

• Andrew Mercer, “5 key things to know about the margin of error in election polls”, Pew Research Center, September 8, 2016. pewresearch.org (lessons 1-4 and 6: what the margin covers, the lead about twice the margin, change between polls, subgroups, design effect).
• American Association for Public Opinion Research, "Understanding a credibility interval and how it differs from the margin of sampling error in a public opinion poll". aapor.org (lesson 7).
• AAPOR, “Margin of Sampling Error / Credibility Interval” guidance. aapor.org (PDF).
• The formulas are the standard ones for a simple random sample (normal approximation), as used in any introductory statistics text; the tools show one decimal place.

Approval Rating is not affiliated with Pew Research Center or AAPOR.