Is a lead within the margin of error a tie?

Not quite. It means the poll can't say with 95% confidence who's ahead. The margin on the lead is about twice the reported one, so a 2-point lead in a poll of ±3 points is well inside it. The candidate ahead is still more likely to be leading than trailing, but the poll alone can't settle it.

How can 1,000 people speak for millions of voters?

The margin of error depends on how many people are asked, not how many live there: 1.96 × √(0.25 ÷ n) has no population in it. A random 1,000 gives about ±3.1 points for a city or for the whole country. That only holds if everyone had a fair chance of being asked and of answering.

What does 95% confidence mean?

If the same poll were run 100 times, about 95 of the results would land within the margin of the true value. On the stage 95 of 100 do; another seed gives 91 or 97. It describes the method over many polls, so any one poll can be among the misses without anything going wrong.

Why were the polls off in 2016 and 2020?

AAPOR's reviews point to who answered. In 2016, state polls had a historically bad year, and some pollsters didn't weight by education, which split voters sharply. In 2020, polls overstated Biden's national margin by 3.9 points on average, and AAPOR found nonresponse caused at least some of the error. The margin of sampling error measures neither.

Live model · Poll margins of error

How a Poll's Margin of Error Actually Works

A poll's margin of error says how far its result could land from the truth by chance alone, from who happened to be asked. At the usual 95% confidence, about 95 of 100 polls like it land within the margin. For 1,000 people that's about ±3 points. It says nothing about people who won't answer, or about bad questions.

100 polls of the same voters

52% of voters back Gold and 48% Magenta. Each poll calls random voters until 1,000 answer. Its margin of error at 95% confidence is 1.96 × √(0.25 ÷ 1,000) = ±3.1 points.

StopPolling

Gold's true support, %People per pollGold voters who answer, per 100 MagentaSeed

Watch the Python behind this model run, line by line

How to read the stage

Gold truly has 52%. Each bar is one poll of 1,000 voters: its Gold share, give or take 3.1 points. A bar that crosses the dashed true line is a hit. Of 100 polls, 95 cross it and 5 miss, by chance alone. The margin is 1.96 × √(0.25 ÷ n), so it shrinks slowly: 2,000 people still leaves ±2.2. Change People per poll, or the seed, and count again.

The margin on a lead is twice as wide

When one candidate gains a point the other usually loses one, so the margin on the gap between them is about twice the poll's margin: ±6.2 points here, as Pew notes. Gold truly leads by 4, yet 13 of the 100 polls show Magenta ahead or tied. None of those polls was wrong in any way the margin promises to catch.

What the margin doesn't cover

Now Gold voters answer 90 calls for every 100 that Magenta voters answer. Each poll still reports ±3.1, but only about 49.4% of the people who answer back Gold. Just 56 of 100 polls catch the true 52%, and 66 show Magenta ahead or tied. AAPOR found 2020's polling error the highest in 40 years for the national vote, and nonresponse caused at least some of it.

The same polls when Gold voters answer less

52% of voters back Gold, but Gold voters answer 90 calls for every 100 Magenta voters answer, so about 49.4% of the people who answer back Gold. The margin of error is still ±3.1 points.

StopPolling

Gold's true support, %People per pollGold voters who answer, per 100 MagentaSeed

Watch the Python behind this model run, line by line

What the model leaves out

Real polls are rarely simple random samples. Pollsters weight answers to match the population by traits such as age and education, which fixes nonresponse only when it lines up with those traits, and weighting itself widens the margin. AAPOR's margin of sampling error applies only to probability samples, not opt-in online panels. Likely-voter screens, question wording and late shifts add error the model doesn't show.

Questions

Is a lead within the margin of error a tie?: Not quite. It means the poll can't say with 95% confidence who's ahead. The margin on the lead is about twice the reported one, so a 2-point lead in a poll of ±3 points is well inside it. The candidate ahead is still more likely to be leading than trailing, but the poll alone can't settle it.
How can 1,000 people speak for millions of voters?: The margin of error depends on how many people are asked, not how many live there: 1.96 × √(0.25 ÷ n) has no population in it. A random 1,000 gives about ±3.1 points for a city or for the whole country. That only holds if everyone had a fair chance of being asked and of answering.
What does 95% confidence mean?: If the same poll were run 100 times, about 95 of the results would land within the margin of the true value. On the stage 95 of 100 do; another seed gives 91 or 97. It describes the method over many polls, so any one poll can be among the misses without anything going wrong.
Why were the polls off in 2016 and 2020?: AAPOR's reviews point to who answered. In 2016, state polls had a historically bad year, and some pollsters didn't weight by education, which split voters sharply. In 2020, polls overstated Biden's national margin by 3.9 points on average, and AAPOR found nonresponse caused at least some of the error. The margin of sampling error measures neither.