How wrong were the polls in Ohio? Half were within the margin of error.
COLUMBUS, Ohio (WCMH) – Most polls of Ohio before Election Day said the state would be a toss-up, but after the voters were cast, President Donald Trump repeated his 8-point victory from 2016 and even flipped two traditionally Democratic counties in Northeast Ohio.
So, why were the polls so wrong?
Polls are rarely as certain as they seem. Although headlines can declare razor-thin leads for a candidate, polls really show a range of possible outcomes. That headlining figure is just the outcome if the poll is a perfect crystal ball, which it seldom is.
“Pollsters are always trying to get more attention on (the margin of error), because that's really what you have to use to hedge your bets,” said Thomas Sutton, director of Baldwin Wallace University’s Community Research Institute, which polled the presidential race in Ohio and other Great Lakes states.
Take a hypothetical poll that has Trump and Joe Biden tied at 50% each, and the poll has a 2-point margin of error. This means that if the poll were to be done again 100 times, in 95 of those times the results would be within these two extremes:
- Trump 52, Biden 48 (+2 more for Trump, -2 fewer for Biden)
- Biden 52, Trump 48 (+2 more for Biden, -2 fewer for Trump)
Even though this hypothetical poll shows Trump and Biden tied, it leaves open the possibility of 4-point victories for either candidate.
“Polling is a snapshot of the day or that week,” Sutton said, “not a predictor.”
Applying the margins of error
Looking at the 24 polls of Ohio from October to Election Day, 11 had margins of error that left the door open for Trump’s eventual 8.17-point victory. And another nine polls indicated a possible Trump victory of six points or more.
“The margin of error really does mean that there’s a chance – it could be slim, but there’s a chance – that the election will not go the way that 2- or 3- or 4-point lead might make people assume it’s going,” Sutton said.
Sutton expects Trump’s 8.17-point margin of victory in Ohio to shrink after counties tabulate the remaining 311,519 provisional and absentee ballots, the latter of which have skewed toward Biden. Boards of elections must finish counting by Nov. 24.
Biden won as polls expected, but why was it close?
2020 was still a better polling year than 2016, Sutton said, because the nationally favored candidate, Joe Biden, won the election, and that is something that cannot be said for Hillary Clinton’s loss four years ago.
“The margins didn’t fit but the direction did,” he said.
Baldwin Wallace, partnered with Ohio Northern University and Oakland University in Michigan, conducted polls of Michigan, Ohio, Pennsylvania and Wisconsin from Sept. 29-Oct. 8. Their results showed a toss-up in Ohio and Biden victories between four and seven points in the three other states.
The predicted winners held true, but the eventual margins shifted heavily toward Trump (even though all outcomes except for Ohio’s were within the margins of error).
Sutton has a couple hypotheses for why Biden’s winning margins were slimmer than estimated. First, “the wonderful problem of increased voter turnout,” he said.
Ohio broke its 2008 record with more than 5.8 million voters, and the U.S. is on pace to have its highest turnout percentage since 1900.
“So, then the question becomes, well, “who registered and voted?” Sutton said, “And our hypothesis is that President Trump had really strong turnout, and that's where the increase happened that we did not capture in the polls. Certainly, that was the case in Ohio.”
Sutton said this can throw off polls because many pollsters, including BW, use likely voters. BW makes sure its polling subjects are registered to vote, and subjects not aged 18-29 must have voted in either of the past two presidential elections.
“What that means is we're going to miss a lot of people who didn't vote in those elections and voted in this one,” he said.
Trump voters distrust polling
Sutton’s second hypothesis is that polls missed Trump voters this year because they are less likely to answer polls.
“As a result, we wind up with an undercount of the Trump support and potentially an overcount of the Biden support,” he said.
In 2016, polls missed Trump voters because they did not accurately account for education level, as Trump massively turned out voters without a college degree. But despite most pollsters correcting for education this year, results were still off.
It’s not that Trump voters are shy, Sutton said, it’s that they don’t trust pollsters in the way they tend not to trust mainstream news media, politicians and other institutions.
“We have this culture of distrust that’s growing,” Sutton said.
Sutton added an additional, more relatable reason it’s hard for pollsters to find subjects: “People are tired of getting unsolicited phone calls.”
How did forecasters do?
Polling averages may have made the race in Ohio feel tighter than it ended up. RealClearPolitics’ simple polling average had Trump up just 1 point before Election Day, and FiveThirtyEight’s sophisticated polling average gave him a 0.8-point advantage.
But major forecasters did pay attention to scenarios in which a 2016-style polling error could give Biden his eventual narrow path to winning the Electoral College, as well as keep Ohio for Trump.
The New York Times tried to account for an error like this in one of its forecasts. The final version of that hypothetical scenario gave Trump a 6-point win in Ohio and Biden narrow wins in Pennsylvania, Georgia and Arizona, which mirror results.
Similarly, FiveThirtyEight Editor-in-Chief Nate Silver wrote this of his outlet’s final 2020 forecast (emphasis added):
“But what’s tricky about this race is that — because of Trump’s Electoral College advantage, which he largely carries over from 2016 — it wouldn’t take that big of a polling error in Trump’s favor to make the election interesting. Importantly, interesting isn’t the same thing as a likely Trump win; instead, the probable result of a 2016-style polling error would be a Biden victory but one that took some time to resolve and which could imperil Democrats’ chances of taking over the Senate.
Nate Silver, FiveThirtyEight Editor-in-Chief
FiveThirtyEight’s final forecast had Trump winning Ohio by less than one point, and by almost four points with a 2016-sized error.
Was 2020 a bad year for polling?
“To call it a bad year would be largely due to factors we couldn't control for,” Sutton said, “which were the effect of COVID on voting and the mail ballots, early voting, etc.”
As pollsters begin to adjust their methods to be more accurate, Sutton said polls may need to weigh factors other than traditional demographics, such as the number of people voting early, the number of undecided voters or even the type of candidate.
“There's a Trump factor that's affected the polling in both of these races that probably won't be the same if we look at 2024 and don't see someone like him,” Sutton said, “that kind of burn-down-the-house, charisma, loyal following, all those different factors.”
Sutton also advises people not to cling to every poll that comes out before an election.
“People need to not base their own behaviors on the outcome of polls but based on their own commitment to a candidate or an issue,” he said.