Cohen: What the Selzer Exit Says About Polling
Veteran pollster Michael Cohen writes the cure to American political polling and how to reach hidden Trump voters
Friends, I’m excited to announce the launch of a weekly column from the great and thoughtful Michael Cohen, author of the book Modern Political Campaigns, president of Cohen Research Group and a 30-year veteran of the polling industry. I’ve long relied on Michael as an even-keeled source with keen insights into politics and the electorate, and am honored he’s agreed to be the very first 24sight News columnist, now delivering his insights directly to you.
Look for more as I continue building out, and shift gears for the second phase of 24sight News — the mission of clear, grounded, objective journalism remains the same. But now with some new features and content.
Enjoy!
Tom LoBianco
National political reporter and co-founder, 24sight News
tom@24sight.news
Even Tom Brady lost two Super Bowls.
I thought about this as a different GOAT hung it up after a celebrated career.
Ann Selzer, the best pollster in politics, announced last week she is retiring from conducting her top-rated Iowa Poll, published since 1997 with the Des Moines Register. By her own admission, it was a tough exit, coming after she published a catastrophic outlier of a presidential poll which missed the mark by16 percentage points, with a stunning finding that more that Harris was winning men in the state by had 3 percentage points.
Combative even (and especially) in victory, incoming president Donald Trump, claimed Selzer “knew exactly what she was doing.” (He also said he wanted her investigated, though he did not say by who or how.)
Well, what if that flop of a poll wasn’t premeditated, as the president-elect alleged?
We’ve all seen the signs (and maybe Selzer has seen them, too): The Trump Years have been a mixed bag for pollsters of all stripes. In 2020, Selzer pulled her final poll during the Iowa Caucuses because one candidate’s name was left off the list in the survey. But, her final Iowa Poll in the general election 2020 had Trump ahead by 7%, which nailed the outcome.
Then came Tuesday, where Trump won Iowa by 13% and didn’t lose by 3%. This error was not marginal.
What does this say about polling?
Let’s look at the overall averages nationwide. The final polling average from Real Clear Politics had the race essentially tied at Harris +0.1%. As of this writing, Trump is up 1.8% nationally in the popular vote, with some House races yet to be entered into the final tally. If you are looking at this fairly, where polling at best can get you to about a 3% margin of error, being off 1.9% is a defensible result.
What about the battleground states? Overall, the polling average showed Trump was leading in the battlegrounds by just under a point on Election Day. But in the only poll that matters, the election itself, Trump outperformed in each, but well within the margins of polling error: Arizona 1.7% in Arizona, 2.5% in Nevada, 1.2% in Wisconsin, 0.9% in Michigan, 1.4% in Pennsylvania, 2% in North Carolina, and 0.9% in Georgia. Pretty solid for a margin of error of 3.5% for most statewide polls.
Contrary to the seemingly universal feeling that polling is broken, it is just as good, if not better this time. In 2020, the battleground states’ actual results all fell within the margins of error — and Pennsylvania was perfect. But Wisconsin was a polling disaster. The average showed Biden ahead of Trump by 6.7%. Biden won by 0.7%.
Based on what we have seen, Trump polls somewhere higher than we’d expect but not necessarily outside the margin of error nationally and even in battlegrounds. “The Polls” weren’t wrong; Selzer was, and on the one day you cannot be wrong. Selzer lost Iowa, forfeited in 2020 during the caucuses and got blown out on Election Day 2024.
And the big winner in the predictions game this past election? A French gambler who won an estimated $48 million on Polymarket hedging for the “shy” Trump vote.
So let’s return to the broader question: What does the persistent undercounting of Trump voters say about polling?
The core problem that remains with modern polling is modern voters.
Since 2016 Trump has relentlessly attacked polls as fake, much like he attacks as the news that covers them. Trump’s supporters are unusually bought into his worldview and they have, around the margins, exited polling as an activity they participate in during elections.
Our public life silos have hardened, including everything from the words we use, news we consume, the policies we support, as well as the candidates. We can no longer debate politely. Politics is beyond personal and now definitional. It’s exhausting.
There are just enough Trump voters unwilling to be counted before Election Day that we undercount them on Election Day. In close races, this is where things can go wrong.
This creates non-response bias, a problem that surfaces when you do all the right things: questionnaire design, sampling, and weighting, but end up with the wrong estimates. There is something different about the people in the surveys compared with the people who refuse to participate in them.
We are no longer just bowling alone, we’re voting alone, cutting ourselves off from the broader conversation and pollsters.
My view is this doesn’t self-correct in 2028, which may be a reason why Selzer exited the public political polling business. The only way through this is expensive: commit to interviewing a subset of voters who look like our samples but who refuse to do surveys. It is no longer an honor and a privilege to be chosen for a poll. It’s a pain-in-the-tuckus and it might out you in a way you’d rather keep private.
Here are three suggestions for identifying gaps in traditional polling:
1. A more significant time commitment, 4-5 days instead of 2-3. This will allow people time to respond. Political polls are notoriously quick for publication exercises.
2. Cash payments to respondents because their time is valuable. We are inundated with requests for money from politicians, paying respondents flips that.
3. Polls need to be paired with one-on-one in-depth qualitative interviews, not focus groups where social biases can limit truthful conversation. We need to understand why they don’t do polls and how much of an undercount they believe exists.
Selzer’s exit from the polling industry is a canary in the coal mine crying for those of us who conduct polls to do more to get us closer to the truth.
 | A guest post by
|