Is Political Polling Broken?

Political polling has had a black eye since at least the 2016 presidential election.

That’s when national polls had Hillary Clinton leading Donald Trump by a significant margin all the way up to Election Day. Democrats started measuring the curtains. Unfortunately, the polls massively underestimated the strength of Trump’s support, and Trump went on to win the Electoral College and the White House.

More recently, pollsters have suffered embarrassing failures in high-profile contested primaries. In Michigan’s Democratic Senate primary, polls predicted a blowout for Abdul El-Sayed; he won by less than a percentage point. In Wisconsin, polls showed Democratic Socialist Francesca Hong enjoying double-digit leads in the gubernatorial primary; she narrowly lost to David Crowley. Think pieces abounded on what’s gone wrong with political polling.

In truth, polls have always been unreliable.

Numbers create the illusion of infallibility, but polls are an imperfect gauge of unpredictable behavior. “You’re relying a lot on a human being predicting what they’re going to do and giving an honest answer about what they’re going to do, which is not what humans are great at doing,” says veteran Democratic pollster Stefan Hankin, founder and CEO of Lincoln Park Strategies.

I talked to Stefan for this week’s interview, which you can watch above. Our conversation pointed to several factors that help explain why political polling seems especially unreliable as of late:

Unreasonable expectations of polling’s accuracy

One big problem is that pollsters and the polling industry, along with the political press and aggregators like Five Thirty Eight (now the Silver Bulletin), have overhyped polls’ “accuracy” and predictive value. Accuracy is a mirage, which Stefan acknowledges that pollsters themselves perpetuate. “A lot of times, pollsters will walk into the room and say, ‘Hey, our poll said [the race is] 40 to 46, and that is the word of God,’” he says.

Reality is nowhere near that precise.

Every poll has a margin of error, which many people downplay or conveniently forget. Take, for instance, a seemingly tight race with a four-point margin of error. If a poll shows Candidate A leading Candidate B with 51 percent of likely voters, the real level of support is between 55 percent and 47 percent (they’re either crushing it or losing significantly). Meanwhile, the opponent who’s “losing” with 49 percent of the vote could in fact be winning comfortably. 

In other words, the only thing this hypothetical poll is showing is that the race is tight, and no one really knows who’s winning. That’s not exactly insightful, so you can hardly blame pollsters for trumpeting more definitive pronouncements. Yet pollsters’ reluctance to acknowledge uncertainty serves to magnify failures when outcomes don’t match these confident predictions.

Good polls are swimming in a sea of slop

In the same way that AI-generated romantasy slop is drowning out decent fiction by human authors, low-quality polls are deluging the political landscape and dragging down polling’s reputation. 

It may surprise you to learn that unlike medicine, law, or even cosmetology, there are no enforceable professional standards for the polling industry: no licensing, no certification, no gatekeeping. “There’s nothing to stop anyone from claiming that they’re a pollster,” Stefan says. 

Some outfits may cut corners with sample sizes that are too small to be credible. Or they may employ shoddy methodology, like cheap Internet opt-in panels where “self-selection” bias becomes a problem. Some “pollsters” are also using AI-generated “synthetic” data—i.e., fake responses for fake polls. 

Political campaigns moreover misuse polls in their fundraising appeals, Stefan says. Campaign polls are almost always biased—assuming there’s a poll at all. “Some of [polling’s] bad reputation is based on numbers that I don’t even actually believe exist,” he says. Among the dozens of fundraising appeals he sees every day, “half of them are talking about some poll that they don’t have a link to,” he continues. “They don’t reference who did the poll, or when the poll came out … It’s not helping the industry.”

Good polling is harder to do

Quality polling has always been expensive and time-consuming, which means that only a handful of well-heeled organizations (e.g., Gallup) can pull it off. But even the most rigorous polling operations are having a harder time collecting reliable data. 

One major challenge is simply finding enough people to poll. “It used to be super easy because everyone had a landline, and we knew everyone was pretty much going to be home after 6 p.m,” Stefan says. “You could call until 9 p.m., and you were going to get a pretty good representative sample.” Today, landlines are rare, while caller ID is ubiquitous. “Most of us, myself included, don’t pick up a number that we don’t recognize,” says Stefan. 

Back in the early 2000s, it would take about 25 calls to reach a willing respondent, Stefan says. Now it takes 100 to 125 calls or more. Polls that took three days to field now take five or six. More time means more money, which means greater temptation to take shortcuts. 

Another problem is identifying “likely voters,” which is especially hard to do in low turnout primaries. This was probably why the Michigan and Wisconsin polls were so far off the mark, Stefan says. The voters who actually voted were not the “likely” voters pollsters queried. 

Deciding who is a “likely” voter is more art than science, Stefan says. Pollsters look at voting histories and ask screening questions. Both methods are imperfect. Voting histories are less reliable for predicting voter behavior in primaries, when turnout is typically very low. And as for screening questions, like asking someone if they plan to vote, “Everyone says yes, because that’s the ‘right’ answer,” says Stefan. 

The bottom line is that because humans are fallible, polls are too. If you watch this week’s video on our Substack, Stefan offers up some useful tips on how to be a responsible consumer of political polling this fall. His primary piece of advice: Polls can be a great for a vibe check. But never treat them as gospel. 

Don’t miss at the Monthly…

In electoral politics, clean energy is off the grid. Unlike the fossils fuels industry, which has had no qualms showering candidates with money, clean energy companies have been surprisingly reticent in their willingness to play hardball electoral politics. That’s because they believe their technology “will win on the merits,” writes Editor Kainoa Lowman. Whether this is optimism or naïveté, the industry risks getting steamrolled if it doesn’t get more aggressive. Read here. 

Blue Wave? Or “Red Undertow”? Trump’s journey from “unpopular to toxic, and then to radioactive” may hand the Senate to Democrats, legendary political prognosticator Charlie Cook tells Contributing Editor Jonathan Alter. But Democrats should also be wary of assuming a mandate if they do indeed take the majority. That’s because Cook doesn’t think a Blue Wave is coming. Rather, he says, “I think it’s a Red Undertow election, where Republicans are getting sucked down.” Voters aren’t thrilled with Democrats either, Cook continues. “I haven’t seen any data whatsoever suggesting people are happy with the Democratic Party,” Cook says. “None, zero. This is 100 percent about Trump.” Read here.

The truth behind the Census Bureau’s numbers on poverty. The Census Bureau recently released its 2025 numbers on the poverty rate, boasting of progress on the problem. But what these numbers actually reflect is the status quo ante before Trump and his catastrophic gutting of the safety net. Next year’s numbers will be radically different, writes Wendy Chun-Hoon, CEO of CLASP. “Despite what the ‘official’ numbers suggest, far too many people in the United States already live in poverty, far too many households do not see a way out of it, and the administration’s policies are making things worse,” she writes. Read here. 

Siege of Troy. Pundits have recently wondered why Maine Senate Democratic candidate Troy Jackson seems to be struggling in his race against Republican incumbent Susan Collins. Some argue he’s overcoming a Kamala-like disadvantage of entering the race so late, while The New Yorker has theorized that slick D.C. political consultants stripped him of his authenticity. Politics Editor Bill Scher has another theory: Jackson’s long record in the state legislature is giving Collins ample fodder for attack. “Jackson carries the burden of being part of the status quo, owning a voting record, and being tagged—however unfairly and disingenuously—as part of the problem,” Bill writes. Read here. 

A fresh blow to struggling counties. The “One Big Beautiful Bill” passed by the GOP Congress last fall has already had catastrophic impacts on needy families who rely on federal food assistance (SNAP). Now counties are facing another burden. “States and counties are now on the hook for 75 percent of the cost to administer what is still, ostensibly, a federal program,” Features Editor Gillen Tener Martin reports. For counties already struggling with high poverty rates and slim budgets, the extra costs could mean a severe hit to government services, painful trade-offs, and higher taxes on cash-strapped residents. Read here. 

Plus…

Merrill Goozner reviews a new book on the damaging impacts of private equity’s involvement in the hospital sector. Keith Humphreys and his colleagues write about a remarkable and unheralded phenomenon: the end of mass incarceration.  David Atkins weighs in on what the left should be doing on AI safety.  Jonah Blank writes that while the current big issue in AI is “alignment” (i.e., the “alignment” of AI with human interests and existence), the real problem is the misalignment of tech lords and the rest of us. 

Parting note…

A new initiative by Yale’s Health Care Affordability Lab shows that you don’t need “Medicare for All” to have a dramatic impact on health care costs. In its “1% Steps for Health Care Reform” project, Yale lays out a smorgasbord of smaller, but achievable, ideas that in the aggregate would lead to billions of dollars in savings. 

Granted, many of their proposals don’t sound glamorous. For instance, “Authorizing the VA to Recover Payments from Medicare Advantage Plans for Dually Enrolled Veterans’ Care” would hardly fit on a bumper sticker. But what this refers to is the fact that federal law currently prohibits the Veterans Administration from billing Medicare or Medicare Advantage plans when veterans go to the VA for healthcare. That means the federal government pays twice for the same services (once through the VA and again through Medicare). Ending the double payment, Yale says, could save as much as $357 billion over ten years—certainly not peanuts. 

Many progressives hunger for transformative change, especially on issues like health care. But it’s worth remembering that singles and doubles can get you to home plate too. 

As always, thank you for reading, and we welcome your feedback! Leave a comment, or write to us at editors@washingtonmonthly.com

Enjoy the start to fall!

Anne Kim, Senior Editor

The post Is Political Polling Broken? appeared first on Washington Monthly.