Has Polling Become Weaponized?

As fewer and fewer objective and reliable polls are conducted, Republicans have been weaponizing polling to skew narratives in their favor.

Source: YouTube/Washington Post

The feeling is familiar and dreadful. One more terrible poll after another, confirming our worst fears. Biden, the historically unpopular and aging candidate, weighed down by inflation and world crises, underwater in key battleground states. The other side trades in anti-democratic extremism, sure, but there are strong economic headwinds, and people just don’t like Joe Biden’s policies. 

This was not only the state of the polling and the vast consensus of political punditry just a few months ago. It was also the situation, and what most of the pundits believed, going into the 2022 midterms. That was when the vaunted “red wave” was supposed to sweep across the U.S. and return Congress to the GOP, with likely a solid majority in both chambers.

Only, that didn’t happen.

The polling averages were off, and several of the individual polls were way off. Democrats overperformed in the battleground states and held the Senate. And they held the GOP to such a narrow majority in the House that Republicans spent the whole next year fighting amongst themselves and deposing and rejecting their own leaders.

But what precisely happened back in 2022 that caused the media to run with the false red wave narrative? What harm might it have caused? And why does it seem like the reporting on polls is repeating these same mistakes this election cycle?

Today I’ll take a closer look at the way polling was skewed, even weaponized, in 2022. These were preventable mistakes back then, and they are preventable now. But for some reason, we’re heading down the same path again. And it has to stop.

A quick pledge break before we dive in! If you’ve been meaning to support our work, now’s a great time!

Republican pollsters flooded the zone with biased polls

In the weeks leading up to the 2022 midterms, Democratic strategist Simon Rosenberg was ringing the alarm bell, but few people were listening. 

"In six major battleground states, more than half the polls conducted in October have been conducted by Republican firms… Basically we can't trust the data on RealClearPolitics or FiveThirtyEight any longer... it's essentially Republican propaganda," he told Joy Reid on MSNBC on October 31st, just eight days before the election. 

Rosenberg noted that the two main poll agglomeration sources that we had come to rely on had become corrupted by a flood of Republican polling. Yet Rosenberg was fairly much ignored, even ridiculed as hopelessly optimistic by the media. 

On the eve of the midterm elections, I wrote a piece for The Status Kuo concurring with Rosenberg that, if you screened for polls that were partisan and biased, the story around the midterms looked far different:

Current polling also doesn’t really support a GOP landslide. The media narrative in October was uniformly about such a blow-out, based in part around a series of polls that came out showing the GOP had regained the upper hand in the generic Congressional ballot. But that may have turned out to be a temporary phenomenon driven by pollsters with strong Republican partisan biases. As Democratic strategist Simon Rosenberg argues, if you screen those partisan polls out and look at only the non-partisan polling, it tells a very different story of a country that is pretty much dead even.

Indeed, if you looked primarily at early voting data, I argued that there already was a fairly solid “Blue Sea Wall” and that you’d need a fairly strong Red Wave to overcome it. That didn’t materialize for the GOP in the battlegrounds, and they came away sorely disappointed by the election results. 

The media falls short in 2022

One of the key issues around the 2022 polling was how absent major media was from the business of polling. As Politico reported, there was a “dearth of public, independent surveys” that year. Major news outlets that normally commissioned quality polls simply sat out the 2022 election. For example, in 2018 NBC News had commissioned 16 polls from Marist College in the fall of that election year. But in 2022? Zero.

In 2018 The New York Times conducted 100 polls, mostly in House races. But in 2022 it was just four House races and five statewides. And as Nate Cohn noted in his report one week out from the election, the Times poll showed Senate races that were looking much stronger for the Democrats than the polling averages were indicating, signaling that something was off.

In fairness to the mainstream media, polling has gotten a lot more expensive. Getting someone to respond to a call has become that much harder. As Cohn lamented in October of 2022, 

In the poll we have in the field right now, only 0.4 percent of dials have yielded a completed interview. If you were employed as one of our interviewers at a call center, you would have to dial numbers for two hours to get a single completed interview.

And that figure is down from a yield of 1.6 percent in 2018, meaning a 75 percent drop in just four years.

No one is talking about what reponse rates are in 2024, but one thing is clear: It doesn’t make any economic sense for major media companies to invest heavily into a lot of polls.

And that has opened the door for political operatives to exploit the void left by the dearth of reliable polling.

Impact of the bad polling on donations and momentum

Bad polls can have an extremely negative effect on a campaign for at least one clear reason: money. In a world where donations from big donors are in high demand but in short supply, negative polling can drive a narrative that becomes self-fulfilling.

Here’s one painful example. The polling averages in the fall of 2022 for the U.S. Senate race in Wisconsin showed the Democratic candidate, Lt. Gov. Mandela Barnes, trailing the incumbent Republican, Sen. Ron Johnson, by as much as five points. A common narrative was that Barnes was likely to lose, and that Democrats needed to shore up places where they had a better chance of winning or holding a senate race, such as Arizona and Pennsylvania.

But Barnes was much more competitive than the polling averages showed, losing to Johnson by a spread of less than one percent—meaning if Barnes had done half a percent better, he would have won. The polls had created a misallocation of resources because Democratic leaders and big donors believed the polling averages, when in fact those averages had been corrupted by a flood of Republican polling.

And while there isn’t direct evidence that the GOP intentionally flooded the zone with bad polls to create an illusion of a losing battle, it would be naive to assume that it was all aboveboard. And it would be doubly foolish to assume that Republican operatives would simply ignore how well the skewing of the averages worked in 2022. Who knows how much weaker the red wave would have been had the polls showed that Republicans were actually trailing far more than had been reported.

Rosenberg believes that the effect of the partisan polling was enough to cost Democrats their House majority because the media ran with the red wave story without questioning its origin. “There was a massive media failure” in 2022, Rosenberg told Vox in an interview. “The failure that just took place is more grave than the polling error [in 2020] because there were a lot of really smart people who basically misled tens of millions of people through their political commentary in the final few weeks.”

2023 polls paint a false picture using questionable polling samples

The failure of the media to invest in quality polling is showing up again in this election cycle. This time, it is evident in the very small sample sizes that are being used to drive big narratives.

Take what Rosenberg calls the “dreaded NYT polls” from just a few months ago that showed Biden losing to Trump in six battleground states. The sample sizes in those states wound up being very small, just 600 per state, likely because it really is quite difficult and costly to get people to answer polls. That meant the polls’ margins for error were very big comparatively. And any attempt to drill down into “subgroups” in the “crosstabs” of these polls, (say, Latino voters) left the pollsters with an even smaller sample size that shouldn’t normally, on its own, be newsworthy. 

Those same NYT Battleground polls—the ones that gave so many Democrats agita and even led to calls for Biden to step aside—had some peculiar numbers behind them. For instance, they had Biden winning 18-29 year olds by only one percent, when other gold standard youth polls had Biden up by 24 percent. They can’t both be right, but the latter is far closer to historical norms and expectations than the Times polls.

And just as we saw in 2022, for every high quality poll out there that shows Biden tied or leading Trump, there are a number of poor quality ones that have him losing by several percentage points, pulling the polling averages down for him.

Are the Republicans flooding the zone again with bad polls to make Biden appear more vulnerable than he is? It would stand to reason. One bad consequence is this: The narrative of Biden as a “loser” pushes more undecideds toward third party options. After all, if Biden is going to lose to Trump (which the better polls indicate isn’t likely), then people might as well vote third party.

The media and the public need to be aware of the danger that skewed, partisan polling presents. This is especially important as polling becomes less reliable and more expensive to conduct, incentivizing bad actors to create noise and mischief.

This means we all need to be a bit more poll savvy. Before we freak out, report on, or amplify a poll, we need to understand who commissioned it, what the sample size was, how the data was collected, and what actual experts like Rosenberg and Cohn think of it.

And by all means, we should stop paying any attention to polling averages that let just anyone anywhere weigh in. So as November approaches, stop checking 538 and RealClearPolitics obsessively. They are more apt to mislead than enlighten.