Let’s deconstruct this Kennedy campaign stunt.
|
Listen To This Story
 |
To hear Robert F. Kennedy Jr. tell it, President Joe Biden is the spoiler in the 2024 presidential election and should drop out. And, Kennedy has told us, he’s got the polling numbers to back that up.
The Kennedy campaign put out a press release Wednesday trumpeting a poll, commissioned and paid for by the campaign, that showed Biden losing to Donald Trump and Kennedy beating Trump in hypothetical head-to-head matchups. “Kennedy,” the headline screams, is the “Only Candidate Who Can Beat Trump.”
According to the survey, conducted by John Zogby Strategies, Trump would take 294 Electoral College votes to Biden’s 244 while Kennedy would beat Trump by the slimmest of possible margins, 270-268.
Kennedy pivoted on the poll results to insist that Biden was the “real spoiler” in the race — undermining Kennedy’s prospects of defeating Trump — and whipped up a “No Spoiler Pledge,” essentially challenging Biden to abide by the results of the poll, if they are corroborated in October, and drop out of the presidential race.
As a former polling analyst, it took me all of two minutes to spot the sleight of hand, but I suspect non-pollsters or non-statisticians may find the top-line results impressive and thus the poll may well serve its political purpose, which is clearly to damage Biden’s electoral standing.
That flash grenade has been greeted with everything from credulous shock to skeptical shrugs and yawns. But I have not yet seen a clear explanation of why the poll should be taken with a great lake of salt and how the Kennedy campaign’s representation of it is meant to deceive.
The Skinny on This Big Fat Poll
As a former polling analyst, it took me all of two minutes to spot the sleight of hand, but I suspect non-pollsters or non-statisticians may find the top-line results impressive and thus the poll may well serve its political purpose, which is clearly to damage Biden’s electoral standing. So let me get right to the deconstruction of this campaign stunt.
First off, I want to make an important distinction between possible issues intrinsic to the poll and how the Kennedy campaign misleadingly packaged the results to create a spectacular impression well beyond anything those results could be deemed to support.
Zogby, the polling firm engaged by Kennedy, has been in the polling business for decades (full disclosure: I have engaged them in the past, pro bono, to conduct surveys I used in support of my own election integrity advocacy) and has a strong reputation for professionalism and integrity. They rely, for this and other surveys, on an online polling protocol that some others in the profession have criticized. A full explanation of the arguments pro and con leads deep into the survey research and statistical weeds, but the essence of the critique as I interpret it centers around the potential for what is known as response bias.
Basically, courtesy of Jeremy Zogby, managing partner at John Zogby Strategies, the way the methodology works is this: Zogby engages a standing panel of roughly 15 million individuals who are chosen, to the best of his firm’s ability, to be demographically representative of the American voting population. This is the pool to which Zogby, when conducting a poll such as this, sends out, to some number of randomly selected panel members, online invitations to take the survey.
Whatever proportion of the invitee group responds becomes the sample, with slight weighting adjustments to account for such problems as the underrepresentation of more difficult to reach demographic groups.
Given that conventional (i.e., telephone) polling has its own ever-growing access problems and low response rates, any blanket criticism of Zogby’s online opt-in approach strikes me as unfair. There is, however, one aspect of the sample selection process that I find concerning, for this survey specifically: a particular form of response bias I would call “techno-bias.”
It seems to me that such bias might take two forms: By definition, the panel of 15 million potential respondents exists online, which excludes the small but not entirely negligible slice of the population that cannot get or chooses not to have such access. More significantly, the more time a member of the panel spends online — the “more online” (or on their phones) they are — the more likely to come across, and respond to, the survey invite.
Thus, the resulting sample might overrepresent those living more of their lives online, or more techie, or both. And that characteristic would stand a fair chance of correlating with support for Kennedy, whose appeal to young voters and tech-bros is generally acknowledged.
It is true that demographic, and particularly age-based, weighting of the sample might serve to somewhat mitigate any such bias. This is pretty weedy and I’ll have to oversimplify a bit, but here goes.
If the sample, for example, consisted of a greater proportion of 18- to 29-year-olds than census data told you was present in the voting population, downweighting the responses from that group (that is, tallying each of their responses as, say, 0.8 or some other fraction of a response) would help restore the age-group representativeness of the sample and — if we know or assume the young spend more time online — might also partially address a suspected techno-bias.
But it is simply not possible to make such an adjustment directly based on affinity for the internet or technology, and impossible as well to quantify precisely what political bias the online opt-in sampling methodology might introduce. This makes for a bit of statistical dancing in the dark.
Sound the Kazoos!
These days, just about every poll comes — or should come — with a caveat or two, and this one is no exception. But any intrinsic problems with the poll itself pale by comparison to the deceptive way in which the poll results have been used by the Kennedy campaign.
The Kennedy press release presented the poll as “the largest Presidential poll of the 2024 election” and began by stating that it “surveyed more than 26,000 likely voters with a margin of error of +/- 0.6%.” Which sounds awfully impressive, given that most national polls have a sample size of less than 2,000. (It’s worth noting that the number of voters surveyed is given by the Zogby Strategies data sheet as 23,683, though the margin of error is unaffected by the disparity.)
But Kennedy is not interested in the national vote. In fact, as far as I could tell, he doesn’t even bother presenting any nationwide results.
Instead he presents three colorful Electoral College maps, showing states “won” in each of three head-to-head matchups: Trump-Biden, Kennedy-Biden, and Kennedy-Trump.
Although he shellacks Biden 367-171, that matchup is really of no interest to him, as he seems not to have even considered suggesting that Trump drop out of the race. No, Kennedy’s No Spoiler Pledge is directed squarely and exclusively at Biden (hmmm…), and so the only maps of interest are the ones that show Trump beating Biden and Kennedy beating Trump.
The difference between those two maps comes down to exactly five states — Georgia, Iowa, Maine, Utah, and Virginia — that move from the Trump column in Biden-Trump to the Kennedy column in Kennedy-Trump.
The impression given: 1) A highly accurate poll, way more respondents and way lower margin of error than all the other polls presented for comparison; 2) A big movement of key states from Trump to Kennedy; 3) Only Kennedy, not Biden, can beat Trump.
Why is this impression so deliberately deceiving? Because Kennedy touts a huge national sample to build the state-by-state Electoral College maps! And those state samples are, well, tiny!
Here’s what we find when we pull out the relevant numbers from the survey data the Kennedy campaign helpfully supplies without ever once qualifying:
Table prepared by the author; sample size and contest margin data from John Zogby Strategies Interactive Survey of US Likely Voters 4/13/24 – 4/21/24; MOEs calculated using Raosoft sample size calculator. Photo credit: Jonathan Simon / WhoWhatWhy
The sample sizes for these five key states range from 350 to 635. A little mathematical polling secret — highly counterintuitive but true! — is that, once you get to a population as large as a state, for a desired level of accuracy you need just about exactly the same sample size for that state as you’d need for the whole country.
Put another way, breaking that great big 26K national sample down to 50 individual states means the margin of error (MOE) for each state is nowhere near that ballyhooed +/- 0.6%.
In fact, the corresponding MOEs, shown in the table above, are far greater. And because the MOE is calculated for the vote share of each candidate, when we’re looking instead at the margin between the candidates, we have to effectively double the MOE, as shown (column in bold font).
So, as even a casual peruser of polls will recognize, those are some whopping error margins! Now let’s look, in the right hand columns, at the margins of victory predicted by the poll in these five states. They’re all quite small. In fact, every one of them is less than the error margin of the poll for each state, most of them dramatically less. Here, to illustrate it more graphically, are the numbers for Georgia and Virginia:
Chart prepared by the author from data in above Table. Photo credit: Jonathan Simon / WhoWhatWhy
Chart prepared by the author from data in above Table. Photo credit: Jonathan Simon / WhoWhatWhy
The gray bars show the error margins and the red and purple bars the Trump-Biden and Kennedy-Trump margins respectively. Both red and purple are well within the poll’s error margins, which means that putting such states in the Trump column (against Biden) or the Kennedy column (against Trump) is highly speculative, not much better than a coin flip. And that’s before any consideration of intrinsic techno-bias towards Kennedy, discussed above.
It is, in short, a laughably unreliable poll — at least for the purposes to which Kennedy is trying to put it.
To reiterate, this is not the fault of Zogby Strategies, which delivered the poll its client paid for. It is, however, why client-commissioned polls are generally viewed with more skepticism than independent polls and why a close eye must be kept on huckster candidates pulling what Kennedy pulls here.
What would the Electoral College results be if just Georgia and Virginia went the other way, with Biden beating Trump and Kennedy losing to Trump? Trump vs. Biden would now result in a 273-265 Biden win, and Trump vs. Kennedy in a 297-241 Trump win — a complete reversal of Kennedy’s vaunt. The other three states, if flipped, would just add to the turnaround.
The Kennedy campaign has presented the poll in such a way as to deliberately misrepresent its import. It is, at best, statistical quicksand and any halfway ethical candidate would put up a sign to warn the unwary.
None of this is to say that Biden will in fact defeat Trump or that Kennedy would lose to Trump. It is to say that the Kennedy campaign has presented the poll in such a way as to deliberately misrepresent its import. It is, at best, statistical quicksand and any halfway ethical candidate would put up a sign to warn the unwary.
Instead, Kennedy hides the ball where it is very unlikely to be found by most media outlets, let alone the voters, and splashes out a deceptive message meant to breathe sudden life into his campaign and fulfill his evident mission of dragging Biden down.
Back on planet earth, Kennedy’s still polling the same 10 to 11 percent, and even that is likely to shrink by November. That’s good to know because, with stunts like this, he keeps making it clearer and clearer that, in this fraught year, he poses his own brand of danger to the survival of our republic.
Jonathan D. Simon is a senior editor at WhoWhatWhy and author of CODE RED: Computerized Elections and the War on American Democracy.
