I am hoping to spend the next week or so looking at what the elections have told us about the various issues we discussed here in the weeks before the election: likely voter models, weighting by party, the incumbent rule, etc. But first, one last thought about exit polls.
Many readers have emailed about various summaries of exit poll results collected at different times on election night. Many are looking for evidence of fraud (or looking to debunk claims of fraud) in the apparent discrepancies between the vote count and the “early” results of the exit polls.
Exit polls are certainly a powerful tool in uncovering vote fraud, when it exists. If, as some allege, a particular type of voting equipment was used to perpetuate a wide scale deception last week, the raw exit poll data could help reveal it. I’m dubious about these claims, to say the least, but that’s my opinion. Those who disagree are right to press the exit pollsters for more answers.
Unfortunately, the various exit poll results that were captured online on election night are not much help. First, a lot of the speculation classifies the type of voting equipment used at the state level. Most states use a combination of different types of counting equipment. In Illinois, for example, equipment varies by county (as this web site shows). Some variation may occur within counties or even individual precincts. My precinct in Washington DC, for example, allowed voters to choose between an optical scan ballot and an experimental touch screen voting machine.
Second, regarding the exit poll results captured at various times of the evening: We may know the time they appeared on a given website, but we do not know how old those tabulations were when they were posted, and more important, we do not know the extent to which any given sample was corrected to conform with the actual vote.
That last point is important and not well understood. Remember two things: First, the weighting of exit polls to match actual results is not new, but a standard procedure used since the early days of exit polls. Second, the weighting to actual returns does not occur all at once but continuously, precinct by precinct, over the course of election night. The exit pollsters weight their sample to match incoming actual results for each sampled precinct as actual returns become available. Thus, the exit poll results get continuously updated in what bloggers might call “real time.” Some of the online postings may reflect that updating; some may not. We have no way of knowing. There is also one more step: The sampled precincts are still just a sample, so even when all the sampled precincts have been weighted to the actual result, sampling error may cause the survey to differ from the statewide result. At that point, near the end of vote counting, the exit pollsters will apply another overall weight so that the vote on the survey matches the actual statewide result.
Although the data now in the public domain is not much help, the raw data puts the exit pollster in a strong position to evaluate some of the speculation about vote fraud. If, for example, someone tampered with tabulations from touch screen voting machines that lacked a paper trail, then an analysis of the poll data should show a greater discrepancy in precincts with such machines.
A New York Times article that appeared last Friday implies that such an analysis has already occurred. The National Election Pool (NEP) officials that conducted the exit polls wrote a report to debrief their clients on the apparent mishap. The Times article said their report “debunked” the theory of a fraudulent vote count, but did not elaborate. As far as I know, that report has not been made public.
Skeptics are certainly right to want to see the data behind that conclusion. As should be obvious, the raw data is under the control of the organizations — ABC, CBS, CNN, FOX, NBC and the Associated Press – that collectively own it. In past years they have deposited the raw data at the archives of the Roper Center, where it is available for analysis to the general public. Mr. Kaus is right: It would be relatively easy for a major network or newspaper to use the raw exit poll data to debunk – or at least explore – the claims of voter fraud. I’m all for it.
One last thought: I said I am dubious that this data will show evidence of vote tampering. Why? No pollster ever wants to be wrong, to have their numbers called into question. Yet, fair or not, that is exactly what people are saying about last week’s exit polls.
The NEP officials who ran this year’s exit polling, Warren Mitofsky and Joe Lenski, are people I greatly respect. They are not partisans, but extraordinarily skilled survey methodologists. Mitofsky, along with a small team of colleagues at CBS, helped invent the exit poll almost 40 years ago, along with many other methods and models still in use by survey researchers today. Although no one is infallible, Mitofsky is deservedly a legend in the field of survey research. Yet despite that stellar standing, last week’s perceived glitch still threatens his reputation and the continuing livelihood of his collegues.
So apply a bit of common sense: If Mitofsky has evidence that his exit poll was right but the vote tally was wrong, do you believe for a minute that he would suppress it? I certainly do not.
How have the exit polls that are currently still posted (at CNN, for example) been corrected? Purely by weighting the raw precinct data by turnout? Or is there some other weighting mechanism at work which forces the exit poll numbers to match the actual vote count? When the exit polls posted at the end of the started getting “better,” was that because of better weighting the older data, or was there newer data coming in also that wasn’t as bad? Clearly, there was something systematic wrong with the early data — in your experience, could a bias in non-responders explain the entire problem, as the official explanations seem to hint? And if the problem was just bias, doesn’t that worry you regarding the reliability of polling in general? Or are non-responders less biased or non-biased in phone surveys? One final question — the current exit poll data shows that 37% of the voters in this last election voted for Gore in 2000 while 43% voted for Bush in 2000. That’s just plain crazy, so we have to assume that a lot of people “mis-spoke” when they responded to this question. I’m sure this “bandwagon” effect has been well studied — are the numbers above in line with what we’d expect — i.e., about 5 or 6% of those who vote for the loser will later “recollect” that they actually voted for the winner? Thanks for providing this illuminating service!
I posted a comment yesterday in response to the “morality/values” discussion regarding how troubled I am by this Gore voters/Bush voters anomaly (and I apologize in advance if my comments today end up being double-posted,as happened yesterday).
I join with Scott W. in hoping that Mark will address this issue and shed some light on what it tells us about the reliability of the exit polling both before and after it was cross-tabulated.
I am very skeptical that this anomaly can be attributed to any known “bandwagon effect,” and I would respectfully point out to Scott that if one assumes that the Gore/Bush split among 2004 voters should have been approximately equal, say 43% apiece, then it would not have been “5 or 6 %” of 2000 Gore voters who “mis-spoke,” but rather 14%! (6 divided by 43). I find this very difficult to believe.
But the exit polls did a lot worse in Florida and Ohio. . .
Let me just say that I am a physician and I went through the great AIDS-blood bank crisis in the early nineties (or eighties. . .it’s been a while.) “Should we worry about banked blood?” patients asked. Everyone involved in blood banking at that time (save a blood bank director at Stanford Hospital) told patients not to worry. . .things were fine. I don’t think they wanted to hurt patients–but it was the IDEA of a contaminated blood supply and the implications that it had–there was no way to test for HIV at that time–terrified everyone. Denial was the easiest way to handle the problem. . .
When we into this election, I thought: theres just one thing that would portend disaster and this is if the exit polls don’t match the results. . .
If I was your physician and gave you contaminated blood, you would be justifiably angry. . .so to if you as a pollster reassured me that everything is fine just because the implications of contamination are too disasterous to contemplate, I would accuse you of pollster malpractice.
Pam, MD
Oh, and I basically never prescribed Vioxx. . .the increase in untoward cardiac events risk assoc. with its use has been apparent to the sceptical practitioner since 2000-2001.
With apologies to all, but especially Scott W., I realize upon reflection that his approximation of the extent of any possible “bandwagon effect” was more accurate than mine ( I would calculate it to be approximately 7.5% based on an assumption that the 2004 electorate would have actually consisted of @ 40% voters who voted for Gore in 2000 and @ 40% who voted for Bush in 2000, with the remainder consisting of those who voted 3rd party in 2000 or did not vote). Nonetheless, I still find the exit poll result on this topic troubling and am not persuaded that it can be attributed to forgetfullness or a “bandwagon effect.”
Did Bush really win?
The tin foil hat brigade is out in full force, pointing to dodgy vote irregularities as signs of possible election fraud. After the raging, year-long conspiracy theories regarding Diebold voting machines, new conspiracy theories are no surprise. Howe…
“Although the data now in the public domain is not much help, the raw data puts the exit pollster in a strong position to evaluate some of the speculation about vote fraud.”
Are you saying what I think you’re saying? If so, thank you, thank you. Final Exit Polls, pre-cross tabulation, COULD be used to verify the validity of a vote count. We could only assume this were true due to international standards and Dick Morris’ article on Mexico.
Mark, could you confirm your statement in light of arguments that this year’s Exit Polls were biased due to GOP non-response, gender imbalance, the latino GOP vote, etc.? Would it still be possible if these concerns hold true?
Thank you also for endorsing the call to have the owners of the data conduct that analysis. And, why not? What has the country got to lose? This call should be repeated across the country. I think we are all prepared to accept the outcome of that analysis. It can only make our democracy stronger.
Thank you!
PS We understand the pressure pollsters and Mr. Mitofsky in particular might be under. There is always a danger to the messenger, isn’t there? I pray for Mr. Mitofsky and all pollsters who may be under pressure to their livelihoods.
Wow, Lawrence Lessig is joining the call to analyze the Exit Poll Data:
http://www.lessig.org/blog/archives/002282.shtml
This may happen.
Check out this excellent summary of links on Exit Polls:
http://pigsandbattleships.blogspot.com/2004/11/did-bush-really-win.html
Please address the wedge between exit polls and reported vote counts introduced by spoiled and provisional votes.
How is this addressed by the Exit Pollsters?
I can vote. I can be exit polled. But, if my vote was spoiled without my knowing it, the exit poll is corrupted. Or if I voted provisionally the exit poll should take that into account. Does it? How?
Thank you!
More people joining the call to investigate Exit Polls.
This time it is the founder of Air America and he is calling for a movement to lead the call, fundraise, and hire an army of CPA’s and Statisticians to cut through the numbers of the reported vote totals.
http://www.opednews.com/drobny_110904_election_investigation.htm
I would like to add just one small note about my position on possible fraud, since it’s not clear from my original laundry list of questions about anomalies in the exit polls. While I know first-hand that in one heavily Democratic precinct in Toldeo, at least a half a dozen voters left the polls without voting because the lines were too long, and while I assume that similar problems occurred elsewhere in Ohio, I do not count myself among those who suspect that there was widespread fraud in this election. I’m just very curious about the explanation for what was clearly some sort of systematic problem with the 2004 exit polls. Unless those explanations are grounded in a concrete and verifable reality and are not just hand-waiving speculations on the part of the pollsters, it will be risky to draw any conclusions about “why we lost,” and, more importantly, “what do we do next,” from that data.
Compare and Contrast or, The Election Was
StolenNot StolenCompare and contrast. First, Greg Palast, the voter fraud gadfly: Most voters in Ohio thought they were voting for Kerry. At 1:05 a.m. Wednesday morning, CNN’s exit poll showed Kerry beating Bush among Ohio women by 53 percent to…
I think I found the answer to one of my questions right here on this site. My apologies to Mark for not reading over his entire archive once I discovered this amazing site late last month.
In any case, the issue about the “bandwagon effect” that I and others have wondered about might instead be a case of the “social discomfort” phenomenom discussed by Mark on October 8, in http://www.mysterypollster.com/main/2004/10/the_bush_landsl.html
In that article on polls in general, Mark notes that people are typically embarrassed about admitting that they didn’t vote in the previous election, and so as many as 10% of them will lie about it. Perhaps that’s where the “extra” Bush 2000 supporters come from — people who may have supported Bush in 2000 but didn’t actually vote for him, and then lied about that on the exit poll, thus pushing up the rate of “Bush recidivism.”
From today’s Washington Post:
“MIT political scientist Charles Stewart III also has said focusing solely on the early polls favoring Kerry in Ohio and Florida is the wrong approach because exit polls in some Democratic-leaning states tilted toward Bush, evening out the national picture.”
If the polls in Dem leaning states in fact evened things out, does this eliminate the idea that Kerry’s faring better in the exit polls of battleground states than in the actual vote totals could not have been the result of random variation due to sampling error as you mentioned earlier?
Mark — I’m not a professional pollster. Just an interested voter. As you’re doing your analysis over the next week, please check out these folks at “scoop”: http://www.scoop.co.nz/mason/stories/HL0411/S00142.htm
In twelve critical states, as opposed to 35 non-critical states, they have detected a DOUBLING of the shift to Bush in the exit polls as the night wore on. The chances of that happening are miniscule without tampering.
Mark — I’m not a professional pollster. Just an interested voter. As you’re doing your analysis over the next week, please check out these folks at “scoop”: http://www.scoop.co.nz/mason/stories/HL0411/S00142.htm
In twelve critical states, as opposed to 35 non-critical states, they have detected a DOUBLING of the shift to Bush in the exit polls as the night wore on. The chances of that happening are miniscule without tampering.
You might be interested in this article (pdf) by a professor at UPenn:
http://www.buzzflash.com/alerts/04/11/The_unexplained_exit_poll_discrepancy_v00k.pdf
Here is another academic “study” of exit poll bias.
http://www.vote.caltech.edu/Reports/VotingMachines3.pdf
However, this study makes the almost inexcusable error of analyzing the currently available CROSS-TABULATED exit polls, i.e. the “corrected” cnn.com exit polls available now.
The study concludes:
“there is no any apparent systematic bias when we take this same
analysis to the state level.”
Wow. Caltech and MIT don’t even bother to learn the rudimentaries of exit poll methodologies. I attended Caltech, and will be informing my professor contacts of this embarrasment.
Andrea Moro made this same initial mistake in his exit poll analysis.
http://www.econ.umn.edu/~amoro/Research/presprobs.html
He corrected himself when he learned of his error. Let’s hope Caltech/MIT do the same.
Here is the study’s contact list in case anyone is curious:
http://www.vote.caltech.edu/contact.html
Huh? Exit poll respondents were far more likely to respond if they were Kerry voters — yet only 37% of respondents said they’d voted for Gore in 2000? That seems pretty weird! I am hesitant to post a comment here, since I may have misread the above posts & I’m far from highly numerate — but how could (presumably) way- oversampled Kerry voters also be claiming in great numbers that they’d voted for Bush in 2000? Please explain, mystery pollster and others!
also, please comment on the paper by the Penn professor claiming that the likelihood that FL, OH & PA polls were biased by random chance to the degree they apparently were is on the order of 250 million to one. http://www.buzzflash.com/alerts/04/11/ale04090.html
It has been shown, in a simple mathematical analysis by Steve Sailer, that the exit poll had impossibly high percentages of Latino Bush supporters concealed in the states which were not reporting their separated Hispanic proportions. From this showing of delberate falsification of the exit-poll data, it can be assumed that this was not a caprice. Rather, it would have been a device to conceal the anomalies resulting from the pollsters having decided to remove millions of Latino Kerry votes from their reporting. They would have oversampled the Hispanic vote to such an extent, that they were left with this crude deceit of trying to hide the resulting impossible percentages of Latino Bush vote, left over from the adjustment process. If they release the raw data, this subterfuge will stand exposed.
I just received an email from Reno, Nevada, journalist Patricia Axelrod, who has written about the use of Depleted Uranium in Iraq and Kosovo, and consequent military and civilian illnesses, among other topics. Axelrod is trying to get public scrutiny of vote-counting in Nevada before the state submits its closet tally to be certified as “final.”
Axelrod can be reached at paxelrod1675@yahoo.com (775-787-1909).
“Despite their best efforts to contain me,” Axelrod says, “I have obtained a copy of the Washoe County Abstract/Voter Statement and it is very obvious that Washoe and the State of Nevada are playing fast and loose with the vote tally.”
“There are at least 2400 votes unaccounted for with no under vote indicated on the tally to adjust for missing numbers. There are a number of other mysterious disappearances of vote to include a total category of voters whose vote was not entered into the final tally. The list of wrongdoings is extraordinary here because there is no oversight at all. I’m it.”
On Monday, November 21 (tomorrow), Axelrod plans to go to court to get a temporary restraining order, “to stop Nevada State Secretary of State from submitting their ‘final’ tally to the Nevada Supreme Court to be accepted.”
Tomorrow is important. When the state supreme court accepts the results, they are forwarded to the US Senate as Nevada’s final tally.”
She says, “Briefly the Nevada Secretary of State has denied me access to all abstracts/voter statements” other than the data on their website. “I have been met with outright hostility from the Washoe County Registrar Dan Burke, who told me to ‘behave myself’ when I asked him questions about the county process and final tallys. When I insisted upon explanations he threatened to have me thrown out of the office . . . His direct words were ‘shut up and get out or I’ll have you thrown out’ by police force.”
Axelrod is asking for some help with the filing and service fees, which might add up to $300 or $500, in filing pro se on Monday. Documentation from the Nevada Secretary of State’s office will be charged at one dollar per page. She says she’s being punished for, as they put it, “calling them liars.”
(Actually, recount provisions have always been built into US election law. The 2000 election has had the unfortunate effect of calling recounts into question.)
Axelrod notes: “Washoe County Nevada is quite important in the grand scheme of electoral politics. Clark County (Las Vegas et al) and Washoe (Reno et al) together make up about three quarters of the Nevada vote. Clark has gone for Kerry… WE need Washoe and we win Nevada.”
In other words, these few thousand votes may have shifted Nevada. It’s interesting that we’re not reading about a story like this in major media outlets. Pass the word along.
http://wgate.ichef.tv/forum/recipebox/messages/7040.html complimentwhosewondered