The Freeman Paper Revisted

Exit Polls Legacy blog posts

I want to revisit my post from Wednesday night on the paper by Dr. Steven Freeman on the "Unexplained Exit Poll Discrepancy." First, Dr. Freeman himself posted an answer last night that many of you may have missed (even I did until tonight – I have copied his complete response at then end of this post). Second, several other comments convinced me that my last three paragraphs were not as clear, and perhaps a bit more incendiary, than they should have been.

First, my main argument is that we have not yet seen any empirical evidence in the exit polls to prove the existence of vote fraud, nor any evidence that the exit poll discrepancy can be explained by any such fraud. Warren Mitofsky strengthened that argument when he told the Chris Johnson, of the blog MayflowerHill, that his initial analysis showed no deviation from the discrepancy "in precincts with touch screen computers that don’t leave paper trails, or any other type of machine for that matter."

However, as I wrote yesterday – and it bears repeating – the fact that the exit polls show no evidence of vote fraud does not disprove vote fraud. It may have occurred on a scale too small to be detectable by the exit polls.

I realize that I confused this issue with my own language in the third to last paragraph, which began, "So to summarize: Absent further data from NEP, you can choose to believe…" It sounds as if I am assuming there are only two possibilities with respect to the possibility of vote fraud. Obviously, that is not the case.

Had I written that better, I might have said: "So to summarize, if you want to explain the exit poll discrepancy, absent further data from NEP, you can choose to believe…" My point is that there are two competing theories for the discrepancy: The first is that the exit polls were slightly biased to Kerry due to a consistent pattern of what methodologists call "differential non-response" that has been evident in exit polls to a lesser degree for a dozen years (Republicans were more likely to refuse to fill out the exit poll than Democrats). The second theory is that systematic and consistent vote fraud occurred in almost every state and using every type of voting equipment. The first hypothesis seems plausible to me; the second wildly improbable.

Steven Freeman and others are right that no one has conclusively proven either hypothesis, but I never suggested that I offered conclusive proof, only a far more plausible explanation for the discrepancy.

Finally, I realize I probably would have been better off omitting the word "delusional" from the last paragraph. It obviously conveyed a broader judgment than I intended, a conclusion brought home by this comment by reader Anthony England:

Apparently reasonable people cannot ask questions about this issue without being accused of promulgating wild allegations that professional pollsters have "deliberately suppressed evidence of … fraud" or without being derided as delusional conspiracy nuts.

I did not intend to condemn all who see a "possibility of count errors" as "delusional," as Steven Freeman heard it. Nor did I intend to imply that there is anything "delusional" about simply asking questions. Hopefully, asking questions is what this site is about.   In retrospect, the word "delusional" was a mistake.  Good blogging, I am told, is about correcting mistakes as soon as possible.  My apologies.

One last thing:  Loyal reader CW made this point via email: "Machines that generate no confirming bit of paper for a voter to review are a disaster for public trust in elections."  Fraud or no fraud, on that point I completely agree.

Professor Freeman’s complete response:

Hello, Mark:

I’d like to thank you for taking the time for offering this detailed critique of my paper, and more generally for your knowledgeable commentary on polling processes. Since writing the draft you read, I’ve learned a great deal about polling – in large part from reading through your site. I’ll have a revision of my paper out in a few days, which will be much stronger for having read your commentary.

Regarding your post, I’m going to respond to several points and then give a general response to what I see as the big question:

1. Data. I’m happy to make my CNN data available. I have 49 states & DC (only Virginia missing if anyone has that), although for a few I don’t have sample size. Just tell me where you’d like me to send it to or post. (My own personal and University websites have been going down from too much traffic.)

2. High degree of certainty (Your point 1). I agree that I overstated the case, should never have cited Hartmann, and did not understand the logistical challenges you explain. NEVERTHELESS, logic and evidence still indicate that exit polls should be a good basis for prediction, and although I can understand why the logistical challenges would increase the margin of error, it’s not at all clear why they should skew the results.

3. 250-million-to-1 (Your point 2). I see that I did put too much faith in stratification counterbalancing the effects of clustering, and will redo the calculations with the 30% increase. That’s a very good citation. NEVERTHELESS, as you point out, it doesn’t change the finding that **random error can be ruled out as an explanation.** This is really the main point of the first draft, because once chance is ruled out, some other explanation needs to be found.

4. Official “explanations. (Your point 3). My key point about explanations is that all we have — at best — are hypotheses. Perhaps Bush-voter refusal is a better hypothesis than I gave credit for, but it still is only a hypothesis. (Too many women would be irrelevant to the CNN data. Male and female preferences are reported separately and thus automatically weighted appropriately.) On the other hand, there are also creditable hypotheses, some with substantial evidence, which could have effected the tally.

I object most to belittling dismissals of these second set of hypotheses and allegations (e.g, Manuel Roig-Franzia and Dan Keating, “Latest Conspiracy Theory — Kerry Won — Hits the Ether” Washington Post, November 11, 2004; Tom Zeller, Jr. "Vote Fraud Theories, Spread by Blogs, Are Quickly Buried" New York Times November 12, 2004-Page 1), along with unquestioning acceptance as “explanations” the hypotheses and allegations about poll error.

In summary, I think that perhaps I biased my paper somewhat unfairly towards suggesting count errors as explanations, but that was probably in response to what I still see as an extreme bias at the press in dismissing them.

When you say that suggesting the possibility of count errors is delusional, perhaps you have done the same? (It seems as though you spend a lot of time on the tin foil hat circuit.)

Thinking coolly and scientifically: Is it delusional to question the Bush-voter-refusal hypothesis as conclusive without independent evidence? On the other hand, considering the scores of allegations, the history (especially in Florida), the lack of safeguards with electronic voting, the conflict-of-interest in election oversight, etc…, etc… (and now the Berkeley study) is it delusional to consider that, just possibly, even part of the discrepancy might be due to the possibility of miscount?

Yours truly, Steve Freeman

Mark Blumenthal

Mark Blumenthal is political pollster with deep and varied experience across survey research, campaigns, and media. The original "Mystery Pollster" and co-creator of Pollster.com, he explains complex concepts to a multitude of audiences and how data informs politics and decision-making. A researcher and consultant who crafts effective questions and identifies innovative solutions to deliver results. An award winning political journalist who brings insights and crafts compelling narratives from chaotic data.