More on the Berkeley Report

Exit Polls Legacy blog posts

Having raised the issue of the Cal Berkeley Report on alleged voting irregularities in Florida, I have been struggling with how much to comment on the ongoing debate among statisticians on their findings (some of it in the comments section of this blog). While this topic fascinates statisticians, it tends to leave the rest of us a bit puzzled. I think Keith Olberman spoke for many:

I have made four passes at "The Effect of Electronic Voting Machines on Change in Support for Bush in the 2004 Florida Elections," and the thing has still got me pinned to the floor.

Most of the paper is so academically dense that it seems to have been written not just in another language, but in some form of code. There is one table captioned "OLS Regression with Robust Standard Errors." Another is titled "OLS regressions with frequency weights for county size." Only the summary produced by Professor Michael Hout and the Berkeley Quantitative Methods Research Time is intelligible.

I have been following the debate, and thought about doing a reader’s guide to some of the statistical issues. I am holding off, at least for now, as I am not sure most readers of this are as obsessed by this subject as those who have left comments.  I could be wrong — let me know if you’d really like to learn more.

For now, let me share a few web sites that have done a good job summarizing the key issues. The best, and easiest to understand, was the post by Kevin Drum on Saturday. It is short and worth reading in full, but here’s the gist:

It turns out that [Berkeley Prof. Michael] Hout’s entire result is due to only two outliers: Broward County and Palm Beach County. This suggests several things:

    • There was almost certainly not any systemic fraud. If there were, it would have showed up in more than just two counties.
    • The results in Broward and Palm Beach are unusual, but it’s hard to draw any conclusion from just two anomolies. As Kieran says, "it seems more likely that these results show the Republican Party Machine was really, really well-organized in Palm Beach and Broward, and they were able to mobilize their vote better than the Democrats."
    • Anyone who wants to continue investigating possible fraud in Florida anyway should focus on Broward and Palm Beach.

Drum based his summary largely on more detailed critiques by Kieren Healy at Crooked Timber (see also the comments) and Columbia University Political Science and Statistics Professor Andrew Gelman. In the comments section of this humble blog, you will find a post from George Mason University Political Science Professor Michael McDonald eviscerates the Berkeley Study as "completely worthless." Other critiques come from bloggers Newmark’s Door, Rich Hasen’s Election Law Blog, and Alex Strashny.

The bottom line is that the Berkeley Study’s conclusions are something less than a slam-dunk. As Kaus’ "Feiler Faster" theory might predict, peer-review-by-Internet has moved at lightning speed. Yes, Michael Hout and his colleagues have impressive academic credentials, but then so do Michael McDonald and Andrew Gelman [and B.D. McCullough and Florenz Plassman — see update below]. The results in Broward and Palm Beach counties are unusual, but the fact that these two counties are among the biggest and most Democratic in Florida with the greatest populations of Jewish voters (see NewmarksDoor) cripples the county level analysis.  Continue investigating? Certainly, but the conclusion in the Berkeley report that "electronic voting raised President Bush’s advantage from the tiney edge he held in 2000 to a clearer margin of victory in 2004" looks premature at best. 

Again, if readers would like to me to attempt to "demystify" the underlying statistical issues, leave or comment or email me. Otherwise, I’ll return to issues of polling methodology.

UPDATE:  Prof. McDonald’s complete write-up of his critique of the Berkeley study is now posted on his website.

BONUS UPDATE (For Kausfiles Readers): A paper by two economics professorsB.D. McCullough of Drexel University and Florenz Plassman of SUNY Binghamton — that rebuts the Berkeley/Hout Study point by point. Money quote: "We conclude that the study is entirely without merit and its ‘results’ are meaningless."

Mark Blumenthal

Mark Blumenthal is political pollster with deep and varied experience across survey research, campaigns, and media. The original "Mystery Pollster" and co-creator of Pollster.com, he explains complex concepts to a multitude of audiences and how data informs politics and decision-making. A researcher and consultant who crafts effective questions and identifies innovative solutions to deliver results. An award winning political journalist who brings insights and crafts compelling narratives from chaotic data.