Following up on a lesson learned from the last post, that a story can sometimes make a point more powerfully than a lot of arcane data, I decided to share an excerpts from a series of emails I received about the experiences of four NEP interviewers from the state of Minnesota. The information comes from the Minnesota college professor – let’s call him Professor M — who helped recruit these college student interviewers for NEP. I have shared much of the substance of this story in previous posts, but in light of the findings of the Edison-Mitofsky report, I thought it would be useful to share his verbatim comments.
In early November, intrigued by the controversy surrounding the exit polls, Professor M decided to interview his four students about their experiences as interviewers. As he points out, a sample size of 4 is truly "anecdotal" — it is by no means representative of the experiences of the 1400 odd interviewers who worked for NEP on Election Day. However, it is remarkable how many of the problems he notes help explain patterns in the data on "within precinct error" in the Edison-Mitofsky report.
The following are excerpts from our email dialogue:
The information that I got from my students is quite intriguing, but of course it cannot in any way be considered a representative sample. Also, the students I spoke with kept no independent notes of response rates or other details while serving as interviewers; their impressions of who responded and who didn’t were entirely from memory.
The geographic distribution of the four interviewers was as follows: one outer exurbia, one inner ring suburb, one "exclusive" upscale suburb and one precinct in an ethnically and economically diverse Minneapolis neighborhood.
The [NEP] badge did display the logos of the networks prominently.
However, this could not be easily seen from a distance, and at least one of my students was hampered by the fact that a contingent of folks from MoveOn.Org was stationed right next to her at the 100-ft line. This made her appear from a distance as being connected with them, and being forced to stand 100 ft from the polls, people were able to easily turn aside to avoid her and MoveOn. Also, it gets dark early up here, so the badge was not visible from a distance after 4pm or so.
The two students in suburban areas commented that they had the hardest trouble getting participation in the early morning — probably due to lines and people needing to get to work.
I believe all of the students reported receiving requests from voters who wanted to participate in the survey despite the fact that they were not the nth person to emerge. Nearly all of the students reported some inclusions that were somewhat less than random. Most commonly this occurred when a couple emerged together and the person the poll worker approached refused but the partner offered to participate. None of the students saw any "difference" in which one of the two participated as long as one of them did.
As I understood what the students told me (who did not see themselves as doing anything wrong, by the way) they would not have coded a refusal at all in that situation.
One student reported at least one instance of a person simply taking a survey from her supplies (which were out in the open at her table) filling it out, and dropping it into the survey box. By the time she realized what had happened (she was busy trying to buttonhole legitimate respondents), there was no way to determine for certain which of the surveys in the box had been incorrectly included.
A few additional observations from [a fourth student] — she noted that she had more refusals among white males, although she was not sure if that was related to her own appearance (she is African-American). Also, she observed (and this makes sense, when you think about it) that her response rate improved over the course of the day as she became better at honing her "sales pitch." Still, despite the fact that she had perhaps the most advantageous placement of any of my four students (she was indoors at the only entrance/exit and had full cooperation from the staff on-site), she still recalls a fairly low response rate — 40-50% perhaps.
[Emphasis added]
To clarify one point: Each interviewer was given an "interviewing rate" which ranged from 1 to 10 nationally. Here is the way the Edison-Mitofsky training materials (passed along by Professor M) describe what was supposed to happen:
We set an interviewing rate based on how many voters we expect at your polling place. If your interviewing rate is 3, you will interview every 3rd voter that passes you. If it is 5, you will interview every 5th voter that passes you, etc. We set an interviewing rate to make sure you end up with the correct number of completed interviews over the course of the day, and to ensure that every voter has an equal chance of being interviewed.
If the targeted voter declines to participate or if you miss the voter and do not get a chance to ask him or her to participate, you should mark them as a "Refusal" or "Miss" on your Refusals and Misses Tally Sheet and start counting voters again (for a more thorough explanation of refusals and misses, refer to page 9). For example, if your interviewing rate is 3 and the 3rd "person refuses to participate, you do not interview the 4th person. Instead, start counting again to three with the next person. [Emphasis added]
The point: If interviewers allowed "inclusions that were somewhat less than random" but did not tally refusals appropriately, then the "completion rates" now getting so much scrutiny in the Edison-Mitofsky report are not only inaccurate, but the inaccuracies will probably occur most, on average, in the same precincts shwoing the biggest "within precinct error" (WPE).
The bigger point: Consider that all of the above comes from just four interviewers. Imagine how much we might learn if we could talk to hundreds. Apparently, that is exactly what Edison Mitofsky says they will soon do (or are already doing) with the interviewers in Ohio and Pennsylvania (p. 13):
We are in the process of an in-depth evaluation of the exit poll process in Ohio and Pennsylvania…We will follow up with in-depth interviews with the exit poll interviewers in the precincts in which we saw the largest errors in an attempt to determine if there were any factors that we have missed thus far in our investigation of Within Precinct Error.
I think I can speak for others in the survey research profession when I say we hope they ultimately share more of what they learn. It will help us all do better work.
[Typo fixed 1/28]
Ummm, Mark, I don’t mean to be rude, but, ahem, where are the anecdotes say, from Ohio, suggesting that there was some philandering there with the voting. I’m not saying that stuff is right, I’m just saying that when you’re starting to post anecdotes like this to explain things like WPE you’re really, really, and I mean really scraping the bottom of the barrel. Just give it up and say, “Well maybe there was some hanky panky in the voting.”
Mark, this is plain silly. Please stick to statistics! There, the main question that needs to be answered is why, for the first time in five presidential elections, WPE was seriously skewed towards one candidate.
Mark,
Don’t these anecdotes deny — rather than confirm — the Edison/Mitofsky hypothesis?
Here’s why: either these sorts of “interviewer issues” have plagued exit-polling in every election (and Edison/Mitofsky has not suggested any reason to think otherwise) or else they suggest a certain exuberance on the part of voters: particularly, Kerry voters. [Arguendo].
So, which way does that cut? If Kerry voters were more excited about their candidate than Bush voters, does that tend to make it more or less likely that Bush won the election 51%-48%?
As a lawyer, I know that the fact that voters overall were more excited about the election — and perhaps more Kerry voters than Bush voters (arguendo) — is “relevant” to the extent that it makes a single fact in dispute more likely to be true: that John Kerry won the election.
How Kerry voters being more excited about Kerry than Bush voters were about Bush *supports* a final “result” which shows Bush the winner is something I simply cannot understand. Indeed, isn’t that why the mainstream media’s “theory” on the election is that many more people were excited about Bush than the nation had previously thought from pre-election polls and approval ratings?
Now that we’re perhaps learning that the *opposite* was true, is someone really going to argue that the conclusion (Bush won) should remain unchanged? What does it say about a theory that it can be used to prove either side of an argument?
The News Editor
The Nashua Advocate
http://www.nashuaadvocate.blogspot.com/
That does seem to be quite a leap. If Kerry supporters exuberance led to extra Kerry supporters answering the exit poll questions, then the exit poll should have more Kerry responses than would scale to the overall election. If the people conducting the poll also wound up asking slightly more Kerry supporters than they should, would that not also tilt the exit poll to having an inflated percentage of Kerry voters compared to what shows in the actual election. A very small shift in response pattern is all that would be required to throw the exit poll numbers off.
Considering your defense of those accused of vandalism, might it not be more a matter of your personal biases leading you to skew all evidence in favor of a preferred conclusion instead of dispassionately evaluating the claims?
Sjerp wrote:
“the main question that needs to be answered is why, for the first time in five presidential elections, WPE was seriously skewed towards one candidate.”
Because the Republicans engaged in systematic vote fraud to steal the election.
Nashua Advocate wrote:
“Now that we’re perhaps learning that the *opposite* was true, is someone really going to argue that the conclusion (Bush won) should remain unchanged?”
Not me! I’ve been arguing all along that Bush’s election is illegitimate and he should be removed from the White House. I’m glad someone influential in the media agrees. Now if only you guys could help to do something about it.
“Richard,”
I don’t waste much breath responding to disingenuous personal attacks — particularly on someone else’s blog.
Of course, I need only direct you to read The Advocate — something you’ve clearly not done adequately or competently — to answer your own question. For example, the part where we wrote —
“Of course, some might ask The Advocate, why is this important? And, alternately, are you defending vandalism? Second question first: no. It should be punished as any vandalism would be punished.”
You can find it at http://nashuaadvocate.blogspot.com/2005/01/news-advocate-continues-its-tiregate.html.
Otherwise, no comment. People can compare your libel to what we wrote and judge for themselves.
The News Editor
The Nashua Advocate
Fixed link (HTML coding was wrong on post):
http://www.nashuaadvocate.blogspot.com/2005/01/news-advocate-continues-its-tiregate.html
— TNE
The Nashua Advocate,
You seem to imply that a lower response rate from Bush supporters indicates less exuberance/excitement for Bush (and the converse for Kerry supporters.) If this is a correct reading of your post, I think the argument is weak.
Your first thought – “either these sorts of ‘interviewer issues’ have plagued exit-polling in every election” is the most likely problem, since these types of problems *actually have* plagued exit polling in the past.
“is someone really going to argue that the conclusion (Bush won) should remain unchanged?” Based on an exuberance deficit? Really?
Nashua Advocate: It was a mistake mentioning the vandalism story in my previous response since that had no relevance to my point.
However, you seem to be downplaying the impact of any extra exit poll responses generated by excessive exuberance. That one alleged extra response to the exit poll would have an equivalent impact to having almost 100,000 extra voters show up at that precinct would have on the total election. Now, to be sure, other exit poll techniques reduce the impact of that extra response by weighting the responses by the size of the precinct.
But with the small number of responses in many precincts, that one extra response can alter the precentages supporting various candidates within a given precinct by several percent. If the reported indication held throughout the exit poll with every fourth interviewer getting a bonus response, the exit poll will be well beyond the predicted error margin even excluding any other errors inherent in exit polling. I hope there were fewer than that many volunteered responses hampering the statistical relevance but just a few would account for much of the divergence between exit poll prediction and election count.
“Mark, this is plain silly. Please stick to statistics! There, the main question that needs to be answered is why, for the first time in five presidential elections, WPE was seriously skewed towards one candidate.”
really? I take it you’re unfamiliar with 1992.
“As a lawyer, I know that the fact that voters overall were more excited about the election — and perhaps more Kerry voters than Bush voters (arguendo)”
? as a lawyer….this doesn’t even deserve the term “bullshit.” what exactly did you learn in law school (or in practice) that gives you the special knowledge to reach such a conclusion?
and this is speaking as an attorney….people like you give my profession a bad name….when we somehow think that our three years of hardly arduous post-college schooling gives us any insights at all into any non-legal issue….
To Jeff and Nathan:
Nathan first. Nathan, you jumped the gun, which is another thing we attorneys do which often gives us a bad name.
You jumped the gun for this reason: I was making only a slight and inconsequential reference to being a lawyer, not using any authority granted to me by the profession (there is none) to “win” an argument. That is, I was referencing the fact that, in the law, the rules of evidence hold that something is “relevant” to a legal dispute if it tends to make a fact in dispute more or less likely to be true. So, my comment meant no more and no less than, “As a lawyer, having come across the notion of ‘relevance’ as a construct pertaining to ‘probative value’ with regard to an issue in dispute, I question whether or not the evidence being presented here is actually ‘relevant.'” Seems simple, but as you well know, Nathan, if you’re ever been in a courtroom, intelligent people argue over “relevance” — in the more formal sense of the word — all the time. I hardly meant to suggest I was any more qualified to debate this issue than you or anyone else.
Why would Mark’s anecdotes not be relevant? Well, how much time do you have? First, because they’re anecdotes, not (as Mark readily concedes, though by presenting the evidence here he imparts it some sense of “relevance” anyway) any sort of statistical study. Second, because if they *did* constitute a statistical study (i.e., even if the sample size were increased), it would be methodologically unsound. Third, because even if it were a study and it were methodologically sound, it nevertheless rests upon basic assumptions which prejudice the entire construct of the study in the first place: for example, do *either* Democrats or Republicans, in a hypothetical all-things-equal scenario, actually move *toward* a gaggle of MoveOn protestors outside a U.S. polling place, such that Democrats statistically have a better chance than Republicans of “seeing” an exit-pollster standing nearby? What was the political makeup of these communities? What’s the political makeup of the state? Is there evidence to support the presumption that conservatives fear the media more than progressives? Is there evidence to support the presumption that conservatives avoid younger people at a higher rate than progressives? Is there evidence to support the presumption that filling out an exit poll suggests exuberance toward a candidate — rather than, say, disgust towards his opponent, or boredom, or a desire to feel important, or curiosity, or a whole host of absurd potentialities, such as wanting to flirt with the exit pollster, or wanting to avoid the rush of the crowd as it walks collectively back to the polling-place parking area? The point is, no competent scientist would draw the conclusions Mitofsky has drawn from the available evidence. Nor any competent lawyer. Nor any competent thinker.
Jeff,
I agree with you. I do think the best argument is that these things have happened before without such stunning WPEs arising. The point I made about reassessing the results of the election was rhetorical, at least in this sense: there was no outcry from the social science crowd when Bush defied the pre-election polling to beat Kerry by three points. It was posited that Bush supporters were simply more exuberant than their Democratic counterparts come Election Day, and that somehow the media had never picked up on it before then. Was there evidence for that proposition? Not really. An equal percentage of Bush and Kerry voters told the press (see the Annenberg Election Study) that they couldn’t envision voting for the other guy. Everyone ignored that study.
So now Mitofsky comes out swinging with a hypothesis that has no statistical proof, and Mark (who’s done a fine job thus far) comes out with an irrelevent anecdote supporting Mitofsky’s ungrounded claim. My point was, if Mitofsky is telling us now that Kerry voters were more exuberant, doesn’t that contradict the media’s explanation for the outcome of the election? I certainly recognize that exuberance alone proves nothing — in fact, that’s my point. Indeed, in an ironic twist, I happen to have made my point by making my point: that is, when I suggest that exuberance proves something, it’s regarded as absurd (and indeed it is) — when Mitofsky suggests it, it makes sense. Why? Has anyone ever found that being enthusiastic about a candidate makes you more likely to spend ten minutes talking to an exit-pollster after you vote? Does polling the spouse of the “nth voter” really suggest that you’re polling an “exuberant” person, just because they’ve “agreed” to cooperate with a request their spouse declined? Is it not a tautology to say that if more Kerry voters were found in the exit polls, it’s explained by the fact that…more Kerry voters *took* the exit poll? Really? It’s that simple? Yet what does that prove? That Kerry voters were more exuberant — or that there were more Kerry voters *in fact*?
To the same extent that Kerry voters’ supposed “exuberance” suggests that they were more likely to take an exit poll, I could say that the fact that more Kerry voters participated in the exit poll suggests that there were more actual Kerry voters than Bush voters on November 2nd, 2004. Right? Is the exit poll “relevant” to show the predilections of voters on Election Day?
Because if it doesn’t even meet that barest standard of relevence, why are we discussing it in the first place?
*That’s* why “relevance” is important. And this post by Mark, bless his heart, ain’t relevant.
— TNE
TNE-
You are from New Hampshire. As you must be aware, New Hampshire had by far the highest difference between exit polls and official vote tally of any of the battleground states cited by Freeman. In fact, NH had the third largest discrepancy of any state according to the CalTech/MIT data. Yet enough of the New Hampshire vote was recounted by hand to satisfy Ralph Nader that the vote tally was correct. The wards recounted were those chosen by Nader because they had an unusually high Bush percentage compared to historic voting patterns.
If you accept the recount as valid (and again, Ralph Nader did) then you must accept that the exit poll in New Hampshire was wildly wrong. If the exit polling in New Hampshire is way off, on what basis can you assume that the NEP exit polling would be right elsewhere?
Using the NEP exit polls to buttress any claim of a “stolen election” won’t fly.
Marty,
A number of thoughts:
First, I didn’t say the exit polls proved a “stolen election.” But they certainly “buttress” suspicions that something went wrong on Election Day. Who else believes that? United States Count Votes, for one, comprised entirely of mathematicians and statisticians from top universities around the world. [Do a Google News search for “USCV” and you’ll find the article from The Nashua Advocate and the USCV Report just issued on 1/29/05]. Who else believes exit polls can “buttress” suspicions over an election? The Bush Administration. That’s what they said about the Ukraine. Were you skeptical then? Did you oppose a second run-off there?
No one thinks the exit polls are a “smoking gun.” But they warrant further investigation, and cause me — and many others — to believe that the election itself likewise warrants further investigation. Who else believes that? Besides USCV, The Democratic National Committee, for starters — which has just committed $500,000 to “open up the machines” in Ohio. They’re conceding that Bush won — but also pointing out that if there was no manipulation of votes, no one should be contesting opening up the machines.
As for New Hampshire, we use all optical scan machines — not the machines which have caused so much concern in Ohio (those are DREs, such as those in Mahoning County which, when they switched votes — which they did routinely on Election Day — switched those votes from Kerry to Bush 97% of the time, as opposed to from Bush to Kerry only 1% of the time [with 2% of the time from Kerry to other candidates, according to the non-partisan Election Incident Reporting System]). Also, Nader’s recounts were incredibly limited in scope, in a state whose voting population is, what, *one-fourteenth* that of Ohio? Do you realize how much larger the OH exit poll sample was than the NH sample?
By the way, I think “stolen” connotes a number of things short of fraud — for example, biased placement of voting machines, which caused (according to The Washington Post) 15,000 voters in heavily-Democratic Franklin County *alone* to leave the polls without voting (due to long lines). Or Kenyon College having 11-hour lines, and two blocks away a small, conservative parochial school having no wait whatsoever. These aren’t old wives’ tales — they’re reported in The Post, The Times, the AP. So, if I were John Kerry, would I require a showing of “fraud” to feel like the election *may have been* (notice I don’t say “was”) “stolen” from me and my supporters — or simply a showing that more people left their homes on November 2nd, 2004 to vote for the Democratic ticket than the Republican ticket?
If you lost out on a promotion because of an illegal and/or improper series of promotion procedures put in place by your rivals’ supporters, would you be upset? Or would you reserve your ire for *proof* the boss was actually paid cold, hard cash to promote someone else? Seriously?
— TNE
I think the point is not that Democratic voters were more “exuberant”, but instead that while both side may have been equally enthusiastic about their candidates one side avoided exit pollsters like the plague. In other words, they enthusiastically shunned the NEP folks.
As other posts on this site make clear this isn’t a new problem. Reference the “Exit Poll FAQ” section, which points out a consistant pattern of error in the NEP and its predecessor which consistently favor the Democratic candidate. In fact, the internal panel assembled by CNN to review the use of exit polling in 2003 recommended against using exit polls to call states for one candidate or another in the future. I am pretty sure Mitofsky himself said exactly the same thing, although I can’t find the link right now.
TNE-
When I read the following on your blog:
“It seems no one who has made an effort to do so can adequately explain, in scientific, rational, or even merely logical terms, how it is that George W. Bush came to be President of the United States for a second term. His Presidency, in short, defies all of the available mathematical data.”
It sure sounds like you are saying that the election was stolen. And you admit that the exit polls are part of the data that lead you to believe-or at least, explore- this idea.
Back to my main point: the exit polls were wrong in New Hampshire-there is no reason to think that they were right anywhere else.
Your point about the relative sizes of the states supports my argument. New Hampshire is a much more homogenous state than Ohio-yet the NEP exit polls were way wrong in New Hampshire. If they can screw up in the tiny state of New Hampshire that badly, why think that the exit polls in Ohio, a larger, more diverse state, were correct? And if the size of the state explains the discrepancy, why were New York’s exit polls off by even more than New Hamsphire’s?
My goal is to try to stay on point while playing in Mystery Pollster’s sandbox by limiting my discussions to polling issues, so I will close by answering your question directly:
If I lost out on a promotion, my first thought would not be that the person who got the promotion cheated. If I believe that they did cheat, I would try to expose the cheating. The Kerry campaign were relcutant participants in the Ohio recount. Once can easily conclude from their actions that they did not believe that they were the legitimate winners in Ohio.
Marty,
Let me make sure I understand you: we’re on a site dedicated to the science of polling, and you’re asking me to credit what a gaggle of Democratic politicians thought on November 2nd, 2004, before any information about polling irregularities had been revealed (and it would be adjudged “too late to do anything about it” every day thereafter) as opposed to crediting the ongoing bewilderment and demand for more information — or discovery and dissemination of contradictory information — being made by folks like U.S. Count Votes, The Annenberg Election Survey, The National Research Commission on Elections and Voting, every pre-election polling entity, and all the post-election polling which showed Bush with an under 50% approval rating *immediately* after the election? And in order to convince me to discard the most accurate form of polling known to mankind as irrelevant, you’re relying on a) a partial hand recount in one of the smallest states in the United States, and b) a self-serving and entirely non-scientific explanation by a man (Warren Mitofsky) who’s made his career by depending upon scientific explanations for phenomena — except, apparently, when it hurts him professionally to do so? Am I to make this man’s wounded pride my rallying cry, as you have?
And do I understand you correctly that I should bend over backwards to make all of these half-assed assumptions in the context of the Bush Administration having called for a revote in the Ukraine based upon *exactly* the same sort of exit poll discrepancies we’ve now seen in the U.S.?
And do I understand you correctly that on this site devoted to the *science* of exit polling you are asking me to privilege wild, unsubstantiated presumptions such as a) Republicans fear young people, b) Republicans fear the media, c) bad weather favors Democrats (oh, sorry, only when it comes to exit-polling, apparently–as we all know it depresses their turnout generally), and d) Republicans are somehow simply more “retiring” and shy than Democrats?
Are you serious? Do you know that this is a site devoted to *serious* science?
And if you are aware of that — what in God’s name are you thinking, besides that you’re thrilled to pieces your man won?
— TNE
TNE-
You cover a lot of ground in your post-too much ground to try to address point by point. I’ll try to cover my main points.
Since you refer twice to the science of polling in your post, I would like to apply the scientific method to this discussion. In the scientific method, you create a hypothesis and test that hypothesis to determine if it is correct.
The hypothesis to test is: The NEP exit polls were correct, and the final vote tally was flawed.
The best way to test this hypothesis is to select a state to recount. The ideal state would have the following characteristics:
1) A large discrepancy between the official vote tally and the exit poll.
2) A paper trail.
3) A small state so that the vote can be recounted by hand quickly.
4) Minimal “funny business” such as claims of voter suppression, etc.
In order to test the hypothesis, a hand recount would be done and results compared to the official tally.
This was done in New Hampshire. The hypothesis being tested fails. We can clearly say that in the state of New Hampshire the exit polls were wrong.
Now we can apply logic. If the exit polls in New Hampshire were wrong, we can logically conclude that the exit polls in other states may be wrong also. Thus, the search for the explanation as to why the NEP polls were wrong.
You dodged this question previously, so I will ask it again: was the NEP exit poll in New Hampshire wrong?
You mention Ukraine. The exit polling was not the only reason for the suspicion; the Organization for Security and Co-operation in Europe (OSCE) report goes into detail of the shortcomings of the election. It was this report, not the exit polls, that was cited in the State Department’s briefings on the subject. The exit polls made a compelling narrative, but they are not the main reason the US government determined that the vote in the Ukraine was fraudulent.
Let me close by pointing out that the “art” of polling gets equal billing with the “science” of polling on this site.
Okay, Marty, I’ll bite.
There are 126 WARDS in New Hampshire.
Nader recounted 11 WARDS.
There were 675,000 VOTERS in New Hampshire on Election Day.
Nader recounted ONE WARD EACH in the towns of Litchfield, Sandown, Newton, Danville, Salem, Pelham,
and Somersworth.
He recounted FOUR WARDS in Manchester.
Being generous, we’ll say that Nader recounted 8.7% of the ballots in NH (11/126).
That’s 59,000 VOTES.
Kerry won New Hampshire 50.3%/49%. Predicted outcome of those 59,000 votes: 29,677 Kerry/28,910 Bush.
Exit polls showed Kerry won New Hampshire 54.9%/44.1%. Predicted outcome of those same 59,000 votes, if they’re representative of the State as a whole (see below): 32,391 Kerry/26,019 Bush.
That means that, out of the hypothetically Nader-recounted 59,000 votes, Kerry got 2,714 “TOO FEW” votes. Bush got 2,891 “TOO MANY” votes.
The discrepancy averages out to 2,800 VOTES.
That would come out to a discrepancy of 32,000 VOTES statewide.
That’s 1/21 of the votes cast in New Hampshire.
A switch of that “magnitude” (32,000 VOTES) in Ohio would constitute 1/176 of the votes cast there. [Perhaps 4 votes per Ohio precinct].
See the effect of sample-size?
But that’s just where the fun begins. How about these “fun facts”:
1. STRAIGHT-TICKET VOTING. New Hampshire has it, most other states don’t, and it could have led to exit-poll discrepancies. According to The Nation’s Russ Baker, “Officials I spoke to in New Hampshire conceded that, in all likelihood, many of those choosing the straight-ticket option hadn’t read or properly absorbed the instructions, and so wrongly assumed they were merely indicating their party affiliation or registration rather than their intention to give all their votes to one party.”
[http://www.thenation.com/doc.mhtml?i=20041206&c=1&s=baker]
2. DIFFERENT MACHINES. The Nader press release revealing the results of the recount makes a vital point: “No conclusions can be drawn about the reliability of electronic voting machines on the basis of the New Hampshire recount, because the machines used in the 11 selected wards predate those showing irregularities in Ohio and other states, where votes were counted backward on some machines and votes were assigned to the wrong candidate on others. Secretary of State William Gardner reported that the machines used in New Hampshire also predate the Diebold Corporation’s purchase of the company that manufactured them…[t]he voting machines in the selected New Hampshire wards were purchased after manufacture and installation by Diebold Corporation, whose top officers were major contributors to the Bush campaign. The software on which votes are counted is protected as a corporate trade secret and voting rights groups report no independent audit is performed on the machines or the software, making the machines vulnerable to undetected programming error, equipment malfunction, and malicious tampering.” So, the recount, even under the best light, could not have resolved the issues seen on DieBold machines in Ohio.
3. I FUDGED MY MATH TO FAVOR YOUR HYPOTHESIS. We don’t know a) that Nader actually recounted 59,000 ballots (my information suggests it was half that or less, I gave the higher number only “arguendo”); b) whether the votes in those selected precincts in any way corresponded to the exit polls. For example, Nader announced prior to the recount that he was seeking to recount wards where Bush’s percentage of the vote exceeded by “5% to 15%” what was expected. But it may well be that most of those wards were in the “5%” category — meaning that whereas the statewide exit polls “overpolled” Kerry supporters by 9.5%, these wards may have been less inaccurate. Here’s an example: I did my calculation of “real-life” and “exit-poll” results for those 59,000 votes based on the reported 9.5% discrepancy between actual and anticipated tallies. But the recounted wards may a) have totaled closer to 25,000 votes, and b) had only a 5% discrepancy between actual and anticipated figures. A 5% discrepancy out of 25,000 votes is 1,250 votes, not the 2,800 I’ve referenced above. So what if Nader’s recount showed nothing more than Bush “overpolling” to the tune of about a thousand votes on machines different from those used in Ohio? Given that Ohio had 5.63 million voters on Election Day, and used different machines, what would the Nader “recount” in New Hampshire (was it of 8.7% of ballots? or closer to 3%?) tell us about election fraud in Ohio, or even the exit polls in New Hampshire?
Do 1,250 votes in New Hampshire probably attributable to confusion over a unique “straight-ticket balloting” system disprove a hypothesis? And by the way, since when do election doubters faced with suspicious evidence have to “prove” a hypothesis, rather than those conducting an election have to “prove” it has any integrity whatsoever? Aren’t you “burden-shifting,” Marty?
Let me make one thing clear: I didn’t assert the election was definitely “stolen,” though that hypothesis remains intact after all this New Hampshire-based bluster; all I’m saying — all I’ve said — is that an investigation needs to be done into the 2004 presidential election — the Democrats (cf. Conyers) want one, and the Republicans (cf. Sensenbrenner) are, predictably, stone-walling (just as 87 of Ohio’s 88 counties violated Ohio recount law, somewhat suspiciously I might add, in order to avoid doing the sort of manual recount which, to you, Marty, “proved” no fraud in the Granite State) — and Mitofsky’s explanation is, can we just say it, finally, obvious horse-poop.
There’s no “smoking gun” as yet, but there’s certainly a lot of smoke, particularly in Ohio, which had irregularities of the sort New Hampshire did not.
You haven’t diffused any of that smoke with your New Hampshire smokescreen, I’m afraid, Marty.
— TNE
TNE-
You totally miss the point of the recount. After the recount, the data for those eleven wards is definitive. A hand recount is the gold standard, gospel, Truth with a capital “T”. It does not matter if the count is 80% Bush, or 80% Kerry, or anywhere in between. It is what it is.
So now you can take this authoritative recount data and use it as a measuring stick. The first thing to compare it to is the original machine tally, ward by ward. The first ward compared is almost identical-Kerry and Bush each gain three votes out of 5,000 ballots cast. The second ward compared has a difference of +3 votes for Kerry. Each remaining ward has similar small discrepancies, until all eleven are compared to the machine tally. The machine tally for each of the eleven wards is confirmed as correct.
At this point you can see a pretty clear trend developing-the machine count was accurate in the eleven wards counted to date. One can easily assume that the machine count was accurate for the remaining 100 plus wards. Ralph Nader and Ida Briggs certainly did.
Notice that the exit polls have not been mentioned yet. That is the next thing we can measure. Which do you think is going to be more accurate:
A statewide machine count that has been verified by a hand recount of ~7% of the state’s ballots;
or an exit polling sample of ~.01% of the state’s voters?
Obviously the exit poll is insubstantial fluff compared to the weight of the verified machine counts.
Now onto to your fun facts:
1) Straight line party ticket voting cannot account for the exit poll differential because a mark for a candidate overrides the party line vote. The article you referenced said,
“When a person marks the “Straight Republican” or “Straight Democratic” option, all unmarked specific races are assigned to that party’s candidate.”
So if a user marks the “Straight Republican” ticket and then fills in the circle next to Kerry’s name, it is a vote for Kerry, and the tells the exit pollster that he voted for Kerry. No discrepancy there.
Regarding different machines, you support my point. you can’t blame the incorrect exit polls on Diebold stealing votes. No DRE’s were used in NH. The vote was fair; it was fairly tallied; and it proves the exit polls wrong.
By the way, there were exactly ZERO Diebold DRE’s used in Ohio. That’s right-none, nada, nil, zilch, zero, zip. Blackwell banned Diebold DRE’s from Ohio in June.
Form the press release from the Ohio Democratic Party on November 10:
No Ohio County used Diebold Electronic Voting Machines (See Press Release Below)
The Press release indicates that Blackwell halted the rollout of Diebold machines in July over security concerns.
http://ohiodems.org/index.php?display=ReleaseDetails&id=191201
Regarding your point 3, my hypothesis doesn’t need you to fudge math to support it. It stands on its won: the machine count of the New Hampshire vote was correct. As a result, the exit polls in New Hampshire are incorrect. If the exit polls in New Hamsphire are wrong, they certainly can be wrong in Ohio. That’s all I’m saying on this polling relating site-that the exit polls cannot be used to support any claim of a stolen election because they are clearly wrong in New Hamspshire.
If you want to argue Ohio, argue it with The Nation’s Russ Baker, whom you cite above-and who concludes that the Ohio election was not stolen. You can read about it at RussBaker.com.
Marty,
Your conclusion doesn’t follow from the evidence you use to reach it.
Your conclusion: Exit polls can’t bolster claims of a stolen election.
Reason for your conclusion: New Hampshire exit polls were wrong.
The problem, Marty, is that 10 of 11 battleground states deviated to Bush, and 43 of 50 states overall. That is, whereas you can single out New Hampshire and make a case that the error in the exit-polling was “random,” you’d have a hard time making that case nationally, because the odds of so many states deviating so wildly toward Bush are infinitesmal. And saying that bad weather or young interviewers just happens to explain those 250,000,000:1 odds is likely say Bush’s poor performance in the three presidential debates can be explained by the fact that he is actually an alien in disguise.
Sure, it’s possible, but does anyone think that explanation’s the magic key to unlocking Bush’s inability to construct a sentence?
I disagree with your New Hampshire assessment, not least of which because two of the first four wards they recounted (as The Nation “recounts”, no pun intended) were wildly off in their calculations, a “mistake” which was later “remedied” but without any explanation as to how the disparities were resolved. Not suggesting conspiracy here, by any means — I think New Hampshire runs a pretty tight ship — what I’m suggesting is that when you’re working with such a small number of ballots problems can appear and disappear more readily than would have been the case if Ken Blackwell had allowed Ohio to be effectively recounted — you can’t hide when it’s 5.63 million ballots, instead of maybe 25,000, which are being recounted. Consider, for example, the race for New York State Senator, discussed here —
http://www.cnn.com/2005/ALLPOLITICS/02/08/runon.recount.ap/index.html
— in which 127,000 votes were cast (almost like a mini-New Hampshire, which had only 5 times as many votes) and the *machines erroneously recounted the ballots* to the tune of a 1,700-vote disparity between the candidates ultimately being revealed to be an 18-vote disparity: a movement of 1,682 votes out of 127,000 (or one out of every 76 [!] ballots being incorrectly counted, not maliciously, just due to error). A similar rate of error in New Hampshire would mean that 8,880 ballots were miscounted. A similar rate of error in Ohio would mean that over 74,000 votes were miscounted (i.e., similar error in Ohio would *net* one candidate or the other that many votes). So, there’s 100 reasons the New Hampshire exit polls could be off which do indeed suggest an inaccurate Election Day tally, even if there is no fraud.
But even if New Hampshire’s exit polls were off due to, say, the fact that Erica Exit-Pollster was 27 rather than 72, and the sky was cloudly rather than clear — a fact you’ve done precisely nothing to prove is relevant to the debate, but which Mitofsky stakes his career on — it says nothing about how all the other polls were wrong in the same direction. So, here’s my query to you:
1. Do you deny that the Edison/Mitofsky Report does *not* present sufficient statistical evidence to support its theory, and in fact presents much evidence which contradicts it, as U.S. Count Votes has detailed in its recent report (available from The Nashua Advocate website)?
2. Do you deny that the Edison/Mitofsky Report presents the only *possible* explanations that we are currently aware of for the exit-poll discrepancies?
3. Do you therefore agree that if the Report does not adequately explain the discrepancies, then it is a fact that there is no current statistical explanation for the discrepancies?
4. Do you agree that a discrepancy in exit polls for a presidential election, which has no current statistical explanation, must be further investigated, given that it suggests an Election Day tally whose odds of being correct (or, conversely, a block of exit-polling whose odds of being incorrect) are infinitesmally small?
5. Do you agree that the odds of New Hampshire’s exit poll being wrong due to random chance are wildly different (exponentially different) than the odds of 43 of 50 state exit polls being wrong in Bush’s favor, along with (just to make things more fun) nearly 98% of all reported DRE failures in Ohio?
— TNE
“And saying that bad weather or young interviewers just happens to explain those 250,000,000:1 odds”
TNA…you’re an idiot. whatever odds you come up with only apply if exit-poll error is random…going back to 1992 (when it was almost as skewed for one candidate as this one)…it hasn’t been random. in other words your 25000000000000 to 1 number doesn’t apply if “young interviewers” skewed the sample…because it wasn’t a random skew.
as for needing to substantiate the notion that Republicans don’t like the media….you don’t know any Republicans do you? but there are numerous studies showing exactly that.
TNE-
I’ll answer your question first and maybe add more thoughts later.
1)I’m not an expert on statistics, so I will accept your premise for the sake of this argument-that M/E does not have the statistical evidence to back up his theory.
2)M/E has to be considered the definitive authority on the analysis of the data presented. They have had full access to it for four months. So if there is to be an analysis that comes from the data, M/E are the experts on it.
3)I agree that there is no statistical explanation for why the exit polls deviated from the election tally. That does not make the exit polls right. It could also mean that there is not enough data or granularity to explain why they are wrong.
Here’s an example of how an exit poll can be wrong without being able to be proven statistically.
African Americans voted ~84% for Kerry. Suppose that questions about race were left off the exit poll, and that African Americans were inadvertantly oversampled by the poll takers. The exit poll would give Kerry a higher percentage of the vote than he actually got because they oversampled a characteristic that they had no control for. There would be no statistical way to find this error because you can’t back this information out of the data.
Again, my point is that just because there is no statistical explanation for the discrepancy does not make the exit polls correct.
4)Sure, the exit poll discrepancies should be explored. It’s not like the discrepancy is a secret, however. I am sure that all concerned parties-from new organizations, to M/E, to the Democratic party each checked various sources to see if it’s possible that the exit polls were right. Consider the stakes:
For newsies, it’s bigger than Watergate. Anyone breaking a story proving that the statewide tallies were wrong would be awarded the Pultizer every year for the rest of her life. The fact that this story has not been broken indicates that there isn’t one there.
For the Democratic Party, it’s vindication-and control of the White House for at least four years.
For E/M, it’s their professional reputations at stake. The fact that their paper is a “mea culpa” is extremely telling. If Einstein told me that “e” does not equal “mc squared,” I’d listen to him.
The fact that these groups accept the vote tally and thus are rejecting the exit polls is very strong evidence that the exit polls are wrong. They are a lot closer to the election process than you or I; certainly they would protest if they saw something amiss.
I’ve got to go-maybe I will address finish up later.
I’m back. Where was I?
5) Again, I’m not arguing with you that the fact that the exit polls were so off indicates that something is wrong; I just disagree with what the exit poll numbers prove.
Here’s an analog that may help: In manufacturing, you may order small parts like screws 5000 at a time. A lot of times you bag them in quantities of 100 so that you can rapidly inventory them. Rather than count 100 screws, you set up a counting scale. You calibrate the scale and then pour in screws until the scale says that there are 100. You bag these screws and repeat the process. If, after doing this, you end up at forty nine bags and 23 leftover screws, you know there is a problem.
The question is, were you screwed out of screws by your vendor, or is the count on the bags wrong?
So you grab a bag, pour the contents into the scale, and it says “100”. Then you hand count them and find out that there really are 102. At this point you can say that the scale is wrong. You haven’t necessarily proven that your vendor sent you 5000 screws, but you know that the scale is broken. To me, that is what New Hamsphire proved-the exit poll scale is broken. In fact, you shouldn’t be suprised if most of the bags have the wrong count if the scale had a systematic bias.
Again, to me, New Hamsphire proved that at least one exit poll was off on an individual candidate’s tally by 4.5%. This becomes the margin of error by default; anything less than that can be considered within the margin of error. Strictly speaking it may not be not correct, but from a practical point it is.
Marty,
Hi — just a quick note to say that I haven’t forgotten about our conversation. My computer & the TNA website have been on the fritz and I’ve been focusing on getting back up to speed. I hope to check back in here soon. In the meantime, I do hope that Mark (and others, such as yourself) are continuing to look into the E/M Report, as well as comparing the data E/M released to the data you need to do your analysis. If data is missing, I hope folks will militate for the relase of that data — whether or not you think it will be any sort of “smoking gun” re: the election.
— TNE
Marty H,
You’re missing the point! Mitofsky and Lenski have been taking payments from Halliburton! Either that, or some Bush crony is holding a gun to the head of their distant cousins somewhere in Eastern Europe! They are in on it! Can’t you see!
HaHaHaHa .. okay…
I agree with Nashua Advocate to a point. The statistical evidence doesn’t *prove* anything, but it is highly suggestive.
When you take the entirety of the report (don’t just focus on a few tables that look a little funny), it seems to me that at some point common sense has to click in. But maybe I’m wrong.
I’m a Republican and I can see how the conspiracy theories are hurting the Dems more than my party. Makes most of middle America (Dean’s target demo) raise a lip, squint, and look sideways at them, and say, “huh?”
Maybe I should just keep my mouth shut… But… that would be bad for Democracy…
I’m working on a critical review of the EMR report, and I hope it will push them into releasing a bit more data, but, I’m just a peon. The EMR report is good, but it’s not as good as it could be; therefore, I suppose they deserve much of the criticism that has been leveled thus far.
Give me 2-3 weeks (day job is killing me and I’m finalizing two other papers on Freeman and Simon/Baiman’s papers).
Again, I think the point is that the source of the error in the NEP was systemic rather than random. Once again I would direct people’s attention to the “Exit Poll FAQ” section of this site, which asserts that:
1) There has been a systemic bias in the NEP and its predecessor which favors Democratic candidates.
2) This bias is evidant from polls dating back to the 1980’s.
3) The problem has reached sufficient proportions that the panel convened by CNN to review exit polling data recommended to CNN that exit polls should not be used to predict the electoral returns in specific states.
If that information is correct then I think it’s pretty clear that there is a problem with the exit poll itself. What’s the source of that problem? I don’t think there’s any way to definitively prove one hypothesis as superior to another. That’s not the fatal flaw that TNA seems to believe it is however. There are plenty of phenomena in the sciences where the underlying mechanism is not well understood. That doesn’t mean that those phenomena are imaginary however.