The War Room

January 27, 2005November 12, 2019 Mark Blumenthal16 Comments

Of the newly disclosed data in the Edison/Mitofsky report on this year’s exit polls, some of the most important concern results from past elections. Although I had found snippets before, the data was not nearly as comprehensive as what is now available in the report. The implication of all the numbers is clear: The 1992 exit polls were off by nearly as much as those in 2004. Even better, there’s a movie version.

But I’m getting a bit ahead of the story. Much of the speculation on the blogosphere and elsewhere about the problems of this year’s exit polls begins with the premise that these problems are new. While it is is true that the average “within precinct error” (WPE) of 6.5 percentage points in Kerry’s favor was, as the report states, “the largest…we have observed on the national level in the last five elections” (p. 31) there was also a similar error in 1992 and a bias favoring Democrats in every national exit poll conducted since the networks started doing a combined exit poll in 1988. To be more specific, the report shows that:

The average WPE this year (-6.5) was almost as high (-5.0) in 1992 and favored the Democrats in 2000 (-1.8), 1996 (-2.2) and 1998 (-2.2 – p. 31).
Within states, the degree of error in 2004 at the state level tended to correlate with the degree of error in past elections, especially 2000 and 1992 (p. 32). As the report put it, “seven of the ten states with the largest WPE in 1992 were also among the fifteen states with the largest WPE in 2004 (California, Connecticut, Delaware, Maryland, New Hampshire, New Jersey and Vermont)” (p. 32).

But, as Ruy Teixeira points out, the presentation of these statistics is a bit arcane for some. So let’s watch the movie version.

The classic documentary “The War Room” followed the backroom exploits of James Carville and George Stephanopoulos on behalf of Bill Clinton’s 1992 campaign. The film ends on Election Day, and one of the final scenes shows Clinton’s operatives reacting to the first leaked exit poll results.

At one point the camera peers over the shoulder as a staffer jots down state-by-state poll results coming in over the telephone. A clock on the wall shows the time: 1:15 Central (2:15 EST). Next, a campaign worker appears reading to someone over the phone the “latest numbers, 2:00 o’clock” (presumably 3:00 EST) for California and Colorado. With a little help from the DVD, I reproduced the handwritten numbers in the table below, and added the the actual Clinton margin in each state, the difference and also the average “within-precinct-error” that VNS for each state included in last weeks’ Edison/Mitofsky report.

[A few caveats: First, given the time of day, the numbers had to be the so-called “first call” numbers, the most raw and partial results that interviewers called in at roughly noon local time. The difference I calculated at the state level is a very different statistic than the average WPE that Edison/Mitofsky reported this week for each state in 1992 – the latter was a precinct level estimate based on the full day’s exit poll. Also keep in mind the possibility that the leaker or leakee may have confused some of the numbers in haste].

Nonetheless, it is obvious that these mid-day reports in 1992 were off by nearly as much as the leaked numbers everyone saw on the Internet in the middle of the Election Day, 2004. Eleven of twelve states had an error in Clinton’s favor; seven had errors on the margin of six points or greater. These first call estimates showed Clinton ahead in three ~~four~~ states (Alabama, Florida, ~~Indiana~~ and Kansas) that he ultimately lost. The average state level error for these twelve states (-5%) was the same as the overall nationwide WPE on the complete exit poll.

But there were two big differences between 1992 and 2004. First, Bill Clinton still won the election, so few came away feeling fooled or suspicious that an election had been stolen. Second, the Internet was in its infancy, so the leaked results spread to a few hundred reporters and politicos, not the millions that saw the leaked exit polls in 2004.

Finally, one last gossipy tidbit with appeal to those still smarting over the “bloggers blew it” meme: The cameras caught Carville charging into George’s office with the very first leaked exit poll results. Carvile says:

“Popkin talked to Warren Whatever, the head of the VRS, and he’s going to talk to him again in four minutes, but his initial impression is landslide, could be up to 12 [points], maybe 400 electoral votes.”

Translation: Samuel L. Popkin, a University of California San Diego ~~State University~~ political science professor who had been advising the Clinton campaign in 1992 had talked to Warren Mitofsky, the head of Voter News Services (VNS, the forerunner of NEP) about the “first call” numbers. You can draw your own conclusions about whether the “initial impressions” belonged to Popkin or Mitofsky. The final result was a bit different. Clinton won the national popular vote by a 6.4% margin, not 12% and won 370 Electoral College votes, not 400.

So, assuming we believe James, the leak came not from some irresponsible blogger but from Mitofsky to Popkin to Carville.

Gotta love cinema verite.

[Typos fixed – 1/27]

Mark Blumenthal

Mark Blumenthal is political pollster with deep and varied experience across survey research, campaigns, and media. The original "Mystery Pollster" and co-creator of Pollster.com, he explains complex concepts to a multitude of audiences and how data informs politics and decision-making. A researcher and consultant who crafts effective questions and identifies innovative solutions to deliver results. An award winning political journalist who brings insights and crafts compelling narratives from chaotic data.