The Difference Between “Partial” and “Final” Exit Polls

Exit Polls Legacy blog posts

A few days ago John Kesich, a commenter on this site, complained about the way the term "exit polls" has been used to describe both projections and raw data. He had a point. Confusion over the term "exit poll" runs far deeper, as I have seen the modifiers "early" and "final" applied inconsistently to a wide variety of exit poll tabulations. Much of the confusion stems from the reluctance of the National Election Pool (NEP) to discuss the various tabulations they generated on Election Day.  Since they will not comment, confusion about the terminology is inevitable.

I have a backlog of questions to cover about exit polls, but I want to start by reviewing NEP’s various tabulations and projections and suggesting some terms to identify them clearly.

First a review of the process: On November 2, 2004, the NEP conducted separate exit polls in all 50 states and the District of Columbia plus a separate, stand-alone "national" sampling. NEP instructed its interviewers to call in to report their results three times on Election Day (all times are local): At about noon, at about 3:00 p.m. and roughly an hour before the polls closed. NEP started releasing tabulations of the vote preference question for the national survey and for most of the battleground states on an hourly basis beginning at 1:00 p.m. Eastern Time. Here is what I know about the various tabulations and projections:

1) Early-afternoon unweighted tabulations – NEP released tabulations for many states between 1:00 and 3:00 p.m. Eastern Time on Election Day based on partial data that were completely raw and unweighted.

2) Late afternoon tabulations (weighted by turnout) – At about 3:00 p.m. (local time) interviewers obtain a hard counts of actual turnout from election officials for their covered precincts. NEP officials use this data to weight their late afternoon exit poll tabulations to match the actual turnout in the sampled precincts. Presumably, NEP started to deliver weighted data for eastern states at about 4:00 EST, but weighted data for states in western states may not have been available until 6 or 7 pm EST.

3) Just-before-poll-closing tabulations (weighted by turnout) – Interviewers call in roughly an hour before polls close with their final tabulations and another hard count of actual turnout. NEP uses these final reports to prepare exit poll tabulations for each state a few minutes before the polls close.  The data are weighted by actual turnout.  The weighting procedure assures that the mix of precints within each state — urban vs. rural, Democratic vs. Republican, etc. — matches the actual turnout patterns recorded that day.  The network "decision desks" use these tabulations to "call" winners, but only in states where the leader’s margin far exceeds statistical significance (the tests of statistical significance assume "confidence levels" of at least 99%, not 95%).

NEP also generates cross-tabular tables (weighted by turnout) for each state just before the polls close. These are tables similar in format to those now available on CNN.com (though the results are now different) that show how respondents answered each question and the vote preference calculated across answers to each question. The cross-tab tables play no role in projecting winners. Rather, networks and newspapers use them to prepare "analytical" stories about the election.

4) Projections after the polls close – Once the polls close, NEP gathers actual results for the precincts sampled in the exit polls and also for another larger sample of precincts (typically referred to as a sample of "key precincts"). Since not all precinct data is available at once, NEP gradually combines the exit poll results and the actual vote counts into an evolving hybrid of projections and estimates that gradually improves over the course of the evening. Although the projection models and tabulations are reportedly quite elaborate, NEP and its forerunners have disclosed very little about them.

5) "Corrected" Exit Poll Tabulations – Once the actual results have been counted in the wee hours of election night, NEP re-weights the results of each exit poll so that the vote preference on the poll matches the actual count. They then release new cross-tabular tables for each state to the general public. In theory, weighting to match the vote preference to actual results makes the complete exit poll more accurate.

6) "Final" Tabulations? – The tabulations put out the day after the election may not be truly "final." I have heard rumors of additional revisions that either have occurred or are about to be released. For example, I heard about a week ago that NEP was about to "revise" its estimates for Hispanic voters. The national survey posted on CNN.com estimates that President Bush received 44% of the Hispanic vote, yet a story in Sunday’s Washington Post puts his Hispanic support at 42%. Does this difference reflect the rumored revision? I have no idea, but will report if I learn more.

7) Raw data deposited with the Roper Center – Consistent with past practices, NEP has promised to make a copy of its raw data available to scholars at the Roper Center archives, only on a more accelerated timetable than usual (about three months).

Those who are concerned about the discrepancy between the exit polls and the actual vote count should focus only on #3, the tabulations prepared just before poll closing for each state. Unfortunately, these were not officially released. The early releases (#1 & #2) were widely leaked and posted on the Internet, but had bigger discrepancies resulting from incomplete samples or the use of completely unweighted data. The later releases now available through official channels (#5) are not helpful for analysis of the discrepancy since they were "corrected" to conform to actual vote results. The only source of "just-before-poll-closing" results appears to be the data in the paper by Steven Freeman (more on this in the next post).

Let me anticipate a few pertinent questions:

Why so many different tabulations? As discussed here before, the exit polls serve at least three functions: (a) They help give producers and reporters a head start in preparing their election night broadcasts and stories, (b) they assist the networks in "calling" winners and (c) they provide a resource to help reporters and the general public interpret the results of the election. The mid-day tabulations and before-poll-closing cross tabs help provide a "head start" that gradually improves during the day. The before-poll-closing tabulations (#3) and the later estimates that incorporate actual votes (#4) facilitate official "projections." The various "corrected" releases (#5 to #7 above) – the only ones meant for wide release – serve the third function, providing an analytical tool for reporters, scholars and the general public.

Are "uncorrected" mid-day and before-poll-closing tabulations available from prior years? Generally, no. Again, these tabulations were never intended for public release. Leaked mid-day numbers are available only to the extent that the "leakees" saved them. Smatterings of before-poll-closing tabulations have appeared in journal articles but are otherwise unavailable through official sources. Moreover, I assume that researchers cannot easily replicate the "before-poll-closing" tabulations using the raw data available through the Roper Center – if they could, analysts like Ruy Teixeira would have run them rather than relying on raw, unweighted tabulations from past years.

Why will it take at least three months to release the raw data? The process of opening the raw data to scholars is slow because (presumably) the archivists need to format the data and prepare documentation so researchers can use it appropriately. Remember, we are talking about raw data from 150,000 interviews, 50 separate state exit polls, plus D.C. and the separate national survey, each with a slightly different questionnaire. The task of preparing it all is huge.

However, as John Kesich’s complaint implied, nothing prevents the immediate release of the various Election Day exit poll tabulations except NEP’s reluctance to do so. These results were disseminated in electronic documents to NEP members and subscribers on November 2 that were presumably saved somewhere. I am not sure what purpose continuing secrecy serves except to provide greater fuel to those spinning conspiracy theories.

Mark Blumenthal

Mark Blumenthal is political pollster with deep and varied experience across survey research, campaigns, and media. The original "Mystery Pollster" and co-creator of Pollster.com, he explains complex concepts to a multitude of audiences and how data informs politics and decision-making. A researcher and consultant who crafts effective questions and identifies innovative solutions to deliver results. An award winning political journalist who brings insights and crafts compelling narratives from chaotic data.