2008 Presidential Polling in 2005: a REALLY Big Grain of Salt

The folks at National Journal’s Hotline raised an interesting question yesterday.  How useful are the head-to-head preference polls being released now on the White House 2008 primary races?  Not very, they concluded.  MP concurs, only more so.

Since the Hotline links will be of little use unless you’re lucky enough to have one of those coveted, pricey subscriptions, here’s the gist:

We took a look back at previous WH elections to see what polls look like three years out. Looking at the same point in the ’04 and ’00 cycles, we averaged the poll results for the first eight months of the year, meaning January through August in ’97 for the ’00 cycle, and January through August in ’01 for the ’04 cycle. For the ’96 election, there were only two polls for the year, one from 9/93 and the other from 10/93.

Their produced the following averages. Three years before the 2000 election, the head-to-head primary polls on the Democratic side correctly identified Al Gore as the front runner, but had George W. Bush running behind Colin Powell, who never ran:

WH ’00 Dem Primary Averages

46%  Al Gore
10% Jesse Jackson

9% Bill Bradley

7% Dick Gephardt

4% Bob Kerrey

3% John Kerry

WH ’00 GOP Primary Averages

29% Colin Powell

17% George W. Bush

13% Jack Kemp

11% Elizabeth Dole

10% Dan Quayle

The polls for the 2004 Democratic primary provided "the best example of tainted primary polls."  Three candidates who did not run (Gore, Clinton & Bradley) dominated the early trial heats, while the ultimate "frontrunners" (Kerry, Edwards & Dean) barely registered:

WH ’04 Dem Primary Averages

41% Al Gore

19% Hilary Clinton

9% Joe Lieberman

8% Bill Bradley

7% Dick Gephardt

4% John Kerry

2% John Edwards

2% Bob Kerrey

The 1996 polls had Bob Dole, the ultimate winner, ahead but once again suggested a competitive race between Dole and two candidates who did run (Powell and Kemp):

WH ’96 GOP Primary Averages

31% Bob Dole

24% Colin Powell

16% Jack Kemp

"Why," the Hotline asks,

do people spend time and money polling a race that may never happen? On the off chance it does, these numbers suddenly become very important. Of the candidates that were in the lead of their respective primary polls three years before the election, among those who actually ran (not Powell — either time) no one failed to get their party’s nod in the years we looked at — Dole in ’96 and Gore in ’00. (And at some point Bush in ’00 once Powell was not included in the primary matchups.) Although the "among those who a actually ran" is a large caveat. With no one on either side fessing-up to a WH bid yet, the primary polls become a Choose Your Own Adventure for politicos.

True enough.  But MP hastens to add a bigger problem.  The population of voters typically sampled by these early trial heat questions (the sample "frame") bears little resemblance to the relatively small slice of actual primary voters that will ultimately decide the 2008 nominations. 

Consider the recent Gallup poll that created the "buzz" noted by the Hotline.   Gallup started with a random sample of 1004 adults and used their standard party identification question to select two samples of self-reported registered voters:  406 "Republicans and Republican leaners" and 424 "Democrats and Democratic leaners."  Thus, the two samples amounted to 40% and 42% of the adult population respectively.  [MP does not mean to single out Gallup; other pollsters conduct these early trial heats in essentially the same way.  For example, a recent Marist poll provided results for Democrats and Republicans, groups that were 37% and 35% of adults respectively]. 

Now consider how many Americans actually vote in the primaries that decide the nomination.  Curtis Gans  of the Committee for the Study of the American Electorate (CSAE) regularly releases estimates of voter turnout.   His estimates of presidential primary turnout as a percentage of eligible adults** are as follows for the most recent contested presidential primaries:

2004 Democratic 11.4% (estimated)

2000 Democratic 10.1%

2000 Republican 14.9%

1996 Republican 11.2%

1992 Democratic 12.7%

1992 Republican 10.1%

1988 Democratic 15.8%

1988 Republican 10.0%

In other words, the horse race questions you are seeing on the 2008 race for the White House are sampling segments of the population that are three to four times larger than the electorates that will actually decide each nomination.  And keep in mind, we do not conduct a national primary, but a series of statewide primaries.   In recent years, the nominations have been decided after the first few primary states.  Do these primary polls have  some entertainment value?  Yes.  Do they have influence over early donors and endorsers?  Unfortunately, yes.   Are they useful is this in determining what will happen in three years?  Sometimes yes, sometimes not at all. 

It may be helpful to consider that private campaign pollsters — the people hired by the presidential candidates — do not bother with this sort of national primary horserace poll.  When they begin to do their internal surveys for presidential candidates, campaign pollsters will focus more on sampling individual states that come early in the process (Iowa, New Hampshire, South Carolina, etc) rather than looking at a national sample.  And even then, they pay far less attention to horse-race questions at this stage in the race than to favorable ratings that tell us how well each potential candidate is known.

Which brings us to the horse race numbers for 2008.  You can look at each result over at the Polling Report, or check these averages computed by the Hotline:

WH ’08 Dem Primary Average

40% Hilary Clinton

19% John Kerry

14% John Edwards

6% Joe Biden

3% Wesley Clark

WH ’08 GOP Primary Average

31% Rudy Guliani

27% John McCain

16% Condoleezza Rice*

10% Jeb Bush*

7% Newt Gingrich

*Rice & Bush included on only two polls

Speculate away if you wish, but MP recommends as big a grain of salt as you can find. 

**MP reminds readers that political scientists disagree about the most appropriate definition of eligible voters to use in calculating turnout.  Gans and the CSAE have historically used the "voting age population" (VAP) as estimated by the Census.  Prof. Michael McDonald of George Mason University has identified shortcomings and proposed an alternative statistic.  For more discussion, see McDonald’s web page, the CSAE Report and my own discussion back in October.