Polling the Props: Part I

The polling story in California just gets more and more interesting.  It involves next week’s special election to consider, among others, four ballot propositions supported by Governor Arnold Schwarzenegger.  This week brings a batch of new surveys based on an unusually wide variety of methodologies and getting some very different results.  If ever there were a story tailor made for MP, this is it.

For now, let’s start at the beginning and review the special challenges pollsters face when conducting pre-elections polls on initiative and referenda.  MP sympathizes with California’s pollsters, because the task before them can be treacherous.  It is tough enough to measure where voter preferences stand when they involve candidates, but when trying to gauge support for complex propositions in a special election and pollsters must deal with some special challenges: 

1) Selecting "Likely Voters" – The task of identifying likely voters, a challenge in any pre-election survey, is even greater for a special election when turnouts are typically low and variable.  California has had only two statewide special elections in the last fifteen years in an odd numbered "off year." In 1993, according to data from the California Secretary of State, only 27.7% of the eligible population voted.  In the 2003 special recall election that put Schwarzenegger in office, turnout was 43.1%, but the year before the election for governor drew only 36.1% of eligible voters.   

In the 2002 general election, the low turnout took California’s three preeminent pollsters (Field, PPIC and the LA Times) by surprise according to a paper co-authored by principals from each of those organizations (that was presented at the 2003 AAPOR conference).   The turnout was especially low among minority voters.  All three polls showed a greater percentage of Latinos and African Americans in their final surveys than actually voted.  The three polls averaged 15% Latino and 7% African American.  The LA Times exit poll showed the electorate to be 10% Latino and 4% black.  Since both groups tended to vote overwhelming for Democratic candidates, the race for governor turned out to be closer than the polls suggested.   

What will the turnout pattern be next week?  Who knows?  Pollsters have few reliable benchmarks to guide them regarding turnout for this special election.**

2) "Fall Off"– Predicting the likely electorate is just the first step.  A second wrinkle comes from the habit of some voters of skipping some or all of the ballot propositions.  According to the California Secretary of State, the percentage of voters that skipped any one of the propositions varied between 6% and 16% of those that cast ballots in 2003 and 2004. Pollsters differ in how they allow for the possibility that a vote might skip a proposition in ways that may affect the results.

Presumably, fall off will not be much of an issue in the 2005 California election, since the special ballot includes little else (if anything) to draw voters to the polls.  Still, some of the 2005 propositions interest voters more than others and fall-off may still be an issue for some of the lesser known propositions in California.  For those looking at polling on initiative and referenda in other states, especially in even numbered year general elections, gauging fall-off can be critical. 

3) Modeling the Ballot Language. This is the really big challenge.  Ballot propositions are notorious for confusing or even indecipherable language.  Yet we know that many voters make up their minds by reading the actual language on the ballot while standing in the voting booth or filling out their absentee ballot.  In most states, partisans wrangle over the language and form of the question (which side is yes or no), which may often be created with the explicit purpose of confusing voters or influencing them to vote one way or anther.  As such, it is important for pollsters to try to simulate the ballot language as closely as possible. 

For telephone polls, simulating the ballot language can be difficult to impossible given the length and complexity of the ballot questions.  This year’s California propositions are 60 to 70 words in length, a length most pollsters consider prohibitive for a telephone survey.  Reading that much text strains the patience of the respondent, and concentrating on that much dense ballot language read aloud is a task most people are simply unwilling to do.  So telephone pollsters try to simply and condense their descriptions, yet the words they choose can make very big differences in the result (see this week’s experiment by SurveyUSA on language for Prop 76). 

MP is hoping to discuss various approaches for simulating ballot language in a subsequent post, but for now suffice it to say the California polls have been taking some very different approaches to this vexing problem.

4) Tracking Opinion Over Time.  As in any election, partisans on both sides wage campaigns to raise awareness and persuade voters to vote one way or another.  The difference is that in many candidate races (especially those involving an incumbent) voters begin with at least some preexisting awareness of the candidates.  Propositions are created anew every election, so awareness starts at zero.  And since the heaviest campaigning (and advertising) occurs near the end, support can change rapidly in the final week or so of the campaign.  The timing of the poll is critical and last minute shifts of support are not uncommon. 

California’s Field poll has asks regularly whether voters have "seen, read or heard anything" about each proposition.  Awareness of the propositions pushed by the Governor has been high throughout the summer but still increased significantly during the campaign.  For example in June, Field found 70% of Californians aware of an upcoming special election "having to do with state spending and school funding."  This week, Field found 81% aware of "Proposition 76 having to do with state spending and school funding limits."   Similarly, awareness grew from 60% to 76% for the redistricting proposal (Prop 77) and from 64% to 8 grew from 60% to 76%.   

The most dramatic shifts in awareness occur for the lesser known propositions (a pattern more typical in other states).  In August, Field found only 17% familiar with Proposition 80 which involves regulation of electric service providers.  Prop 80 awareness grew to 41% on their October 18-24 survey, and another nine points (to 50%) this week.

Put all of this together, and it’s not surprising that polls on initiative and referenda are prone to a lot more error than other election polls.  In 2004, according to a paper by SurveyUSA’s Joe Shipman, polling on ballot measures had triple the rate of error of presidential statewide polls and double the error of other statewide polls. 

This difficulty is not lost on pollsters and consultants that work on initiative and referenda campaigns.   We tend to avoid taking polling results at face value and intead interpret the results in the context of some commonly observed patterns.   For one, support for these ballot propositions typically falls as the campaign progresses.  Compare the tracking polls (looking separately at each survey organization) and it is obvious that the usual pattern is at work in California.  Support has deccreased during the campaign for virtually every proposition.  Another common pattern is that voters who go into the polling place knowing little about a proposition tend to vote no, especially when the language or issues involved are complex.  As MP was painfully reminded last fall, heuristic "rules" of this sort do not always apply, but if the percentage supporting a measure has fallen behind opposition by the final poll, it usually foretells defeat.   All of this bode poorly for Schwarzenegger’s propositions.

Unfortunately, MP’s time is short this morning (as it has been all week), so he will need to end here on a bit of a cliff-hanger.  In the next post, we will take at look at each of the California polls, how their methodology differs and how these differences may be leading to some very different results.

**Update:  Add two more polling challenges that MP initially overlooked.  First, as many as half the ballots may be cast by absentee ballot, according to California election officials interviewed by the San Francisco Chronicle.  which means many voters have already voted or are casting their ballots right now.  Pollsters need to take that behavior into account on their questionnaires.

A new wrinkle on turnout concerns something called the Permanent Absentee Voter (PAV).  In California, one registered voter in five has now signed up to automatically receive an absentee ballot fo reach election, rather than having to make a new request each time.  That amounts to 3.3 million voters who have received the special election ballot in the mail.  The SF Chronicle piece has more details.  How many of these voters will return their ballots for next week’s election?  It’s anyone’s guess.