Time to revisit "incumbent rule," thanks to Mickey Kaus who highlighted this observation last week by Michael Barone’s column in U.S. News & World Report:
It may be time to revise one of the cardinal rules of poll interpretation–that an incumbent is not going to get a higher percentage in an election than he got in the polls. Lieberman was clocked at 41 and 45 percent in recent Quinnipiac polls; he got 48 percent in the primary election. The assumption has been that voters know an incumbent, and any voter who is not for him will vote against him. But the numbers suggest that Lieberman’s campaigning over the last weekend may have boosted his numbers-or that the good feelings many Democratic voters have had for him over the years may have overcome their opposition to his stands on Iraq and foreign policy.
I wrote about the incumbent rule quite a bit in the run-up to the 2004 elections (especially here and here), applied it the polls in Ohio and then considered how the rule came up short (here and here). Reconsidering the rule has been buried on my MP to-do list for some time, and while I lack the data to provide conclusive answers, today is as good as any to think out loud about some of the key issues involved.
The best known empirical assessment of this "cardinal" rule was written by Chicago pollster Nick Panagakis for the Polling Report in 1989. He gathered 155 final polls spanning the period from the 1970s to 1988 (though most came from 1986 and 1988) and found that for 82% of the polls, the majority of the undecided broke to the challenger. Note, that this statistic tells us how many polls showed undecideds breaking for challengers, not the proportion of the undecided voters that broke that way.
In September 2004, MyDD’s Chris Bowers persuaded Panagakis to share his database and updated it with polls conducted from 1992 to early 2004. Bowers took the process a step further, calculating the average split of the undecided vote over all the elections. He noticed something obviously important in retrospect. The incumbent rule seemed to be weakening (although he had little data from 1996): 80% of the undecided vote broke to challengers in the poll Panagakis collected between 1976 and 1988, but only 60% went to the challenger in the polls Bowers gathered between 1992 and the summer of 2004. And challengers did worst of all in the polls in 2002 and the spring/summer of 2004 (42% to the incumbent, 58% to the challenger).
I have not attempted the same sort of comprehensive review of all of final polls from the fall of 2004, but on the final national presidential surveys an average of roughly 40% of the undecided vote broke toward challenger Kerry. And the break of undecided voters in battleground states looks closer to 50/50. "According to the exit polls," as Slate‘s David Kenner and Will Saletan pointed out, "Bush got 46 percent of those who made up their minds in the last week of the campaign and 44 percent of those who made up their minds in the final three days."
One question I have wondered about is whether the apparent weakening between the 1980s and 1990s could have been an artifact of the changes in the nature of pre-election polling or the particular races included in the database. For example, did the 1990s see more polling in contests for Senate, Governor and local offices and less in presidential races? Did long term changes in the timing or volume of pre-election polling affect the statistics?
The more important question is why undecided voters have stopped breaking toward challengers in the final week of the campaign. There are many theories.
- One possibility is that post 9/11 politics makes voters more reluctant to take a chance on challengers. Are undecided voters more averse to change given the current emphasis on war and terrorism in our campaigns? Some of the high profile Senate and Gubernatorial races saw a break favoring in incumbents in 2002 (though the incumbents were not exclusively Republican). Consider also this bit of purely anecdotal evidence from MyDD’s Matt Stoller:
I phone-banked a bunch of undecideds who in all likelihood flipped to Lieberman in the waning days of the campaign. "I hate the war, I hate Bush, but I’m just not sure we can pull out right now" was the way they put it.
- There is also the alternative theory Barone articulated in his column last week:
The left is noisy, assertive, in your face, quick to declare its passionate support. Voters on the right and in the center may be quieter but then stubbornly resist the instruction of the mainstream media and show up on Election Day and vote Republican, as they did in 2004, or for Lieberman, as some apparently did this week.
- Or could this change reflect a change in the nature of campaigning? Negative television ads were a rarity in the 1970s, but have grown increasingly commonplace in the years since. Has the willingness of incumbents to "go negative" limited the ability of challengers to make the race a referendum on the incumbent and shifted the attention of late breaking voters to the alleged shortcomings of the challengers?
Unfortunately, I have no answers tonight. What is clear is that past trends are not much help in interpreting the pre-election polls of 2006. How the undecideds will "break"in the final days of the 2006 campaign is anyone’s guess.
UPDATE 8/15: Readers have made a number of points worth reviewing in the comments section about possible shortcomings in the speculation above, as well as with the previous analysis of the incumbent rule. One thing worth noting is that academic political scientists and survey researchers have devoted little if any attention to the incumbent rule. We certainly have a lot to learn about this "cardinal rule," despite its past popularity with campaign pollsters including yours truly.
Doesn’t the incumbent rule need to be researched and evaluated via subcategories? It didn’t seem realistic for Lieberman to be trounced by the undecideds, in a race he once led by 40 or 50 points, then fell behind by 10+.
It reminds me of this section of a PDF I read several years ago, regarding patterns of poll movement in presidential races, but I think it logically applies elsewhere: http://www.nuff.ox.ac.uk/Politics/papers/2002/w27/wlezien.pdf
“Once in the lead, however, the winner’s margin tends to shrink. Leads from the Labor Day period, for instance, eventually are halved by Election Day. Even the final pre-election polls distort the lead, though this partly reflects the lack of late polls in a number of years, as noted above. These results imply a persistent underdog effect, where the projected loser gains support as the campaign persists.”
I remembered that prior to August 8. Once Lieberman became the projected loser, particularly by big margin and with more than two weeks remaining, I fully expected him to do well with the late polls and late deciders. Similarly, other incumbents who trail in the polls and are the persistent underdog, namely Santorum (PA), DeWine (OH), Burns (MT), Ehrlich (MD), may do better than expected with the late polls and late deciders. Granholm (MI) as well, if she stays behind the remainder of the race. And Chafee (RI), if he survives the GOP primary. I can guarantee heading toward election day I’ll see posts on political blogs estimating the leads by those challengers are actually larger than the polls indicate, due to the incumbent rule, but I dispute that already.
We all know an incumbent who leads 45-41 with high number of undecideds is in potential peril, but I’d like to see the incumbent rule broken down to the more rare circumstance of incumbents who trail. Particularly when they were the underdog for weeks/months prior to election day, not tossup situations with polls favoring either candidate, like Carter/Reagan in ’80 or the ’98 D’Amato/Schumer senate race in New York, ones that broke sharply to the challenger at the end.
For example, incumbent Don Siegelman (AL) trailed throughout the final months and was down by roughly 4-8 points in the final polls of the 2002 governors race, yet had the lead all night before losing an extremely close disputed result. Incumbent Lisa Murkowski trailed every poll for a year in the 2004 Alaska senate race, normally by 3-6 points, before defeating Tony Knowles by something like 10,000 votes. Admittedly, that’s a strange case since the exit polls indicate Knowles actually won the late deciders by huge margin. I’ve come to believe Alaska may be worse than Georgia in terms of overstating the Democrat in pre-election polls. Knowles didn’t threaten his projected percentage in the ’94 or ’98 gov races, and Fran Ulmer failed by 15 points in the 2002 gov race against Frank Murkowski despite poll numbers much closer than that.
Also, the obvious question is how much we know about the incumbent rule in primaries? Panagakis mentioned primary results were included in his sample, but not how many, or any examples. Chris Bowers of MyDD did a great job with that followup study in 2004, but I’m not sure if he had access to primary polls, or if they were included in the added data.
I think with respect to Lieberman/Lamont, the real question is one of turnout: How many voters did the Q-Poll think were going to come vote on the 8th? It seems like the likely voter pool slowly drifted to Lamont, and Lieberman’s late strength may be due more to those who decided to vote in the last couple of days than a “rebounding” among undecideds and Lamont supporters.
Hello Mark, long time no talk. I hope all is going well for you and yours.
I vaguely remember trying to persuade you, over the summer of 2004 during the run-up to the election, that the incumbent rule wasn’t all it was cracked up to be. I am glad that Bowers pulled together the data to back my skepticism.
Regards,
Gerry
Maybe there are more self-identified undecideds now due to dissatisfaction with the incumbent and these voters in the end are more likely to stick with the incumbent. It would be interesting to compare the undecided break % with the % undecided and the incumbent approval rating.
You realize that Barone’s “theory” is hardly that, right? That it’s just a partisan slam disguised as an explanation? I’m surprised to see you give it any air time. If you switched the words “left” and “right” his complaint would be equally valid. (I.e., equally a partisan talking point, and hardly worth the notice.)
I wonder how much of Lieberman’s uptick in the final three days is related to the idea that, at least publicly and politically, Lieberman has more substance than a one issue candidate like Lamont.
One of the most interesting phenomena is the bigger the national stature of a Congressman or a Senator, the more they lose touch with the constituents that put them in office, and the narrower their margin of victory becomes. This obviously does not occur in every leadership race, but ask Tom Daschle if he thinks it had anything to do with his loss. Lamont, before he has even been offically elected, had a national reputation, whether deserved or not, as an anti-war leader and that may have turned off some voters who were making their decision in the final 3-5 days.
I also tend to think that the quick national exposure and the constant hammering by the liberal netroots about the war being the only issued trapped Lamont into a campaign that did not serve him well at the end. For example, does anyone know Lamont’s stances on energy policy, for example? What about immigration? What about education? Chances are most voters in Connecticut don’t and I suspect that Lamont’s positions are not all that different than Lieberman’s.
A cautionary note about the incumbent rule, based on my replication of/fiddling with Chris Bowers’ analysis for PRESIDENTIAL races. Bowers doesn’t really look at the allocation of undecideds at all. He looks at the change in incumbent share from final poll(s) to official return, and the change in challenger share, and figures that as a proportion. So if ten different polls all showed the incumbent up 52/47 with 1% undecided, and the incumbent won 51/49 — well, actually in that case his formula would blow up. It depends on both the challenger and the incumbent increasing their average vote shares.
I took 31 final PRESIDENTIAL polls from 6 races with incumbents — I think this is Bowers’ data set plus one more election. (To give an idea of how Bowers’ formula works, if you add up all the incumbent and challenger poll shares, and compare them with the vote shares, the challengers do a net 55 points better, and the incumbents do a net 11 points better. So Bowers would reckon that as a 5:1 split of undecideds to the challenger.) I reckoned the optimal allocation of undecideds to minimize the error in the margin. I got a 4.5 point margin to the challenger (as in 52-48, ignoring third-party candidates), which is statistically indistinguishable from splitting them evenly. Similarly, if you want to predict vote share, you should allocate almost as large a proportion of undecideds to the incumbent as to the challenger.
The incumbent rule could work regardless of NOMINAL undecideds — maybe incumbents tend to pick up soft leaners in polls and then lose them in the voting. But the presidential data don’t really support that, either. Simply ignoring undecideds, there are two presidential races out of six (1976 and 1984) in which the incumbent appears to get a larger vote share than the polls predicted.
I think we knew all along that the incumbent rule was likely to work better in smaller races with more undecideds. And in those cases, Bowers’ formula may be a better approximation of actual undecided allocation. I haven’t tried it.
Another thought: the point of the “incumbent rule” is to try to predict how self-identified undecided voters will actually vote in order to predict the outcome of the election. Perhaps if pollsters asked more questions of the undecideds, such as name recognition, candidate they lean toward and how strongly, approval ratings of each candidate, “if you had to pick right now, who would you choose”, etc., a better prediction could be made.
Hmm..Matt Johnston’s comment is also merely a slam disguised as an argument. Hint: calling Ned Lamont a “one issue candidate” reveals you to be somewhat less than objective in your analysis. There are plenty of blogs around for slamming individual candidates. I had hoped this one would restrict its attention to the actual difficulties in polling, and not let itself get mired in pettiness.
Naive and stupid question here. Why don’t pollsters ask about switch voters. There are questions about late deciders(not enough), but I never see questions about people who changed their mind, but clearly that happened in CT and is happening in a variety of races. Why would someone favor Casey one week and Santorum the next? It’s unfathomable to me that you could ever switch between these two choices or are all elections decided by “low information” voters. I can possibly see someone wavering between undecided and for a candidate based on a lesser of two evils if they disliked both, but to actually have a preference and then switch seems to indicate muddled thinking.
Er. Regarding the 2004 Presidential election, don’t forget the bin Laden tape–I think that had a lot to do with it.
In the McKinney race in Georgia, in the runoff, she secured a lower percentage of the votes in the run-off than she had in the primary. The biggest change in her runoff campaign was a promise to work to impeach Bush and to introduce the resolution if no one else would. I suspect this was critical in lowering the percentage of votes she got. And there weren’t that many crossover Republican votes this time (as there were when Majette beat her) – the Lt. Governor’s race with Ralph Reed running was too important(some for him, more against him).
On another subject, yes many voters are uninformed or minimally informed. Only since I have retired have I had to time to go over all candidates stands on everything. General elections can be fairly easy if you are a Republican or a Democrat – you usually know where you are going to cross lines; primaries are more difficult. . .the subtleties among candidates are often difficult to find.
My guess concerning the overcounting of Democrats in Georgia polls is that a number of socially conservative African-Americans tell pollsters they will vote Democrat then vote Republican. Also, I understand that many affluent African-American men (but not thier wives) tend to vote their pocketbooks, but may tell a pollster that they plan to vote Democratic if wife is in the room.
One more time, how do we get a better method of tracking votes than taking them from a touch screen and then counting them as if by magic? I don’t think a major election has been stolen yet, but we need to get this fixed before one is.
If pollsters have slanted the polls even slightly toward Democratic challengers during in past 4-5 years, it’s not surprising that Republican incumbents have done better than the incumbent rule would predict.
In other words, IF some pollsters have used polls that favor Democratic candidates and IF it is true that a larger percentage of undecided voters have chosen the incumbent since 2002, then the fact that more undecideds choose the incumbent would make sense because there are more Republican incumbents than Democratic incumbents.