MoveOn vs. Gallup

Legacy blog posts Likely Voters Weighting by Party

I have to admit I didn’t see the MoveOn.org newspaper ad attacking Gallup until very late last night. And when I did I was somewhat taken aback by its ferocity, but given the subject matter of this blog, a response is obviously in order.

Please remember as you read what follows that I too have questions about Gallup’s likely voter model. I doubt that their selection procedure – which includes probes of knowledge of polling place location, interest in the campaign and reports of past voting – is as appropriate in July and August as it may a week before the election. Gallup’s methods are also worthy of scrutiny given the disproportionate attention their surveys receive due to their prominence on CNN and in USAToday.

It is also obvious that Gallup’s likely voter results in August and early September had Bush farther ahead than other polls of likely voters conducted in the same period, although MoveOn picked the week when that difference was greatest. Gallup’s survey of likely voters conducted September 9-14 did show a 14 -point Bush lead on the three-way vote (54%-40%), and there were seven survey organizations that reported over a roughly comparable period (IBD/TIPP, CBS/New York Times, Pew, Harris, Democracy Corps, New Democratic Network/Penn Schoen Berland and ICR) that average to a 48%-43% Bush lead among likely voters on the three way vote. Note that two of these (Democracy Corps and NDN) were conducted by Democratic polling firms.

So yes, it is appropriate to question Gallup’s likely voter model, and likely voter models generally, but the tone and substance of the MoveOn advertisement just goes too far. If Ruy Teixeira dances on the line between spin and demagoguery in his daily calls for weighting by party, this attack by MoveOn leaps across it.

Whatever doubts I have about Gallup’s model, I don’t believe for a minute that they are intentionally “Gallup-ing to the Right,” as MoveOn loudly charges. They say Gallup “refuses to fix a longstanding problem with their likely voter methodology” and imply that weighting by party is the fix, never mind that most of the “other publicly available national likely voter polls” they tout to counter Gallup do no such thing. And then they slime everyone involved by implying that the company slants its surveys to suit the whims of George Gallup, Jr, an evangelical Christian no longer involved in Gallup’s political polling operation. According to the MoveOn ad, Gallup, Jr. called his polling work a “kind of a ministry” whose “most profound purpose…is to see how people are responding to God.”

Call me a partisan, but I always thought that this sort of guilt-by-association smear of someone based on an exercise of a constitutional right – no matter how disagreeable – was something that Liberals fought against.

Also, consider this observation by Richard C. Rockwell, a professor of Sociology at the University of Connecticut (and former director of the Roper Center) who posted the following on an listserv of survey researchers (quoted with permission):

The Gallup Organization has historically been among the most forthcoming of all polling organizations about their methods and about any problems that might arise from those particular methods. This goes back to the 1940s, when Gallup (i.e., George [Sr.]) was among the founders of AAPOR. Moreover, the Gallup Organization makes its data available for public inspection through the archives of the Roper Center for Public Opinion Research at the University of Connecticut – the raw data, not just the tabular reports. Anyone can check out these data for any evidence of error or bias. You can even re-weight the data as you wish. The Gallup archives go back to the 1930s. Given the public availability of their data on a site not owned or controlled by the Gallup Organization, it would be extraordinarily difficult for Gallup to mess with the data for political or any other reasons.

I may not always agree with the decisions of the methodologists at Gallup, but I have no doubt they are professionals who exercise their best objective judgment in an atmosphere of intellectual freedom. We should respect that.

Finally, let me take off my survey research hat for a moment and put on my Democratic Party hat. I have admired MoveOn’s efforts, but I have to ask, is it now so flush with cash that it can afford to buy a full page ad in the New York Times a few weeks before “the most important election of our lifetimes” attacking a polling company? Do swing or less-than-likely voters really care? Wouldn’t it be better to spend that money, say, making a case against George Bush or just turning out the vote?

Advice from one Democrat to another: let’s keep our eyes on the prize.

Mark Blumenthal

Mark Blumenthal is political pollster with deep and varied experience across survey research, campaigns, and media. The original "Mystery Pollster" and co-creator of Pollster.com, he explains complex concepts to a multitude of audiences and how data informs politics and decision-making. A researcher and consultant who crafts effective questions and identifies innovative solutions to deliver results. An award winning political journalist who brings insights and crafts compelling narratives from chaotic data.