I have to admit I didn’t see the MoveOn.org newspaper ad attacking Gallup until very late last night. And when I did I was somewhat taken aback by its ferocity, but given the subject matter of this blog, a response is obviously in order.
Please remember as you read what follows that I too have questions about Gallup’s likely voter model. I doubt that their selection procedure – which includes probes of knowledge of polling place location, interest in the campaign and reports of past voting – is as appropriate in July and August as it may a week before the election. Gallup’s methods are also worthy of scrutiny given the disproportionate attention their surveys receive due to their prominence on CNN and in USAToday.
It is also obvious that Gallup’s likely voter results in August and early September had Bush farther ahead than other polls of likely voters conducted in the same period, although MoveOn picked the week when that difference was greatest. Gallup’s survey of likely voters conducted September 9-14 did show a 14 -point Bush lead on the three-way vote (54%-40%), and there were seven survey organizations that reported over a roughly comparable period (IBD/TIPP, CBS/New York Times, Pew, Harris, Democracy Corps, New Democratic Network/Penn Schoen Berland and ICR) that average to a 48%-43% Bush lead among likely voters on the three way vote. Note that two of these (Democracy Corps and NDN) were conducted by Democratic polling firms.
So yes, it is appropriate to question Gallup’s likely voter model, and likely voter models generally, but the tone and substance of the MoveOn advertisement just goes too far. If Ruy Teixeira dances on the line between spin and demagoguery in his daily calls for weighting by party, this attack by MoveOn leaps across it.
Whatever doubts I have about Gallup’s model, I don’t believe for a minute that they are intentionally “Gallup-ing to the Right,” as MoveOn loudly charges. They say Gallup “refuses to fix a longstanding problem with their likely voter methodology” and imply that weighting by party is the fix, never mind that most of the “other publicly available national likely voter polls” they tout to counter Gallup do no such thing. And then they slime everyone involved by implying that the company slants its surveys to suit the whims of George Gallup, Jr, an evangelical Christian no longer involved in Gallup’s political polling operation. According to the MoveOn ad, Gallup, Jr. called his polling work a “kind of a ministry” whose “most profound purpose…is to see how people are responding to God.”
Call me a partisan, but I always thought that this sort of guilt-by-association smear of someone based on an exercise of a constitutional right – no matter how disagreeable – was something that Liberals fought against.
Also, consider this observation by Richard C. Rockwell, a professor of Sociology at the University of Connecticut (and former director of the Roper Center) who posted the following on an listserv of survey researchers (quoted with permission):
The Gallup Organization has historically been among the most forthcoming of all polling organizations about their methods and about any problems that might arise from those particular methods. This goes back to the 1940s, when Gallup (i.e., George [Sr.]) was among the founders of AAPOR. Moreover, the Gallup Organization makes its data available for public inspection through the archives of the Roper Center for Public Opinion Research at the University of Connecticut – the raw data, not just the tabular reports. Anyone can check out these data for any evidence of error or bias. You can even re-weight the data as you wish. The Gallup archives go back to the 1930s. Given the public availability of their data on a site not owned or controlled by the Gallup Organization, it would be extraordinarily difficult for Gallup to mess with the data for political or any other reasons.
I may not always agree with the decisions of the methodologists at Gallup, but I have no doubt they are professionals who exercise their best objective judgment in an atmosphere of intellectual freedom. We should respect that.
Finally, let me take off my survey research hat for a moment and put on my Democratic Party hat. I have admired MoveOn’s efforts, but I have to ask, is it now so flush with cash that it can afford to buy a full page ad in the New York Times a few weeks before “the most important election of our lifetimes” attacking a polling company? Do swing or less-than-likely voters really care? Wouldn’t it be better to spend that money, say, making a case against George Bush or just turning out the vote?
Advice from one Democrat to another: let’s keep our eyes on the prize.
MoveOn is a political organization. Gallup’s political polls are, of course, fodder for political commentary and political news.
Given Gallup’s relationship with CNN, it’s polls are picked up and spread by CNN as close to the gospel truth, with little regard for it’s methodology or it’s relationship with other polls.
That’s not really Gallup’s fault, but the media’s fault. (CNN for trumpeting “their” poll, for instance).
Sure, in a perfect world, MoveOn could take out a full-page add attacking CNN for blantantly ignoring the huge disparity between Gallup and the bulk of the polling world, and that would be the end of it.
Neither politics nor people work that way, and I applaud MoveOn for being pragmatic enough to take the fight where it’s needed.
Distasteful or not, MoveOn is playing to win. As are their counterparts on the right.
I would gently suggest that you are missing the point. MoveOn is devoted to getting Democrats elected. They’ve made the decision to do this by trying to influence both voters and the media. This is simply a way to influence the media narrative around polling. By buying a large flashy and vitrolic ad in the Times, they will force news outlets to at least mention the fact that there might be problems. To be honest, this is more than half the battle – if the narrative can be changed to be at least neutral towards Kerry, then the MoveOn ad has done its work. Is the ad over the top? Sure. Does it have elements of smear by association? yup. Does that bother me? yes, much as it bothers you.
Alas, in our current “news” climate, this is what works. We’ll never see an actual conversation about the relative merits of various likely voters models, sampling, weighting, etc. by you or others like you. Even if we did see such a thing, no news organization would be able to distill it into the requisite 5 second explanation.
MoveOn has been moving in this direction during this election cycle – less content and more attack. This is why I no longer give money to them. But, I do appreciate the results…
I would also look to the response by both CNN and USA Today – in neither case anything approaching a thoughtful look at the issue. Whether we like it or not (and I don’t), this election is caught in a perception trap.
When Jeff Greenfield (for example) can blandly state an out-and-out untruth (that Fox News broke the Bush DUI story in 2000), and not be held accountable; when CNN and USA Today can back Gallup without addressing the issues raised (whether the critics are right or not, there are issues); when there are massive differences among seemingly competent pollsters (go to http://anonymous.coward.free.fr/polls/pollbias.html and then all the way to the bottom of the page for a graph); when one might argue that CBS was shut down by forgery to avoid an airing of both the Niger story and – perhaps more important – any more prison pictures; then MoveOn bringing attention to this issue is not only right, but necessary. Most mass media outlets are avoiding any issues other than soap opera plotlines – we deserve better than that.
As an amateur sociologist, I can sit back and sort of scratch my head and wonder why Gallup’s results are so out of line with other pollsters, and acknowledge that different organizations have different sampling methodologies and they’re all run by competent people who are sincerely interested in the truth, and who knows, maybe Gallup has the best technique and all its competitors are off the mark, etc., etc.
As a Democrat, I am terrified that a steady flow of “Bush ahead by double digits” headlines will dissuade the soft Kerry supporters from voting (volunteering, donating), and the poll will become a self-fulfilling prophecy.
MoveOn is just working the refs[*], and as long as they do so without actually lying, I’m not going to quibble about their tone. The Republicans have spent twenty years learning to make editors cower before accusations of “liberal media bias,” and it’s about time Democrats learned to use the same techniques.
The ad only works if MoveOn’s ‘brand’ is more trusted than Gallup’s. In other words, the ad only works with Kerry’s base……which is okay as far as it goes.
Seth’s comment about the impact of polling is right on – dispirited supporters are not going to bang on doors the day before the election. So reaction is absolutely required – and I’m not sure what else they could do to attempt to discredit Gallup. But there are other polls out there that are equally devestating: Time Bush +6 and CBS Bush +9. Since they are probably doing the same weighting as Gallup I think MoveOn would have been wise to attack all three at once for identical reasons.
The Cowboy And The Patrician
I don’t know about you, but I’m glad our president is from a state like Texas. I’ve read numerous accounts of George Bush’s “swagger.” In my opinion, this is something only an insecure man would have a problem with. After all, why would a real ma…
What’s funny about this is that it was conservatives last time who insisted that Gallup’s polls were off, particularly the polls from October.
All the complaining in the world didn’t make a difference to press reports of the polls. So, fair being fair, they wouldn’t make any difference this time.
For those who recall, those on the right were insisting that people should pay attention only to the Rasmussen polls–those were straight down the middle, these partisans said, and always showed a Bush lead. (I think that’s how one was supposed to know that they were straight.)
So much has changed, but not so much.
What’s funny about this is that it was conservatives last time who insisted that Gallup’s polls were off, particularly the polls from October.
All the complaining in the world didn’t make a difference to press reports of the polls. So, fair being fair, they wouldn’t make any difference this time.
For those who recall, those on the right were insisting that people should pay attention only to the Rasmussen polls–those were straight down the middle, these partisans said, and always showed a Bush lead. (I think that’s how one was supposed to know that they were straight.)
So much has changed, but not so much.
Gallup Defends Poll
Gallup’s chief pollster defended their presidential poll last night on CNN’s Inside Politics after withering criticism from bloggers Chris Bowers and Ruy Teixeira. Gallup partner USA Today also defends the poll. Will bloggers take down yet another esta…
Buzz Potamkin —
Why should either CNN or USA Today be expected to respond thoughtfully to a demagogic, sliming, guilt-by-association attack ad? If there are serious, reasonable points to be made that CNN and USA Today should answer, it’s MoveOn’s responsibility to bring them up in a serious, responsible manner.
If, however, MoveOn is just trying to rally the base by throwing out irresponsible attacks, CNN and USA Today have no obligation to treat their claims with any more respect than they have to answer the wild rantings of the John Birch Society, whether or not there’s a point buried deep within the slime.
Call me a partisan, but I always thought that this sort of guilt-by-association smear of someone based on an exercise of a constitutional right – no matter how disagreeable – was something that Liberals fought against.
There seem to be a lot of Democrats who aren’t what used to be called Liberals.
Hasn’t Gallup basically come back into line with the other polling outfits?
They now have Bush up by 8, and so does Pew and CBS(!). The average over at RealClearPolitics is Bush by 5.9.
I think MoveOn just wasted an ad.
And They Will Move On — When?
The Mystery Pollster eviscerates MoveOn.org’s recent ad criticizing the Gallup Poll….
moveon is trash. i wouldn’t expect any different.
Moveon V. Gallup–My Solution
If MoveOn is so peeved about the Gallup poll, they should use my poll instead. Take a look at how good Kerry’s poll numbers are: These numbers are so much better than that dull Gallup thingy.
The most interesting point of the article:
“Wouldn’t it be better to spend that money, say, making a case against George Bush or just turning out the vote?”
That wouldn’t matter, either, because of the one thing MoveOn can’t figure out how to do, and which you don’t address: making a case FOR Kerry. That is what really makes the MoveOn ads irrelevant.
Moveon.org Attacks Gallup
Mystery Pollster: MoveOn vs. Gallup Democratic pollster Mark Blumenthal responds to Moveon.org’s “ferocious attack” on the Gallup organization. Call me a partisan, but I always thought that this sort of guilt-by-association smear of someone based on an…
Mr. Blumenthal,
MoveOn’s strategy is clear (and I am sure totally uncoordinated with the Kerry campaign *cough* *cough*).
The Kerry campaign badly needs to fight off the perception that he is out of the race, not so much because of any bandwagon effect (which is a myth in my estimation) but because the media drumbeat on stories about how his campaign is failing and how it failed would drown out any hope he would have of getting his own message out.
The fact that Moveon exists and can make the attack rather than Kerry makes it all the better.
The fact that it involves the type of smear you said you thought liberals fought against might cause you to open your eyes a bit. Your side is not as pristine as you would like, if it ever was. People like you, on one side, and like me on the other, have to speak out loudly and forcefully against our own side when they do crap like this. It is the only way we can pressure the mud out of the system.
Thanks for taking a stand against it.
Moveon.org didn’t waste it’s money – they can get more from George Soros anytime it wants. And he’ll make it back by selling the dollar short when Kerry wins the election (at least that’s his plan).
MoveOn drank the “working the refs” Kool-Aid and are playing the role of hyper-rational bias hunter of pollsters.
According to Open Secrets MoveOn has spent $8 million more than their reported revenue.
http://www.opensecrets.org/527s/527cmtes.asp?cycle=2004&format=&level=C
I keep on waiting for Move On to Move On. Until they quit the mud slinging, I think that the opposite of their commercials are true.
The Gallop organization will not doubt survive the “fit of stupidity” on the part of MoveOn.org. No Doubt, MoveOn.org will too.
I find it hard to put any faith in any of the polls at this time. There are predictions that this election will have record turnout from the deceased. Since the poll takers refuse to hold a seance, they just can’t predict how the vote will go.
Who gives a rat’s ass what that bunch of homos think? Move on up an ass.
I think MoveOn.org should concentrate on
Sen. Kerry’s most important issue.
What he should wear to President Bush’es
re-inauguration late January 2005.
Anything else is really just a waste
of MoveOn.org’s contributer’s money.
Why in the world would Moveon.org spend that kind of money advertising in the NYT where they are complaining to the faithful? Wouldn’t it have made more sense to split up that money and buy ads in several different major newspapers in SWING states such as Florida and Ohio. So what if it was for national exposure because at the national level the message is somewhat diluted and wasted on committed states and voters. Oh well, I guess Moveon.org can spend George Soros’ money any way it sees fit. Democrats just crack me up.
As far as money well spent, by placing the ad in the NYT, more people are talking about it. I don’t know if it would have gotten that type of buzz in a large number of regional papers. I think that Gallup does have some valid points on party affiliation. I am not registered with any one party, but I’ll be rooting for a Republican victory, as will my wife (also unregistered). My parents, also unregistered, are voting for Kerry. In this election, I’m leaning more Republican, and would probably identify as such to a pollster, even though I’m not registered.
Perhaps the question should be changed to “Which political party are you currently registered with?”
Typical liberal. They can’t debate, they have to insult. So lowly…
The interesting thing is this…If MoveOn was a site anyone visited they need not have taken out an ad at the New York Times but need merely have posted at their site instead. So they have confessed their own irrelevance in the most public way possible. Sucks to be them. ^_~
I actually have some polling experience from the 2000 election with the head of CNN’s polling (who is directly involved in both the Gallup/CNN/USA Today and Harris/TIME/CNN polls). If there were bias introduced by a survey he was affiliated with, I suspect it would go to the left, and not toward Bush.
However, when I worked with him during the 2000 election, he made every attempt to be an honest public opinion researcher, and I would be shocked if he or any poll he was associated with intentionally introduced any bias. For that reason, I believe that MoveOn.org is almost certainly wrong.
I suspect that what probably happened with that poll is that it was just outside the confidence interval. When polls give a margin of error, the margin of error is with 95% confidence (generally). That means that one out of every twenty polls is outside the margin of error, if the general population were to take the same survey. When one considers this, it is not at all surprising that some polls come out with results that are out of line of other polls. Given the number of polls that have been done on the national Presidential election in 2004, it is surprising that more outlier results aren’t noticed.
Weighting for party ID doesn’t solve this problem in the least. It is a result of the underlying problems of RDD polling.
Ultimately, this is just another example in the laundry list of examples of political and media commentators acting on polling data without understanding polling methodology and statistics.
Scott Pauls,
You said: “- if the narrative can be changed to be at least neutral towards Kerry”. Why are you seeking a dimunition of the positive MSM coverage of Kerry? I would agree that there hasn’t been much positive coverage of his past (actually there hasn’t been much said about his past track record at all most likely because there isn’t much there to comment on). I think the “narrative” has been very positive with regards to Kerry, especially if you count the ‘on screen time’ he gets and the positive spin given his proclamations. Be careful what you wish for regarding the “narrative”. It appears the MSM is the only thing he has going for him and if they go neutral he might disappear from the polls all together.
Nice post Mark.
I think there’s little question that Gallup’s poll showing Bush up by 14% was seriously flawed. I’m not sure that should have been released. But Gallup has since polled pretty consistently and not out of the mainstream.
The moveon.org is really disturbing in many ways. I do wonder that if trying to smear a group by linking it to ‘evangelical Christian’ is a bright idea. I think it’s a bit offensive to most Americans. The right-wing equivavlent would be something to the effect of “move.on org, funded by George Soros, a devout Jew.” I think it skews the message and is reflects badly on the attackers.
Politics and media is returning to a more bitter era of the turn of the 19th century when Jefferson vs. Adams fought a nasty fight using partisan media.
DC Hoo writes:
>>The right-wing equivavlent would be something to the effect of “move.on org, funded by George Soros, a devout Jew.” I think it skews the message and is reflects badly on the attackers.<< Hmm, didn't Dennis Hastert (two missed heartbeats away from being our Pres) say something about Soros getting his money from drug-dealing? And didn't he re-affirm it when he had the chance to "clarify"? When it comes to smearing, moveon.org has nothing on the Republicans - don't believe me, read a few of the vicious attacks in this thread - poor Mark, all he wants is a discussion, and he gets flames from anonymous posters with fake e-mail addresses.
I downloaded the PDF. The chart in the middle shows poll results for August and September. Here’s what struck me: the average Sept. result look like Gallup’s Aug. result. So, perhaps Gallop’s methodology just caught a real trend earlier? I have no other evidence that’s the case, but it would be an interesting lead for someone to pursue.
Half-Canadian,
It’s possible that more people are talking about Moveon.org’s ad, however, less swing voters will be actually reading it than if they had placed ads in regional swing state newspapers. It seems to me that relying on a word-of-mouth campaign to rebut polling data that shows your weaknesses with such short time left is dumb. I don’t care it’s just an observation.
And for the record Soros is an atheist and diehard socialist whose only gods are money and power. Sure Soros only wants to create a more just world and society. It’s just that he wants to be in charge of it too, because only Soros and those who believe as he does in the incredibly unending promise of socialism (which has a dismal record of failure and an unfathomable number of bodies in its wake) knows what’s best for the rest of us. If we are just willing to submit to his wisdom, ditch the Constitution of the United States including the Bill of Rights, ignore American sovereignty then in time then we will see the truth, be set truly free, and reap the rewards of his glorius vision. (Call it the Cult of The Open Society.) Hmmm…I guess some socialists are just more equal than most socialists (shades of Animal Farm, creepy ain’t it). I mean really someone attempting to influence a democratically held election with his massive wealth is just, well, so un-Democrat like. What a bunch of propagandizing hypocrites. I say to hell with George Soros and his ilk! Live Free or Die MFs!!
later comrades.