Zogby, Wal-Mart & Ethics

Last week, syndicated columnist Joel Mowbray took pollster John Zogby to task in the Washington Times for "serious ethical issues which call into question the integrity of the much-ballyhooed survey" on the retailer Wal-Mart.  Zogby’s poll, which had been commissioned by a labor-backed group known as WakeUpWalMart.com, reported among other things that 56 percent of Americans agreed with the statement, "Wal-Mart is bad for America."   While MP has never been a big fan of Zogby’s methodology, in this case at least, the criticism seems a bit unfair. 

Some background:  MP will leave for another day an in-depth look at Mr. Zogby’s colorful history and unique approach to survey methods.  For now, suffice it to say that MP has long been skeptical of Zogby’s practices of interviewing during daytime hours and sampling from listed telephone directories, adjusting samples by party to match past exit polls or "his judgment of ‘what is happening on the ground.’" The amazing last minute "corrections" that brought his results for the 2004 New Hampshire and Georgia primaries into line with those of other pollsters and the ill-fated Kerry victory projection made at 5:00 p.m. EST on Election Day (a projection which neatly coincided the results of exit polls that had been widely leaked earlier that afternoon) did not exactly bolster my confidence (more details here, here and here; use CTRL-F to find "Zogby"). 

Now before delving into the specifics of Joel Mowbray’s argument, it is worth noting that Zogby has also been criticized in the past for failing to disclose the identity of clients who paid for results released into the public domain.  For example, consider this passage from last year’s classic Zogby profile by the New Yorker‘s Larissa MacFarquhar:

Zogby is also willing to guard the identity of his clients if they want to remain anonymous, while AAPOR [the American Association for Public Opinion Research] insists that an ethical pollster will always make the sponsor known if a poll’s results are published, so that the public can judge whether or not he had a vested interest in the outcome. Zogby argues, reasonably enough, that if his methods are transparent and correct, the sponsor should be irrelevant. He tells clients that if they wish to publish any part of a survey they have to publish the whole thing, including the questions. But if a client wishes to publish only part of a survey-the results that he likes-Zogby offers another, rather underhanded option: he will conduct a second survey, using only the questions from the first one whose answers served the client’s purposes, and the client can publish that and keep the first poll private.

In the case of the Wal-Mart poll, however, disclosure of the client’s identity was not an issue. The various press releases and the articles by the Associated Press and Reuters all clearly stated that the poll was "commissioned" or "released" by WakeUpWalMart.com.

Yes, Zogby was hired by the interest group in question, but he is certainly not the first pollster to work on behalf of a private interest.  Results from polls conducted for political campaigns and interest groups are released almost every day.  The pollsters that conduct that research — including  MP — operate under essentially the same ethical guidelines as all survey researchers.  At the risk of oversimplifying AAPOR’s Code of Professional Ethics and Practices, those obligations include taking "all reasonable steps" to measure public opinion fairly and accurately and then, if releasing the results publicly, describing the results honestly and disclosing both the sponsor and "essential" details about methodology.  Full disclosure of sponsorship allows the public to take privately funded research with the appropriate grain of salt. 

So if in this case Zogby fully disclosed the sponsor, what was Mowbray’s beef?  He alleges two specific failings:

1)  Before conducting this poll, Zogby had done other work for anti-Wal-Mart interests:

In recent years, Mr. Zogby has pocketed roughly $90,000 to serve as an expert witness for individuals suing Wal-Mart, according to testimony he gave in a deposition last year in an Arizona case. Nowhere is Mr. Zogby’s prior work on behalf of plaintiffs mentioned in the press release announcing the poll results.

Had Zogby hidden WakeUpWalMart’s sponsorship of his poll, Mobrey would have had a point, but I am not sure what the reader gains from knowing that Zogby also did other work for essentially the same client.  Mowbray claims that "what no journalist would have known without digging is that Mr. Zogby cannot be considered an objective third-party when it comes to Wal-Mart."  If payments from an anti-Wal-Mart interest compromise a pollster’s "objectivity," did journalists really need to dig farther than the press releases disclosing who sponsored the poll?   A researcher’s prior work may be relevant, but asking every pollster to disclose all work done for the same interest that is paying for the poll seems over the top.  It is certainly not required by the AAPOR Code. 

2)  Zogby’s client wrote the first draft of the press release on the results: 

Though Mr. Zogby insisted that being paid tens of thousands by people suing the retailer did not compromise his objectivity, he was careful to note that the press release announcing the poll results was drafted by the client, Big Labor-backed WakeUpWalMart.com. But when reached the following morning, Mr. Zogby conceded that his staff "heavily edited" the release and even posted it on the group’s Web site and put the release out over its wire.

MP will grant that letting a client draft the press release that goes out over the pollster’s letterhead is not something that most in our field would consider a "best practice." However, from Mowbray’s account, it is not clear that is what happened here.  The press release posted on the WakeUpWalMart site is different than the one that appears on Zogby.com.  Which release did Zogby attempt to distance himself from?  Which did Zogby’s staff "heavily edit?"  I know of no pollster code of ethics – including the AAPOR code – that would prohibit a client from putting out a press release put out on their own letterhead.

There is an important ethical question raised here, but one not quite as dramatic nor as theoretically damaging to this particular poll.   It involves Zogby’s habit of conducting polls for both mainstream media outlets and political interest groups.    Mowbray is probably right to assume that WakeUpWalMart hired Zogby because of his "extra panache and an air of instant credibility of his reputation."  That reputation comes as result of his high profile media polls.  Consumers of Zogby’s media-sponsored surveys have the right to ask whether his work for private interests clouds his objectivity.  Needless to say, Zogby argues vehemently in his response to Mobray that it does not. 

This is a conversation worth having, but we should be fair to John Zogby.  He is certainly not the only "public" pollster whose client list also includes interest groups or partisans.  His approach may be a bit more flamboyant and unorthodox — to put it mildly — but on this general ethical issue he is not alone. 

Interest disclosed:  I conduct "internal" polls for Democratic candidates and interest groups, some of which get released into the public domain.  I have never conducted a survey for a news organization. 

LATE UPDATE (12/15):  The Pew Research Center just released a survey (report, full results, tables) that probed attitudes on Wal-Mart, including favorable ratings for Wal-Mart and Target that appear to be based on a very similarly worded question.  In the Pew survey, 65% of Americans have a favorable opinion of Wal-Mart, 30% have an unfavorable opinion.   On the Zogby survey, 58% of Americans rated Wal-Mart favorably and 38% unfavorably. 

The Pew and Zogby surveys were more in synch in measuring opinions of Target:  Pew reported a 76% favorable, 14% unfavorable rating.  Zogby reported 73% favorable, 13% unfavorable rating.

The Pew survey had much more to say about perceptions of Wal-Mart:

Nearly every American lives near enough a Wal-Mart to shop there, and 84% say
they have done so in the past year. Praise for the retailer’s low prices, wide
selection and convenience flow freely, and 81% of those with a Wal-Mart nearby
say it is a good place to shop.

Somewhat less glowing, however, are
judgments about Wal-Mart’s effect on communities and the nation as a whole, and
a third of the public (34%) rates it a bad place to work. Overall, 69% of those
familiar with Wal-Mart have a favorable opinion of the company. Still, 31% have
an unfavorable view, which is a considerably higher negative rating than is
accorded to many other major corporations