Finally (for today) I have a surprise and something of a holiday gift from the editors of Public Opinion Quarterly.
As a personal venture, this blog has paid some truly gratifying and utterly unexpected dividends. Topping the list during 2005 was an invitation I received from the editors of Public Opinion Quarterly (POQ) to submit an article for their special issue on the polls, the media and the 2004 campaign. That issue is out this week, and the editors POQ have consented to allow MP readers to access my article — "Toward an Open-Source Methodology: What We Can Learn from the Blogosphere" — free of charge. There are two versions, a PDF replica of the printed article and an HTML version with live links to footnotes and sources.
For years I have turned to POQ for the latest gold standard research and commentary on survey methodology from the most noted authorities in the field. This issue is no exception. The authors of this special issue include names that should be familiar to regular MP readers including Frank Newport of Gallup, Kathleen Hall Jamieson of the Annenberg School, Kathy Frankovic of CBS and Gary Langer of ABC, as well as others from the worlds of media and academia whose names may not be as familiar but are equally prominent, if not more so.
The great honor of the inclusion in this special issue is not lost on MP, nor should it be on his readers. It is a byproduct of the changes that the Internet has produced and of the growing influence of all of you who read and write blogs. Thank you — all of you — for that.
In their introduction, special issue editors Lawrence Jacobs and Robert Shapiro explain this issue’s purpose:
This special issue of Public Opinion Quarterly assembles a wide range of perspectives to critically evaluate polling and American politics in recent election campaigns and the 2004 presidential race in particular. This volume, which follows the 1984 special issue on polls and the media, addresses the major questions that have been raised about election polls today. How accurate are they? Are candidates and government officials poll-hooked addicts who slavishly follow the preferences or whims of the public? Polls and politicians are two elements of a complex process of political strategy and communication. The press is also a critical-or the critical-mediating element. What is the impact of the old and new media’s large and ever-growing attention to polling? Does the press serve as a watchdog that ferrets out flawed polls and facilitates strong responsiveness to the public’s concern? Or does it do something quite different?
At the urging of the editors, my article provides a review of Internet and blog commentary on polling methodology during the 2004 campaign as well as a review of the emerging methodologies such as automated telephone surveys and Internet polling. Jacobs and Shapiro also nicely summarize my underlying thesis:
[Blumenthal] also suggests that the Internet may help make it possible to widen public dissemination of polling results, conduct experiments with surveys, and facilitate a broader and richer scrutiny of polling. New survey methods and modes of public discussion raise questions and challenges for future campaigns.
Again, the editors of POQ have granted permission to MP readers to download my article for free. While online access to the rest of this special edition is limited to subscribers and AAPOR members, I encourage readers to visit the special issue page and, view the free abstracts and consider ordering a copy of the special issues (a late stocking stuffer perhaps?). The single issue price is just $10.
If you would like to order the complete issue, just follow this link. If you want to order online, go to Single Issues – Personal – Print and click the link for "add to basket." When the "My Basket" window comes up, select Volume 69, Issue 5. Ignore the fact that price is listed in British pounds. I am told that it will display the price in US dollars (or whatever the correct currency is) "at the checkout point" once you enter your payment information.
If you prefer to order by mail or by fax, you also have the option of starting at the order page, going to Single Issues – Personal – Print and click the link for "$10" which will allow you to download an order form.
Thanks again to the editors of Public Opinion Quarterly and to all of you for supporting this blog.
UPDATE (12/24): As the article explains, I use the term “open source” as a metaphor to describe the notion of full disclosure of all methodological details, analogous to the way open-source software makes source code available. Again, as noted in the article, most academic opinion surveys already operate under what is essentially an “open source” model.
However, unlike the idea that Jeff Jarvis floated in April (as he points out in the comments section below), I am not proposing a methodology that would open up survey participation (as a respondent) to all comers. As commenter Andrew Tyndall also points out in the comments, “opt-in polls are useless as statistical samples.” Open participation would also raise some profound issues of respondent confidentiality. However, both Jarvis and Tyndall are asking some provocative questions that are worth taking up in the New Year.
Ah…so many good topics, so little time…
From a far less knowledgable perspective, I wished for open-source polling back in April here:
http://www.buzzmachine.com/archives/2005_04_01.html#009389
I’m glad an expert is working with the people who know what they’re doing on this.
At mediachannel.org, we conducted a series of opt-in surveys, called the Citizens Debate Scorecard, after the Campaign 2004 Presidential debates.
A crude way in which we tried to deal with the problem of the non-representative composition of the self-selecting panel was a simple segmentation of the results, contrasting the views of Bush supporters with Kerry supporters.
Crude as this was, it revealed overnight findings that Kerry won the first debate and that the “moral values” of the candidates was a crucial dimension on which the two groups of partisans diverged. This is the press release we produced in response to the exit polls which vindicated that finding: http://www.mediachannel.org/views/dissector/affalert287.shtml.
Obviously opt-in polls are useless as statistical samples. I do believe, however, they have the potential as an open-source, inexpensive, distributed, opt-in format to contrast various segments of an overall population, even if the members of those segments are non-representative partisans.
Why don’t you allow any critical comments?
You have a penchant for over analyzing everything to the point where you miss the most simple truths.
Mo: Public Opinion Quartely is an academic journal published by Oxford University Press. They control the online format, not I. Obviously you are welcome to leave critical comments here.
And Happy Holidays to you too.
“Obviously opt-in polls are useless as statistical samples.”
Only because we don’t know the probability that an “opt-in’er” participates in a survey, as compared to any other respondent–that’s why we do sampling. I suspect there’s a BIG research question here, which might make several dozen Ph.D.s on its way to being answered.
Can Blogs Improve Polling?
Mark Blumenthal (a.k.a. Mystery Pollster) has a piece in Public Opinion Quarterly on “the lessons to be learned from the spirit of innovation and openness of the Internet in evaluating new survey methods such as automated polls and those conducted over…
Mark,
Great article. Moveon’s ad and Dowd’s email were meant not to attack the pollster, but to attack the conventional wisdom generated by the poll. I suspect that is so in the blogosphere as well.
You might want to consider recommending changes in how polling is covered in major papers, ie. push for less sensationalism and fewer unwarranted conclusions.
I’ve read with pleasure. Maybe it’s offtopic, but i just wanted to say, that it’s really interesting to read everything this with the comments… You discuss here a lot of interesting things on different useful themes. Thanks for that =)
“Obviously opt-in polls are useless as statistical samples”
While as first blush this may seem obvious, with a little more thought it seems that it may not always be the case. On many public policy topics the conversation is not necessarily set by the popular opinion but rather by those more active in pursuing a specific policy. An opt-in poll could well be a way to gage the opinion of those more likey to actually be a part of framing and directing the discussion of the topic. For example, there is a known and marked difference in responses between “registered” and “likely” voters. The inherent ability of an opt-in poll to attract greater participation from those more interested in the sublect material may have utility in both estimating the likelihood of voting as well as the positions of those likely voters.
Is traditional public opinion polling dying a slow death?
Angus Reid talks to Canadian Business about federal politics, refusal rates and web-enabled interview panels. Oh, and how traditional market research organizations, so dependent upon phone interviews, are dinosaurs. One quote from the reporter: “In the…
Is traditional public opinion polling dying a slow death?
Angus Reid talks to Canadian Business about federal politics, refusal rates and web-enabled interview panels. Oh, and how traditional market research organizations, so dependent upon phone interviews, are dinosaurs. One quote from the reporter: “In the…
The future of public opinion polls will be the Internet-based polls. The sheer number of participants will compensate the sampling issues. One poll, two polls … many such internet wiki style polls will average out the biases… The emergence of the so-called web 2.0 applications encourages people to participate, and the collective intelligence of the crowd will surprise a lot of the traditional polling agencies. See such applications at http://www.wipoll.com and http://www.dpoll.com