Party identification, the question that measures whether Americans consider themselves Democrats, Republicans or Independents, got a lot of attention in the 2004 campaign. Some pollsters choose to weight by party ID — that is they statistically adjust the number of self-identified Republicans and Democrats in their samples — but most public pollsters do not. Last week, MP attended a seminar on pre-election polling in 2004, sponsored by the DC chapter of AAPOR (the American Association for Public Opinion Research). Not surprisingly, the party identification debate was a central topic.
After the seminar, I posed asked a question of the participants – Frank Newport of the Gallup Organization, Scott Keeter of the Pew Research Center and Mark Schulman of Schulman, Ronca, and Bucuvalas (SRBI), the firm that conducts the Time Magazine poll – about their willingness to disclose party identification and received some truly unexpected responses that are good news for those of us pushing for greater transparency in public polling I want to share those with this week, starting with Gallup’s Editor in Chief Frank Newport.
Some background. At the seminar, Newport said he is in the midst of "zero-basing everything we know about party [identification] in an election year." Newport is preparing a paper on the topic to be delivered at AAPOR’s May conference, and he devoted much of his presentation to a preliminary discussion of the topic.
While Newport has not reached final conclusions, he did reiterate that he considers party identification more a "survey statistic" than a "stable population parameter," meaning that party is a potentially changeable attitude that surveys measure rather than a stable demographic like race or gender. Pew’s Scott Keeter agreed, describing party identification as a measure that "provides useful information about the political climate." MP generally agrees with this philosophy, as explained in these two posts, but like many political data consumers, is often frustrated by the habit of many public pollsters of withholding party identification when releasing results online. So after the seminar, I sent Newport, Keeter and SRBI’s Schulman the following question via email (edited slightly to correct an error in the orignal):
If party identification is a "survey statistic" and not a "stable population parameter" (as Frank Newport put it), if it provides important and useful information on the changing political environment (to paraphrase Scott Keeter — and I agree on both counts), why do your organizations routinely exclude results for party identification from survey releases?
To be more specific, I cannot find results for party identification in the otherwise excellent and comprehensive questionnaire/reports put out for the most recent surveys for Pew, Time/SRBI and Gallup (neither in USAToday nor to paid subscribers on Gallup.com), nor in other such releases from Gallup or Pew for the last six months or so (though Time/SRBI did routinely report party ID results on surveys released last fall).
Now, obviously, I am aware that your organizations have disclosed much party ID data at academic conferences, in selected special reports online (especially this superb summary from Pew last September) or through raw data made available to the Roper Center. I also know from personal experience that your organizations have been very responsive to specific requests for such data — even from your harshest critics. You deserve praise for doing so. But given that you routinely put out long summary documents online, again, why are such specific requests necessary? Why not include the results for party ID in each release?
I also do not want to pick on Gallup, Pew and Time/SRBI alone. Other organizations that — as far as I can tell — routinely omit party ID from releases include the American Research Group, Fox/Opinion Dynamics, Harris, LA Times, Marist, Mason Dixon, Newsweek, Quinnipiac and Washington Post/ABC News (although ABC News occasionally reports on the partisanship of its samples in written releases). Although Zogby and Rasumussen weight by party, they do not routinely disclose their weight targets in their releases or on their web sites.
The noteworthy exceptions — those that routinely include party ID results in online releases — are CBS/New York Times, NBC/Wall Street Journal, AP/IPSOS (albeit to paid subscribers only) and SurveyUSA.
Here is the response from Gallup’s Frank Newport:
As you can tell from that meeting and my forthcoming paper at the national AAPOR conference in May, party ID is a matter of significant interest to us here at Gallup. I think there’s been a good deal of misinformation and misunderstanding about what party ID measures and what its importance is — as you gathered from the paper I gave in Washington. Your commentary on your site has been very helpful in clarifying this area to those who don’t know a lot about it.
Gallup has always been happy to give party ID figures for any survey to anyone who wants them. In addition, we write articles and reviews about party ID when we think the aggregated trends are showing something significant (e.g., http://www.gallup.com/poll/content/login.aspx?ci=14347 [subscription required]) , and have discussed party ID and its implications at great length in our Gallup editors’ blog [free to all] on our website.As far as I know, Gallup has no history over the last 70 years of routinely posting the party ID composition of each survey we conduct, just as we routinely don’t report ideology and a lot of other measures regularly asked in each survey. As noted, we send the party ID composition percentages to anyone who is interested (actually, we really don’t get that many requests for them). But since this seems to be an area in which there is perhaps bourgeoning interest, we’ll probably start posting them on our website for each survey, along with rolling trends and some explanations of how Gallup measures party ID and what it’s significance is [emphasis added].
On behalf political survey data consumers everywhere, let me way, thank you Dr. Newport! If Gallup, the most important brand name in survey research, is willing to take this step, other survey organizations are likely will follow your lead to greater transparency.
Next up: Responses from Pew’s Scott Keeter and SRBI’s Mark Schulman.
“meaning that party is a potentially changeable attitude that surveys measure rather than a stable demographic like race or gender.”
This is an interesting theory and one where current political science research actually intersects survey methodology. Personally I think Green Palmquist and Schickler make the most convincing argument in “Partisan Hearts and Minds” that party identification is VERY stable. Although they cite international data, the American South is probably the best example for stability. The point being that its not surprising that Southern voters vote Republican, but it is surprising that it took so long.