On “Oversamples” and the AP-IPSOS MLK Holiday Poll

Legacy blog posts Polls in the News

The Associated Press, in its usual partnership with IPSOS, released a survey yesterday (story, results & methodology) on attitudes on today’s holiday honoring Martin Luther King, Jr. and his dream of racial equality:

Three-quarters of those surveyed say there has been significant progress on achieving King’s dream. But only 66 percent of blacks felt that way…

Only 23 percent of respondents say they will do anything to commemorate the national holiday that took effect in 1986 after a lengthy campaign in Congress to honor King. A solid majority of blacks, 60 percent, say they will get involved in holiday activities.

If you do nothing else to commemorate the holiday, read all of Will Lester’s AP story on the poll and its implications. 

To allow for a sufficiently large sample of African Americans, AP-IPSOS conducted an "oversample." That term often confuses readers and deserves a bit more explanation.  The AP methodology summary tells us that they interviewed 1,242 adults, including 312 blacks and adds this sentence: 

The total sample includes an oversample of blacks, completed in part by interviewing people who self-identified as black in previous Ipsos telephone surveys.

Do you find that explanation a bit confusing?  If so, here’s how it works: AP-IPSOS typically conducts a representative sample of about 1,000 adults.  I am going to assume they did the same on this poll, and guess at the other numbers involved.  A base sample of 1,000 would have included a representative sample of African Americans.  In this case it appears to have been roughly 70 interviews.**  Media pollsters typically shy away from reporting on subgroups that small since they have very large statistical sampling error (for n=70 it would be at least  11%).   So they sometimes conduct an "oversample" of additional interviews among respondents in the subgroup of interest to increase the sample size.  In this case AP-IPSOS conducted (again, my guess) roughly 242 additional interviews among "people how self identified as black in previous IPSOS telephone surveys."

In order to report results among all adults, the pollsters weight the total pool of interviews (n=1,242 in this case) so that the proportion of African Americans in the weighted sample matches Census estimates for the U.S. Population (roughly 11%).  This is an important point: Results reported for "all adults" in this AP-IPSOS poll come from a statistically adjusted sample that "weights down" the African-American oversample to its appropriate size in the U.S. adult population. 

Most pollsters will also use Census estimates to weight on other demographic factors to correct for small differences resulting from sampling error or non-response bias.  In this case, AP-IPSOS tells us that they also weighted by other factors "such as" age, sex, region and income."

**Again, if I’m guessing right, the AP-IPSOS base sample included roughly 70 interviews, or 7%, among African Americans.  A perfectly representative sample of U.S. adults would have been roughly 11% African-American.  Why the (apparent) difference?  Response rates tend to be lower in urban areas, and as a result, unweighted national samples typically under-represent African Americans.  As noted here previously, most national pollsters typically weight African-Americans up slightly to match US Census estimates.

Mark Blumenthal

Mark Blumenthal is political pollster with deep and varied experience across survey research, campaigns, and media. The original "Mystery Pollster" and co-creator of Pollster.com, he explains complex concepts to a multitude of audiences and how data informs politics and decision-making. A researcher and consultant who crafts effective questions and identifies innovative solutions to deliver results. An award winning political journalist who brings insights and crafts compelling narratives from chaotic data.