Post/ABC on NSA Records – Part II

Divergent Polls Legacy blog posts Polls in the News President Bush

As expected, Washington Post and ABC News continued conducting interviews over the weekend and released complete results Tuesday afternoon.  The release last Friday morning was based on the first 505 interviews — the complete sample consisted of 1,103 adults.  The overall survey has much bad news for President Bush and the Republicans, as reported separately by the Dan Balz and Rich Morin of the Post and Gary Langer of ABC News.  However, the summary of results put out by the Washington Post includes new data on the NSA questions we have been puzzling over for the last few days. 

One thing is clear:  The new results obtained from Friday through Saturday night are virtually identical to the original Thursday night sample. The latest release from the Post (excerpted below) includes two lines of data for each question:  The first line, labeled 5/11, shows the now controversial results from the first night of interviewing.  The second line, labeled 5/15, shows the data collected from Friday through Monday night.  As the tables show, the results differ by at most a percentage point or two:

The Post summary also breaks out results for party, ideology, vote registration and religion so that we can compare the Thursday night interviews to those collected since.  There are a few small differences — the Thursday night interviews have slightly more respondents reporting an income over $50,000 annually, for example — but none that can explain why the Post/ABC results on the NSA records questions differ so much from the subsequent poll questions asked by Newsweek and USAToday/Gallup (see previous discussion of these differences here and here). 

The bottom line:  These new results help rule out two widely floated hypotheses for the discrepancies.  One was that “views changed that much in one day” (as Editor and Publisher speculated Saturday).  The new results show virtually no change in the Post/ABC results after Thursday.  Another theory was a skewed sample resulting from the relatively small number of interviews conducted in a single evening.  The new results show that on party, ideology and religion the Thursday night interviews are well within sampling error of those conducted since. 

So with respect to the questions on the NSA phone records database, these new data suggest that the differences between the Post/ABC survey were mostly about question wording and question order. 

Mark Blumenthal

Mark Blumenthal is political pollster with deep and varied experience across survey research, campaigns, and media. The original "Mystery Pollster" and co-creator of Pollster.com, he explains complex concepts to a multitude of audiences and how data informs politics and decision-making. A researcher and consultant who crafts effective questions and identifies innovative solutions to deliver results. An award winning political journalist who brings insights and crafts compelling narratives from chaotic data.