So When Is An Attitude Really An Attitude?

Legacy blog posts Measurement Issues Polls in the News

I want to add a few quick follow-up thoughts on yesterday’s topic:  The large differences across four national surveys in the number who offered an opinion on whether Samuel Alito should be confirmed as a Supreme Court justice.   What should we make of the roughly 50% of Americans who will offer an opinion on Alito’s confirmation when pushed by an interviewer but not when offered the option “or can’t you say?”   Survey researchers have long debated this issue, and the answer is not as easy or obvious as it may appear. 

One theory says that those “pushable” respondents have no real opinion or attitude on the Alito nomination but just make an answer up on the spot under the pressure of the interview.   Taken to an extreme, this theory would have respondents just making up meaningless answers at random.   Under this first theory, we would probably want to ignore these “pushed” attitudes. 

Another, and somewhat more enlightened theory argues that some respondents opt for “I can’t say” when offered because they would rather not expend the mental energy necessary to answer the question.  To put it in academic terms, they “satisifice” and take the easy way out rather than expending the “cognitive effort” necessary.  So some of those 50% may have a real attitude about Alito in there somwhere, and the gentle push from the question format or interviewer forces them to think it through.   

Even when respondents form opinions and make them up on the spot, their answers are often drawn from real attitudes.    Thus, when pushed, respondents will use what information and cues the question provides to work their way to an answer.  For example, if you know only that Alito was nominated by President Bush, and you have strong feelings one way or another about President Bush, you may base your answer on those attitudes. 

With that in mind, consider the results to the question of whether Alito should be confirmed when tabulated by party in the table below:

Notice that among Republicans most of the difference between the CBS poll (which offered an explicit “can’t say” option) and the Gallup and ABC/Washington Post polls (which did not) is in the support category.   When pushed, Republicans are supportive by a roughly four-to-one margin.  But independents and Democrats appear to divide about evenly when pushed.  About half support and about half oppose.

So this example does not settle the debate.  When pushed, Republicans appear to draw on their partisanship in forming (or deepening) an opinion on whether Alito should be confirmed.  But it is not at all clear what independents and Democrats are doing when pushed.  Are they drawing on other attitudes?  Are some basing their answer on partisanship while others are just “acquiescing” (see satisficing under the “cognitive psychology” section).   Are they just mentally flipping coins?  It’s hard to say.

Either way, again, the lesson for poll consumers is to watch out when we ask about issues or people that the public are not following closely.   The safest conclusion is that when offering an explicit “don’t know” option boosts that that category by 50 percentage points, the opinions offered by those pushed to an answer are very soft. 

Source links: CBS News poll, Gallup poll, ABC/Washington Post Poll

Mark Blumenthal

Mark Blumenthal is political pollster with deep and varied experience across survey research, campaigns, and media. The original "Mystery Pollster" and co-creator of Pollster.com, he explains complex concepts to a multitude of audiences and how data informs politics and decision-making. A researcher and consultant who crafts effective questions and identifies innovative solutions to deliver results. An award winning political journalist who brings insights and crafts compelling narratives from chaotic data.