Mobile Phones

Legacy blog posts Mobile phones Sampling Issues

Arianna Huffington, a frequent critic of political polling, is weighing in once again with an indictment of the "downright dangerous impact that polls are having on our democracy," including an assertion about mobile phones that has gotten a lot of attention lately (here, here and here): 

Pollsters never call cell phones – of which there are now close to 170 million. And even though most cell phone users also have a hard line, a growing number don’t – especially young people, an underpolled and hard-to-gauge demographic that could easily turn out to be the margin of difference in this year’s race.

Is she right? 

National surveys draw samples of randomly generated telephone numbers, a technique with the power to reach any working wired telephone (or "landline") in the United States.   Ariana is right that this method excludes mobile phone numbers* and, obviously, cannot reach those who lack telephone service altogether.  If the pool of excluded respondents is large and different from those covered by the sample, the results of the survey could suffer from what survey methodologists call "coverage error."  Are polls suffering from coverage error this year?  That is a more difficult question. 

[*Why do pollsters exclude mobile phone numbers from random digit dial samples?  See my note in the comments section]

For the last 20 or so years, telephone surveys have excluded the roughly 5% of U.S. households without any form of home phone service.  Those who lack phone service are disproportionately younger, non-white, and lower income, but their numbers are small, they vote at much lower rates than other adults, and pollsters typically weight by age, race and income, so the impact on political polling has been negligible. 

However, the willingness of some to disconnect their landlines in favor of mobile phones threatens to increase the size of those missed by telephone samples significantly.  So let’s try to answer three questions: 

1) What percentage of households are wireless only?
2) How are wireless households different?
3) Given the answers to the first two questions, how likely are these differences to affect political polls this year? 

What percentage of households are wireless only?

The best data on this issue come from the enormous, high response rate surveys conducted in-person (rather than over the telephone) by government agencies like the Census, the Bureau of Labor Statistics (BLS) and Centers for Disease Control (CDC). 

  • In the first quarter of 2003, the Consumer Expenditure Survey (BLS) estimated wireless only households at 4.3%, having risen steadily from 0.8% in just two years (n=5,000-8,000 per quarter).
  • In the first nine months of 2003, the CDC’s National Health Interview Survey (n=23,372) showed 3.6% of U.S. Civilian Households had wireless service but no landline.  They also estimated that 3.0% of all adults lived in such households.
  • In February of 2004, the Current Population Study (CPS) conducted by BLS and the Census put the number of wireless only households at 6.0% (n=34,219).

How different are wireless only adults?

Wireless-only adults are certainly younger than other adults.  The CDC study reported that 6.8% of 18-24 year olds live in wireless only households, compared to much smaller share of those aged 45-64 (1.6%) or 65+ (0.5%). 

But beware of concluding that all of the missing respondents are college age.  In fact, less than a third (29%) of wireless-only adults are 18-24 – most are either age 25-44 (52%) and many are 45 or older (19%).

Still, if not exclusively college age, wireless only adults are predominantly under age 45 (81%).  They also tend to live in large metropolitan areas (82%), earn less than $40,000 annually (66%) and rent rather than own a home (62%; the comparable percentages for adults with a landline are 51% age 18-44, 73% metro area, 39% <$40K and 24% renter). 

We do not have data on the political attitudes of wireless only adults, but their demographic profile suggests a Democratic skew.  In the latest AP-IPSOS survey, for example, John Kerry leads President Bush by a wide margin among renters (68% to 29%) and those with incomes under $40,000 (60% to 37%).   AP-IPSOS had the race even among younger voters, but Kerry is ahead among 18-30 year olds in the recent surveys by CBS (56% to 35%) and the Washington Post (50% to 43% on Oct. 11-13)

We also lack data on the likely turnout of wireless only adults, although their demographic profile suggests they have been much less likely to vote in past elections.  According the Current Population Study by the U.S. Census, turnout in the 2000 election was much lower among adults earning less than $35,000 (51%) than those earning over $35,000 (70%), lower among 18-24 year old (36%) than among those over 35 (66%) and lower among renters (44%) than home owners (65%).

How likely are these differences to cause error in the political polls?

We could calculate the "coverage error" that results from excluding wireless-only adults from political polls if we knew two things:  (1) How the vote preferences of wireless only adults differ from those with working landlines and (2) the percentage of all likely voters with only wireless service.  Unfortunately, both numbers are unknown. 

Still, assume for the sake of argument that wireless adults are 5% of the electorate, that a survey of wired households shows a 48%-48% tie and that the missing wireless-only voters prefer John Kerry by a 20-point margin (58% to 38% – a pure but plausible guess based on the numbers for renters, low income, etc).  If we were able to include the wireless only adults, it would change the overall preference by only one point – Kerry would lead 48.5% to 47.5%.

Keep in mind that two factors will work to reduce this small potential error:  Wireless-only voters are likely to turn out at a lesser rate than those with wired phones, and pollsters typically weight to make up for overall differences in gender, age, race and education.

Of course, that’s this year.  Things could be very different next time.  A recent study by the market research firm In-Stat/MDR estimates wireless only households growing to 30% in 2008.  If that estimate holds, telephone polls will face enormous challenges in the very near future. 

(Offline sources on the jump)

[Continue with More on Mobile Phones]

Offline sources: 

Julian V. Luke, Stephen J. Blumberg, and Marcie L. Cynamon. "The Prevalence of Wireless Substitution." Paper presented at the Annual Meeting of the American Association for Public Opinion Research, May 15, 2004, in Phoenix Arizona

Clyde Tucker, Brian Meekins and J. Michael Brick.  "Household Telephone Service and Usage Patterns in the United States in 2004."  Paper presented at the Annual Meeting of the American Association for Public Opinion Research, May 15, 2004, in Phoenix Arizona

See also the discussion of cell phones by the ABC News Polling Unit

 

Mark Blumenthal

Mark Blumenthal is political pollster with deep and varied experience across survey research, campaigns, and media. The original "Mystery Pollster" and co-creator of Pollster.com, he explains complex concepts to a multitude of audiences and how data informs politics and decision-making. A researcher and consultant who crafts effective questions and identifies innovative solutions to deliver results. An award winning political journalist who brings insights and crafts compelling narratives from chaotic data.