Last week’s "Numbers Guy" column by Carl Bialik of the Wall Street Journal Online looked at another online survey about an online activity, in this case about online alcohol sales to teenagers. Bialik noticed something in the release that the other outlets should have at least checked. The "online" sample was not drawn from all teenagers, but rather from teenagers who had volunteered to participate in online surveys. It was a non-random online survey of an online activity, something with the potential to create a significant bias.
I will let Bialik take it from there:
People who agree to participate in online surveys are, by definition, Internet users, something that not all teens are. (Also, people who actually take the time to complete such surveys may be more likely to be active, or heavy, Internet users.) It’s safe to say that kids who use the Internet regularly are more likely to shop online than those who don’t. Teenage Research Unlimited told me it weighted the survey results to adjust for age, sex, ethnicity and geography of respondents, but had no way to adjust for degree of Internet usage.
Regardless, the survey found that, after weighting, just 2.1% of the 1,001 respondents bought alcohol online — compared with 56% who had consumed alcohol. Making the questionable assumption that their sample was representative of all Americans aged 14 to 20 with access to the Internet — and not just those with the time and inclination to participate in online surveys — the researchers concluded that 551,000 were buying alcohol online.
Bialik goes on to raise even more fundamental problems with the release from the survey sponsor — the Wine and Spirits Wholesalers of America — a group that according to Bialik, "has long fought efforts to expand online sales of alcohol." For one, their headline claims that "Millions of Kids Buy Internet Alcohol," while even the questionable survey estimate adds up to just 551,000.
One point not raised in Bialik’s excellent piece is that the survey reports a margin of error ("plus or minus three percentage points"). The margin of error is a measure of random sampling error, which applies only where there is a random sample. This survey was based on a sample drawn from a volunteer panel, not a random sample survey. I have raised this issue before (here and here). Yes, random sample surveys face challenges of their own, but if a sound statistical basis exists for reporting a "margin of error" for non-probability samples, I am not yet aware of it.
No one in US government officials got the blame on September 11, 2001. George W. Bush should take responsibility on the event. He tried every way possible to cover up. More and more people know the truth of the event. He is guilty 100% on his crime on September 11, 2001. He is guilty 100% on lying to congress to start the war in Iraq. He is guilty 100% for his signing statements on the the laws that he signed. He is 100% guilty on foreign prisoner torture at Guantanamo bay prison. He is guilty 100% on wiretapping telephone conversation of American people. He is a war criminal against humanity and international law.