Katrina: Evacuees & Random Samples

Today we have two different perspectives on "random sampling" in gauging reactions to the disaster from Katrina evacuees living in shelters in Texas.  One comes from a true random sample survey conducted by the Washington Post, the Henry J. Kaiser Family Foundation and the Harvard School of Public Health before the last night’s televised address by President Bush (article here, full results here).  The second comes from the interviews with Katrina evacuees just outside the Houston Astrodome done live by ABC’s Dean Reynolds just after Bush’s speech.   

Let’s start with the ABC interviews.  The right-wing of the blogosphere is working itself into tizzy over the surprisingly positive reactions from six African-American evacuees interviewed in the parking lot of the Houston Astrodome.   The Media Research Center’s Brent Baker provides a concise summary (as well as video excerpts and a full transcript,
Ian Schwarz also has a video clip):

Instead of denouncing Bush and blaming him for their plight, they praised Bush and blamed local officials. Reynolds asked Connie London: "Did you harbor any anger toward the President because of the slow federal response?" She rejected the premise: "No, none whatsoever, because I feel like our city and our state government should have been there before the federal government was called in." She pointed out: "They had RTA buses, Greyhound buses, school buses, that was just sitting there going under water when they could have been evacuating people." 

Not one of the six people interviewed on camera had a bad word for Bush — despite Reynolds’ best efforts. . .

Reynolds pressed another woman: "Did you feel that the President was sincere tonight?" She affirmed: "Yes, he was." Reynolds soon wondered who they held culpable for the levee breaks. Unlike the national media, London did not blame supposed Bush-mandated budget cuts: "They’ve been allocated federal funds to fix the levee system, and it never got done. I fault the mayor of our city personally. I really do."

[Emphasis added].

Now, again, the Post/Kaiser/Harvard researchers fielded their survey of 680 evacuees living in Houston shelters before Bush’s speech.  However, the attitudes of their respondents toward Bush and the local officials were quite different: 

• Only 15% approved of "the way George W. Bush handled the situation caused by Hurricane Katrina;" 70% disapproved.

• Approval ratings for the governor and mayor were also low, but not as bad as for Bush:  27% approve and 58% disapprove of Governor Blanco’s handling of the Hurricane; 33% approve and 53% disapprove of Mayor Nagin’s post Hurricane performance. 

• Evacuees were largely split on who they blamed most.  Roughly the same number blamed the federal government (28%) as blamed either the state of Louisiana (12%) or the city of New Orleans (19%). 

It is worth noting that the survey also shows that ABC interviewee Connie London is not alone in her feeling that, "here in Texas, they have been truly good to us:"

• Almost all rate the "conditions at their shelter" as excellent (43%) or good (42%).  Only 10% rate them not so good, and only 3% rate them as poor.

• 82% said they felt "grateful," and 87% that they were "hopeful" about their future.

Now let’s consider the methodology of the Washington Post/Kaiser/Harvard study, because at Mystery Pollster, that’s what we do.  It was truly a "scientific" random sample survey of evacuees living in Houston area shelters.  As this very complete methodology summary tells us, interviews were conducted in person, from September 10 to12, by professional Houston based interviewers employed by ICR, a survey research company that regularly conducts face-to-face interviews.

Interviewers selected respondents suing a systematic and random procedure that involved choosing every "nth" evacuee.  In other words, they were given a random number — let’s say 20 — and told to interview every 20th person.  Where evacuees "had limited mobility or were non-mobile" (e.g. sitting on cots or chairs), the interview counted off every "nth" cot or chair and attempte to interview that person where they sat.  Where evacuees were walking around, they stood in a fixed position and attempted to interview every "nth" person that walked by.  The response rate was quite high: 90% of those approached agreed to be interviewed. 

So, MP wonders, how did ABC choose the women they interviewed last night? Obviously, they were not attempting a scientific survey, but did ABC producers select evacuees more or less at random before the speech, asking them to watch and comment afterward?  Or did evacuees gather on their own to watch not knowing Reynolds would interview them later?  Or did they gather near the cameras in hopes of volunteering to be interview subjects? 

MP also wonders, aside from the ABC interviews, just how many evacuees in the Astrodome chose to watch last night’s speech?  Moreover, the cynic in MP (and, perhaps, the Democratic partisan) wonders if someone might have tried to coax Bush friendly evacuees to be first in line to be interviewed by ABC’s cameras.   

MP hopes an enterprising reporter on scene will try to look into these questions.  It might teach all a lesson or two about the advantages of random sampling . . .  and maybe more . . .