Deaths in Iraq

There's been a lot of really, really bad commentary on the Lancet study in the blogosphere. Apparently everyone becomes a self-described expert in statistics when the statistics disagree with your previously held beliefs. At least we have Timothy Lambert and Daniel Davies to set the record straight. Lambert does an excellent job (as usual) of debunking some of the most common criticisms, and links to other defenses of the study, while Davies provides a thorough look at other common criticisms.

I really can't help but wonder why so many people who've clearly never taken a statistics course, and some who obviously haven't even read the paper, feel qualified to comment on its methodology and findings. Even those who do have some statistical knowledge (or at least should), seem to forget everything they've learned (or should have learned) when commenting on the paper. Thus, John Ray, who has published some admittedly unimportant psychological research that used statistics, writes:

Nobody, however, seems to have commented on the fact that the findings were a product of cluster sampling. The major fault I see with the study is that estimating low-incidence phenomena via cluster samples is inherently dodgy. I have had many findings derived from cluster samples reported in the academic journals so I know a little bit about it. You just have to get one or two clusters being a-typical (either by chance or intentionally) to arrive at totally distorted results. Basing such an important conclusion on a sample-size of only 33 is really quite ludicrous. I have used as few as 10 clusters in some of my surveys but I was concerned only to find whether some effect existed at all. I was not trying to estimate it precisely.

I'm going to be charitable and assume that it's been so long since Ray has published anything using cluster sampling that he doesn't remember how it works, and what sort of estimation errors one is likely to get using it. If that's the case (instead of Ray being so upset that the study disagrees with his personal beliefs about Iraq that he is rendered "temporarily" stupid), I hope that Ray and others will read Davies' post, because he explains where Ray goes wrong. He writes:

Although sampling textbooks warn against the cluster methodology in cases like this, they are very clear about the fact that the reason why it is risky is that it carries a very significant danger of underestimating the rare effects, not overestimating them.

Hopefully, further research will be done on the effects of the war in Iraq on Iraqis. This study is really just a preliminary one, though its findings should be taken seriously. However, as the results of future studies are likely to further upset conservatives, I can't imagine their responses will be any different, no matter how many studies demonstrate that Iraqis are worse, rather than better off, in post-war Iraq.