Everybody's an Expert: When Cognitive Science Goes Public

I have heard many linguists complain, in Rodney Dangerfield voices, that they can't get no respect. A popular belief among the general public (and even among many scientists) is that just about everybody learns to talk at a pretty young age, so language can't be that hard. All of those alphas, thetas, and X-bar thingies seem pretty silly when you think language is really simple. The same is unfortunately true of cognitive science in general. Try to explain to your grandmother that you study how the visual system recognizes and individuates objects, and she'll probably say something like, "I look at a table, and see it's a table. What don't you understand about that?" In other words, people think, perceive, and speak everyday, and have for as long as they can remember, so they figure they've got a pretty good handle on how it works. As it turns out, many (if not most) of our conscious intuitions about our own thought processes are pretty damn wrong, but that can't be helped. We simply don't have access to the relevant information, so we're essentially creating stories to explain the information we do have. That's pretty much what our conscious minds do. And when we actually study cognition empirically, to get at the information we can't get introspectively, it turns out that thinking is a pretty complex thing.

You're probably wondering, what brought all this on. Well, I'll tell you what brought all this on. This did! Now, I assume that Todd Zywicki is a fairly intelligent guy. He's on the faculty at what, for all I know, is a good law school. Yet, it appears even this career intellectual isn't immune to the, "I know how my mind works" syndrome when it comes to evaluating cognitive science research. If you follow that link, you will find one of the most intellectually vapid critiques of a scientific research program ever. Zywicki demonstrates quite well that he knows nothing about IAT or implicit attitude research, writing nonsense like:

Is it really plausible that my impression of Bill and Hillary is driven more by whether I have a messy desk than my personal perception that Bill Clinton is a liar and Hillary Clinton is a megalomaniac and opportunist?

No, it's not, Todd, but that's now what the IAT is about. The IAT is about stable attitudes that may not be measurable through explicit measures (which is not to say that they are never measurable explicitly, another misunderstanding Todd seems to have), not ephemeral attitudes that are based on things as irrelevant as how organized your desk is at the moment. I don't know if such attitudes exist, but if they do, the IAT was not designed to discover them.

Now, I'm no fan of the IAT (as I've said before), but my problems with it are methodological. Other implicit tests that avoid the IAT's methodological problems (e.g., evaluative movement assessment) have also shown that implicit attitudes (or evaluations, or valences, or whatever you want to call them) do in fact exist, can be measured, and correlate quite nicely with several behaviors. Now, neither the IAT nor EMA can tell us where these evaluations come from, but that's not their goal. The fact remains that a lot of rigorous research has demonstrated the existence of such attitudes or evaluations, and Todd's disbelief in them doesn't amount to anything like a critique of that research.

Yet, Todd thinks, so he assumes he's qualified to talk about thinking. I wonder, if a physicist had conducted some research that produced a finding that Todd disagreed with, but Todd had not bothered to read up on the actual research, would he write a post calling it absurd? I mean, Todd has existed in the physical universe for as long as he can remember (I'm assuming, of course, that Todd hasn't had any major bouts of psychosis in the past), so he should be able to comment on any findings that concern that universe, right?

My real suspicion is not that Todd doesn't like the IAT. He knows shit about it, so how could he not like it? What I really think is that Todd doesn't like the idea that much of his mental life goes on below the level of his awareness. That's not surprising. I'd bet that a lot of people dislike the idea of that. So, Todd lashes out at a methodology, an area of research, and an entire scientific field, because he's uncomfortable with the truth. I just have to keep reminding myself that this is one of the hazards of my chosen field. When you study something people already think they're experts on, you're going to have to deal with stupidity sometimes. I suppose I should thank Todd for reminding me of that.

(Link to Zywicki's post via Universal Acid, who also thinks the criticisms are stupid.)

UPDATE: In an update at the end of his post, Zywicki responds to my criticisms, and those of Andrew at Universal Acid. I'll ignore the fact that he endorses evolutionary psychology (which is never a good sign), and focus on the remarks that are actually relevant. He writes:

I think the study of cognition and unconscious reasoning is very useful and explains much. I just think that it is important in studying this, as with everything else, that we remain aware of the limitations of the work and, in particular, make sure that the conclusions and implications we draw are actually supported by what the experiments are actually calibrated to test.

Of course, this is what IAT researchers, and others in the field are already doing, and nowhere does Zywicki provide evidence that they are not "Aware of the limitations of the work" or "[making] sure that the conclusions and implications we draw are actually supported by what the experiments are actually calibrated to test." Of course, as I've noted, there may be problems with inferences of "prejudice" from the IAT, but there is still a debate going on about that, and since Zywicki neither references that debate or any of the evidence, his criticisms seem misguided.

Zywicki also confirms my suspicion that his real beef is with the idea that many of his attitudes may have unconscious sources, as he writes:

Clearly many of my beliefs and actions are motivated primarily by my subconscious, equally clearly to me many of my other beliefs and actions are motivated primarily by my conscious, and most is in-between. I recognize that my love for my family or the Pittsburgh Steelers is heavily rooted in my subconscious mind; but I also find it much more likely that my slight preference for Bill versus Hillary Clinton has a lot more to do with my conscious.

The problem is that what research like that on IAT, EMA, goals and evaluations, source monitoring, conscious will, and automaticity has demonstrated is that Zywicki is in no position to determine the extent to which the sources of his attitudes are conscious or unconscious. He can, of course, consciously access his attitudes towards Bill and Hillary, but their sources may (in fact probably do) allude him. That he insists on denying this, without reference to any evidence, only exacerbates my frustration with yet another non-expert who feels qualified to speak with authority on issues he knows little or nothing about.

Which brings me to Zywicki's dumbest remark of all, the one with which he justifies criticizing the work despite his ignorance of it. He compares IAT to astrology, writing:

Do I have to be an expert in the "science" of horoscope reading in order to reject the proposition that "the stars" are in control of my life? I think not.

No, Todd, but you should at least be aware of the differences between science (which IAT research is, and you've offered no reasoned argument to the contrary) and astrology, before comparing the two.

UPDATE II: Zywicki responds again, this time in an entirely new post (once again, via Andrew of Universal Acid, who happens to have added an update to his original post as well). The gist of Zywicki's argument, which he originally used in an email to me, is that if most of the sources of our attitudes are not available to awareness, then it's likely that his (and my) attitudes toward the IAT are not available to awareness, and therefore there's no reason for me, or Andrew, or anyone else to try to convince him to change his mind. In making this argument, Zywicki once again indicates a complete ignorance of cognitive psychology, and I promise you that because he insists on commenting out of complete ignorance, this will be my last attempt to address his "arguments." To do so, I will clarify a few things:

1.) Just because we are aware of, or consciously guiding, our reasoning on a particular topic, doesn't mean that our reasoning cannot be influenced by unconscious processes, associations, and attitudes (valences). In fact, the information of which we are consciously aware is largely the product of unconscious processes which produce a coherent and generally verbalizable (except, of course, in the case of some aspects of visual consciousness) output. Put differently, the information of which we are consciously aware has been polished by unconscious processes, which have largely determined their content.
2.) Just because we are not aware of our cognitive processes and representations (in part or in their entirety), doesn't mean that information of which we are aware (e.g., arguments or facts that others communicate to us) can't influence those processes or representations. In most cases, complex conceptual information must be attended to be processed, and that means we're probably going to be aware of that information, at least while we're encoding it (i.e., while it's present in working memory).
3.) Just because we are consciously aware of our attitudes, behaviors, etc., and perhaps even have some conscious idea of our reasons for them (or consciously reason about them), doesn't mean that the ultimate attitudes, behaviors, etc. are in fact the product of conscious thought. As Dan Wegner has shown over and over again, our inferences of conscious will or forethought are often mistaken.
4.) Just because our ideas are the products of unconscious processes doesn't make them any less rational. If we had evolved to have a largely irrational cognitive system that is supplemented by a fairly small, rational component, we wouldn't have lasted very long. In fact, I would argue that in many cases, the unconscious processes are more likely to be rational (in the behavioral sense, which means something like "optimal") than those that are influenced by conscious thought. Consider a basketball player who is shooting free throws. Most of the time, he does so "unthinkingly," which is to say, utilizing wholly automatic and largely unconscious processes. Occasionally (especially if he's missed a few in a row) however, he will start to "think" about the shots, and reason about how to shoot free throws consciously. His accuracy will almost certainly decrease when he does this, because the automatic processes have been honed through years of practice, and all the conscious reasoning does is interrupt those processes. Zywicki is likely to object that this example isn't quite like his well-reasoned, conscious beliefs about the Clintons, and he'd be right, to some degree, but he'd also be missing the larger point. The unconscious mind isn't some Freudian playground in which nefarious drives and forces produce beliefs and attitudes willy nilly, with no justification. The unconscious mind is dominated by processes that have been honed through millions of years of evolution, and years of life experience, to produce actionable, adaptive outputs.
5.) While it's understandable that Zywicki would hold an antiquated view of the mind, since he's clearly unacquainted with any psychology literature whatsoever, it is not clear why he holds one particular misconception, namely his belief that the IAT only measures unconscious attitudes. Nowhere on the IAT site, or in any of the various publications on IAT in the peer reviewed literature, is this claim made. The word "implicit" in "Implicit Attitude Test" denotes the nature of the test, and not necessarily that of the attitudes it is supposed to be measuring. If Zywicki had actually read the Project Implicit website, he would have learned this, and perhaps even learned that the use of implicit tests is necessitated by the problems involved in getting people to report their attitudes when those attitudes may be socially undesirable. It is true that the IAT people tend to believe that some, and perhaps many attitudes are implicit, and the empirical evidence supports that belief, but critiquing that belief does not count as a criticism of the IAT.