If you've read this far, you probably recognize that my view of my own discipline isn't too different from Quine's. Here is a passage from Quine that Lindsay quotes in her post:
The old epistemology aspired to contain, in a sense, natural science; it would construct its somehow from sense data. Epistemology in its new setting, conversely, is contained in natural science, as a chapter of psychology. But the old containment remains valid, too, in its way. We are studying how the human subject of our study posits bodies and projects his physicis from his data, and we appreciate that our position in the world is just like his. Our very epistemological enterprise, therefore, and the psychology wherein it is a component chapter, and the whole of natural science wherein psychology is a component book—all of this is our own construction or projection from simulations like those we were meting out to our epistemological subject. Thus there is reciprocal containment, though containment in different senses: epistemology in natural science and natural science in epistemology.—W.V.O. Quine, Epistemology Naturalized, pg 25.Lindsay uses this passage, and the sentiments therein, as a launching point for a discussion of what is today (rather than in the 18th century) called experimental philosophy. I, however, wonder if experimental philosophy, as its practiced by philosophers like Stich, is really what Quine had in mind. The general method of experimental philosophers seems to be to present people (college students, most often) with stories, and to answer a few questions related to important philosophical issues, designed to get people to tell them what they think is going on. This practice is meant to solve the problem of philosophers doing their own individual thought experiments (experiments with an N of 1, as Lindsay puts it) made up of problematic (and often counterfactual) situations, and using their intuitions to develop philosophical pictures of the concepts involved in those situations. What experimental philosophy, in this sense, can do is a.) show that there are several possible intuitions (each of which is held by members of different populations, perhaps), and then b.) investigate the extent to which those different intuitions are justified.
What I wonder is, can experimental philosophy do this? And is this really even something that needs to be done? To answer the second question first, I'm pretty sure the fact that different philosophers, usually from within the same population, have different intuitions about the same situations renders experiments designed to show that this is true for non-philosophers as well fairly superfluous. One can think of the last 50 years (or 2500 years, depending on your preference) of epistemological inquiry as an extended experiment with an N in the thousands, and the data should force us to admit that there is a diversity of intuitions about most, if not all of the sorts of epistemological problems and concepts that philosophers consider.
My answer to the first question, which asks whether experimental philosophy can tell us anything about the intuitions epistemologists use, is just this side of no. For one, I don't think this is the sort of thing that Quine had in mind. It's certainly nothing like the way psychology is done by psychologists. Outside of clinical and social psychology (which I seriously doubt experimental philosophers want to take as their methodological models), psychologists almost never conduct experiments designed to get participants to tell them what they think about something. That's what people in marketing departments do. Sure, in cognitive psychology, self-report measures are used, and with frequency in some sub-areas, but in most cases, what the researcher is interested in is not so much what a person says he or she thinks about the stimuli, but the cognitive and percpetual mechanisms that underlie what he or she thinks about them. For example, some psychologists (I'm not one of them) believe that they can understand the subtelties of the phenomenological character of different types of memory, or memory for different types of information, by asking people to indicate whether theythey "remember" seeing the information during the learning phase (i.e., they remember the presentation itself), or only "know" that they saw it (i.e., they have a feeling that they saw it, but don't remember the presentation of that item) - the so-called "remember-know" procedure (see this paper for an example of its use). In most cases, though, psychologists tend to be wary of self-report measures, and prefer indirect measures of the phenomena being studied (even the remember-know procedure is pretty indirect). This is in part because experiment participants tend to say what they think you want to hear, but it's also because much of what is interesting, scientifically, about cognition is not intrpospectively available. The consensus among cognitive psychologists is that the science of cognition has to rely primarily on third-person methodologies, using first-person reports for primarily abductive purposes.
One criticism of experimental philosophy and its methods, therefore, is that it doesn't look a heck of a lot like the rest of experimental psychology. Then there's the question of what the hell the results of experimental philosophy experiments demonstrate. For one, they tend to present fairly strange circumstances, the sort that philosophers use all the time, but which nonphilosophers will probably never consider (except in drunken conversations with philosophy majors late at night in dorm rooms). Of course, this is a criticism of many cognitive psychology experiments as well, but the awkwardness of those is usually caused by the indirect nature of the experimental measure. In the case of experimental philosophy, the awkwardness is intentional, and the measurement direct. Do peoples' intuitions about reference in unusual counterfactual scenarios really tell us about their intuitions in ordinary situations? Philosophers are trained to use certain pathological cases to help them develop generalizable intuitions; undergraduate social work majors are not. Then there's the types of questions experimental philosophers ask about their awkward situations. A common criticism from people like J. L. Austin and Norman Malcolm is that philosophers use words that ordinary people never do to refer to ordinary things, and that they use words that ordinary people use to refer to things that ordinary people never refer to. Are we really getting at peoples' intuitions about "intentionality" and "reference" by asking them questions with those terms, which they probably never use (how often, really, do ordinary people say, "I intended to do that?") in those contexts?
I think the sort of thing that I do is the real experimental philosophy (actually, I think all science is experimental philosophy in a way), and that if philosophers want to develop better intuitions, then it's the sort of work they should be doing, or at least paying attention to. If you want better intuitions about reference, you don't go out and ask a bunch of people what their intuitions about reference are. Instead, you go out and look at how people refer to things, and then use that information to develop more educated intuitions about reference. Seriously, isn't that what philosophers who use other branches of philosophy do? When philosophers want better intuitions about time, they don't go out and ask a bunch of people what they think about time. Instead, they look at what physics experiments and theories have to say about time, and use that knowledge to develop better intuitions. Granted, time is supposed to be an objective property of the world, while reference is at least partially subjective (it takes a subject and an object to do any real referring), but I think the principle is still the same.
Have I ranted long enough? Probably too long. I'll just shut up now.