I Didn't Even Notice That You Had a New Head

As I've discussed in previous posts (e.g., this one), our minds are schema machines. We recognize regularities in our environment, and where irregularities differ from our expectations in ways that are not relevant to those expectations, we tend to miss them. The classic, and always-cited (except, so far, in my posts) example of this is Bartlett's "War of the Ghosts" experiment¹, in which people's memories for a story were distorted in ways that made them more culturally relevant. The implications of the schematicity of thought for higher-level cognition are far reaching. Many of our fundamental intuitions about our memories, concepts, creativity, and belief systems are challenged by the nature of schematic thought. The study of things like stereotypes, political views, and even science itself have been greatly influenced by research on the role of schemas and expectations in thought (consider, for example, the role of the confirmation bias in all three). Still, we may at first believe that the influence of schematicity is limited to these higher-order phenomena. We might think that intuitions about lower-level phenomena like visual perception are safe. We'd be wrong. In this post, I am going to discuss one example of how many of our intuitions about visual perception are wrong, an example that may itself have far-reaching implications for how we conceive of our representation of the world. The example is change blindness, a phenomena that is at times so striking and counterintuitive that even experts on vision and attention have had a difficult time believing the results of change detection experiments. Hopefully by the end of this post, all of us will be believers.

If you're like most people, you believe that your conscious visual representation of the world is fairly stable from moment to moment, and that, as long as you're not lost in thought or otherwise distracted, you have a good grasp of what's going on right in front of your eyes. What we are doing, under this intuitive view, is maintaining an internal representation of the visual scene that dynamically changes with the environment, along with visual memories of the preceding scenes. If this is true, then we should have no problem detecting large-scale changes in our visual field, especially those in the center, or focal areas of this field. However, researchers have been aware for decades that there are circumstances under which we may fail to notice such large-scale changes, e.g., when a change occurs while our eyes are performing a saccade. An example often used to illustrate this is the existence of errors in movies. A person's clothing may be different at different moments in a single scene, or the location of an object may change. Directors, editors, and audience members often fail to notice these changes, and production teams often include people whose sole job is to insure scene continuity. Even these well-trained and highly motivated change-detectors often fail to notice continuity errors, and thus the movies are released with the errors. The failure to detect such changes is commonly referred to as change blindness.

Recently, researchers have begun to pay much more attention to change blindness. Experiments in the mid-90s essentially demonstrated what we already knew about change that occurs during saccades. In one experiment², participants were instructed to focus on a photo of two people, and told to expect changes in the scene. The changes (which included the people's hats being switched in one version, and their entire heads being switched in another) were timed to occur during a saccade. Even the most salient of these changes (the switching of the two people's heads) was only noticed by about half of the participants. However, experiments like these may be explainable in ways that do not require the questioning of our assumptions about vision. For instance, it may be that we fail to detect change during saccades because, as our eyes are rapidly moving over the visual field, change is occuring over the entire retina. These changes in our current visual field may serve to mask the change in the scene³.

This explanation is further supported by experiments designed to mimic the effect that a saccade has on the information received by the retina, by creating change across the retina without a saccade. To do this, researchers very briefly insert a blank screen between presentations of the image. When the image returns after the blank screeen, the change has occurred. For example, participants were presented with a photo of a large jet airliner, followed by the brief presentation of a blank screen, and then the photo of the airliner again, but this time with one of the big jet engines missing⁴. Even when such sequences (photo-blank-changed photo) were presented over and over again, participants often either did not notice the change, or required several presentations to detect it. When the change occurred without the intervening scene, however, people had no problem noticing that the engine disappeared.

The results of another set of studies, however, called this explanation into question. In these studies, the "global" change affected by a saccade or the presentation of a blank screen was replaced by local change in parts of the visual field at the same time that the critical change (the one people were supposed to notice) occurred. For example, as the plane's engine was disappearing, dot patters would appear briefly in other locations on the screen⁵. Under these conditions, people often failed to detect the change. This can't be explained by arguing that change across the entire retina masked the critical change. In these experiments, change only occurred in a few locations in the visual field. Thus, it appears that what is actually happening is that we fail to detect the change because we don't have an exhaustive visual memory of the scene as it was just before the change.

These early studies were criticized by some because they lacked ecological validity. The presentation of large changes in a static image (such as the switching of hats or heads, or the disappearance of a plane engine) that occur entirely during a saccade, or immediately after a blank screen are rare in our everyday experience. To answer these criticisms, studies were conducted using moving images (videos), and finally real-world interactions. In the video studies, participants watched a video in which one actor was replaced by another during a camera movement, and most failed to detect the change⁶.

The most impressive example of change blindness comes from the real-world interaction experiments. In these experiments, researchers approached people on the street and asked them for directions. While the participant was giving directions, two more experimenters walked between them carrying a door. Behind the door was a fourth experimenter who, when the experimenter who had originally asked for directions was completely occluded from the participant's view by the door, switched places with the original experimenter. In this experiment, 50% of the participants failed to notice that the person to whom they were giving directions changed⁷.

With this last experiment, we can begin to see the parallels between the higher-level cognitive phenomena associated with schema and change blindness. In this experiment, participants were much more likely to notice their new interlocutor if the participants were college students. This was likely because the experimenters, who were also college-aged, were members of their social group, and the participants were therefore more likely to notice details about them. When the participants were older, or the researchers were dressed as construction workers, even the student-aged participants failed to notice the change. This implies that attention and expectations may play a role in change detection.

Previous research on memory for change is consistent with this interpretation. When participants were presented with pictures during a learning phase, and later shown either the same pictures, or pictures with a single change, participants tended to mistakenly remember the change pictures as having been shown during the learning phase unless the change was schema-relevant, but inconsistent⁸, a result consistent with other findings in the schematic memory literature (see here for a discussion of these). Subsequent change blindness experiments using static and video presentations have also demonstrated that unless people encode specific details, due to, e.g., the violation of expectations, they will fail to notice change⁹.

So, under even ordinary circumstances, people often fail to notice large-scale changes in their visual field, and this failure appears to be associated with attention and expectations. An important question, then, is how and what we are visually representing the world from moment to moment. There are two schools of thought on this. The first, and the one that is most directly supported by the research I've just described, is that we either don't form lasting visual representations of the world, and instead continually monitor the world, using it as, in essence, its own representation, or we form but do not retain these representations (these views are the basis of the sensorimotor account of vision). A second view, which has recently received empirical support, argues that we do form and retain visual representations, but that we either fail to or inadequately compare current representations to preceding ones. The evidence for this view comes from change blindness studies using videos¹⁰ or real-world interactions¹¹ in which memory tests are administered subsequent to the change blindness task. Participants show evidence of memory for the scenes, both before and after the changes, even when they failed to detect the changes in the original task. In another study¹² in which participants observed (and either detected or did not detect) a change and were then given forced choice questions in which they had to indicate which of two objects had been in the scenes, participants chose both the pre and post-change options well above chance, indicating that they had some enduring representation of the scene from both before and after the change. Furthermore, their performance on the pre-change forced choice questions was worse than that for the post-change questions, indicating that the pre-change representation was weaker. This may mean that, while people have representations of the scene both before and after the change, the pre-change representation may have been partially "overwritten" by the post-change representation, making a comparison of the two difficult.

One last result is interesting to note. In a recent study¹³, participants were presented with a change blindness task in which the timing of the change was varied. After participants were able to detect the change, they were asked when the change occurred. Participants consistently indicated that the change had occurred much later (the time they indicated involved a delay prior to the onset of the change that was, on average, 85X greater than the actual delay), indicating that they were unaware that they had been blind to the change for so long. In other words, they failed to detect their own detection failure. This hints at both the source of our intuitions about the completeness and veridicality of our visual representation of the world, and the sorts of errors that our commonsense intuitions can lead to.

If you are interested, you can view change blindness demos here.

¹ Bartlett, F.C. (1932). Remembering: a study in experimental and social psycology. New York: The Macmillan Company.
² Grimes, J. (1996). On the failure to detect changes in scenes across saccads. In Atkins, K. (ed.), Perception (Vancouver Studies in Cognitive Science, Vol 2. 89-110. Oxford University Press.
³ Simons, D.J., & Levin, D.T. (1997). Change blindness. Trends in Cognitive Science, 1(7), 261-267.
⁴ Rensink, R.A., O'Regan, J.K., & Clark, J.J. (1997). To see or not to see: The need for attention to perceive changes in scenes. Psychological Sciences, 8, 368-373.
⁵ O'Reagan, J.K., Rensink, R.A., & Clark, J.J. (1996). "Mud splashes" render picture changes invisible. Investigative Opthamology & Visual Science, 37, 5213.
⁶ Levin, D.T., & Simons, D.J. (1997). Failure to detect changes to attended objects in motion pictures. Psychonomic Bulletin and Review, 4, 501-506.
⁷ Simons, D.J., & Levin, D.T. (1998). Failure to detect changes to people during a real-world interaction. Psychonomic Bulletin & Review 5 (4), 644-649.
⁸ Friedman, A. (1979). Framing pictures: The role of knowledge in automatized encoding and memory for gist. Journal of Experimental Psychology: General, 108, 316-355.
⁹ Simons & Levin (1997).
¹⁰ Angelone, B. L., Levin, D. T., & Simons, D. J. (2003). Representation and comparison failures in change blindness. Perception, 32, 947-962.
¹¹ Simons, D. J., Chabris, C. F., Schnur, T., & Levin, D. T. (2002). Evidence for preserved representations in change blindness. Consciousness and Cognition, 11, 78-97.
¹² Mitroff, S. R., & Simons, D. J., & Levin, D. T. (In Press). Nothing compares 2 views: Change blindness results from failures to compare retained information. Perception & Psychophysics.
¹³ Scholl, BJ, Simons, DJ, & Levin, DT (In Press). Change blindness blindness: An implicit measure of a metacognitive error. Psychonomic Bulletin & Review.