Monday, September 28, 2009

Bang! – The science behind the show

In our segment on Bang Goes the Theory (Mon 28th Sept 7:30pm BBC 1; catch it on BBC iPlayer if you missed it) Peter Lamont and I discussed some of the everyday psychological phenomenon used by magicians to fool our perception and create illusions. There were two main phenomena demonstrated in the piece: Inattentional Blindness and Change Blindness. If you want to try the change blindness test used in the show for yourself skip down to the 'Now its your turn!' section below.

Inattentional Blindness

In the piece, Peter demonstrated how a magician can make an object seem to disappear by controlling where you are looking. We have the impression that we see all of the visual world in great detail at the same time because as soon as we turn our attention to an object in the world we can see it. However, this is an illusion created by our brains to overcome the limitations of our eyes. Our eyes are actually only able to pick out visual detail from a very small portion of the world at any one moment. This is because the light sensitive surface at the back of the eye, known as the retina, has the greatest concentration of cells in a small region around its centre, known as the fovea. The light landing on the fovea comes from a region out in the world roughly the size of your thumb nail held out at arm’s length. When we look at an object we move our eyes so that the light reflected off an object lands on the fovea. The light reflected off objects away from the centre of our attention land on less sensitive parts of the eye resulting in a lower quality image. We may have the impression that we can see everything but in reality if our eyes are not pointing at an object all we can see is a blurry image with poor definition.

However, because we move our eyes on average 3-5 times every second our brains use all the information we get across this sequence of fixations (when the eyes are still) to piece together a detailed impression of the world. Our perception of the world is constructed over time from minimal detail and we assume that we see more than we actually do. This assumption is used by magicians to make you think that if you didn’t see something happen, such as a ball being pocketed, then it didn’t happen!

In the Bang! piece, Peter uses misdirection to ensure that you are not looking at the hand which is hiding the object. He does this by using cues to encourage you to look elsewhere. These cues may involve a sudden flourish of a hand, the waving of a wand, directing his eyes to an object, his posture and referring to an object by name. All of these cues direct our attention to one object whilst misdirecting us from the hand actually performing the trick. Social cues such as these are so powerful and we respond to them so consistently that magicians are able to reliably influence the attention of individuals or entire audiences. The result is that we are blind to the method of the trick because we failed to attend to it. The most famous inattentional blindness demonstration is Simon’s & Chabris ‘Gorillas in our Midst' experiment in which they made viewers fail to notice a person in a gorilla suit by focussing viewer attention on basketball players in the same scene. For a demonstration of the Simon’s & Chabris demonstration and a similar magic trick utilising inattentional blindness see the Colour Changing Card Trick we previously described on this blog.

For further discussion of the use of natural inattentional blindness phenomenon used by magicians see Peter’s book and article below.

Lamont, P. & Wiseman, R. (1999). Magic in theory: an introduction to the theoretical and psychological elements of conjuring (Hatfield: University of Hertfordshire Press)

Lamont, P., Henderson, J. M., & Smith, T. (in press). Where science and magic meet: the illusion of a ‘science of magic’. Review of General Psychology. (* e-mail Peter for a pre-print)

Change Blindness

Another everyday phenomenon used by magicians to create illusions is Change Blindness. Change Blindness refers to when a detail of the visual world changes without us noticing. For example, the most famous change blindness demonstration was performed by Simons and Levin (1997). In this demonstration they made strangers in the street fail to notice when the person they were talking to changed into a different person. They achieved this by hiding the change behind a door which was rudely carried between the stranger and the experimenter during the conversation. See a demonstration of the effect here. And the original article here.

Change Blindness occurs for three reasons: 1) either we don’t perceive the object before it changes, 2) our attention isn’t attracted to the change, or 3) we don’t compare the changed object to our memory of the original object. The first reason is similar to inattention blindness in that it is a failure to take in enough detail of the object. This is probably due to not looking at the object and only taking in sketchy information about the object from our peripheral vision (away from the fovea). The second reason is essential for us to experience change blindness. In the Simons and Levin example the change was hidden behind the door. If the door hadn’t been present and somehow one person had spontaneously changed into another person the change itself would have captured our attention. This is because the change creates visual transients: sudden unnatural changes in the light landing on our retina. In nature visual transients are often caused by dangerous events such as rapidly approaching predators or unexpected object suddenly lunging towards us. As a consequence our visual system is tuned to transients and responds by directing our eyes towards them. Our faith in the our ability to automatically response to such changes is so great that if these transient are hidden we often fail to check whether a change has happened and fail to notice them.

In the Bang! piece we used a change blindness demonstration to show how we rely on the visual world to tell us what has changed. The method we used to hide the change was more subtle than the door example: we used the viewer’s own eye movements. Every 200-300ms we rapidly shift our eyes in order to look at a new object. These eye movements are called saccades. We generally do not blink during a saccade so light continues to land on our retinas. Saccades are so rapid that the light projected into our eyes blurs across our retina. Clearly we do not perceive this blur as otherwise every time you moved your eyes – such as right now, as you are reading this text – we would experience flashes of blurring. To ensure we don’t see the blur our visual system stops processing the light during a saccade. We are effectively blind for the 20-50ms it takes to shift the eyes to a new stable position! If a change is timed to coincide with a saccade the visual transients associated with the change are hidden and our attention is not drawn to it. The change is hidden and the viewer is completely unaware that anything has happened.

In our lab we use an eye tracker to monitor people’s eye movements and detect when a saccade occurs. An eye tracker uses a high-speed infra red camera to locate the viewer’s pupil and record its movements. From the movement of the pupil we can work out where the viewer is looking on a computer screen. In the Bang! piece we showed Dallas a photograph of a real scene and changed objects in the scene as he moved his eyes. For anybody else watching the scene, the changes are immediately apparent because the changes are not timed to coincide with our eye movements. The changes create visual transients which capture out attention. But for Dallas it is much harder for him to detect the changes as the transients are hidden and instead he has to rely on his memory to check what has changed. As our memory for a visual scene is not perfect we often fail to detect changes either because we didn’t store enough information about the objects in memory before the change or we fail to compare the object to our memory. This demonstration shows how minimal our perception of the visual world is, how little information we store in our memory, and how we rely on the visual world itself to tell us when something important has happened.

Now its your turn!

Without an eye tracker we can create similar change blindness effects by hiding the visual transients behind a flicker. I have recreated the change blindness test performed by Dallas by repeatedly flickering the photograph. Every so often an object in the scene will change. Your task is to watch the first video below and detect how many changes occur.

Did you get the right answer? Now watch what happens when we make the changes without the flickers:

The changes seem to ‘pop out’ because the visual transients capture our attention.

The list of changes and their times in the video are written below in invisi-text. Uncover the answer by highlighting the white text.

*Hold down the left mouse button here and drag it to the bottom à

9 sec = Plane in centre appears

10 sec = VUE sign disappears

12 sec = Top floors of the tower block on the right disappear.

14 sec = Safety barriers change colour

16 sec = Man and child change to a woman

18 sec = Rear wheel of left-most bike disappears

20 sec = lowest branch of the tree disappears

21 sec = Man appears next to dustbin

23 sec = Bus disappears

25 sec = Left tower block disappears

*ß finish highlighting text here.

The first time you viewed the scene you may have found it easier to detect some changes rather than others. Dallas had the same experience during his version of the demonstration. The variability in change detection is due to where your eyes are at the time of the change and where they have been up to that point. Work in our lab has shown that you are more likely to detect changes to objects that you are looking at or moving your eyes to compared to objects you haven’t looked at (Henderson & Hollingworth, 1999). When the changes happen without being hidden by a flicker or an eye movement these factors don’t matter. All changes are equally likely to capture your attention. This difference tells us that what you perceive in a visual scene and how the details are stored in memory is very closely related to what you look at. You may think you are aware of peripheral details in a scene but if you do not look at them you will not be able to detect when they change.

These phenomena, inattentional blindness and change blindness are key tools used by magicians to control our visual experience and create fantastical effects. However, they are not exclusive to magic as they are a natural product of how our visual system works. We fail to detect changes or see details of our visual world during every day life. It is only when this failure is brought to our attention during demonstrations like the one above or during magic tricks that we are experience surprise. By looking at these phenomena and how magicians utilise them we can learn more about how these operate in every day life and how much our experience of the visual world is an illusion.

For further information on using eye tracking to investigate change blindness look at the papers below and the rest of the research coming out of our lab:

Henderson, J. M., & Hollingworth, A. (1999). The role of fixation position in detecting scene changes across saccades. Psychological Science, 5, 438-443

Simons, D. J. (2000). Attentional capture and inattentional blindness. Trends in Cognitive Sciences, 4, 147-155.

Wednesday, September 23, 2009

Bang! Goes the Theory

A few weeks ago I had the great privilege of hosting the BBC in our Visual Cognition lab. The BBC filmed a piece for the new science show, Bang! Goes the Theory (Mondays 7:30pm, BBC1). In the piece my colleague Pater Lamont, a psychologist and talented magician explained some of the psychological phenomenon utilised by magicians during close-up magic and then I demonstrated how these phenomenon can be investigated using eye tracking. The whole piece was filmed in and around Prof. John Henderson's Visual Cognition lab.

The piece will be broadcast on Monday 28th September BBC1 at 7:30pm (10:35pm, BBC1Wales). If you miss it you can view it on the BBC iPlayer (UK only). Catch it, I'm sure it will be fun and informative.

Friday, September 04, 2009

Film as a source of visual knowledge

A colleague of mine, Dorota Ostrowska from Birkbeck has written an article on the potential for interdisciplinary film research. As a case study Dorota has used my PhD thesis. Her summary of my research and its implications for broader film studies is very insightful and complimentary. I especially enjoyed her discussion of the implications of my cognitive theory of continuity editing on the classical socio-political interpretation of the 'Hollywood Style' of film editing:

(pg. 108) "Smith’s research offers us a new insight into the view of continuity
editing as a basis of bourgeois and capitalist ideology. His findings show a
psychological basis for continuity editing, revealing its reliance on some
powerful cognitive mechanisms. Such an understanding of continuity
editing does not make the ideological tenets of the cinema that it underpins
right. Instead, it presents us with a different paradigm (scientific and empirical)
to explain why, even today, and in spite of vehement criticism, continuity
editing is still a predominant mode in audiovisual production across
the world. The reasons for the lasting success of this type of editing may be
as much cognitive and perceptive as they are ideological. Furthermore,
Smith’s conclusions change the status of certain editing practices, such as
continuity editing, from a series of arbitrary choices imposed on audiences
by editors working under pressure from money-hungry studios, into
choices based on the cognitive expectations of these audiences. Smith’s
research gives us better understanding why continuity editing has been so
pervasive and the cognitive basis of its ‘naturalness’ to spectators."

This is indeed the implications of my theory and empirical evidence. Although, as a cognitive scientist I would never have stated these implications so boldly as Dorota :) I do not dismiss the political, ideological, or practical forces influencing the path of film form evolution. The intention of my research is to add an extra influence to the evolution, namely cognitive compatibility. I will leave it up to film theorists such as Dorota to decide what this means for classic film theory.

I do not agree with all of Dorota's interpretations of my research but I think the article is very thought provoking and it is nice to see my research influencing others.

Film as a source of visual knowledge in informatics, architecture and music

Ostrowska, D. (2009), Studies in European Cinema, Vol. 5 (2), pgs. 105-116.

This article is a series of questions and reflections, which grow out of my recent
research projects which centred on the issue of interdisciplinarity in relation to
film studies and cinema. Through my work at the Cinema Interdisciplinary
Network (CINET), which we have been running at the University of Edinburgh
since 2005, I have come across a number of academics and researchers who work
with cinema or use moving images in their research but who are not film studies
scholars and are not working in the areas more traditionally associated with film,
such as literature or cultural studies. Could the CINET members’ engagement with
cinema yield some new insights about the object of their study for film scholars?
What does it mean that cinema is becoming a repository of visual knowledge?
How is this knowledge to be treated and organised? What does it mean that film is
becoming not only an aesthetic object but also a tool to conduct intellectual analysis?
What are the difficulties and pitfalls of such research projects? In order to
address these questions, I will discuss two projects presented in the context of
CINET – ‘An Attentional Theory of Continuity Editing’ which is Tim Smith’s doctoral
project completed in the department of Informatics (Computer Science) and
‘Inflecting Space: Correlating the attributes of voice with the character of urban
spaces’, which is a collaboration between architecture and music.