Bang! – The science behind the show

In our segment on Bang Goes the Theory (Mon 28^th Sept 7:30pm BBC 1; catch it on BBC iPlayer if you missed it) Peter Lamont and I discussed some of the everyday psychological phenomenon used by magicians to fool our perception and create illusions. There were two main phenomena demonstrated in the piece: Inattentional Blindness and Change Blindness. If you want to try the change blindness test used in the show for yourself skip down to the 'Now its your turn!' section below.

Inattentional Blindness

In the piece, Peter demonstrated how a magician can make an object seem to disappear by controlling where you are looking. We have the impression that we see all of the visual world in great detail at the same time because as soon as we turn our attention to an object in the world we can see it. However, this is an illusion created by our brains to overcome the limitations of our eyes. Our eyes are actually only able to pick out visual detail from a very small portion of the world at any one moment. This is because the light sensitive surface at the back of the eye, known as the retina, has the greatest concentration of cells in a small region around its centre, known as the fovea. The light landing on the fovea comes from a region out in the world roughly the size of your thumb nail held out at arm’s length. When we look at an object we move our eyes so that the light reflected off an object lands on the fovea. The light reflected off objects away from the centre of our attention land on less sensitive parts of the eye resulting in a lower quality image. We may have the impression that we can see everything but in reality if our eyes are not pointing at an object all we can see is a blurry image with poor definition.

However, because we move our eyes on average 3-5 times every second our brains use all the information we get across this sequence of fixations (when the eyes are still) to piece together a detailed impression of the world. Our perception of the world is constructed over time from minimal detail and we assume that we see more than we actually do. This assumption is used by magicians to make you think that if you didn’t see something happen, such as a ball being pocketed, then it didn’t happen!

In the Bang! piece, Peter uses misdirection to ensure that you are not looking at the hand which is hiding the object. He does this by using cues to encourage you to look elsewhere. These cues may involve a sudden flourish of a hand, the waving of a wand, directing his eyes to an object, his posture and referring to an object by name. All of these cues direct our attention to one object whilst misdirecting us from the hand actually performing the trick. Social cues such as these are so powerful and we respond to them so consistently that magicians are able to reliably influence the attention of individuals or entire audiences. The result is that we are blind to the method of the trick because we failed to attend to it. The most famous inattentional blindness demonstration is Simon’s & Chabris ‘Gorillas in our Midst' experiment in which they made viewers fail to notice a person in a gorilla suit by focussing viewer attention on basketball players in the same scene. For a demonstration of the Simon’s & Chabris demonstration and a similar magic trick utilising inattentional blindness see the Colour Changing Card Trick we previously described on this blog.

For further discussion of the use of natural inattentional blindness phenomenon used by magicians see Peter’s book and article below.

Lamont, P. & Wiseman, R. (1999). Magic in theory: an introduction to the theoretical and psychological elements of conjuring (Hatfield: University of Hertfordshire Press)

Lamont, P., Henderson, J. M., & Smith, T. (in press). Where science and magic meet: the illusion of a ‘science of magic’. Review of General Psychology. (* e-mail Peter for a pre-print)

Change Blindness

Another everyday phenomenon used by magicians to create illusions is Change Blindness. Change Blindness refers to when a detail of the visual world changes without us noticing. For example, the most famous change blindness demonstration was performed by Simons and Levin (1997). In this demonstration they made strangers in the street fail to notice when the person they were talking to changed into a different person. They achieved this by hiding the change behind a door which was rudely carried between the stranger and the experimenter during the conversation. See a demonstration of the effect here. And the original article here.

Change Blindness occurs for three reasons: 1) either we don’t perceive the object before it changes, 2) our attention isn’t attracted to the change, or 3) we don’t compare the changed object to our memory of the original object. The first reason is similar to inattention blindness in that it is a failure to take in enough detail of the object. This is probably due to not looking at the object and only taking in sketchy information about the object from our peripheral vision (away from the fovea). The second reason is essential for us to experience change blindness. In the Simons and Levin example the change was hidden behind the door. If the door hadn’t been present and somehow one person had spontaneously changed into another person the change itself would have captured our attention. This is because the change creates visual transients: sudden unnatural changes in the light landing on our retina. In nature visual transients are often caused by dangerous events such as rapidly approaching predators or unexpected object suddenly lunging towards us. As a consequence our visual system is tuned to transients and responds by directing our eyes towards them. Our faith in the our ability to automatically response to such changes is so great that if these transient are hidden we often fail to check whether a change has happened and fail to notice them.

In the Bang! piece we used a change blindness demonstration to show how we rely on the visual world to tell us what has changed. The method we used to hide the change was more subtle than the door example: we used the viewer’s own eye movements. Every 200-300ms we rapidly shift our eyes in order to look at a new object. These eye movements are called saccades. We generally do not blink during a saccade so light continues to land on our retinas. Saccades are so rapid that the light projected into our eyes blurs across our retina. Clearly we do not perceive this blur as otherwise every time you moved your eyes – such as right now, as you are reading this text – we would experience flashes of blurring. To ensure we don’t see the blur our visual system stops processing the light during a saccade. We are effectively blind for the 20-50ms it takes to shift the eyes to a new stable position! If a change is timed to coincide with a saccade the visual transients associated with the change are hidden and our attention is not drawn to it. The change is hidden and the viewer is completely unaware that anything has happened.

In our lab we use an eye tracker to monitor people’s eye movements and detect when a saccade occurs. An eye tracker uses a high-speed infra red camera to locate the viewer’s pupil and record its movements. From the movement of the pupil we can work out where the viewer is looking on a computer screen. In the Bang! piece we showed Dallas a photograph of a real scene and changed objects in the scene as he moved his eyes. For anybody else watching the scene, the changes are immediately apparent because the changes are not timed to coincide with our eye movements. The changes create visual transients which capture out attention. But for Dallas it is much harder for him to detect the changes as the transients are hidden and instead he has to rely on his memory to check what has changed. As our memory for a visual scene is not perfect we often fail to detect changes either because we didn’t store enough information about the objects in memory before the change or we fail to compare the object to our memory. This demonstration shows how minimal our perception of the visual world is, how little information we store in our memory, and how we rely on the visual world itself to tell us when something important has happened.

Now its your turn!

Without an eye tracker we can create similar change blindness effects by hiding the visual transients behind a flicker. I have recreated the change blindness test performed by Dallas by repeatedly flickering the photograph. Every so often an object in the scene will change. Your task is to watch the first video below and detect how many changes occur.

Did you get the right answer? Now watch what happens when we make the changes without the flickers:

The changes seem to ‘pop out’ because the visual transients capture our attention.

The list of changes and their times in the video are written below in invisi-text. Uncover the answer by highlighting the white text.

*Hold down the left mouse button here and drag it to the bottom à

9 sec = Plane in centre appears

10 sec = VUE sign disappears

12 sec = Top floors of the tower block on the right disappear.

14 sec = Safety barriers change colour

16 sec = Man and child change to a woman

18 sec = Rear wheel of left-most bike disappears

20 sec = lowest branch of the tree disappears

21 sec = Man appears next to dustbin

23 sec = Bus disappears

25 sec = Left tower block disappears

*ß finish highlighting text here.

The first time you viewed the scene you may have found it easier to detect some changes rather than others. Dallas had the same experience during his version of the demonstration. The variability in change detection is due to where your eyes are at the time of the change and where they have been up to that point. Work in our lab has shown that you are more likely to detect changes to objects that you are looking at or moving your eyes to compared to objects you haven’t looked at (Henderson & Hollingworth, 1999). When the changes happen without being hidden by a flicker or an eye movement these factors don’t matter. All changes are equally likely to capture your attention. This difference tells us that what you perceive in a visual scene and how the details are stored in memory is very closely related to what you look at. You may think you are aware of peripheral details in a scene but if you do not look at them you will not be able to detect when they change.

These phenomena, inattentional blindness and change blindness are key tools used by magicians to control our visual experience and create fantastical effects. However, they are not exclusive to magic as they are a natural product of how our visual system works. We fail to detect changes or see details of our visual world during every day life. It is only when this failure is brought to our attention during demonstrations like the one above or during magic tricks that we are experience surprise. By looking at these phenomena and how magicians utilise them we can learn more about how these operate in every day life and how much our experience of the visual world is an illusion.

For further information on using eye tracking to investigate change blindness look at the papers below and the rest of the research coming out of our lab:

http://www.psy.ed.ac.uk/people/jhender9/Lab/

Henderson, J. M., & Hollingworth, A. (1999). The role of fixation position in detecting scene changes across saccades. Psychological Science, 5, 438-443

Simons, D. J. (2000). Attentional capture and inattentional blindness. Trends in Cognitive Sciences, 4, 147-155.

Bang! Goes the Theory

A few weeks ago I had the great privilege of hosting the BBC in our Visual Cognition lab. The BBC filmed a piece for the new science show, Bang! Goes the Theory (Mondays 7:30pm, BBC1). In the piece my colleague Pater Lamont, a psychologist and talented magician explained some of the psychological phenomenon utilised by magicians during close-up magic and then I demonstrated how these phenomenon can be investigated using eye tracking. The whole piece was filmed in and around Prof. John Henderson's Visual Cognition lab.

The piece will be broadcast on Monday 28th September BBC1 at 7:30pm (10:35pm, BBC1Wales). If you miss it you can view it on the BBC iPlayer (UK only). Catch it, I'm sure it will be fun and informative.

Film as a source of visual knowledge

A colleague of mine, Dorota Ostrowska from Birkbeck has written an article on the potential for interdisciplinary film research. As a case study Dorota has used my PhD thesis. Her summary of my research and its implications for broader film studies is very insightful and complimentary. I especially enjoyed her discussion of the implications of my cognitive theory of continuity editing on the classical socio-political interpretation of the 'Hollywood Style' of film editing:

(pg. 108) "Smith’s research offers us a new insight into the view of continuity

editing as a basis of bourgeois and capitalist ideology. His findings show a

psychological basis for continuity editing, revealing its reliance on some

powerful cognitive mechanisms. Such an understanding of continuity

editing does not make the ideological tenets of the cinema that it underpins

right. Instead, it presents us with a different paradigm (scientific and empirical)

to explain why, even today, and in spite of vehement criticism, continuity

editing is still a predominant mode in audiovisual production across

the world. The reasons for the lasting success of this type of editing may be

as much cognitive and perceptive as they are ideological. Furthermore,

Smith’s conclusions change the status of certain editing practices, such as

continuity editing, from a series of arbitrary choices imposed on audiences

by editors working under pressure from money-hungry studios, into

choices based on the cognitive expectations of these audiences. Smith’s

research gives us better understanding why continuity editing has been so

pervasive and the cognitive basis of its ‘naturalness’ to spectators."

This is indeed the implications of my theory and empirical evidence. Although, as a cognitive scientist I would never have stated these implications so boldly as Dorota :) I do not dismiss the political, ideological, or practical forces influencing the path of film form evolution. The intention of my research is to add an extra influence to the evolution, namely cognitive compatibility. I will leave it up to film theorists such as Dorota to decide what this means for classic film theory.

I do not agree with all of Dorota's interpretations of my research but I think the article is very thought provoking and it is nice to see my research influencing others.

Film as a source of visual knowledge in informatics, architecture and music

Ostrowska, D. (2009), Studies in European Cinema, Vol. 5 (2), pgs. 105-116.

This article is a series of questions and reflections, which grow out of my recent

research projects which centred on the issue of interdisciplinarity in relation to

film studies and cinema. Through my work at the Cinema Interdisciplinary

Network (CINET), which we have been running at the University of Edinburgh

since 2005, I have come across a number of academics and researchers who work

with cinema or use moving images in their research but who are not film studies

scholars and are not working in the areas more traditionally associated with film,

such as literature or cultural studies. Could the CINET members’ engagement with

cinema yield some new insights about the object of their study for film scholars?

What does it mean that cinema is becoming a repository of visual knowledge?

How is this knowledge to be treated and organised? What does it mean that film is

becoming not only an aesthetic object but also a tool to conduct intellectual analysis?

What are the difficulties and pitfalls of such research projects? In order to

address these questions, I will discuss two projects presented in the context of

CINET – ‘An Attentional Theory of Continuity Editing’ which is Tim Smith’s doctoral

project completed in the department of Informatics (Computer Science) and

‘Inflecting Space: Correlating the attributes of voice with the character of urban

spaces’, which is a collaboration between architecture and music.

http://www.atypon-link.com/INT/doi/abs/10.1386/seci.5.2.105_1

The educational power of film

A new journal article published this week in Psychological Science asks whether watching popular Hollywood films depicting historical events can enhance learning of historical facts. This is a question that has been asked many times in the past and educators often assume that there is a benefit to watching video reenactments, not least of all because they are entertaining. This study aimed to test whether there can be both benefits (when the hostorical informationd epicted in the film is accurate) and costs (when the details are inaccurate) to using historical films in a history lesson. They used selection of excerpts from well-known feature films including The Last Samurai (Ed Zwick, 2003), Amistad (Steven Spielberg, 1997) and Glory (Ed Zwick, 1989). They recorded better memory for historical facts when the students watched a film version but only when the film depicted the facts accurately. When the facts were inaccurate the false details in the film sometimes take precendence over the accurate details taught to the students using traditional means.

This is a very interesting study as it shows both the strengths of using films to teach history but also the weaknesses. Any educator considering using a feature film in their lessons should consider that the primary purpose of a feature film is to entertain, not be historically accurate. The most entertaining way of telling a story is rarely the most authentic. Film makers often distort the truth to increase drama and excitement. As outlined in the study below, educators should make their students aware of this so that they don't believe everything they see on the screen.

I personally learned this lesson the hard way by very confidentally answering "Ben Hur" to the question "Who helped Jesus with the cross?" in a Religious Education class. Damn artistic licence!

(*according to the New Testament the answer is Simon The Cyrene...... not Charlton Heston)

Using Popular Films to Enhance Classroom Learning: The Good, the Bad, and the Interesting

Butler, A. C., Zaromb, F. Z., Lyle, K.B. & Roediger, H. L. III (in press) Psychological Science

ABSTRACT—Popular history films sometimes contain major historical inaccuracies. Two experiments investigated how watching such films influences people's ability to remember associated texts. Subjects watched film clips and studied texts about various historical topics. Whereas the texts contained only correct information, the film clips contained both correct information (consistent with the text) and misinformation (contradicted by the text). Before watching each clip, subjects received a specific warning, a general warning, or no warning about the misinformation. One week later, they returned for a cued-recall test about the texts. Watching a film clip increased correct recall of consistent information relative to recall of the same information when subjects did not see the clip. However, when the information in the film contradicted the text, subjects often (falsely) recalled misinformation from the film. The specific warning substantially reduced this misinformation effect. Teachers should use popular history films with caution and should warn students about major inaccuracies in the films.

http://www3.interscience.wiley.com.ezproxy.webfeat.lib.ed.ac.uk/journal/122521484/abstract

Antichrist and the anti-cut

Lars von Trier’s Antichrist is an unsettling film for many reasons. It will be discussed in depth for its more obvious unsettling traits such as its intense graphic violence and self-mutilation and lack of a clear message especially with regard to gender relationships and the nature of evil. Some of these traits, such as the horror imagery, settings and plot-devices may seem unique within Trier’s work to date. However, there are other, more subtle methods used to unsettle the viewer which Trier has been refining since Dogme 95. I’m speaking, of course about his use of discontinuous editing.

Antichrist is a two-header centring around the grief of a married couple (Charlotte Gainsborough and Willem Dafoe) after the loss of their young son during the film’s prologue. The majority of the film depicts the two main actors whilst they try to come to terms with the loss and confront the wife’s fears about their holiday cottage in the woods, Eden. The film employs stunning high-speed black and white cinematography to add incredible beauty to some brutal scenes including the son’s fall to his death and the copulation of the couple during the prologue. These scenes are offset by, uncharacteristically glossy scenes of the couple interacting initially at home and then at the cottage. These scenes have a “Hollwood”-esque appearance: very beautifully lit, shallowly focussed with frequent focus-pulls into the shot, and shot from mostly stable camera positions. This is in stark contrast to the compositional style developed by Trier and his cinematographer Anthony Dod Mantle over their last few films culminating in computer-controlled camera and lighting system (Automavision) employed by Trier in his last film, The Boss Of It All. In a previous blog post I discussed how this system randomly shifted the camera position, zoom and focus of the camera. This resulted in the focal object (e.g. a talking character) drifting out of shot or being framed in such an obtuse manner that the viewer had to struggle to fixate the region of interest such as the characters eyes or mouth. The unpredictable camera movements also force the viewer to actively pursue the focal object with their eyes rather than stablise their gaze (i.e. fixate) the object. Even before Automavision, Trier’s use of hand-held cameras and improvised staging and acting in earlier films (see Idioterne and Dancer in the Dark) meant that the viewer always had to be active in their viewing of his films. Unlike Hollywood films edited according to the Continuity Editing Style (see my thesis) which keep focal objects relatively stationary on the screen and always establish predictable screen locations or the modern Intensified Continuity style (see David Bordwell’s discussion ) which use rapidly edited close shots , Trier’s films make no concessions to the viewer’s attention, actively engaging them in the staging of a scene by forcing them to search the screen for the most relevant details. In Dancer In The Dark the hand-held camera roams around the main characters often failing to keep up with changes in action. Trier compensates by cutting to different takes of the same scene with a slightly different camera position on the same actors. This creates, what is traditionally referred to as a Jump Cut: a sudden displacement of an object or actor on the screen caused by a cut to a camera position less than 30 degrees away from the original position (for an in-depth discussion of perceptual reasons for Jump Cuts see Joseph Anderson’s The Reality of Illusion). Prior to the cut, the viewer is trying to compensate for the actor’s movement relative to the screen by pursuing the actor or saccading. A sudden unexpected Jump Cut then displaces the actor on the screen requiring the viewer to reorient to the new shot. The consequence is much more active eye movement behaviour than is usually required when viewing a film edited in the Continuity style. This level of active viewing is so alien to a cinema audience that the experience can be exhausting and unsettling. Such affective response is utilised by Trier to add to the overall experience of watching his films.

At the recent Society for Cognitive Studies of the Moving Images annual meeting in Copenhagen we were treated to an advanced screening of Antichrist and Q&A with Lars von Trier. I asked him how his discontinuous editing style had come about and whether he used it deliberately to unsettle his viewer. Trier enthusiastically described the way that he and his long-term editor Molly Marlene Stensgaard had experimented in the editing suite and devised a style of cut that, whilst violating classical editing conventions, they believed accurately portrayed the mood of a scene. I referred to this cut as a Jump Cut but Trier seemed uncomfortable with this name and possibly its negative connotations. Trier explained that he didn’t consider the cut to be discontinuous because he ensured that the audio track was continuous across the cut. The camera position would suddenly shift across the cut as if it had been moving smoothly around the principal object but a portion of the movement had been omitted by the edit. However, because the camera positions were actually filmed at different times the shots either side of the cut could depict a (somewhat) continuous action and the audio track could be stitched together to make it sound as if the scene were continuous. Trier stated that he thought the most important element for creating the illusion of continuity was the audio track. As long as the audio implied that the scene was continuous the audience would perceive the scene as continuous. A Jump Cut traditionally implies an omission of time. Trier’s cut overrules this implication by using continuous audio. Not perceptual time is omitted. As such we will refer to Trier’s type of cut as an Anti Cut.

The effect of an Anti Cut on the viewer is cognitive dissonance. Cognitive dissonance comes about when a person holds two (or more) conflicting concepts in their mind at the same time. The experience of cognitive dissonance is unpleasant. We endeavour to find order in our experiences and ideas and when conflicting concepts cannot be resolved we are uncertain how to act. If these concepts come from two different sensory modalities the brain tries to reconcile the two sources and come up with an amalgamation. A classic example of this is the Mc Gurk Effect (see video below). The McGurk effect is a demonstration of how we did not evolve to process our senses in isolation. When watching someone say a word our perception of the sounds they make is informed by both the audio and way their lips mouth the word. If these two sources are in conflict e.g. the mouth says “ba” but the audio track says “ga”, we perceive something in between, e.g. “da” . This demonstrates how sensory perception is multimodal and the brain will endeavour to resolve cognitive dissonance even if it means distorting reality and creating an illusory perception.

In the case of Trier’s Anti Cut it appears that the audio track takes precedence over the visuals to create the perception of temporal continuity. This may only be possible as long as the visuals do not conflict with the audio. If the cut involved a shift in camera position and a repetition of action we would expect the viewer to perceive the repetition irrespective of how temporally continuous the soundtrack is. However, Trier typically cuts in between actions when no visual events are occurring and, therefore cannot be used to perceive the omission of time. The sudden shift in camera position and changes in actor posture, position, lighting, etc all imply a temporal omission but it appears that these are overridden by the temporal continuity implied by the audio track. Scenes edited using Anti Cuts, such as the bedroom scenes in the first act of Antichrist or the scene in which Bjork’s character’s son receives a new bike in Dancer in the Dark, create a strange, almost abstract sense of time. Many aspects of the scene imply temporal continuity such as the continuous audio and the depicted events. However, the sudden shifts in camera position caused by the Anti Cut imply discontinuity. The resulting perception may be of general temporal continuity but with residual conflicting cues which, if reasoned about may confuse the viewer and undermine the general perception of continuity.

Trier’s last film, The Boss of it All contains many examples of such conflicting cues. In order to compensate for the frequent, inexplicable camera moves caused the computer controlled camera system (Automavision) Trier had to use frequent Anti Cuts to stitch together different takes. Often the lighting and sound balance would also change wildly across such cuts. Unlike in Antichrist when the audio was carefully controlled to imply temporal continuity, the sudden changes in The Boss of it All implied temporal discontinuity. The result was the perception of a scene as a construction, clearly stitched together from takes filmed at different times. The depicted actions implied temporal continuity and if the viewer chose to engage only with this dimension and ignore the other conflicting dimensions they could perceive scenes as temporally continuous but my feeling of watching the film was that such active reconstruction of the scene required much more effort than watching a film edited according to the Continuity Style or even some of Trier’s other films.

As always, Lars Von Trier is pushing the boundaries or editing in order to experiment with our perception of film. As a film viewer I adore Trier’s introspective experiments and as a scientist I would love to see empirical investigations of these issues. What is the relationship between the different sensory modalities in film and which modality takes precedence and leads to the resulting perception of continuity. In Chapter 5 of my thesis I laid out a theory about how continuity perception in film may operate and I went on to empirically investigate time perception across cuts. This investigation was purely in the visual dimension although I acknowledged how powerful the audio dimension appeared to be and how editors placed such emphasis on audio in the creation of continuity. Direct empirical investigation of the role of audio in continuity perception is required but for now we can occupy ourselves with marvelling the masterful way Trier manipulates our experience of time and space in order to create an unsettling viewing experience.

Come and work in our lab: PhD studentships

Would you like to come and work in Visual Cognition? Would you like to be part of John Henderson's great research lab and learn how to investigate static, dynamic, and film cognition using eye tracking?

Well we now have funded PhD studentships available.

Details below:

UNIVERSITY OF EDINBURGH
SCHOOL OF PHILOSOPHY, PSYCHOLOGY AND LANGUAGE SCIENCES

PPLS (PSYCHOLOGY) CAREER DEVELOPMENT STUDENTSHIPS
Deadline for Application: Friday 10th July 2009

A minimum of four Career Development Studentships are offered by Psychology within the School of Philosophy, Psychology and Language Sciences

Career Development Studentships are designed to provide students with teaching
experience, training, and other career development opportunities. A stipend of £10,000 per
annum will be provided in addition to tuition fees at the Home/EU level and additional
programme costs of £600 per annum. The studentships will start in September 2009.

Further information on the topics/research areas are available online at:
http://www.ppls.ed.ac.uk/postgraduate/CareerDevStudentship.html
Eligibility
Candidates should have an academic background in Psychology holding (or expecting to hold
by September 09) a postgraduate Masters level qualification or equivalent and be eligible to
apply for PhD level study at the University of Edinburgh. Students entering the second year
of their PhD Psychology study at the University of Edinburgh are also encouraged to apply.

Both International and Home/EU students are eligible to apply although please note that the
tuition fees provided in the studentship are at a Home/EU level only.

Application Procedure
Candidates are invited to apply through the University of Edinburgh online application system
(EUCLID); please ensure that you note your intention to apply for this studentship under the
funding section of the application form. When you submit your application, please also email
pplspg@ed.ac.uk to indicate that you have done so.

The application form and further guidance is available online at:
http://www.ppls.ed.ac.uk/postgraduate/CareerDevStudentship.html
If you have already submitted an application for PhD Psychology and would like to be
considered for these studentships, please email pplspg@ed.ac.uk to register your interest.

Deadline for application: Friday 10th July 2009
Selection Process
Short listed candidates will be invited to attend an interview at the University of Edinburgh. Successful candidates will be informed by Friday 14th August 2009