Continuity Boy

SCSMI 2012: where Kuleshov is King!

2011-12-14T12:31:00.002Z

This is a rather late notice to anybody interested in Cognitive Film Theory/Film Cognition. The annual meeting of the Society for the Cognitive Studies of the Moving Images (SCSMI) will be hosted by Sarah Lawrence College and New York University next June 13-16th in New York and the proposal deadline is TOMORROW, December 15th 2011.

This is my favourite conference for a variety of reasons:

1) It is the only place where film makers, theorists, psychologists, cognitive scientists, neuroscientists and anybody else interested in film come together to present their work, exchange ideas and contribute to the international community of film cognition.

2) All of the big names in the field attend and the small scale of the conference means that you will get the opportunity to meet them in person and exchange ideas.

3) The presentation format is incredibly generous: 25 minute presentations + 20 mins questions! Traditionally there have never been more than 3 parallel strands. This is incredibly conducive to open, sprawling and exciting discussions.

4) The society is full of generally lovely people and we always have a great time.

If you are interested in attending the deadline is TOMORROW, December 15th 2011. Details of how to submit are available here: http://scsmi-online.org/conference

Hope to see lots of the old faces and many, many new faces in New York next June.

Smooth Pursuit on BBC Breakfast

2011-10-12T15:38:00.003Z

BBC Breakfast contacted me yesterday about my eye tracking and film research. They were putting together a piece about how we can use eye tracking (and other methods such as EEG) to monitor people's experience of entertainment such as film and literature. I didn't have much input to the piece but they did start with a clip of my There Will Be Blood study. The original report of that study is available here:http://www.davidbordwell.net/blog/2011/02/14/watching-you-watch-there-will-be-blood/

The BBC Breakfast pieces can me seen here (it starts 15:22s):

http://www.bbc.co.uk/news/12550706

The clip they chose to feature was not included in my earlier discussion. It is very interesting from the eye movement perspective because it shows how our attention picks out motion and tracks it even when it is momentarily out of view. For instance, when the shot begins the viewers do not know what they are looking at so their gaze defaults to screen centre and the vanishing point of the train tracks.

The sound of a car's engine and the slight motion in the distance captures viewer attention and everybody looks in the same place.

As the camera pans right the viewers continue tracking the car. Because the car and the camera are both moving this means that the viewer's eyes may actually be stationary on the screen but they perceive the car as moving. In the real-world we would either pursue the car by rotating our eyes with it or rotate our head to keep the car at the centre of our gaze without moving our eyes. The pan in this example serves the same purpose as a head rotation.

As the car disappears behind the building the fact that the camera continues to move with the car and we hear the car's engine implies that it will reappear. Viewers try to find the car by saccading to the screen edge in anticipation of the car's reappearance.

Whenever a door or window appears in the building all viewers zoom in on it trying to catch a glimpse of the car. This high degree of attentional synchrony is expressed by the heatmap that appears periodically during the clip. If you want to know more about how this heatmap is calculated look at our paper:

Mital, P.K., Smith, T. J., Hill, R. and Henderson, J. M. (2011) Clustering of gaze during dynamic scene viewing is predicted by motion. Cognitive Computation, 3(1), 5-24 http://www.bbk.ac.uk/psychology/our-staff/academic/tim-smith/documents/Clustering_of_Gaze_During_Dynamic_Scene_Viewing_is_Predicted.pdf

The building ends and all viewers saccade to the edge that they believe the car will appear. The continued engine sounds reinforces our belief that it continues to exist and as the camera pans left and stops tracking we wait patiently. When the car finally appears our diligence as viewers is rewarded.

That one gracious shot demonstrates how a combination of slow, deliberate camera movements and choreography of action can lead to incredibly focussed viewing and induce viewers to perceive actions and details, such as the car's existence and motion behind the building, without actually showing them. Compare this shot to the long-take in my previous analysis to see how two completely different shots can lead to similar coordination of viewer gaze. Masterful!

So, that was what was going on in that brief clip from BBC Breakfast. Now all I need is to get on the couch with Sian and Bill!

ECEM 2011

2011-09-05T09:48:00.001Z

August 21-25th 2011 I had the great fortune of attending the European Conference on Eye Movements (ECEM) in Marseille, France. This conference occurs every two years and I having attended it since 2003 I can say that this year was a roaring success. The quality of the presentations was second to none, the atmosphere was fun and sociable and Marseille is fantastically charismatic city in which to hold a conference. I presented two pieces of work. On the Monday I presented a poster on using eyetracking to inform film theory (see below or pdf here) which was very well received and gave me plenty of opportunity to ramble on about Von Trier, Welles, Intensified Continuity and the Society for Cognitive Studies of Moving Images (SCSMI). The poster was presented outside in a beautifully tree-line courtyard at the University of Provence, Marseille. Unfortunately, on the day of presentation the wind had whipped up and I spent most of my presentation holding on to my poster as it attempted to fly away. The result was rather comical for the audience but, the trooper that I am, I did not let it phase me and continued as enthusiastically as always.

If you are interested in reading more about the work presented in the poster you can see details of the cognitive readings of film in my Edit Blindness paper (Smith & Henderson, 2008; JEMR; here) and my modelling paper with Parag Mital (Mital, Smith, Hill & Henderson, 2011; Cog. Comp; here). The cinematic feature analysis will appear as a chapter in the book Psychocinematics published next year.

The second presentation I gave at ECEM was in a symposium on The Perception of Dynamic Scenes which I organised. This was a very successful symposium which turned out even better than I had hoped. Michael Dorr (Schepens Eye Hospital), Halszka Jarodzka (Heerlen), Sam Wass (Birkbeck), Daniel Richardson (UCL), and Ben Tatler (Dundee) all contributed to the symposium and presented a wonderful variety of perspectives of how we attend to and perceive complex, dynamic scenes. There was a real buzz about the potential for investigating dynamic scene perception at ECEM which culminated with the symposium. Over the last few ECEM there has been a growing interest in studying eye movements on video but technical, methodological, and theoretical limitations meant that very few people attempted it. Since the last ECEM in Southampton two years ago many labs have overcome these issues through significant innovations such as those provided by the DIEM project. We are now at an exciting tipping point which I am certain will lead to a sudden upsurge of empirical investigations of visual cognition with more naturalistic, dynamic stimuli. The potential insights into human behaviour and cognition this will provide are unlimited and I am looking forward to seeing the area develop and assisting it in anyway I can.

Thanks again for all the organisers of ECEM 2011, Marseille including Francoise Vitu and her helpers and I am looking forward to ECEM 2013 in Lund, Sweden!

Hollywood calling?

2011-07-24T13:00:00.001Z

A couple of months ago, I got an unexpected email with the subject “Would you like to speak at DreamWorks Animation”. Naturally, I initially considered it a scam and replied cautiously. Their response left no doubt that it was indeed Hollywood calling.

The idea that a cognitive psychologist from London might be of interest to a Hollywood animation studio might seem odd. What the filmmakers were interested in was my recent work on how we attend to and perceive the real world.

My research usually involves showing volunteers simple patterns or photographs on a computer screen and recording how they move their eyes when trying to make sense of the image. This provides me with insights into how someone uses their eyes to sample the bits of the visual world they are interested in, stitch the details together and store them in memory. In the last few years I began applying the same methods to investigate how we watch film after the realisation that we use the same cognitive processes to watch film as we use to look at the real-world.

Films present an artificial world across a series of camera shots edited together so that only the bits important for the narrative are presented. When Shrek leaves his swamp to find Princess Fiona, for example, we only need to see him leave his house and then arrive at the Fairytale castle to work out the journey that must have happened in between. Films can create fantastical events and spaces that we can comprehend without any effort, as long as the film is edited correctly. But what distinguishes a “good” from a “bad” edit? In my research I have been trying to understand the Psychology of film viewing to answer this question.

Like all Hollywood film studios, Dreamworks Animation want to make their films as enjoyable and as effortless to watch as possible. Confusing films don't lead to big box office receipts, especially not when the film is intended for children.

At every moment during a film, the director needs to know exactly where the viewer is looking, how they are understanding the story and what they are feeling- this is no trivial task. If an edit occurs at the “wrong” time during an action sequence or cuts to the “wrong” camera position, the viewer can become disorientated and confused. In film terms, the cut is said to create a “discontinuity”. Over the 116 years cinema has existed, a suite of heuristics (rules-of-thumb) have evolved that filmmakers can use to help them avoid bad edits. These rules of continuity editing suggest, for example, that when filming a scene with two actors in a conversation, all shots of the action should be filmed from the same side of an imaginary line connecting the two actors. This 180 degree rule (named because the cameras will map out a 180 degree arc around the actors) ensures that the actors don't suddenly reverse direction on the screen across a cut and appear to be facing away from each other. Virtually all film and TV is constructed according to these rules. Watch a scene from any TV show and you will see how the cameras always stay on one side of the action.

While the continuity rules are believed to work by filmmakers the world over nobody understands why they work. Dreamworks Animation wanted to know if my experiments in film viewing could shed any light on this question.

By recording the eye movements of viewers as they watch film sequences I have been able to see which cinematic techniques succeed in guiding the viewers to the point of interest in a scene and whether a cut leads to disorientation. For example, if the action of a scene is easy to follow all viewers will watch the scene in the same way, leading to a clustering in the location of their gaze on the screen.

Puss In Boots teaser trailer with gaze location of 16 viewers from Tim J. Smith on Vimeo.

To get a sense of this gaze clustering watch this trailer for Dreamworks Animations upcoming Puss In Boots (http://vimeo.com/25033301). The gaze locations of 16 viewers are each represented as a dot and a hotspot overlaid on to the video. As the gaze of multiple people clusters together the colours become hotter. Notice how the gaze is clustered on Puss throughout the clip without taking in much of the background. This clip also uses the continuity editing rules to ensure that viewers shift their attention seamlessly across a cut. When Puss tosses his hat off the screen the cut is made right after the hat starts flying. The next shot continues the hat’s motion until caught by an enamoured admirer. Such a cut is referred to as a “match on action”. By using the sudden onset of motion to capture viewer attention and lead the eyes across the cut, the director ensures that the viewer perceives the two shots as being continuous. You can see this in the smooth shift in eye movements from one shot to the next.

At Dreamworks, I was struck with how intimately they engaged with the issues I was presenting. Their day-by-day concerns are with the minutae of film, the nuanced animation of a facial expression, the placement of characters across a cut, the correct lighting to pick out the main character. To make these decisions, though, they have to imagine themselves as their eventual viewer and, until now, they have had no way of knowing what was going in the mind of these viewers. I hope that combining some of the methods and theories from cognitive psychology with their own insights about film, they will get a clearer glimpse of this insight. I believe that studying the psychology of film will help filmmakers continue to improve upon the kinds of unique, exciting, and moving experiences that enraptured me when I was a kid and continue to fascinate us all today.

PostDoc in Neurocinematics

2011-07-18T16:34:00.000Z

http://www.aalto.fi/en/current/jobs/teaching_and_research/postdoctoral_researcher_and_doctoral_student-school_of_art_and_design-neurocine/

Aalto University School of Art and Design, Department of Motion Picture, Television and Production Design is looking for two team members to conduct research in neurocinematics:

A post-doctoral researcher (Cognitive neuroscience) AND a doctoral student (Cinema studies)

1) POST DOC EXPERIENCED IN COGNITIVE NEUROSCIENCE AND fMRI. The applicant will work together with PI, doctoral student, and associated neuroscientists at the aivoAALTO. She or he is expected to independently collect and analyze functional magnetic resonance imaging (fMRI) data. Experience in fMRI is required, and experience in magnetoencephalography (MEG), electroencephalography (EEG) and physiological measures (e.g. eye tracking) are appreciated. A degree in neuroscience, medical sciences, engineering, mathematics, physics, or equivalent is requested. An emphasis is on the scientific writing skills, and applicants are expected to present a selection of first author publications.

2) DOCTORAL STUDENT IN FILM STUDIES: The doctoral student must have a Master’s degree in film or media studies, or equivalent, with explicit interest in psychology and cognitive neuroscience, and possess a post-graduate study place in a University. PI, Doctor of Arts Pia Tikka will supervise the thesis work in neurocinematics in collaboration with neuroscientists at the aivoAALTO.

For both positions, applicants' research potential and co-operation skills will be given particular emphasis during the selection process. Application materials should include a cover letter, a curriculum vitae, a complete list of publications, and 1 page description of future research interests related to neurocinematics. In addition, reprints of publications (max 2) and reference letters (max 2) are appreciated.

Both the positions are available for two years starting on 1st September, 2011. The salary is determined by the salary system of Aalto University.

The applications are to be submitted to the Registry of Aalto University, preferably on a single pdf-file by email no later than on August 16, 2011. The right to extend the search or not to fill a position is reserved.

Note, all single pdf-files should contain applicants last name: “NeuroCine_LASTNAME_otherinfo”. In addition, the email should be named “Application: NeuroCine”.

The email address of the registry is rekry-taik@aalto.fi Applications can also be sent via mail to: Aalto University School of Art and Design, Registry, P.O. Box 31000, FI‐00076 Aalto, Finland (visiting address Hämeentie 135 C, 00560 Helsinki). The registry closes at 3.00 p.m. The application documents will not be returned.

Please feel free to direct further questions to:

Pia Tikka, Ph.D.
aivoAALTO research project
Aalto University, Helsinki, Finland
- Dept. of Motion Picture, Television and Production Design
US online phone + 1 213 785 7048
Finland mobile phone +358 50 347 7432
Skype: piatikka
e-mail: pia.tikka@aalto.fi

Due to the summer holidays in July and August phone or skype inquiries are only taken on Friday’s between 12-4 pm Finnish time (GTM +2).

Neural signature of The Uncanny valley?

2011-07-18T14:25:00.002Z

An fMRI study by a group from UC San Diego led by Ayse Pinar Saygin has investigated the phenomenon known as "The Uncanny Valley": that eerie feeling you get when watching a robot or CG animation that is attempting to be photorealistic but fails. we have all experienced this phenomenon when watching films such as Final Fantasy, or Robert Zemeckis' performance capture pieces, The Polar Express, Beowulf, A Christmas Carol and the recent film that sounded the death-knell for Zemeckis' studio, Mars Needs Moms. Given the popularity of using performance capture and even facial expression capture in films like Avatar and in the upcoming Tintin, there is a lot of interest in how to present naturalistic motion capture animation without wandering into the Uncanny Valley. The results of this new study seem to suggest that the discomfort comes from the perceptual mismatch between the authentic human motion and the inadequate appearance. We are highly sensitive to human appearance and motion and have more brain areas dedicated to the processing of these features than any other visual category. Motion capture gives us the ability to trick the brain into seeing human motion even in the absence of the corresponding appearance. This is clearly displayed in point-light walkers (check out this fun interactive demo http://www.biomotionlab.ca/Demos/BMLwalker.html). However, when the authentic biological motion is combined with an appearance that doesn't match the authenticity of the motion it results in cognitive dissonance. This new study shows the brains response in such a situation.

I am very intrigued to see how Steven Spielberg and Peter Jackson deal with this issue in the upcoming Tintin movies. In the first teaser trailer he avoided showing faces, perhaps to avoid the audience's negative response to seeing an Uncanny Tintin. In the first full trailer (below) the faces walk an interesting line between naturalism and cartoon, very accurately capturing the character of the original Herge cartoon. Perhaps Spielberg has dodged the bullet by mismatching the motion and appearance enough to avoid cognitive dissonance. We'll have to wait until Christmas to find out.

From the press release:

"[Image: Brain response as measured by fMRI to videos of a robot, android and human]

Your Brain on Androids

July 14, 2011
By Inga Kiderra

Ever get the heebie-jeebies at a wax museum? Feel uneasy with an anthropomorphic robot? What about playing a video game or watching an animated movie, where the human characters are pretty realistic but just not quite right and maybe a bit creepy? If yes, then you’ve probably been a visitor to what’s called the “uncanny valley.”

T! he phenomenon has been described anecdotally for years, but how and why this happens is still a subject of debate in robotics, computer graphics and neuroscience. Now an international team of researchers, led by Ayse Pinar Saygin of the University of California, San Diego, has taken a peek inside the brains of people viewing videos of an uncanny android (compared to videos of a human and a robot-looking robot).

Published in the Oxford University Press journal Social Cognitive and Affective Neuroscience, the functional MRI study suggests that what may be going on is due to a perceptual mismatch between appearance and motion.

The term “uncanny valley” refers to an artificial agent’s drop in likeability when it becomes too humanlike. People respond positively to an agent that shares some characteristics with humans – think dolls, cartoon animals, R2D2. As the agent becomes more human-like, it becomes more likeable. But at some point! that upward trajectory stops and instead the agent is perceived as strange and disconcerting. Many viewers, for example, find the characters in the animated film “Polar Express” to be off-putting. And most modern androids, including the Japanese Repliee Q2 used in the study here, are also thought to fall into the uncanny valley.

Saygin and her colleagues set out to discover if what they call the “action perception system” in the human brain is tuned more to human appearance or human motion, with the general goal, they write, “of identifying the functional properties of brain systems that allow us to understand others’ body movements and actions.”

They tested 20 subjects aged 20 to 36 who had no experience working with robots and hadn’t spent time in Japan, where there’s potentially more cultural exposure to and acceptance of androids, or even had friends or family from Japan.

The subjects were shown 12 videos of Repliee Q2 performing such o! rdinary actions as waving, nodding, taking a drink of water and picking up a piece of paper from a table. They were also shown videos of the same actions performed by the human on whom the android was modeled and by a stripped version of the android – skinned to its underlying metal joints and wiring, revealing its mechanics until it could no longer be mistaken for a human. That is, they set up three conditions: a human with biological appearance and movement; a robot with mechanical appearance and mechanical motion; and a human-seeming agent with the exact same mechanical movement as the robot.

At the start of the experiment, the subjects were shown each of the videos outside the fMRI scanner and were informed about which was a robot and which human.

The biggest difference in brain response the researchers noticed was during the android condition – in the parietal cortex, on both sides of the brain, specifically in the areas that connect the part of the brain! s visual cortex that processes bodily movements with the section of the motor cortex thought to contain mirror neurons (neurons also known as “monkey-see, monkey-do neurons” or “empathy neurons”).

According to their interpretation of the fMRI results, the researchers say they saw, in essence, evidence of mismatch. The brain “lit up” when the human-like appearance of the android and its robotic motion “didn’t compute.”

“The brain doesn’t seem tuned to care about either biological appearance or biological motion per se,” said Saygin, an assistant professor of cognitive science at UC San Diego and alumna of the same department. “What it seems to be doing is looking for its expectations to be met – for appearance and motion to be congruent.”

In other words, if it looks human and moves likes a human, we are OK with that. If it looks like a robot and acts like a robot, we are OK with that, too; our brains have no difficulty processin! g the information. The trouble arises when – contrary to a lifetime of expectations – appearance and motion are at odds.

“As human-like artificial agents become more commonplace, perhaps our perceptual systems will be re-tuned to accommodate these new social partners,” the researchers write. “Or perhaps, we will decide it is not a good idea to make them so closely in our image after all.”

Saygin thinks it’s “not so crazy to suggest we brain-test-drive robots or animated characters before spending millions of dollars on their development.”

It’s not too practical, though, to do these test-drives in expensive and hard-to-come-by fMRI scanners. So Saygin and her students are currently on the hunt for an analogous EEG signal. EEG technology is cheap enough that the electrode caps are being developed for home use.

The research was funded by the Kavli Institute for Brain and Mind at UC San Diego. Saygin was additionally supported by the Cal! ifornia Institute of Telecommunication and Information Technology (Calit2) at UCSD.

Saygin’s coauthors are Thierry Chaminade of Mediterranean Institute for Cognitive Neuroscience, France; Hiroshi Ishiguro of Osaka University and ATR, Japan; Jon Driver of University College London; and Chris Firth of University of Aarhus, Denmark.

Media Contact: Inga Kiderra, 858-822-0661, or ikiderra@ucsd.edu"

What is the point of Art? A view from neuroscience

2011-07-06T10:22:00.000Z

In conjunction with Jericho House (http://www.jerichohouse.org.uk/) there will be a talk by Prof Colin Blakemore, FRS:

What's the Point of Art? A view from neuroscience.

Time: 7-8pm

Date: 20th July 2011

Location: Lecture theatre, 33 Queen Square

There will be a discussion after the talk, led by Prof Geraint Rees.

Prof Sophie Scott will chair the event.

The talk is open to all. Please register by sending an email to sophie.scott@ucl.ac.uk

What's The Point of Art?

In the aftermath of the recent cuts arguments about arts funding are becoming increasingly heated, yet crucial discussions as to the value and place of art in our world are distinguished by their absence. As the wider society experiences the kind of structural economic changes unseen in the UK for sixty years, the time has never been riper for a serious investigation of the role of art in our lives, and of its relationship with the individual, the state and the market.

For 2011, in collaboration with University College London, we are developing a sequence of six events on the theme ‘what is the point of art?’, each featuring a single speaker, to take place in London on dates throughout the year. The aim is to have accumulated by the end a compelling portfolio of perspectives on the value of art in our society.

The Illusion of Continuity, Berliner and Cohen (2011)

2011-07-05T11:23:00.003Z

How we perceive spatiotemporal continuity across edited sequences of film is clearly a major interest of mine (hence the blog name!). Several theorists have written about this topic going all the way back to Munsterberg (1916), Hochberg & Brooks in the 70s and early 80s, and the pioneering Change Blindness studies of Levin & Simons in the 90s. In my thesis I discussed in depth the paradox of perceiving continuous visual scenes across visually discontinuous shots. For instance, why is a series of shots all filmed from the same side of an action such as a conversation perceived as being spatially continuous where as a shot that crosses to the other side of the action leads to spatial confusion? This filming and editing convention is known as the 180 Degree Rule. You can see clear demonstrations of it in this video:

Now a recent paper on the perception of continuity in film by Berliner and Cohen (2011) wonderfully brings together psychological evidence and film theory to provide an accessible and insightful overview of the topic. I highly recommend this article if you are looking for a quick reader on continuity perception.

Berliner and Cohen (2011) clearly outline the psychological motivation for the continuity editing rules but also acknowledge the role of intuition and creativity of filmmakers in choosing the right techniques to convey their particular story.

"Continuity conventions have remained relatively stable for about ninety years. The primary reason for their stability is not, as some scholars think, Hollywood’s marketing dominance or other externalities but rather that the early filmmakers who first developed the conventions were guided by their intuitive understanding of space perception and the reactions of cinema spectators. Just as expert pool players learn— not through direct study but intuitively, through trial and error—the principles of Newtonian physics that govern pool playing, as well as matter and energy generally, the filmmakers in the early twentieth century who first developed the conventions of the classical editing system, without directly studying psychology, discovered the structure of human perception."

Berliner, T. & Cohen, D. J. (2011) The Illusion of Continuity: Active Perception and the Classical Editing System.”Journal of Film and Video 63.1: 44-63.

Interview on CBC radio Canada

2011-06-13T14:56:00.002Z

I had a great conversation with Michael Bhardwaj, the science correspondent for CBC radio Canada last week about eyetracking and film, the There Will Be Blood research, and my visit to Dreamworks Animation. Michael did a great job of describing my work on his radio show last Friday (10/06/11) and was kind enough to send me an mp3.

Enjoy

http://www.bbk.ac.uk/psychology/our-staff/academic/tim-smith/files/Movies.MP3/at_download/file

Guardian Science podcast

2011-06-06T11:58:00.003Z

I was very fortunate to have the opportunity to discuss my research with Alok Jha on the Guardian Science Weekly podcast. If you would like to hear my views on visual attention, eye movements, the film experience and how it relates to magic check out the podcast:

http://www.guardian.co.uk/science/blog/audio/2011/jun/06/science-weekly-podcast-music-movies

If you are interested in learning more about my research you can find my publications and links to my research projects on my website.

DIEM and thanks for all the hits

2011-02-24T09:36:00.002Z

My guest post on David Bordwell's blog last week was a roaring success. I could never have imagined that it would capture the interest of so many people across so many disciplines. You can get a sense of the interest by looking at the comments and statistics for the main eye movement video. In the week the post has been up it has been viewed 145,000 times and pages embedding the video have been read 728,000 times! The video cropped up on twitter (thanks Roger Ebert and others), facebook, numerous blogs, websites and newspapers. I am incredibly happy that my research reached out to film makers, theorists, and eager consumers to inform their appreciation of film. Hopefully, you can all now get a sense of how miraculous and complex our perception of film is and how we can inform our understanding by applying methods from empirical psychology.

I plan to build on the momentum created by the blog post by posting similar cognitive readings of films here on my own blog. In the meantime, I can point you to my existing publications on the topic:

For information on the Dynamic Images and Eye Movement project (DIEM) and its analysis of the influence of visual and cinematic features on how we watch movies as presented in my analysis of There Will Be Blood, check out:

Mital, P.K., Smith, T. J., Hill, R. and Henderson, J. M. (in press) Clustering of gaze during dynamic scene viewing is predicted by motion. Cognitive Computation

On how we perceive film and the issues related to continuity, read: Smith, T.J. (2010) Film (Cinema) Perception. In E.B. Goldstein (ed.)The Sage Encyclopedia of Perception.

On the illusion of the "invisible edit" and how it relates to natural attentional shifts when watching film, see: Smith, T.J. and Henderson, J.M. (2008). Edit Blindness: The relationship between attention and global change blindness in dynamic scenes. Journal of Eye Movement Research, 2(2):6, 1-17.

Finally, if you want to see more of the DIEM eye movement videos, new videos as they are created and download the analysis software (i.e. CARPE) go to the DIEM project page and subscribe to our Vimeo channel. As a taster, here is a showreel from the DIEM videos. Enjoy!

Guest blog on davidbordwell.net

2011-02-14T15:43:00.002Z

As mentioned in my last post, David Bordwell asked me to write a guest post on his much read blog. Skip over to davidbordwell.net to see my overview of how recording eye movements during film viewing can inform our understanding of how we watch and make sense of films. As an example I analysed a sequence from PT Anderson's There Will Be Blood and visualised the gaze data using tools created as part of the Dynamic Images and Eye Movements (DIEM) project. To give you a sneak preview, here is a "peekthrough heatmap" made from the gaze data of 11 viewers.

Details about how this was created and what it tells us can be found on the blog post.

There Will Be Blood + eye movement peekthrough from TheDIEMProject on Vimeo.

Bordwell on Eyes

2011-02-07T10:56:00.002Z

Cognitive Film Theorist extraordinaire, David Bordwell has recently posted two wonderful blog posts about the power of eyes in film. His first post on emotional communication and acting in Fincher's The Social Network is a brilliant introduction to the psychology of facial expressions and its subtle mastery by the actors. A very fun and clever exercise in how we can use cognitive psychology to inform our understanding and analysis of film.

http://www.davidbordwell.net/blog/?p=12186

In David's second post he extends the discussion to a topic very dear to my heart: the eye movements of film viewers.

http://www.davidbordwell.net/blog/?p=12332

By providing an introduction to eye movements and summarising Yarbus' work on the influence of task on eye movements during picture viewing he provides the foundations from which we can start hypothesising about the relationship between film form and viewer attention. In next week's post, David has given me the honour of building on this foundation by explaining first-hand some of my empirical research into film viewing utilising my eyetracking methods. Watch this space to find out more!

Horizon - Is seeing believing?

2010-10-27T13:19:00.000Z

BBC TWO in the UK broadcast a wonderful factual programme on sensory illusions (Horizon, 9pm, 18/10/10). The programme covered many well known sensory illusions such as the McGurk effect, multimodal illusions such as the effect of sound and colour on taste perception, magic tricks such as Gustav Kuhn's disappearing ball illusion, and the perceptual completion of impossible figures. If you are interested in sensory illusions the programme provides a nice, succinct introduction. The programme also goes on to talk about the science of illusions and how illusions allow us to understand the cognitive processes and neurology of perception.

The programme also introduced the phenomenon of Synaesthesia in which perception through one modality (e.g. vision) triggers perceptual experiences in another modality (e.g. taste). We all have some synaesthetic experiences such as smells triggering specific visual memories but for people with more developed synaesthesia these cross-modal percepts are richer and more complex. Synaesthesia is a fascinating research area which up until a few years ago was dismissed by the Psychological research community as hokum. Thankfully a new generation of researchers such as Jules Simner (Edinburgh) have used clever behavioural measures combined with neuroimaging to show that synaeasthesia is a real phenomena and one which can seriously advance our understanding of sensory perception.

Viewers in the UK can catch the programme on the BBC iPlayer:

http://www.bbc.co.uk/i/vhw1d/

For other recent public presentations of sensory illusions see the work of Susanna Martinez-Conde (Barrow institute), the special issue of Scientific American, and the Illusion of the Year contest at the Vision Sciences Society annual meeting.

Discontinuity Boy

2010-10-27T13:10:00.001Z

After 13 years in Edinburgh, a BSc. in Artificial Intelligence and Psychology, a PhD in Cognitive Science, and a couple of post-docs I have finally left! 13 years of continuity have come to an end. This sudden discontinuity has been brought about by my move to Birkbeck, University of London to take up my first lectureship in Psychology. Thank you to everybody who made my time in Edinburgh so enjoyable, my friends, colleagues, collaborators, and students. And thank you Birkbeck for offering me this big break just when I needed it.

My new contact details are :

Tim J. Smith, Ph.D.
Lecturer
Rm 501d
Department of Psychological Sciences
Birkbeck, University of London
Malet Street,
London WC1E 7HX

Tel: +44(0)20 7631 6503
Fax: +44 (0)20 7631 6312

e-mail: tj.smith@bbk.ac.uk
url: http://www.bbk.ac.uk/psyc/staff/academic/tsmith

Don't worry, this does not signify the end of 'Continuity Boy'. Far from it. I will continue updating this research blog and I look forward to regaling you with details of the research coming out of my lab.

Amsterdam PhD Position Opening Absorption in Film Viewing

2010-08-03T09:16:00.001Z

A great opportunity to work with/learn from one of the great minds in Cognitive Film Theory:

>>>
The Amsterdam School for Communications Research currently have a four year PhD vacancy.

We are looking for a researcher who has knowledge of (cognitive) theories of narrative and film and wants to study experiential states in film viewers. Experience in psychological experimentation is also absolutely necessary. Interest in manufacturing film materials for the experiments is a recommendation.

A scholarship sufficient for costs of living and tuition is related to the position.

Please pass this announcement on to potential candidates. Deadline for applications is Sept. 12 2010. For details, applications and all correspondence see:

http://www.ascor.uva.nl/vac4.html

Ed Tan

Amsterdam School for Communications Research

The other type of continuity

2010-05-26T20:05:00.002Z

Along with cinema there is another medium that is obsessed with continuity: comics. The universes created by comic giants DC and Marvel have strictly enforced continuity so that super heroes like Superman can pop up in other characters plot arcs such as Batman and all actions fit within their own overarching stories.

The lengths that fans, writers, artists, and licence holders go to to maintain continuity within comic universes (along with the related Sci-Fi and fantasy universes such as Star Trek, Star Wars, and Doctor Who) is remarkable. The recent spate of Marvel movies including Iron Man, The Hulk, and next year's Thor have shown how universe continuity can be maintained in cinema and create innovative sequels/crossovers. Whether or not they work for the general public is yet to be seen. I personally love the Avengers super-hero team but is the world really ready for cinematic versions of AntMan or the Wasp?

I guess we'll find out over the next few years but for now lets enjoy a musical ribbing of the whole phenomenon.

Cannes 2010!!

2010-05-13T20:09:00.003Z

Greetings from the Cannes Film Festival 2010! After years of eagerly reading about all the gossip, film news, and screenings at Cannes I finally got the opportunity to attend the festival. Friends Calum Waddell and Naomi Holwill from new Edinburgh based production company, High Rising Productions (http://www.highrisingproductions.com/) asked me to join them on their Cannes experience and help out promoting some new film projects. How could I resist!

My first impressions of Cannes have been surprise and awe. You really get a sense of how much film is an industry when you are wandering around the film market and attending the market screenings. The glossy public face of the festival, the Official Competition is really just the polished face of what is really a massive buying and selling market for films. Cannes is completely mis-labelled as a festival as only industry representatives or press can get access to the 'festival' and unless you are pretty high up in the industry most of the competition and red carpet screenings will be out of bounds. Most people attending the festival spend their time in the many tiny impromptu screening rooms scattered around hotels or the palais. This year there are over 1400 screenings during the two weeks with 40 cinemas showing films simultaneously!!! Most of the films will be receiving their world premiers here at Cannes and almost all of them are completely unknown commodities. There is a great chance of stumbling across some real gems that may not make it to your local multiplex for a couple more years. Unfortunately, there is a much greater chance of sitting through some absolute dross :)

Whilst I'm here I'll try to post some reviews as I go along. Lets kick off with a couple of British gems and a few stinkers.

2B, directed by Richard Koehling.

I have to confess to being a bit of a Sci-Fi geek (was it not evident from this blog already!) and loving all things AI. The synopsis for this film got me excited: set in a near future New York, a brilliant scientist is killed by the first post-human that he personally created and attempts to transfer his mind into a new body so that he can live forever. The premise is good, if rather familiar and could have made an intriguing, contemplative sci-fi film. Sadly the budget appeared to be micro and while containing some recognisable faces from US TV, Kevin Corrigan (Fringe) and James Remar (Sex and the City) there was either too little budget to afford action sequences of no inclination to include them. Instead the film consists of a series of philosophical debates on the nature of human life, mind and soul. The ideas are good but the execution is without vision and ultimately tedious.

Edinburgh Informatics department, where I studied Artificial Intelligence has a DVD library devoted to all things vaguely related to AI, cybernetics and robots. 2B would definitely make it into their DVD library. I doubt it will make it into anybody elses though.

Redline by Takeshi Koike

Redline is for those of you who love your Japanese animation loud, fast, and intense. After watching this cinematic hit of amphetamines I felt like I needed a lie down! Koike, the director of the Animatrix section 'World Record' has crafted future universe populated by bizarre humanoid characters and an intense form of car racing where anything goes. The story follows a racer, Sweet JP who is notorious for race fixing. After almost dying in a race he unexpectedly finds himself called up for the Universe's most intense race, the Redline which is to be held illegally on the hostile Roboworld. As he prepares for the race a romance blossoms between JP and fellow racer, Sonosee a typical manga heroin with massive eyes, disproportionate breasts, a school girl innocence and a huge car. The action culminates in the spectacular Redline race in which Wacky Races-esque competitors jostle for position whilst also fighting to stay alive against missile attacks from the Roboworld denizens, mecha-suited warriors, giant gelatinous monsters and huge space lasers.

This film is an intense shot of traditional cell animation with a huge dose of Japanese surrealism. Highly fun if you have ADHD but possibly a bit too intense for the general viewing public. Not to be recommended if you suffer from an existing heart condition.

New Scientist and the 1/f structure in film editing

2010-02-17T23:50:00.004Z

My research on eye movement behaviour during film viewing and its relationship to editing style is mentioned (briefly) in the print edition of New Scientist this week (issue 2748) and on-line here

http://www.newscientist.com/article/mg20527482.600-dawn-of-neurocinematics.html

and here

http://www.newscientist.com/article/mg20527483.900-cinemaths-grabs-our-fickle-attention.html

The New Scientist article is in response to James Cutting's (Cornell) recent Psychological Science article on the 1/f structure in film editing. The scale of James' study is unbelievable. He and his collaborators basically created their own handcoded cinemetrics (http://www.cinemetrics.lv) database identifying every cut in 150 films! They then used this database to perform incredibly clever pattern analysis to look for repeating structures in adjacent shot lengths. They found that over the last 70 years Hollywood has been evolving towards a 1/f structure. I'll leave the technical explanation of what this means to James (below) but the basic gist is that shot lengths have begun to cluster in sequences of localised repeating lengths. Similar patterns have been observed in all aspects of nature such as tides, music, dance, the spatial frequency of the visual world, and neural firing rates. 1/f is thought by some to be a primitive pattern of processing in the human brain and influences attention fluctuation. James hypothesises that the adoption of a 1/f structure in editing structure may enable film to synchronise with viewer attention and create a more harmonious viewing experience. Of course, given that this was also the hypothesis I put forward in my Ph.D. thesis I am overjoyed by James' finding :) Let's hope this ushers in a new era of mathematically advanced cinemetrics and introduces more people to the area of Cognitive Film Theory.

Attention and the Evolution of Hollywood Film

James E. Cutting, Jordan E. DeLong, and Christine E. Nothelfer

Psychological Science (in press)

http://pss.sagepub.com/content/early/2010/02/04/0956797610361679

Reaction times exhibit a spectral patterning known as 1/f, and these patterns can be thought of as reflecting time-varying changes in attention. We investigated the shot structure of Hollywood films to determine if these same patterns are found. We parsed 150 films with release dates from 1935 to 2005 into their sequences of shots and then analyzed the pattern of shot lengths in each film. Autoregressive and power analyses showed that, across that span of 70 years, shots became increasingly more correlated in length with their neighbors and created power spectra approaching 1/f. We suggest, as have others, that 1/f patterns reflect world structure and mental process. Moreover, a 1/f temporal shot structure may help harness observers’ attention to the narrative of a film.

Bang! – The science behind the show

2009-09-28T11:49:00.005Z

In our segment on Bang Goes the Theory (Mon 28^th Sept 7:30pm BBC 1; catch it on BBC iPlayer if you missed it) Peter Lamont and I discussed some of the everyday psychological phenomenon used by magicians to fool our perception and create illusions. There were two main phenomena demonstrated in the piece: Inattentional Blindness and Change Blindness. If you want to try the change blindness test used in the show for yourself skip down to the 'Now its your turn!' section below.

Inattentional Blindness

In the piece, Peter demonstrated how a magician can make an object seem to disappear by controlling where you are looking. We have the impression that we see all of the visual world in great detail at the same time because as soon as we turn our attention to an object in the world we can see it. However, this is an illusion created by our brains to overcome the limitations of our eyes. Our eyes are actually only able to pick out visual detail from a very small portion of the world at any one moment. This is because the light sensitive surface at the back of the eye, known as the retina, has the greatest concentration of cells in a small region around its centre, known as the fovea. The light landing on the fovea comes from a region out in the world roughly the size of your thumb nail held out at arm’s length. When we look at an object we move our eyes so that the light reflected off an object lands on the fovea. The light reflected off objects away from the centre of our attention land on less sensitive parts of the eye resulting in a lower quality image. We may have the impression that we can see everything but in reality if our eyes are not pointing at an object all we can see is a blurry image with poor definition.

However, because we move our eyes on average 3-5 times every second our brains use all the information we get across this sequence of fixations (when the eyes are still) to piece together a detailed impression of the world. Our perception of the world is constructed over time from minimal detail and we assume that we see more than we actually do. This assumption is used by magicians to make you think that if you didn’t see something happen, such as a ball being pocketed, then it didn’t happen!

In the Bang! piece, Peter uses misdirection to ensure that you are not looking at the hand which is hiding the object. He does this by using cues to encourage you to look elsewhere. These cues may involve a sudden flourish of a hand, the waving of a wand, directing his eyes to an object, his posture and referring to an object by name. All of these cues direct our attention to one object whilst misdirecting us from the hand actually performing the trick. Social cues such as these are so powerful and we respond to them so consistently that magicians are able to reliably influence the attention of individuals or entire audiences. The result is that we are blind to the method of the trick because we failed to attend to it. The most famous inattentional blindness demonstration is Simon’s & Chabris ‘Gorillas in our Midst' experiment in which they made viewers fail to notice a person in a gorilla suit by focussing viewer attention on basketball players in the same scene. For a demonstration of the Simon’s & Chabris demonstration and a similar magic trick utilising inattentional blindness see the Colour Changing Card Trick we previously described on this blog.

For further discussion of the use of natural inattentional blindness phenomenon used by magicians see Peter’s book and article below.

Lamont, P. & Wiseman, R. (1999). Magic in theory: an introduction to the theoretical and psychological elements of conjuring (Hatfield: University of Hertfordshire Press)

Lamont, P., Henderson, J. M., & Smith, T. (in press). Where science and magic meet: the illusion of a ‘science of magic’. Review of General Psychology. (* e-mail Peter for a pre-print)

Change Blindness

Another everyday phenomenon used by magicians to create illusions is Change Blindness. Change Blindness refers to when a detail of the visual world changes without us noticing. For example, the most famous change blindness demonstration was performed by Simons and Levin (1997). In this demonstration they made strangers in the street fail to notice when the person they were talking to changed into a different person. They achieved this by hiding the change behind a door which was rudely carried between the stranger and the experimenter during the conversation. See a demonstration of the effect here. And the original article here.

Change Blindness occurs for three reasons: 1) either we don’t perceive the object before it changes, 2) our attention isn’t attracted to the change, or 3) we don’t compare the changed object to our memory of the original object. The first reason is similar to inattention blindness in that it is a failure to take in enough detail of the object. This is probably due to not looking at the object and only taking in sketchy information about the object from our peripheral vision (away from the fovea). The second reason is essential for us to experience change blindness. In the Simons and Levin example the change was hidden behind the door. If the door hadn’t been present and somehow one person had spontaneously changed into another person the change itself would have captured our attention. This is because the change creates visual transients: sudden unnatural changes in the light landing on our retina. In nature visual transients are often caused by dangerous events such as rapidly approaching predators or unexpected object suddenly lunging towards us. As a consequence our visual system is tuned to transients and responds by directing our eyes towards them. Our faith in the our ability to automatically response to such changes is so great that if these transient are hidden we often fail to check whether a change has happened and fail to notice them.

In the Bang! piece we used a change blindness demonstration to show how we rely on the visual world to tell us what has changed. The method we used to hide the change was more subtle than the door example: we used the viewer’s own eye movements. Every 200-300ms we rapidly shift our eyes in order to look at a new object. These eye movements are called saccades. We generally do not blink during a saccade so light continues to land on our retinas. Saccades are so rapid that the light projected into our eyes blurs across our retina. Clearly we do not perceive this blur as otherwise every time you moved your eyes – such as right now, as you are reading this text – we would experience flashes of blurring. To ensure we don’t see the blur our visual system stops processing the light during a saccade. We are effectively blind for the 20-50ms it takes to shift the eyes to a new stable position! If a change is timed to coincide with a saccade the visual transients associated with the change are hidden and our attention is not drawn to it. The change is hidden and the viewer is completely unaware that anything has happened.

In our lab we use an eye tracker to monitor people’s eye movements and detect when a saccade occurs. An eye tracker uses a high-speed infra red camera to locate the viewer’s pupil and record its movements. From the movement of the pupil we can work out where the viewer is looking on a computer screen. In the Bang! piece we showed Dallas a photograph of a real scene and changed objects in the scene as he moved his eyes. For anybody else watching the scene, the changes are immediately apparent because the changes are not timed to coincide with our eye movements. The changes create visual transients which capture out attention. But for Dallas it is much harder for him to detect the changes as the transients are hidden and instead he has to rely on his memory to check what has changed. As our memory for a visual scene is not perfect we often fail to detect changes either because we didn’t store enough information about the objects in memory before the change or we fail to compare the object to our memory. This demonstration shows how minimal our perception of the visual world is, how little information we store in our memory, and how we rely on the visual world itself to tell us when something important has happened.

Now its your turn!

Without an eye tracker we can create similar change blindness effects by hiding the visual transients behind a flicker. I have recreated the change blindness test performed by Dallas by repeatedly flickering the photograph. Every so often an object in the scene will change. Your task is to watch the first video below and detect how many changes occur.

Did you get the right answer? Now watch what happens when we make the changes without the flickers:

The changes seem to ‘pop out’ because the visual transients capture our attention.

The list of changes and their times in the video are written below in invisi-text. Uncover the answer by highlighting the white text.

*Hold down the left mouse button here and drag it to the bottom à

9 sec = Plane in centre appears

10 sec = VUE sign disappears

12 sec = Top floors of the tower block on the right disappear.

14 sec = Safety barriers change colour

16 sec = Man and child change to a woman

18 sec = Rear wheel of left-most bike disappears

20 sec = lowest branch of the tree disappears

21 sec = Man appears next to dustbin

23 sec = Bus disappears

25 sec = Left tower block disappears

*ß finish highlighting text here.

The first time you viewed the scene you may have found it easier to detect some changes rather than others. Dallas had the same experience during his version of the demonstration. The variability in change detection is due to where your eyes are at the time of the change and where they have been up to that point. Work in our lab has shown that you are more likely to detect changes to objects that you are looking at or moving your eyes to compared to objects you haven’t looked at (Henderson & Hollingworth, 1999). When the changes happen without being hidden by a flicker or an eye movement these factors don’t matter. All changes are equally likely to capture your attention. This difference tells us that what you perceive in a visual scene and how the details are stored in memory is very closely related to what you look at. You may think you are aware of peripheral details in a scene but if you do not look at them you will not be able to detect when they change.

These phenomena, inattentional blindness and change blindness are key tools used by magicians to control our visual experience and create fantastical effects. However, they are not exclusive to magic as they are a natural product of how our visual system works. We fail to detect changes or see details of our visual world during every day life. It is only when this failure is brought to our attention during demonstrations like the one above or during magic tricks that we are experience surprise. By looking at these phenomena and how magicians utilise them we can learn more about how these operate in every day life and how much our experience of the visual world is an illusion.

For further information on using eye tracking to investigate change blindness look at the papers below and the rest of the research coming out of our lab:

http://www.psy.ed.ac.uk/people/jhender9/Lab/

Henderson, J. M., & Hollingworth, A. (1999). The role of fixation position in detecting scene changes across saccades. Psychological Science, 5, 438-443

Simons, D. J. (2000). Attentional capture and inattentional blindness. Trends in Cognitive Sciences, 4, 147-155.

Bang! Goes the Theory

2009-09-23T13:55:00.002Z

A few weeks ago I had the great privilege of hosting the BBC in our Visual Cognition lab. The BBC filmed a piece for the new science show, Bang! Goes the Theory (Mondays 7:30pm, BBC1). In the piece my colleague Pater Lamont, a psychologist and talented magician explained some of the psychological phenomenon utilised by magicians during close-up magic and then I demonstrated how these phenomenon can be investigated using eye tracking. The whole piece was filmed in and around Prof. John Henderson's Visual Cognition lab.

The piece will be broadcast on Monday 28th September BBC1 at 7:30pm (10:35pm, BBC1Wales). If you miss it you can view it on the BBC iPlayer (UK only). Catch it, I'm sure it will be fun and informative.

Film as a source of visual knowledge

2009-09-04T13:14:00.003Z

A colleague of mine, Dorota Ostrowska from Birkbeck has written an article on the potential for interdisciplinary film research. As a case study Dorota has used my PhD thesis. Her summary of my research and its implications for broader film studies is very insightful and complimentary. I especially enjoyed her discussion of the implications of my cognitive theory of continuity editing on the classical socio-political interpretation of the 'Hollywood Style' of film editing:

(pg. 108) "Smith’s research offers us a new insight into the view of continuity

editing as a basis of bourgeois and capitalist ideology. His findings show a

psychological basis for continuity editing, revealing its reliance on some

powerful cognitive mechanisms. Such an understanding of continuity

editing does not make the ideological tenets of the cinema that it underpins

right. Instead, it presents us with a different paradigm (scientific and empirical)

to explain why, even today, and in spite of vehement criticism, continuity

editing is still a predominant mode in audiovisual production across

the world. The reasons for the lasting success of this type of editing may be

as much cognitive and perceptive as they are ideological. Furthermore,

Smith’s conclusions change the status of certain editing practices, such as

continuity editing, from a series of arbitrary choices imposed on audiences

by editors working under pressure from money-hungry studios, into

choices based on the cognitive expectations of these audiences. Smith’s

research gives us better understanding why continuity editing has been so

pervasive and the cognitive basis of its ‘naturalness’ to spectators."

This is indeed the implications of my theory and empirical evidence. Although, as a cognitive scientist I would never have stated these implications so boldly as Dorota :) I do not dismiss the political, ideological, or practical forces influencing the path of film form evolution. The intention of my research is to add an extra influence to the evolution, namely cognitive compatibility. I will leave it up to film theorists such as Dorota to decide what this means for classic film theory.

I do not agree with all of Dorota's interpretations of my research but I think the article is very thought provoking and it is nice to see my research influencing others.

Film as a source of visual knowledge in informatics, architecture and music

Ostrowska, D. (2009), Studies in European Cinema, Vol. 5 (2), pgs. 105-116.

This article is a series of questions and reflections, which grow out of my recent

research projects which centred on the issue of interdisciplinarity in relation to

film studies and cinema. Through my work at the Cinema Interdisciplinary

Network (CINET), which we have been running at the University of Edinburgh

since 2005, I have come across a number of academics and researchers who work

with cinema or use moving images in their research but who are not film studies

scholars and are not working in the areas more traditionally associated with film,

such as literature or cultural studies. Could the CINET members’ engagement with

cinema yield some new insights about the object of their study for film scholars?

What does it mean that cinema is becoming a repository of visual knowledge?

How is this knowledge to be treated and organised? What does it mean that film is

becoming not only an aesthetic object but also a tool to conduct intellectual analysis?

What are the difficulties and pitfalls of such research projects? In order to

address these questions, I will discuss two projects presented in the context of

CINET – ‘An Attentional Theory of Continuity Editing’ which is Tim Smith’s doctoral

project completed in the department of Informatics (Computer Science) and

‘Inflecting Space: Correlating the attributes of voice with the character of urban

spaces’, which is a collaboration between architecture and music.

http://www.atypon-link.com/INT/doi/abs/10.1386/seci.5.2.105_1

The educational power of film

2009-08-11T17:03:00.002Z

A new journal article published this week in Psychological Science asks whether watching popular Hollywood films depicting historical events can enhance learning of historical facts. This is a question that has been asked many times in the past and educators often assume that there is a benefit to watching video reenactments, not least of all because they are entertaining. This study aimed to test whether there can be both benefits (when the hostorical informationd epicted in the film is accurate) and costs (when the details are inaccurate) to using historical films in a history lesson. They used selection of excerpts from well-known feature films including The Last Samurai (Ed Zwick, 2003), Amistad (Steven Spielberg, 1997) and Glory (Ed Zwick, 1989). They recorded better memory for historical facts when the students watched a film version but only when the film depicted the facts accurately. When the facts were inaccurate the false details in the film sometimes take precendence over the accurate details taught to the students using traditional means.

This is a very interesting study as it shows both the strengths of using films to teach history but also the weaknesses. Any educator considering using a feature film in their lessons should consider that the primary purpose of a feature film is to entertain, not be historically accurate. The most entertaining way of telling a story is rarely the most authentic. Film makers often distort the truth to increase drama and excitement. As outlined in the study below, educators should make their students aware of this so that they don't believe everything they see on the screen.

I personally learned this lesson the hard way by very confidentally answering "Ben Hur" to the question "Who helped Jesus with the cross?" in a Religious Education class. Damn artistic licence!

(*according to the New Testament the answer is Simon The Cyrene...... not Charlton Heston)

Using Popular Films to Enhance Classroom Learning: The Good, the Bad, and the Interesting

Butler, A. C., Zaromb, F. Z., Lyle, K.B. & Roediger, H. L. III (in press) Psychological Science

ABSTRACT—Popular history films sometimes contain major historical inaccuracies. Two experiments investigated how watching such films influences people's ability to remember associated texts. Subjects watched film clips and studied texts about various historical topics. Whereas the texts contained only correct information, the film clips contained both correct information (consistent with the text) and misinformation (contradicted by the text). Before watching each clip, subjects received a specific warning, a general warning, or no warning about the misinformation. One week later, they returned for a cued-recall test about the texts. Watching a film clip increased correct recall of consistent information relative to recall of the same information when subjects did not see the clip. However, when the information in the film contradicted the text, subjects often (falsely) recalled misinformation from the film. The specific warning substantially reduced this misinformation effect. Teachers should use popular history films with caution and should warn students about major inaccuracies in the films.

http://www3.interscience.wiley.com.ezproxy.webfeat.lib.ed.ac.uk/journal/122521484/abstract

Antichrist and the anti-cut

2009-08-01T14:18:00.004Z

Lars von Trier’s Antichrist is an unsettling film for many reasons. It will be discussed in depth for its more obvious unsettling traits such as its intense graphic violence and self-mutilation and lack of a clear message especially with regard to gender relationships and the nature of evil. Some of these traits, such as the horror imagery, settings and plot-devices may seem unique within Trier’s work to date. However, there are other, more subtle methods used to unsettle the viewer which Trier has been refining since Dogme 95. I’m speaking, of course about his use of discontinuous editing.

Antichrist is a two-header centring around the grief of a married couple (Charlotte Gainsborough and Willem Dafoe) after the loss of their young son during the film’s prologue. The majority of the film depicts the two main actors whilst they try to come to terms with the loss and confront the wife’s fears about their holiday cottage in the woods, Eden. The film employs stunning high-speed black and white cinematography to add incredible beauty to some brutal scenes including the son’s fall to his death and the copulation of the couple during the prologue. These scenes are offset by, uncharacteristically glossy scenes of the couple interacting initially at home and then at the cottage. These scenes have a “Hollwood”-esque appearance: very beautifully lit, shallowly focussed with frequent focus-pulls into the shot, and shot from mostly stable camera positions. This is in stark contrast to the compositional style developed by Trier and his cinematographer Anthony Dod Mantle over their last few films culminating in computer-controlled camera and lighting system (Automavision) employed by Trier in his last film, The Boss Of It All. In a previous blog post I discussed how this system randomly shifted the camera position, zoom and focus of the camera. This resulted in the focal object (e.g. a talking character) drifting out of shot or being framed in such an obtuse manner that the viewer had to struggle to fixate the region of interest such as the characters eyes or mouth. The unpredictable camera movements also force the viewer to actively pursue the focal object with their eyes rather than stablise their gaze (i.e. fixate) the object. Even before Automavision, Trier’s use of hand-held cameras and improvised staging and acting in earlier films (see Idioterne and Dancer in the Dark) meant that the viewer always had to be active in their viewing of his films. Unlike Hollywood films edited according to the Continuity Editing Style (see my thesis) which keep focal objects relatively stationary on the screen and always establish predictable screen locations or the modern Intensified Continuity style (see David Bordwell’s discussion ) which use rapidly edited close shots , Trier’s films make no concessions to the viewer’s attention, actively engaging them in the staging of a scene by forcing them to search the screen for the most relevant details. In Dancer In The Dark the hand-held camera roams around the main characters often failing to keep up with changes in action. Trier compensates by cutting to different takes of the same scene with a slightly different camera position on the same actors. This creates, what is traditionally referred to as a Jump Cut: a sudden displacement of an object or actor on the screen caused by a cut to a camera position less than 30 degrees away from the original position (for an in-depth discussion of perceptual reasons for Jump Cuts see Joseph Anderson’s The Reality of Illusion). Prior to the cut, the viewer is trying to compensate for the actor’s movement relative to the screen by pursuing the actor or saccading. A sudden unexpected Jump Cut then displaces the actor on the screen requiring the viewer to reorient to the new shot. The consequence is much more active eye movement behaviour than is usually required when viewing a film edited in the Continuity style. This level of active viewing is so alien to a cinema audience that the experience can be exhausting and unsettling. Such affective response is utilised by Trier to add to the overall experience of watching his films.

At the recent Society for Cognitive Studies of the Moving Images annual meeting in Copenhagen we were treated to an advanced screening of Antichrist and Q&A with Lars von Trier. I asked him how his discontinuous editing style had come about and whether he used it deliberately to unsettle his viewer. Trier enthusiastically described the way that he and his long-term editor Molly Marlene Stensgaard had experimented in the editing suite and devised a style of cut that, whilst violating classical editing conventions, they believed accurately portrayed the mood of a scene. I referred to this cut as a Jump Cut but Trier seemed uncomfortable with this name and possibly its negative connotations. Trier explained that he didn’t consider the cut to be discontinuous because he ensured that the audio track was continuous across the cut. The camera position would suddenly shift across the cut as if it had been moving smoothly around the principal object but a portion of the movement had been omitted by the edit. However, because the camera positions were actually filmed at different times the shots either side of the cut could depict a (somewhat) continuous action and the audio track could be stitched together to make it sound as if the scene were continuous. Trier stated that he thought the most important element for creating the illusion of continuity was the audio track. As long as the audio implied that the scene was continuous the audience would perceive the scene as continuous. A Jump Cut traditionally implies an omission of time. Trier’s cut overrules this implication by using continuous audio. Not perceptual time is omitted. As such we will refer to Trier’s type of cut as an Anti Cut.

The effect of an Anti Cut on the viewer is cognitive dissonance. Cognitive dissonance comes about when a person holds two (or more) conflicting concepts in their mind at the same time. The experience of cognitive dissonance is unpleasant. We endeavour to find order in our experiences and ideas and when conflicting concepts cannot be resolved we are uncertain how to act. If these concepts come from two different sensory modalities the brain tries to reconcile the two sources and come up with an amalgamation. A classic example of this is the Mc Gurk Effect (see video below). The McGurk effect is a demonstration of how we did not evolve to process our senses in isolation. When watching someone say a word our perception of the sounds they make is informed by both the audio and way their lips mouth the word. If these two sources are in conflict e.g. the mouth says “ba” but the audio track says “ga”, we perceive something in between, e.g. “da” . This demonstrates how sensory perception is multimodal and the brain will endeavour to resolve cognitive dissonance even if it means distorting reality and creating an illusory perception.

In the case of Trier’s Anti Cut it appears that the audio track takes precedence over the visuals to create the perception of temporal continuity. This may only be possible as long as the visuals do not conflict with the audio. If the cut involved a shift in camera position and a repetition of action we would expect the viewer to perceive the repetition irrespective of how temporally continuous the soundtrack is. However, Trier typically cuts in between actions when no visual events are occurring and, therefore cannot be used to perceive the omission of time. The sudden shift in camera position and changes in actor posture, position, lighting, etc all imply a temporal omission but it appears that these are overridden by the temporal continuity implied by the audio track. Scenes edited using Anti Cuts, such as the bedroom scenes in the first act of Antichrist or the scene in which Bjork’s character’s son receives a new bike in Dancer in the Dark, create a strange, almost abstract sense of time. Many aspects of the scene imply temporal continuity such as the continuous audio and the depicted events. However, the sudden shifts in camera position caused by the Anti Cut imply discontinuity. The resulting perception may be of general temporal continuity but with residual conflicting cues which, if reasoned about may confuse the viewer and undermine the general perception of continuity.

Trier’s last film, The Boss of it All contains many examples of such conflicting cues. In order to compensate for the frequent, inexplicable camera moves caused the computer controlled camera system (Automavision) Trier had to use frequent Anti Cuts to stitch together different takes. Often the lighting and sound balance would also change wildly across such cuts. Unlike in Antichrist when the audio was carefully controlled to imply temporal continuity, the sudden changes in The Boss of it All implied temporal discontinuity. The result was the perception of a scene as a construction, clearly stitched together from takes filmed at different times. The depicted actions implied temporal continuity and if the viewer chose to engage only with this dimension and ignore the other conflicting dimensions they could perceive scenes as temporally continuous but my feeling of watching the film was that such active reconstruction of the scene required much more effort than watching a film edited according to the Continuity Style or even some of Trier’s other films.

As always, Lars Von Trier is pushing the boundaries or editing in order to experiment with our perception of film. As a film viewer I adore Trier’s introspective experiments and as a scientist I would love to see empirical investigations of these issues. What is the relationship between the different sensory modalities in film and which modality takes precedence and leads to the resulting perception of continuity. In Chapter 5 of my thesis I laid out a theory about how continuity perception in film may operate and I went on to empirically investigate time perception across cuts. This investigation was purely in the visual dimension although I acknowledged how powerful the audio dimension appeared to be and how editors placed such emphasis on audio in the creation of continuity. Direct empirical investigation of the role of audio in continuity perception is required but for now we can occupy ourselves with marvelling the masterful way Trier manipulates our experience of time and space in order to create an unsettling viewing experience.

Come and work in our lab: PhD studentships

2009-06-29T14:35:00.002Z

Would you like to come and work in Visual Cognition? Would you like to be part of John Henderson's great research lab and learn how to investigate static, dynamic, and film cognition using eye tracking?

Well we now have funded PhD studentships available.

Details below:

UNIVERSITY OF EDINBURGH
SCHOOL OF PHILOSOPHY, PSYCHOLOGY AND LANGUAGE SCIENCES

PPLS (PSYCHOLOGY) CAREER DEVELOPMENT STUDENTSHIPS
Deadline for Application: Friday 10th July 2009

A minimum of four Career Development Studentships are offered by Psychology within the School of Philosophy, Psychology and Language Sciences

Career Development Studentships are designed to provide students with teaching
experience, training, and other career development opportunities. A stipend of £10,000 per
annum will be provided in addition to tuition fees at the Home/EU level and additional
programme costs of £600 per annum. The studentships will start in September 2009.

Further information on the topics/research areas are available online at:
http://www.ppls.ed.ac.uk/postgraduate/CareerDevStudentship.html
Eligibility
Candidates should have an academic background in Psychology holding (or expecting to hold
by September 09) a postgraduate Masters level qualification or equivalent and be eligible to
apply for PhD level study at the University of Edinburgh. Students entering the second year
of their PhD Psychology study at the University of Edinburgh are also encouraged to apply.

Both International and Home/EU students are eligible to apply although please note that the
tuition fees provided in the studentship are at a Home/EU level only.

Application Procedure
Candidates are invited to apply through the University of Edinburgh online application system
(EUCLID); please ensure that you note your intention to apply for this studentship under the
funding section of the application form. When you submit your application, please also email
pplspg@ed.ac.uk to indicate that you have done so.

The application form and further guidance is available online at:
http://www.ppls.ed.ac.uk/postgraduate/CareerDevStudentship.html
If you have already submitted an application for PhD Psychology and would like to be
considered for these studentships, please email pplspg@ed.ac.uk to register your interest.

Deadline for application: Friday 10th July 2009
Selection Process
Short listed candidates will be invited to attend an interview at the University of Edinburgh. Successful candidates will be informed by Friday 14th August 2009

ECHOES

2008-07-14T09:33:00.003Z

I am proud to announce a very exciting new project I am involved in, ECHOES II: Improving Children's Social Interaction through Exploratory Learning in a Multimodal Environment.

The intention of the ECHOES project is to develop a safe, motivating, and inclusive multimodal learning environment allowing Aspergers and Typically Developing children to explore different social situations to improve their social interaction and communication skills. ECHOES II will provide a unique opportunity to explore a place in which learning happens and in which learning can be studied.

The ECHOES project is a multi-site (Edinburgh, London, Sussex, Birmingham, Dundee, Glasgow, and Cardiff), three year project funded jointly by ESRC and EPSRC via the TLRP TEL call. This project which will combine the latest and most innovative methods in participatory design of education technology with cutting-edge multimodal interfaces including motion, gesture, and facial expression capture, audio reactivity and gaze tracking within an augmented reality 3D environment. Further details on the project can be found here:

http://www.lkl.ac.uk/cms/index.php?option=com_content&task=view&id=287&Itemid=91

The ECHOES project is currently at the recruitment stage. We are looking for a range of full-time, part-time RAs and PhD students. These posts are spread across the various sites. Details are below. If you are interested in any of these posts please follow the links below and contact the people named in each advert.

*Check each advert for closing date as it varies across posts*

Full-Time Research Fellow (School of Informatics, University of Edinburgh)

The successful applicant will contribute to the design and development of an intelligent multimodal interface, using a combination of an interactive whiteboard, 3D graphics, speech synthesis, and video input technology. The post will involve development of the ECHOES software architecture and organisation of software modules developed at the other sites. Experience in software engineering and multi-modal interfaces is highly desirable.

http://www.jobs.ed.ac.uk/vacancies/index.cfm?fuseaction=vacancies.detail&vacancy_ref=3009321

Part-Time Research Assistant (School of Informatics, University of Edinburgh)

Job description same as above except RA will focus more on combining existing interface technologies, e.g. motion tracking + gaze tracking, into the ECHOES architecture.

http://www.jobs.ed.ac.uk/vacancies/index.cfm?fuseaction=vacancies.detail&vacancy_ref=3009321

Full-Time Research Fellow (London Knowledge Lab, University of London)

The successful applicant will collaborate in the development of the ECHOES II environment with other researchers, focusing particularly on the design and implementation of the learning activities and interventions along with the ‘action engine’ responsible for the selection of the activities based on user input .

http://jobs.ioe.ac.uk/Vacancy.aspx?ref=7AC-CPLKL-4665

Part-Time Research Assistant (London Knowledge Lab, University of London)

The successful applicant will collaborate in the design of the ECHOES II environment with other researchers, focusing particularly on the design and testing of the learning activities and the evaluation of the educational impact of the environment.

http://jobs.ioe.ac.uk/Vacancy.aspx?ref=7AC-CPLKL-4664

Full-Time Research Fellow (School of Informatics, University of Sussex)

The successful candidate will use participatory design methods with children in order to design and implement the interface for the ECHOES II environment, and collaborate in the develompent of the environment with the other researchers, focusing particularly on the development of 3D characters and avatars and “mirroring” the user’s actions through video input.

Enquiries should be address to Dr. Judith Good (J.Good@sussex.ac.uk).

Full-Time PhD. (School of Computing, University of Dundee)

The School of Computing invites applications for a PhD student to work on the functional communication of children with autistic spectrum disorder (ASD) and similar disabilities within a wider project to develop and evaluate a technology enhanced multimodal learning environment designed to scaffold children’s social interaction and communication skills.

The successful candidate will collaborate with other researchers who will be designing a system for the exploration and learning of social interaction skills, and will be focussing of how the ECHOES environment impacts on the interactive communication of children with ASD, specifically Asperger Syndrome.

A first class or good 2:1 Honours or Masters degree, or equivalent, in Applied Computing, Computer Science, Psychology, Cognitive Science or related discipline is essential, as are knowledge of user-centred design methods, computer vision, and an ability to organise and run studies involving children, parents and professionals. Good written and communication skills are essential.

Enquiries and application requests should be addressed to Dr Annalu Waller (awaller@computing.dundee.ac.uk ).

Closing date for applications: 15 September 2008

Completing the Circle

2008-07-14T09:30:00.000Z

The Lansdown Centre for Electronic Arts (Middlesex University) is hosting a one-day symposium focussing on novel methods, or methods newly borrowed from other disciplines, in evaluating the user's or audience's response to media such as websites, portable media (such as iPods, PSPs), pervasive games, film, videogames, technology-rich performance, interactive art. The organising committee have put out a special call for researchers using eye tracking to evaluate film or media.

For further information see:

http://www.cea.mdx.ac.uk/?location_id=59&item=31

Research Assistant in Active Viewing of Dynamic Scenes

2008-07-05T16:27:00.002Z

John Henderson and I have recently been awarded a 2-year Leverhulme research grant investigating Active Viewing of Dynamic Scenes. We are very excited about the project as it will enable us to explore issues of attentional control, eye movements, and scene processing using dynamic scenes such as naturalistic social scenes, film and television. This will allow us to build upon the initial research I have been doing in this area and provide the foundations for other research groups to follow.

The project will begin later this year and we are currently advertising for a part-time Research Assistant to join the project. Details copied below. If you are interested please contact me or John asap as the deadline is July 11th.

"Research Assistant, Edinburgh University

Active Viewing of Dynamic Scenes: Eye Movements in Video

The University of Edinburgh invites applications for a two-year, part-time (20 hours per week), fixed term Research Assistant related to a new project funded by the Leverhulme Trust titled: Active Viewing of Dynamic Scenes: Eye Movements in Video. The post requires technical expertise with computational methods in computer graphics with particular application to video. Knowledge of human visual perception and eyetracking methods is desirable. The research assistant will interact with cognitive psychologists and cognitive scientists in the research team, and should have excellent interpersonal and communication skills. Applications are welcomed from individuals with a good first degree in computer science, informatics, or related computational discipline.

Informal enquiries: Prof John M. Henderson (john.m.henderson@ed.ac.uk).

Apply online (www.jobs.ed.ac.uk, Vacancy Reference: 3009312) including a CV and statement of relevant experience. Alternatively, telephone the recruitment line on 0131 650 2511. "

MSC in Visual Cognition

2008-07-02T08:39:00.002Z

I am happy to announce that as of the academic year 2008-2009 Edinburgh University Psychology department will be hosting an MSc programme in Visual Cognition. This one-year taught MSc will give students the opportunity of learning from leading researchers in the field of Visual Cognition and Percention in Action: Prof. John M Henderson, Dr. James Brockmole, Dr. Robert McIntosh, and Dr. Antje Nuthman. Students will also benefit from the broad and varied visual cognition and eye movement research groups across the University. The taught course will culminate in a Summer research project in which the students can explore their own research ideas under the tutelage of researchers such as myself. This provides a framework under which students interested in Psychology and Cognitive Science can improve their empirical skills in the area of eye tracking and apply it to new and innovative areas. I would specifically welcome applicants who are interested in eye movement and attentional control during static and dynamic scene viewing and during film and media viewing.

Funding may be available for this MSc.

For further information follow this link:
http://www.psy.ed.ac.uk/postgrad/msc/vc

If you have any specific questions about the course or want to discuss potential research ideas contact me directly (tim [dot] smith [at] ed.ac.uk).

Attention: experimental film project

2008-03-18T17:38:00.001Z

How do we watch film? How does the way that I watch a film differ from the way you watch a film? If we were to make a film of the way I watch a film, what would it look like?

These are some of the questions a group of Design and Digital Media MSc. students under my mentoring are exploring in an art installation entitled Attention: experimental film project. Parag Mital, Stefanie Tan Su Ann, and Dave Stewart have create an interactive film experience for their Digital Media Studio Project that will be premiered this Friday (21^st March), 12-5pm in Teviot Row House, Edinburgh University.

The installation is inspired by my research into how we attend to film, how this shapes our experience of a film, and how film manipulates this attention (see my thesis). Film can be interpreted as an analogue of our experience of the visual world. Snapshots of fragmented but related visual information is presented to us in a way that we can comprehend by forming conceptual linkages between them. This is similar to the way we acquire information from the world by moving our eyes. If we take this analogy to its logical conclusion, a visual recreation of the information acquired from a scene as a person shifts their eyes may resemble an edited filmic representation of that scene.

Creating such a filmic recreation of a viewers experience is the intention of this project.

How is this accomplished and what is the result? Check out the project website or come along to the premier to find out

http://www.theexperimentalfilm.com/

Rank Prize Funds Mini-Syposium

2007-12-11T16:10:00.000Z

Last week (3-6th December, 2007) I attended a Mini-Symposium on Representations of the Visual World in the Brain organised by the Rank Prize Funds. The symposium was a small gathering of highly respected established researchers and young researchers across the field of Visual Science. Attendees included Ron Rensink (UBC), Nancy Kanwisher (MIT), Mike Land (Sussex), Ben Tatler and Ben Vincent (Dundee), Jens Helmert (Dresden), Jim Brockmole (Edinburgh), Melissa Vo (Munich), and many others from across Europe and North America.

The symposium was an incredibly stimulating, intense experience and I have to express my immense gratitude to the Rank Prize Funds for organising it (aside: the Fund was established by the late Lord Rank to support scientific research in his two main interests Nutrition and OptoElectronics; Rank was also the founder of Rank Film....a rather coincidental overlap with my interests). The symposium was held in the wonderfully picaresque Wordsworth Hotel, Grasmere, Cumbria. Not that I got to appreciate much of it as I was too busy being intellectually stimulated.

My enjoyment of the symposium was rounded off by my being awarded the Prize for the Best Presentation by a Young Researcher for my presentation entitled 'Facilitation of Return'. It is a great compliment for the work I am doing with John Henderson to be acknowledge by such a distinguished group of researchers.

.....and Lord Rank

C.V. updated

2007-12-11T16:07:00.000Z

Just a quick note to say that I have updated my C.V. and my homepage. Both were old and rather poor representations of what I have been up to. Much better now :)

The Famous Colour Changing Card Trick

2007-11-15T23:27:00.000Z

Richard Wiseman, the magician, psychologist, science communicator and author of Quirkology: the Curious Science of Everyday Lives asked me and John Henderson to measure viewers’ eye movements whilst they watched one of his magic tricks. The Colour Changing Card Trick is a very clever use of a perceptual phenomenon known as ‘inattentional blindness’. Check out the trick before reading on:

Inattentional Blindness is an absence of awareness of some detail or event in the visual world due to a failure to attend to it. This absence can often be strikingly large (as in Richard’s card trick) or, most famously in Simons & Chabris (1999) ‘Gorillas in our Midst' experiment. In the Simons & Chabris experiment subjects were told to watch a video of two teams, one wearing white, the other wearing black pass basketballs within their teams. Half of the subjects were told to count the number or passes made by the team wearing white. The other half were told to count the passes made by the team wearing black. Half way through the video a man wearing a black Gorilla suit walked through the scene, stopped in the middle of the scene, waved at the camera and then walked out of shot. When asked after the video if they had noticed anything odd during the video, the majority of subjects failed to report the Gorilla! The probability of noticing the Gorilla was greater when the subjects had been instructed to attend to the black team (58% detection) compared with the white team (27%) indicating that the task had biased the subjects attention either towards black or white objects. The subject’s selective attention shapes the details of the scene that reach the level of conscious awareness and subsequent memory but, importantly the subject is not aware that their awareness is in any way partial. This mismatch between what the subject think they see and what they actually see is what creates the shock at the end of the Gorilla experiment or the Colour Changing Card Trick.

The Colour Changing Card Trick uses a simple card trick to distract viewer attention from what is actually going on, namely the changing of both presenters’ T-shirts, the backdrop, and the table cloth. Such misdirection is a classic tool of any magic performance. All changes are made off camera when the continuous camera shot zooms in to a close-up. This removes the actual change itself from view, leaving only the result. In order for viewers to notice the change they must have previously attended to the object that has changed and have sufficient memory of that object to notice that its current form is different. By measuring viewer eye movements during the trick we can see whether viewers attend to the objects before the changes and whether there is any increase in attention to the objects after the change. Such an increase may indicate the precise moment at which the viewer notices the change.

The results are still being analysed but for now Richard has posted a video illustrating the eye movements of 9 subjects whilst they watched the trick (5 men and 4 women). The one red spot is the gaze location of a woman who detected the Female presenter’s T-shirt change.

This video was created using my own Gazeatron software.

As can be clearly seen from the video most viewers look in roughly the same parts of the scene at the same time. This close control over where viewers are looking is exactly the intention of the magician. By ensuring such systematic viewing the magician can hide changes/manipulations in the unattended spaces. It is only once one of the viewers notices the change (the red spot) that their gaze location begins to differ from everyone else’s.

If you want to know more about the psychology of misdirection and its relationship to eye movements check out Gustav Kuhn’s research at the University of Durham.

Eyetracking and Dynamic Scenes Interest Group

2007-07-10T16:59:00.000Z

The technology available for recording the direction of a person’s gaze has taken massive leaps forward in the last few years. The cost, precision, usability, and, most importantly discomfort of using an eye tracker has now reached a level where any psychology research lab or usability/HCI assessment centre can use eye tracking. Recording the focus of a person’s overt attention (as well as other measures such as pupil dilation, blink rate and eyelid closure) can provide a real-time measure of their experience and an indication of their cognitive processes. Such an insight can complement existing methodologies and allow researchers to understand the dynamics of human experience.

At the recent Experimental Psychology Society conference held here in Edinburgh, eye tracking was clearly rising in popularity as many different researchers applied it to areas such as reading studies, lexical processing of speech, facial expression recognition, social attention, object perception, working memory, visual search, scene perception, and, of course, attention research. The current popularity of eye tracking can be directly related to the increases in technology. Most of the researchers currently using eye tracking are not, primarily attention researchers. The tools provided by the eye trackers allow them to use eye movements as an index of other phenomenon such as real-time cognitive processes. Previously, a research lab would be required to build their own display and analysis tools from scratch (using programming environments such as C or Matlab). This placed eye tracking research clearly out of reach for most people and required a considerable understanding of basic oculomotor control and attention. I believe the current renaissance of eye tracking (of which Edinburgh is part of with our multiple research groups using eye tracking) is a hugely positive step-forward and I hope it continues into other areas of psychology.

One big obstruction to the advancement of eyetracking in to other areas of psychology is its current incompatibility with dynamic scenes. Many areas of psychology are interested in understanding human behaviour in realistic settings such as social interactions, conversations in the real-world, moving through the world, performing actions, complex tasks such as driving, and even watching TV (or is that just me :). As anybody who has ever tried to record eye movements in one of these settings will know, it is phenomenally complicated and time consuming. For example, Mike Land’s influential research on goal-oriented attention during tea making required the construction of novel eye movement technology and the hand coding of every frame of the resulting video! It is no wonder that eye tracking of dynamic scenes is so uncommon when the analysis is so laborious.

The situation has recently got a lot better with the introduction of new eye tracking software. The Tobii and Eyelink (SR Research) eye tracking systems now come with software for displaying dynamic scenes (e.g. videos, animations, etc). However, being able to display the videos is pointless if there is no easy way to analyse the resulting data. No systems, that I am currently aware of assist in the analysis of eye movements in dynamic scenes.

This lack of support for eye movement researchers interested in using dynamic stimuli has motivated me to make a call-to-arms. I know of a growing number of researchers, both in academia and industry who are struggling with the problems, both practical and theoretical associated with recording eye movements in dynamic scenes. No support structure exists for these researchers, no common source of knowledge or tools, and no where they can go to ask for help. Because of this I’ve decided to put out a call to all researchers using or interested in performing eyetracking in dynamic scenes. The Eyetracking and Dynamic Scenes [EDS] Interest Group will comprise a mailing list to which members can post information relating to their research, ask for help, and post useful resources (e.g. software) and references.

If you are interested in signing up for the Eyetracking and Dynamic Scenes [EDS] mailing list, please e-mail me at tim.smith [at] ed.ac.uk with the subject “[EDS] registration” and include the following content in the body of your e-mail:

Name
Position e.g. Research Fellow
Affiliation e.g. name of University or business
Location
Short summary of your research interests.
Existing eye tracking equipment you use.

Together we can make the experience of researching eye movements in dynamic scenes pleasurable, practical, and painless.

Kent Presentation

2007-03-17T16:40:00.000Z

Quick plug.

I’ve been invited to give a presentation to the Centre of Cognitive Neuroscience and Cognitive Systems at the University of Kent next Wednesday (21^st March, 2007). I’ll be presenting the results from my eye movement and film study that I discussed at SCMS and Madison but with the focus shifted to the science. If you happen to be a University of Kent student or in the neighbourhood you should com and check it out (details).

The following morning I’m giving a guest lecture as part of Murray Smith’s Cognition and Emotion in Film course. This sounds like a fantastic course. I wish there had been a similar course when I was an undergraduate. Murray Smith is a very active member of the cognitive film theory community and particularly the Society for Cognitive Studies of Moving-Images. His work on emotion and empathy in film viewing is very influential.

I’m looking forward ton bouncing ideas around with his students.

Seeing Spots

2007-03-17T15:14:00.000Z

Thanks to David Bordwell for his very flattering comments about my presentation on his blog. David, and the rest of the Madison Communication Arts department were incredibly welcoming when I presented there last Monday (12^th March) and the session was an absolute pleasure.

Now that David has alluded to my findings and given a brief description of my presentation you’re probably interested in finding out more. Sadly, I’m going to have to ask you to watch this space just a while longer. As is the way in academia, the publication of academic papers takes a long time and the paper that describes my findings is not yet ready for public distribution…..I know, I know: I’m a big tease. I promise you it will be worth the wait and I’ll publish the paper on my blog as soon as it is available.

In the meantime I can give you a glimpse of the “little yellow dots” David refers to.

This image is a screengrab of a software tool I created called (rather cheesily) Gazeatron. The tool allows me to plot the gaze position of multiple viewers on to the video they were viewing. The image above illustrates where 17 people were looking during this frame of the film (the yellow spots were viewers who could hear the audio and the pink were viewers who could not). Gazeatron allows you to see the same data in real-time as the video plays. By observing the swarming behaviour of the gaze positions whilst multiple viewers watch a film you gain an incredibly detailed insight into the viewing experience. Gazeatron also provides automated analysis for features of the eye movements to provide objective measures in supplement to the subjective observations.

Existing eye tracking tools do not allow you to analyse film viewing in this way and, I would argue reducing viewer attention to a film to static screenshots or data points does not give you a feel for the dynamics of the viewing experience. I’ll work on posting a video of Gazeatron so you can all see what I mean.

A bit of background on eye tracking. Each spot in the image above represents the point where a single viewer is looking. This is important as it tells us, roughly the part of the visual field they are attending to and, therefore processing at any moment in time. You may think you are aware of the whole visual field but in fact you are only able to process a very small portion to a high degree of accuracy at any one time. When you want to process a new part of the visual field you shift your eyes (perform a saccadic eye movement) so that the light from the new target is projected into the region of highest sensitivity in the eye, referred to as the fovea. These saccadic eye movements are very quick and we are not aware of them as our brains “stitch” the images either side together to create the impression of a stable visual world. By recording these eye movements we can infer the moment-by-moment experience of a viewer.

Eye movements can be recorded using a technique known as Eye tracking. There a variety of ways to track somebody’s eyes such as sclera coil and dual-purkinje (some clearly more scary than others). The most common technique used today, and the one I use is Corneal-Reflection. These trackers shine infrared lights onto the eye and film the reflected image using an infrared camera. By locating the iris and the infrared light reflected off the cornea the gaze of the viewer can be calculated. The gaze is simply a vector pointing out from the viewer’s eye into space. Therefore, eye trackers can be used to tell us where people are looking on a computer screen, table top, real-world interaction or….whilst watching a film.

The eye trackers I use are the Eyelink II head-mounted and the Eyelink 1000 tower mounted tracker, both from SR Research. These trackers are located around the University of Edinburgh, mostly in John Henderson’s lab which I am part of.

Tracking a viewer’s eyes whilst they watch a film is not as simple as you might think. The Eyelink trackers all come with software that allows you to present videos but they do not, currently have accompanying tools for analysing the eye movement data in the way I’ve described above. Most other trackers do not provide assistance in presenting films and a lot of previous researchers have resorted to tracking viewers using a head-mounted real-world tracker and recording a video to see what they are looking at (a similar technique is used in driving studies). The only other tracker I have used that is suitable for presenting films is Tobii. This system is incredibly easy to use as it is focussed at usability studies and as a hands-free interface for disabled users. The Tobii eye trackers are incredibly well designed but their price, ~£17,000 puts them out of the reach of most users (the price issue is the same with all eye trackers). Their accuracy is also not as good as the Eyelink systems which is why most vision researchers don’t use them.

If you’re looking for a cheaper option there is the option of building your own eye tracker. Derrick Parkhurst has developed open-source software and instructions for how to construct the necessary hardware to build your own eye tracker. The openEyes project is a great idea although I’m yet to have a go. If you have a go, best of luck and tell me how it goes.

If anybody has any further questions about eye tracking and film please either post a comment below or e-mail me.

As for what I have found by eye tracking film viewers well……that’ll still have to wait. Sorry. For the time being I hope you enjoy the picture of little yellow and pink spots. Who’d have thought seeing spots could be so useful!

SCMS and State Street Brats

2007-03-13T05:53:00.000Z

I am writing this blog post from a hotel room in Madison, Wisconsin (note: the picture above is not the hotel I’m staying in. That’s the Chicago Hilton; more later). Why am I in Madison? Other than the fact they have great frozen custard, Afghan, Mexican food and hotdogs (http://www.statestreetbrats.com/), I am here to present my eye tracking research to the Department of Communication Arts at the University of Wisconsin-Madison. If you are at all interested in Cognitive Film Theory you will be aware of Madison’s Comm. Arts department. They began teaching cognitive film theory before most of us were even aware that such an approach existed. People like Ben Singer, Jonathan Frome (now of Uni. Central Florida) and, of course the hugely influential David Bordwell (yes that guy who’s books you are always being told to read by your film teacher/lecturer/film geek friend/me). The department is so infused by cognitivist ideas that it was an absolute pleasure to present my research there. The intense discussion that my presentation invoked was incredibly invigorating. Its great to get such an intrigued and welcome reception of, what is essentially cognitive science research (although, research focussed on film). I look forward to many future exchanges with this group. I’m probably not supposed to mention this as I’m not sure if the details are finalised but, David Bordwell and the Department of Communication Arts will be hosting the Society for Cognitive Studies of the Moving-Images conference next Summer (2008) so I shall be eagerly returning to Madison next year. I also want to encourage as many other people interested in issues related to cognition and film, whether from arts, humanities, social sciences, sciences, or anywhere to attend this conference. It is going to be a hoot J.

Thanks to The Comm. Arts department for being so incredible hospitable, special thanks to Jeff Smith for being my guide and David Bordwell for being so receptive to my imposition.

Now some background: the reason why this visit to Madison was possible was because I was presenting at the Society of Cinema and Media Studies (SCMS) conference in Chicago last week. The conference is one of the foremost gatherings of Cinema, Television, and Media theorists from around the world. I attended two years ago when it was held in London and my appetite was whetted. This year I presented a paper about my recent eye tracking examinations of different editing techniques across a range of films. The paper was really well received and a lot of people were very intrigued by the potential for eye tracking as a tool in their analysis of film. The technology is getting close to the point that most researchers would be able to perform the kind of analysis I do on films. However, there are a couple of key components such as analysis and visualisation tools that are currently missing from most commercial eye tracking systems that would be required for the technique to really take off. I have developed my own tools that allow me to fill these blanks but most researchers would not be able to do this. And, of course the cost of most easy to use eye tracking systems is still prohibitive.

Who knows, maybe the technology will suddenly take both a cost and technological leap forward and it’ll become accessible to all. Watch this space……

Returning to SCMS, the conference was held in the Chicago Hilton (very swish….well the lobby is anyway; the conference presentation rooms/bedrooms are a tad odd). I was a complete conference geek, attending almost every session. Considering that the days ran 8:15am-8pm this is quite an achievement. The reason I attended so many sessions was because of the incredible range of interesting presentations. Everything from a bit of Cognitive Film Theory (Jonathan Frome, Joe Kickasola, Mark Minett), masses on New Media, Interactive Media, and Videogames, emotions and film, including discussion of automatic facial expression recognition (Kelly Gates), and even a presentation on the Queering of Kevin Smith (it doesn’t take much ;)…Carter Soles). This year there seemed to be a lot of presentations on the impact on-line distribution, web video, and interactive TV and media such as videogames were having on our classical theories of film and television. Fascinating stuff. One of my most satisfying panels was debating the implications of interfaces for interactive TV content e.g. TIVO and PVRs, and their effect on our relationship to the film/TV content. Does the interface, which is meant to empower the viewer by allowing them access to the content actually compete with the content itself?

So, all-in-all a great conference and trip to Madison. I look forward to coming back in 2008.

Examples of Automavision and a proposal for Automavision 2.0

2007-01-08T15:32:00.000Z

Following on from my discussion of Lars Von Trier’s Automavision and Lookey I thought you would like to see some examples of Automavision. David Bordwell, the author of a phenomenal number of outstanding books on the subject of film has written a very interesting blog post on the subject here. He has had the good fortune of viewing The Boss Of It All, unlike myself and capturing some screenshots. The effect is intriguing. Automavision appears to create unmotivated, and classically imperfect framings which Von Trier accentuates by cutting rapidly between very similar shots. Bordwell notes that the result is that almost every cut is a Jump Cut, a violation of the 30 degree rule that causes the image to jump uncomfortably and creates ambiguous temporal relationships between the shots. The same effect occurred in Dancer in the Dark (see a video example here) and can be said to have contributed to the overall discomfort felt by viewers of the film.

What is most interesting about Von Trier’s use of Jump Cuts is that, whilst they abandon the classic continuity style’s preservation of temporal continuity within scenes they still retain a clear cohesion that allows the viewer to understand the action represented. It is almost as if Von Trier, in consciously violating the dimensions of continuity prescribed by the classic continuity style he is revealing extra dimensions of continuity that he uses to create extra significances within his films.

Also, as noted by Antithesis Boy in his comment on my last post the Automavision technique shares a lot of similarities with virtual camera control in videogames. Automavision uses a computer to randomly generate framings for the camera and the result is non-classical framings. Videogames often take place in a 3D virtual environment and require a virtual camera to follow the action within this space. This camera is computer controlled and the objective is to create the best framings possible (e.g. Jhala and Young, 2006) but the result is often unacceptable and bizarre framings. The difference between the two systems is that a Virtual Camera is positioned relative to the objects within a scene where as Automavision (as far as I can make out) does not care what the scene is. This is why Automavision loses significant objects off the edge of the screen or frames them oddly. In his blog post, Bordwell notes that Von Trier does not always choose the most outrageous framing generated by Automavision indicating that he realises that the unconventional framings have a particular effect on the viewer and should be mixed with more conventional compositions to create the intended viewing experience. If Automavision is truly random, as implied by Von Trier then there is no way to control the degree to which the resulting framings are unconventional. This means that Von Trier must keep refilming and regenerating framings with Automavision in order to get suitable shots.

The next generation Automavision system could improve its ability to generate unconventional framings by incorporating some of the intelligence of Virtual Cameras. If it is able to apply the classical framing conventions then it can knowingly violate them. This would also allow it to modify the conventionality of its framings by varying the relative influence of chance and the framing conventions. To do this it would need to begin processing the visual scene, which is no easy task and deciding where the significant objects should be positioned in the frame. An interesting reverse application of this technique would then be to apply Automavision 2.0 to a videogame. Anybody up for a game of Halo directed by Lars Von Trier?

It Lookey like Lars Von Trier is at it again.

2006-12-21T01:51:00.000Z

Not content with creating a revolution in filmmaking by spearheading the Dogme 95 movement, Lars Von Trier is now experimenting with how his films are shot and how viewers engage with them. His new film, The Boss Of It All, is filmed using a new camera control technique developed by Von Trier called Automavision. The system removes the need for a human camera operator, replacing them with a computer which randomly selects camera shots and movements based around an initial camera position selected by the director. The system removes the usual control the director and cinematographer have over the composition of each shot and, specifically it’s framing. Lars Von Trier explains his desire for developing this technique:

“I am a man who likes to control things, and if I can't control them totally I will not control them at all. After doing Europa with very very fixed shots and camera movements, I was tempted to do something totally different. I started using a handheld camera and we invented a form of framing, or non-framing, called pointing of the camera, because I hate framing.” (guardian.co.uk)

This hand-held, almost haphazard method of framing combined with non-continuity editing had recently become Von Trier’s trade mark. Even after he deviated from the original strict edict of the Dogme 95 manifesto with Dancer in the Dark, Dogville and Manderlay he retained this style of framing. The style deliberately breaks with the traditions of the Hollywood continuity style by forcing the viewer to actively search each shot for the most significant elements. In the Hollywood style these elements, for instance a protagonist would be centred within the frame and lit in a way which made them highly salient even in busy scenes. In Von Trier’s recent films the protagonist is often cut off by the frame, visually diminished by other less important elements, or moving at odds with the hand-held camera movement. In combination with his often uncomfortable subject matter, this unpredictable framing technique often leads viewers to describe the experience of watching a Von Trier film as “hard work”.

In a series of recent eye tracking experiments I have shown that this difficulty in following the action of a Von Trier film can be very clearly seen in viewers’ eye movements. When watching a film composed according to the classic continuity style, all viewers will focus their attention on a small number of objects within a shot. In most shots there will only be one clear centre of attention, usually the face of a principle actor and it will be this that all viewers track within the shot and across cuts. By comparison, viewers watching Dancer in the Dark or Dogville distribute their attention across more of the screen and show less agreement of what they believe to be the most significant object. When a cut then happens (which they often do at unexpected moments in Von Trier’s films) viewers are not guided to the new centre of attention by the director so they have to actively search the scene. This active engagement with the visual constituents of the film creates a viewing experience that is completely counter to the normal smooth, direct, almost passive viewing experience of a classical continuity film.

Von Trier’s desire to create films that actively engage viewers can also be seen in his use of Brechtian theatrical technique. In Dogville and Manderlay the use of black backdrops, minimal props, and transparent scenery expose the artificiality of the film. Bertolt Brecht developed these techniques, amongst others, as a way of encouraging his theatre audiences to adopt a critical mindset. By actively counteracting the “suspension of disbelief” encouraged in classical Artistotelian theatre, Brecht was trying to engage his audience in the critical interpretation of the depicted action, the act of its construction, and its place within real-life. The application of these techniques to cinema by Von Trier also resulted in an extra level of visual engagement beyond that created by the non-framing camerawork. By populating the set with transparent scenery the director is unable to hide insignificant actions. All actors must be present on set at the same time and act even though their actions are not important to the current shot. These peripheral actions crowd in on the main action, drawing the viewer’s attention away from the main action and creating further disagreement between viewers.

However, the natural instinct for framing is hard to overcome and it appears that Von Trier realised that his desire for completely un-framed shots would not be possible so long as he or his camera operator were controlling the camera. A hand-held camera is often described as a visual prosthesis: an extension of the camera operator’s eye; seeing what they see. The movement of a hand-held camera may be rough and the framing imperfect but, like the human eye it will always eventually settle on the most important parts of a scene. If Von Trier is to create shots in which viewers are unable to predict what is the most significant part of a scene or how the camera is going to move he needs to take the human camera operator out of the equation. Hence, Automavision.

Automavision is not the only innovation in Von Trier’s new film, The Boss Of It All. Von Trier recently announced that he has embedded five to seven “Lookeys” in the film:

"For the casual observer, it's just a glitch or a mistake….For the initiated, it's a riddle to be solved. All Lookeys can be decoded by a system that is unique." (Von Trier quoted on Screendaily.com)

Von Trier is offering 30,000 Danish kroner (£2,700) to the first Danish viewer that identifies all the Lookeys. The Lookeys are described as “visual elements that are out of place” (www.lookey.dk) and are intended to turn the film into a “mind game”. By informing his viewers of the presence of these Lookeys Von Trier is again encouraging his viewers to actively engage with his films in a way in conflict with the normal film viewing. Spotting continuity errors, which is how these Lookey’s would be described if they were unintentional, has been an occupation of film viewers throughout the history of film. The pastime has escalated to such a level that there are even books and websites devoted to it. Continuity errors are typically mistakes made during production that are spotted by viewers on repeated viewings of a film. The most common errors are unintentional costume changes across shots or cigarettes and drinks that disappear or refill.

Finding errors is always good fun although it can often be very difficult. In my Ph.D. thesis I created a taxonomy of continuity errors that classifies them according to when they are made during the film’s production process and what is required to spot them (page 186 of my thesis). The detection of errors is very dependent on how the viewers watch the film. If the viewer looks at the parts of the screen the director wants them to look at they should never be aware of any continuity errors. The focal objects (those at the centre of attention) should never have errors as they would have been spotted by the director, cinematographer, or editor. Errors are more likely to be located in the periphery of the screen, areas where the production crew and the average viewer are unlikely to look. As film is a dynamic medium and a viewer can only focus their attention on one small part of the screen at any one time, peripheral errors should be missed as viewers are rationing their attention to the most significant parts of the screen.

In a traditional Hollywood film composed for continuity, errors are hard to spot as viewer attention is so carefully influenced by the director and editor. In a Von Trier film, as already discussed, the intention appears to be to encourage active search of the screen and disagreement between where different viewers look. With the addition of Automavision the likelihood that a viewer looks at a part of the scene that would traditionally be insignificant probably increases. This may mean that viewers of The Boss Of It All may be more likely to spot the Lookeys than if they had been included in a traditionally composed film.

However, without knowing exactly what form these Lookeys take (see page 186 of my thesis for a taxonomy of errors) or what expectations the viewer will have to have to realise that the Lookeys are errors we cannot know whether they are easier or harder to detect than traditional continuity errors. What we do know is that the experience of watching The Boss Of It All is, like in all Lars Von Trier films going to be unlike watching any other film.

Re-viewed

2006-10-29T14:08:00.000Z

Last Monday (23/10/06) I had the good fortune of attending the opening of a new interactive art exhibit at the Leeds Metropolitan Gallery. The exhibit is called Re-possessed and presents a collection of interactive art installations inspired by Alfred Hitchcock’s Vertigo. Each installation takes a section of the film and provides the participants with novel ways of exploring the film, their experience of it, and the issues of power, gender, and voyeurism that are so prominent in the film. The exhibit runs until 18^th November and I highly recommend it if you are interested in film or just want to play with some highly innovative interactive exhibits.

The main reason I was invited down for the opening was because one of the artists, Richard Stevens had stumbled across my research whilst looking for references about applying eye tracking to film. His installation, Re-viewed uses a Tobii eyetracker to record viewer’s eye movements whilst watching scenes from Vertigo and then represents the eye movements using some very interesting visualisations that Richard and his students have created. These visualisations give the visitors an insight into their perceptual experience of film that they have never previously had. It also emphasises group differences such as between Male and Female viewers. Given that gender politics and the resulting power relationships are very prominent themes in Hitchcock’s work (see Laura Mulvey’s article on the Male Gaze in her 1975 book “Visual Pleasure in Narrative Cinema”) this method of using eyetracking to extract the gender differences in the viewing experience could prove very informative.

The reason why Richard contacted me is because he and his collaborator, Tony Renshaw were uncertain how to interpret the eye movement data given that so little work has been done in relation to film. They also needed some clear way of expressing any group differences that they observed. Currently, Richard’s visualisations can display the moment by moment differences but in order to report and quantify these differences precise measurements are required. I have encountered similar problems with my own eye tracking work and have been developing suitable measurements. Hopefully I will be able to aid Richard and Tony in the analysis of their data and we will find some interesting group differences. We’ll then work towards publications based on the results. I’ll keep you posted if anything emerges.

I am very excited about this exhibit as, to my knowledge it is the first time eye tracking of film has been performed on such a large scale with a large cross-section of the population. Most eye tracking studies (all Psychology studies for that matter) are usually conducted on University undergraduates; the most easily accessible subject pool for university researchers. The number of subjects eye tracked is also usually less than 20 so there is no scope for exploring group differences. Hopefully, through the development of new measurement techniques and automation of the analysis we can extract some interesting results from this study which will further our understanding of the viewing experience.

So, if you are any where near Leeds in the next two weeks head down to the Leeds Met Gallery. You’ll be furthering scientific knowledge whilst having a fun and informative time.

End of an era + Catch up

2006-09-11T17:19:00.000Z

Today I moved out of my PhD office. Even though I finished my thesis in January, graduated in June, and have been working on a completely different project (Le Active Math) since last October I had somehow managed to remain at my old desk. Today marks the start of Edinburgh University Fresher’s so new PhD students were coming to inhabit my office which meant that I finally had to cut my ties with my old desk. So sad….so many memories….and too many long hours spent at that desk.

My departure from the office was eased by the fact that most of the office mates I had shared it with during my Ph.D. had also left. Scott Nowson started his new life at Macquarie University in Sydney three weeks ago (you can check on his progress at http://nowsonexitmusic.wordpress.com/). Sarah Gingell finally finished her Ph.D. (after too long to record here) last year and has moved on to a real life. Colin Fraser went part-time on his Ph.D. and is now touring Europe promoting e-Democracy (you’ll have to ask Colin to explain what this entails). Zoe Bruce (and Morgan the Dog) left a couple of years ago to pursue a career in academic publishing (Zoe that is, not Morgan). We all had some great times together. In fact we may have had too many great times considering how long our research ended up taking J. Still, I wouldn’t have traded you guys in for anything. Thanks for keeping me company.

Now I find myself divided, literally. I’ve got a “hot desk” (what a cool name for what is essentially a desk that anybody can sit at) in the same building in Informatics which I shall be using most days for my Le Active Math work. I also have a very nice office all to myself in John Henderson’s lab in Psychology. That is where all the interesting blog-relevant work will take place. I’m really excited about starting full time over there but this won’t happen until January. In the meantime I have to run a MASSIVE evaluation of Le Active Math….. Wish me look.

Blog posts I should have written and still might:

Cognitive Studies of the Moving Images Conference, Potsdam, Germany (20/7-23/7/06). Fantastic conference, great people, interesting research, and fabulous reception for my work. The future of Cognitive Film Theory is bright. See Bruce Hutchinson’s account of it at bdhutch.blogspot.com.

Narrative and Interactive Learning Environments conference, Edinburgh (8/08-11/08/06). I really enjoy this conference as it attracts such a small, friendly, yet hugely intelligent group of people. I was one of the organisers this year and found myself stage managing and being sound and lighting engineer for a multimedia performance group called Palindrome. These guys do visionary things with motion capture and real-time music production. We were really lucky to have them take part in NILE.

Active Vision workshop, Dundee (1/09/06). This is a great little workshop organised by Ben Tatler. Lots of interesting presentations about eye movement research from a diverse range of disciplines. The atmosphere was really relaxed which resulted in a lot of discussion and humour. This workshop was a great example of how many interesting eye movement researchers there are across the UK. I’m really looking forward to being more “active” in this community (Sorry, I couldn’t help it)

Edinburgh International Film Festival (14/08-25/08/06). God I love living in Edinburgh! August is a month of cultural, artistic and pleasurable excess that always amazes me. The high point for me is always the film festival. I see an unhealthy number of films each year and 2006 was no difference. Highlights include Brothers of the Head, Black Sheep, Host (Gwoemul), Snow Cake, Clerks 2 (and attending a Q&A with Kevin Smith), and Looking for Cheyenne (Oublier Cheyenne). Thanks to Shane Danielsen the festival director who programmed a fantastic final festival. You shall be missed. Especially your bizarre digressional Q&A style and uncontrollable flirting. Fantastic.

Actually I think I just ended up writing those blog posts!

You can’t get rid of me that easily.....

2006-07-28T12:07:00.000Z

I am very happy to announce that I shall be continuing my research for at least another year. I have been accepted for a post-doctoral research position in John M. Henderson’s Visual Cognition lab in the Edinburgh University Psychology department. Now I know what you’re all thinking: “I thought he was already in Edinburgh Uni”. You are correct. My current status is as a Research Associate for the Le Active Math project employed by Moray House School of Education at Edinburgh University. However, my move to Psychology will be a very fortuitous one as it will allow me to conduct my research without distraction and within a group of likeminded people.

This move to psychology has only been made possible by the uncannily well-timed move of John Henderson’s visual cognition lab from Michigan State University to Edinburgh. John and his partner Fernanda Ferreira were offered chairs in our psychology department and they were wise enough to accept. In the simple signing of a contract Edinburgh has suddenly become a hot-bed of eyetracking and visual cognition research. John, for those of you who don’t know, is one of the foremost researchers in the field of visual search, scene perception, and visual cognition in general. If you read my thesis you will find that he is one of my most cited people. His work is highly influential and I am sure that it will continue to be so from his base in Edinburgh.

John has been very encouraging of my research and interested in the theories I have been developing. As a post-doc under John I shall be continuing my research into the use of attentional cues in film to create the perception of continuity. I will also be expanding my theoretical, methodological, and writing skills to begin establishing myself as a visual cognition researcher.

Thank you, John for this opportunity.

So it looks as if you (reader) won’t be getting rid of me or this blog for a few years to come :)

Rock ‘n’ Roll Research

2006-07-28T11:32:00.000Z

or “How to do an experiment in two days”.

I was contacted at the start of July by a Gustav Kuhn from Durham University. Gustav is doing some really interesting research looking at how magicians use misdirection and perceptual expectations to make you perceive something that actually isn’t there (Gustav is a highly talented magician himself. He’s available for children’s parties, corporate events, bar mitzvah’s …..). He’s done a series of eyetracking experiments to examine where people look whilst watching magic tricks. To satisfy a reviewer of one of his papers he needed to run an eyetracking study and he needed to do it fast. *Cue the Mission Impossible music*

Unfortunately, Gustav’s eyelink II system was not setup yet so he asked if he could run his study using the equipment we have here. I agreed with three small provisos:

1) the study had to take place after the 11^th July as I was at a project meeting in Germany before then;

2) I would have to run some of my own stimuli on the subjects whilst they were being eyetracked;

3) the study had to be finished by the 14^th July as I had to fly to Germany (again) to present at a conference (CCSMI, more later).

In the course of 3 days I managed to prepare two sets of stimuli for my part of the study, design and implement the experiment using the Experiment Builder software which I had never used before, test, pilot, and run 13 subjects through the experiment, and process the data so that Gustav could do his analysis. Phew! Gustav was up in Edinburgh for a day and a half and in that time we managed to get all the data we needed and, through subsequent analysis, Gustav got the result he needed to appease his reviewer. Now that’s how science should be :)

Unfortunately, the pace for my part of the experiment has slowed down somewhat. I tested two types of stimuli. The first were the videos I used in the editing memory experiment. I wanted to see which details of the videos viewers were using to detect the editing discontinuities. I also had some hypotheses about how the discontinuities would effect their eye movements (see d'Ydewalle, G., Desmet, G., & Van Rensbergen, J., 1998 for similar effects). To extract these effects I need to examine each video by hand and then perform some complex statistics on the eye movement data. Sadly this can not be done in a day L I’ll publish these results in the Editing Memory sometime soon.

The second stimuli I eyetracked were feature films. I have been desperate to get eyetracking data for films with different degrees of continuity for ages and Gustav’s urgency finally motivated me to do it. In my thesis I develop a series of hypotheses about how eye movements should be controlled by an editor in order to create the perception of continuity. This data should provide direct evidence of these techniques (if they exist) and motivate further, in depth studies. The films I eyetracked are also interesting to film theorists as they cover the most significant styles of films: Blade Runner (Continuity), Citizen Kane (Deep Focus), Koyaaniqsatsi (Non-narrative), Dancer in the dark (Dogme-esque), Eisenstein’s October (Dialectical Montage), Hulk (Digital Composition/Collage), Requiem for a Dream (Quick Cutting/MTV style), and a few more.

I’m really excited about the findings of this study and I’ll keep this blog updated as to what I find. Unfortunately, analysis of eyetracking data for long sequences of feature films is not easy and there exists no tools to assist. I’m currently developing my own methodologies, tools, and analyses to make this possible. Fingers crossed everything goes to plan and I have some interesting results to present asap.

So in conclusion, high speed experimentation is by far the most Rock ‘n’ Roll way to run experiments. Unfortunately, there is nothing Rock ‘n’ Roll about analysis. (Any suggestions on how I can make my analysis “rock” are highly appreciated)

Corrected version of my Ph.D. thesis submitted

2006-05-18T12:09:00.000Z

I have just resubmitted my Ph.D. thesis in its final hard-bound version. When I had my viva (see Viva! Viva) back in February I was awarded a pass with minor corrections. The corrections amounted to the modification of a few graphs, fixing of typos, rephrasing of a few sentences to make things clear and general reformatting. Not much really. I finally got round to making the changes to my thesis and I formally resubmitted the thesis last Monday (9^th May). I have also registered my thesis on-line with Edinburgh University’s Research Archive. The archive is a wonderful, publically accessible repository of research documents produced at Edinburgh. You can find my thesis here.

I have also updated the version linked from my website. Now everybody can read what I have been up to for the last 4.5 years and be confident that reputable researchers have given it their seal of approval.

Now all I need to do is channel some of the ideas stemming from my thesis into research proposals for my future career. Too many ideas, too little research funding/job offers J

Science Festival Fun

2006-04-14T08:25:00.000Z

This is just a quick note to say that I am currently running activities at the Edinburgh International Science festival (April 11^th – 15^th, 2006) at the National Museum of Scotland. There are two activities: Editing Memory and Directing Reality.

Editing Memory is a drop in activity in which you get to test your memory for films by reconstructing the film using editing. It is proving to be very VERY popular (my helpers can barely keep up!). Come and have a go (its free) if you want to test you memory or try editing a film. You can visit the website for more information and the eventual results: http://scifest.inf.ed.ac.uk/editing-memory/

Directing Reality is a 1-hour workshop in which kids (and big kids) can come and make Trick Films using really simple filming techniques. The films the kids are making are just like those produced by George Melies over a hundred years ago. The intention of the workshop is to get the kids thinking about the expectations and assumptions we make when perceiving the visual world and how these can be violated to create funny illusions using editing. The kids pick it up really quickly and the tricks they come up with are marvellous. If you want to come and have a go get down to the museum before Saturday although you might want to book on-line before hand (http://www.sciencefestival.co.uk/) as tickets are selling out fast (they cost £3).

Hope to see you there :)

An Attentional Theory of Continuity Editing

2006-03-10T19:32:00.000Z

So here it is: my thesis. After successfully passing my viva a few weeks ago I have finally finished tweaking and polishing my thesis so that it is now in a format that I can distribute. Feel free to download the thesis and print it off although I will warn you it is 389 pages long. Selective printing might be better for the rainforests.

If you wish to reference my thesis please use the reference below.

Smith, T. J. (2006) An Attentional Theory of Continuity Editing. Unpublished Doctoral Thesis. University of Edinburgh.

I welcome all comments and questions about the thesis as well as peripheral issues. The thesis is a first stab at using cognitive science theories and methodologies to answer questions about how we watch film. The deeper I delved into the issues related to film perception, the more questions emerged. It is my hope that I will have the opportunity to investigate these questions in the future and that other researchers, from both cognitive science and film theory will also take an interest in answering these questions.

Whilst writing this thesis my love of film and adoration of filmmakers has grown inordinately. The insight into the human perceptual system exhibited by talented filmmakers is awe inspiring. The precision with which they appear to shape their audience’s experience of the film highlights how much further cognitive science has to go before we begin to get even close to understanding how perception works. By applying an analytical mind to the conventions and techniques used by filmmakers, it is my belief that cognitive scientists will learn a lot about the human perceptual system.

Film is a deviant reality that we, as members of its audience, have been successfully interacting with for over a century. By understanding how film differs from reality and why this difference has no negative effect on our ability to perceive film we will gain, not only a greater understanding of how we perceive film but also an insight into how we perceive reality.

Viva! Viva

2006-02-25T12:13:00.000Z

Last Tuesday (21^st February) I had my viva. For those readers who are not academics this is an oral examination during which I am grilled by two experienced academics about my thesis, its content, all issues surrounding my thesis, my research goals, and, if they are really pedantic, my dubious sentence construction. Vivas in my department have been known to go on for 3-4 hours on average! Imagine a job interview in which your interviewers know you inside out and challenge you to defend why you are you. That’s the kind of experience a viva can be.

Luckily my viva turned out to be nowhere near as bad as I was dreading it might be. Due to some fortuitous scheduling I was able to have Professor John Henderson of Michigan State University as my external examiner. Usually your external has to come from the UK or Europe as the university have to pay to fly them over and accommodate them. I was able to have John as my external as he happened to be planning a visit to Edinburgh just at the time that I was due to have my viva. In terms of my research topic and the specific approach I took, John was probably one of the most qualified people in the world to examine me. He and his colleagues at Michigan State have produced a wealth of innovative research investigating visual scene perception and memory across eye movements. John is specifically interested in using ecologically valid stimuli to try and understand how real-world scene perception works. I hoped that this would make him receptive to my application of the theories and methodologies of visual cognition to film.

As it turned out in the viva, he appeared to appreciate my approach and my thesis in general. After only an hour of discussion, most of which at a high level of issues arising from and related to by thesis John and my internal examiner (Professor Keith Stenning) decided to recommend that I be awarded a Ph.D. with only minor corrections (giving me one month to do these). This is essentially the best result you can hope for from a viva. I was stunned, to say the least.

I want to say a huge thank you to John and Keith for agreeing to examine me on my thesis, to read the massive document, provide me with feedback on it, and take time out of their busy schedules (in John’s case, super busy) to examine me. I also want to thank John Lee, my primary supervisor for is help in getting to this point and organising the viva and Helen Pain, my second supervisor for reading my thesis multiple times, grilling me in a mock viva, and being there during my real viva. Thank you everyone.

The best part of my viva success for any interested readers out there is that this means I am able to make my thesis available to you. I will post a pdf copy of my thesis on this blog in the next few days. There are a few minor corrections I have to make first but once they are done I’ll post it here.

Thesis Complete

2006-01-27T17:37:00.000Z

I did it! I handed in my Ph.D. thesis. I can't actually believe I finished my thesis. As you might deduce from the lack of blog posts since early December, the Christmas period was busy for me. I had so much of my thesis left to write as I approached Christmas and I had no idea how I would do it. However, I seemed to enter the thesis “zone” where all my ideas coexisted in my brain and I was finally able to see all the connection, relationships, and structures that had been lacking from my ideas for so long. This made the final writing possible and meant that I was able to produce sections at an alarming rate.

I returned on the 27^thDecember to Edinburgh from my parent’s house on the Wirral. Between that day and the 10^th January I had no more than 4 hours sleep a night, spoke to very few people other than my girlfriend on the phone (she was visiting her family in Germany), and saw even less people. I produced at least 30% (~33,000 words) of my thesis in this period. Having now submitted my thesis I feel I can now reflect on the experience and hand out this advice: WRITE YOUR THESIS AS YOU GO ALONG, NOT AT THE LAST MINUTE. Simple, some might say, obvious advice that I completely failed to adhere to when doing my thesis. Luckily my sanity, my fingers (from all the typing), and my Will to live where not completely crushed by the experience. Not completely anyway……

I can now officially record that the Ph.D. thesis entitled ‘An Attentional Theory of Continuity Editing’ was submitted for examination on the 11^th January 2006. My viva (oral examination) should take place around the 20^th February and based on the outcome of that (eek!) I will know if I have succeeded in gaining my doctorate. I have no idea how it will go but I’m doing all I can to be prepared for it. I’ll try to keep this blog updated so you know how things go.

For the time being I am fortunate to report that I am employed as a Research Assistant on a European project developing an intelligent tutoring system (http://www.leactivemath.org/). This project gives me the opportunity to develop my AI interests whilst continuing to perform experiments (I’ll be performing usability, evaluation of a tutoring system). However, I do not intend to end my research interest in film perception with my thesis as there are too many questions left unanswered. I’ll be developing research proposals and making grant applications during my time as an RA with the aim that by 2007 I’ll be able to pursue my research interests. To this end I will be presenting my research at conferences during 2006 (e.g. ETRA, in San Diego in March, www.e-t-r-a.org) and writing papers. Lets hope 2006 proves to be an enjoyable and productive year and sets the foundations of my future research career.

The same wishes are extended to all readers of this blog: may your 2006 bring all you desire.

A human filter

2005-12-16T12:50:00.000Z

I feel like my head has become a filter for all the information that might possibly be relevant to my thesis. All the ideas I have had about continuity editing, the papers I have read, the studies I have performed are all filtering through my head and on to the page. If I achieve nothing with my thesis at least I can be proud or collating such a large reference list! This is the problem with performing cross-disciplinary research you have to be knowledgeable in multiple areas simultaneously. This requires a lot of reading and even more thinking. After I hand in my thesis (less than a month from now) I'm going to need to do something mind-numbing for a while. Maybe I'll get a job in data entry. I did this job for three summers during my undergraduate degree and it is incredible how little of your brain you need to use to do it well (no disrespect to the envelope stuffers out there).

My thesis is finally shaping up. I've got most of the sections written and the main theory is nearing its final form. For anybody who has witnessed/read any of the previous accounts of my research its final form might come as some what of a shock. By reading so many theories written in different areas I started to see the natural crossovers. I think my theory as it now stands can account for a lot of the techniques used in continuity editing. All that I hope is that my examiners also think so.

I'll wait until my thesis is finished before debuting my new theory. However, if you want to get a good insight into why continuity editing is acceptable to the human perceptual system take a look at these books. These editors know what's what:

Dmytryk, Edward (1986) On Filmmaking
Block, Bruce (2001) The Visual Story: Seeing structure of film, TV, and New Media
Murch, Walter (2001) In The Blink Of An Eye: a perspective on film editing
Pepperman, Richard (2004) The Eye is Quicker: film editing; making a good film better

Do you see a theme?

Quick Thesis Update:
Deadline = 9th January (Merry Christmas for me then)
Percentage written = 80% (and then I have to edit; Oh, the irony: the one part of writing I hate)

Remembrance or Mardi Gras?

2005-11-11T11:00:00.000Z

I was in Germany a year ago today and I had a very odd experience. I was walking through Stralsund with my girlfriend’s family at 11am on the 11^th November feeling very odd due to the significance of the time and date. In the UK this is Remembrance Day, the day when we remember the people who died during the First World War and the day when the armistice was signed. It is a very solemn day that makes you dwell on the futility, brutality, and loss of war. So, being in Germany on this day made me feel a bit strange and a bit uncertain whether I should enquire of my girlfriend’s family whether they knew of the day’s significance. However, as 11am came round the streets were suddenly filled with kids wearing streamers and party hats and blowing on party blowers and horns. As we walked towards the town hall we saw that there was some kind of party going on inside. We walked into the Town Hall and found loud German Techno booming from one of the ornate meeting rooms, a table set full of Sekt (German sparkling wine), and people handing out Berliners (donuts). A party was in full swing in Germany whilst at home people would be mourning the death of thousands of soldiers.

The party, I was later informed by my girlfriend was to celebrate the start of Fasching, the Mardi Gras or Carnival. For some reason, traditional preparation for Fasching begins on the 11^th November and it is celebrated by storming the Town Hall. Fasching is a big deal in the Catholic south of Germany but the whole of Germany likes a good party so the idea seems to have spread north. The precise match of the beginning of Fasching with the signing of the WWI armistice was too startling a coincidence to be accidental. It must have been chosen to emphasise the importance of the occasion (11am 11^th day, 11^th month is very easy to remember) yet in Germany nobody remembered the significance of the day. Other than the fact that you get to drink wine and eat donuts at 11am. Another interesting example of how a shared history isn’t always the same history.

Errors/goofs/gaffes/slip-ups

2005-11-11T10:29:00.000Z

No, I’m not talking about my current problems writing my thesis. These are all names given to continuity errors. Now I thought it was about time that I addressed one of the common confusions about my thesis: I am not researching continuity errors. What are generally referred to as continuity errors are mistakes made during production that in some way highlight the artificiality of the film. Classic examples are cigarettes/drinks/food suddenly changing level across shots, actors changing posture or position within the scene, and the all-too-frequent instantaneous changes of costume. A great source of continuity errors is Jon Sandys’ website www.moviemistakes.com and the books that accompany it http://tinyurl.com/8hjxt. He has a massive directory of errors some of which are hilarious and others are so bafflingly complex that it is incredible that anybody noticed them. However, hardly any of them actually have anything to do with editing. Which brings me to a strange contradiction: continuity errors are not usually caused by violating a continuity editing rule. I discuss this in some detail in my thesis (I have just finished writing a taxonomy and explanation of the psychology of continuity errors) but in general my thesis is about editing so continuity errors are not important.

After my thesis I might expand on my taxonomy of errors and work out a webpage devoted to the topic but for now I’d direct you to Jon Sandys', the expert on this subject, and you can amaze at how slack Hollywood can sometimes be.

Quick Thesis Update:
Deadline = 9th January (notice the change)
Percentage written = 50% (still yikes!)

DDD

2005-10-05T18:28:00.000Z

Just finished presenting my Draft Dissertation Defence (DDD). This is curious part of the British Ph.D. process where by you defend the ideas in your thesis before you've finished making them up....Sorry, I mean "writing them up".

My presentation seemed to go well. As always, I was bizarrely calm whilst presenting. The audience seemed to appreciate and follow what I was presenting so I rewarded them with some entertaining visuals (well, I thought they were entertaining). Received some useful feedback from various people after the presentation which will, hopefully find its way into my thesis.

Quick Thesis Update:
Deadline = 16th December
Percentage written = 30% (yikes!)

For those who are interested I'll whet your appetite with the abstract for my DDD. If you have any questions about it or want to know more don't hesitate to contact me. My plan is to write some journal publications based on this data immediately after I submit my thesis so hold on in there for the official unveiling of this stuff.

Title: An Attention-based Theory of Continuity Editing: A Draft Dissertation Defence
Date: Wednesday 5th October 2005

Abstract:

All film and television productions are constructed according to a set of conventions developed during the first two decades of the 20^th century. These conventions are collectively referred to as the “rules of continuity editing”, and enable a filmmaker to construct a film in a manner that makes the artificial visual disruption occurring during an edit invisible to the viewer whilst maintaining the impression of continuous action. Achieving this “continuity” is the goal of all mainstream editing yet the concept is poorly understood and has never received an in depth scientific examination. The intention of this thesis is to rectify this neglect and in doing so advance our understanding of how we perceive edited visual media.

A cut (an instantaneous change from one camera shot to another) creates a sudden disruption of the entire visual scene which, under normal viewing conditions would involuntarily capture a viewer’s attention (Lang, 2000). This would make the viewer aware of the editing and has been shown to have negative effects on comprehension (Frith & Robson, 1975) and arousal (Lang, Zhou, Schwartz, Bolls, & Potter, 2000). For a cut to occur without attentional capture, the continuity editing rules must create viewing conditions under which attention is suppressed or occupied, or the attracting quality of the disruption minimised. During this talk I will present examples of editing rules that I have empirically shown to utilise these processes.

Three main phenomena will be presented that explain how natural modulations of attention during normal scene viewing can be used to explain how continuity editing rules function: event segmentation (Zacks et al., 2001), visual occlusion (Michotte, 1991), and saccadic eye movements (Yarrow, Haggard, Heal, Brown, & Rothwell, 2001). The close compatibility between these natural cognitive phenomena and the editing rules indicates that “continuity” is actually a combination of spatiotemporal continuity and existence constancy. These properties do not have to be created as they are assumed to continue unless we are made aware of their absence. Therefore, the editing rules function by ensuring continuity of attention which in turn guarantees perceptual continuity.

References

Frith, U., & Robson, J. E. (1975). Perceiving the language of films. Perception, 4(1), 97-103.

Lang, A., Zhou, S., Schwartz, N., Bolls, P. D., & Potter, R. F. (2000). The Effects of Edits on Arousal, Attention, and Memory for television Messages: When an Edit is an Edit Can An Edit Be Too Much? Journal of Broadcasting and Electronic Media, 44, 94-109.

Lang, A. (2000). The Limited Capacity model of Mediated Message Processing. Journal of Communication, 50, 46-70.

Michotte, A. (1991). On phenomenal permanence: facts and theories. G. Thines, A. Costall, & A. Butterworth (Editor), Michotte's experimental phenomenology of perception (pp. 122-139). Hillsdale, NJ, US: Erlbaum. Notes: (Original work published 1950)

Yarrow, K., Haggard, P., Heal, R., Brown, P., & Rothwell, J. C. (2001). Illusory perceptions of space and time preserve cross-saccadic perceptual continuity. Nature, 414.

Zacks, J., Braver, T. S., Sheridan, M. A., Donaldson, D. I., Snyder, A. Z., Ollinger, J. M., Buckner, R. L., & Raichle, M. E. (2001). Human brain activity time-locked to perceptual event boundaries. Nature Neuroscience, 4(6), 651-655.

A Coruna: Continued

2005-09-05T17:23:00.000Z

[continued from previous post]

As for the conference it was hit and miss. There seemed to be a large emphasis on neuroscience and psychophysical methods. There is nothing wrong with these methods, in fact they are very powerful ways of investigating visual phenomenon in very precise ways, but they are not the only methods for investigating visual perception. Quite a few people commented to me after the first couple of days that they were a bit disappointed with the lack of higher-order perception studies and applied cognition. As it turned out these types of studies were reserved for the last couple of days and there were quite a number of interesting posters that made up for my general feeling of “intellectual drowning” during most of the seminars.

Before the start of the conference there was a pre-symposium entitled Art and the Visual System. This was really interesting and was exactly the approach I have been trying to take during my PhD: applying scientific theories and methods to the analysis of visual art. There were a number of really interesting presentations including explanations of the Watercolour illusion (Baingio Pinna), Picasso’s cubism via visual crowding (Denis Pelli), and Mona Lisa’s ambiguous smile via peripheral vision (Margaret Livingstone). Very clever stuff. The presentation I found most interesting was Semir Zeki’s (University College London) discussion of the chronoarchitecture of the brain and its relationship to sensory experience. As an example of his ideas he showed an animation of real-time brain activation whilst watching a clip from the James Bond movie Goldeneye. Brilliant stuff. Although I have to acknowledge Uri Hasson (NYU) who has previously presented similar results of neural synchrony between viewers when watching a feature film (his Science paper can be found here). Hasson presented further evidence of this synchrony at ECVP (abstract). He showed that neural activation was mostly the same if a silent film was viewed forwards or backwards with only a couple of regions showing different activation: the intraparietal sulcus (IPS) when the films depicted non-continuous shots of motion and Wernicke’s area when the shots were semantically related (i.e. the films had a plot).

There were a couple of other relevant presentations at ECVP (such as Kazuya Kito’s eye tracking study of Coca-Cola adverts….sweet industrial sponsorship) but the most relevant was Kiyoshige Suzuki’s presentation of ‘The perceptual organisation with serially presented motion picture shots’…..sound familiar? I couldn’t believe the similarity between his study and my own when I first saw it. He even used stimuli that depicted a ball moving laterally across the screen, exiting the screen, and then reappearing on the opposite edge after a cut! His mode of assessment was primarily introspective and he drew conclusions about viewer’s perception of continuity of object identity and motion across the cut where as I was using indirect methods to find evidence for the same effects (you can see my abstract here). Suzuki has presented work at ECVP twice before, one poster was an empirical reproduction of the Kuleshov experiment (2002) and the other was an earlier investigation into continuity across cuts (2004). The fact that we have both been investigating the same questions was a great shock to both of us. Suzuki seemed to get a lot out of my poster when I presented it on the Friday and we spent a lot of time together discussing our methodologies and research interests. I’m sure Suzuki will prove to be a great colleague and I hope we can stay in touch to exchange ideas and keep each other motivated about this area of research.

During Suzuki’s poster session I also had the good fortune to meet Rosanna Actis-Grosso. Rosanna, in conjunction with Luca Tommasi presented two studies investigating the perception of temporal continuity across cuts at ECVP in 1998 and 1999. Their 1998 poster (sorry, can’t find a link) showed that perception of temporal continuity across a magnification cut during a Michottean Launching event was effected by the degree of magnification. This experiment is the first empirical evidence I know of that shows how a temporally discontinuous edit can be made to appear continuous. My Watched Pot / Stopped Clock experiment owes a lot to this experiment. Their 1999 poster showed that this perception of temporal continuity is effected by the relocation and the resulting apparent motion of the main object on the screen across the cut. This conclusion supported my ideas about the effects of saccadic eye movements on continuity perception across edits. Rosanna made some very interesting comments about the poster I presented at ECVP and it is great to see that her interests still lie in this area even if she hasn’t followed up her two innovative studies.

In general ECVP was a very satisfying experience and I am very glad I went. The people I met, both socially and in an intellectual context really made the conference worthwhile and I look forward to staying in contact with as many of them as possible. Thanks again everyone and I hope to see you at future ECVPs or other visual perception conferences.

A Coruna!

2005-09-05T14:37:00.000Z

So I returned from the ECVP conference last weekend (27/8) and thought it was about time I filled you in on how it was. The whole trip was lots of fun. A Coruna is a really interesting city, mixing the heat of southern Spain with the rolling green hills of somewhere like Ireland or the Scottish Highlands. The city is positioned on a hammer-head peninsula with a beautiful golden beach down one neck of the peninsula, a harbour on the other side of the neck and a Roman lighthouse on the rocky head of the peninsula. The cool air off the Atlantic makes the climate really comfortable (if a little wet at times) although the temperature of the water makes for a rather surreal beach-going experience: beautiful sand, hot sun, and F*****g freezing water!

I stayed in university accommodation on the outskirts of the city which turned out to be a rather “interesting” experience. I knew the accommodation was somewhat out of the way but I didn’t expect it to be positioned on top of a hill in the middle of nowhere! The accommodation looked like a mini holiday camp with swimming pool, cafeteria, bar, gym, ping-pong table, laundry, bingo, boules, and about 300 old Spanish women! My first afternoon at the accommodation was very surreal as it slowly dawned on me that the place was over-run by Spanish old people (the accommodation was a retreat for old people from Barcelona and Madrid for whom the summer heat was too much), nobody spoke English (even the receptionist), I couldn’t identify any other conference attendees and I was stuck without any way out of this place. It was like a bad comedy horror film! Luckily I met a couple of people during dinner and started to realise that the accommodation actually had its upside: cheap dinner served with all the wine you could drink and fabulously fun people to share it with. The people I met during the conference were what made the whole trip worthwhile (there were intellectual rewards, I’ll get on to them later). The first two people I met were James and Lucie (so sorry if this isn’t how it is spelt, Lucie) from Calgary, Canada. They proved to be the life and soul of the entire trip, especially Lucie who is a mini-whirlwind of unstoppable fun. James and I hit it off immediately, probably due to our mutual interest in applied visual cognition and eye tracking as well as the fact that we are both originally from the North West of England and seem to share the humour and personality traits associated with that. Lucie is his fiance who originally came from the Dominican Republic. Together they form this wonderfully complimentary couple that are both hilarious, endlessly enthusiastic, and adoring. Thanks guys for making my trip so enjoyable.

Staying at the accommodation bred a sense of community that I’m sure would have been lacking if I had stayed at a hotel in the city. By a couple of days into the conference there was a stable group of ECVP people who all hung around together, socialised, toured the city and attended conference sessions. Here are some of the wonderful people I met and hope to meet again:

Nicholas Pugeault – Stirling University

Zsara Hussain – Hamilton university, Ontario

Karol Myszkowski – Max-Planck Institute, Saarbrucken

Baingio Pinna - Università di Sassari, Italy

Kamal (?) – Tokyo

ECVP and Thesis Progress

2005-08-18T11:03:00.000Z

So, it’s been a while. What have I been up to? Well, in short I have been writing my thesis. Actually there is nothing short about it. Not the time it is taking to write it, the length of the produced material, or the time I still foresee it taking to finish. However, I have a deadline (end of December) and a schedule that I am, mostly, sticking to so I guess that is all I can do. The act of writing the thesis is actually quite enjoyable. It is really satisfying to finally put in to words the ideas I have had throughout my PhD and take time to research the questions that have always been in the back of my mind. Some of the topics I have already written about are: before the second World War to initiate the drama. The cinematography and score of Nirgendwo are absolutely beautiful. Wah-Wah captured some of the beauty of the African landscape but Nirgendwo really caught its soul.

Continuity editing rules and their evolution
The Kuleshov Effect
Pudovkin’s Constructive Editing
Eisenstein’s Dialectical Montage
Facial Expression perception
Distribution of attention about a dynamic visual scene
Matched Exit/Entrance cuts
Cuing and Expecting a cut
Attention and the cut
Eye movements
Saccadic Suppression
How to empirically investigate film perception
As well as describing my first experiment.

This may sound like a lot but some of the sections are only partially complete and there are still a lot of other topics to address. I am going to have to be super productive over the next few months. I’ll try my best to keep the blog updated with my progress as I get closer to completion.

However, this period of intense writing will have to be temporarily put on hold whilst I attend the European Conference of Visual Perception (ECVP). The conference is being held in A Coruna, Spain from Monday 22^nd August to the 26^th. I’m flying out there tomorrow morning for a pre-conference workshop on Art and the Visual system. Should be interesting. I’ll be presenting a poster at the main conference on the Friday morning. It should be really useful to get some feedback from people highly experienced in visual perception research. Precisely the kind of input I need at this point in my write-up. Well, as long as it is positive J. I’ve posted the poster on my website front page for anybody to read who isn’t at the conference. Feel free to send me any comments or questions.

So that’s where I’ll be for the next week and I’ll try and blog the conference on my return.

On an completely unconnected note, the Edinburgh International Film festival (http://www.edfilmfest.org.uk/) opened last night with Richard E. Grant’s autobiographical ‘Wah-Wah’. It was a wonderful film to open the film festival: very satisfying, finely crafted and acted, and very, very British in a way that British film rarely is. It concentrated on the disintegration of the British Empire and its parallels within a family in Swaziland. I found it fascinating to see the life of the colonials, their desperate grasp of a ‘Britishness’ that didn’t really exist anymore in Britain at that time and is long deceased now. I think the topic of ‘British colonial guilt’ should really be dealt with more often as it was such a significant part of our history. Wah-Wah was very brave for tackling it, however incidentally. Check it out when it gets a general release. If you enjoy it I could also highly recommend a German film called ‘Nirgendwo in Afrika’ (Nowhere in Africa) which won the Best Foreign language film in the 2003 Oscars. It tells a similar story about European immigrants in rural Africa but uses the flight of Jews from Germany

Henderson Visit

2005-06-24T14:39:00.000Z

I just wanted to say a quick word about a researcher who visited Edinburgh University last week. John Henderson from Michigan State University is one of the foremost researchers in the area of visual cognition with specific expertise in change blindness, transsaccadic memory, and visual representation. If you are at all interested in understanding how we perceive the visual world then I highly recommend checking out his research.

For the last decade there has been a raging argument in the field of visual cognition about how much of the visual world is represented and stored for future reference. Evidence from Change Blindness studies (see a Levin and Simons 1997 summary here) in which subjects were shown to miss very large changes of the visual scene across a disruption (e.g. an eye movement, a blanking of the screen, or physical obstruction) seemed to indicate that the subjects were storing very little visual information over time. This evidence seemed to conflict with everybody’s natural intuition that our experience of the visual world is a rich and lasting one. The work John Henderson and colleagues have been doing over the last few years has gradually reconfirmed this intuition by showing that change detection does occur when less explicit measures are used. His results indicate that visual memory is a natural result of scene viewing with attended and fixated regions of the scene receiving a high degree of representation. These results are very important to the area of film cognition and specifically understanding perception across edits. Editing is an explicit manipulation of the audience’s attention and as such we will better understand perception across edits by understanding what visual information is represented across shifts of attention.

Whilst John was visiting Edinburgh I had the wonderful opportunity to present my research to him and receive some very useful feedback. John and his wife Fernanda Ferreira (a well known psycholinguist) are two very wonderful and influential people to whom I wish all the best in their research and hope I get to meet again (and possibly collaborate with) real soon.

The Kuleshov Effect

2005-06-24T14:11:00.000Z

Sorry it has been so long since I last wrote. I didn’t mean to imply that I wasn’t interested in you anymore…..you know, things just come up…..distractions……I still love you….

Whoops! I thought I was writing an e-mail for a second (lets hope my girlfriend doesn’t find out). I do regret not blogging, however. I’ve actually had lots of experiences over the last month that I should have blogged but I guess there was just too much and too little time. I’m finally fully immersed in writing my thesis. No distractions (well almost none). I get into my office in the morning and start writing then leave in the evening (with a brief break for lunch to watch trailers on-line…I’m a trailer junky). The thesis is coming long slowly but it is coming. I’m currently writing a section on the Kuleshov effect which should form the bridge between my introduction and the main theoretical and empirical section of my thesis.

Wait, what’s that? You don’t know what the Kuleshov effect is! Shame on you, you haven’t been reading my blog in detail. I summarised the Kuleshov effect in my ‘Edgecodes’ post. Basically it is the effect of juxtaposing the shot of an expressionless face with shots of different objects which results in the audience perceiving different emotions in the face. It, along with other pseudo-experiments conducted by soviet filmmakers Kuleshov and Pudovkin in the 1920s forms the theoretical basis of the cinema we are all familiar with today. It is a really useful reference for my thesis as it shows an attempt to merge film theory with scientific experimentation. As this is exactly what I am attempting to do it serves as a great example of how it should and shouldn’t be done. The Kuleshov experiment has been repeated many times since Pudovkin first reported it, each repetition adding slightly more to our understanding of how it works. The effect now exists in a strange duality where film theorists doubt its authenticity due to evidence that suggests the original experiment never actually took place and recent attempts to repeat it which have failed* whilst psychologists and filmmakers treat the effect as an established fact. In the section I am writing right now I am trying to resolve this two viewpoints and show how better science actually leads to better film theory.

The main reason why we should believe that the Kuleshov effect exists is its continual successful use in cinema. In June’s edition of Sight and Sound magazine, Greg Araki directly referenced the Kuleshov Effect when explaining how it managed to depict the emotionally shattering actions in his film Mysterious Skin without having to show them. The film follows the early life of two boys who were abused by their Little League coach when they were 8. The subject matter of the film is grotesque in the extreme, paedophilia, gay hustling, sexual abuse, baseball, but the film’s aesthetic and its emotional effect on the audience resonates with beauty. Part of the reason for this is probably Araki’s emphasis on facial Close-Ups (CU) for a large portion of the dramatic scenes. Due to the young age of the child actor’s he didn’t want to have to actually show the abuse that was occurring so instead it is all implied through the actor’s facial expressions. The effect is astounding, you believe that the abuse has taken place and that you have been affected by it when in reality you have seen very little. The film orchestrates the viewer’s emotional responses through the use of these Close-ups, beautiful atmospheric music (wonderful use is made of Sigur Ros), quiet Long-shots, and perfect pacing. I left the film in a detached state that stayed with me all evening and still returns to me every time I think of the film.

The interesting thing about Araki’s use of the Kuleshov effect as rationalisation of his use of facial CUs is that he does not surround the CUs with shots of the action or objects which, in the original Kuleshov experiment, imbued the CUs with meaning. In fact Araki’s use is almost counter the Kuleshov effect: the expressive power of the human face without context. As Ingmar Bergman stated so astutely “Everything begins with the actor’s face”.

* Prince, S & Hensley, W. E. (1992) The Kuleshov Effect: Recreating the Classic Experiment, Cinema Journal 31, No.2 Winter

and

Touzard, G (2003)

Where have all the fun films gone?

2005-05-11T11:02:00.000Z

I was discussing this last night with my girlfriend: why are all the fun films all over 10 years old? I’m talking about the films that you watch repeatedly and enjoy thoroughly every time. Films like Back to the Future, Bill & Ted’s Excellent Adventure, Princess Bride, The Never Ending Story, Labyrinth, the original Star Wars trilogy, Dirty Dancing, Grease (ok, maybe the musical/dance films are just me). Everybody probably has similar films even if they don’t share my tastes. Films that are associated with joy, fun, innocence, and fond memories that transport you back to the time and place that you first watched them every time you see them. Where are the comparable films today?

I can be a bit of a snob when it comes to films. I appreciate fine artistic films, seek out independent and foreign language films, and love to watch a film that knowingly toys with cinematic conventions and the restrictions of Hollywood. I’ll often turn my nose up at overly commercial films or blatantly crass or manipulative films…….however inside me there is always that little kid who first became fascinated with cinema through films that existed purely for fun. Some of my fondest memories of childhood are film related: going to watch Labyrinth in the Apollo 6 in Wallasey with my dad, watching cartoons in the primary school hall on rainy days, seeing Karate Kid II in a mini-cinema on a ferry back from Holland whilst holding down a bout of sea sickness/food poisoning (ok the last is less “fond” and more “green”). The glow of the cinema screen and TV set held me enraptured as a child. Not because of the films that were presented on them rather the experiences, emotions, and sensations they induced in me and the way that the stories entered into me during viewing and became my own.

Have I become hardened by film viewing? Does it become harder for films to connect with you on such an instinctive, emotional level as you get older and you start thinking about a film too much and not feeling? Or have the films changed? I can’t think of any films over the last 5 years that have moved me, enraptured me, and left me in a state of absolute joy. The only films that come to mind are animated and predominantly produced by Pixar. Only in the animated realm do filmmakers seem to have a licence to detach their films from the harsh realities of life and focus on stories that are pure escapism. Yes, it is easier to achieve this using cartoons but live dramatists have been achieving the same thing for millennia so why are all films, no matter how FX-driven, currently unable to sever their ties to reality? Take Back to the Future (BTTF) for instance. BTTF takes a cartoony highschool kid and transfers him to a world that is not his own (the 1960s). There he must overcome many obstacles (getting power into the time machine; stopping his teenaged mum falling for him, etc) so that he can return to an improved version of normality. This is a classic Aristotlean dramatic structure and works so well because of the initial detachment of the protagonist (Marty McFly) from his reality. In the unfamiliar world the normal conventions and expectations of reality can be abandoned (or rather replaced by historical, kitsch, cartoony conventions) and so experiences can be heightened and played out for pure fun. The shackles of reality do not bear down upon the protagonist.

I’m sure there are recent films that use a similar dramatic structure but all seem to carry too much of the reality with them. I don’t think I am alone in my disillusionment with the current decrease in film as pure entertainment. You just have to look at the genres that consistently perform well in DVD sales: comedies and horror films. Audiences don’t want to take dramas home. Artistically respectable, innovative, and experimental films are only bought by collectors who place them on a shelf at home almost for prestige value not because they ever actually intend to watch them (sadly, I often fall into this category). The films that audiences constantly return to are those that have a good, easily accessible story, and move them in pleasurable ways. Why, when I have gone to the DVD rental shop recently have I completely failed to find any films that fulfil these requirements? Where’s the nerd-wins-cheerleader’s-heart films? Where’s the girl-becomes-dance/music-diva films? Where’s the fantasy films with imaginations bigger than my own? Where’s the intimate awe, the enrapture, the comfort?

I think Hollywood needs to either start taking itself VERY seriously and start producing artistic, philosophic, and intelligent films that don’t crumble under the pressure of the business men and focus groups. Or, they need to throw caution to the wind, stop trying to appeal to everybody, stop focussing too much on the technology, the glitz, the “cool” factor, and make films from the heart. Then maybe I’ll remember why I originally fell in love with film.

Edgecodes

2005-05-06T11:58:00.000Z

I stumbled across a wonderful documentary about the history/theory/practice/future of film editing the other day. I was reading this month’s Wired article on George Lucas and it mentioned an interview he did for a documentary called Edgecodes. Apparently, Lucas’ true calling is as an avant-garde filmmaker and experimental film editor and it is to this that he is planning to turn after StarWars (once he has finished cashing in with the merchandising, of course). The wired article is very interesting to learn about Lucas’ part in the evolution of special effects and non-linear digital editing (he personally financed the development of one of the first non-linear editing systems, Editdroid in the early 1980s). However, even more exciting is Edgecodes.

Edgecodes is a documentary produced by Travesty productions in Toronto. It is available for purchase on-line and can either be downloaded (thank god for broadband!) or purchased as a DVD from their website: www.edgecodes.com. I really wish I had encountered this documentary three years ago when starting my research. Without a formal film school education I have had to teach myself editing and the theory of continuity from books, dissecting films, and experimenting with my own films/animations. After 3 years (plus a lifetime) of a long hard slog I finally feel as if I have an understanding of how an editor approaches a film and how they regard the art of editing. Edgecodes gives you an insight to this in 75 minutes…..Nuts!

The film is primarily made up of taking heads discussing the development of film editing whilst examples are intercut with their dialogue and the entire film is edited in a way that highlights exactly what they are talking about. It is a wonderful resource for obscure early cinema clips such as Keaton in Sherlock Jr. walking into the cinema screen and being tormented by the editing. There is also a replication of the classic Kuleshov Effect. For those of you who aren’t familiar with this effect it was the first time that the potential of editing for creating meaning through juxtaposition of shots was formally shown. (Lev Kuleshov was a Russian Filmmaker and theorist who, in the 1920s along with the likes of Sergei Eisenstein, Vsevolod Pudovkin, and Dziga Vertov experimented with the potential of editing for the creation and manipulation of meaning, primarily as propaganda. The Kuleshov Effect refers to the juxtaposition of an actor’s expressionless face with one of three point of view (POV) shots: either a bowl of soup, happy child, or a dead woman. The supposed effect is for the viewer to project different emotions on to the face of the actor depending on the POV shot: hunger, parental pride, or loss and sadness.)

The most satisfying part of the documentary for me was to hear renowned film editors theorising on the reasons why they edit and why editing works. Like a lot of supposed opinion in this area most of what was said could be related back to text books, such as Zach Staenberg (editor of The Matrix) describing editing as being about making decisions based on graphic, spatial, temporal, and rhythmical relationships between shots which is a direct reference from Bordwell and Thompson’s ‘Film Art: An Introduction’ (1979/2003). The acceptance of this theory by film editors shows that either it must have some grounding in truth or the editors just don’t have any better way of expressing their intuition about their art. The same can be said for the comments about the reason why editing works: each comment can be directly related to film text books and, whilst thought provoking, don’t really get us any closer to an understanding. However, the overriding impression I gained from Edgecodes is that the editors are eager to understand, they spend their lives experimenting on themselves, trying to induce and control their experiences and so they are eager for any new technology, technique or theory that allows them to do this. I feel that they (and by result, their audience) will be the ones to really benefit from any greater understanding about the psychology of the edit I can produce. Here’s hoping.

Blogademia

2005-04-25T11:42:00.000Z

This is just a quick post to highlight some "sterling" research being done by a close friend of mine. Scott Nowson is studying the use of language in blogs and seeing if it reflects the bloggers personality. He was the person who inspired me to start blogging my own research as he has been doing this for over a year. Check out his work blog, Blogademia to get more insight into his research as well as any other tidbits of information of academic interest to the blogging community.

Blogging is a very interesting cultural phenomenon which is starting to become so popular that it is reshaping the way that information is distributed especially by the journalistic community (see news blogs such as Guardian's). Only recently has it begun to receive attention from academics but the work that is beginning to appear is very interesting. For a quick "gateway" to this research check out Scott's blog.

Sickened by Tarnation

2005-04-21T15:18:00.000Z

Ok, I have been studying continuity editing WAY too much!

I saw a preview of the Jonathan Caouette’s autobiographical film 'Tarnation' on Monday night and the sensory shock it inflicted on me made me physically sick for two days! For those of you not familiar with this festival favourite, Tarnation can be loosely described as an autobiographical documentary chronicling Caouette, a Texan-turned-New Yorker actor-come-filmmaker’s life. The main focus of the film is Caouette’s schizophrenic and institutionally abused mother and her effect on his own psychological development. The reason why such a “small” film has received such coverage and critical acclaim is its innovative use of existing still photographs, audio recordings, and home videos. This, melded together with Caouette’s camp aesthetic/personality/sexual development and his ability to push iMovie to its limits (the film was initially made on Caouette’s home iMac with no budget) makes the film as sensory shock to the system. In my case this shock was obviously literal!

I watched the film on the front row of a very small independent cinema (Cameo Edinburgh, check it out if you get the chance) and the rapid editing, constant visual manipulations (I didn’t know iMovie could make so many after-effects!), and stills-montage sequences bombarded my senses. Don’t get me wrong, this isn’t a criticism, I thoroughly enjoyed his creative use of the graphical potential of the material he was working with. He managed to imbue the photographs and poorly filmed videos of his youth with an energy and “camp” vitality that perfectly suited the subject matter. However, watching an entire film constructed from one such montage sequence after another is exhausting. And to top it off the subject matter isn’t exactly easy going either.

Caouette is obviously a very talented creative editor and he does seem to realise that his film can verge on being too much at times. He periodically presents calm soundtracked sequences of travelling landscapes which serve as a well needed respite but no sooner have you regained composure than he is plunging headfirst into another sensory and emotionally exhausting sequence. In reflection I think the brutality of this film when viewed on a big screen may actually be due to Caouette’s inability to think outside of the iMovie preview window. This is a common error made by rookie editors and is also those trained on television before moving to cinema. Their shots are often too close and too short. This works fine when viewed at a small viewing angle in a living room (in fact, it is preferable) but when projected on a cinema screen there is just too much on the screen for the eye to take in at one time. As the shots are also shorter the viewer doesn’t have time to scan the image with their eyes and pick up the important details and so they can often experience a sense of sea-sickness as the images move across their retinas in an unpredictable fashion.

This is my excuse for why I left the cinema feeling rather ill. That and the dodgy beer and chocolates I ate during the screening (if checking out the Cameo cinema, don’t drink the Stella Artois).

SCMS and C&T

2005-04-14T19:54:00.000Z

Between 31^st March and 8^th April I was travelling around the

attending conferences related to my Ph.D. research. The first was the Society for Cinema and Media Studies (SCMS) conference in London and the second was the Cinema and Technology conference in Lancaster. Both were highly enjoyable and very informative but differed quite considerably in terms of their scale and focus.

SCMS is one of (if not “the”) foremost international conferences in the area of cinema and media studies. As such it is HUGE! There were, on average, 17 parallel sessions in every time slot, covering such disparate topics as queer cinema studies, digital cinema, television, film and philosophy, as well as cognition and cinema (to name a few). My only experience of a conference like this was presenting at the Cognitive Studies of Moving Images conference last summer in Grand Rapids, Michigan By comparison that conference was a very intimate affair attended by academics who all shared a common interest in Cognitive Film Theory. Attending SCMS was a completely different experience as it represented a cross-section of all the different research that falls under the umbrella of “Cinema and Media Studies”. Not having a background in this academic discipline I found it a very useful experience to get an insight into the many different approaches/methodologies/theories/styles of intellectual enquiry that exist. It was also useful to see how Cognitive Film Theory sits within all the other theoretical traditions. Being a cognitive scientist I always approach things from a cognitive perspective and so it strikes me as very odd when other people are resistant to such an approach. I can now see how much work is ahead of us to make this path of intellectual enquiry universally accepted as an integral part of film (and media) studies as well as developing new theories that marry the strengths of cognitive science with the established traditions of film theory.

SCMS proved very beneficial to me as it gave me an opportunity to present by Watched pot/Stopped Clock experiment to a new audience (I won’t go into details here; they can be found on my main website). As always, I got some great feedback about my presentation which will be fed into my thesis and it was a great experience presenting with Lisa Fehsenfeld and Chris Robinson, my fellow panel members. Lisa discussed the potential for camera and actor motion in non-action films as a tool for manipulating the viewer’s experience (i.e. creating excitement, arousal, attracting attention). Chris then supplemented this by discussing the technical and physiological differences between viewing films projected at different frame rates. I then developed the level of scientific detail further and finished off the session by presenting an experiment investigating the perception of time across match-action cuts. (All our abstracts can be found in the proceedings here and my Powerpoint presentation is available on-line here). This progression from theory to experimentation worked really well and, I believe, allowed us to escort a non-scientific audience through the session culminating in a level of enquiry they may not usually be familiar with. The greatest thanks have to go out to Lisa and Chris for their work in making the session such a success.

Given the vast number of competing sessions I managed to follow a rather “cognitive” path through the rest of the conference. There was a session on the mis-use of visualisation in science (organised by Lisa Cartwright), a topic under a lot of debate in the scientific community. Cognition, evolution and cinema presented by Daniel Barratt, Mette Kramer, and Torben Grodal. Daniel has just finished a PhD at the University of Kent and has some great ideas on the psychology of “affect” as it relates to film viewing. The most relevant session for me was on “The Cinematic Mind: Cognition and Cinema”. This session appears to have been setup as an accompaniment for an undergraduate course Todd Berliner and Dale Cohen have been teaching at the University of North Carolina. I was aware of this course before attending the session and already knew that I wholly approved of and agreed with what they had been teaching but to see them present was hugely enjoyable. Their presentation attempted to invalidate the tradition of Sausaurean Semiology in film theory and replace it with theory based in Cognitive Science. They made reference to the same empirical evidence as I do and their line of argument follows a very similar line to my own so it was great to know that I’m on the right track. I don’t totally agree with their dismissal of semiology, I agree with some of the points Murray Smith raised in his response to the panel (generally revolving around the “specificity” of the cinematic image compared to the linguistic sign). I believe a lot can be learnt from semiotics (by this I mean the Peircean tradition, not the Sausurrean; see my introductory essay on semiotics) but Berliner and Cohen’s willingness to apply current cognitive science to film theory is very commendable. I wish them all the best with their own experiments in this area (and the rumoured book on the subject).

By comparison to SCMS’ international feel, the Cinema and Technology conference felt more British. This was in no way due to the attendees, C&T was attended by just as international and distinguished people as SCMS (in fact a large number attended both due their temporal and geographical proximity). I think the feel was largely due to the rain, the cold, the intimacy of Lancaster University’s campus, and the screening of the Mitchell and Kenyon collection which, for me, cast an air of Bygone-Britishness over the whole conference. This feeling was rather odd given that the focus of the conference was on the cutting-edge and multifaceted merger of cinema and technology. I have to confess that my aspirations for the future of cinema does feature the development of “true” interactive cinema (by “true” I mean an experience that resembles current cinematic experience as closely as possible whilst also enabling audience participation/interaction in the plot). As such this conference was right up my alley!

In general I’d have to say that the conference was a great success although I did yearn for a bit more technology and a little less theory (hey, shoot me, I’m a scientist!). It was nice to see so many people interested in the implications of DVD, videogame, digital effects, CG, and interactive media. These technologies are subtly changing what it is that we current see as “cinema” and this conference showed us that we need to work hard to develop new ways of analysing the resulting media and new theoretical frameworks to understand the future of cinema. It is an exciting time for academics and the public alike and I hope that this conference just marks the beginning of the academic interest in these developing technologies.

Just quickly I’d like to mention a few people from C&T:

Lanfranco Aceti from Central St Martin’s College, London is looking at the use of neuroimaging as an interface for avant-garde film generation. He’s a really nice guy who is tackling a very interesting but difficult area. All the best to him.

Jim Bizzocchi is a researcher/lecturer in new media and film studies from Simon Frasier University, Vancouver. I met him last year at CCSMI and he was instantly supportive of my research which I was hugely grateful for. His ideas on the future of interactive media and the theoretical analysis of videogames are right on the button and I always have a great time chatting to him about these topics. If you ever have the opportunity to read some of his work or see him present I would highly recommend it.

Jonathan Frome is a Ph.D. student/lecturer from University of Wisconsin-Madison. His thesis is on ‘Imagination, Immersion, and Emotion: Video Games and Visual Media.’ and he is supervised by David Bordwell (lucky guy). He wasn’t actually at C&T (he was at SCMS) but given that his research is C&T related I thought I’d mention him here. He’s developing some great frameworks for the theoretical analysis of videogames with specific emphasis on the experience of the player/user. Keep an eye on this one, he’s going to do some great things in the near future :)

Well I guess I’ve rambled enough about my conference visits. I returned to Edinburgh motivated and driven to actually start writing my thesis (Thank god!) and also develop this blog/website as a resource for other people interested in this research. So far the blog is going well (even though I need to work on writing smaller posts!) but the thesis writing is starting slow. I’m writing an overview/survey of continuity editing rules which will form the foundation of my thesis. Its fun to work on but collating all the existing definitions and experiences takes a long time. I’m sure this won’t be the last time you’ll hear reference to it.

Wish me luck.

A quick word on "Continuity Boy"

2005-04-12T11:57:00.000Z

I just wanted to take a moment to explain the name of my blog. “Continuity Boy” is a play on the classical Hollywood tradition of referring to the person who supervised the script and the continuity between shots as a “Continuity Girl”. This has recently transmogrified through political correctness to “Script Supervisor”. This person (who is pretty much always female, hence the demeaning use of “Girl”) has the incredibly difficult task of checking that all shots specified in the script are filmed during production, correctly labelled so that the editor can locate them, and that all details of wardrobe, makeup, actor and prop locations remain the same across shots that may be filmed months apart. Whenever film viewers spot “continuity errors” in a film, e.g. a cigarette that is suddenly burnt down across shots, these are mistakes made during production that should have been fixed by the script supervisor. However, the blame for bad continuity cannot be solely placed on the script supervisor as their job requires them to tell many other members of the production staff how to do their job in a way that creates eventual continuity and as such miscommunications and power struggles can often make this task very difficult. Script Supervisors get my up most respect.

A good description of a script supervisor’s job can be found here.

The reason why script supervisors are almost always female is due to the gender differences in our ability to store and compare visual information. Females typically perform better on tasks that require visual information to be processed in parallel with other sensory signals and then stored in memory. This is thought to be due to the fact that the corpus callosum, the part of the brain that links the right and left hemispheres, is a fifth larger in women. This means women can process multiple sensory signals and perform parallel cognitive processes (such as perception and storage in memory) better than men. There is also a theory that oestrogen levels in women give them an added advantage in spatial memory. By comparison men are better at extracting spatial information from a visual scene and reconstructing/transforming this information to compare views of a space (this is best exemplified in mental rotation tasks). These different styles of visual information processing mean that men and women typically differ in how they approach a task such as reading a map. Men can abstract from the spatial relationships represented on a map (e.g. corners, distances, sizes) to the real space quite well whilst women typically prefer to use landmarks and salient visual features.

This increased attention to detail explains why women are typically better at spotting and keeping track of continuity errors across shots (although a Polaroid/digital camera also comes in handy).

Sorry to disappoint the readers who thought the name “Continuity Boy” indicated that I was actually a superhero sidekick. I wish. Although the image induced by that interpretation is one that warrants a cartoon……

And so it begins.....

2005-04-12T10:21:00.000Z

So here it is: the first post to my blog chronicling my research on the perceptual foundations of continuity editing.

The what? (I hear you call).

Well I've decided to start writing this blog because I'm meant to be writing a thesis, and this provides a good distraction.... No seriously (mostly), I'm writing it because, as with all webpages, I built mine over a year ago with the intention to continually update it with details of my Ph.D. research but once it was semi-complete there doesn't seem any reason to tinker with it. As such it has laid in a rather stagnant state which means it is of only temporary use to anybody who is interested in this area of research. I believe one of the main reasons why more people haven't been working in this research area is that there is no "way-in" for the casually interested academic/student/member of the public. During my travels to conferences and my e-mail correspondences I have spoken to a lot of people from many different areas of academia who are asking the same questions but don't know how to answer them. Hopefully, this blog and the associated webpages will provide some insight into the way I have attempted to answer some of these questions. It is not intended to be a definitive guide to this area but by outlining my ideas, references, and methodologies I hope to point interested researchers in the right direction.

But wait! What questions?

Well, the topic of my Ph.D. thesis is the empirical investigation of the cognitive processes involved in film viewing. My specific aim is to use empirical methods to investigate how the natural processes involved in visual perception can be used to validate the conventions used by film editors. When editing a film, a film editor constantly has to make decisions about how and when to cut between shots. These decisions function on many levels. Does the cut drive the narrative of the scene? Does it induce the right emotion in the viewer? Does the action flow smoothly across the cut? To simplify the editor’s task conventions exist that allow the editor to quickly arrange a “rough” sequence of shots which can then be tweaked for stylistic or affective reasons. These conventions, referred to as the rules of continuity editing, can be found in any film theory text book and are taught to film students the world over. Yet, the reason why the conventions exist is not really known. Editors will take a stab at explaining the benefit of one composition resulting from the application of a convention over one which violates a convention but their explanation is only built on introspection and hearsay. As with all artists, an editor’s craft is one of feeling and intuition and as such it is not their job to express the conceptual steps they take whilst creating their art. This task is better suited to somebody who can approach the problem objectively and utilise methodologies that allow the problem to be dissected via the testing of specific hypotheses. By applying empirical methods to the task of understanding the cognitive processes involved in film viewing we can begin to understand the moment-by-moment behaviour of film viewers. This allows us to better understand how the decisions made by an editor affect the resulting experience of the film viewer and, in turn, validate the conventions or continuity editing.

So that is a brief introduction to the research area that will be the subject of this blog. More information on my specific methodologies, experiments, and theories can be found on the rest of this website. As for the future of this blog, well my intention is to use it primarily to discuss my ideas as they develop over the course of writing my thesis. I probably take the occasional detour into related themes (e.g. topical discussion of films) as well as any references, interesting websites, books, etc that crop up. But for now I’ll think I’ll leave it there. Please feel free to comment on any of my posts and contact me if you have any requests for information/facilities I should provide on this website.

All the best.

Tim J. Smith (aka Continuity Boy)