Thursday, April 26, 2018

ZDOK 18 - Attentional theory presentation

I was very honored to be invited by Christian Iseli to present at the Zurcher Dokumentarfilmtagung (ZDOK; Documentary Film Workshop) in Zurich on 22nd March. This gave me a great opportunity to present an overview of my Attentional Theory of Cinematic Continuity (Smith, 2012; official url; free pdf preprint) to filmmakers and hear their thoughts on my cognitive approach.

A video of my talk is available here:
https://blog.zhdk.ch/zdok/2018/exploring-the-cognitive-foundations-of-cinematic-continuity-2/



I'd also highly recommend checking out the other talks from what proved to be a fascinating conference (even if it pushed my German skills to the limit!). Of special interest to me was the always fabulous, Karen Pearlman and a wonderful discussion with one of my favourite editors, Mathilde Bonnefoy on her work on the Edward Snowden documentary, Citizen Four and her longstanding work with Tom Tykwer on various films including Run Lola Run:

https://blog.zhdk.ch/zdok/zdok-18_archiv/

p.s. Thanks to Christian Iseli, Miriam Loertscher and Kristina Jungic for organising my visit and the fantastic real-time translators.

Monday, March 07, 2016

IMC Bootcamp Video - Eye-tracking in the wild and in film

On Tuesday 24th November, 2016 I was lucky enough to be invited by Katrin Heimann and colleagues from the Interacting Minds Centre (IMC) at Aarhus University, Demark to take part in their bootcamp on Visual Attention. The IMC is an incredible interdisciplinary research institute investigating issues of social cognition and related phenomena from a variety of perspectives. The visual attention bootcamp was a great success and I thoroughly enjoyed learning about the work and approaches of the other speakers. 

I had the pleasure of ending the day with a summary of my research on film viewing. In the talk I touched on issues of how to run eye tracking experiments with dynamic stimuli, how to use computer vision to analyse film content, the evolution of cinema and its relationship with visual cognition, an overview of my Attentional Theory of Cinematic Continuity (AToCC; Smith, 2012; available here), and how individual differences manifest in where we look and what we think whilst watching movies. The resulting video is great and provides a great overview of my work.

Check it out and let me know what you think via Twitter @timothy_j_smith.  


Thursday, July 30, 2015

Movies in your brain - The Academy of Motion Pictures event





(This is a terribly belated write-up! But better late than never.)

Last July (29-30th, 2014) I was very privileged to be asked to host a two night event in the Academy of Motion Picture Arts and Sciences (AMPAS),  Linwood Dunn Theatre in Los Angeles. I had been approached for my work on the cognitive science of film viewing as AMPAS were interested in hosting an event that would bring together filmmakers with scientists to explore, for the first time  the way viewers process images, events and stories experienced on the silver screen. Of course, I jumped at this opportunity as it would give me the chance to pick and choose who I considered some of the most interesting filmmakers working today along with some of the most brilliant and inspiring cognitive scientists and orchestrate the discussion I had always dreamed of.  

The title of the event was "Movies in Your Brain: The Science of Cinematic Perception" and attracted a sold-out audience of Academy members, general public, scientists and filmmakers. The structure for the two nights was:

Night 1
The event consisted of film clips, discussions and presentations from:
Walter Murch (film editor/sound designer/director; Apocalypse Now, The English Patient and The Godfather Part II)
Jon Favreau (actor/producer/director; Iron Man, Chef, The Jungle Book)
Prof. Uri Hasson (Princeton U., expert in film perception, social cognition, and neuroimaging)
Prof. James Cutting (Cornell; expert in visual perception, cognition and the evolution of film form)

Night 1 also involved a live audience experiment testing Walter Murch's ideas about blink entrainment during The Conversation and how he believes he can guide an audience's attention and shape their perception during a film by using composition, editing and sound design. AMPAS have created a short video which captures some of this discussion between myself, Walter Murch and Jon Favreau.




During Night 1 we also conducted an eyetracking experiment on the Monaco sequence from Favreau's Iron Man 2. This data was gathered as participants arrived at the event using a Tobii TX300 tracker, quickly rendered into a heatmap video which shows where the densest concentration of viewer gaze is during the video and then I had the joy of discussing this data with the director responsible for creating the film, Jon Favreau. His insights into how this synchrony of attention was created was surprising and often very funny.




Night 2
The evening had a similar structure to night 1 but with a focus on the work of:
Darren Aronofsky (director/writer; Requiem for a Dream, The Fountain, Black Swan, Noah)
Ari Handel (screenwriter/producer/neuroscientist;  The Fountain, The Wrestler, Black Swan)
Prof. Jeff Zacks (Washington U.-St. Louis; expert on event perception, neuroimaging and author of Flicker: Your brain on Movies; see a great video on Jeff's work here)
Prof. Talma Hendler (Tel-Aviv; expert in neuroscience, emotion, social cognition, and empathy in the movies)

Night 2 increased our insight into the what is going on in the brain of viewers whilst watching a movie. As provocation for delving into the extremes of cinematic experience we featured the work of Aronofsky who's films often push the boundaries of audience emotions by using tropes from horror, thrillers and deep portrayals of psychological distress and disorder (such as schizophrenia in Black Swan). Hendler, in conjunction with Uri Hasson ran a neuroimaging study on the final 15 minutes of Black Swan in an fMRI scanner. The data illustrated how audiences both experienced extreme distress at the shocking sequence, empathised with Nina, the main character played by Natalie Portman as she undergoes a psychological and physical break down but also control their emotional responses as a form of psychological self-defense. The research showcased in this demo is truly stunning and paves the way for future work into the neural mechanisms of rich cognitive and emotional cinematic experience. You can find a video on Hendler's work here

Darren Aronosfky and Ari Handel's responses to the science demonstrated a great insight into their craft and how they explicitly make directorial and scriptwriting decisions to either elicit a shared, universal experience in their audience or create moments of fracture in which audience members take something unique and idiosyncratic from their movies. The AMPAS clip below beautifully captures this discussion with myself and Jeff Zacks.



The event ended with the presentation of a short test film the Aceademy Tech Council had created as a dramatic demonstration of the multiple possibilities of high frame rate (HFR) filmmaking. A single-shot, silent short entitled “The Affair” was filmed with identical camera moves and identical performances more than 30 times at different frame rates: 24fps (the cinematic standard), 48fps, 60fps, 120fps and several speeds in between. Four frame rates (24, 48, 60 and 120fps) were presented to the audience. The leap between each was both fascinating and divisive, with a show of hands revealing how much further we all have to go before deciding what the experience of watching and absorbing a movie will mean in the future.

The two nights were a stunning success and I really can't do them justice describing them here. AMPAS have provided a great write-up on their website (here) and we were very fortunate to have the science of the event featured by Wired on-line as a series of articles (here). If you want to get a sense of what was discussed I would immediately direct you to these existing articles.

I have to thank AMPAS for hosting this event, the AMPAS staff (Academy Governor, Bill Kroyer; Maryrose McMahon, Joe di Gennaro and their crew), the filmmakers with taking time out of their busy schedules to blow our minds and the scientists for sharing their work and inspiring me and the audience. I hope this will be the first of many such events the AMPAS and other such societies host in the future. 


Tuesday, June 16, 2015

Poster on using low-cost eye trackers for film cognition experiments (SCSMI 2015, London)

The Society for the Cognitive Studies of the Moving Image (SCSMI) annual conference is about to kick off at Birkbeck. All our preparations are ready and we're really looking forward to hosting old and new colleagues for this exciting international event. Check out the programme for some very interesting events and if you want to pop along you can book a day ticket here: http://scsmi-online.org/conference/registration-information

Tomorrow evening, my student, Jono Batten and I will be presenting a poster and giving a demo of some recent low-cost ($140-$99) commercial eye trackers that have appeared on the market over the last year. Whilst intended as gaming peripherals and interface devices they offer researchers interested in getting into eye tracking but unable to afford the massive price tage of most science-grade systems a potential way in. We have tested and investigated these new low-cost eye trackers in considerable depth and in the poster we present a review of which trackers you should consider for particular experiments.

As I will likely be running around hosting the conference during the session I've also included the poster here.

Some of the links mentioned in the poster are here:

Eye Tracking Hardware
Eye Tracking Software


Wednesday, December 24, 2014

Laptop vs. IMAX: An eyetracking experiment

As part of BBC Radio 4's Cells and Celluloid Christmas special on the science of film I conducted an experiment investigating the impact screen size had on viewing behaviour. Using head-mounted eyetracking equipment (SMI Glasses) I recorded the eye movements of Francine Stock and Adam Rutherford as they watched Fly Me to the Moon 3D (2008) either in the Science Museum's IMAX or on a 13 inch laptop.

Full discussion of the experiment will be available on the BBC iPlayer here:
http://www.bbc.co.uk/programmes/b04vj1x8



As you will see from the video, the IMAX screen filled the entire field of view of the head camera (>60 degrees) where as the laptop screen only filled less than half of the image even when it was viewed at a typical viewing distance (~60cm), This difference in viewing angle meant that the gaze had to explore more of the image in the IMAX presentation producing a greater number of saccades into the periphery. The difference in gaze exploration of the image can clearly be seen in the scatter plot below. All the gaze to the laptop (green crosses) is tightly centred relative to the head camera whereas the gaze during the IMAX viewing (blue circles) is higher (possibly due to the viewer being positioned in the back row of the auditorium) and covers much more of the image.




These data give us an initial indication of how the viewing behaviour between small screen film viewing and large format (such as IMAX) may differ but to check whether these results are robust we would need to conduct a more thorough empirical investigation using exactly the same section of a movie and a larger number of viewers. A lot has been claimed about the "immersive" effect of large-screen viewing and these sorts of studies can begin to identify exactly what these differences might be in terms of viewer experience. There have not been enough real-cinema studies for us to be able to draw strong conclusions but those studies that have been conducted indicate that even when the amount of visual field occupied by a screen and the amount of light reflected off the screen is equated between a home cinema presentation and a large screen (e.g. at the cinema) our knowledge that the screen is physically larger creates an increased experience of immersion (Troscianko, Meese & Hinde, 2012). More research is needed to identify exactly what aspects create the "cinematic experience" and to predict what the future holds for this experience given the increased competition for our attention.

Thursday, June 26, 2014

EPSRC 3.5 year PhD studentship: Eye guidance in real-world scenes (UK students only)

Can't recommend this opportunity more highly. This is a chance to work with two pioneers of active vision (Nuthmann) and computer vision (Fisher) in one of my favourite cities on the planet!

.....
Dear students,

A (last-minute) fully funded(!) PhD studentship is available to work with Dr. Antje Nuthmann (principal supervisor) and Prof. Bob Fisher (co-supervisor) at the University of Edinburgh. 

Details can be found here: http://www.ppls.ed.ac.uk/students/postgraduate/PGFunding.php#EPSRC

Please don't hesitate to contact me if you have any questions.

Best wishes,
Antje Nuthmann
http://nuthmann.de/antje/Site/Welcome.html

Sunday, November 03, 2013

BIMI study day: Cognition at the Movies




I am proud to announce the first 'Cognition at the Movies' study day hosted by myself and Birkbeck Institute for the Moving Image (BIMI).

Saturday November 9th 10am to 6:30pm (B35, Birkbeck, Malet Street, London, WC1E 7HX; number 1 on the map http://www.bbk.ac.uk/downloads/centrallondon.pdf)

Since cinema’s inception filmmakers and theorists have been interested in the relationship between film and its audience. How do directorial decisions influence what we see on the screen and how does a viewer’s prior beliefs and interests influence how they experience a film? Cognitive Science, the interdisciplinary investigation of mental phenomena using theories and techniques from neuroscience, psychology and philosophy has recently begun to be applied to these questions of film cognition. This workshop will bring together film theorists, cognitive psychologists and philosophers in an exploration of the relationship between film and its audience.

Keynote presentation ‘by Prof Torben Grodal (Copenhagen) author of Moving Pictures and Embodied Visions.


Comic Entertainment, Film, and the Embodied Brain
The lecture will first provide a short description of how muscles and action is important for the embodied brain and for our experience of narratives.  The basis for the standard narrative reflects the Brain’s PECMA flow: Perception, Emotion, Cognition, and Motor Action. Characters and viewers want to modify some states of the world by motor action, including verbal actions. The lecture will then discuss the embodied brain’s three ’bail out’ mechanisms where the modification of the world by action is supplanted with self-modification: Crying, as in sad melodramas, laughter, as in comedies, and freeze reactions as effects of sublime submission to the exterior world. The lecture will especially focus on comic entertainment and discuss the processes that allows the brain to evaluate something as ’not real’, as ’not a cause for action’ and redirect the arousal from a given scene from tense world-directedness to laughter. It will finally discuss the social nature of comic entertainment and those mammalian play-functions that serve as facilitators for the reality status evaluations in comic entertainment that makes it possible to experience shame, failure and other negative events with a strongly positive hedonic tone.

Event is free but please register as spaces are limited: https://cognitivism.eventbrite.com/

Schedule:


9:30-10 Registration

10-10:15 Welcome and Introduction
10:15-11 Prof. Ian Christie (Birkbeck) - Psychology in the dark: just what is it we want to know?
11-11:45 Dr. William Brown (Roehampton) - He(u)retical Film Theory: Cinema and the Brain
11:45-12:30 Prof. Sheena Rogers (James Mason U.) - Towards Transcendence: Cognitive Components of the Sublime in Art

12:30-1:30 Lunch break

1:30-2:30 KEYNOTE: Prof. Torben Grodal (Copenhagen) - Comic Entertainment, Film, and the Embodied Brain
2:30-3:15 Dr. Paul Taberham  (Kent-Canterbury) - Avant-Garde Film in an Evolutionary Context
3:15-4:00 Prof. Murray Smith (Kent-Canterbury) - Murder Ballads

4-4:15   Coffee break

4:15-35 Steve Hinde  (Bristol) - A Study of Attention  while People Watch Movies.
4:35-5 Parag K. Mital (Goldsmiths) - Resynthesizing Perception
5-5:45 Dr. Tim J. Smith  (Birkbeck) - Cinematic Universality: Do I see the same movie you see?
 
5:45 - 6:30 Discussion


6:30 + Wine reception

For further information email tj.smith@bbk.ac.uk