It Lookey like Lars Von Trier is at it again.

Not content with creating a revolution in filmmaking by spearheading the Dogme 95 movement, Lars Von Trier is now experimenting with how his films are shot and how viewers engage with them. His new film, The Boss Of It All, is filmed using a new camera control technique developed by Von Trier called Automavision. The system removes the need for a human camera operator, replacing them with a computer which randomly selects camera shots and movements based around an initial camera position selected by the director. The system removes the usual control the director and cinematographer have over the composition of each shot and, specifically it’s framing. Lars Von Trier explains his desire for developing this technique:

“I am a man who likes to control things, and if I can't control them totally I will not control them at all. After doing Europa with very very fixed shots and camera movements, I was tempted to do something totally different. I started using a handheld camera and we invented a form of framing, or non-framing, called pointing of the camera, because I hate framing.” (guardian.co.uk)

This hand-held, almost haphazard method of framing combined with non-continuity editing had recently become Von Trier’s trade mark. Even after he deviated from the original strict edict of the Dogme 95 manifesto with Dancer in the Dark, Dogville and Manderlay he retained this style of framing. The style deliberately breaks with the traditions of the Hollywood continuity style by forcing the viewer to actively search each shot for the most significant elements. In the Hollywood style these elements, for instance a protagonist would be centred within the frame and lit in a way which made them highly salient even in busy scenes. In Von Trier’s recent films the protagonist is often cut off by the frame, visually diminished by other less important elements, or moving at odds with the hand-held camera movement. In combination with his often uncomfortable subject matter, this unpredictable framing technique often leads viewers to describe the experience of watching a Von Trier film as “hard work”.

In a series of recent eye tracking experiments I have shown that this difficulty in following the action of a Von Trier film can be very clearly seen in viewers’ eye movements. When watching a film composed according to the classic continuity style, all viewers will focus their attention on a small number of objects within a shot. In most shots there will only be one clear centre of attention, usually the face of a principle actor and it will be this that all viewers track within the shot and across cuts. By comparison, viewers watching Dancer in the Dark or Dogville distribute their attention across more of the screen and show less agreement of what they believe to be the most significant object. When a cut then happens (which they often do at unexpected moments in Von Trier’s films) viewers are not guided to the new centre of attention by the director so they have to actively search the scene. This active engagement with the visual constituents of the film creates a viewing experience that is completely counter to the normal smooth, direct, almost passive viewing experience of a classical continuity film.

Von Trier’s desire to create films that actively engage viewers can also be seen in his use of Brechtian theatrical technique. In Dogville and Manderlay the use of black backdrops, minimal props, and transparent scenery expose the artificiality of the film. Bertolt Brecht developed these techniques, amongst others, as a way of encouraging his theatre audiences to adopt a critical mindset. By actively counteracting the “suspension of disbelief” encouraged in classical Artistotelian theatre, Brecht was trying to engage his audience in the critical interpretation of the depicted action, the act of its construction, and its place within real-life. The application of these techniques to cinema by Von Trier also resulted in an extra level of visual engagement beyond that created by the non-framing camerawork. By populating the set with transparent scenery the director is unable to hide insignificant actions. All actors must be present on set at the same time and act even though their actions are not important to the current shot. These peripheral actions crowd in on the main action, drawing the viewer’s attention away from the main action and creating further disagreement between viewers.

However, the natural instinct for framing is hard to overcome and it appears that Von Trier realised that his desire for completely un-framed shots would not be possible so long as he or his camera operator were controlling the camera. A hand-held camera is often described as a visual prosthesis: an extension of the camera operator’s eye; seeing what they see. The movement of a hand-held camera may be rough and the framing imperfect but, like the human eye it will always eventually settle on the most important parts of a scene. If Von Trier is to create shots in which viewers are unable to predict what is the most significant part of a scene or how the camera is going to move he needs to take the human camera operator out of the equation. Hence, Automavision.

Automavision is not the only innovation in Von Trier’s new film, The Boss Of It All. Von Trier recently announced that he has embedded five to seven “Lookeys” in the film:

"For the casual observer, it's just a glitch or a mistake….For the initiated, it's a riddle to be solved. All Lookeys can be decoded by a system that is unique." (Von Trier quoted on Screendaily.com)

Von Trier is offering 30,000 Danish kroner (£2,700) to the first Danish viewer that identifies all the Lookeys. The Lookeys are described as “visual elements that are out of place” (www.lookey.dk) and are intended to turn the film into a “mind game”. By informing his viewers of the presence of these Lookeys Von Trier is again encouraging his viewers to actively engage with his films in a way in conflict with the normal film viewing. Spotting continuity errors, which is how these Lookey’s would be described if they were unintentional, has been an occupation of film viewers throughout the history of film. The pastime has escalated to such a level that there are even books and websites devoted to it. Continuity errors are typically mistakes made during production that are spotted by viewers on repeated viewings of a film. The most common errors are unintentional costume changes across shots or cigarettes and drinks that disappear or refill.

Finding errors is always good fun although it can often be very difficult. In my Ph.D. thesis I created a taxonomy of continuity errors that classifies them according to when they are made during the film’s production process and what is required to spot them (page 186 of my thesis). The detection of errors is very dependent on how the viewers watch the film. If the viewer looks at the parts of the screen the director wants them to look at they should never be aware of any continuity errors. The focal objects (those at the centre of attention) should never have errors as they would have been spotted by the director, cinematographer, or editor. Errors are more likely to be located in the periphery of the screen, areas where the production crew and the average viewer are unlikely to look. As film is a dynamic medium and a viewer can only focus their attention on one small part of the screen at any one time, peripheral errors should be missed as viewers are rationing their attention to the most significant parts of the screen.

In a traditional Hollywood film composed for continuity, errors are hard to spot as viewer attention is so carefully influenced by the director and editor. In a Von Trier film, as already discussed, the intention appears to be to encourage active search of the screen and disagreement between where different viewers look. With the addition of Automavision the likelihood that a viewer looks at a part of the scene that would traditionally be insignificant probably increases. This may mean that viewers of The Boss Of It All may be more likely to spot the Lookeys than if they had been included in a traditionally composed film.

However, without knowing exactly what form these Lookeys take (see page 186 of my thesis for a taxonomy of errors) or what expectations the viewer will have to have to realise that the Lookeys are errors we cannot know whether they are easier or harder to detect than traditional continuity errors. What we do know is that the experience of watching The Boss Of It All is, like in all Lars Von Trier films going to be unlike watching any other film.

Re-viewed

Last Monday (23/10/06) I had the good fortune of attending the opening of a new interactive art exhibit at the Leeds Metropolitan Gallery. The exhibit is called Re-possessed and presents a collection of interactive art installations inspired by Alfred Hitchcock’s Vertigo. Each installation takes a section of the film and provides the participants with novel ways of exploring the film, their experience of it, and the issues of power, gender, and voyeurism that are so prominent in the film. The exhibit runs until 18^th November and I highly recommend it if you are interested in film or just want to play with some highly innovative interactive exhibits.

The main reason I was invited down for the opening was because one of the artists, Richard Stevens had stumbled across my research whilst looking for references about applying eye tracking to film. His installation, Re-viewed uses a Tobii eyetracker to record viewer’s eye movements whilst watching scenes from Vertigo and then represents the eye movements using some very interesting visualisations that Richard and his students have created. These visualisations give the visitors an insight into their perceptual experience of film that they have never previously had. It also emphasises group differences such as between Male and Female viewers. Given that gender politics and the resulting power relationships are very prominent themes in Hitchcock’s work (see Laura Mulvey’s article on the Male Gaze in her 1975 book “Visual Pleasure in Narrative Cinema”) this method of using eyetracking to extract the gender differences in the viewing experience could prove very informative.

The reason why Richard contacted me is because he and his collaborator, Tony Renshaw were uncertain how to interpret the eye movement data given that so little work has been done in relation to film. They also needed some clear way of expressing any group differences that they observed. Currently, Richard’s visualisations can display the moment by moment differences but in order to report and quantify these differences precise measurements are required. I have encountered similar problems with my own eye tracking work and have been developing suitable measurements. Hopefully I will be able to aid Richard and Tony in the analysis of their data and we will find some interesting group differences. We’ll then work towards publications based on the results. I’ll keep you posted if anything emerges.

I am very excited about this exhibit as, to my knowledge it is the first time eye tracking of film has been performed on such a large scale with a large cross-section of the population. Most eye tracking studies (all Psychology studies for that matter) are usually conducted on University undergraduates; the most easily accessible subject pool for university researchers. The number of subjects eye tracked is also usually less than 20 so there is no scope for exploring group differences. Hopefully, through the development of new measurement techniques and automation of the analysis we can extract some interesting results from this study which will further our understanding of the viewing experience.

So, if you are any where near Leeds in the next two weeks head down to the Leeds Met Gallery. You’ll be furthering scientific knowledge whilst having a fun and informative time.

End of an era + Catch up

Today I moved out of my PhD office. Even though I finished my thesis in January, graduated in June, and have been working on a completely different project (Le Active Math) since last October I had somehow managed to remain at my old desk. Today marks the start of Edinburgh University Fresher’s so new PhD students were coming to inhabit my office which meant that I finally had to cut my ties with my old desk. So sad….so many memories….and too many long hours spent at that desk.

My departure from the office was eased by the fact that most of the office mates I had shared it with during my Ph.D. had also left. Scott Nowson started his new life at Macquarie University in Sydney three weeks ago (you can check on his progress at http://nowsonexitmusic.wordpress.com/). Sarah Gingell finally finished her Ph.D. (after too long to record here) last year and has moved on to a real life. Colin Fraser went part-time on his Ph.D. and is now touring Europe promoting e-Democracy (you’ll have to ask Colin to explain what this entails). Zoe Bruce (and Morgan the Dog) left a couple of years ago to pursue a career in academic publishing (Zoe that is, not Morgan). We all had some great times together. In fact we may have had too many great times considering how long our research ended up taking J. Still, I wouldn’t have traded you guys in for anything. Thanks for keeping me company.

Now I find myself divided, literally. I’ve got a “hot desk” (what a cool name for what is essentially a desk that anybody can sit at) in the same building in Informatics which I shall be using most days for my Le Active Math work. I also have a very nice office all to myself in John Henderson’s lab in Psychology. That is where all the interesting blog-relevant work will take place. I’m really excited about starting full time over there but this won’t happen until January. In the meantime I have to run a MASSIVE evaluation of Le Active Math….. Wish me look.

Blog posts I should have written and still might:

• Cognitive Studies of the Moving Images Conference, Potsdam, Germany (20/7-23/7/06). Fantastic conference, great people, interesting research, and fabulous reception for my work. The future of Cognitive Film Theory is bright. See Bruce Hutchinson’s account of it at bdhutch.blogspot.com.

• Narrative and Interactive Learning Environments conference, Edinburgh (8/08-11/08/06). I really enjoy this conference as it attracts such a small, friendly, yet hugely intelligent group of people. I was one of the organisers this year and found myself stage managing and being sound and lighting engineer for a multimedia performance group called Palindrome. These guys do visionary things with motion capture and real-time music production. We were really lucky to have them take part in NILE.

• Active Vision workshop, Dundee (1/09/06). This is a great little workshop organised by Ben Tatler. Lots of interesting presentations about eye movement research from a diverse range of disciplines. The atmosphere was really relaxed which resulted in a lot of discussion and humour. This workshop was a great example of how many interesting eye movement researchers there are across the UK. I’m really looking forward to being more “active” in this community (Sorry, I couldn’t help it)

• Edinburgh International Film Festival (14/08-25/08/06). God I love living in Edinburgh! August is a month of cultural, artistic and pleasurable excess that always amazes me. The high point for me is always the film festival. I see an unhealthy number of films each year and 2006 was no difference. Highlights include Brothers of the Head, Black Sheep, Host (Gwoemul), Snow Cake, Clerks 2 (and attending a Q&A with Kevin Smith), and Looking for Cheyenne (Oublier Cheyenne). Thanks to Shane Danielsen the festival director who programmed a fantastic final festival. You shall be missed. Especially your bizarre digressional Q&A style and uncontrollable flirting. Fantastic.

Actually I think I just ended up writing those blog posts!

You can’t get rid of me that easily.....

I am very happy to announce that I shall be continuing my research for at least another year. I have been accepted for a post-doctoral research position in John M. Henderson’s Visual Cognition lab in the Edinburgh University Psychology department. Now I know what you’re all thinking: “I thought he was already in Edinburgh Uni”. You are correct. My current status is as a Research Associate for the Le Active Math project employed by Moray House School of Education at Edinburgh University. However, my move to Psychology will be a very fortuitous one as it will allow me to conduct my research without distraction and within a group of likeminded people.

This move to psychology has only been made possible by the uncannily well-timed move of John Henderson’s visual cognition lab from Michigan State University to Edinburgh. John and his partner Fernanda Ferreira were offered chairs in our psychology department and they were wise enough to accept. In the simple signing of a contract Edinburgh has suddenly become a hot-bed of eyetracking and visual cognition research. John, for those of you who don’t know, is one of the foremost researchers in the field of visual search, scene perception, and visual cognition in general. If you read my thesis you will find that he is one of my most cited people. His work is highly influential and I am sure that it will continue to be so from his base in Edinburgh.

John has been very encouraging of my research and interested in the theories I have been developing. As a post-doc under John I shall be continuing my research into the use of attentional cues in film to create the perception of continuity. I will also be expanding my theoretical, methodological, and writing skills to begin establishing myself as a visual cognition researcher.

Thank you, John for this opportunity.

So it looks as if you (reader) won’t be getting rid of me or this blog for a few years to come :)

Rock ‘n’ Roll Research

or “How to do an experiment in two days”.

I was contacted at the start of July by a Gustav Kuhn from Durham University. Gustav is doing some really interesting research looking at how magicians use misdirection and perceptual expectations to make you perceive something that actually isn’t there (Gustav is a highly talented magician himself. He’s available for children’s parties, corporate events, bar mitzvah’s …..). He’s done a series of eyetracking experiments to examine where people look whilst watching magic tricks. To satisfy a reviewer of one of his papers he needed to run an eyetracking study and he needed to do it fast. *Cue the Mission Impossible music*

Unfortunately, Gustav’s eyelink II system was not setup yet so he asked if he could run his study using the equipment we have here. I agreed with three small provisos:

1) the study had to take place after the 11^th July as I was at a project meeting in Germany before then;

2) I would have to run some of my own stimuli on the subjects whilst they were being eyetracked;

3) the study had to be finished by the 14^th July as I had to fly to Germany (again) to present at a conference (CCSMI, more later).

In the course of 3 days I managed to prepare two sets of stimuli for my part of the study, design and implement the experiment using the Experiment Builder software which I had never used before, test, pilot, and run 13 subjects through the experiment, and process the data so that Gustav could do his analysis. Phew! Gustav was up in Edinburgh for a day and a half and in that time we managed to get all the data we needed and, through subsequent analysis, Gustav got the result he needed to appease his reviewer. Now that’s how science should be :)

Unfortunately, the pace for my part of the experiment has slowed down somewhat. I tested two types of stimuli. The first were the videos I used in the editing memory experiment. I wanted to see which details of the videos viewers were using to detect the editing discontinuities. I also had some hypotheses about how the discontinuities would effect their eye movements (see d'Ydewalle, G., Desmet, G., & Van Rensbergen, J., 1998 for similar effects). To extract these effects I need to examine each video by hand and then perform some complex statistics on the eye movement data. Sadly this can not be done in a day L I’ll publish these results in the Editing Memory sometime soon.

The second stimuli I eyetracked were feature films. I have been desperate to get eyetracking data for films with different degrees of continuity for ages and Gustav’s urgency finally motivated me to do it. In my thesis I develop a series of hypotheses about how eye movements should be controlled by an editor in order to create the perception of continuity. This data should provide direct evidence of these techniques (if they exist) and motivate further, in depth studies. The films I eyetracked are also interesting to film theorists as they cover the most significant styles of films: Blade Runner (Continuity), Citizen Kane (Deep Focus), Koyaaniqsatsi (Non-narrative), Dancer in the dark (Dogme-esque), Eisenstein’s October (Dialectical Montage), Hulk (Digital Composition/Collage), Requiem for a Dream (Quick Cutting/MTV style), and a few more.

I’m really excited about the findings of this study and I’ll keep this blog updated as to what I find. Unfortunately, analysis of eyetracking data for long sequences of feature films is not easy and there exists no tools to assist. I’m currently developing my own methodologies, tools, and analyses to make this possible. Fingers crossed everything goes to plan and I have some interesting results to present asap.

So in conclusion, high speed experimentation is by far the most Rock ‘n’ Roll way to run experiments. Unfortunately, there is nothing Rock ‘n’ Roll about analysis. (Any suggestions on how I can make my analysis “rock” are highly appreciated)

Corrected version of my Ph.D. thesis submitted

I have just resubmitted my Ph.D. thesis in its final hard-bound version. When I had my viva (see Viva! Viva) back in February I was awarded a pass with minor corrections. The corrections amounted to the modification of a few graphs, fixing of typos, rephrasing of a few sentences to make things clear and general reformatting. Not much really. I finally got round to making the changes to my thesis and I formally resubmitted the thesis last Monday (9^th May). I have also registered my thesis on-line with Edinburgh University’s Research Archive. The archive is a wonderful, publically accessible repository of research documents produced at Edinburgh. You can find my thesis here.

I have also updated the version linked from my website. Now everybody can read what I have been up to for the last 4.5 years and be confident that reputable researchers have given it their seal of approval.

Now all I need to do is channel some of the ideas stemming from my thesis into research proposals for my future career. Too many ideas, too little research funding/job offers J

Science Festival Fun

This is just a quick note to say that I am currently running activities at the Edinburgh International Science festival (April 11^th – 15^th, 2006) at the National Museum of Scotland. There are two activities: Editing Memory and Directing Reality.

Editing Memory is a drop in activity in which you get to test your memory for films by reconstructing the film using editing. It is proving to be very VERY popular (my helpers can barely keep up!). Come and have a go (its free) if you want to test you memory or try editing a film. You can visit the website for more information and the eventual results: http://scifest.inf.ed.ac.uk/editing-memory/

Directing Reality is a 1-hour workshop in which kids (and big kids) can come and make Trick Films using really simple filming techniques. The films the kids are making are just like those produced by George Melies over a hundred years ago. The intention of the workshop is to get the kids thinking about the expectations and assumptions we make when perceiving the visual world and how these can be violated to create funny illusions using editing. The kids pick it up really quickly and the tricks they come up with are marvellous. If you want to come and have a go get down to the museum before Saturday although you might want to book on-line before hand (http://www.sciencefestival.co.uk/) as tickets are selling out fast (they cost £3).

Hope to see you there :)

An Attentional Theory of Continuity Editing

So here it is: my thesis. After successfully passing my viva a few weeks ago I have finally finished tweaking and polishing my thesis so that it is now in a format that I can distribute. Feel free to download the thesis and print it off although I will warn you it is 389 pages long. Selective printing might be better for the rainforests.

If you wish to reference my thesis please use the reference below.

Smith, T. J. (2006) An Attentional Theory of Continuity Editing. Unpublished Doctoral Thesis. University of Edinburgh.

I welcome all comments and questions about the thesis as well as peripheral issues. The thesis is a first stab at using cognitive science theories and methodologies to answer questions about how we watch film. The deeper I delved into the issues related to film perception, the more questions emerged. It is my hope that I will have the opportunity to investigate these questions in the future and that other researchers, from both cognitive science and film theory will also take an interest in answering these questions.

Whilst writing this thesis my love of film and adoration of filmmakers has grown inordinately. The insight into the human perceptual system exhibited by talented filmmakers is awe inspiring. The precision with which they appear to shape their audience’s experience of the film highlights how much further cognitive science has to go before we begin to get even close to understanding how perception works. By applying an analytical mind to the conventions and techniques used by filmmakers, it is my belief that cognitive scientists will learn a lot about the human perceptual system.

Film is a deviant reality that we, as members of its audience, have been successfully interacting with for over a century. By understanding how film differs from reality and why this difference has no negative effect on our ability to perceive film we will gain, not only a greater understanding of how we perceive film but also an insight into how we perceive reality.

Viva! Viva

Last Tuesday (21^st February) I had my viva. For those readers who are not academics this is an oral examination during which I am grilled by two experienced academics about my thesis, its content, all issues surrounding my thesis, my research goals, and, if they are really pedantic, my dubious sentence construction. Vivas in my department have been known to go on for 3-4 hours on average! Imagine a job interview in which your interviewers know you inside out and challenge you to defend why you are you. That’s the kind of experience a viva can be.

Luckily my viva turned out to be nowhere near as bad as I was dreading it might be. Due to some fortuitous scheduling I was able to have Professor John Henderson of Michigan State University as my external examiner. Usually your external has to come from the UK or Europe as the university have to pay to fly them over and accommodate them. I was able to have John as my external as he happened to be planning a visit to Edinburgh just at the time that I was due to have my viva. In terms of my research topic and the specific approach I took, John was probably one of the most qualified people in the world to examine me. He and his colleagues at Michigan State have produced a wealth of innovative research investigating visual scene perception and memory across eye movements. John is specifically interested in using ecologically valid stimuli to try and understand how real-world scene perception works. I hoped that this would make him receptive to my application of the theories and methodologies of visual cognition to film.

As it turned out in the viva, he appeared to appreciate my approach and my thesis in general. After only an hour of discussion, most of which at a high level of issues arising from and related to by thesis John and my internal examiner (Professor Keith Stenning) decided to recommend that I be awarded a Ph.D. with only minor corrections (giving me one month to do these). This is essentially the best result you can hope for from a viva. I was stunned, to say the least.

I want to say a huge thank you to John and Keith for agreeing to examine me on my thesis, to read the massive document, provide me with feedback on it, and take time out of their busy schedules (in John’s case, super busy) to examine me. I also want to thank John Lee, my primary supervisor for is help in getting to this point and organising the viva and Helen Pain, my second supervisor for reading my thesis multiple times, grilling me in a mock viva, and being there during my real viva. Thank you everyone.

The best part of my viva success for any interested readers out there is that this means I am able to make my thesis available to you. I will post a pdf copy of my thesis on this blog in the next few days. There are a few minor corrections I have to make first but once they are done I’ll post it here.

Thesis Complete

I did it! I handed in my Ph.D. thesis. I can't actually believe I finished my thesis. As you might deduce from the lack of blog posts since early December, the Christmas period was busy for me. I had so much of my thesis left to write as I approached Christmas and I had no idea how I would do it. However, I seemed to enter the thesis “zone” where all my ideas coexisted in my brain and I was finally able to see all the connection, relationships, and structures that had been lacking from my ideas for so long. This made the final writing possible and meant that I was able to produce sections at an alarming rate.

I returned on the 27^th December to Edinburgh from my parent’s house on the Wirral. Between that day and the 10^th January I had no more than 4 hours sleep a night, spoke to very few people other than my girlfriend on the phone (she was visiting her family in Germany), and saw even less people. I produced at least 30% (~33,000 words) of my thesis in this period. Having now submitted my thesis I feel I can now reflect on the experience and hand out this advice: WRITE YOUR THESIS AS YOU GO ALONG, NOT AT THE LAST MINUTE. Simple, some might say, obvious advice that I completely failed to adhere to when doing my thesis. Luckily my sanity, my fingers (from all the typing), and my Will to live where not completely crushed by the experience. Not completely anyway……

I can now officially record that the Ph.D. thesis entitled ‘An Attentional Theory of Continuity Editing’ was submitted for examination on the 11^th January 2006. My viva (oral examination) should take place around the 20^th February and based on the outcome of that (eek!) I will know if I have succeeded in gaining my doctorate. I have no idea how it will go but I’m doing all I can to be prepared for it. I’ll try to keep this blog updated so you know how things go.

For the time being I am fortunate to report that I am employed as a Research Assistant on a European project developing an intelligent tutoring system (http://www.leactivemath.org/). This project gives me the opportunity to develop my AI interests whilst continuing to perform experiments (I’ll be performing usability, evaluation of a tutoring system). However, I do not intend to end my research interest in film perception with my thesis as there are too many questions left unanswered. I’ll be developing research proposals and making grant applications during my time as an RA with the aim that by 2007 I’ll be able to pursue my research interests. To this end I will be presenting my research at conferences during 2006 (e.g. ETRA, in San Diego in March, www.e-t-r-a.org) and writing papers. Lets hope 2006 proves to be an enjoyable and productive year and sets the foundations of my future research career.

The same wishes are extended to all readers of this blog: may your 2006 bring all you desire.