Abstract

The research within this PhD continues the exploration of brain computer interfaces to service cinematic experiences, which has been the focus of the artistic practice for Richard Ramchurn for several years. The PhD is practice led [1], exploring the production, interaction and reception of a brain controlled narrative film. Theory is informed by the disciplines of Human Computer Interaction (HCI), Brain Computer Interfaces (BCI) and Media Studies. The literature within these fields act as a guide to creating the design, interaction and making of a brain controlled film, it also acts as the grounding for the studies into the films’ making and audience interaction.

Introduction

The field of BCI involves neuroscience, engineering and signal processing, to extract data from either electricity activity or blood flow within a person’s brain. While it has many applications in helping people with medical conditions, it also has novel features which afford interesting interactions for everyone. Before the conception of interacting with a computer via brain waves [6] artists were experimenting with Electroencephalograms to create music [5], artists are now using BCI in various artistic practices [2]. Filmmakers have a long history making interactive films. There is a current challenge as to how to interact with a film without breaking the narrative immersion. The two concepts, of immersion or flow and interaction within film have been postulated to be mutually exclusive [3,4]. Combining BCI with Interactive cinema presents potential to mitigate some disruptive elements that interaction may have on narrative immersion.

**Practice **

Artistic practice takes many roads, in the first year of research Ramchurn travelled to New York to create an interactive installation at BRIC, Brooklyn. He was invited to explore the substantial archive of late poet, activist and member of the Living Theatre steve ben isreal (he was staunchly anti-capitalism) and to make a piece of work that would allow visitors to interact and explore steve ben’s life and work.

The practice leading the research is the making and reception of the films The Disadvantages of Time Travel and The MOMENT both directed by Ramchurn.

The MOMENT

The main practice that is part of the PhD is the practical process of making of a brain controlled film called The MOMENT. The MOMENT is a Sci Fi thriller set in the near future made as a response to the unintended consequences of technology which continues to subvert culture and society. Projecting ambitions for brain computer interfaces from big tech Ramchurn explores a world where thoughts become public property. The intentions behind the narrative are inspired by current events to bring attention how peoples data are used and abused by corporations and political interests now and how that could continue in the future. Using BCI technology Ramchurn invokes what we give away unintentionally every day, how our data effect content and advertising we see and to ask viewers to think critically about their everyday interactions with technology.

Creating NeuroCinema

Making a film is an involved process, as a director it is his or her job to tell a story and to guide the viewer within that story from beginning to end. That is the directors prerogative. But when the story is interactive this presents challenges that are unique. How does one tell a story when some of that authorial control is no longer in the hand of the director? Can filmic techniques be adapted in order to service that ideal? The film making process does not only belong to the director; actors, cinematographer, designers all deliver their crafts collaboratively to create a final film, how does the interactive aspect of the film impact on their practice? The entire process is being studied as to how it differs from traditional linier narrative content; from conception, to script and across departments such as music and sound design, on set production to final editing.

The MOMENT interactive system

Then there is creating the interactive system that the film will be exhibited on. This system has been designed and documented during the research. The specifications of this software will reflect how and where the film will be exhibited. The system will be able to record the data created within the interaction with the film. That data will consist of information about the individual film that is created, which scenes were watched, how fast it cut from one narrative thread to another, and the data from the EEG headset.

Ramchurn’s practice would not have been possible without further external funding and support. The development of this project was supported by successful applications to The Arts Council England, EPRSC’s Telling Tales of Engagement Award and The University of Nottinghams Horizon CDT Impact Fund.

Impact

Ramchurn won the Telling Tales of Engagement prize in 2017 which facilitated the creation of a mobile cinema built from a converted caravan. The MOMENT was screened in the mobile cinema, in auditoriums, and as a live score version. Throughout 2017 and 18 Ramchurn and his team presented The MOMENT at over 15 national and international festivals, in the UK, Switzerland, Spain, Italy, Iceland, China, and Canada.

Research Questions:

What are the considerations on specific filmmaking processes present when making a brain controlled film?

What is the value in having a film that is interactive and interacted with via live brain data both for individuals and for groups?

How does the inclusion of BCI within an interactive film impact on the narrative comprehension, immersion and repeat viewings?

Studies

A number of studies will be included in the thesis. These are qualitative studies complimented by quantitative data from interaction experiences. The studies are done in the wild, that is with the public interacting with a film. In these studies, we explore how the inclusion of a BCI with in a narrative film experience affects the individuals or groups, what it means to the reception of a narrative, the strategies of interaction participants take and their perceived successes and failures.

The first in the wild studies undertaken as part of this research was about adapting and presenting a brain controlled film to be interacted with multiple interactors. This was conducted in 2016 where the film The Disadvantages of Time Travel toured to venues throughout the UK as a three-person experience. We conducted interviews and collected brain scanner data to explore how people interact with each other and with the film. The Second data collection took place when The MOMENT was on its UK tour, Ramchurn et al. has explored this new interaction, how the narrative interaction manifests both from interviews and analysis of the created films. Further to that they will explore repeat experiences of the film and how understanding and value judgements change with repeat performances.

1. Steve Benford, Chris Greenhalgh, Andy Crabtree, Martin Flintham, Brendan Walker, Joe Marshall, Boriana Koleva, Stefan Rennick Egglestone, Gabriella Giannachi, Matt Adams, Nick Tandavanitj, and Ju Row Farr. 2013. Performance-Led Research in the Wild. ACM Transactions on Computer-Human Interaction 20, 3: 14:1-14:22. https://doi.org/10.1145/2491500.2491502
2. Hayrettin Gürkök and Anton Nijholt. 2013. Affective brain-computer interfaces for arts. In Proceedings - 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, ACII 2013, 827–831. https://doi.org/10.1109/ACII.2013.155
3. Lev Manovich. 2001. The Language of New Media. Screen 27, 1: 354. https://doi.org/10.1386/nl.5.1.25/1
4. Andrew Polaine. 2010. The Flow Principle in Interactivity. 151–158.
5. Volker Straebel and Wilm Thoben. 2014. Alvin Lucier’s Music for Solo Performer: Experimental music beyond sonification. Organised Sound 19, 01: 17–29. https://doi.org/10.1017/S135577181300037X
6. J Vidal. 1973. TOWARD DIRECT BRAIN-COMPUTER COMMUNICATION.