The research within this PhD continues the exploration of brain computer interfaces to service cinematic experiences, which has been the focus of the artistic practice for Richard Ramchurn for several years. The PhD is practice led , exploring the production, interaction and reception of a brain controlled narrative film. Theory is informed by the disciplines of Human Computer Interaction (HCI), Brain Computer Interfaces (BCI) and Media Studies. The literature within these fields act as a guide to creating the design, interaction and making of a brain controlled film, it also acts as the grounding for the studies into the films’ making and audience interaction.
The field of BCI involves neuroscience, engineering and signal processing, to extract data from either electricity activity or blood flow within a person’s brain. While it has many applications in helping people with medical conditions, it also has novel features which afford interesting interactions for everyone. Before the conception of interacting with a computer via brain waves  artists were experimenting with Electroencephalograms to create music , artists are now using BCI in various artistic practices . Filmmakers have a long history making interactive films. There is a current challenge as to how to interact with a film without breaking the narrative immersion. The two concepts, of immersion or flow and interaction within film have been postulated to be mutually exclusive [3,4]. Combining BCI with Interactive cinema presents potential to mitigate some disruptive elements that interaction may have on narrative immersion.
Artistic practice takes many roads, in the first year of research Ramchurn travelled to New York to create an interactive installation at BRIC, Brooklyn. He was invited to explore the substantial archive of late poet, activist and member of the Living Theatre steve ben isreal (he was staunchly anti-capitalism) and to make a piece of work that would allow visitors to interact and explore steve ben’s life and work.
The practice leading the research is the making and reception of the films The Disadvantages of Time Travel and The MOMENT both directed by Ramchurn.
The main practice that is part of the PhD is the practical process of making of a brain controlled film called The MOMENT. The MOMENT is a Sci Fi thriller set in the near future made as a response to the unintended consequences of technology which continues to subvert culture and society. Projecting ambitions for brain computer interfaces from big tech Ramchurn explores a world where thoughts become public property. The intentions behind the narrative are inspired by current events to bring attention how peoples data are used and abused by corporations and political interests now and how that could continue in the future. Using BCI technology Ramchurn invokes what we give away unintentionally every day, how our data effect content and advertising we see and to ask viewers to think critically about their everyday interactions with technology.
Making a film is an involved process, as a director it is his or her job to tell a story and to guide the viewer within that story from beginning to end. That is the directors prerogative. But when the story is interactive this presents challenges that are unique. How does one tell a story when some of that authorial control is no longer in the hand of the director? Can filmic techniques be adapted in order to service that ideal? The film making process does not only belong to the director; actors, cinematographer, designers all deliver their crafts collaboratively to create a final film, how does the interactive aspect of the film impact on their practice? The entire process is being studied as to how it differs from traditional linier narrative content; from conception, to script and across departments such as music and sound design, on set production to final editing.
The MOMENT interactive system
Then there is creating the interactive system that the film will be exhibited on. This system has been designed and documented during the research. The specifications of this software will reflect how and where the film will be exhibited. The system will be able to record the data created within the interaction with the film. That data will consist of information about the individual film that is created, which scenes were watched, how fast it cut from one narrative thread to another, and the data from the EEG headset.
Ramchurn’s practice would not have been possible without further external funding and support. The development of this project was supported by successful applications to The Arts Council England, EPRSC’s Telling Tales of Engagement Award and The University of Nottinghams Horizon CDT Impact Fund.
Ramchurn won the Telling Tales of Engagement prize in 2017 which facilitated the creation of a mobile cinema built from a converted caravan. The MOMENT was screened in the mobile cinema, in auditoriums, and as a live score version. Throughout 2017 and 18 Ramchurn and his team presented The MOMENT at over 15 national and international festivals, in the UK, Switzerland, Spain, Italy, Iceland, China, and Canada.
What are the considerations on specific filmmaking processes present when making a brain controlled film?
What is the value in having a film that is interactive and interacted with via live brain data both for individuals and for groups?
How does the inclusion of BCI within an interactive film impact on the narrative comprehension, immersion and repeat viewings?
A number of studies will be included in the thesis. These are qualitative studies complimented by quantitative data from interaction experiences. The studies are done in the wild, that is with the public interacting with a film. In these studies, we explore how the inclusion of a BCI with in a narrative film experience affects the individuals or groups, what it means to the reception of a narrative, the strategies of interaction participants take and their perceived successes and failures.
The first in the wild studies undertaken as part of this research was about adapting and presenting a brain controlled film to be interacted with multiple interactors. This was conducted in 2016 where the film The Disadvantages of Time Travel toured to venues throughout the UK as a three-person experience. We conducted interviews and collected brain scanner data to explore how people interact with each other and with the film. The Second data collection took place when The MOMENT was on its UK tour, Ramchurn et al. has explored this new interaction, how the narrative interaction manifests both from interviews and analysis of the created films. Further to that they will explore repeat experiences of the film and how understanding and value judgements change with repeat performances.
Ramchurn, Richard; Wilson, Max L; Martindale, Sarah; Benford, Steve; Brain Controlled Cinema; Chapter in Artistic Brain Computer Interfaces; Edited by Anton Nijholt; Springer; 2019
Ramchurn, Richard; Wilson, Max L; Martindale, Sarah; Benford, Steve; From Director’s Cut to User’s Cut: to Watch a Brain-Controlled Film is to Edit it; In ACM CHI'19. May 7-12, Glasgow, Scotland, UK
Ramchurn, Richard; Wilson, Max L; Martindale, Sarah; Benford, Steve; # Scanners 2-The MOMENT: A New Brain-Controlled Movie Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems D210 2018 ACM
Ramchurn, R; Maior, HA; Wilson, ML; Martindale, S; Benford, S; fNIRS and Neurocinematics; fNIRS, Tokoyo, Japan; 2018
Ramchurn, Richard; Chamberlian, Alan; Benford, Steve; Designing musical soundtracks for Brain Controlled Interface (BCI) systems Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion 28 2018 ACM
Ramchurn, Richard; Brain controlled cinema Proceedings of the 30th International BCS Human Computer Interaction Conference: Fusion! 6 2016 BCS Learning & Development Ltd.
Pike, Matthew; Wilson, Max L; Benford, Steve; Ramchurn, Richard; # Scanners: A BCI Enhanced Cinematic Experience Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems 293-296 2016 ACM
Pike, M., Ramchurn, R., Benford, S. and Wilson, M. L. (2016) #Scanners: Exploring the Control of Adaptive Films using Brain-Computer Interaction. In ACM CHI'16. May 7-12, San Jose, CA, USA (Winner: Best Art Paper)
Pike, M., Ramchurn, R. and Wilson, M. L. 2015. “## Scanners?: Integrating Physiology into Cinematic Experiences,” C&C '15 Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition 151–52.
Pike, Matthew; Ramchurn, Richard; Wilson, Max L; Two-way affect loops in multimedia experiences Proceedings of the 2015 British HCI Conference 117-118 2015 ACM
This author is supported by the Horizon Centre for Doctoral Training at the University of Nottingham (RCUK Grant No. EP/L015463/1) and FACT and B3 Media.