Horizon CDT Research Highlights

Research Highlights

Brain Controlled Film

  Richard Ramchurn (2015 cohort)   www.albinomosquito.com

Abstract

This research continues the exploration of the application of brain computer interfaces to filmic experiences which has been the main focus of artistic practice for Richard Ramchurn for several years. The PhD is practice led [1], exploring the production, interaction and reception of a brain controlled narrative film. Theory is informed by the disciplines of Human Computer Interaction (HCI), Brain Computer Interfaces (BCI) and Media Studies. The literature within these fields act as a guide to creating the design, interaction and making of a brain controlled film, it also acts as the grounding for the studies into the films’ making and audience interaction.

Introduction

The field of BCI involves neuroscience, engineering and signal processing, to extract data from either electricity activity or blood flow within a person’s brain. While it has many applications in helping disabled people, it has novel features which afford interesting interactions for everyone. Artists are known to be early adopters of new technologies [2], before the conception of interacting with a computer via brain waves [5] artists were experimenting with Electroencephalograms to create music [4]. Filmmakers and artists working with film also have a long history making interactive films, before the time of the internet or even computer games. There is a challenge as to how to interact with a film without breaking the narrative immersion. The two concepts, of immersion and interaction within film have been postulated to be at opposite ends of the same spectrum and thus mutually exclusive [3]. Combining BCI and Interactive film presents potential to mitigate some disruptive elements that interaction may have on narrative immersion.

Practice

Artistic practice takes many roads, in the first year of research Ramchurn travelled to New York to create an interactive installation at BRIC, Brooklyn. He was invited to explore the substantial archive of late poet and activist and member of the Living Theatre steve ben isreal (his name is never capitalised as he was staunchly anti capitalism) and to make a piece of work that would allow visitors to interact and explore steve ben’s live and work. The installation called The Den was a replica of his study where he would entertain guests and write material, visitors would spend time in the space wearing a BCI headset, glimpses of his performances and interviews would be projected onto the walls while a soundtrack made up of his music and specially recorded jazz for some of his favourite musicians that was also remixed via the visitors’ brain data would play.

The main practice that is part of the PhD is the practical process of making of a brain controlled Film called The MOMENT. The moment is a Sci Fi thriller set in the near future made as a response to the unintended consequences of technology which continues to subvert culture and society. Projecting ambitions for brain computer interfaces from big tech Ramchurn explores a world where thoughts become public property. The intentions behind the narrative are inspired by current events to bring attention how peoples data are used and abused by co-operations and political interests now and how they could be in the future. Using BCI technology we invoke what we give away unintentionally everyday, how effects content and advertising they see and ask viewers to think critically about their interaction with technology.

Making a film is an involved process, as a director it is his or her job to tell a story and to guide the viewer within that story from beginning to end. That is the directors prerogative. But when the the story is interactive this presents challenges that are unique. How does one tell a story when some of that authorial control is no longer in the hand of the director? Can filmic techniques can be adapted in order to service that ideal. The film making process is not only belonging to the director, actors, cinematographer, designers all deliver their crafts collaboratively to create a final film, how does the interactive aspect of the film impact on their practice? The entire process is being studied in how it differs from traditional linier narrative content; from conception, to script and across departments such as music and sound design, on set production to final editing.

Then there is creating the interactive system that the film will be exhibited on. This system will be designed and documented during the research. The specifications of this software will reflect how and where the film will be exhibited. The system will be able to record the data created within the interaction with the film. That data will consist of information about the individual film that is created, which scenes were watched, how fast it cut from one narrative thread to another, and the data from the EEG headset.

For maximum impact we want as many people as possible to experience the film and in the best possible conditions. To screen the interactive film, we wanted to have a controlled environment, be mobile so we can take it to places all over the UK and beyond, have control over how it is screened as this can negatively effect peoples experience of interaction. All these specifications let to refurbishing a caravan into an mobile cinema. We hired a company with experience making art installations and converting caravans into art spaces to do this. Part of the project is to create a sustainable model that continues to give public impact and so the mobile cinema will be available to tour throughout 2018 and beyond, to festivals, venues and events.

As a piece of PhD research the work is not typical, the practice being studied would not have been possible without further external funding and support. The development of this project was supported by successful applications to The Arts Council England, EPRSC’s Telling Tales of Engagement Award and The University of Nottinghams Horizon CDT Impact Fund.

Research Questions:

What are the considerations on specific filmmaking processes present when making a brain controlled film?

What is the value in having a film that is interactive and interacted with via live brain data both for individuals and for groups?

How does the inclusion of BCI within an interactive film impact on the narrative comprehension, immersion and repeat viewings?

Studies

A number of studies will be included in the thesis. These are qualitative studies complimented by quantitative data from interaction experiences. The studies are done in the wild, that is with the public interacting with a film. In these studies, we explore how the inclusion of a BCI with in a narrative film experience affects the individuals or groups, what it means to the the reception of a narrative, the strategies of interaction participants take and their successes and failures.

The first in the wild studies undertaken as part of this research was about adapting and presenting a brain controlled film to be interacted with multiple interactors. This was conducted in 2016 where the film The Disadvantages of Time Travel toured to venues throughout the UK as a three-person experience. We conducted interviews and collected brain scanner data to explore how people interact with each other and with the film. The Second data collection will take place when The MOMENT is on its UK tour, we will be probing this new interaction how the narrative interaction manifests both from interviews and analysis of the created films. Further to that we will explore repeat experiences of the film and how understanding and value judgements change with repeat performances.

Prizes and publications

Our paper ## Scanners: Integrating Physiology into Cinematic Experiences was published at CHI2016 and was the winner of the Best Art Paper Award. My work has also won the EPRSC’s Telling Tales of Engagement Award to take my interactive film work on tour across the country.

Project Website

My practice is being supported by industry partners B3 Media and the Foundation for Art and Creative Technology.

FACT

B3Media

References

  1. Steve Benford, Chris Greenhalgh, Andy Crabtree, Martin Flintham, Brendan Walker, Joe Marshall, Boriana Koleva, Stefan Rennick Egglestone, Gabriella Giannachi, Matt Adams, Nick Tandavanitj, and Ju Row Farr. 2013. Performance-Led Research in the Wild. ACM Transactions on Computer-Human Interaction 20, 3: 14:1-14:22.
  2. Hayrettin Gürkök and Anton Nijholt. 2013. Affective brain-computer interfaces for arts. In Proceedings - 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, ACII 2013, 827–831.
  3. Lev Manovich. 2001. The Language of New Media. Screen 27, 1: 354. https://doi.org/10.1386/nl.5.1.25/1
  4. Volker Straebel and Wilm Thoben. 2014. Alvin Lucier’s Music for Solo Performer: Experimental music beyond sonification. Organised Sound 19, 1: 17–29. https://doi.org/10.1017/S135577181300037X 5.J Vidal. 1973. TOWARD DIRECT BRAIN-COMPUTER COMMUNICATION.

Publications

This author is supported by the Horizon Centre for Doctoral Training at the University of Nottingham (RCUK Grant No. EP/L015463/1) and FACT and B3 Media.