My practice based PhD research is a multidisciplinary study (Theatre, HCI and Critical Theory) examining the performative application of interactive projective technologies. The research is informed by the methodology of Practise as Research in the Arts (PaR) [1] and the concept of Performative Research [2]. Critical theory drawn from intermediality [3] authenticity and liveness [4], the uncanny and the double [5] are imbricated within the practice to produce praxis. The research is contextualised historically and with reference to other practitioners and documented in the form of an ongoing research blog on the website kinectic.net
iMorphia (see figure 1 below), a prototype system using the Microsoft Kinect, the Unity games engine and projected characters sourced from Daz3D Studio [6] and MakeHuman [7] was used as the basis for a series of studies culminating with a workshop and an ethnographic study involving sixteen participants. The results of the study led to a number of key observations and directions for further research as described below.
The user study confirmed that the aesthetics of the projected character, the ‘bodymask’ effected performative behaviour. It was acknowledged that the characters used in the evaluation present stereotypical images. One of the challenges was in fact sourcing non-idealized characters, when the sources originate from within the world of gaming where stereotypes abound. A strand identified for further research is the creation and testing of a range of characters with more realistic body types enabling research into the effect of varying body types of different ages, gender and ethnicity. In addition a range of clothing might also be created, thereby addressing the reading of the signs and semiotics associated with clothing. However such research would require the creation of a range of virtual characters with differing ages, gender, build and ethnicity, a wardrobe of virtual clothing and a series of user studies in order to evaluate the effects of differing bodymasks on performative behaviour. This avenue is difficult to scope and is felt to be out of remit for two reasons - the creation of multiple characters and clothing would be a time consuming task and require the skills of a 3D designer and an informed evaluation of the project would require knowledge and skills in areas such as psychology and gender studies. Such research however might have applications in mental health and well-being and the cultural studies of stereotypes.
This research strand focusses on Human Computer Interaction and involves three overlapping areas of research - control, feedback and dynamics. The control aspect would investigate the mapping between the time-based 3D data of the performer’s body movements to the projected character - can it be improved in terms of speed and accuracy? Feedback involves investigating the form of information delivered to the performer on the coherence between the performer’s body and the projection – how might the ability to follow or control the digital bodymask be improved? The implementation of bodymask dynamics would involve the addition of interactive characteristics to the bodymask such that it has performative behaviours – it might only be able to move slowly, or may direct the performer into following certain actions or movements.
This avenue of research is certainly within scope though does present technical and implementation challenges requiring low level programming and dealing with raw data rather than working at a higher level using a game engine and scripting. The strong technical focus may also steer the research away from any creative and performative practice. The research avenue may find application in training and exercise, however examples of research in this area exist and is thus not entirely novel [8].
The concept of possession however has an interesting artistic aspect, suggesting themes of the supernatural and the performer as a medium; and in this context marries performance with HCI.
An informal evaluation workshop was carried out with two performers with the premise that two performers transformed at the same time might result in improvisation. The two performers did playfully interact with each other but not in a way that suggested the emergence of improvisation. Discussion afterwards led to suggestions that a third element was required, a game, an object they could work with or perhaps a scenario designed to encourage improvisation. The system currently lacks the facility to enable the performative bodymask to interact with its virtual environment; it is simply a projected character that follows the movements of the performer. However iMorphia is implemented using the Unity Games Engine and supports the scripting of virtual objects such that they can be imbued with performative and responsive behaviours. In addition a virtual set such as a room or a forest can be designed so supplying a context for an interactive scenario designed to encourage improvisation. The Unity platform represents a suitable vehicle for exploring the potential of creating improvisation through the addition of interactive objects and scenarios. An improvisational performance is associated with the quality of liveness, it occurs in the here and now and has a distinct quality of edginess and risk in comparison to the rehearsed and choreographed. The idea of a live interactive stage that supports improvisation resonates with the concept of intermedial performance as described by performer Jo Scott. “Within live intermedial performance, liveness exists within, through and in relation to the media employed and is enacted and engendered by manipulation of such media by the performer/activator in the real time of performance” [9]
The uncanny is a problematic term operating in the domains of phenomenology and subjectivity. According to the OED the uncanny is defined as “strange or mysterious, especially in an unsettling way” and originated from the Scottish – not-canny relating to the occult and the malicious. The etymology of the word resonates with the German Das Unheimliche, "the opposite of what is familiar”, a concept expounded upon by Sigmund Freud in his 1919 essay “The Uncanny” [10]. The Uncanny Valley [11] describes a trough to be avoided in the design of robotics and computer graphics, on one side we perceive things as non-human and on the other they are recognised as clearly human, the trough is the in-between space, a liminal and discomforting area where objects appear uncomfortable and disturbing neither really alive in a human sense or dead as in non-living. In his essay Freud refers to the earlier work of Jentsch [12] and shares a definition of the circumstances that evoke a sense of the uncanny which mirrors the uncanny valley as described above. “doubt as to whether an apparently living being really is animate and, conversely, doubt as to whether a lifeless object may not in fact be animate” [13] Freud also refers to the writings of Otto Rank and his association of the uncanny with the double or doppelganger. This theme has been more recently taken up in the field of performance and technology by writers such as Causey [5] and Dixon [14] who write about the manifestation and cultural significance of the digital double.
The practice led methodology informed by critical theory resulted in the creation of a prototype platform which has generated a number of potential strands for further research. The strands I believe are novel and have potential application beyond the arts, they are however emergent from an arts practice based form of research.
Praxis, the combination of theory and practice advocated by Nelson [1] is recognized as a valuable means of ensuring creativity is informed by critical theory and that practise is seen as a method of validating and driving the research forward.
The proposal for the next stage of the research is to conduct a number of performative exercises which intend to imbricate theories of the uncanny with practice. These are seen as creative exercises, means of validating theory through “user testing” and also a method of generating further research.
The uncanny is regarded as a conceptually rich area, especially in relationship to technological mediation and I end with two provocative questions which will be used as a basis to catalyse future research.
How using interactive projective technologies might the uncanny be made manifest?
What is an uncanny interface?
This author is supported by the Horizon Centre for Doctoral Training at the University of Nottingham (RCUK Grant No. EP/G037574/1) and by the RCUK’s Horizon Digital Economy Research Institute (RCUK Grant No. EP/G065802/1).