Outline

Film critic and theorist André Bazin defined two schools of filmmaking: mise-en-scène and montage [6]. While this binary does not hold up as film and television have progressed, the two concepts provide a useful way to reason about film, television, video games and other digital media with a strong visual component.
Mise-en-scène is about the composition of the scene and the shot, and is concerned with elements including set construction, objects, period costumes, and camera position and angle. Consider the work of Wes Anderson, whose sets are often elaborately constructed with bright colours, strong symmetry, and finely detailed compositions.

Montage on the other hand occurs after filming, using cuts and juxtapositions to convey meaning. For example, we might think of the “match-shot” early in 2001: A Space Odyssey [9] where one of our ancestors throws a bone into the air which then cuts to a white spaceship in the same position in space with a similar appearance, suggesting tool-use and technological progress as links to our ancestors.

In most digital and immersive media, we do not see significant usage of montage and cuts, and in contexts such as VR cuts are generally ill-advised as they can be jarring, nausea-inducing and detracting from the immersive and user-agency focused nature of these experiences. Meanwhile, virtual production and level design to provide environments for the user to explore are vital in digital media and align well with the concept of mise-en-scène [17]. That said, change within these environments is often explicit and user-centric: move the object, shoot the bad guy, open the door.

Attempts have been made in narrative-focused video games to provide dynamic outcomes from minimal interaction [16]. Take the example of Dear Esther [12], where the narration changes with each playthrough. This however is randomised, and not reacting to the player. Another, more reactive example, is Bastion [15], where a narrator responds to the player’s actions. However, these narrations are surface details only, and no meaningful change is effected.

Some argue that interactivity is incompatible with “traditional narrative forms” [13]. They claim that interactivity requires a different form of immersion and engagement on the part of the player. Jenkins suggests instead that interaction and narrative in video games are not incompatible. Rather games (and related forms) provide narrative architectures and are, “[…] spaces ripe with narrative possibility.” [8]

This research involves the creation of a television-like experience within a narrative-focused space, using the viewer’s attention and proximity (where they are looking and where they are located) to scenes and objects within them, to reactively decide the viewer’s access to future areas and scenes, and the composition of objects within them. It will expand on concepts seen in narrative-focused “walkers” [11] and their successors [3–5,12,14].

Following a Research-through-Design [18] informed methodology, on completion of the exemplar experience, the tools used in its creation will be distributed to other creators [7] and used in game jams [1,2,10] to explore the possibilities of the design approach.

References

1.    Alan Chatham, Ben A.M. Schouten, Cagdas Toprak, Florian Mueller, Menno Deen, Regina Bernhaupt, Rohit Khot, and Sebastiaan Pijnappel. 2013. Game jam. In CHI ’13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA ’13, 3175. https://doi.org/10.1145/2468356.2479640

2.    Menno Deen, Robert Cercos, Alan Chatman, Amani Naseem, Regina Bernhaupt, Allan Fowler, Ben Schouten, and Florian Mueller. 2014. Game jam: [4 research]. In Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems - CHI EA ’14, 25–28. https://doi.org/10.1145/2559206.2559225

3.    Fullbright. 2013. Gone Home: A Story Exploration Video Game. Retrieved February 24, 2019 from https://gonehome.game/

4.    Fullbright. 2017. Tacoma - Fullbright. Retrieved February 24, 2019 from https://tacoma.game/

5.    Giant Sparrow. 2017. What Remains of Edith Finch. Retrieved February 24, 2019 from http://www.giantsparrow.com/games/finch/

6.    Henderson, B. 1971. THE LONG TAKE. Film Comment. 7, 2 (1971), 6–11.

7.    Hilary Hutchinson, Heiko Hansen, Nicolas Roussel, Björn Eiderbäck, Wendy Mackay, Bo Westerlund, Benjamin B. Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, and Helen Evans. 2003. Technology probes. In Proceedings of the conference on Human factors in computing systems - CHI ’03, 17. https://doi.org/10.1145/642611.642616

8.    Henry Jenkins. 2004. Game Design as Narrative Architecture. Computer 44, 3: 118–130. Retrieved January 10, 2019 from http://blogs.bgsu.edu/honors1120/files/2013/08/Jenkins_Narrative_Architecture.pdf

9.    Kubrick, S. 1968. 2001: A Space Odyssey. Metro-Goldwyn-Mayer (MGM),  Stanley Kubrick Productions.

10.    Annakaisa Kultima. 2015. Defining Game Jam. In Proceedings of the 10th International Conference on the Foundations of Digital Games (FDG 2015), 1–9. Retrieved October 2, 2019 from https://www.researchgate.net/profile/Kultima_Annakaisa/publication/281748266_Defining_Game_Jam/links/55f729d908ae07629dc114bd.pdf

11.    Alexander Muscat and Jonathan Duckworth. 2018. WORLD4: Designing Ambiguity for First-Person Exploration Games. In The Annual Symposium on Computer-Human Interaction in Play Extended Abstracts - CHI PLAY ’18, 341–351. https://doi.org/10.1145/3242671.3242705

12.    Dan Pinchbeck. 2008. Dear Esther: An Interactive Ghost Story Built Using the Source Engine. In Joint International Conference on Interactive Digital Storytelling, 51–54. https://doi.org/10.1007/978-3-540-89454-4_9

13.    Andrew Polaine. 2005. The Flow Principle in Interactivity. In Proceedings of the second Australasian conference on Interactive entertainment, 151–158. Retrieved April 13, 2019 from http://dl.acm.org/citation.cfm?id=1109180.1109204

14.    Lucas Pope. 2018. Return of the Obra Dinn. Retrieved February 24, 2019 from https://obradinn.com/

15.    Rao, A. 2011. Bastion. Supergiant Games.

16.    Chris Remo. 2019. Interactive Story Without Challenge Mechanics: The Design of “Firewatch.” In Game Developers Conference. Retrieved April 13, 2019 from https://schedule.gdconf.com/session/interactive-story-without-challenge-mechanics-the-design-of-firewatch/861240

17.    Signposting, Mise-en-Scene, and Environmental Storytelling: Understanding signposting as part of the embedded narrative in environmental storytelling: 2017. http://localhost/handle/1874/352414. Accessed: 2020-12-18.

18.    John Zimmerman and Jodi Forlizzi. 2014. Research Through Design in HCI. In Ways of Knowing in HCI. Springer New York, New York, NY, 167–189. https://doi.org/10.1007/978-1-4939-0378-8_8