Current research in Applied Linguistics focuses mainly on human to human interactions and provides approaches and methodologies to describe patterns in face to face communication. Research into human-agent interaction, on the other hand, is in its infancy and although the main shared feature of interpersonal and human-agent interaction is language, the existing methodologies in Applied Linguistics are of limited use when it comes to analysing these new contexts. While research exists on the functionality, language and design of agent systems, limited research of agent-learner interaction has been undertaken. A first study undertaken as part of this project contributed to a better understanding of the effect of agent voice on teen user perception, while the second study, the main data collection of this thesis, aims to describe and understand structures of multimodal agent feedback and develop a framework for users interacting with feedback loops in mobile language learning. The data collected will be analysed and discussed in terms of the contribution to the fields of HCI and Applied Linguistics as fields of enquiry that are increasingly called upon to help us understand communication in these emerging contexts.
The work undertaken as part of this thesis aims at the development of a framework for the analysis of multimodal feedback loops in agent based language learning environments. and in mobile assisted language teaching and learning materials. The objective of this research is to contribute to the body of knowledge in Human Computer Interaction (HCI) and Mobile Assisted Language Learning (MALL) by offering a better understanding of the structure and role of multimodal feedback loops in MALL interactions between system and user. The results of the data analyses regarding user experience may inform future MALL app development and policy making by means of a UX best-practice guide for feedback loops in MALL. Investigating the structure and types of multimodal feedback loops in agent based language learning environments and how these relate to the user experience of language learning in different interactional contexts is a niche field of interdisciplinary research. Specifically, the research project brings to bear linguistic approaches and challenges on the broader question around users’ perception and attitudes towards interacting with different feedback systems in agent based language learning environments.
Research questions investigated as part of this PhD include:
• What multimodal feedback loops do users encounter in agent based language learning environments and how can these feedback loops be described for the purpose of linguistic analysis?
• How do users respond to positive and negative multimodal feedback loops in their interaction with mobile assisted language learning environments?
Multimodal feedback has been investigated in the context of agent based environments for over two decades and researchers in the field emphasise the importance and potential of multimodality in the efforts of developing more human-like embodied agents.
For further questions on this project or any other comments, please do not hesitate to contact me.
This author is supported by the Horizon Centre for Doctoral Training at the University of Nottingham (RCUK Grant No. EP/G037574/1) and by the RCUK’s Horizon Digital Economy Research Institute (RCUK Grant No. EP/G065802/1).