Zhuang Chang

Zhuang Chang

Cross-Reality Agent; Biosensing; HCI

Postures and Locomotion in Mixed Reality Agents: Effects on Social Perception of Virtual Opponents and Assistants

Published in International Journal of Human–Computer Interaction, 2025

Assistant_Oppoent
Figure 1: Experimental conditions and hardware setup across both studies: (a) Speech (S), (b) Speech+Locomotion (S+L), and (c) Speech+Locomotion+Posture (S+L+P) from Study 1, where the virtual agent acted as an opponent in a turn-based Gobang game (adapted from Chang et al., 2024). (d) S, (e) S+L, and (f) S+L+P from Study 2, where the virtual agent served as an assistant, providing strategic hints on demand. (g): Hardware setup used in both studies, including the HoloLens 2 for MR interaction and an EEG headset for measuring cognitive engagement (originally presented as (d) in Chang et al., 2024, reordered here for clarity.

Integrating non-verbal cues into Mixed Reality Agents (MiRAs) enhances their ability to engage users and foster socially rich interactions. This paper investigates the role of locomotion and body posture in shaping user engagement, social presence, and interaction quality through two user studies involving a turn-based Gobang game. From these studies we found that in a competitive context, MiRAs’ locomotion and posture enhanced social presence and engagement, but while in a cooperative context, these behaviors fostered rapport but not trust. By integrating subjective, behavioral, and physiological measures, including EEG, this study provides a holistic understanding of MiRAs’ impact. The findings offer actionable design implications for creating engaging and socially effective virtual agents, advancing the field of human-agent interaction in Mixed Reality. Future research directions include exploring long-term effects, using diverse application domains, and supporting multimodal interactions.

Recommended citation: Chang, Zhuang, Kunal Gupta, Jiashuo Cao, Huidong Bai, and Mark Billinghurst. Exploring the Effects of Mixed Reality Agents’ Locomotion and Postures on Social Perception Through a Board Game. International Journal of Human–Computer Interaction (2025): 1-30. DOI: 10.1080/10447318.2025.2583473
Download Paper