Empathetic video clip experience through timely multimodal interaction

Abstract

In this article, we describe a video clip playing system, named "Empatheater," that is controlled by multimodal interaction. As the video clip is played, the user can interact and emulate predefined video "events" through guidance and multimodal natural interaction (e.g. following the main character's motion, gestures or voice). Without the timely interaction, the video stops. The system shows guidance information as how to properly react and continue the video playing. The purpose of Empatheater is to provide indirect experience of the given video content by eliciting the user to mimic and empathize with the main character. The user is given the illusion (or suspended disbelief) of playing an active role in the unraveling video content. We first discuss the overall system architecture, and important components and features of Empatheater. In addition, we report on the results of the experiments that were carried out to evaluate perceived empathy, presence/immersion and subjective user experience with it as compared to conventional passive video viewing for different content genres and forms of interaction.