Effective quality factors of multimodal interaction in simple and complex tasks of using a smart television

Abstract

Influencing quality factors, related to the user and system, need to be considered when building a well-designed multimodal interaction system. User groups, access to input modes and tasks were defined as the user and system factors to examine effective factors of multimodal interaction with a smart TV, and its input modes consisted of voice, arrow key, and motion-based pointer modes and their combinations. User group 1 had experience of multimodal interaction with another device, while user group 2 had only experience of unimodal interaction. In addition, the sequential/simultaneous input modes and simple/complex tasks were considered as the system factors. Depending on the task complexity, two experiments were conducted. Nine input modes (three unimodes and six multimodes), sequentially and simultaneously given to both user groups, were investigated for the simple task of menu traversal and the complex task of manipulating broadcasting content, menu traversal, and web content navigation. A subjective rating of the level of preference was recorded in the sequential input mode using a modified Likert-type rating scale, while each participant's preferred mode was observed in the simultaneous scenario. Additionally, the completion time and error rate were measured in both experiments. When performing the simple task, user group 1 used multimodes more so than group 2. However, in the complex task, both user groups preferred multimodes when modes were simultaneously presented. Considering effective quality factors, input modes of a smart TV should be simultaneously provided with a voice and motion-based pointer multimode.