Integrating breakdown detection into dialogue systems to improve knowledge management: encoding temporal utterances with memory attention

Abstract

With the increasing pervasiveness of smart phones and smart devices, dialogue systems are gaining ever growing attention from both academic and industry. These systems can be broadly classified into two categories, one that is aimed at helping user to gain new knowledge and one that can chat with users without completing any specific tasks. Although dialogue systems are improving substantially, the user experience of such systems are still unsatisfactory as there are no specific rules covering all possible situations of real human-machine dialogue, resulting in breakdowns. There are two technical issues affecting the detection of dialogue breakdown in an open domain conversation: human resources to prepare and annotate a large chunk of conversation data and dialogue histories containing words that don't appear directly in training data. To tackle these issues, we propose a novel encoding method for temporal utterances with memory attention based on end-to-end dialogue breakdown detection. Specifically, long short-term memory (LSTM) is employed to encode each word of all previous user and system utterances. Encoded vectors from LSTM (user and system utterances), along with system and user utterances from sentence embedding, are then stored in memory wherein an attention mechanism is applied to select the most relevant piece of words from system and user utterances for breakdown detection. An evaluation of the proposed approach on a breakdown detection task (DBDC3) showed that the model for single-labeled breakdown detection outperforms other state-of-the-art methods in a classification task. In conclusion, a more effective knowledge gain and management can be achieved by integration of our proposed breakdown detection into dialogue systems.