Fusion-ConvBERT: Parallel Convolution and BERT Fusion for Speech Emotion Recognition
- Authors
- Lee, Sanghyun; Han, David K.; Ko, Hanseok
- Issue Date
- 11월-2020
- Publisher
- MDPI
- Keywords
- speech emotion recognition; bidirectional encoder representations from transformers (BERT); convolutional neural networks (CNNs); transformer; representation; spatiotemporal representation; fusion model
- Citation
- SENSORS, v.20, no.22
- Indexed
- SCIE
SCOPUS
- Journal Title
- SENSORS
- Volume
- 20
- Number
- 22
- URI
- https://scholar.korea.ac.kr/handle/2021.sw.korea/52051
- DOI
- 10.3390/s20226688
- ISSN
- 1424-8220
- Abstract
- Speech emotion recognition predicts the emotional state of a speaker based on the person's speech. It brings an additional element for creating more natural human-computer interactions. Earlier studies on emotional recognition have been primarily based on handcrafted features and manual labels. With the advent of deep learning, there have been some efforts in applying the deep-network-based approach to the problem of emotion recognition. As deep learning automatically extracts salient features correlated to speaker emotion, it brings certain advantages over the handcrafted-feature-based methods. There are, however, some challenges in applying them to the emotion recognition problem, because data required for properly training deep networks are often lacking. Therefore, there is a need for a new deep-learning-based approach which can exploit available information from given speech signals to the maximum extent possible. Our proposed method, called "Fusion-ConvBERT", is a parallel fusion model consisting of bidirectional encoder representations from transformers and convolutional neural networks. Extensive experiments were conducted on the proposed model using the EMO-DB and Interactive Emotional Dyadic Motion Capture Database emotion corpus, and it was shown that the proposed method outperformed state-of-the-art techniques in most of the test configurations.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Engineering > School of Electrical Engineering > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.