UnitCorrect: Unit-Based Mispronunciation Correcting System With a DTW-Based Detection

초록

With recent globalization trends, the importance of language education for nonnative speakers has significantly increased. Research on pronunciation correction and speech editing continues to be actively explored to reduce the pronunciation gap between native and nonnative speakers. Traditional speech editing models frequently suffer from unnaturalness at the boundary of the corrected parts and demand substantial input information to effectively synthesize corrected speech. Furthermore, the lack of well-curated annotations for mispronounced speech presents a challenge for effective training. This paper introduces UnitCorrect, a mispronunciation correction system that leverages self-supervised unit representations for synthesizing natural-sounding speech and utilizing more phoneme information effectively. Additionally, we propose a mispronunciation detection approach based on dynamic time warping, which efficiently identifies mispronounced segments by aligning input audio with the target text, helping to mitigate the data scarcity issue. In addition, frame-level text is incorporated into the decoder's input to supplement the text information, and a conditional flow-matching decoder enables high-quality speech synthesis. The experimental results demonstrate that UnitCorrect outperforms existing models in terms of mispronunciation corrections and naturalness of speech.

키워드