딥러닝 언어모형의 평가와 언어학

Abstract

This article addresses how the deep learning-based language models can be evaluated with respect to linguistic knowledge. Building upon the overlook, this article discusses how linguistics can make a substantial contribution to the development of the artificial intelligence systems. As many transformer-based models have been competitively implemented for the last few years, it is required to evaluate the multiple models in a common and reliable way. For this purpose, a wide range of linguistic evaluation metrics have been designed and constructed. The evaluation datasets involve the concepts used in theoretical linguistics, such as syntax, semantics, and pragmatics. The evaluation process follows the guideline used in psycholinguistic experiments. As such, the linguistic knowledge enhances interpretability of the deep leaning-based natural language processing techniques. It is contended that linguistics will play a pivotal role in evaluating and improving the language models in further research.