Do Multilingual Language Models Transfer Knowledge of Pro-Drop Across Languages?

초록

This study investigates whether two representative multilingual masked language models encode and cross-lingually transfer information relevant to pro-drop prediction across four languages with varying constraints on subject realization: English, Spanish, Korean, and Chinese. We evaluate mBERT and XLM-R on a binary subject-drop prediction task derived from an annotated parallel corpus of The Little Prince. Results show that Spanish subject-drop is better predicted than Korean and Chinese subject-drop, consistent with the greater availability of sentence-internal morphological cues; that same-language fine-tuning generally improves performance; and that cross-lingual fine-tuning generally degrades target-language performance. The relatively small negative transfer between Korean and Chinese is consistent with their similarity in discourse-oriented subject-drop licensing, although class distribution and effectiveness of training data may also contribute.

키워드