Investigating LLMs' Processing of Binding Relations in Korean

초록

This study examines the linguistic competence of large language models (LLMs) by evaluating their performance on anaphoric binding in Korean, a language that permits both local and long-distance binding. Using 280 test sentences adapted from prior experiments on native speakers’ preferences for local versus long-distance binding, we tested three Generative Pretrained Transformers—GPT-3.5-turbo, GPT-4, and GPT-4o—on their ability to resolve reflexives and pronouns. Across all models, we observed a strong and consistent preference for long-distance binding, even in contexts where native Korean speakers reliably favor local antecedents. This pattern was especially pronounced with morphologically complex anaphors such as caki-casin and ku-casin, for which the models selected long-distance antecedents in 66.7% of cases for GPT-3.5-turbo, 72.5% for GPT-4, and 50% for GPT-4o. These results suggest that the models do not replicate the nuanced patterns of native speaker judgments, particularly in contexts involving morphological or syntactic complexity. However, despite these discrepancies, the models—especially GPT-4o—did exhibit some sensitivity to semantic constraints. In gender-controlled conditions, GPT-4o selected the appropriate local antecedent for ku-casin in 95% of cases in Experiment 1 and 70% in Experiment 2, and the correct long-distance antecedent for kunye in 90% of both experiments. This indicates that while LLMs do not fully internalize binding principles in a native-like way, they are capable of utilizing lexical-semantic cues such as gender to guide antecedent resolution. The findings highlight both the potential and limitations of current LLMs at the syntax-semantics interface and underscore the importance of evaluating these models on linguistically rich and typologically diverse phenomena to gain a deeper understanding of their representational capabilities and theoretical adequacy. ©2025 Institute for Cognitive Science, Seoul National University.

키워드