언어모델은 한국어 시간 직시 표현을 얼마만큼 이해하는가?

초록

This study evaluates the ability of large language models (LLMs) to interpret Korean temporal deictic expressions in conversational contexts by constructing a dedicated benchmark dataset. Deixis, whose interpretation varies according to the spatiotemporal context of interlocutors, manifests across diverse grammatical categories including adverbs, nouns, and demonstratives. Existing work on deictic comprehension in LLMs has been largely confined to English and addressed only as a subsidiary component of pragmatic inference benchmarks. Drawing on the spoken corpus provided by the National Institute of Korean Language (NIKL), this study selected dialogues containing temporal deictic expressions, specifically those involving reported speech constructions and complex temporal adverbials, and constructed four question types per dialogue: multiple choice, short answer, true/false, and truth choice selection. Three multilingual LLMs (GPT-5.5, Claude-sonnet-4.6, Gemini-3.1-pro) achieved a mean accuracy of 79%, with notably lower performance on reported speech items exhibiting deictic center mismatches between utterance and syntactic structure. This study presents a Korean-specific benchmark for quantitatively measuring pragmatic inference capabilities in LLMs.

키워드