A Comparative Analysis of GPT-Based Distractor Generation for Passage-Based Questions

초록

High-quality distractors are essential in multiple-choice questions for diagnosing student misconceptions, yet their manual creation remains labor-intensive. This study presents a comparative investigation of three GPT-based automatic distractor generation strategies applied to passage-based questions from a Korean language question dataset on AI-Hub: supervised fine-tuning, in-context learning, and chain-of-thought prompting. We evaluate each approach against a zero-shot GPT-4.1-mini baseline across six automatic metrics measuring semantic and lexical alignment with ground-truth distractors, as well as a functional plausibility assessment using GPT-5.4 as an automated judge. Supervised fine-tuning consistently achieves the strongest alignment across all metrics and approaches and produces the highest proportion of well-functioning distractor sets as judged by GPT-5.4. In-context learning yields higher alignment than the zero-shot baseline across all alignment metrics but generates the highest rate of distractor sets that did not allow the judge model to identify the ground-truth answer among the correct answers. Chain-of-thought prompting scores below the zero-shot baseline on all alignment metrics and produces the lowest rate of well-functioning distractor sets. Our findings illustrate how different strategies serve distinct roles under varying resource conditions, with implications for AI-assisted assessment design in Korean language education and broader EFL contexts.

키워드