Evaluating over-empathizing in emotional support conversations: A user-centered framework

초록

Emotional Support Conversation (ESC) aims to improve a help-seeker's emotional state through supportive multi-turn dialogue. While large language models (LLMs) generate fluent and empathetic responses, they often exhibit over-empathy-such as repetitive expressions and excessive use of support strategies. Our preliminary analysis demonstrates that these behaviors reduce user satisfaction and perceived support, highlighting a mismatch between model output and user experience. However, existing evaluation metrics focus on single-turn or reference-based comparisons, failing to capture this problem. To address this gap, we propose UPEval, a user-centered evaluation framework that measures over-empathy in multi-turn ESC by incorporating user satisfaction as a central criterion. Experiments across closed-, open-, and ESC-oriented LLMs demonstrate that UPEval effectively detects patterned responses and strategy overuse, aligning closely with human judgments. We show that existing LLMs tend to produce patterned responses and repeatedly apply similar support strategies, regardless of dialogue context or progression. Our findings suggest that modeling emotional support from a user-centered perspective-by adapting strategies based on user feedback and dialogue flow-is essential for building systems that provide authentic and effective support. This work focuses on over-empathy-a phenomenon that has received limited attention in prior ESC research-and introduces a user-centered evaluation framework that captures user-perceived support quality beyond existing model-centric metrics.

키워드