대규모 언어모델에 기반한 생성형 인공지능의 규제 필요성 - AI 리터러시를 위한 적극적 설명의무 제안 -

초록

The phenomenon of AI sycophancy—where generative AI based on large language models unconditionally seeks approval from users by agreeing with them, even at the expense of the accuracy, truthfulness, and safety of its responses—is not only a structural issue that inevitably arises during the training process of large language models but also a fundamental problem stemming from universal human cognitive biases. This is exacerbated by the commercial incentives of AI companies, which stand to gain more profit from the reactions of users who prefer AI sycophancy. Although South Korea and the EU have recently enacted general laws on AI and begun implementing cautious regulations, these measures are insufficient to resolve the sycophancy issue. While there are exceptional cases where regulations can be applied in South Korea, relying solely on requiring AI operators to implement technical measures is insufficient to address the structural nature of AI sycophancy. This is because the phenomenon of AI sycophancy is not a problem that AI operators can control through technical measures alone. In the EU, regulation ranges from outright bans on AI activities that are highly influential or pose significant risks if they cause serious physical harm or violations of fundamental rights, to transparency obligations requiring the disclosure of information regarding the design and operation of AI models—or declarations of AI literacy obligations— in cases where the risks are not as severe. This approach results in regulation that is either excessive or insufficient. Ultimately, the phenomenon of AI sycophancy reflects the fundamental limitations of the current AI alignment paradigm. Since we cannot simply wait idly while technological innovations to address these limitations remain insufficient, we need to devise a reasonable regulatory approach to protect our society and its members immediately. First, users must cultivate AI literacy including the capability of critical thinking to the greatest extent possible so that they can understand the potential biases and limitations of the answers provided by AI models and protect themselves from AI sycophancy. For the legal reality that users employ AI models at their own risk to hold legitimacy, sufficient information must be provided to assist users in making accurate judgments. As a concrete measure to fulfill this duty of active disclosure, AI service providers must regularly provide users with detailed, specific usage guidelines. This should involve providing data verified through various AI tests on the AI sycophancy phenomenon—specifying in which areas and in what ways such sycophancy may occur—and notifying users of the risks of potential harm they may face as a result. Regulatory authorities should oversee whether AI service providers are adequately fulfilling this proactive duty to explain, and provide relevant guidelines to them.

키워드