Augmentation leak-prevention scheme using an auxiliary classifier in GAN-based image generation

초록

Although a generative adversarial network (GAN) can generate realistic and distinct images, it requires numerous training data. Data augmentation is a popular method of incrementing data using various aug-mentation operations. However, data augmentation for training GANs can lead to augmentation leaks where unintended distortion effects in augmented images appear in images generated by the GAN. This augmentation leaks occur because the GAN is trained to generate images similar to the training data-set containing augmented images. Although recent studies have revealed that maintaining the invertibil-ity of data augmentation can prevent augmentation leaks, maintaining it for various data augmentation configurations and datasets is challenging. This paper proposes a novel auxiliary neural network called an auxiliary augmentation operation classifier (AAOC) to prevent augmentation leaks. The AAOC learns the classification of augmentation operations during GAN training, guiding the GAN to avoid generating images with unintended distortions. Due to its simple and lightweight architecture, the AAOC does not increase the training time of GANs. Extensive experiments on the three public datasets revealed that the AAOC reduced the occurrence of augmentation leaks by a factor of 30 compared to the basic method and improved image quality by 50% for leak-free images.& COPY; 2023 The Author(s). Published by Elsevier B.V. on behalf of King Saud University. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

키워드