Controllable Generative Adversarial Network

Abstract

Recently introduced generative adversarial networks (GANs) have been shown numerous promising results to generate realistic samples. In the last couple of years, it has been studied to control features in synthetic samples generated by the GAN. Auxiliary classifier GAN (ACGAN), a conventional method to generate conditional samples, employs a classification layer in discriminator to solve the problem. However, in this paper, we demonstrate that the auxiliary classifier can hardly provide good guidance for training of the generator, where the classifier suffers from overfitting. Since the generator learns from classification loss, such a problem has a chance to hinder the training. To overcome this limitation, here, we propose a controllable GAN (ControlGAN) structure. By separating a feature classifier from the discriminator, the classifier can be trained with data augmentation technique, which can support to make a fine classifier. Evaluated with the CIFAR-10 dataset, ControlGAN outperforms AC-WGAN-GP which is an improved version of the ACGAN, where Inception score of the ControlGAN is 8.61 +/- 0.10. Furthermore, we demonstrate that the ControlGAN can generate intermediate features and opposite features for interpolated input and extrapolated input labels that are not used in the training process. It implies that the ControlGAN can significantly contribute to the variety of generated samples.