Generative Adversarial Networks, or GANs, are an architecture for training generative models, such as deep convolutional neural networks for generating images.<\/p>\n
The conditional generative adversarial network, or cGAN for short, is a type of GAN that involves the conditional generation of images by a generator model. Image generation can be conditional on a class label, if available, allowing the targeted generated of images of a given type.<\/p>\n
The Auxiliary Classifier GAN, or AC-GAN for short, is an extension of the conditional GAN that changes the discriminator to predict the class label of a given image rather than receive it as input. It has the effect of stabilizing the training process and allowing the generation of large high-quality images whilst learning a representation in the latent space that is independent of the class label.<\/p>\n
In this tutorial, you will discover how to develop an auxiliary classifier generative adversarial network for generating photographs of clothing.<\/p>\n
After completing this tutorial, you will know:<\/p>\n
- \n
- The auxiliary classifier GAN is a type of conditional GAN that requires that the discriminator predict the class label of a given image.<\/li>\n
- How to develop generator, discriminator, and composite models for the AC-GAN.<\/li>\n
- How to train, evaluate, and use an AC-GAN to generate photographs of clothing from the Fashion-MNIST dataset.<\/li>\n<\/ul>\n
Discover how to develop DCGANs, conditional GANs, Pix2Pix, CycleGANs, and more with Keras in my new GANs book<\/a>, with 29 step-by-step tutorials and full source code.<\/p>\n
Let\u2019s get started.<\/p>\n
![\"How](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/07\/How-to-Develop-an-Auxiliary-Classifier-GAN-AC-GAN-From-Scratch-with-Keras.jpg\")
<\/p>\nHow to Develop an Auxiliary Classifier GAN (AC-GAN) From Scratch with Keras
Photo by ebbe ostebo<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
This tutorial is divided into five parts; they are:<\/p>\n
- \n
- Auxiliary Classifier Generative Adversarial Networks<\/li>\n
- Fashion-MNIST Clothing Photograph Dataset<\/li>\n
- How to Define AC-GAN Models<\/li>\n
- How to Develop an AC-GAN for Fashion-MNIST<\/li>\n
- How to Generate Items of Clothing With the AC-GAN<\/li>\n<\/ol>\n
Auxiliary Classifier Generative Adversarial Networks<\/h2>\n
The generative adversarial network is an architecture for training a generative model, typically deep convolutional neural networks for generating image.<\/p>\n
The architecture is comprised of both a generator model that takes random points from a latent space as input and generates images, and a discriminator for classifying images as either real (from the dataset) or fake (generate). Both models are then trained simultaneously in a zero-sum game.<\/p>\n
A conditional GAN, cGAN or CGAN for short, is an extension of the GAN architecture that adds structure to the latent space. The training of the GAN model is changed so that the generator is provided both with a point in the latent space and a class label as input, and attempts to generate an image for that class. The discriminator is provided with both an image and the class label and must classify whether the image is real or fake as before.<\/p>\n
The addition of the class as input makes the image generation process, and image classification process, conditional on the class label, hence the name. The effect is both a more stable training process and a resulting generator model that can be used to generate images of a given specific type, e.g. for a class label.<\/p>\n
The Auxiliary Classifier GAN, or AC-GAN for short, is a further extension of the GAN architecture building upon the CGAN extension. It was introduced by Augustus Odena<\/a>, et al. from Google Brain in the 2016 paper titled \u201cConditional Image Synthesis with Auxiliary Classifier GANs<\/a>.\u201d<\/p>\n
As with the conditional GAN, the generator model in the AC-GAN is provided both with a point in the latent space and the class label as input, e.g. the image generation process is conditional.<\/p>\n
The main difference is in the discriminator model, which is only provided with the image as input, unlike the conditional GAN that is provided with the image and class label as input. The discriminator model must then predict whether the given image is real or fake as before, and must also predict the class label of the image.<\/p>\n
\n
\u2026 the model [\u2026] is class conditional, but with an auxiliary decoder that is tasked with reconstructing class labels.<\/p>\n<\/blockquote>\n
\u2014 Conditional Image Synthesis With Auxiliary Classifier GANs<\/a>, 2016.<\/p>\n
The architecture is described in such a way that the discriminator and auxiliary classifier may be considered separate models that share model weights. In practice, the discriminator and auxiliary classifier can be implemented as a single neural network model with two outputs.<\/p>\n
The first output is a single probability via the sigmoid activation function that indicates the \u201crealness<\/em>\u201d of the input image and is optimized using binary cross entropy like a normal GAN discriminator model.<\/p>\n
The second output is a probability of the image belonging to each class via the softmax activation function, like any given multi-class classification neural network model, and is optimized using categorical cross entropy.<\/p>\n
To summarize:<\/p>\n
Generator Model:<\/h4>\n
- \n
- Input<\/strong>: Random point from the latent space, and the class label.<\/li>\n
- Output<\/strong>: Generated image.<\/li>\n<\/ul>\n
Discriminator Model:<\/h4>\n
- \n
- Input<\/strong>: Image.<\/li>\n
- Output<\/strong>: Probability that the provided image is real, probability of the image belonging to each known class.<\/li>\n<\/ul>\n
The plot below summarizes the inputs and outputs of a range of conditional GANs, including the AC-GAN, providing some context for the differences.<\/p>\n
![\"Summary](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/05\/Summary-of-the-Differences-Between-the-Conditional-GAN-Semi-Supervised-GAN-InfoGAN-and-AC-GAN-1024x423.png\")
<\/p>\nSummary of the Differences Between the Conditional GAN, Semi-Supervised GAN, InfoGAN, and AC-GAN.
Taken from: Version of Conditional Image Synthesis With Auxiliary Classifier GANs.<\/p>\n<\/div>\nThe discriminator seeks to maximize the probability of correctly classifying real and fake images (LS) and correctly predicting the class label (LC) of a real or fake image (e.g. LS + LC). The generator seeks to minimize the ability of the discriminator to discriminate real and fake images whilst also maximizing the ability of the discriminator predicting the class label of real and fake images (e.g. LC \u2013 LS).<\/p>\n
\n
The objective function has two parts: the log-likelihood of the correct source, LS, and the log-likelihood of the correct class, LC. [\u2026] D is trained to maximize LS + LC while G is trained to maximize LC \u2212 LS.<\/p>\n<\/blockquote>\n
\u2014 Conditional Image Synthesis With Auxiliary Classifier GANs<\/a>, 2016.<\/p>\n
The resulting generator learns a latent space representation that is independent of the class label, unlike the conditional GAN.<\/p>\n
\n
AC-GANs learn a representation for z that is independent of class label.<\/p>\n<\/blockquote>\n
\u2014 Conditional Image Synthesis With Auxiliary Classifier GANs<\/a>, 2016.<\/p>\n
The effect of changing the conditional GAN in this way is both a more stable training process and the ability of the model to generate higher quality images with a larger size than had been previously possible, e.g. 128\u00d7128 pixels.<\/p>\n
\n
\u2026 we demonstrate that adding more structure to the GAN latent space along with a specialized cost function results in higher quality samples. [\u2026] Importantly, we demonstrate quantitatively that our high resolution samples are not just naive resizings of low resolution samples.<\/p>\n<\/blockquote>\n
- Output<\/strong>: Probability that the provided image is real, probability of the image belonging to each known class.<\/li>\n<\/ul>\n
- Output<\/strong>: Generated image.<\/li>\n<\/ul>\n
- Input<\/strong>: Random point from the latent space, and the class label.<\/li>\n