The Cycle Generative Adversarial Network, or CycleGAN, is an approach to training a deep convolutional neural network for image-to-image translation tasks.<\/p>\n
Unlike other GAN models for image translation, the CycleGAN does not require a dataset of paired images. For example, if we are interested in translating photographs of oranges to apples, we do not require a training dataset of oranges that have been manually converted to apples. This allows the development of a translation model on problems where training datasets may not exist, such as translating paintings to photographs.<\/p>\n
In this tutorial, you will discover how to develop a CycleGAN model to translate photos of horses to zebras, and back again.<\/p>\n
After completing this tutorial, you will know:<\/p>\n
- \n
- How to load and prepare the horses to zebras image translation dataset for modeling.<\/li>\n
- How to train a pair of CycleGAN generator models for translating horses to zebras and zebras to horses.<\/li>\n
- How to load saved CycleGAN models and use them to translate photographs.<\/li>\n<\/ul>\n
Discover how to develop DCGANs, conditional GANs, Pix2Pix, CycleGANs, and more with Keras in my new GANs book<\/a>, with 29 step-by-step tutorials and full source code.<\/p>\n
Let\u2019s get started.<\/p>\n
![\"How](\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/08\/How-to-Develop-a-CycleGAN-for-Image-to-Image-Translation-with-Keras.jpg\")
<\/p>\nHow to Develop a CycleGAN for Image-to-Image Translation with Keras
Photo by A. Munar<\/a>, some rights reserved.<\/p>\n<\/div>\nTutorial Overview<\/h2>\n
This tutorial is divided into four parts; they are:<\/p>\n
- \n
- What Is the CycleGAN?<\/li>\n
- How to Prepare the Horses to Zebras Dataset<\/li>\n
- How to Develop a CycleGAN to Translate Horses to Zebras<\/li>\n
- How to Perform Image Translation with CycleGAN Generators<\/li>\n<\/ol>\n
What Is the CycleGAN?<\/h2>\n
The CycleGAN model was described by Jun-Yan Zhu<\/a>, et al. in their 2017 paper titled \u201cUnpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks<\/a>.\u201d<\/p>\n
The benefit of the CycleGAN model is that it can be trained without paired examples. That is, it does not require examples of photographs before and after the translation in order to train the model, e.g. photos of the same city landscape during the day and at night. Instead, the model is able to use a collection of photographs from each domain and extract and harness the underlying style of images in the collection in order to perform the translation.<\/p>\n
The model architecture is comprised of two generator models: one generator (Generator-A) for generating images for the first domain (Domain-A) and the second generator (Generator-B) for generating images for the second domain (Domain-B).<\/p>\n
- \n
- Generator-A -> Domain-A<\/li>\n
- Generator-B -> Domain-B<\/li>\n<\/ul>\n
The generator models perform image translation, meaning that the image generation process is conditional on an input image, specifically an image from the other domain. Generator-A takes an image from Domain-B as input and Generator-B takes an image from Domain-A as input.<\/p>\n
- \n
- Domain-B -> Generator-A -> Domain-A<\/li>\n
- Domain-A -> Generator-B -> Domain-B<\/li>\n<\/ul>\n
Each generator has a corresponding discriminator model. The first discriminator model (Discriminator-A) takes real images from Domain-A and generated images from Generator-A and predicts whether they are real or fake. The second discriminator model (Discriminator-B) takes real images from Domain-B and generated images from Generator-B and predicts whether they are real or fake.<\/p>\n
- \n
- Domain-A -> Discriminator-A -> [Real\/Fake]<\/li>\n
- Domain-B -> Generator-A -> Discriminator-A -> [Real\/Fake]<\/li>\n
- Domain-B -> Discriminator-B -> [Real\/Fake]<\/li>\n
- Domain-A -> Generator-B -> Discriminator-B -> [Real\/Fake]<\/li>\n<\/ul>\n
The discriminator and generator models are trained in an adversarial zero-sum process, like normal GAN models. The generators learn to better fool the discriminators and the discriminator learn to better detect fake images. Together, the models find an equilibrium during the training process.<\/p>\n
Additionally, the generator models are regularized to not just create new images in the target domain, but instead translate more reconstructed versions of the input images from the source domain. This is achieved by using generated images as input to the corresponding generator model and comparing the output image to the original images. Passing an image through both generators is called a cycle. Together, each pair of generator models are trained to better reproduce the original source image, referred to as cycle consistency.<\/p>\n
- \n
- Domain-B -> Generator-A -> Domain-A -> Generator-B -> Domain-B<\/li>\n
- Domain-A -> Generator-B -> Domain-B -> Generator-A -> Domain-A<\/li>\n<\/ul>\n
There is one further element to the architecture, referred to as the identity mapping. This is where a generator is provided with images as input from the target domain and is expected to generate the same image without change. This addition to the architecture is optional, although results in a better matching of the color profile of the input image.<\/p>\n
- \n
- Domain-A -> Generator-A -> Domain-A<\/li>\n
- Domain-B -> Generator-B -> Domain-B<\/li>\n<\/ul>\n
Now that we are familiar with the model architecture, we can take a closer look at each model in turn and how they can be implemented.<\/p>\n
The paper<\/a> provides a good description of the models and training process, although the official Torch implementation<\/a> was used as the definitive description for each model and training process and provides the basis for the the model implementations described below.<\/p>\n
<\/div>\n<\/p>\n Want to Develop GANs from Scratch?<\/h3>\n
Take my free 7-day email crash course now (with sample code).<\/p>\n
Click to sign-up and also get a free PDF Ebook version of the course.<\/p>\n