{"id":1961,"date":"2019-04-02T18:00:23","date_gmt":"2019-04-02T18:00:23","guid":{"rendered":"https:\/\/www.aiproblog.com\/index.php\/2019\/04\/02\/how-to-normalize-center-and-standardize-images-with-the-imagedatagenerator-in-keras\/"},"modified":"2019-04-02T18:00:23","modified_gmt":"2019-04-02T18:00:23","slug":"how-to-normalize-center-and-standardize-images-with-the-imagedatagenerator-in-keras","status":"publish","type":"post","link":"https:\/\/www.aiproblog.com\/index.php\/2019\/04\/02\/how-to-normalize-center-and-standardize-images-with-the-imagedatagenerator-in-keras\/","title":{"rendered":"How to Normalize, Center, and Standardize Images With the ImageDataGenerator in Keras"},"content":{"rendered":"<p>Author: Jason Brownlee<\/p>\n<div>\n<p>The pixel values in images must be scaled prior to providing the images as input to a deep learning neural network model during the training or evaluation of the model.<\/p>\n<p>Traditionally, the images would have to be scaled prior to the development of the model and stored in memory or on disk in the scaled format.<\/p>\n<p>An alternative approach is to scale the images using a preferred scaling technique just-in-time during the training or model evaluation process. Keras supports this type of data preparation for image data via the ImageDataGenerator class and API.<\/p>\n<p>In this tutorial, you will discover how to use the ImageDataGenerator class to scale pixel data just-in-time when fitting and evaluating deep learning neural network models.<\/p>\n<p>After completing this tutorial, you will know:<\/p>\n<ul>\n<li>How to configure and a use the ImageDataGenerator class for train, validation, and test datasets of images.<\/li>\n<li>How to use the ImageDataGenerator to normalize pixel values when fitting and evaluating a convolutional neural network model.<\/li>\n<li>How to use the ImageDataGenerator to center and standardize pixel values when fitting and evaluating a convolutional neural network model.<\/li>\n<\/ul>\n<p>Let\u2019s get started.<\/p>\n<div id=\"attachment_7394\" style=\"width: 650px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7394\" class=\"size-full wp-image-7394\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/04\/How-to-Normalize-Center-and-Standardize-Images-With-the-ImageDataGenerator-in-Keras.jpg\" alt=\"How to Normalize, Center, and Standardize Images With the ImageDataGenerator in Keras\" width=\"640\" height=\"359\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/04\/How-to-Normalize-Center-and-Standardize-Images-With-the-ImageDataGenerator-in-Keras.jpg 640w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/04\/How-to-Normalize-Center-and-Standardize-Images-With-the-ImageDataGenerator-in-Keras-300x168.jpg 300w\" sizes=\"(max-width: 640px) 100vw, 640px\"><\/p>\n<p id=\"caption-attachment-7394\" class=\"wp-caption-text\">How to Normalize, Center, and Standardize Images With the ImageDataGenerator in Keras<br \/>Photo by <a href=\"https:\/\/www.flickr.com\/photos\/27568572@N06\/14580422837\/\">Sagar<\/a>, some rights reserved.<\/p>\n<\/div>\n<h2>Tutorial Overview<\/h2>\n<p>This tutorial is divided into five parts; they are:<\/p>\n<ol>\n<li>MNIST Handwritten Image Classification Dataset<\/li>\n<li>ImageDataGenerator class for Pixel Scaling<\/li>\n<li>How to Normalize Images With ImageDataGenerator<\/li>\n<li>How to Center Images With ImageDataGenerator<\/li>\n<li>How to Standardize Image With ImageDataGenerator<\/li>\n<\/ol>\n<h2>MNIST Handwritten Image Classification Dataset<\/h2>\n<p>Before we dive into the usage of the ImageDataGenerator class for preparing image data, we must select an image dataset on which to test the generator.<\/p>\n<p>The <a href=\"https:\/\/en.wikipedia.org\/wiki\/MNIST_database\">MNIST problem<\/a>, or MNIST for short, is an image classification problem comprised of 70,000 images of handwritten digits.<\/p>\n<p>The goal of the problem is to classify a given image of a handwritten digit as an integer from 0 to 9. As such, it is a multiclass image classification problem.<\/p>\n<p>This dataset is provided as part of the Keras library and can be automatically downloaded (if needed) and loaded into memory by a call to the <a href=\"https:\/\/keras.io\/datasets\/#mnist-database-of-handwritten-digits\">keras.datasets.mnist.load_data() function<\/a>.<\/p>\n<p>The function returns two tuples: one for the training inputs and outputs and one for the test inputs and outputs. For example:<\/p>\n<pre class=\"crayon-plain-tag\"># example of loading the MNIST dataset\r\nfrom keras.datasets import mnist\r\n(x_train, y_train), (x_test, y_test) = mnist.load_data()<\/pre>\n<p>We can load the MNIST dataset and summarize the dataset. The complete example is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># load and summarize the MNIST dataset\r\nfrom keras.datasets import mnist\r\n# load dataset\r\n(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\r\n# summarize dataset shape\r\nprint('Train', train_images.shape, train_labels.shape)\r\nprint('Test', (test_images.shape, test_labels.shape))\r\n# summarize pixel values\r\nprint('Train', train_images.min(), train_images.max(), train_images.mean(), train_images.std())\r\nprint('Train', test_images.min(), test_images.max(), test_images.mean(), test_images.std())<\/pre>\n<p>Running the example first loads the dataset into memory. Then the shape of the train and test datasets is reported.<\/p>\n<p>We can see that all images are 28 by 28 pixels with a single channel for black-and-white images. There are 60,000 images for the training dataset and 10,000 for the test dataset.<\/p>\n<p>We can also see that pixel values are integer values between 0 and 255 and that the mean and standard deviation of the pixel values are similar between the two datasets.<\/p>\n<pre class=\"crayon-plain-tag\">Train (60000, 28, 28) (60000,)\r\nTest ((10000, 28, 28), (10000,))\r\nTrain 0 255 33.318421449829934 78.56748998339798\r\nTrain 0 255 33.791224489795916 79.17246322228644<\/pre>\n<p>We will use this dataset to explore different pixel scaling methods using the ImageDataGenerator class in Keras.<\/p>\n<div class=\"woo-sc-hr\"><\/div>\n<p><center><\/p>\n<h3>Want Results with Deep Learning for Computer Vision?<\/h3>\n<p>Take my free 7-day email crash course now (with sample code).<\/p>\n<p>Click to sign-up and also get a free PDF Ebook version of the course.<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.lpages.co\/leadbox\/1458ca1e0972a2%3A164f8be4f346dc\/4715926590455808\/\" target=\"_blank\" style=\"background: rgb(255, 206, 10); color: rgb(255, 255, 255); text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-weight: bold; font-size: 16px; line-height: 20px; padding: 10px; display: inline-block; max-width: 300px; border-radius: 5px; text-shadow: rgba(0, 0, 0, 0.25) 0px -1px 1px; box-shadow: rgba(255, 255, 255, 0.5) 0px 1px 3px inset, rgba(0, 0, 0, 0.5) 0px 1px 3px;\" rel=\"noopener noreferrer\">Download Your FREE Mini-Course<\/a><script data-leadbox=\"1458ca1e0972a2:164f8be4f346dc\" data-url=\"https:\/\/machinelearningmastery.lpages.co\/leadbox\/1458ca1e0972a2%3A164f8be4f346dc\/4715926590455808\/\" data-config=\"%7B%7D\" type=\"text\/javascript\" src=\"https:\/\/machinelearningmastery.lpages.co\/leadbox-1553357564.js\"><\/script><\/p>\n<p><\/center><\/p>\n<div class=\"woo-sc-hr\"><\/div>\n<h2>ImageDataGenerator Class for Pixel Scaling<\/h2>\n<p>The ImageDataGenerator class in Keras provides a suite of techniques for scaling pixel values in your image dataset prior to modeling.<\/p>\n<p>The class will wrap your image dataset, then when requested, it will return images in batches to the algorithm during training, validation, or evaluation and apply the scaling operations just-in-time. This provides an efficient and convenient approach to scaling image data when modeling with neural networks.<\/p>\n<p>The usage of the ImageDataGenerator class is as follows.<\/p>\n<ul>\n<li>1. Load your dataset.<\/li>\n<li>2. Configure the ImageDataGenerator (e.g. construct an instance).<\/li>\n<li>3. Calculate image statistics (e.g. call the <em>fit()<\/em> function).<\/li>\n<li>4. Use the generator to fit the model (e.g. pass the instance to the <em>fit_generator()<\/em> function).<\/li>\n<li>5. Use the generator to evaluate the model (e.g. pass the instance to the <em>evaluate_generator()<\/em> function).<\/li>\n<\/ul>\n<p>The ImageDataGenerator class supports a number of pixel scaling methods, as well as a range of data augmentation techniques. We will focus on the pixel scaling techniques and leave the data augmentation methods to a later discussion.<\/p>\n<p>The three main types of pixel scaling techniques supported by the ImageDataGenerator class are as follows:<\/p>\n<ul>\n<li><strong>Pixel Normalization<\/strong>: scale pixel values to the range 0-1.<\/li>\n<li><strong>Pixel Centering<\/strong>: scale pixel values to have a zero mean.<\/li>\n<li><strong>Pixel Standardization<\/strong>: scale pixel values to have a zero mean and unit variance.<\/li>\n<\/ul>\n<p>The pixel standardization is supported at two levels: either per-image (called sample-wise) or per-dataset (called feature-wise). Specifically, the mean and\/or mean and standard deviation statistics required to standardize pixel values can be calculated from the pixel values in each image only (sample-wise) or across the entire training dataset (feature-wise).<\/p>\n<p>Other pixel scaling methods are supported, such as ZCA, brightening, and more, but wel will focus on these three most common methods.<\/p>\n<p>The choice of pixel scaling is selected by specifying arguments to the ImageDataGenerator when an instance is constructed; for example:<\/p>\n<pre class=\"crayon-plain-tag\"># create and configure the data generator\r\ndatagen = ImageDataGenerator(...)<\/pre>\n<p>Next, if the chosen scaling method requires that statistics be calculated across the training dataset, then these statistics can be calculated and stored by calling the <em>fit()<\/em> function.<\/p>\n<p>When evaluating and selecting a model, it is common to calculate these statistics on the training dataset and then apply them to the validation and test datasets.<\/p>\n<pre class=\"crayon-plain-tag\"># calculate scaling statistics on the training dataset\r\ndatagen.fit(trainX)<\/pre>\n<p>Once prepared, the data generator can be used to fit a neural network model by calling the <em>flow()<\/em> function to retrieve an iterator that returns batches of samples and passing it to the <em>fit_generator()<\/em> function.<\/p>\n<pre class=\"crayon-plain-tag\"># get batch iterator\r\ntrain_iterator = datagen.flow(trainX, trainy)\r\n# fit model\r\nmodel.fit_generator(train_iterator, ...)<\/pre>\n<p>If a validation dataset is required, a separate batch iterator can be created from the same data generator that will perform the same pixel scaling operations and use any required statistics calculated on the training dataset.<\/p>\n<pre class=\"crayon-plain-tag\"># get batch iterator for training\r\ntrain_iterator = datagen.flow(trainX, trainy)\r\n# get batch iterator for validation\r\nval_iterator = datagen.flow(valX, valy)\r\n# fit model\r\nmodel.fit_generator(train_iterator, validation_data=val_iterator, ...)<\/pre>\n<p>Once fit, the model can be evaluated by creating a batch iterator for the test dataset and calling the <em>evaluate_generator()<\/em> function on the model.<\/p>\n<p>Again, the same pixel scaling operations will be performed and any statistics calculated on the training dataset will be used, if needed.<\/p>\n<pre class=\"crayon-plain-tag\"># get batch iterator for testing\r\ntest_iterator = datagen.flow(testX, testy)\r\n# evaluate model loss on test dataset\r\nloss = model.evaluate_generator(test_iterator, ...)<\/pre>\n<p>Now that we are familiar with how to use the ImageDataGenerator class for scaling pixel values, let\u2019s look at some specific examples.<\/p>\n<h2>How to Normalize Images With ImageDataGenerator<\/h2>\n<p>The ImageDataGenerator class can be used to rescale pixel values from the range of 0-255 to the range 0-1 preferred for neural network models.<\/p>\n<p>Scaling data to the range of 0-1 is traditionally referred to as normalization.<\/p>\n<p>This can be achieved by setting the rescale argument to a ratio by which each pixel can be multiplied to achieve the desired range.<\/p>\n<p>In this case, the ratio is 1\/255 or about 0.0039. For example:<\/p>\n<pre class=\"crayon-plain-tag\"># create generator (1.0\/255.0 = 0.003921568627451)\r\ndatagen = ImageDataGenerator(rescale=1.0\/255.0)<\/pre>\n<p>The ImageDataGenerator does not need to be fit in this case because there are no global statistics that need to be calculated.<\/p>\n<p>Next, iterators can be created using the generator for both the train and test datasets. We will use a batch size of 64. This means that each of the train and test datasets of images are divided into groups of 64 images that will then be scaled when returned from the iterator.<\/p>\n<p>We can see how many batches there will be in one epoch, e.g. one pass through the training dataset, by printing the length of each iterator.<\/p>\n<pre class=\"crayon-plain-tag\"># prepare an iterators to scale images\r\ntrain_iterator = datagen.flow(trainX, trainY, batch_size=64)\r\ntest_iterator = datagen.flow(testX, testY, batch_size=64)\r\nprint('Batches train=%d, test=%d' % (len(train_iterator), len(test_iterator)))<\/pre>\n<p>We can then confirm that the pixel normalization has been performed as expected by retrieving the first batch of scaled images and inspecting the min and max pixel values.<\/p>\n<pre class=\"crayon-plain-tag\"># confirm the scaling works\r\nbatchX, batchy = train_iterator.next()\r\nprint('Batch shape=%s, min=%.3f, max=%.3f' % (batchX.shape, batchX.min(), batchX.max()))<\/pre>\n<p>Next, we can use the data generator to fit and evaluate a model. We will define a simple convolutional neural network model and fit it on the <em>train_iterator<\/em> for five epochs with 60,000 samples divided by 64 samples per batch, or about 938 batches per epoch.<\/p>\n<pre class=\"crayon-plain-tag\"># fit model with generator\r\nmodel.fit_generator(train_iterator, steps_per_epoch=len(train_iterator), epochs=5)<\/pre>\n<p>Once fit, we will evaluate the model on the test dataset, with about 10,000 images divided by 64 samples per batch, or about 157 steps in a single epoch.<\/p>\n<pre class=\"crayon-plain-tag\">_, acc = model.evaluate_generator(test_iterator, steps=len(test_iterator), verbose=0)\r\nprint('Test Accuracy: %.3f' % (acc * 100))<\/pre>\n<p>We can tie all of this together; the complete example is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># example of using ImageDataGenerator to normalize images\r\nfrom keras.datasets import mnist\r\nfrom keras.utils import to_categorical\r\nfrom keras.models import Sequential\r\nfrom keras.layers import Conv2D\r\nfrom keras.layers import MaxPooling2D\r\nfrom keras.layers import Dense\r\nfrom keras.layers import Flatten\r\nfrom keras.preprocessing.image import ImageDataGenerator\r\n# load dataset\r\n(trainX, trainY), (testX, testY) = mnist.load_data()\r\n# reshape dataset to have a single channel\r\nwidth, height, channels = trainX.shape[1], trainX.shape[2], 1\r\ntrainX = trainX.reshape((trainX.shape[0], width, height, channels))\r\ntestX = testX.reshape((testX.shape[0], width, height, channels))\r\n# one hot encode target values\r\ntrainY = to_categorical(trainY)\r\ntestY = to_categorical(testY)\r\n# confirm scale of pixels\r\nprint('Train min=%.3f, max=%.3f' % (trainX.min(), trainX.max()))\r\nprint('Test min=%.3f, max=%.3f' % (testX.min(), testX.max()))\r\n# create generator (1.0\/255.0 = 0.003921568627451)\r\ndatagen = ImageDataGenerator(rescale=1.0\/255.0)\r\n# prepare an iterators to scale images\r\ntrain_iterator = datagen.flow(trainX, trainY, batch_size=64)\r\ntest_iterator = datagen.flow(testX, testY, batch_size=64)\r\nprint('Batches train=%d, test=%d' % (len(train_iterator), len(test_iterator)))\r\n# confirm the scaling works\r\nbatchX, batchy = train_iterator.next()\r\nprint('Batch shape=%s, min=%.3f, max=%.3f' % (batchX.shape, batchX.min(), batchX.max()))\r\n# define model\r\nmodel = Sequential()\r\nmodel.add(Conv2D(32, (3, 3), activation='relu', input_shape=(width, height, channels)))\r\nmodel.add(MaxPooling2D((2, 2)))\r\nmodel.add(Conv2D(64, (3, 3), activation='relu'))\r\nmodel.add(MaxPooling2D((2, 2)))\r\nmodel.add(Flatten())\r\nmodel.add(Dense(64, activation='relu'))\r\nmodel.add(Dense(10, activation='softmax'))\r\n# compile model\r\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\r\n# fit model with generator\r\nmodel.fit_generator(train_iterator, steps_per_epoch=len(train_iterator), epochs=5)\r\n# evaluate model\r\n_, acc = model.evaluate_generator(test_iterator, steps=len(test_iterator), verbose=0)\r\nprint('Test Accuracy: %.3f' % (acc * 100))<\/pre>\n<p>Running the example first reports the min and max pixel values on the train and test sets. This confirms that indeed the raw data has pixel values in the range 0-255.<\/p>\n<p>Next, the data generator is created and the iterators are prepared. We can see that we have 938 batches per epoch with the training dataset and 157 batches per epoch with the test dataset.<\/p>\n<p>We retrieve the first batch from the dataset and confirm that it contains 64 images with the height and width (rows and columns) of 28 pixels and 1 channel, and that the new minimum and maximum pixel values are 0 and 1 respectively. This confirms that the normalization has had the desired effect.<\/p>\n<pre class=\"crayon-plain-tag\">Train min=0.000, max=255.000\r\nTest min=0.000, max=255.000\r\nBatches train=938, test=157\r\nBatch shape=(64, 28, 28, 1), min=0.000, max=1.000<\/pre>\n<p>The model is then fit on the normalized image data. Training does not take long on the CPU. Finally, the model is evaluated in the test dataset, applying the same normalization.<\/p>\n<pre class=\"crayon-plain-tag\">Epoch 1\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.1841 - acc: 0.9448\r\nEpoch 2\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.0573 - acc: 0.9826\r\nEpoch 3\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.0407 - acc: 0.9870\r\nEpoch 4\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.0299 - acc: 0.9904\r\nEpoch 5\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.0238 - acc: 0.9928\r\nTest Accuracy: 99.050<\/pre>\n<p>Now that we are familiar with how to use the ImageDataGenerator in general and specifically for image normalization, let\u2019s look at examples of pixel centering and standardization.<\/p>\n<h2>How to Center Images With ImageDataGenerator<\/h2>\n<p>Another popular pixel scaling method is to calculate the mean pixel value across the entire training dataset, then subtract it from each image.<\/p>\n<p>This is called centering and has the effect of centering the distribution of pixel values on zero: that is, the mean pixel value for centered images will be zero.<\/p>\n<p>The ImageDataGenerator class refers to centering that uses the mean calculated on the training dataset as feature-wise centering. It requires that the statistic is calculated on the training dataset prior to scaling.<\/p>\n<pre class=\"crayon-plain-tag\"># create generator that centers pixel values\r\ndatagen = ImageDataGenerator(featurewise_center=True)\r\n# calculate the mean on the training dataset\r\ndatagen.fit(trainX)<\/pre>\n<p>It is different to calculating of the mean pixel value for each image, which Keras refers to as sample-wise centering and does not require any statistics to be calculated on the training dataset.<\/p>\n<pre class=\"crayon-plain-tag\"># create generator that centers pixel values\r\ndatagen = ImageDataGenerator(samplewise_center=True)<\/pre>\n<p>We will demonstrate feature-wise centering in this section. Once the statistic is calculated on the training dataset, we can confirm the value by accessing and printing it; for example:<\/p>\n<pre class=\"crayon-plain-tag\"># print the mean calculated on the training dataset.\r\nprint(datagen.mean)<\/pre>\n<p>We can also confirm that the scaling procedure has had the desired effect by calculating the mean of a batch of images returned from the batch iterator. We would expect the mean to be a small value close to zero, but not zero because of the small number of images in the batch.<\/p>\n<pre class=\"crayon-plain-tag\"># get a batch\r\nbatchX, batchy = iterator.next()\r\n# mean pixel value in the batch\r\nprint(batchX.shape, batchX.mean())<\/pre>\n<p>A better check would be to set the batch size to the size of the training dataset (e.g. 60,000 samples), retrieve one batch, then calculate the mean. It should be a very small value close to zero.<\/p>\n<pre class=\"crayon-plain-tag\"># try to flow the entire training dataset\r\niterator = datagen.flow(trainX, trainy, batch_size=len(trainX), shuffle=False)\r\n# get a batch\r\nbatchX, batchy = iterator.next()\r\n# mean pixel value in the batch\r\nprint(batchX.shape, batchX.mean())<\/pre>\n<p>The complete example is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># example of centering a image dataset\r\nfrom keras.datasets import mnist\r\nfrom keras.preprocessing.image import ImageDataGenerator\r\n# load dataset\r\n(trainX, trainy), (testX, testy) = mnist.load_data()\r\n# reshape dataset to have a single channel\r\nwidth, height, channels = trainX.shape[1], trainX.shape[2], 1\r\ntrainX = trainX.reshape((trainX.shape[0], width, height, channels))\r\ntestX = testX.reshape((testX.shape[0], width, height, channels))\r\n# report per-image mean\r\nprint('Means train=%.3f, test=%.3f' % (trainX.mean(), testX.mean()))\r\n# create generator that centers pixel values\r\ndatagen = ImageDataGenerator(featurewise_center=True)\r\n# calculate the mean on the training dataset\r\ndatagen.fit(trainX)\r\nprint('Data Generator Mean: %.3f' % datagen.mean)\r\n# demonstrate effect on a single batch of samples\r\niterator = datagen.flow(trainX, trainy, batch_size=64)\r\n# get a batch\r\nbatchX, batchy = iterator.next()\r\n# mean pixel value in the batch\r\nprint(batchX.shape, batchX.mean())\r\n# demonstrate effect on entire training dataset\r\niterator = datagen.flow(trainX, trainy, batch_size=len(trainX), shuffle=False)\r\n# get a batch\r\nbatchX, batchy = iterator.next()\r\n# mean pixel value in the batch\r\nprint(batchX.shape, batchX.mean())<\/pre>\n<p>Running the example first reports the mean pixel value for the train and test datasets.<\/p>\n<p>The MNIST dataset only has a single channel because the images are black and white (grayscale), but if the images were color, the mean pixel values would be calculated across all channels in all images in the training dataset, i.e. there would not be a separate mean value for each channel.<\/p>\n<p>The ImageDataGenerator is fit on the training dataset and we can confirm that the mean pixel value matches our own manual calculation.<\/p>\n<p>A single batch of centered images is retrieved and we can confirm that the mean pixel value is a small-ish value close to zero. The test is repeated using the entire training dataset as a the batch size, and in this case, the mean pixel value for the scaled dataset is a number very close to zero, confirming that centering is having the desired effect.<\/p>\n<pre class=\"crayon-plain-tag\">Means train=33.318, test=33.791\r\nData Generator Mean: 33.318\r\n(64, 28, 28, 1) 0.09971977\r\n(60000, 28, 28, 1) -1.9512918e-05<\/pre>\n<p>We can demonstrate centering with our convolutional neural network developed in the previous section.<\/p>\n<p>The complete example with feature-wise centering is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># example of using ImageDataGenerator to center images\r\nfrom keras.datasets import mnist\r\nfrom keras.utils import to_categorical\r\nfrom keras.models import Sequential\r\nfrom keras.layers import Conv2D\r\nfrom keras.layers import MaxPooling2D\r\nfrom keras.layers import Dense\r\nfrom keras.layers import Flatten\r\nfrom keras.preprocessing.image import ImageDataGenerator\r\n# load dataset\r\n(trainX, trainY), (testX, testY) = mnist.load_data()\r\n# reshape dataset to have a single channel\r\nwidth, height, channels = trainX.shape[1], trainX.shape[2], 1\r\ntrainX = trainX.reshape((trainX.shape[0], width, height, channels))\r\ntestX = testX.reshape((testX.shape[0], width, height, channels))\r\n# one hot encode target values\r\ntrainY = to_categorical(trainY)\r\ntestY = to_categorical(testY)\r\n# create generator to center images\r\ndatagen = ImageDataGenerator(featurewise_center=True)\r\n# calculate mean on training dataset\r\ndatagen.fit(trainX)\r\n# prepare an iterators to scale images\r\ntrain_iterator = datagen.flow(trainX, trainY, batch_size=64)\r\ntest_iterator = datagen.flow(testX, testY, batch_size=64)\r\nprint('Batches train=%d, test=%d' % (len(train_iterator), len(test_iterator)))\r\n# define model\r\nmodel = Sequential()\r\nmodel.add(Conv2D(32, (3, 3), activation='relu', input_shape=(width, height, channels)))\r\nmodel.add(MaxPooling2D((2, 2)))\r\nmodel.add(Conv2D(64, (3, 3), activation='relu'))\r\nmodel.add(MaxPooling2D((2, 2)))\r\nmodel.add(Flatten())\r\nmodel.add(Dense(64, activation='relu'))\r\nmodel.add(Dense(10, activation='softmax'))\r\n# compile model\r\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\r\n# fit model with generator\r\nmodel.fit_generator(train_iterator, steps_per_epoch=len(train_iterator), epochs=5)\r\n# evaluate model\r\n_, acc = model.evaluate_generator(test_iterator, steps=len(test_iterator), verbose=0)\r\nprint('Test Accuracy: %.3f' % (acc * 100))<\/pre>\n<p>Running the example prepares the ImageDataGenerator, centering images using statistics calculated on the training dataset.<\/p>\n<p>We can see that performance starts off poor but does improve. The centered pixel values will have a range of about -227 to 227, and neural networks often train more efficiently with small inputs. Normalizing followed by centering would be a better approach in practice.<\/p>\n<p>Importantly, the model is evaluated on the test dataset, where the images in the test dataset were centered using the mean value calculated on the training dataset. This is to avoid any data leakage.<\/p>\n<pre class=\"crayon-plain-tag\">Batches train=938, test=157\r\nEpoch 1\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 12.8824 - acc: 0.2001\r\nEpoch 2\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 6.1425 - acc: 0.5958\r\nEpoch 3\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.0678 - acc: 0.9796\r\nEpoch 4\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.0464 - acc: 0.9857\r\nEpoch 5\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.0373 - acc: 0.9880\r\nTest Accuracy: 98.540<\/pre>\n<\/p>\n<h2>How to Standardize Image With ImageDataGenerator<\/h2>\n<p>Standardization is a data scaling technique that assumes that the distribution of the data is Gaussian and shifts the distribution of the data to have a mean of zero and a standard deviation of one.<\/p>\n<p>Data with this distribution is referred to as a standard Gaussian. It can be beneficial when training neural networks as the dataset sums to zero and the inputs are small values in the rough range of about -3.0 to 3.0 (e.g. <a href=\"https:\/\/en.wikipedia.org\/wiki\/68%E2%80%9395%E2%80%9399.7_rule\">99.7 of the values will fall within three standard deviations of the mean<\/a>).<\/p>\n<p>Standardization of images is achieved by subtracting the mean pixel value and dividing the result by the standard deviation of the pixel values.<\/p>\n<p>The mean and standard deviation statistics can be calculated on the training dataset, and as discussed in the previous section, Keras refers to this as feature-wise.<\/p>\n<pre class=\"crayon-plain-tag\"># feature-wise generator\r\ndatagen = ImageDataGenerator(featurewise_center=True, featurewise_std_normalization=True)\r\n# calculate mean and standard deviation on the training dataset\r\ndatagen.fit(trainX)<\/pre>\n<p>The statistics can also be calculated then used to standardize each image separately, and Keras refers to this as sample-wise standardization.<\/p>\n<pre class=\"crayon-plain-tag\"># sample-wise standardization\r\ndatagen = ImageDataGenerator(samplewise_center=True, samplewise_std_normalization=True)<\/pre>\n<p>We will demonstrate the former or feature-wise approach to image standardization in this section. The effect will be batches of images with an approximate mean of zero and a standard deviation of one.<\/p>\n<p>As with the previous section, we can confirm this with some simple experiments. The complete example is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># example of standardizing a image dataset\r\nfrom keras.datasets import mnist\r\nfrom keras.preprocessing.image import ImageDataGenerator\r\n# load dataset\r\n(trainX, trainy), (testX, testy) = mnist.load_data()\r\n# reshape dataset to have a single channel\r\nwidth, height, channels = trainX.shape[1], trainX.shape[2], 1\r\ntrainX = trainX.reshape((trainX.shape[0], width, height, channels))\r\ntestX = testX.reshape((testX.shape[0], width, height, channels))\r\n# report pixel means and standard deviations\r\nprint('Statistics train=%.3f (%.3f), test=%.3f (%.3f)' % (trainX.mean(), trainX.std(), testX.mean(), testX.std()))\r\n# create generator that centers pixel values\r\ndatagen = ImageDataGenerator(featurewise_center=True, featurewise_std_normalization=True)\r\n# calculate the mean on the training dataset\r\ndatagen.fit(trainX)\r\nprint('Data Generator mean=%.3f, std=%.3f' % (datagen.mean, datagen.std))\r\n# demonstrate effect on a single batch of samples\r\niterator = datagen.flow(trainX, trainy, batch_size=64)\r\n# get a batch\r\nbatchX, batchy = iterator.next()\r\n# pixel stats in the batch\r\nprint(batchX.shape, batchX.mean(), batchX.std())\r\n# demonstrate effect on entire training dataset\r\niterator = datagen.flow(trainX, trainy, batch_size=len(trainX), shuffle=False)\r\n# get a batch\r\nbatchX, batchy = iterator.next()\r\n# pixel stats in the batch\r\nprint(batchX.shape, batchX.mean(), batchX.std())<\/pre>\n<p>Running the example first reports the mean and standard deviation of pixel values in the train and test datasets.<\/p>\n<p>The data generator is then configured for feature-wise standardization and the statistics are calculated on the training dataset, matching what we would expect when the statistics are calculated manually.<\/p>\n<p>A single batch of 64 standardized images is then retrieved and we can confirm that the mean and standard deviation of this small sample is close to the expected standard Gaussian.<\/p>\n<p>The test is then repeated on the entire training dataset and we can confirm that the mean is indeed a very small value close to 0.0 and the standard deviation is a value very close to 1.0.<\/p>\n<pre class=\"crayon-plain-tag\">Statistics train=33.318 (78.567), test=33.791 (79.172)\r\nData Generator mean=33.318, std=78.567\r\n(64, 28, 28, 1) 0.010656365 1.0107679\r\n(60000, 28, 28, 1) -3.4560264e-07 0.9999998<\/pre>\n<p>Now that we have confirmed that the standardization of pixel values is being performed as we expect, we can apply the pixel scaling while fitting and evaluating a convolutional neural network model.<\/p>\n<p>The complete example is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># example of using ImageDataGenerator to standardize images\r\nfrom keras.datasets import mnist\r\nfrom keras.utils import to_categorical\r\nfrom keras.models import Sequential\r\nfrom keras.layers import Conv2D\r\nfrom keras.layers import MaxPooling2D\r\nfrom keras.layers import Dense\r\nfrom keras.layers import Flatten\r\nfrom keras.preprocessing.image import ImageDataGenerator\r\n# load dataset\r\n(trainX, trainY), (testX, testY) = mnist.load_data()\r\n# reshape dataset to have a single channel\r\nwidth, height, channels = trainX.shape[1], trainX.shape[2], 1\r\ntrainX = trainX.reshape((trainX.shape[0], width, height, channels))\r\ntestX = testX.reshape((testX.shape[0], width, height, channels))\r\n# one hot encode target values\r\ntrainY = to_categorical(trainY)\r\ntestY = to_categorical(testY)\r\n# create generator to standardize images\r\ndatagen = ImageDataGenerator(featurewise_center=True, featurewise_std_normalization=True)\r\n# calculate mean on training dataset\r\ndatagen.fit(trainX)\r\n# prepare an iterators to scale images\r\ntrain_iterator = datagen.flow(trainX, trainY, batch_size=64)\r\ntest_iterator = datagen.flow(testX, testY, batch_size=64)\r\nprint('Batches train=%d, test=%d' % (len(train_iterator), len(test_iterator)))\r\n# define model\r\nmodel = Sequential()\r\nmodel.add(Conv2D(32, (3, 3), activation='relu', input_shape=(width, height, channels)))\r\nmodel.add(MaxPooling2D((2, 2)))\r\nmodel.add(Conv2D(64, (3, 3), activation='relu'))\r\nmodel.add(MaxPooling2D((2, 2)))\r\nmodel.add(Flatten())\r\nmodel.add(Dense(64, activation='relu'))\r\nmodel.add(Dense(10, activation='softmax'))\r\n# compile model\r\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\r\n# fit model with generator\r\nmodel.fit_generator(train_iterator, steps_per_epoch=len(train_iterator), epochs=5)\r\n# evaluate model\r\n_, acc = model.evaluate_generator(test_iterator, steps=len(test_iterator), verbose=0)\r\nprint('Test Accuracy: %.3f' % (acc * 100))<\/pre>\n<p>Running the example configures the ImageDataGenerator class to standardize images, calculates the required statistics on the training set only, then prepares the train and test iterators for fitting and evaluating the model respectively.<\/p>\n<pre class=\"crayon-plain-tag\">Epoch 1\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.1342 - acc: 0.9592\r\nEpoch 2\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.0451 - acc: 0.9859\r\nEpoch 3\/5\r\n938\/938 [==============================] - 12s 13ms\/step - loss: 0.0309 - acc: 0.9906\r\nEpoch 4\/5\r\n938\/938 [==============================] - 13s 13ms\/step - loss: 0.0230 - acc: 0.9924\r\nEpoch 5\/5\r\n938\/938 [==============================] - 13s 14ms\/step - loss: 0.0182 - acc: 0.9941\r\nTest Accuracy: 99.120<\/pre>\n<\/p>\n<h2>Extensions<\/h2>\n<p>This section lists some ideas for extending the tutorial that you may wish to explore.<\/p>\n<ul>\n<li><strong>Color<\/strong>. Update an example to use an image dataset with color images and confirm that scaling is performed across the entire image rather than per-channel.<\/li>\n<li><strong>Sample-Wise<\/strong>. Demonstrate an example of sample-wise centering or standardization of pixel images.<\/li>\n<li><strong>ZCA Whitening<\/strong>. Demonstrate an example of using the ZCA approach to image data preparation.<\/li>\n<\/ul>\n<p>If you explore any of these extensions, I\u2019d love to know.<br \/>\nPost your findings in the comments below.<\/p>\n<h2>Further Reading<\/h2>\n<p>This section provides more resources on the topic if you are looking to go deeper.<\/p>\n<h3>API<\/h3>\n<ul>\n<li><a href=\"https:\/\/keras.io\/datasets\/#mnist-database-of-handwritten-digits\">MNIST database of handwritten digits, Keras API<\/a>.<\/li>\n<li><a href=\"https:\/\/keras.io\/preprocessing\/image\/\">Image Preprocessing Keras API<\/a><\/li>\n<li><a href=\"https:\/\/keras.io\/models\/sequential\/\">Sequential Model Keras API<\/a><\/li>\n<\/ul>\n<h3>Articles<\/h3>\n<ul>\n<li><a href=\"https:\/\/en.wikipedia.org\/wiki\/MNIST_database\">MNIST database, Wikipedia<\/a>.<\/li>\n<li><a href=\"https:\/\/en.wikipedia.org\/wiki\/68%E2%80%9395%E2%80%9399.7_rule\">68\u201395\u201399.7 rule, Wikipedia<\/a>.<\/li>\n<\/ul>\n<h2>Summary<\/h2>\n<p>In this tutorial, you discovered how to use the ImageDataGenerator class to scale pixel data just-in-time when fitting and evaluating deep learning neural network models.<\/p>\n<p>Specifically, you learned:<\/p>\n<ul>\n<li>How to configure and a use the ImageDataGenerator class for train, validation, and test datasets of images.<\/li>\n<li>How to use the ImageDataGenerator to normalize pixel values when fitting and evaluating a convolutional neural network model.<\/li>\n<li>How to use the ImageDataGenerator to center and standardize pixel values when fitting and evaluating a convolutional neural network model.<\/li>\n<\/ul>\n<p>Do you have any questions?<br \/>\nAsk your questions in the comments below and I will do my best to answer.<\/p>\n<p>The post <a rel=\"nofollow\" href=\"https:\/\/machinelearningmastery.com\/how-to-normalize-center-and-standardize-images-with-the-imagedatagenerator-in-keras\/\">How to Normalize, Center, and Standardize Images With the ImageDataGenerator in Keras<\/a> appeared first on <a rel=\"nofollow\" href=\"https:\/\/machinelearningmastery.com\/\">Machine Learning Mastery<\/a>.<\/p>\n<\/div>\n<p><a href=\"https:\/\/machinelearningmastery.com\/how-to-normalize-center-and-standardize-images-with-the-imagedatagenerator-in-keras\/\">Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Author: Jason Brownlee The pixel values in images must be scaled prior to providing the images as input to a deep learning neural network model [&hellip;] <span class=\"read-more-link\"><a class=\"read-more\" href=\"https:\/\/www.aiproblog.com\/index.php\/2019\/04\/02\/how-to-normalize-center-and-standardize-images-with-the-imagedatagenerator-in-keras\/\">Read More<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":1962,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[24],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/1961"}],"collection":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=1961"}],"version-history":[{"count":0,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/1961\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media\/1962"}],"wp:attachment":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=1961"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=1961"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=1961"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}