{"id":2237,"date":"2019-06-06T19:00:22","date_gmt":"2019-06-06T19:00:22","guid":{"rendered":"https:\/\/www.aiproblog.com\/index.php\/2019\/06\/06\/how-to-develop-a-face-recognition-system-using-facenet-in-keras\/"},"modified":"2019-06-06T19:00:22","modified_gmt":"2019-06-06T19:00:22","slug":"how-to-develop-a-face-recognition-system-using-facenet-in-keras","status":"publish","type":"post","link":"https:\/\/www.aiproblog.com\/index.php\/2019\/06\/06\/how-to-develop-a-face-recognition-system-using-facenet-in-keras\/","title":{"rendered":"How to Develop a Face Recognition System Using FaceNet in Keras"},"content":{"rendered":"<p>Author: Jason Brownlee<\/p>\n<div>\n<p>Face recognition is a computer vision task of identifying and verifying a person based on a photograph of their face.<\/p>\n<p>FaceNet is a face recognition system developed in 2015 by researchers at Google that achieved then state-of-the-art results on a range of face recognition benchmark datasets. The FaceNet system can be used broadly thanks to multiple third-party open source implementations of the model and the availability of pre-trained models.<\/p>\n<p>The FaceNet system can be used to extract high-quality features from faces, called face embeddings, that can then be used to train a face identification system.<\/p>\n<p>In this tutorial, you will discover how to develop a face detection system using FaceNet and an SVM classifier to identify people from photographs.<\/p>\n<p>After completing this tutorial, you will know:<\/p>\n<ul>\n<li>About the FaceNet face recognition system developed by Google and open source implementations and pre-trained models.<\/li>\n<li>How to prepare a face detection dataset including first extracting faces via a face detection system and then extracting face features via face embeddings.<\/li>\n<li>How to fit, evaluate, and demonstrate an SVM model to predict identities from faces embeddings.<\/li>\n<\/ul>\n<p>Let\u2019s get started.<\/p>\n<div id=\"attachment_7827\" style=\"width: 650px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7827\" class=\"size-full wp-image-7827\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/06\/How-to-Develop-a-Face-Recognition-System-Using-FaceNet-in-Keras-and-an-SVM-Classifier.jpg\" alt=\"How to Develop a Face Recognition System Using FaceNet in Keras and an SVM Classifier\" width=\"640\" height=\"427\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/06\/How-to-Develop-a-Face-Recognition-System-Using-FaceNet-in-Keras-and-an-SVM-Classifier.jpg 640w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/06\/How-to-Develop-a-Face-Recognition-System-Using-FaceNet-in-Keras-and-an-SVM-Classifier-300x200.jpg 300w\" sizes=\"(max-width: 640px) 100vw, 640px\"><\/p>\n<p id=\"caption-attachment-7827\" class=\"wp-caption-text\">How to Develop a Face Recognition System Using FaceNet in Keras and an SVM Classifier<br \/>Photo by <a href=\"https:\/\/www.flickr.com\/photos\/petervalverde\/45786176624\/\">Peter Valverde<\/a>, some rights reserved.<\/p>\n<\/div>\n<h2>Tutorial Overview<\/h2>\n<p>This tutorial is divided into five parts; they are:<\/p>\n<ol>\n<li>Face Recognition<\/li>\n<li>FaceNet Model<\/li>\n<li>How to Load a FaceNet Model in Keras<\/li>\n<li>How to Detect Faces for Face Recognition<\/li>\n<li>How to Develop a Face Classification System<\/li>\n<\/ol>\n<h2>Face Recognition<\/h2>\n<p>Face recognition is the general task of identifying and verifying people from photographs of their face.<\/p>\n<p>The 2011 book on face recognition titled \u201c<a href=\"https:\/\/amzn.to\/2EuR8Oo\">Handbook of Face Recognition<\/a>\u201d describes two main modes for face recognition, as:<\/p>\n<ul>\n<li><strong>Face Verification<\/strong>. A one-to-one mapping of a given face against a known identity (e.g. <em>is this the person?<\/em>).<\/li>\n<li><strong>Face Identification<\/strong>. A one-to-many mapping for a given face against a database of known faces (e.g. <em>who is this person?<\/em>).<\/li>\n<\/ul>\n<blockquote>\n<p>A face recognition system is expected to identify faces present in images and videos automatically. It can operate in either or both of two modes: (1) face verification (or authentication), and (2) face identification (or recognition).<\/p>\n<\/blockquote>\n<p>\u2014 Page 1, <a href=\"https:\/\/amzn.to\/2EuR8Oo\">Handbook of Face Recognition<\/a>. 2011.<\/p>\n<p>We will focus on the face identification task in this tutorial.<\/p>\n<div class=\"woo-sc-hr\"><\/div>\n<p><center><\/p>\n<h3>Want Results with Deep Learning for Computer Vision?<\/h3>\n<p>Take my free 7-day email crash course now (with sample code).<\/p>\n<p>Click to sign-up and also get a free PDF Ebook version of the course.<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.lpages.co\/leadbox\/1458ca1e0972a2%3A164f8be4f346dc\/4715926590455808\/\" target=\"_blank\" style=\"background: rgb(255, 206, 10); color: rgb(255, 255, 255); text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-weight: bold; font-size: 16px; line-height: 20px; padding: 10px; display: inline-block; max-width: 300px; border-radius: 5px; text-shadow: rgba(0, 0, 0, 0.25) 0px -1px 1px; box-shadow: rgba(255, 255, 255, 0.5) 0px 1px 3px inset, rgba(0, 0, 0, 0.5) 0px 1px 3px;\" rel=\"noopener noreferrer\">Download Your FREE Mini-Course<\/a><script data-leadbox=\"1458ca1e0972a2:164f8be4f346dc\" data-url=\"https:\/\/machinelearningmastery.lpages.co\/leadbox\/1458ca1e0972a2%3A164f8be4f346dc\/4715926590455808\/\" data-config=\"%7B%7D\" type=\"text\/javascript\" src=\"https:\/\/machinelearningmastery.lpages.co\/leadbox-1553357564.js\"><\/script><\/p>\n<p><\/center><\/p>\n<div class=\"woo-sc-hr\"><\/div>\n<h2>FaceNet Model<\/h2>\n<p>FaceNet is a face recognition system that was described by <a href=\"http:\/\/www.florian-schroff.de\/\">Florian Schroff<\/a>, et al. at Google in their 2015 paper titled \u201c<a href=\"https:\/\/arxiv.org\/abs\/1503.03832\">FaceNet: A Unified Embedding for Face Recognition and Clustering<\/a>.\u201d<\/p>\n<p>It is a system that, given a picture of a face, will extract high-quality features from the face and predict a 128 element vector representation these features, called a face embedding.<\/p>\n<blockquote>\n<p>FaceNet, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity.<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/arxiv.org\/abs\/1503.03832\">FaceNet: A Unified Embedding for Face Recognition and Clustering<\/a>, 2015.<\/p>\n<p>The model is a deep convolutional neural network trained via a triplet loss function that encourages vectors for the same identity to become more similar (smaller distance), whereas vectors for different identities are expected to become less similar (larger distance). The focus on training a model to create embeddings directly (rather than extracting them from an intermediate layer of a model) was an important innovation in this work.<\/p>\n<blockquote>\n<p>Our method uses a deep convolutional network trained to directly optimize the embedding itself, rather than an intermediate bottleneck layer as in previous deep learning approaches.<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/arxiv.org\/abs\/1503.03832\">FaceNet: A Unified Embedding for Face Recognition and Clustering<\/a>, 2015.<\/p>\n<p>These face embeddings were then used as the basis for training classifier systems on standard face recognition benchmark datasets, achieving then-state-of-the-art results.<\/p>\n<blockquote>\n<p>Our system cuts the error rate in comparison to the best published result by 30% \u2026<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/arxiv.org\/abs\/1503.03832\">FaceNet: A Unified Embedding for Face Recognition and Clustering<\/a>, 2015.<\/p>\n<p>The paper also explores other uses of the embeddings, such as clustering to group like-faces based on their extracted features.<\/p>\n<p>It is a robust and effective face recognition system, and the general nature of the extracted face embeddings lends the approach to a range of applications.<\/p>\n<h2>How to Load a FaceNet Model in Keras<\/h2>\n<p>There are a number of projects that provide tools to train FaceNet-based models and make use of pre-trained models.<\/p>\n<p>Perhaps the most prominent is called <a href=\"http:\/\/cmusatyalab.github.io\/openface\/\">OpenFace<\/a> that provides FaceNet models built and trained using the PyTorch deep learning framework. There is a port of OpenFace to Keras, called <a href=\"https:\/\/github.com\/iwantooxxoox\/Keras-OpenFace\">Keras OpenFace<\/a>, but at the time of writing, the models appear to require Python 2, which is quite limiting.<\/p>\n<p>Another prominent project is called <a href=\"https:\/\/github.com\/davidsandberg\/facenet\">FaceNet by David Sandberg<\/a> that provides FaceNet models built and trained using TensorFlow. The project looks mature, although at the time of writing does not provide a library-based installation nor clean API. Usefully, David\u2019s project provides a number of high-performing pre-trained FaceNet models and there are a number of projects that port or convert these models for use in Keras.<\/p>\n<p>A notable example is Keras <a href=\"https:\/\/github.com\/nyoki-mtl\/keras-facenet\">FaceNet by Hiroki Taniai<\/a>. His project provides a script for converting the Inception ResNet v1 model from TensorFlow to Keras. He also provides a pre-trained Keras model ready for use.<\/p>\n<p>We will use the pre-trained Keras FaceNet model provided by <a href=\"https:\/\/github.com\/nyoki-mtl\">Hiroki Taniai<\/a> in this tutorial. It was trained on <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/ms-celeb-1m-challenge-recognizing-one-million-celebrities-real-world\/\">MS-Celeb-1M dataset<\/a> and expects input images to be color, to have their pixel values whitened (standardized across all three channels), and to have a square shape of 160\u00d7160 pixels.<\/p>\n<p>The model can be downloaded from here:<\/p>\n<ul>\n<li><a href=\"https:\/\/drive.google.com\/open?id=1pwQ3H4aJ8a6yyJHZkTwtjcL4wYWQb7bn\">Keras FaceNet Pre-Trained Model (88 megabytes)<\/a><\/li>\n<\/ul>\n<p>Download the model file and place it in your current working directory with the filename \u2018<em>facenet_keras.h5<\/em>\u2018.<\/p>\n<p>We can load the model directly in Keras using the <em>load_model()<\/em> function; for example:<\/p>\n<pre class=\"crayon-plain-tag\"># example of loading the keras facenet model\r\nfrom keras.models import load_model\r\n# load the model\r\nmodel = load_model('facenet_keras.h5')\r\n# summarize input and output shape\r\nprint(model.inputs)\r\nprint(model.outputs)<\/pre>\n<p>Running the example loads the model and prints the shape of the input and output tensors.<\/p>\n<p>We can see that the model indeed expects square color images as input with the shape 160\u00d7160, and will output a face embedding as a 128 element vector.<\/p>\n<pre class=\"crayon-plain-tag\"># [<tf.Tensor 'input_1:0' shape=(?, 160, 160, 3) dtype=float32>]\r\n# [<tf.Tensor 'Bottleneck_BatchNorm\/cond\/Merge:0' shape=(?, 128) dtype=float32>]<\/pre>\n<p>Now that we have a FaceNet model, we can explore using it.<\/p>\n<h2>How to Detect Faces for Face Recognition<\/h2>\n<p>Before we can perform face recognition, we need to detect faces.<\/p>\n<p>Face detection is the process of automatically locating faces in a photograph and localizing them by drawing a bounding box around their extent.<\/p>\n<p>In this tutorial, we will also use the Multi-Task Cascaded Convolutional Neural Network, or MTCNN, for face detection, e.g. finding and extracting faces from photos. This is a state-of-the-art deep learning model for face detection, described in the 2016 paper titled \u201c<a href=\"https:\/\/arxiv.org\/abs\/1604.02878\">Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks<\/a>.\u201d<\/p>\n<p>We will use the implementation provided by <a href=\"https:\/\/www.linkedin.com\/in\/ivandepazcenteno\/\">Iv\u00e1n de Paz Centeno<\/a> in the <a href=\"https:\/\/github.com\/ipazc\/mtcnn\">ipazc\/mtcnn project<\/a>. This can also be installed via pip as follows:<\/p>\n<pre class=\"crayon-plain-tag\">sudo pip install mtcnn<\/pre>\n<p>We can confirm that the library was installed correctly by importing the library and printing the version; for example:<\/p>\n<pre class=\"crayon-plain-tag\"># confirm mtcnn was installed correctly\r\nimport mtcnn\r\n# print version\r\nprint(mtcnn.__version__)<\/pre>\n<p>Running the example prints the current version of the library.<\/p>\n<pre class=\"crayon-plain-tag\">0.0.8<\/pre>\n<p>We can use the mtcnn library to create a face detector and extract faces for our use with the FaceNet face detector models in subsequent sections.<\/p>\n<p>The first step is to load an image as a NumPy array, which we can achieve using the PIL library and the <em>open()<\/em> function. We will also convert the image to RGB, just in case the image has an alpha channel or is black and white.<\/p>\n<pre class=\"crayon-plain-tag\"># load image from file\r\nimage = Image.open(filename)\r\n# convert to RGB, if needed\r\nimage = image.convert('RGB')\r\n# convert to array\r\npixels = asarray(image)<\/pre>\n<p>Next, we can create an MTCNN face detector class and use it to detect all faces in the loaded photograph.<\/p>\n<pre class=\"crayon-plain-tag\"># create the detector, using default weights\r\ndetector = MTCNN()\r\n# detect faces in the image\r\nresults = detector.detect_faces(pixels)<\/pre>\n<p>The result is a list of bounding boxes, where each bounding box defines a lower-left-corner of the bounding box, as well as the width and height.<\/p>\n<p>If we assume there is only one face in the photo for our experiments, we can determine the pixel coordinates of the bounding box as follows. Sometimes the library will return a negative pixel index, and I think this is a bug. We can fix this by taking the absolute value of the coordinates.<\/p>\n<pre class=\"crayon-plain-tag\"># extract the bounding box from the first face\r\nx1, y1, width, height = results[0]['box']\r\n# bug fix\r\nx1, y1 = abs(x1), abs(y1)\r\nx2, y2 = x1 + width, y1 + height<\/pre>\n<p>We can use these coordinates to extract the face.<\/p>\n<pre class=\"crayon-plain-tag\"># extract the face\r\nface = pixels[y1:y2, x1:x2]<\/pre>\n<p>We can then use the PIL library to resize this small image of the face to the required size; specifically, the model expects square input faces with the shape 160\u00d7160.<\/p>\n<pre class=\"crayon-plain-tag\"># resize pixels to the model size\r\nimage = Image.fromarray(face)\r\nimage = image.resize((160, 160))\r\nface_array = asarray(image)<\/pre>\n<p>Tying all of this together, the function <em>extract_face()<\/em> will load a photograph from the loaded filename and return the extracted face. It assumes that the photo contains one face and will return the first face detected.<\/p>\n<pre class=\"crayon-plain-tag\"># function for face detection with mtcnn\r\nfrom PIL import Image\r\nfrom numpy import asarray\r\nfrom mtcnn.mtcnn import MTCNN\r\n\r\n# extract a single face from a given photograph\r\ndef extract_face(filename, required_size=(160, 160)):\r\n\t# load image from file\r\n\timage = Image.open(filename)\r\n\t# convert to RGB, if needed\r\n\timage = image.convert('RGB')\r\n\t# convert to array\r\n\tpixels = asarray(image)\r\n\t# create the detector, using default weights\r\n\tdetector = MTCNN()\r\n\t# detect faces in the image\r\n\tresults = detector.detect_faces(pixels)\r\n\t# extract the bounding box from the first face\r\n\tx1, y1, width, height = results[0]['box']\r\n\t# bug fix\r\n\tx1, y1 = abs(x1), abs(y1)\r\n\tx2, y2 = x1 + width, y1 + height\r\n\t# extract the face\r\n\tface = pixels[y1:y2, x1:x2]\r\n\t# resize pixels to the model size\r\n\timage = Image.fromarray(face)\r\n\timage = image.resize(required_size)\r\n\tface_array = asarray(image)\r\n\treturn face_array\r\n\r\n# load the photo and extract the face\r\npixels = extract_face('...')<\/pre>\n<p>We can use this function to extract faces as needed in the next section that can be provided as input to the FaceNet model.<\/p>\n<h2>How to Develop a Face Classification System<\/h2>\n<p>In this section, we will develop a face detection system to predict the identity of a given face.<\/p>\n<p>The model will be trained and tested using the \u2018<em>5 Celebrity Faces Dataset<\/em>\u2018 that contains many photographs of five different celebrities.<\/p>\n<p>We will use an MTCNN model for face detection, the FaceNet model will be used to create a face embedding for each detected face, then we will develop a <a href=\"https:\/\/machinelearningmastery.com\/support-vector-machines-for-machine-learning\/\">Linear Support Vector Machine (SVM)<\/a> classifier model to predict the identity of a given face.<\/p>\n<h3>5 Celebrity Faces Dataset<\/h3>\n<p>The 5 Celebrity Faces Dataset is a small dataset that contains photographs of celebrities.<\/p>\n<p>It includes photos of: <a href=\"https:\/\/en.wikipedia.org\/wiki\/Ben_Affleck\">Ben Affleck<\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/Elton_John\">Elton John<\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/Jerry_Seinfeld\">Jerry Seinfeld<\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/Madonna_(entertainer)\">Madonna<\/a>, and <a href=\"https:\/\/en.wikipedia.org\/wiki\/Mindy_Kaling\">Mindy Kaling<\/a>.<\/p>\n<p>The dataset was prepared and made available by <a href=\"https:\/\/www.linkedin.com\/in\/dansbecker\/\">Dan Becker<\/a> and provided for <a href=\"https:\/\/www.kaggle.com\/dansbecker\/5-celebrity-faces-dataset\">free download on Kaggle<\/a>. Note, a Kaggle account is required to download the dataset.<\/p>\n<ul>\n<li><a href=\"https:\/\/www.kaggle.com\/dansbecker\/5-celebrity-faces-dataset\">5 Celebrity Faces Dataset, Kaggle<\/a>.<\/li>\n<\/ul>\n<p>Download the dataset (this may require a Kaggle login), data.zip (2.5 megabytes), and unzip it in your local directory with the folder name \u2018<em>5-celebrity-faces-dataset<\/em>\u2018.<\/p>\n<p>You should now have a directory with the following structure (note, there are spelling mistakes in some directory names, and they were left as-is in this example):<\/p>\n<pre class=\"crayon-plain-tag\">5-celebrity-faces-dataset\r\n\u251c\u2500\u2500 train\r\n\u2502   \u251c\u2500\u2500 ben_afflek\r\n\u2502   \u251c\u2500\u2500 elton_john\r\n\u2502   \u251c\u2500\u2500 jerry_seinfeld\r\n\u2502   \u251c\u2500\u2500 madonna\r\n\u2502   \u2514\u2500\u2500 mindy_kaling\r\n\u2514\u2500\u2500 val\r\n    \u251c\u2500\u2500 ben_afflek\r\n    \u251c\u2500\u2500 elton_john\r\n    \u251c\u2500\u2500 jerry_seinfeld\r\n    \u251c\u2500\u2500 madonna\r\n    \u2514\u2500\u2500 mindy_kaling<\/pre>\n<p>We can see that there is a training dataset and a validation or test dataset.<\/p>\n<p>Looking at some of the photos in the directories, we can see that the photos provide faces with a range of orientations, lighting, and in various sizes. Importantly, each photo contains one face of the person.<\/p>\n<p>We will use this dataset as the basis for our classifier, trained on the \u2018<em>train<\/em>\u2018 dataset only and classify faces in the \u2018<em>val<\/em>\u2018 dataset. You can use this same structure to develop a classifier with your own photographs.<\/p>\n<h3>Detect Faces<\/h3>\n<p>The first step is to detect the face in each photograph and reduce the dataset to a series of faces only.<\/p>\n<p>Let\u2019s test out our face detector function defined in the previous section, specifically <em>extract_face()<\/em>.<\/p>\n<p>Looking in the \u2018<em>5-celebrity-faces-dataset\/train\/ben_afflek\/<\/em>\u2018 directory, we can see that there are 14 photographs of Ben Affleck in the training dataset. We can detect the face in each photograph, and create a plot with 14 faces, with two rows of seven images each.<\/p>\n<p>The complete example is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># demonstrate face detection on 5 Celebrity Faces Dataset\r\nfrom os import listdir\r\nfrom PIL import Image\r\nfrom numpy import asarray\r\nfrom matplotlib import pyplot\r\nfrom mtcnn.mtcnn import MTCNN\r\n\r\n# extract a single face from a given photograph\r\ndef extract_face(filename, required_size=(160, 160)):\r\n\t# load image from file\r\n\timage = Image.open(filename)\r\n\t# convert to RGB, if needed\r\n\timage = image.convert('RGB')\r\n\t# convert to array\r\n\tpixels = asarray(image)\r\n\t# create the detector, using default weights\r\n\tdetector = MTCNN()\r\n\t# detect faces in the image\r\n\tresults = detector.detect_faces(pixels)\r\n\t# extract the bounding box from the first face\r\n\tx1, y1, width, height = results[0]['box']\r\n\t# bug fix\r\n\tx1, y1 = abs(x1), abs(y1)\r\n\tx2, y2 = x1 + width, y1 + height\r\n\t# extract the face\r\n\tface = pixels[y1:y2, x1:x2]\r\n\t# resize pixels to the model size\r\n\timage = Image.fromarray(face)\r\n\timage = image.resize(required_size)\r\n\tface_array = asarray(image)\r\n\treturn face_array\r\n\r\n# specify folder to plot\r\nfolder = '5-celebrity-faces-dataset\/train\/ben_afflek\/'\r\ni = 1\r\n# enumerate files\r\nfor filename in listdir(folder):\r\n\t# path\r\n\tpath = folder + filename\r\n\t# get face\r\n\tface = extract_face(path)\r\n\tprint(i, face.shape)\r\n\t# plot\r\n\tpyplot.subplot(2, 7, i)\r\n\tpyplot.axis('off')\r\n\tpyplot.imshow(face)\r\n\ti += 1\r\npyplot.show()<\/pre>\n<p>Running the example takes a moment and reports the progress of each loaded photograph along the way and the shape of the NumPy array containing the face pixel data.<\/p>\n<pre class=\"crayon-plain-tag\">1 (160, 160, 3)\r\n2 (160, 160, 3)\r\n3 (160, 160, 3)\r\n4 (160, 160, 3)\r\n5 (160, 160, 3)\r\n6 (160, 160, 3)\r\n7 (160, 160, 3)\r\n8 (160, 160, 3)\r\n9 (160, 160, 3)\r\n10 (160, 160, 3)\r\n11 (160, 160, 3)\r\n12 (160, 160, 3)\r\n13 (160, 160, 3)\r\n14 (160, 160, 3)<\/pre>\n<p>A figure is created containing the faces detected in the Ben Affleck directory.<\/p>\n<p>We can see that each face was correctly detected and that we have a range of lighting, skin tones, and orientations in the detected faces.<\/p>\n<div id=\"attachment_7825\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7825\" class=\"size-large wp-image-7825\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Plot-of-14-Faces-of-Ben-Afflek-Detected-from-the-Training-Dataset-of-the-5-Celebrity-Faces-Dataset-1024x768.png\" alt=\"Plot of 14 Faces of Ben Affleck Detected From the Training Dataset of the 5 Celebrity Faces Dataset\" width=\"1024\" height=\"768\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Plot-of-14-Faces-of-Ben-Afflek-Detected-from-the-Training-Dataset-of-the-5-Celebrity-Faces-Dataset-1024x768.png 1024w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Plot-of-14-Faces-of-Ben-Afflek-Detected-from-the-Training-Dataset-of-the-5-Celebrity-Faces-Dataset-300x225.png 300w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Plot-of-14-Faces-of-Ben-Afflek-Detected-from-the-Training-Dataset-of-the-5-Celebrity-Faces-Dataset-768x576.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Plot-of-14-Faces-of-Ben-Afflek-Detected-from-the-Training-Dataset-of-the-5-Celebrity-Faces-Dataset.png 1280w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/p>\n<p id=\"caption-attachment-7825\" class=\"wp-caption-text\">Plot of 14 Faces of Ben Affleck Detected From the Training Dataset of the 5 Celebrity Faces Dataset<\/p>\n<\/div>\n<p>So far, so good.<\/p>\n<p>Next, we can extend this example to step over each subdirectory for a given dataset (e.g. \u2018<em>train<\/em>\u2018 or \u2018<em>val<\/em>\u2018), extract the faces, and prepare a dataset with the name as the output label for each detected face.<\/p>\n<p>The <em>load_faces()<\/em> function below will load all of the faces into a list for a given directory, e.g. \u2018<em>5-celebrity-faces-dataset\/train\/ben_afflek\/<\/em>\u2018.<\/p>\n<pre class=\"crayon-plain-tag\"># load images and extract faces for all images in a directory\r\ndef load_faces(directory):\r\n\tfaces = list()\r\n\t# enumerate files\r\n\tfor filename in listdir(directory):\r\n\t\t# path\r\n\t\tpath = directory + filename\r\n\t\t# get face\r\n\t\tface = extract_face(path)\r\n\t\t# store\r\n\t\tfaces.append(face)\r\n\treturn faces<\/pre>\n<p>We can call the <em>load_faces()<\/em> function for each subdirectory in the \u2018<em>train<\/em>\u2018 or \u2018<em>val<\/em>\u2018 folders. Each face has one label, the name of the celebrity, which we can take from the directory name.<\/p>\n<p>The <em>load_dataset()<\/em> function below takes a directory name such as \u2018<em>5-celebrity-faces-dataset\/train\/<\/em>\u2018 and detects faces for each subdirectory (celebrity), assigning labels to each detected face.<\/p>\n<p>It returns the <em>X<\/em> and <em>y<\/em> elements of the dataset as NumPy arrays.<\/p>\n<pre class=\"crayon-plain-tag\"># load a dataset that contains one subdir for each class that in turn contains images\r\ndef load_dataset(directory):\r\n\tX, y = list(), list()\r\n\t# enumerate folders, on per class\r\n\tfor subdir in listdir(directory):\r\n\t\t# path\r\n\t\tpath = directory + subdir + '\/'\r\n\t\t# skip any files that might be in the dir\r\n\t\tif not isdir(path):\r\n\t\t\tcontinue\r\n\t\t# load all faces in the subdirectory\r\n\t\tfaces = load_faces(path)\r\n\t\t# create labels\r\n\t\tlabels = [subdir for _ in range(len(faces))]\r\n\t\t# summarize progress\r\n\t\tprint('>loaded %d examples for class: %s' % (len(faces), subdir))\r\n\t\t# store\r\n\t\tX.extend(faces)\r\n\t\ty.extend(labels)\r\n\treturn asarray(X), asarray(y)<\/pre>\n<p>We can then call this function for the \u2018train\u2019 and \u2018val\u2019 folders to load all of the data, then save the results in a single compressed NumPy array file via the <a href=\"https:\/\/docs.scipy.org\/doc\/numpy\/reference\/generated\/numpy.savez_compressed.html\">savez_compressed() function<\/a>.<\/p>\n<pre class=\"crayon-plain-tag\"># load train dataset\r\ntrainX, trainy = load_dataset('5-celebrity-faces-dataset\/train\/')\r\nprint(trainX.shape, trainy.shape)\r\n# load test dataset\r\ntestX, testy = load_dataset('5-celebrity-faces-dataset\/val\/')\r\nprint(testX.shape, testy.shape)\r\n# save arrays to one file in compressed format\r\nsavez_compressed('5-celebrity-faces-dataset.npz', trainX, trainy, testX, testy)<\/pre>\n<p>Tying all of this together, the complete example of detecting all of the faces in the 5 Celebrity Faces Dataset is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># face detection for the 5 Celebrity Faces Dataset\r\nfrom os import listdir\r\nfrom os.path import isdir\r\nfrom PIL import Image\r\nfrom matplotlib import pyplot\r\nfrom numpy import savez_compressed\r\nfrom numpy import asarray\r\nfrom mtcnn.mtcnn import MTCNN\r\n\r\n# extract a single face from a given photograph\r\ndef extract_face(filename, required_size=(160, 160)):\r\n\t# load image from file\r\n\timage = Image.open(filename)\r\n\t# convert to RGB, if needed\r\n\timage = image.convert('RGB')\r\n\t# convert to array\r\n\tpixels = asarray(image)\r\n\t# create the detector, using default weights\r\n\tdetector = MTCNN()\r\n\t# detect faces in the image\r\n\tresults = detector.detect_faces(pixels)\r\n\t# extract the bounding box from the first face\r\n\tx1, y1, width, height = results[0]['box']\r\n\t# bug fix\r\n\tx1, y1 = abs(x1), abs(y1)\r\n\tx2, y2 = x1 + width, y1 + height\r\n\t# extract the face\r\n\tface = pixels[y1:y2, x1:x2]\r\n\t# resize pixels to the model size\r\n\timage = Image.fromarray(face)\r\n\timage = image.resize(required_size)\r\n\tface_array = asarray(image)\r\n\treturn face_array\r\n\r\n# load images and extract faces for all images in a directory\r\ndef load_faces(directory):\r\n\tfaces = list()\r\n\t# enumerate files\r\n\tfor filename in listdir(directory):\r\n\t\t# path\r\n\t\tpath = directory + filename\r\n\t\t# get face\r\n\t\tface = extract_face(path)\r\n\t\t# store\r\n\t\tfaces.append(face)\r\n\treturn faces\r\n\r\n# load a dataset that contains one subdir for each class that in turn contains images\r\ndef load_dataset(directory):\r\n\tX, y = list(), list()\r\n\t# enumerate folders, on per class\r\n\tfor subdir in listdir(directory):\r\n\t\t# path\r\n\t\tpath = directory + subdir + '\/'\r\n\t\t# skip any files that might be in the dir\r\n\t\tif not isdir(path):\r\n\t\t\tcontinue\r\n\t\t# load all faces in the subdirectory\r\n\t\tfaces = load_faces(path)\r\n\t\t# create labels\r\n\t\tlabels = [subdir for _ in range(len(faces))]\r\n\t\t# summarize progress\r\n\t\tprint('>loaded %d examples for class: %s' % (len(faces), subdir))\r\n\t\t# store\r\n\t\tX.extend(faces)\r\n\t\ty.extend(labels)\r\n\treturn asarray(X), asarray(y)\r\n\r\n# load train dataset\r\ntrainX, trainy = load_dataset('5-celebrity-faces-dataset\/train\/')\r\nprint(trainX.shape, trainy.shape)\r\n# load test dataset\r\ntestX, testy = load_dataset('5-celebrity-faces-dataset\/val\/')\r\n# save arrays to one file in compressed format\r\nsavez_compressed('5-celebrity-faces-dataset.npz', trainX, trainy, testX, testy)<\/pre>\n<p>Running the example may take a moment.<\/p>\n<p>First, all of the photos in the \u2018<em>train<\/em>\u2018 dataset are loaded, then faces are extracted, resulting in 93 samples with square face input and a class label string as output. Then the \u2018<em>val<\/em>\u2018 dataset is loaded, providing 25 samples that can be used as a test dataset.<\/p>\n<p>Both datasets are then saved to a compressed NumPy array file called \u2018<em>5-celebrity-faces-dataset.npz<\/em>\u2018 that is about three megabytes and is stored in the current working directory.<\/p>\n<pre class=\"crayon-plain-tag\">>loaded 14 examples for class: ben_afflek\r\n>loaded 19 examples for class: madonna\r\n>loaded 17 examples for class: elton_john\r\n>loaded 22 examples for class: mindy_kaling\r\n>loaded 21 examples for class: jerry_seinfeld\r\n(93, 160, 160, 3) (93,)\r\n>loaded 5 examples for class: ben_afflek\r\n>loaded 5 examples for class: madonna\r\n>loaded 5 examples for class: elton_john\r\n>loaded 5 examples for class: mindy_kaling\r\n>loaded 5 examples for class: jerry_seinfeld\r\n(25, 160, 160, 3) (25,)<\/pre>\n<p>This dataset is ready to be provided to a face detection model.<\/p>\n<h3>Create Face Embeddings<\/h3>\n<p>The next step is to create a face embedding.<\/p>\n<p>A face embedding is a vector that represents the features extracted from the face. This can then be compared with the vectors generated for other faces. For example, another vector that is close (by some measure) may be the same person, whereas another vector that is far (by some measure) may be a different person.<\/p>\n<p>The classifier model that we want to develop will take a face embedding as input and predict the identity of the face. The FaceNet model will generate this embedding for a given image of a face.<\/p>\n<p>The FaceNet model can be used as part of the classifier itself, or we can use the FaceNet model to pre-process a face to create a face embedding that can be stored and used as input to our classifier model. This latter approach is preferred as the FaceNet model is both large and slow to create a face embedding.<\/p>\n<p>We can, therefore, pre-compute the face embeddings for all faces in the train and test (formally \u2018<em>val<\/em>\u2018) sets in our 5 Celebrity Faces Dataset.<\/p>\n<p>First, we can load our detected faces dataset using the <a href=\"https:\/\/docs.scipy.org\/doc\/numpy\/reference\/generated\/numpy.load.html\">load() NumPy function<\/a>.<\/p>\n<pre class=\"crayon-plain-tag\"># load the face dataset\r\ndata = load('5-celebrity-faces-dataset.npz')\r\ntrainX, trainy, testX, testy = data['arr_0'], data['arr_1'], data['arr_2'], data['arr_3']\r\nprint('Loaded: ', trainX.shape, trainy.shape, testX.shape, testy.shape)<\/pre>\n<p>Next, we can load our FaceNet model ready for converting faces into face embeddings.<\/p>\n<pre class=\"crayon-plain-tag\"># load the facenet model\r\nmodel = load_model('facenet_keras.h5')\r\nprint('Loaded Model')<\/pre>\n<p>We can then enumerate each face in the train and test datasets to predict an embedding.<\/p>\n<p>To predict an embedding, first the pixel values of the image need to be suitably prepared to meet the expectations of the FaceNet model. This specific implementation of the FaceNet model expects that the pixel values are standardized.<\/p>\n<pre class=\"crayon-plain-tag\"># scale pixel values\r\nface_pixels = face_pixels.astype('float32')\r\n# standardize pixel values across channels (global)\r\nmean, std = face_pixels.mean(), face_pixels.std()\r\nface_pixels = (face_pixels - mean) \/ std<\/pre>\n<p>In order to make a prediction for one example in Keras, we must expand the dimensions so that the face array is one sample.<\/p>\n<pre class=\"crayon-plain-tag\"># transform face into one sample\r\nsamples = expand_dims(face_pixels, axis=0)<\/pre>\n<p>We can then use the model to make a prediction and extract the embedding vector.<\/p>\n<pre class=\"crayon-plain-tag\"># make prediction to get embedding\r\nyhat = model.predict(samples)\r\n# get embedding\r\nembedding = yhat[0]<\/pre>\n<p>The <em>get_embedding()<\/em> function defined below implements these behaviors and will return a face embedding given a single image of a face and the loaded FaceNet model.<\/p>\n<pre class=\"crayon-plain-tag\"># get the face embedding for one face\r\ndef get_embedding(model, face_pixels):\r\n\t# scale pixel values\r\n\tface_pixels = face_pixels.astype('float32')\r\n\t# standardize pixel values across channels (global)\r\n\tmean, std = face_pixels.mean(), face_pixels.std()\r\n\tface_pixels = (face_pixels - mean) \/ std\r\n\t# transform face into one sample\r\n\tsamples = expand_dims(face_pixels, axis=0)\r\n\t# make prediction to get embedding\r\n\tyhat = model.predict(samples)\r\n\treturn yhat[0]<\/pre>\n<p>Tying all of this together, the complete example of converting each face into a face embedding in the train and test datasets is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># calculate a face embedding for each face in the dataset using facenet\r\nfrom numpy import load\r\nfrom numpy import expand_dims\r\nfrom numpy import asarray\r\nfrom numpy import savez_compressed\r\nfrom keras.models import load_model\r\n\r\n# get the face embedding for one face\r\ndef get_embedding(model, face_pixels):\r\n\t# scale pixel values\r\n\tface_pixels = face_pixels.astype('float32')\r\n\t# standardize pixel values across channels (global)\r\n\tmean, std = face_pixels.mean(), face_pixels.std()\r\n\tface_pixels = (face_pixels - mean) \/ std\r\n\t# transform face into one sample\r\n\tsamples = expand_dims(face_pixels, axis=0)\r\n\t# make prediction to get embedding\r\n\tyhat = model.predict(samples)\r\n\treturn yhat[0]\r\n\r\n# load the face dataset\r\ndata = load('5-celebrity-faces-dataset.npz')\r\ntrainX, trainy, testX, testy = data['arr_0'], data['arr_1'], data['arr_2'], data['arr_3']\r\nprint('Loaded: ', trainX.shape, trainy.shape, testX.shape, testy.shape)\r\n# load the facenet model\r\nmodel = load_model('facenet_keras.h5')\r\nprint('Loaded Model')\r\n# convert each face in the train set to an embedding\r\nnewTrainX = list()\r\nfor face_pixels in trainX:\r\n\tembedding = get_embedding(model, face_pixels)\r\n\tnewTrainX.append(embedding)\r\nnewTrainX = asarray(newTrainX)\r\nprint(newTrainX.shape)\r\n# convert each face in the test set to an embedding\r\nnewTestX = list()\r\nfor face_pixels in testX:\r\n\tembedding = get_embedding(model, face_pixels)\r\n\tnewTestX.append(embedding)\r\nnewTestX = asarray(newTestX)\r\nprint(newTestX.shape)\r\n# save arrays to one file in compressed format\r\nsavez_compressed('5-celebrity-faces-embeddings.npz', newTrainX, trainy, newTestX, testy)<\/pre>\n<p>Running the example reports progress along the way.<\/p>\n<p>We can see that the face dataset was loaded correctly and so was the model. The train dataset was then transformed into 93 face embeddings, each comprised of a 128 element vector. The 25 examples in the test dataset were also suitably converted to face embeddings.<\/p>\n<p>The resulting datasets were then saved to a compressed NumPy array that is about 50 kilobytes with the name \u2018<em>5-celebrity-faces-embeddings.npz<\/em>\u2018 in the current working directory.<\/p>\n<pre class=\"crayon-plain-tag\">Loaded:  (93, 160, 160, 3) (93,) (25, 160, 160, 3) (25,)\r\nLoaded Model\r\n(93, 128)\r\n(25, 128)<\/pre>\n<p>We are now ready to develop our face classifier system.<\/p>\n<h3>Perform Face Classification<\/h3>\n<p>In this section, we will develop a model to classify face embeddings as one of the known celebrities in the 5 Celebrity Faces Dataset.<\/p>\n<p>First, we must load the face embeddings dataset.<\/p>\n<pre class=\"crayon-plain-tag\"># load dataset\r\ndata = load('5-celebrity-faces-embeddings.npz')\r\ntrainX, trainy, testX, testy = data['arr_0'], data['arr_1'], data['arr_2'], data['arr_3']\r\nprint('Dataset: train=%d, test=%d' % (trainX.shape[0], testX.shape[0]))<\/pre>\n<p>Next, the data requires some minor preparation prior to modeling.<\/p>\n<p>First, it is a good practice to normalize the face embedding vectors. It is a good practice because the vectors are often compared to each other using a distance metric.<\/p>\n<p>In this context, <a href=\"https:\/\/machinelearningmastery.com\/vector-norms-machine-learning\/\">vector normalization<\/a> means scaling the values until the length or magnitude of the vectors is 1 or unit length. This can be achieved using the <a href=\"https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.preprocessing.Normalizer.html\">Normalizer class in scikit-learn<\/a>. It might even be more convenient to perform this step when the face embeddings are created in the previous step.<\/p>\n<pre class=\"crayon-plain-tag\"># normalize input vectors\r\nin_encoder = Normalizer(norm='l2')\r\ntrainX = in_encoder.transform(trainX)\r\ntestX = in_encoder.transform(testX)<\/pre>\n<p>Next, the string target variables for each celebrity name need to be converted to integers.<\/p>\n<p>This can be achieved via the <a href=\"https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.preprocessing.LabelEncoder.html\">LabelEncoder class in scikit-learn<\/a>.<\/p>\n<pre class=\"crayon-plain-tag\"># label encode targets\r\nout_encoder = LabelEncoder()\r\nout_encoder.fit(trainy)\r\ntrainy = out_encoder.transform(trainy)\r\ntesty = out_encoder.transform(testy)<\/pre>\n<p>Next, we can fit a model.<\/p>\n<p>It is common to use a <a href=\"https:\/\/machinelearningmastery.com\/support-vector-machines-for-machine-learning\/\">Linear Support Vector Machine (SVM)<\/a> when working with normalized face embedding inputs. This is because the method is very effective at separating the face embedding vectors. We can fit a linear SVM to the training data using the <a href=\"https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.svm.SVC.html\">SVC class in scikit-learn<\/a> and setting the \u2018<em>kernel<\/em>\u2018 attribute to \u2018<em>linear<\/em>\u2018. We may also want probabilities later when making predictions, which can be configured by setting \u2018<em>probability<\/em>\u2018 to \u2018<em>True<\/em>\u2018.<\/p>\n<pre class=\"crayon-plain-tag\"># fit model\r\nmodel = SVC(kernel='linear')\r\nmodel.fit(trainX, trainy)<\/pre>\n<p>Next, we can evaluate the model.<\/p>\n<p>This can be achieved by using the fit model to make a prediction for each example in the train and test datasets and then calculating the classification accuracy.<\/p>\n<pre class=\"crayon-plain-tag\"># predict\r\nyhat_train = model.predict(trainX)\r\nyhat_test = model.predict(testX)\r\n# score\r\nscore_train = accuracy_score(trainy, yhat_train)\r\nscore_test = accuracy_score(testy, yhat_test)\r\n# summarize\r\nprint('Accuracy: train=%.3f, test=%.3f' % (score_train*100, score_test*100))<\/pre>\n<p>Tying all of this together, the complete example of fitting a Linear SVM on the face embeddings for the 5 Celebrity Faces Dataset is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># develop a classifier for the 5 Celebrity Faces Dataset\r\nfrom numpy import load\r\nfrom sklearn.metrics import accuracy_score\r\nfrom sklearn.preprocessing import LabelEncoder\r\nfrom sklearn.preprocessing import Normalizer\r\nfrom sklearn.svm import SVC\r\n# load dataset\r\ndata = load('5-celebrity-faces-embeddings.npz')\r\ntrainX, trainy, testX, testy = data['arr_0'], data['arr_1'], data['arr_2'], data['arr_3']\r\nprint('Dataset: train=%d, test=%d' % (trainX.shape[0], testX.shape[0]))\r\n# normalize input vectors\r\nin_encoder = Normalizer(norm='l2')\r\ntrainX = in_encoder.transform(trainX)\r\ntestX = in_encoder.transform(testX)\r\n# label encode targets\r\nout_encoder = LabelEncoder()\r\nout_encoder.fit(trainy)\r\ntrainy = out_encoder.transform(trainy)\r\ntesty = out_encoder.transform(testy)\r\n# fit model\r\nmodel = SVC(kernel='linear', probability=True)\r\nmodel.fit(trainX, trainy)\r\n# predict\r\nyhat_train = model.predict(trainX)\r\nyhat_test = model.predict(testX)\r\n# score\r\nscore_train = accuracy_score(trainy, yhat_train)\r\nscore_test = accuracy_score(testy, yhat_test)\r\n# summarize\r\nprint('Accuracy: train=%.3f, test=%.3f' % (score_train*100, score_test*100))<\/pre>\n<p>Running the example first confirms that the number of samples in the train and test datasets is as we expect<\/p>\n<p>Next, the model is evaluated on the train and test dataset, showing perfect classification accuracy. This is not surprising given the size of the dataset and the power of the face detection and face recognition models used.<\/p>\n<pre class=\"crayon-plain-tag\">Dataset: train=93, test=25\r\nAccuracy: train=100.000, test=100.000<\/pre>\n<p>We can make it more interesting by plotting the original face and the prediction.<\/p>\n<p>First, we need to load the face dataset, specifically the faces in the test dataset. We could also load the original photos to make it even more interesting.<\/p>\n<pre class=\"crayon-plain-tag\"># load faces\r\ndata = load('5-celebrity-faces-dataset.npz')\r\ntestX_faces = data['arr_2']<\/pre>\n<p>The rest of the example is the same up until we fit the model.<\/p>\n<p>First, we need to select a random example from the test set, then get the embedding, face pixels, expected class prediction, and the corresponding name for the class.<\/p>\n<pre class=\"crayon-plain-tag\"># test model on a random example from the test dataset\r\nselection = choice([i for i in range(testX.shape[0])])\r\nrandom_face_pixels = testX_faces[selection]\r\nrandom_face_emb = testX[selection]\r\nrandom_face_class = testy[selection]\r\nrandom_face_name = out_encoder.inverse_transform([random_face_class])<\/pre>\n<p>Next, we can use the face embedding as an input to make a single prediction with the fit model.<\/p>\n<p>We can predict both the class integer and the probability of the prediction.<\/p>\n<pre class=\"crayon-plain-tag\"># prediction for the face\r\nsamples = expand_dims(random_face_emb, axis=0)\r\nyhat_class = model.predict(samples)\r\nyhat_prob = model.predict_proba(samples)<\/pre>\n<p>We can then get the name for the predicted class integer, and the probability for this prediction.<\/p>\n<pre class=\"crayon-plain-tag\"># get name\r\nclass_index = yhat_class[0]\r\nclass_probability = yhat_prob[0,class_index] * 100\r\npredict_names = out_encoder.inverse_transform(yhat_class)<\/pre>\n<p>We can then print this information.<\/p>\n<pre class=\"crayon-plain-tag\">print('Predicted: %s (%.3f)' % (predict_names[0], class_probability))\r\nprint('Expected: %s' % random_face_name[0])<\/pre>\n<p>We can also plot the face pixels along with the predicted name and probability.<\/p>\n<pre class=\"crayon-plain-tag\"># plot for fun\r\npyplot.imshow(random_face_pixels)\r\ntitle = '%s (%.3f)' % (predict_names[0], class_probability)\r\npyplot.title(title)\r\npyplot.show()<\/pre>\n<p>Tying all of this together, the complete example for predicting the identity for a given unseen photo in the test dataset is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># develop a classifier for the 5 Celebrity Faces Dataset\r\nfrom random import choice\r\nfrom numpy import load\r\nfrom numpy import expand_dims\r\nfrom sklearn.preprocessing import LabelEncoder\r\nfrom sklearn.preprocessing import Normalizer\r\nfrom sklearn.svm import SVC\r\nfrom matplotlib import pyplot\r\n# load faces\r\ndata = load('5-celebrity-faces-dataset.npz')\r\ntestX_faces = data['arr_2']\r\n# load face embeddings\r\ndata = load('5-celebrity-faces-embeddings.npz')\r\ntrainX, trainy, testX, testy = data['arr_0'], data['arr_1'], data['arr_2'], data['arr_3']\r\n# normalize input vectors\r\nin_encoder = Normalizer(norm='l2')\r\ntrainX = in_encoder.transform(trainX)\r\ntestX = in_encoder.transform(testX)\r\n# label encode targets\r\nout_encoder = LabelEncoder()\r\nout_encoder.fit(trainy)\r\ntrainy = out_encoder.transform(trainy)\r\ntesty = out_encoder.transform(testy)\r\n# fit model\r\nmodel = SVC(kernel='linear', probability=True)\r\nmodel.fit(trainX, trainy)\r\n# test model on a random example from the test dataset\r\nselection = choice([i for i in range(testX.shape[0])])\r\nrandom_face_pixels = testX_faces[selection]\r\nrandom_face_emb = testX[selection]\r\nrandom_face_class = testy[selection]\r\nrandom_face_name = out_encoder.inverse_transform([random_face_class])\r\n# prediction for the face\r\nsamples = expand_dims(random_face_emb, axis=0)\r\nyhat_class = model.predict(samples)\r\nyhat_prob = model.predict_proba(samples)\r\n# get name\r\nclass_index = yhat_class[0]\r\nclass_probability = yhat_prob[0,class_index] * 100\r\npredict_names = out_encoder.inverse_transform(yhat_class)\r\nprint('Predicted: %s (%.3f)' % (predict_names[0], class_probability))\r\nprint('Expected: %s' % random_face_name[0])\r\n# plot for fun\r\npyplot.imshow(random_face_pixels)\r\ntitle = '%s (%.3f)' % (predict_names[0], class_probability)\r\npyplot.title(title)\r\npyplot.show()<\/pre>\n<p>A different random example from the test dataset will be selected each time the code is run.<\/p>\n<p>Try running it a few times.<\/p>\n<p>In this case, a photo of Jerry Seinfeld is selected and correctly predicted.<\/p>\n<pre class=\"crayon-plain-tag\">Predicted: jerry_seinfeld (88.476)\r\nExpected: jerry_seinfeld<\/pre>\n<p>A plot of the chosen face is also created, showing the predicted name and probability in the image title.<\/p>\n<div id=\"attachment_7826\" style=\"width: 310px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7826\" class=\"wp-image-7826 size-medium\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Detected-Face-of-Jerry-Seinfeld-Correctly-Identified-by-the-SVM-Classifier-300x225.png\" alt=\"Detected Face of Jerry Seinfeld, Correctly Identified by the SVM Classifier\" width=\"300\" height=\"225\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Detected-Face-of-Jerry-Seinfeld-Correctly-Identified-by-the-SVM-Classifier-300x225.png 300w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Detected-Face-of-Jerry-Seinfeld-Correctly-Identified-by-the-SVM-Classifier-768x576.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Detected-Face-of-Jerry-Seinfeld-Correctly-Identified-by-the-SVM-Classifier-1024x768.png 1024w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Detected-Face-of-Jerry-Seinfeld-Correctly-Identified-by-the-SVM-Classifier.png 1280w\" sizes=\"(max-width: 300px) 100vw, 300px\"><\/p>\n<p id=\"caption-attachment-7826\" class=\"wp-caption-text\">Detected Face of Jerry Seinfeld, Correctly Identified by the SVM Classifier<\/p>\n<\/div>\n<h2>Further Reading<\/h2>\n<p>This section provides more resources on the topic if you are looking to go deeper.<\/p>\n<h3>Papers<\/h3>\n<ul>\n<li><a href=\"https:\/\/arxiv.org\/abs\/1503.03832\">FaceNet: A Unified Embedding for Face Recognition and Clustering<\/a>, 2015.<\/li>\n<\/ul>\n<h3>Books<\/h3>\n<ul>\n<li><a href=\"https:\/\/amzn.to\/2EuR8Oo\">Handbook of Face Recognition,<\/a>\u00a02011.<\/li>\n<\/ul>\n<h3>Projects<\/h3>\n<ul>\n<li><a href=\"http:\/\/cmusatyalab.github.io\/openface\/\">OpenFace PyTorch Project<\/a>.<\/li>\n<li><a href=\"https:\/\/github.com\/iwantooxxoox\/Keras-OpenFace\">OpenFace Keras Project, GitHub<\/a>.<\/li>\n<li><a href=\"https:\/\/github.com\/nyoki-mtl\/keras-facenet\">Keras FaceNet Project, GitHub<\/a>.<\/li>\n<li><a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/ms-celeb-1m-challenge-recognizing-one-million-celebrities-real-world\/\">MS-Celeb 1M Dataset<\/a>.<\/li>\n<\/ul>\n<h3>APIs<\/h3>\n<ul>\n<li><a href=\"https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.preprocessing.Normalizer.html\">sklearn.preprocessing.Normalizer API<\/a><\/li>\n<li><a href=\"https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.preprocessing.LabelEncoder.html\">sklearn.preprocessing.LabelEncoder API<\/a><\/li>\n<li><a href=\"https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.svm.SVC.html\">sklearn.svm.SVC API<\/a><\/li>\n<\/ul>\n<h2>Summary<\/h2>\n<p>In this tutorial, you discovered how to develop a face detection system using FaceNet and an SVM classifier to identify people from photographs.<\/p>\n<p>Specifically, you learned:<\/p>\n<ul>\n<li>About the FaceNet face recognition system developed by Google and open source implementations and pre-trained models.<\/li>\n<li>How to prepare a face detection dataset including first extracting faces via a face detection system and then extracting face features via face embeddings.<\/li>\n<li>How to fit, evaluate, and demonstrate an SVM model to predict identities from faces embeddings.<\/li>\n<\/ul>\n<p>Do you have any questions?<br \/>\nAsk your questions in the comments below and I will do my best to answer.<\/p>\n<p>The post <a rel=\"nofollow\" href=\"https:\/\/machinelearningmastery.com\/how-to-develop-a-face-recognition-system-using-facenet-in-keras-and-an-svm-classifier\/\">How to Develop a Face Recognition System Using FaceNet in Keras<\/a> appeared first on <a rel=\"nofollow\" href=\"https:\/\/machinelearningmastery.com\/\">Machine Learning Mastery<\/a>.<\/p>\n<\/div>\n<p><a href=\"https:\/\/machinelearningmastery.com\/how-to-develop-a-face-recognition-system-using-facenet-in-keras-and-an-svm-classifier\/\">Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Author: Jason Brownlee Face recognition is a computer vision task of identifying and verifying a person based on a photograph of their face. FaceNet is [&hellip;] <span class=\"read-more-link\"><a class=\"read-more\" href=\"https:\/\/www.aiproblog.com\/index.php\/2019\/06\/06\/how-to-develop-a-face-recognition-system-using-facenet-in-keras\/\">Read More<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":2238,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[24],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/2237"}],"collection":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=2237"}],"version-history":[{"count":0,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/2237\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media\/2238"}],"wp:attachment":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=2237"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=2237"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=2237"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}