{"id":2219,"date":"2019-06-02T19:00:17","date_gmt":"2019-06-02T19:00:17","guid":{"rendered":"https:\/\/www.aiproblog.com\/index.php\/2019\/06\/02\/how-to-perform-face-detection-with-deep-learning-in-keras\/"},"modified":"2019-06-02T19:00:17","modified_gmt":"2019-06-02T19:00:17","slug":"how-to-perform-face-detection-with-deep-learning-in-keras","status":"publish","type":"post","link":"https:\/\/www.aiproblog.com\/index.php\/2019\/06\/02\/how-to-perform-face-detection-with-deep-learning-in-keras\/","title":{"rendered":"How to Perform Face Detection with Deep Learning in Keras"},"content":{"rendered":"<p>Author: Jason Brownlee<\/p>\n<div>\n<p>Face detection is a computer vision problem that involves finding faces in photos.<\/p>\n<p>It is a trivial problem for humans to solve and has been solved reasonably well by classical feature-based techniques, such as the cascade classifier. More recently deep learning methods have achieved state-of-the-art results on standard benchmark face detection datasets. One example is the Multi-task Cascade Convolutional Neural Network, or MTCNN for short.<\/p>\n<p>In this tutorial, you will discover how to perform face detection in Python using classical and deep learning models.<\/p>\n<p>After completing this tutorial, you will know:<\/p>\n<ul>\n<li>Face detection is a non-trivial computer vision problem for identifying and localizing faces in images.<\/li>\n<li>Face detection can be performed using the classical feature-based cascade classifier using the OpenCV library.<\/li>\n<li>State-of-the-art face detection can be achieved using a Multi-task Cascade CNN via the MTCNN library.<\/li>\n<\/ul>\n<p>Let\u2019s get started.<\/p>\n<div id=\"attachment_7787\" style=\"width: 650px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7787\" class=\"size-full wp-image-7787\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/06\/How-to-Perform-Face-Detection-With-Classical-and-Deep-Learning-Methods-in-Python-with-Keras.jpg\" alt=\"How to Perform Face Detection With Classical and Deep Learning Methods (in Python with Keras)\" width=\"640\" height=\"360\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/06\/How-to-Perform-Face-Detection-With-Classical-and-Deep-Learning-Methods-in-Python-with-Keras.jpg 640w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/06\/How-to-Perform-Face-Detection-With-Classical-and-Deep-Learning-Methods-in-Python-with-Keras-300x169.jpg 300w\" sizes=\"(max-width: 640px) 100vw, 640px\"><\/p>\n<p id=\"caption-attachment-7787\" class=\"wp-caption-text\">How to Perform Face Detection With Classical and Deep Learning Methods (in Python with Keras)<br \/>Photo by <a href=\"https:\/\/www.flickr.com\/photos\/miguel_discart_vrac_3\/46300410915\/\">Miguel Discart<\/a>, some rights reserved.<\/p>\n<\/div>\n<h2>Tutorial Overview<\/h2>\n<p>This tutorial is divided into four parts; they are:<\/p>\n<ol>\n<li>Face Detection<\/li>\n<li>Test Photographs<\/li>\n<li>Face Detection With OpenCV<\/li>\n<li>Face Detection With Deep Learning<\/li>\n<\/ol>\n<h2>Face Detection<\/h2>\n<p>Face detection is a problem in computer vision of locating and localizing one or more faces in a photograph.<\/p>\n<p>Locating a face in a photograph refers to finding the coordinate of the face in the image, whereas localization refers to demarcating the extent of the face, often via a bounding box around the face.<\/p>\n<blockquote>\n<p>A general statement of the problem can be defined as follows: Given a still or video image, detect and localize an unknown number (if any) of faces<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S107731420190921X\">Face Detection: A Survey<\/a>, 2001.<\/p>\n<p>Detecting faces in a photograph is easily solved by humans, although has historically been challenging for computers given the dynamic nature of faces. For example, faces must be detected regardless of orientation or angle they are facing, light levels, clothing, accessories, hair color, facial hair, makeup, age, and so on.<\/p>\n<blockquote>\n<p>The human face is a dynamic object and has a high degree of variability in its appearance, which makes face detection a difficult problem in computer vision.<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S107731420190921X\">Face Detection: A Survey<\/a>, 2001.<\/p>\n<p>Given a photograph, a face detection system will output zero or more bounding boxes that contain faces. Detected faces can then be provided as input to a subsequent system, such as a face recognition system.<\/p>\n<blockquote>\n<p>Face detection is a necessary first-step in face recognition systems, with the purpose of localizing and extracting the face region from the background.<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S107731420190921X\">Face Detection: A Survey<\/a>, 2001.<\/p>\n<p>There are perhaps two main approaches to face recognition: feature-based methods that use hand-crafted filters to search for and detect faces, and image-based methods that learn holistically how to extract faces from the entire image.<\/p>\n<div class=\"woo-sc-hr\"><\/div>\n<p><center><\/p>\n<h3>Want Results with Deep Learning for Computer Vision?<\/h3>\n<p>Take my free 7-day email crash course now (with sample code).<\/p>\n<p>Click to sign-up and also get a free PDF Ebook version of the course.<\/p>\n<p><a href=\"https:\/\/machinelearningmastery.lpages.co\/leadbox\/1458ca1e0972a2%3A164f8be4f346dc\/4715926590455808\/\" target=\"_blank\" style=\"background: rgb(255, 206, 10); color: rgb(255, 255, 255); text-decoration: none; font-family: Helvetica, Arial, sans-serif; font-weight: bold; font-size: 16px; line-height: 20px; padding: 10px; display: inline-block; max-width: 300px; border-radius: 5px; text-shadow: rgba(0, 0, 0, 0.25) 0px -1px 1px; box-shadow: rgba(255, 255, 255, 0.5) 0px 1px 3px inset, rgba(0, 0, 0, 0.5) 0px 1px 3px;\" rel=\"noopener noreferrer\">Download Your FREE Mini-Course<\/a><script data-leadbox=\"1458ca1e0972a2:164f8be4f346dc\" data-url=\"https:\/\/machinelearningmastery.lpages.co\/leadbox\/1458ca1e0972a2%3A164f8be4f346dc\/4715926590455808\/\" data-config=\"%7B%7D\" type=\"text\/javascript\" src=\"https:\/\/machinelearningmastery.lpages.co\/leadbox-1553357564.js\"><\/script><\/p>\n<p><\/center><\/p>\n<div class=\"woo-sc-hr\"><\/div>\n<h2>Test Photographs<\/h2>\n<p>We need test images for face detection in this tutorial.<\/p>\n<p>To keep things simple, we will use two test images: one with two faces, and one with many faces. We\u2019re not trying to push the limits of face detection, just demonstrate how to perform face detection with normal front-on photographs of people.<\/p>\n<p>The first image is a photo of two college students taken by <a href=\"https:\/\/www.flickr.com\/photos\/83633410@N07\/7658261288\/\">CollegeDegrees360<\/a> and made available under a permissive license.<\/p>\n<p>Download the image and place it in your current working directory with the filename \u2018<em>test1.jpg<\/em>\u2018.<\/p>\n<div id=\"attachment_7777\" style=\"width: 650px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7777\" class=\"size-full wp-image-7777\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/test1.jpg\" alt=\"College Students.\" width=\"640\" height=\"427\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/test1.jpg 640w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/test1-300x200.jpg 300w\" sizes=\"(max-width: 640px) 100vw, 640px\"><\/p>\n<p id=\"caption-attachment-7777\" class=\"wp-caption-text\">College Students (test1.jpg)<br \/>Photo by CollegeDegrees360, some rights reserved.<\/p>\n<\/div>\n<ul>\n<li><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/test1.jpg\">College Students (test1.jpg)<\/a><\/li>\n<\/ul>\n<p>The second image is a photograph of a number of people on a swim team taken by <a href=\"https:\/\/www.flickr.com\/photos\/bobnrenee\/2370369784\/\">Bob n Renee<\/a> and released under a permissive license.<\/p>\n<p>Download the image and place it in your current working directory with the filename \u2018<em>test2.jpg<\/em>\u2018.<\/p>\n<div id=\"attachment_7778\" style=\"width: 650px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7778\" class=\"size-full wp-image-7778\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/test2.jpg\" alt=\"Swim Team.\" width=\"640\" height=\"486\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/test2.jpg 640w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/test2-300x228.jpg 300w\" sizes=\"(max-width: 640px) 100vw, 640px\"><\/p>\n<p id=\"caption-attachment-7778\" class=\"wp-caption-text\">Swim Team (test2.jpg)<br \/>Photo by Bob n Renee, some rights reserved.<\/p>\n<\/div>\n<ul>\n<li><a href=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/test2.jpg\">Swim Team (test2.jpg)<\/a><\/li>\n<\/ul>\n<h2>Face Detection With OpenCV<\/h2>\n<p>Feature-based face detection algorithms are fast and effective and have been used successfully for decades.<\/p>\n<p>Perhaps the most successful example is a technique called cascade classifiers first described by Paul Viola and Michael Jones and their 2001 paper titled \u201c<a href=\"https:\/\/ieeexplore.ieee.org\/document\/990517\">Rapid Object Detection using a Boosted Cascade of Simple Features<\/a>.\u201d<\/p>\n<p>In the paper, effective features are learned using the <a href=\"https:\/\/machinelearningmastery.com\/boosting-and-adaboost-for-machine-learning\/\">AdaBoost algorithm<\/a>, although importantly, multiple models are organized into a hierarchy or \u201c<em>cascade<\/em>.\u201d<\/p>\n<p>In the paper, the AdaBoost model is used to learn a range of very simple or weak features in each face, that together provide a robust classifier.<\/p>\n<blockquote>\n<p>\u2026 feature selection is achieved through a simple modification of the AdaBoost procedure: the weak learner is constrained so that each weak classifier returned can depend on only a single feature . As a result each stage of the boosting process, which selects a new weak classifier, can be viewed as a feature selection process.<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/ieeexplore.ieee.org\/document\/990517\">Rapid Object Detection using a Boosted Cascade of Simple Features<\/a>, 2001.<\/p>\n<p>The models are then organized into a hierarchy of increasing complexity, called a \u201c<em>cascade<\/em>\u201c.<\/p>\n<p>Simpler classifiers operate on candidate face regions directly, acting like a coarse filter, whereas complex classifiers operate only on those candidate regions that show the most promise as faces.<\/p>\n<blockquote>\n<p>\u2026 a method for combining successively more complex classifiers in a cascade structure which dramatically increases the speed of the detector by focusing attention on promising regions of the image.<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/ieeexplore.ieee.org\/document\/990517\">Rapid Object Detection using a Boosted Cascade of Simple Features<\/a>, 2001.<\/p>\n<p>The result is a very fast and effective face detection algorithm that has been the basis for face detection in consumer products, such as cameras.<\/p>\n<blockquote>\n<p>Their detector, called detector cascade, consists of a sequence of simple-to-complex face classifiers and has attracted extensive research efforts. Moreover, detector cascade has been deployed in many commercial products such as smartphones and digital cameras.<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/arxiv.org\/abs\/1502.02766\">Multi-view Face Detection Using Deep Convolutional Neural Networks<\/a>, 2015.<\/p>\n<p>It is a modestly complex classifier that has also been tweaked and refined over the last nearly 20 years.<\/p>\n<p>A modern implementation of the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Cascading_classifiers\">Classifier Cascade<\/a> face detection algorithm is provided in the <a href=\"https:\/\/opencv.org\/\">OpenCV library<\/a>. This is a C++ computer vision library that provides a python interface. The benefit of this implementation is that it provides pre-trained face detection models, and provides an interface to train a model on your own dataset.<\/p>\n<p>OpenCV can be installed by the package manager system on your platform, or via pip; for example:<\/p>\n<pre class=\"crayon-plain-tag\">sudo pip install opencv-python<\/pre>\n<p>Once the installation process is complete, it is important to confirm that the library was installed correctly.<\/p>\n<p>This can be achieved by importing the library and checking the version number; for example:<\/p>\n<pre class=\"crayon-plain-tag\"># check opencv version\r\nimport cv2\r\n# print version number\r\nprint(cv2.__version__)<\/pre>\n<p>Running the example will import the library and print the version. In this case, we are using version 3 of the library.<\/p>\n<pre class=\"crayon-plain-tag\">3.4.3<\/pre>\n<p>OpenCV provides the <a href=\"https:\/\/docs.opencv.org\/3.4.3\/d1\/de5\/classcv_1_1CascadeClassifier.html\">CascadeClassifier class<\/a> that can be used to create a cascade classifier for face detection. The constructor can take a filename as an argument that specifies the XML file for a pre-trained model.<\/p>\n<p>OpenCV provides a number of pre-trained models as part of the installation. These are available on your system and are also available on the <a href=\"https:\/\/github.com\/opencv\/opencv\/tree\/master\/data\/haarcascades\">OpenCV GitHub project<\/a>.<\/p>\n<p>Download a pre-trained model for frontal face detection from the OpenCV GitHub project and place it in your current working directory with the filename \u2018<em>haarcascade_frontalface_default.xml<\/em>\u2018.<\/p>\n<ul>\n<li><a href=\"https:\/\/raw.githubusercontent.com\/opencv\/opencv\/master\/data\/haarcascades\/haarcascade_frontalface_default.xml\">Download Open Frontal Face Detection Model (haarcascade_frontalface_default.xml)<\/a><\/li>\n<\/ul>\n<p>Once downloaded, we can load the model as follows:<\/p>\n<pre class=\"crayon-plain-tag\"># load the pre-trained model\r\nclassifier = CascadeClassifier('haarcascade_frontalface_default.xml')<\/pre>\n<p>Once loaded, the model can be used to perform face detection on a photograph by calling the <a href=\"https:\/\/docs.opencv.org\/3.4.3\/d1\/de5\/classcv_1_1CascadeClassifier.html#aaf8181cb63968136476ec4204ffca498\">detectMultiScale() function<\/a>.<\/p>\n<p>This function will return a list of bounding boxes for all faces detected in the photograph.<\/p>\n<pre class=\"crayon-plain-tag\"># perform face detection\r\nbboxes = classifier.detectMultiScale(pixels)\r\n# print bounding box for each detected face\r\nfor box in bboxes:\r\n\tprint(box)<\/pre>\n<p>We can demonstrate this with an example with the college students photograph (<em>test.jpg<\/em>).<\/p>\n<p>The photo can be loaded using OpenCV via the <em>imread()<\/em> function.<\/p>\n<pre class=\"crayon-plain-tag\"># load the photograph\r\npixels = imread('test1.jpg')<\/pre>\n<p>The complete example of performing face detection on the college students photograph with a pre-trained cascade classifier in OpenCV is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># example of face detection with opencv cascade classifier\r\nfrom cv2 import imread\r\nfrom cv2 import CascadeClassifier\r\n# load the photograph\r\npixels = imread('test1.jpg')\r\n# load the pre-trained model\r\nclassifier = CascadeClassifier('haarcascade_frontalface_default.xml')\r\n# perform face detection\r\nbboxes = classifier.detectMultiScale(pixels)\r\n# print bounding box for each detected face\r\nfor box in bboxes:\r\n\tprint(box)<\/pre>\n<p>Running the example first loads the photograph, then loads and configures the cascade classifier; faces are detected and each bounding box is printed.<\/p>\n<p>Each box lists the <em>x<\/em> and <em>y<\/em> coordinates for the bottom-left-hand-corner of the bounding box, as well as the width and the height. The results suggest that two bounding boxes were detected.<\/p>\n<pre class=\"crayon-plain-tag\">[174  75 107 107]\r\n[360 102 101 101]<\/pre>\n<p>We can update the example to plot the photograph and draw each bounding box.<\/p>\n<p>This can be achieved by drawing a rectangle for each box directly over the pixels of the loaded image using the <em>rectangle()<\/em> function that takes two points.<\/p>\n<pre class=\"crayon-plain-tag\"># extract\r\nx, y, width, height = box\r\nx2, y2 = x + width, y + height\r\n# draw a rectangle over the pixels\r\nrectangle(pixels, (x, y), (x2, y2), (0,0,255), 1)<\/pre>\n<p>We can then plot the photograph and keep the window open until we press a key to close it.<\/p>\n<pre class=\"crayon-plain-tag\"># show the image\r\nimshow('face detection', pixels)\r\n# keep the window open until we press a key\r\nwaitKey(0)\r\n# close the window\r\ndestroyAllWindows()<\/pre>\n<p>The complete example is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># plot photo with detected faces using opencv cascade classifier\r\nfrom cv2 import imread\r\nfrom cv2 import imshow\r\nfrom cv2 import waitKey\r\nfrom cv2 import destroyAllWindows\r\nfrom cv2 import CascadeClassifier\r\nfrom cv2 import rectangle\r\n# load the photograph\r\npixels = imread('test1.jpg')\r\n# load the pre-trained model\r\nclassifier = CascadeClassifier('haarcascade_frontalface_default.xml')\r\n# perform face detection\r\nbboxes = classifier.detectMultiScale(pixels)\r\n# print bounding box for each detected face\r\nfor box in bboxes:\r\n\t# extract\r\n\tx, y, width, height = box\r\n\tx2, y2 = x + width, y + height\r\n\t# draw a rectangle over the pixels\r\n\trectangle(pixels, (x, y), (x2, y2), (0,0,255), 1)\r\n# show the image\r\nimshow('face detection', pixels)\r\n# keep the window open until we press a key\r\nwaitKey(0)\r\n# close the window\r\ndestroyAllWindows()<\/pre>\n<p>Running the example, we can see that the photograph was plotted correctly and that each face was correctly detected.<\/p>\n<div id=\"attachment_7779\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7779\" class=\"size-large wp-image-7779\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-1024x685.png\" alt=\"College Students Photograph With Faces Detected using OpenCV Cascade Classifier\" width=\"1024\" height=\"685\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-1024x685.png 1024w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-300x201.png 300w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-768x514.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier.png 1280w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/p>\n<p id=\"caption-attachment-7779\" class=\"wp-caption-text\">College Students Photograph With Faces Detected using OpenCV Cascade Classifier<\/p>\n<\/div>\n<p>We can try the same code on the second photograph of the swim team, specifically \u2018<em>test2.jpg<\/em>\u2018.<\/p>\n<pre class=\"crayon-plain-tag\"># load the photograph\r\npixels = imread('test2.jpg')<\/pre>\n<p>Running the example, we can see that many of the faces were detected correctly, but the result is not perfect.<\/p>\n<p>We can see that a face on the first or bottom row of people was detected twice, that a face on the middle row of people was not detected, and that the background on the third or top row was detected as a face.<\/p>\n<div id=\"attachment_7780\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7780\" class=\"size-large wp-image-7780\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-1024x778.png\" alt=\"Swim Team Photograph With Faces Detected using OpenCV Cascade Classifier\" width=\"1024\" height=\"778\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-1024x778.png 1024w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-300x228.png 300w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-768x584.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier.png 1276w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/p>\n<p id=\"caption-attachment-7780\" class=\"wp-caption-text\">Swim Team Photograph With Faces Detected using OpenCV Cascade Classifier<\/p>\n<\/div>\n<p>The <em>detectMultiScale()<\/em> function provides some arguments to help tune the usage of the classifier. Two parameters of note are <em>scaleFactor<\/em> and <em>minNeighbors<\/em>; for example:<\/p>\n<pre class=\"crayon-plain-tag\"># perform face detection\r\nbboxes = classifier.detectMultiScale(pixels, 1.1, 3)<\/pre>\n<p>The <em>scaleFactor<\/em> controls how the input image is scaled prior to detection, e.g. is it scaled up or down, which can help to better find the faces in the image. The default value is 1.1 (10% increase), although this can be lowered to values such as 1.05 (5% increase) or raised to values such as 1.4 (40% increase).<\/p>\n<p>The <em>minNeighbors<\/em> determines how robust each detection must be in order to be reported, e.g. the number of candidate rectangles that found the face. The default is 3, but this can be lowered to 1 to detect a lot more faces and will likely increase the false positives, or increase to 6 or more to require a lot more confidence before a face is detected.<\/p>\n<p>The <em>scaleFactor<\/em> and <em>minNeighbors<\/em> often require tuning for a given image or dataset in order to best detect the faces. It may be helpful to perform a sensitivity analysis across a grid of values and see what works well or best in general on one or multiple photographs.<\/p>\n<p>A fast strategy may be to lower (or increase for small photos) the scaleFactor until all faces are detected, then increase the <em>minNeighbors<\/em> until all false positives disappear, or close to it.<\/p>\n<p>With some tuning, I found that a <em>scaleFactor<\/em> of 1.05 successfully detected all of the faces, but the background detected as a face did not disappear until a <em>minNeighbors<\/em> of 8, after which three faces on the middle row were no longer detected.<\/p>\n<pre class=\"crayon-plain-tag\"># perform face detection\r\nbboxes = classifier.detectMultiScale(pixels, 1.05, 8)<\/pre>\n<p>The results are not perfect, and perhaps better results can be achieved with further tuning, and perhaps post-processing of the bounding boxes.<\/p>\n<div id=\"attachment_7781\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7781\" class=\"size-large wp-image-7781\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-After-Some-Tuning-1024x779.png\" alt=\"Swim Team Photograph With Faces Detected Using OpenCV Cascade Classifier After Some Tuning\" width=\"1024\" height=\"779\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-After-Some-Tuning-1024x779.png 1024w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-After-Some-Tuning-300x228.png 300w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-After-Some-Tuning-768x584.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-With-Faces-Detected-using-OpenCV-Cascade-Classifier-After-Some-Tuning.png 1272w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/p>\n<p id=\"caption-attachment-7781\" class=\"wp-caption-text\">Swim Team Photograph With Faces Detected Using OpenCV Cascade Classifier After Some Tuning<\/p>\n<\/div>\n<h2>Face Detection With Deep Learning<\/h2>\n<p>A number of deep learning methods have been developed and demonstrated for face detection.<\/p>\n<p>Perhaps one of the more popular approaches is called the \u201c<em>Multi-Task Cascaded Convolutional Neural Network<\/em>,\u201d or MTCNN for short, described by <a href=\"http:\/\/kpzhang93.github.io\/\">Kaipeng Zhang<\/a>, et al. in the 2016 paper titled \u201c<a href=\"https:\/\/arxiv.org\/abs\/1604.02878\">Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks<\/a>.\u201d<\/p>\n<p>The MTCNN is popular because it achieved then state-of-the-art results on a range of benchmark datasets, and because it is capable of also recognizing other facial features such as eyes and mouth, called landmark detection.<\/p>\n<p>The network uses a cascade structure with three networks; first the image is rescaled to a range of different sizes (called an image pyramid), then the first model (Proposal Network or P-Net) proposes candidate facial regions, the second model (Refine Network or R-Net) filters the bounding boxes, and the third model (Output Network or O-Net) proposes facial landmarks.<\/p>\n<blockquote>\n<p>The proposed CNNs consist of three stages. In the first stage, it produces candidate windows quickly through a shallow CNN. Then, it refines the windows to reject a large number of non-faces windows through a more complex CNN. Finally, it uses a more powerful CNN to refine the result and output facial landmarks positions.<\/p>\n<\/blockquote>\n<p>\u2014 <a href=\"https:\/\/arxiv.org\/abs\/1604.02878\">Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks<\/a>, 2016.<\/p>\n<p>The image below taken from the paper provides a helpful summary of the three stages from top-to-bottom and the output of each stage left-to-right.<\/p>\n<div id=\"attachment_7782\" style=\"width: 872px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7782\" class=\"size-large wp-image-7782\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Pipeline-for-the-Multi-Task-Cascaded-Convolutional-Neural-Network-862x1024.png\" alt=\"Pipeline for the Multi-Task Cascaded Convolutional Neural Network\" width=\"862\" height=\"1024\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Pipeline-for-the-Multi-Task-Cascaded-Convolutional-Neural-Network-862x1024.png 862w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Pipeline-for-the-Multi-Task-Cascaded-Convolutional-Neural-Network-253x300.png 253w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Pipeline-for-the-Multi-Task-Cascaded-Convolutional-Neural-Network-768x912.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Pipeline-for-the-Multi-Task-Cascaded-Convolutional-Neural-Network.png 938w\" sizes=\"(max-width: 862px) 100vw, 862px\"><\/p>\n<p id=\"caption-attachment-7782\" class=\"wp-caption-text\">Pipeline for the Multi-Task Cascaded Convolutional Neural NetworkTaken from: Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks.<\/p>\n<\/div>\n<p>The model is called a multi-task network because each of the three models in the cascade (P-Net, R-Net and O-Net) are trained on three tasks, e.g. make three types of predictions; they are: face classification, bounding box regression, and facial landmark localization.<\/p>\n<p>The three models are not connected directly; instead, outputs of the previous stage are fed as input to the next stage. This allows additional processing to be performed between stages; for example, non-maximum suppression (NMS) is used to filter the candidate bounding boxes proposed by the first-stage P-Net prior to providing them to the second stage R-Net model.<\/p>\n<p>The MTCNN architecture is reasonably complex to implement. Thankfully, there are open source implementations of the architecture that can be trained on new datasets, as well as pre-trained models that can be used directly for face detection. Of note is the <a href=\"https:\/\/github.com\/kpzhang93\/MTCNN_face_detection_alignment\">official release<\/a> with the code and models used in the paper, with the implementation provided in the <a href=\"http:\/\/caffe.berkeleyvision.org\/\">Caffe deep learning framework<\/a>.<\/p>\n<p>Perhaps the best-of-breed third-party Python-based MTCNN project is called \u201c<a href=\"https:\/\/github.com\/ipazc\/mtcnn\">MTCNN<\/a>\u201d by <a href=\"https:\/\/www.linkedin.com\/in\/ivandepazcenteno\/\">Iv\u00e1n de Paz Centeno<\/a>, or ipazc, made available under a permissive MIT open source license. As a third-party open-source project, it is subject to change, therefore I have a fork of the project at the time of writing <a href=\"https:\/\/github.com\/jbrownlee\/mtcnn\">available here<\/a>.<\/p>\n<p>The MTCNN project, which we will refer to as <em>ipazc\/MTCNN<\/em> to differentiate it from the name of the network, provides an implementation of the MTCNN architecture using TensorFlow and OpenCV. There are two main benefits to this project; first, it provides a top-performing pre-trained model and the second is that it can be installed as a library ready for use in your own code.<\/p>\n<p>The library can be installed via pip; for example:<\/p>\n<pre class=\"crayon-plain-tag\">sudo pip install mtcnn<\/pre>\n<p>After successful installation, you should see a message like:<\/p>\n<pre class=\"crayon-plain-tag\">Successfully installed mtcnn-0.0.8<\/pre>\n<p>You can then confirm that the library was installed correctly via pip; for example:<\/p>\n<pre class=\"crayon-plain-tag\">sudo pip show mtcnn<\/pre>\n<p>You should see output like that listed below. In this case, you can see that we are using version 0.0.8 of the library.<\/p>\n<pre class=\"crayon-plain-tag\">Name: mtcnn\r\nVersion: 0.0.8\r\nSummary: Multi-task Cascaded Convolutional Neural Networks for Face Detection, based on TensorFlow\r\nHome-page: http:\/\/github.com\/ipazc\/mtcnn\r\nAuthor: Iv\u00e1n de Paz Centeno\r\nAuthor-email: ipazc@unileon.es\r\nLicense: MIT\r\nLocation: ...\r\nRequires:\r\nRequired-by:<\/pre>\n<p>You can also confirm that the library was installed correctly via Python, as follows:<\/p>\n<pre class=\"crayon-plain-tag\"># confirm mtcnn was installed correctly\r\nimport mtcnn\r\n# print version\r\nprint(mtcnn.__version__)<\/pre>\n<p>Running the example will load the library, confirming it was installed correctly; and print the version.<\/p>\n<pre class=\"crayon-plain-tag\">0.0.8<\/pre>\n<p>Now that we are confident that the library was installed correctly, we can use it for face detection.<\/p>\n<p>An instance of the network can be created by calling the <em>MTCNN()<\/em> constructor.<\/p>\n<p>By default, the library will use the pre-trained model, although you can specify your own model via the \u2018<em>weights_file<\/em>\u2018 argument and specify a path or URL, for example:<\/p>\n<pre class=\"crayon-plain-tag\">model = MTCNN(weights_file='filename.npy')<\/pre>\n<p>The minimum box size for detecting a face can be specified via the \u2018<em>min_face_size<\/em>\u2018 argument, which defaults to 20 pixels. The constructor also provides a \u2018<em>scale_factor<\/em>\u2018 argument to specify the scale factor for the input image, which defaults to 0.709.<\/p>\n<p>Once the model is configured and loaded, it can be used directly to detect faces in photographs by calling the <em>detect_faces()<\/em> function.<\/p>\n<p>This returns a list of dict object, each providing a number of keys for the details of each face detected, including:<\/p>\n<ul>\n<li>\u2018<em>box<\/em>\u2018: Providing the <em>x<\/em>, <em>y<\/em> of the bottom left of the bounding box, as well as the <em>width<\/em> and <em>height<\/em> of the box.<\/li>\n<li>\u2018<em>confidence<\/em>\u2018: The probability confidence of the prediction.<\/li>\n<li>\u2018<em>keypoints<\/em>\u2018: Providing a dict with dots for the \u2018<em>left_eye<\/em>\u2018, \u2018<em>right_eye<\/em>\u2018, \u2018<em>nose<\/em>\u2018, \u2018<em>mouth_left<\/em>\u2018, and \u2018<em>mouth_right<\/em>\u2018.<\/li>\n<\/ul>\n<p>For example, we can perform face detection on the college students photograph as follows:<\/p>\n<pre class=\"crayon-plain-tag\"># face detection with mtcnn on a photograph\r\nfrom matplotlib import pyplot\r\nfrom mtcnn.mtcnn import MTCNN\r\n# load image from file\r\nfilename = 'test1.jpg'\r\npixels = pyplot.imread(filename)\r\n# create the detector, using default weights\r\ndetector = MTCNN()\r\n# detect faces in the image\r\nfaces = detector.detect_faces(pixels)\r\nfor face in faces:\r\n\tprint(face)<\/pre>\n<p>Running the example loads the photograph, loads the model, performs face detection, and prints a list of each face detected.<\/p>\n<pre class=\"crayon-plain-tag\">{'box': [186, 71, 87, 115], 'confidence': 0.9994562268257141, 'keypoints': {'left_eye': (207, 110), 'right_eye': (252, 119), 'nose': (220, 143), 'mouth_left': (200, 148), 'mouth_right': (244, 159)}}\r\n{'box': [368, 75, 108, 138], 'confidence': 0.998593270778656, 'keypoints': {'left_eye': (392, 133), 'right_eye': (441, 140), 'nose': (407, 170), 'mouth_left': (388, 180), 'mouth_right': (438, 185)}}<\/pre>\n<p>We can draw the boxes on the image by first plotting the image with matplotlib, then creating a <a href=\"https:\/\/matplotlib.org\/api\/_as_gen\/matplotlib.patches.Rectangle.html\">Rectangle object<\/a> using the <em>x<\/em>, <em>y<\/em> and <em>width<\/em> and <em>height<\/em> of a given bounding box; for example:<\/p>\n<pre class=\"crayon-plain-tag\"># get coordinates\r\nx, y, width, height = result['box']\r\n# create the shape\r\nrect = Rectangle((x, y), width, height, fill=False, color='red')<\/pre>\n<p>Below is a function named <em>draw_image_with_boxes()<\/em> that shows the photograph and then draws a box for each bounding box detected.<\/p>\n<pre class=\"crayon-plain-tag\"># draw an image with detected objects\r\ndef draw_image_with_boxes(filename, result_list):\r\n\t# load the image\r\n\tdata = pyplot.imread(filename)\r\n\t# plot the image\r\n\tpyplot.imshow(data)\r\n\t# get the context for drawing boxes\r\n\tax = pyplot.gca()\r\n\t# plot each box\r\n\tfor result in result_list:\r\n\t\t# get coordinates\r\n\t\tx, y, width, height = result['box']\r\n\t\t# create the shape\r\n\t\trect = Rectangle((x, y), width, height, fill=False, color='red')\r\n\t\t# draw the box\r\n\t\tax.add_patch(rect)\r\n\t# show the plot\r\n\tpyplot.show()<\/pre>\n<p>The complete example making use of this function is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># face detection with mtcnn on a photograph\r\nfrom matplotlib import pyplot\r\nfrom matplotlib.patches import Rectangle\r\nfrom mtcnn.mtcnn import MTCNN\r\n\r\n# draw an image with detected objects\r\ndef draw_image_with_boxes(filename, result_list):\r\n\t# load the image\r\n\tdata = pyplot.imread(filename)\r\n\t# plot the image\r\n\tpyplot.imshow(data)\r\n\t# get the context for drawing boxes\r\n\tax = pyplot.gca()\r\n\t# plot each box\r\n\tfor result in result_list:\r\n\t\t# get coordinates\r\n\t\tx, y, width, height = result['box']\r\n\t\t# create the shape\r\n\t\trect = Rectangle((x, y), width, height, fill=False, color='red')\r\n\t\t# draw the box\r\n\t\tax.add_patch(rect)\r\n\t# show the plot\r\n\tpyplot.show()\r\n\r\nfilename = 'test1.jpg'\r\n# load image from file\r\npixels = pyplot.imread(filename)\r\n# create the detector, using default weights\r\ndetector = MTCNN()\r\n# detect faces in the image\r\nfaces = detector.detect_faces(pixels)\r\n# display faces on the original image\r\ndraw_image_with_boxes(filename, faces)<\/pre>\n<p>Running the example plots the photograph then draws a bounding box for each of the detected faces.<\/p>\n<p>We can see that both faces were detected correctly.<\/p>\n<div id=\"attachment_7783\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7783\" class=\"size-large wp-image-7783\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-Drawn-For-Each-Detected-Face-using-MTCNN-1024x768.png\" alt=\"College Students Photograph With Bounding Boxes Drawn for Each Detected Face Using MTCNN\" width=\"1024\" height=\"768\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-Drawn-For-Each-Detected-Face-using-MTCNN-1024x768.png 1024w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-Drawn-For-Each-Detected-Face-using-MTCNN-300x225.png 300w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-Drawn-For-Each-Detected-Face-using-MTCNN-768x576.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-Drawn-For-Each-Detected-Face-using-MTCNN.png 1280w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/p>\n<p id=\"caption-attachment-7783\" class=\"wp-caption-text\">College Students Photograph With Bounding Boxes Drawn for Each Detected Face Using MTCNN<\/p>\n<\/div>\n<p>We can draw a circle via the <a href=\"https:\/\/matplotlib.org\/api\/_as_gen\/matplotlib.patches.Circle.html\">Circle class<\/a> for the eyes, nose, and mouth; for example<\/p>\n<pre class=\"crayon-plain-tag\"># draw the dots\r\nfor key, value in result['keypoints'].items():\r\n\t# create and draw dot\r\n\tdot = Circle(value, radius=2, color='red')\r\n\tax.add_patch(dot)<\/pre>\n<p>The complete example with this addition to the <em>draw_image_with_boxes()<\/em> function is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># face detection with mtcnn on a photograph\r\nfrom matplotlib import pyplot\r\nfrom matplotlib.patches import Rectangle\r\nfrom matplotlib.patches import Circle\r\nfrom mtcnn.mtcnn import MTCNN\r\n\r\n# draw an image with detected objects\r\ndef draw_image_with_boxes(filename, result_list):\r\n\t# load the image\r\n\tdata = pyplot.imread(filename)\r\n\t# plot the image\r\n\tpyplot.imshow(data)\r\n\t# get the context for drawing boxes\r\n\tax = pyplot.gca()\r\n\t# plot each box\r\n\tfor result in result_list:\r\n\t\t# get coordinates\r\n\t\tx, y, width, height = result['box']\r\n\t\t# create the shape\r\n\t\trect = Rectangle((x, y), width, height, fill=False, color='red')\r\n\t\t# draw the box\r\n\t\tax.add_patch(rect)\r\n\t\t# draw the dots\r\n\t\tfor key, value in result['keypoints'].items():\r\n\t\t\t# create and draw dot\r\n\t\t\tdot = Circle(value, radius=2, color='red')\r\n\t\t\tax.add_patch(dot)\r\n\t# show the plot\r\n\tpyplot.show()\r\n\r\nfilename = 'test1.jpg'\r\n# load image from file\r\npixels = pyplot.imread(filename)\r\n# create the detector, using default weights\r\ndetector = MTCNN()\r\n# detect faces in the image\r\nfaces = detector.detect_faces(pixels)\r\n# display faces on the original image\r\ndraw_image_with_boxes(filename, faces)<\/pre>\n<p>The example plots the photograph again with bounding boxes and facial key points.<\/p>\n<p>We can see that eyes, nose, and mouth are detected well on each face, although the mouth on the right face could be better detected, with the points looking a little lower than the corners of the mouth.<\/p>\n<div id=\"attachment_7784\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7784\" class=\"size-large wp-image-7784\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN-1024x768.png\" alt=\"College Students Photograph With Bounding Boxes and Facial Keypoints Drawn for Each Detected Face Using MTCNN\" width=\"1024\" height=\"768\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN-1024x768.png 1024w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN-300x225.png 300w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN-768x576.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Collage-Students-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN.png 1280w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/p>\n<p id=\"caption-attachment-7784\" class=\"wp-caption-text\">College Students Photograph With Bounding Boxes and Facial Keypoints Drawn for Each Detected Face Using MTCNN<\/p>\n<\/div>\n<p>We can now try face detection on the swim team photograph, e.g. the image test2.jpg.<\/p>\n<p>Running the example, we can see that all thirteen faces were correctly detected and that it looks roughly like all of the facial keypoints are also correct.<\/p>\n<div id=\"attachment_7785\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7785\" class=\"size-large wp-image-7785\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN-1024x768.png\" alt=\"Swim Team Photograph With Bounding Boxes and Facial Keypoints Drawn for Each Detected Face Using MTCNN\" width=\"1024\" height=\"768\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN-1024x768.png 1024w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN-300x225.png 300w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN-768x576.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Swim-Team-Photograph-with-Bounding-Boxes-and-Facial-Keypoints-Drawn-For-Each-Detected-Face-using-MTCNN.png 1280w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/p>\n<p id=\"caption-attachment-7785\" class=\"wp-caption-text\">Swim Team Photograph With Bounding Boxes and Facial Keypoints Drawn for Each Detected Face Using MTCNN<\/p>\n<\/div>\n<p>We may want to extract the detected faces and pass them as input to another system.<\/p>\n<p>This can be achieved by extracting the pixel data directly out of the photograph; for example:<\/p>\n<pre class=\"crayon-plain-tag\"># get coordinates\r\nx1, y1, width, height = result['box']\r\nx2, y2 = x1 + width, y1 + height\r\n# extract face\r\nface = data[y1:y2, x1:x2]<\/pre>\n<p>We can demonstrate this by extracting each face and plotting them as separate subplots. You could just as easily save them to file. The <em>draw_faces()<\/em> below extracts and plots each detected face in a photograph.<\/p>\n<pre class=\"crayon-plain-tag\"># draw each face separately\r\ndef draw_faces(filename, result_list):\r\n\t# load the image\r\n\tdata = pyplot.imread(filename)\r\n\t# plot each face as a subplot\r\n\tfor i in range(len(result_list)):\r\n\t\t# get coordinates\r\n\t\tx1, y1, width, height = result_list[i]['box']\r\n\t\tx2, y2 = x1 + width, y1 + height\r\n\t\t# define subplot\r\n\t\tpyplot.subplot(1, len(result_list), i+1)\r\n\t\tpyplot.axis('off')\r\n\t\t# plot face\r\n\t\tpyplot.imshow(data[y1:y2, x1:x2])\r\n\t# show the plot\r\n\tpyplot.show()<\/pre>\n<p>The complete example demonstrating this function for the swim team photo is listed below.<\/p>\n<pre class=\"crayon-plain-tag\"># extract and plot each detected face in a photograph\r\nfrom matplotlib import pyplot\r\nfrom matplotlib.patches import Rectangle\r\nfrom matplotlib.patches import Circle\r\nfrom mtcnn.mtcnn import MTCNN\r\n\r\n# draw each face separately\r\ndef draw_faces(filename, result_list):\r\n\t# load the image\r\n\tdata = pyplot.imread(filename)\r\n\t# plot each face as a subplot\r\n\tfor i in range(len(result_list)):\r\n\t\t# get coordinates\r\n\t\tx1, y1, width, height = result_list[i]['box']\r\n\t\tx2, y2 = x1 + width, y1 + height\r\n\t\t# define subplot\r\n\t\tpyplot.subplot(1, len(result_list), i+1)\r\n\t\tpyplot.axis('off')\r\n\t\t# plot face\r\n\t\tpyplot.imshow(data[y1:y2, x1:x2])\r\n\t# show the plot\r\n\tpyplot.show()\r\n\r\nfilename = 'test2.jpg'\r\n# load image from file\r\npixels = pyplot.imread(filename)\r\n# create the detector, using default weights\r\ndetector = MTCNN()\r\n# detect faces in the image\r\nfaces = detector.detect_faces(pixels)\r\n# display faces on the original image\r\ndraw_faces(filename, faces)<\/pre>\n<p>Running the example creates a plot that shows each separate face detected in the photograph of the swim team.<\/p>\n<div id=\"attachment_7786\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7786\" class=\"size-large wp-image-7786\" src=\"https:\/\/machinelearningmastery.com\/wp-content\/uploads\/2019\/03\/Plot-of-Each-Separate-Face-Detected-in-a-Photograph-of-a-Swim-Team-1024x169.png\" alt=\"Plot of Each Separate Face Detected in a Photograph of a Swim Team\" width=\"1024\" height=\"169\" srcset=\"http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Plot-of-Each-Separate-Face-Detected-in-a-Photograph-of-a-Swim-Team-1024x169.png 1024w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Plot-of-Each-Separate-Face-Detected-in-a-Photograph-of-a-Swim-Team-300x49.png 300w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Plot-of-Each-Separate-Face-Detected-in-a-Photograph-of-a-Swim-Team-768x127.png 768w, http:\/\/3qeqpr26caki16dnhd19sv6by6v.wpengine.netdna-cdn.com\/wp-content\/uploads\/2019\/03\/Plot-of-Each-Separate-Face-Detected-in-a-Photograph-of-a-Swim-Team.png 1056w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/p>\n<p id=\"caption-attachment-7786\" class=\"wp-caption-text\">Plot of Each Separate Face Detected in a Photograph of a Swim Team<\/p>\n<\/div>\n<h2>Further Reading<\/h2>\n<p>This section provides more resources on the topic if you are looking to go deeper.<\/p>\n<h3>Papers<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S107731420190921X\">Face Detection: A Survey<\/a>, 2001.<\/li>\n<li><a href=\"https:\/\/ieeexplore.ieee.org\/document\/990517\">Rapid Object Detection using a Boosted Cascade of Simple Features<\/a>, 2001.<\/li>\n<li><a href=\"https:\/\/arxiv.org\/abs\/1502.02766\">Multi-view Face Detection Using Deep Convolutional Neural Networks<\/a>, 2015.<\/li>\n<li><a href=\"https:\/\/arxiv.org\/abs\/1604.02878\">Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks<\/a>, 2016.<\/li>\n<\/ul>\n<h3>Books<\/h3>\n<ul>\n<li>Chapter 11 Face Detection, <a href=\"https:\/\/amzn.to\/2EuR8Oo\">Handbook of Face Recognition<\/a>, Second Edition, 2011.<\/li>\n<\/ul>\n<h3>API<\/h3>\n<ul>\n<li><a href=\"https:\/\/opencv.org\/\">OpenCV Homepage<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/opencv\/opencv\">OpenCV GitHub Project<\/a><\/li>\n<li><a href=\"https:\/\/docs.opencv.org\/3.4.3\/d7\/d8b\/tutorial_py_face_detection.html\">Face Detection using Haar Cascades, OpenCV<\/a>.<\/li>\n<li><a href=\"https:\/\/docs.opencv.org\/3.4.3\/dc\/d88\/tutorial_traincascade.html\">Cascade Classifier Training, OpenCV<\/a>.<\/li>\n<li><a href=\"https:\/\/docs.opencv.org\/3.4.3\/db\/d28\/tutorial_cascade_classifier.html\">Cascade Classifier, OpenCV<\/a>.<\/li>\n<li><a href=\"https:\/\/github.com\/kpzhang93\/MTCNN_face_detection_alignment\">Official MTCNN Project<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ipazc\/mtcnn\">Python MTCNN Project<\/a><\/li>\n<li><a href=\"https:\/\/matplotlib.org\/api\/_as_gen\/matplotlib.patches.Rectangle.html\">matplotlib.patches.Rectangle API<\/a><\/li>\n<li><a href=\"https:\/\/matplotlib.org\/api\/_as_gen\/matplotlib.patches.Circle.html\">matplotlib.patches.Circle API<\/a><\/li>\n<\/ul>\n<h3>Articles<\/h3>\n<ul>\n<li><a href=\"https:\/\/en.wikipedia.org\/wiki\/Face_detection\">Face detection, Wikipedia<\/a>.<\/li>\n<li><a href=\"https:\/\/en.wikipedia.org\/wiki\/Cascading_classifiers\">Cascading classifiers, Wikipedia<\/a>.<\/li>\n<\/ul>\n<h3>Summary<\/h3>\n<p>In this tutorial, you discovered how to perform face detection in Python using classical and deep learning models.<\/p>\n<p>Specifically, you learned:<\/p>\n<ul>\n<li>Face detection is a computer vision problem for identifying and localizing faces in images.<\/li>\n<li>Face detection can be performed using the classical feature-based cascade classifier using the OpenCV library.<\/li>\n<li>State-of-the-art face detection can be achieved using a Multi-task Cascade CNN via the MTCNN library.<\/li>\n<\/ul>\n<p>Do you have any questions?<br \/>\nAsk your questions in the comments below and I will do my best to answer.<\/p>\n<p>The post <a rel=\"nofollow\" href=\"https:\/\/machinelearningmastery.com\/how-to-perform-face-detection-with-classical-and-deep-learning-methods-in-python-with-keras\/\">How to Perform Face Detection with Deep Learning in Keras<\/a> appeared first on <a rel=\"nofollow\" href=\"https:\/\/machinelearningmastery.com\/\">Machine Learning Mastery<\/a>.<\/p>\n<\/div>\n<p><a href=\"https:\/\/machinelearningmastery.com\/how-to-perform-face-detection-with-classical-and-deep-learning-methods-in-python-with-keras\/\">Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Author: Jason Brownlee Face detection is a computer vision problem that involves finding faces in photos. It is a trivial problem for humans to solve [&hellip;] <span class=\"read-more-link\"><a class=\"read-more\" href=\"https:\/\/www.aiproblog.com\/index.php\/2019\/06\/02\/how-to-perform-face-detection-with-deep-learning-in-keras\/\">Read More<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":2220,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[24],"tags":[],"_links":{"self":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/2219"}],"collection":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/comments?post=2219"}],"version-history":[{"count":0,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/posts\/2219\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media\/2220"}],"wp:attachment":[{"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/media?parent=2219"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/categories?post=2219"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aiproblog.com\/index.php\/wp-json\/wp\/v2\/tags?post=2219"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}