A Simple Introduction to Facial Recognition (with Python codes)

guest_blog 14 May, 2020 • 8 min read

Introduction

Did you know that every time you upload a photo to Facebook, the platform uses facial recognition algorithms to identify the people in that image? Or that certain governments around the world use face recognition technology to identify and catch criminals? I don’t need to tell you that you can now unlock smartphones with your face!

The applications of this sub-domain of computer vision are vast and businesses around the world are already reaping the benefits. The usage of face recognition models is only going to increase in the next few years so why not teach yourself how to build one from scratch?

In this article, we are going to do just that. We will first understand the inner workings of face recognition, and then take a simple case study and implement it in Python. By the end of the article you will have built your very first facial recognition model!

 

Table of Contents

  • Understanding how Face Recognition works
  • Case Study
  • Implementation in Python
  • Understanding the Python code
  • Applications of Facial Recognition Algorithms

 

Understanding how Face Recognition works

In order to understand how Face Recognition works, let us first get an idea of the concept of a feature vector.

Every Machine Learning algorithm takes a dataset as input and learns from this data. The algorithm goes through the data and identifies patterns in the data. For instance, suppose we wish to identify whose face is present in a given image, there are multiple things we can look at as a pattern:

  • Height/width of the face.
  • Height and width may not be reliable since the image could be rescaled to a smaller face. However, even after rescaling, what remains unchanged are the ratios – the ratio of height of the face to the width of the face won’t change.
  • Color of the face.
  • Width of other parts of the face like lips, nose, etc.

Clearly, there is a pattern here – different faces have different dimensions like the ones above. Similar faces have similar dimensions. The challenging part is to convert a particular face into numbers – Machine Learning algorithms only understand numbers. This numerical representation of a “face” (or an element in the training set) is termed as a feature vector. A feature vector comprises of various numbers in a specific order.

As a simple example, we can map a “face” into a feature vector which can comprise various features like:

  • Height of face (cm)
  • Width of face (cm)
  • Average color of face (R, G, B)
  • Width of lips (cm)
  • Height of nose (cm)

Essentially, given an image, we can map out various features and convert it into a feature vector like:

Height of face (cm) Width of face (cm) Average color of face (RGB) Width of lips (cm) Height of nose (cm)
23.1 15.8 (255, 224, 189) 5.2 4.4

 

So, our image is now a vector that could be represented as (23.1, 15.8, 255, 224, 189, 5.2, 4.4). Of course there could be countless other features that could be derived from the image (for instance, hair color, facial hair, spectacles, etc). However, for the example, let us consider just these 5 simple features.

Now, once we have encoded each image into a feature vector, the problem becomes much simpler. Clearly, when we have 2 faces (images) that represent the same person, the feature vectors derived will be quite similar. Put it the other way, the “distance” between the 2 feature vectors will be quite small.

Machine Learning can help us here with 2 things:

  1. Deriving the feature vector: it is difficult to manually list down all of the features because there are just so many. A Machine Learning algorithm can intelligently label out many of such features. For instance, a complex features could be: ratio of height of nose and width of forehead. Now it will be quite difficult for a human to list down all such “second order” features.
  2. Matching algorithms: Once the feature vectors have been obtained, a Machine Learning algorithm needs to match a new image with the set of feature vectors present in the corpus.

Now that we have a basic understanding of how Face Recognition works, let us build our own Face Recognition algorithm using some of the well-known Python libraries.

 

Case Study

We are given a bunch of faces – possibly of celebrities like Mark Zuckerberg, Warren Buffett, Bill Gates, Shah Rukh Khan, etc. Call this bunch of faces as our “corpus”. Now, we are given image of yet another celebrity (“new celebrity”). The task is simple – identify if this “new celebrity” is among those present in the “corpus”.

Here are some of the images in the corpus:

As you can see, we have celebrities like Barack Obama, Bill Gates, Jeff Bezos, Mark Zuckerberg, Ray Dalio and Shah Rukh Khan.

Now, here is the “new celebrity”:

Note: all of the above images have been taken from Google images.

It is obvious that this is Shah Rukh Khan. However, for a computer this is a challenging task. The challenge is because of the fact that for us humans, it is easy to combine so many features of the images to see which one is which celebrity. However, for a computer, it isn’t straightforward to learn how to recognize these faces.

There is an amazingly simple Python library that encapsulates all of what we learn above – creating feature vectors out of faces and knowing how to differentiate across faces. This Python library is called as face_recognition and deep within, it employs dlib – a modern C++ toolkit that contains several machine learning algorithms that help in writing sophisticated C++ based applications.

face_recognition library in Python can perform a large number of tasks:

  • Find all the faces in a given image
  • Find and manipulate facial features in an image
  • Identify faces in images
  • Real-time face recognition

Here, we will talk about the 3rd use case – identify faces in images.

You can find the source code of face_recognition library here on Github: https://github.com/ageitgey/face_recognition.

In fact, there is also a tutorial on how to install face_recognition library: https://github.com/ageitgey/face_recognition#installation-options. Before you install face_recognition, you need to install dlib as well. You can find the instructions to install dlib over here: https://gist.github.com/ageitgey/629d75c1baac34dfa5ca2a1928a7aeaf.

 

Implementation in Python

This section contains the code for a building a straightforward face recognition system using the face_recognition library. This is the implementation part, we will go through the code to understand it in more detail in the next section.

# import the libraries
import os
import face_recognition

# make a list of all the available images
images = os.listdir('images')

# load your image
image_to_be_matched = face_recognition.load_image_file('my_image.jpg')

# encoded the loaded image into a feature vector
image_to_be_matched_encoded = face_recognition.face_encodings(
    image_to_be_matched)[0]

# iterate over each image
for image in images:
    # load the image
    current_image = face_recognition.load_image_file("images/" + image)
    # encode the loaded image into a feature vector
    current_image_encoded = face_recognition.face_encodings(current_image)[0]
    # match your image with the image and check if it matches
    result = face_recognition.compare_faces(
        [image_to_be_matched_encoded], current_image_encoded)
    # check if it was a match
    if result[0] == True:
        print "Matched: " + image
    else:
        print "Not matched: " + image

The folder structure is as follows:

facialrecognition:

  • fr.py
  • my_image.jpg
  • images/
    • barack_obama.jpg
    • bill_gates.jpg
    • jeff_bezos.jpg
    • mark_zuckerberg.jpg
    • ray_dalio.jpg
    • shah_rukh_khan.jpg
    • warren_buffett.jpg

Our root directory, facialrecognition contains:

  • Our face recognition code above in the form of fr.py.
  • my_image.jpg – the image to be recognized (“new celebrity”).
  • images/ – the “corpus”.

When you create the folder structure as above and run the above code, here is what you get as the output:

Matched: shah_rukh_khan.jpg
Not matched: warren_buffett.jpg
Not matched: barack_obama.jpg
Not matched: ray_dalio.jpg
Not matched: bill_gates.jpg
Not matched: jeff_bezos.jpg
Not matched: mark_zuckerberg.jpg

Clearly, the “new celebrity” is Shah Rukh Khan and our face recognition system is able to detect it!

 

Understanding the Python code

Now, let us go through the code to understand how it works:

# import the libraries
import os
import face_recognition

These are simply the imports. We will be using the built-in os library to read all the images in our corpus and we will use face_recognition for the purpose of writing the algorithm.

# make a list of all the available images
images = os.listdir('images')

This simple code helps us identify the path of all of the images in the corpus. Once this line is executed, we will have:

images = ['shah_rukh_khan.jpg', 'warren_buffett.jpg', 'barack_obama.jpg', 'ray_dalio.jpg', 'bill_gates.jpg', 'jeff_bezos.jpg', 'mark_zuckerberg.jpg']

Now, the code below loads the new celebrity’s image:

# load your image
image_to_be_matched = face_recognition.load_image_file('my_image.jpg')

To make sure that the algorithms are able to interpret the image, we convert the image to a feature vector:

# encoded the loaded image into a feature vector

image_to_be_matched_encoded = face_recognition.face_encodings(

    image_to_be_matched)[0]

The rest of the code now is fairly easy:

# iterate over each image
for image in images:
    # load the image
    current_image = face_recognition.load_image_file("images/" + image)

    # encode the loaded image into a feature vector
    current_image_encoded = face_recognition.face_encodings(current_image)[0]

    # match your image with the image and check if it matches
    result = face_recognition.compare_faces(
        [image_to_be_matched_encoded], current_image_encoded)

    # check if it was a match
    if result[0] == True:
        print "Matched: " + image
    else:
        print "Not matched: " + image

Here, we are:

  • Looping over each image.
  • Encoding the image into a feature vector.
  • Comparing the loaded image with the image to be recognized.
  • If it is a match, we print that. If it is a mismatch, we print that as well.

The output as shown above clearly suggests that this simple face recognition algorithm works amazingly well. Let us try replacing my_image with another image:

When you run the algorithm again, you will see the following output:

Not matched: shah_rukh_khan.jpg
Not matched: warren_buffett.jpg
Not matched: barack_obama.jpg
Not matched: ray_dalio.jpg
Not matched: bill_gates.jpg
Not matched: jeff_bezos.jpg
Not matched: mark_zuckerberg.jpg

Clearly, the system did not identify Jack Ma as any of the above celebrities. This indicates that our algorithm is quite good in both:

  • Correctly identifying those that are present in the corpus
  • Flagging a mismatch for those that are not present in the corpus

 

Face Recognition Applications

Face Recognition is a well researched problem and is widely used in both industry and in academia. As an example, a criminal in China was caught because a Face Recognition system in a mall detected his face and raised an alarm. Clearly, Face Recognition can be used to mitigate crime. There are many other interesting use cases of Face Recognition:

  • Facial Authentication: Apple has brought in Face ID for Facial Authentication in iPhones. Some of the leading banks are trying to use Facial Authentication for lockers.
  • Customer Service: Some of the banks in Malaysia have installed systems which use Face Recognition to detect valuable customers of the bank so that the bank can provide the personalized service. This way, banks are able to generate more revenues by retaining such customers and keeping them happy.
  • Insurance Underwriting: Many insurance companies are using Face Recognition to match the face of the person with that provided in the photo ID proof. This way, the underwriting process becomes much faster.

 

End Notes

To summarize, Face Recognition is an interesting problem with lots of powerful use cases which can significantly help society across various dimensions. While there will always be an ethical risk attached to commercialzing such techniques, that is a debate we will shelve for another time.

I hope you found this article useful. Please provide you feedback and suggestions in the comments section below!

 

About the author

Aman Goel – Co-Founder and CEO at AllinCall Research & Solutions
Aman Goel is an IIT-Bombay Alumnus and is an entrepreneur, coder, and a fan of air crash investigation. He writes programming blogs for Hackr.io, a programming community to find the best programming tutorials. He is a co-founder at AllinCall Research & Solutions and loves helping people in cracking IIT JEE problems.
guest_blog 14 May 2020

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Responses From Readers

Clear

Vijay Kumar Machadapally
Vijay Kumar Machadapally 30 Aug, 2018

Excellently and in simple - explained about Facial recognition process with python code. Hats off. Continue the good work.

Vidyush
Vidyush 30 Aug, 2018

A very simple and effective tutorial tyvm!

Nimshi Kollu
Nimshi Kollu 02 Sep, 2018

Superb

Nikhil
Nikhil 08 Sep, 2018

Thank you very much for such a simple and elegant explanation. I am trying to implement same but getting below error. I also googled and found some installation pre-requisite like a Visual Studio. But still, I am getting below error while installing face_recognition: CMake Error in CMakeLists.txt: The CMAKE_CXX_COMPILER: Could you please help me to resolve the error. Thank you in advanced!

Sanjay Nath
Sanjay Nath 08 Sep, 2018

Can you suggest any resources to further go deeper into this area? Amazing article.Thank you.

Dadagouda
Dadagouda 09 Sep, 2018

Put in very simplistic way so that anyone can understand! Thanks!

Hector Alvaro Rojas
Hector Alvaro Rojas 17 Sep, 2018

Hi Aman: Great article? Do you know how to install face_recognition and dlib (and all stuff necessary to run face_recognition) under Windows 7, 64 bits platform? Any chance to get the information?

Omar
Omar 30 Oct, 2018

A well written tutorial and easy to understand Thanks a lot

NAGENDRA GIDDALURU
NAGENDRA GIDDALURU 10 Nov, 2018

Simply superb article

Zahoor Ahmad
Zahoor Ahmad 22 Nov, 2018

Well-done Dear Aman: Great article. Do you know how to install face_recognition and dlib (and all stuff necessary to run face_recognition) under Windows 10, 64 bits platform? Any chance to get the information?

Olcha
Olcha 23 Dec, 2018

Excellent and Simple. Thank you.

Aminu Ibrahim
Aminu Ibrahim 22 Jan, 2019

Well done, simple and easy to understand, pls can you help, I'm working on a project that will recognize face and identify some social network account of the victim ( detected face)

Santosh
Santosh 03 Apr, 2019

Are you serious? is it that simple :). You are awesome dude.

Vuyiso
Vuyiso 11 Apr, 2019

I love how you explained it

Sandip Bhagwat
Sandip Bhagwat 12 Apr, 2019

Great article! Excellent and easy to understand. Thank you.

Related Courses

image.name
0 Hrs 70 Lessons
5

Introduction to Python

Free

Computer Vision
Become a full stack data scientist