Learn How to Implement Face Recognition using OpenCV with Python!

This article was published as a part of the Data Science Blogathon

Pre-requisites

– Basic understanding of Image Classification

– Knowledge of Python and Deep Learning

– Conceptual understanding of various modules from deep learning

Introduction

In this article, we are going to see what is face recognition? and how it is different from face detection also? We will understand briefly the theory of face recognition and then jump to the coding section!! At the end of this article, you will be able to develop a face recognition program for recognizing faces in images!!!

Agenda of this Article

1. Overview of Face Detection

2. Overview of Face Recognition

3. Understand what is OpenCV

4. Implementation using Python

Overview of Face Detection

What if the machine is able to detect objects automatically in an image without human involvement? Let us see: Face detection can be such a problem where we detect human faces in an image. There might be slight differences in human faces, but after all, it is safe to say that there are specific features that are associated with all human faces. Various face detection algorithms are there but the Viola-Jones Algorithm is the oldest method that is also used today.

Face detection is generally the first step towards many face-related applications like face recognition or face verification. But, face detection has very useful applications. One of the most successful applications of face detection is probably “photo-taking”.

Example: When you click a photo of your friends, the camera in which the face detection algorithm has built-in detects where the faces are and adjusts focus accordingly.

Overview of Face Recognition

Now we have seen our algorithms can detect faces but can we also recognize whose faces are there? And what if an algorithm is able to recognize faces?

Generally, Face Recognition is a method of identifying or verifying the identity of an individual by using their face. Various algorithms are there for face recognition but their accuracy might vary. Here I am going to discuss with you that how we can do face recognition using deep learning.

Now let us understand how we can recognize faces using deep learning. Here we use face embeddings in which every face is converted into a vector. The technique of converting the face into a vector is called deep metric learning. Let me divide this process into three simple steps for better and easy understanding:

1. Face Detection:

The first task that we perform is detecting faces in the image(photograph) or video stream. Now we know that the exact coordinates/location of the face, so we extract this face for further processing.

2. Feature Extraction:

Now see we have cropped out the face from the image, so we extract specific features from it. Here we are going to see how to use face embeddings to extract these features of the face. As we know a neural network takes an image of the face of the person as input and outputs a vector that represents the most important features of a face! In machine learning, this vector is nothing but called embedding and hence we call this vector face embedding. Now how this will help in recognizing the faces of different people?

When we train the neural network, the network learns to output similar vectors for faces that look similar. Let us consider an example, if I have multiple images of faces within different timelapse, it’s obvious that some features may change but not too much. So in this problem, the vectors associated with the faces are similar or we can say they are very close in the vector space.

Up to this point, we came to know how this network works, let us see how to use this network on our own data. Here we pass all the images in our data to this pre-trained network to get the respective embeddings and save these embeddings in a file for the next step.

3. Comparing Faces:

  We have face embeddings for each face in our data saved in a file, the next step is to recognize a new image that is not in our data. Hence the first step is to compute the face embedding for the image using the same network we used earlier and then compare this embedding with the rest of the embeddings that we have. We recognize the face if the generated embedding is closer or similar to any other embedding.

Image source: https://cdn-media-1.freecodecamp.org/images/1*fpDngO6lM5pDeIPOOezK1g.jpeg

Understand What is OpenCV

In the Artificial Intelligence field, Computer Vision is one of the most interesting and challenging tasks. Computer Vision acts as a bridge between Computer Software and visualizations. Computer Vision allows computer software to understand and learn about the visualizations in the surroundings.

Let us understand an example: Based on the shape, color, and size that determines the fruit. This task is very easy for the human brain but in the Computer Vision pipeline, first, we need to gather the data, then we perform the data processing operations, and then we train and teach the model to understand how to distinguish between the fruits based on its size, shape, and color of the fruit.

Image source: https://martinapugliese.github.io/assets/posts_images/yolo-predictions.png

Nowadays, various packages are available to perform machine learning, deep learning, and computer vision problems. So far, computer vision is the best module for such complex problems. OpenCV is an open-source library. It is supported by different programming languages such as R, Python, etc. It runs probably on most platforms such as Windows, Linux, and macOS.

Advantages of OpenCV:

1. Open CV is free of cost and an open-source library.

2. Open CV is fast as it is written in C/C++ language as compared to others

3. With less system RAM, OpenCV works better.

4. It supports most of the operating systems like Windows, Linux, and macOS.

Implementation

In this section, we are going to implement face recognition using OpenCV and Python.

First, let us see what  libraries we will need and how to install them:

1. OpenCV

2. dlib

3. Face_recognition

OpenCV is a video and image processing library and it is used for image and video analysis, like facial detection, license plate reading, photo editing, advanced robotic vision, and many more.

The dlib library contains our implementation of ‘deep metric learning’ which is used to construct our face embeddings used for the actual recognition process.

The face_recognition library is super easy to work with and we will be using this in our code. First, remember to install dlib library before you install face_recognition.

The output of any face recognition applications using OpenCV will be like this:-

Image Source: https://spectrum.ieee.org/image/MzU1MTA4Mw.jpeg

To install OpenCV, dlib, and face recognition type the following snippets in the command prompt.

pip install opencv-python

conda install -c conda-forge dlib

pip install face_recognition

Now let’s do code!

Extracting features from Face

First, you need a dataset or even create one of your own. Just make sure to arrange all images in folders with each folder containing images of just one person.

Or you can download the dataset from this link.

Now, save the dataset in a folder the same as you are going to make the file. Here is the code and I have added comments wherever it is required:

from imutils import paths #imutils includes opencv functions
import face_recognition
import pickle
import cv2
import os

#get paths of each file in folder named Images
#Images here that contains data(folders of various people)
imagePath = list(paths.list_images('Images'))
kEncodings = []
kNames = []

# loop over the image paths
for (i, ip) in enumerate(imagePath):
# extract the person name from the image path
name = ip.split(os.path.sep)[-2]
# load the input image and convert it from BGR
image = cv2.imread(ip)
rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

boxes = face_recognition.face_locations(rgb,model='hog')
# compute the facial embedding for the any face
encodings = face_recognition.face_encodings(rgb, boxes)
# loop over the encodings
for encoding in encodings:
kEncodings.append(encoding)
kNames.append(name)

#save emcodings along with their names in dictionary data
data = {"encodings": kEncodings, "names": kNames}
#use pickle to save data into a file for later use
f = open("face_enc", "wb")
f.write(pickle.dumps(data))#to open file in write mode
f.close()#to close file

The next part is that we are going to see how to recognize faces from images.

How to Recognize Face in Images

The following script for detecting and recognizing faces in images. I have given comments next to code wherever it is required. So You will able to understand code efficiently.

import face_recognition
import imutils #imutils includes opencv functions
import pickle
import time
import cv2
import os

#to find path of xml file containing haarCascade file
cfp = os.path.dirname(cv2.__file__) + "/data/haarcascade_frontalface_alt2.xml"
# load the harcaascade in the cascade classifier
fc = cv2.CascadeClassifier(cfp)
# load the known faces and embeddings saved in last file
data = pickle.loads(open('face_enc', "rb").read())

#Find path to the image you want to detect face and pass it here
image = cv2.imread(Path-to-img)
rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
#convert image to Greyscale for HaarCascade
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
faces = fc.detectMultiScale(gray,
scaleFactor=1.1,
minNeighbors=6,
minSize=(60, 60),
flags=cv2.CASCADE_SCALE_IMAGE)

# the facial embeddings for face in input
encodings = face_recognition.face_encodings(rgb)
names = []
# loop over the facial embeddings incase
# we have multiple embeddings for multiple fcaes
for encoding in encodings:
#Compare encodings with encodings in data["encodings"]
#Matches contain array with boolean values True and False
matches = face_recognition.compare_faces(data["encodings"],
encoding)
#set name =unknown if no encoding matches
name = "Unknown"
# check to see if we have found a match
if True in matches:
#Find positions at which we get True and store them
matchedIdxs = [i for (i, b) in enumerate(matches) if b]
count = {}
# loop over the matched indexes and maintain a count for
# each recognized face face
for i in matchedIdxs:
#Check the names at respective indexes we stored in matchedIdxs
name = data["names"][i]
#increase count for the name we got
count[name] = count.get(name, 0) + 1
#set name which has highest count
name = max(count, key=count.get)
# will update the list of names
names.append(name)

# do loop over the recognized faces
for ((x, y, w, h), name) in zip(faces, names):
# rescale the face coordinates
# draw the predicted face name on the image
cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2)
cv2.putText(image, name, (x, y), cv2.FONT_HERSHEY_SIMPLEX,
0.75, (0, 255, 0), 2)
cv2.imshow("Frame", image)
cv2.waitKey(0)

Output

Conclusion

Now you can see this lead us to the end of this article where we learned all about face recognition. Hope you liked this article. Thank You!