Detect the Faces in the Image using the Mediapipe Library

Aman Preet Gulati 06 Apr, 2022 • 7 min read

This article was published as a part of the Data Science Blogathon.

Introduction

In this article, we will learn to detect the faces in the image using the Mediapipe library we might see different algorithms and models that could perform the same task. Here we will discuss the detection pipeline with the help of the Mediapipe library and undergo code by code explanation. Before moving forward please don’t confuse yourself between face detection and facial landmarks detection as in face detection it detects your overall face and draws the bounding box but in facial landmarks, it detects the features of the face i.e. eyes, nose, and mouth though we will also try to detect some landmarks here as well this is not the optimal way we will do that only because the Mediapipes face detection algorithm offers us the same.

Mediapipe Library
Image Source: Adobe Stock

Application of Face Detection

  1. Face recognition: Face detection is just one step before face recognition because before recognizing the face we have to first detect it.
  2. Face emotions detection: Face emotions recognition is also one of the widely applied use cases of face detection.
  3. Lock screen: So whenever we use the lock screen functionality of android/apple phones then that application first detects or recognizes our face in that area face detection is the first step to be followed.

So let’s build our face detection system this time with the help of the Media pipe library.

Import the Libraries

The very first step will be to import all the necessary libraries.

import cv2
import numpy as np
import mediapipe as mp
import matplotlib.pyplot as plt

Face Detection with Mediapipe Library

So this time we will be performing the face detection functionality with Mediapipe’s face detection model when we try to get into the depth of this model we can find out that it is completely based on BlazeFace which is one of the face detection algorithms and the main reason that it is used is because of its lightweight and very accurate predictions when it comes to face detection even that algorithm is derived from the MobileNetV1/V2 state of the art model. The frame per second of this model is 200-1000 depending on the specification of the devices.

Our next step is to initialize the face detection model of the Mediapipe library.

Mediapipe Library

Image Source: CronJ

 

Before using Mediapipe’s face detection model we have to first initialize the model for that we will be using the simple syntax as mp.solution.face_detection and after initializing the model we will call the face detection function with some arguments. Now we will discuss those arguments:

  • model_selection: This argument takes the real integer value only in the range of 0-1 i.e. this model will take the integer value as either 1 or 0. Let us discuss these two types of models.
    1. 0 type model: When we will select the 0 type model then our face detection model will be able to detect the faces within the range of 2 meters from the camera.
    2. 1 type model: When we will select the 1 type model then our face detection model will be able to detect the faces within the range of 5 meters. Though the default value is 0.
  • min_detection_confidence: This argument also takes the integer value but in the range of [0.0,1.0] and the default value for the same is 0.5 which is 50% confidence i.e. when our model will be detecting the faces it should be at least 50% sure that the face is there otherwise it won’t detect anything.

Though we have initialized our model and it is ready to detect the faces but having said that we need to visualize the results too for that we will use the drawing_utils function to see the result in the images/frames.

mp_face_detection = mp.solutions.face_detection

face_detection = mp_face_detection.FaceDetection(model_selection=0, min_detection_confidence=0.5)

mp_drawing = mp.solutions.drawing_utils

Code breakdown

  1. The very first step will be to initialize the Mediapipe’s face detection model.
  2. After initializing the model we will call the face detection function by using the relevant parameters and their values.
  3. At the last it is also necessary to see the results for that we will use the drawing_utils function to draw the results on the image/frames.

Reading the image on which we will perform the face detection

So after the initialization part, we will read a particular image on which we will apply our face detection model for that we will be using the cv2.imread a function that will help us to read the image but during this step it will convert the image format from RGB to BGR.

sample_img = cv2.imread('media/sample.jpg')

plt.figure(figsize = [10, 10])

plt.title("Sample Image");plt.axis('off');plt.imshow(sample_img[:,:,::-1]);plt.show()

Output:

Mediapipe Library
Image Source: iStock

Code breakdown

  1. As discussed here we have read the image after providing its path in the function.
  2. Now, we will use the matplotlib function to plot the image as I’m working with Jupyter notebook so cv2’s show function will not work in this environment hence with Matplotlib’s show function we will plot the image before that we will also use the figure size with the help of figure function of matplotlib.

Face Detection using the Process Function

So we have read the image as well now comes to the main task where we will use our face detection model and implement its functionality on our sample image for that we will be using the process() function from FaceDetection class. This function will return us the major six-coordinate for every face it will detect in the image/frame. Those six coordinates are as follows:

  1. Right Eye
  2. Left Eye
  3. Nose Tip
  4. Mouth Center
  5. Right Ear Tragion
  6. Left Ear Tragion

So after getting these 6 points we will be able to plot the bounding box on the image/frame but we will only plot two major key points so that the output is clear and there should be no extra points over the image which could make it congested.

face_detection_results = face_detection.process(sample_img[:,:,::-1])

if face_detection_results.detections:

    for face_no, face in enumerate(face_detection_results.detections):

        print(f'FACE NUMBER: {face_no+1}')
        print('==============================')

        print(f'FACE CONFIDENCE: {round(face.score[0], 2)}')

        face_data = face.location_data

        print(f'nFACE BOUNDING BOX:n{face_data.relative_bounding_box}')

        for i in range(2):

            print(f'{mp_face_detection.FaceKeyPoint(i).name}:')
            print(f'{face_data.relative_keypoints[mp_face_detection.FaceKeyPoint(i).value]}')

Output:

 

Output | Mediapipe Library

Code breakdown

  1. Firstly we will change the format of the image from the BGR format to its RGB version.
  2. Then, using the if condition we will first detect whether the face is found in the image or not for that we will be using the detection attribute of face_detection.
  3. Then with the help of enumerate function and for loop, we will iterate over the faces found in the image.
  4. Now we will print out the total number of faces found in the image.
  5. We will also print out the confidence value i.e. how much our model is confident that the face is detected or not.
  6. Then we will get the bounding box and major points which we will be displayed on the image.
  7. Now note one thing in the inner for loop we are just iterating over 2 points because we only want to see the two points on the image from those major 6 points and then finally we will display those points – the normalized version of the coordinates.

Visualizing the Results

img_copy = sample_img[:,:,::-1].copy()

if face_detection_results.detections:

    for face_no, face in enumerate(face_detection_results.detections):

        mp_drawing.draw_detection(image=img_copy, detection=face, 
                                 keypoint_drawing_spec=mp_drawing.DrawingSpec(color=(255, 0, 0),
                                                                              thickness=2,
                                                                              circle_radius=2))
fig = plt.figure(figsize = [10, 10])

plt.title("Resultant Image");plt.axis('off');plt.imshow(img_copy);plt.show()

Output:

 

Code breakdown

  1. First, we will create a copy of the image using copy the method so that we won’t lose the original preprocessed part of the image.
  2. Then again we will first check if the faces are found or not then only we will proceed further.
  3. If the faces were found then we will iterate over each face via for loop and enumerate the function.
  4. Now instead of printing the points, we will draw the points on the sample image using the draw_detection function with relevant parameters as discussed.
  5. Now we will set the size of the figure (here image) using Matplotlib’s figure function.
  6. Finally, we will display the image with the bounding box and the points as well.

End Notes

  1. The very first takeaway from this article is that we learned a different algorithm to detect the faces i.e. Mediapipe library to detect the faces.
  2. We understood the process function which is the backbone of the media pipe face detection algorithm and we also had an in-depth discussion on the same.
  3. And at the last, we have visualized the results as well and not only the face but also some landmarks like Eyes, ear, mouth, and the nose the results for the landmarks might not be that accurate for different angles but it is very handy for the frontal face.

Here’s the repo link to this article. Hope you liked my article on Face detection using the Mediapipe library. If you have any opinions or questions, then comment below.

Read our blogs on various predictions using Machine Learning.

About the Author

Greeting to everyone, I’m currently working in TCS and previously, I worked as a Data Science Analyst in Zorba Consulting India. Along with full-time work, I’ve got an immense interest in the same field, i.e. Data Science, along with its other subsets of Artificial Intelligence such as Computer Vision, Machine Learning, and Deep learning; feel free to collaborate with me on any project on the domains mentioned above (LinkedIn).

The media shown in this article is not owned by Analytics Vidhya and are used at the Author’s discretion. 

Aman Preet Gulati 06 Apr 2022

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Responses From Readers

Clear

Computer Vision
Become a full stack data scientist