In this article, we will learn how to make a real-time blink detector application using computer vision. Also, we will be using more libraries and mathematics to build such an application while going through a complete pipeline and code-by-code analysis.
Application of Blink Detection Application
1) Driver drowsiness detection: As the name suggests, this application is very useful in building real-world applications like detecting whether the driver is sleepy or not while detecting the eye moment or blink.
2) Iris tracking: This is another use case while we can also track the iris movement for building an AR kind of application.
3) Virtual gaming: We are in the age of virtual reality evolution, and by far, mostly VR-powered games are either hand or body movement driven, but we can also build eye movement driven games.
How will we achieve this?
Firstly, we will extract only those points located near our eyes to get the enclosed area of the eye, and then we will find out the EAR (Eye Aspect Ratio), which will help us determine the blink as an event has occurred or not.
There is a total of 6 XY coordinates for an eye which starts from the left corner of the eye, and then from that position, it will go in the clockwise direction.
There will be a relation between the height and width of these coordinates.
Let’s Start by Importing the Required Libraries
from scipy.spatial import distance as dist
from imutils.video import FileVideoStream
from imutils.video import VideoStream
from imutils import face_utils
import numpy as np
import imutils
import time
import dlib
import cv2
The functionality of each library:
distance: This library will help us to find the Euclidean distance and remove some burden of applying mathematical calculations.
FileVideoStream: This library will help us to stream the videos from the file explorer i.e. video file (.mp4 or another type).
VideoStream: This library will help us to stream real-time video from the webcam.
face_utils: This library will be responsible for grabbing the face landmarks (here eyes).
NumPy: This library will help us to perform some other mathematical operations like arrays.
time: This library is to get the system time or to get delayed i.e. sleep function.
dlib: This is the heart of this application as this library will help us get access to 68 landmarks of the face in real-time.
cv2: Computer vision library to perform some image processing techniques.
Function to Calculate Eye Aspect Ratio (EAR)
def eye_aspect_ratio(eye):
A = dist.euclidean(eye[1], eye[5])
B = dist.euclidean(eye[2], eye[4])
C = dist.euclidean(eye[0], eye[3])
ear = (A + B)/ (2.0 * C)
return ear
Code breakdown:
1) First, we will get the Euclidean distance between the 2 coordinates of the eyes.
2) While grabbing the coordinates, we will first have the vertical eye landmarks.
3) Then we will have the horizontal eye landmarks using the same algorithm.
4) After grabbing the coordinates, we will calculate the Eye Aspect ratio.
5) Then, at last, we will return the EAR.
Define Constants
EYE_AR_THRESH = 0.3
EYE_AR_CONSEC_FRAMES = 3
Eye aspect ratio constant: This constant value will act as a threshold value to detect the blink.
Count of frames:: This constant value is the threshold value for the number of consecutive frames.
Initializing the Variables
COUNTER = 0
TOTAL = 0
Counter: This value will denote the total number of consecutive frames that will have the threshold value less than the EYE ASPECT RATIO constant.
Total: This value will make a count of the total number of blinks in a certain number of frames.
Initialize the Dlib’s Face Detector
print("Loading the dlib's face detector")
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
Output:
Loading the dlib's face detector
detector: Here we will initialize the dlib library (frontal face detector).
predictor: Now we will use the shape_predictor method to load the .dat file and predict the landmarks accordingly.
Firstly we are grabbing the coordinates values of the left eye using the face_utils function.
Secondly, we will do the same for right_eye.
Loading the Video/Real-time Streaming
print("Starting the video/live stteaming")
vs = FileVideoStream("Video.mp4").start()
fileStream = True
# vs = VideoStream(src = 0).start() # run this line if you want to run it on webcam.
# vs = VideoStream(usePiCamera = True).start()
fileStream = False
time.sleep(1.0)
Output:
Starting the video/live stteaming
Code breakdown:
While using FileVideoStream we will initialize the object with the video file location and then start() the same.
Setting the fileStream value as True after successful streaming of file (video).
If we want a real-time streaming we will be using the VideoStream(src=0).start().
Then we will be using the sleep function from the time library to delay the frame.
Firstly we will loop over the video streaming, and along with that, we will also check whether there are any more frames left in the buffer.
We will first pick all the frames from the video/live streaming, then we will resize them to our desired dimensions, and at last, convert them to grayscale.
Then using the detector function, we will detect the faces.
Now, with the help of the predictor function, we will detect the facial landmarks and then convert them into a NumPy array.
Then we will first grab the left, and right eye coordinates then will compute the Eye Aspect Ratio and do the average by 2, i.e., 2 eyes.
Then we will compute the convex hull for both eyes so that we can visualize the eyes by drawing methods using contours.
Now, we will check that our calculated EAR should be below the threshold value so that we can increase the blink counter.
Else if the EAR is greater than the threshold value, then we will increase the counter of Total frames so that we can check other frames as well. If the eyes were closed for certain frames, then also we will increase the number of blinks.
Then using the put text method, we will draw the number of blinks in each frame and also the Eye Aspect Ratio (EAR) value
Then, at last, using the show function, we will show the mainframe, and along with that, we will also code to exit from the loop, i.e., q, and at last, for the clean-up process, we will destroy all the windows.
Conclusion
Firstly we saw the real-world application of blink detection application then we saw what we would be doing in a nutshell.
The main key takeaway from this article is to segment the eyes by using their coordinates.
We have also learned about the concept of Euclidean distance and its formulae using the specific library.
Along with that, we also came across the concept of Eye Aspect Ratio (EAR), which is the soul of this application.
We also learned how the dlib library could detect the landmarks of the face and, along with that, read the video files as well as live streaming.
Here’s the repo link to this article. Hope you liked my article on the Blink detection application using computer vision. If you have any opinions or questions, then comment below.
Read on AV Blog about various predictions using Machine Learning.
The media shown in this article is not owned by Analytics Vidhya and are used at the Author’s discretion.
A verification link has been sent to your email id
If you have not recieved the link please goto
Sign Up page again
Loading...
Please enter the OTP that is sent to your registered email id
Loading...
Please enter the OTP that is sent to your email id
Loading...
Please enter your registered email id
This email id is not registered with us. Please enter your registered email id.
Don't have an account yet?Register here
Loading...
Please enter the OTP that is sent your registered email id
Loading...
Please create the new password here
We use cookies on Analytics Vidhya websites to deliver our services, analyze web traffic, and improve your experience on the site. By using Analytics Vidhya, you agree to our Privacy Policy and Terms of Use.Accept
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.