Story

We develop our own drowsiness detector that allows us to efficiently prevent traffic accidents by issuing an audible alarm that establishes the alert pattern for the driver.

What do we need?

We connected our camera to the raspberry pi 4 where the computer vision software will be installed to detect drowsiness.

• Python is the chosen programming language, a version greater +3.7
• Scipy is a python lib for scientific calculations we used the distance module to calculate the Euclidean distance that we implemented in the software logic. 

from scipy.spatial import distance

• OpenCV (cv2) is the graphic library that Intel created for the management of images in computer vision. The reading and transformation of the original frames is done with this library.

import cv2

• Imutils It is a python lib that, in union with OpenCV, handles conventions to transform or access arrays of well-defined zones, In our case, it has used to obtain the matrices associated with the eyes after that the face has been detected through the output provided by face detection.

from imutils import face_utils 

• Numpy is a python lib that allowed us with its multiple functions to create, transform, store and apply different operations of the matrix algebra to adapt the section or complete image and obtain the result or transformation that the algorithm required.

import numpy as np

• Pygame is a lib created to make video games in python. We used the graphic and sound alerts offered by pygame, it allowed us to display the message on the video output and trigger the alarm sound.

import pygame 

• Dlib is a lib with a wide variety of utilities ranging from networks to machine learning, In our case we used pre-trained model for face detection with the haar-cascade model
  

import dlib

What is the algorithm?

Step by Step

1.- We activate the camera to obtain the necessary inputs, in this case we used the camera of our smartphone to record the video and then send it in.mp4 format to the folder where all our programming logic is.

2.- Through the shape_predictor() and get_frontal_face_detector() functions of dlib and its pre-trained face detection model, we extract the coordinates associated with the face detected in the video.

3.- All the coordinates (x, y) that make up the facial features are obtained as a numeric matrix by means of the shape_to_np() function of face_utils

4.- Once the coordinates of the face was extracted, only the coordinates that define the eyes are filtered.

5.- A new numerical array is obtained with the coordinates that represent the reference points associated with the location of the eyes.

6.- Extracted us the 6 reference points that determine the degree of opening of an eye according to the model of Tereza Soukupová and Jan Čech then we proceed to calculate the eye aspect radius.

7.- Established us a minimum opening threshold in the eyelid, once we reach values lower than the value set in the threshold, we are in the presence of an eye with a high probability of being closed, in our case we set the threshold at Y (we will delve into this later )

8.- We establish a minimum amount of consecutive frames that share the opening threshold of the eye with high probabilities of being closed, in our case we had X FRAMES ( we will delve into this later ).

9.- If the two previous conditions are met together, the alarm is triggered that allows setting the alert for drowsiness danger.

How to know when there is drowsiness?

To determine when the eyes are closed or open, we rely in part on the algorithm proposed by Tereza Soukupová and Jan Čech in their paper "Real-time blink detection using facial marks" the strange numerical coordinates that represent the points that define the beginning. and horizontal end of each eye (Point P1 and P4) and the points that indicate the current position of the lower eyelid (P5 and P6) and the mobile eyelid (P2 and P3), understanding...