Hypovigilance Detection Based on Eyelids Behavior Study

—The supervision of people in special circumstances, having as their purpose detecting risk situations, is an area which has many applications. In this context we are interested, in this paper, the problems of driver drowsiness detection based on video analysis. We describe a new approach to detect reduced alertness automatically, to prevent the driver. Unlike many other works, our noninvasively approach is based only on the analysis of geometric features captured form faces video sequence without any electronics elements related to human.


INTRODUCTION I.
Drowsiness is the risk of falling asleep for a moment with eyes closed and eyes open at times which is an intermediate state between waking and sleeping. This state is involuntary and is accompanied by reduced alertness. A study of the SNCF [3], the characteristic signs of drowsiness are manifested by behavioral signals such as yawning, decreased reflexes, heavy eyelids, itchy eyes, a desire to close eyes for a moment, a need to stretch, a desire to change positions frequently, phases of"micro sleeps" (about 2-5 seconds), a lack of memory of the last stops and trouble keeping head up.
In literature, many systems based on video analysis have proposed for drowsiness detecting [25] [10] [13] [23] [24]. Special attention is given to the measures related to the speed of eye closure. Indeed, the analysis of the size of the iris that changes its surface according to its state in the video allows the de-termination of the eye closure [6] [11] [13] Other work is based on detecting the distance between the up-per and the lower eyelids in order to locate eye blinks. This distance decreases if the eyes are closed and in-creases when they are open [14] [16] [14] [17].
These so-called single-variable approaches can prevent the driver in case of prolonged eye closure, of its reduced alertness. The duration of eye closure used as an indication varies from one work to another. Sarbjit [11] considers that a person is asleep if the eyes remain closed for a period of 5 to 6 seconds; in this case, we speak of total sleep. On the other side, sleep microphones are detected if the driver goes through a state of sleep for a short time (2-3 seconds). For Horng [15], the driver is considered dozing if he / she close their eyes for 5 consecutive images. Hongbiao [14] estimated that the reduced alertness is determined if the distance between the eyelids is less than 60% for a period of 6.66 seconds.
Yong [17] [16] achieved 100% as a correct detection rate. All the last work calculates the results by a study on subjects varying in number from two to ten individuals (four individuals for Horng [6], five subjects for Hiroshi [4], ten people for Tnkehiro [12] and only two for Yong [17]). The second type of approach is called multi-variable. In this context, the maximum speed reached by the eyelid when the eye is closed (velocity) and the amplitude of blinking calculated from the beginning of blink until the maximum blinking are two indications that have been studied by Murray [8] . The latter shows that the velocity amplitude ratio (A/PCV) is used to prevent the driver one minute in advance. Takuhiro [14] uses an infrared camera and suggests five levels of vigilance namely non-drowsy, slightly drowsy, sleepy, rather sleepy, very sleepy and asleep. Takuhiro proposes several criteria: The percentage of eye opening, the number of blinking, the duration of eye closure, the blinking interval and the surface of eye opening. This work has been tested using a base of ten subjects. A correlation rate that is greater than 0.90 is achieved. Picot presents a synthesis of different sizes as the duration to 50%, the PERCLOS 80%, the frequency of blinking and the velocity amplitude ratio. Picot [9] shows that its criteria are more relevant to the detection of drowsiness. These variables are calculated every second on a sliding window of the length of 20 seconds. They are fused by fuzzy logic to improve the reliability of the decision. This study shows a percentage of 80% of good detections and 22% of false alarms. The mono-variable approach enables us to detect the state of drowsiness, but in a very advanced stage. We, hence, focus on studying the characteristics which aim at foretelling the driver, about his fatigue, before she falls asleep, by analyzing the speed of closing his eyes.
In the case of multi-variable approaches, we find that some methods are based on the analysis of the EOG signal. This kind of analysis requires technical cooperation between the hardware and the driver. Moreover, these methods need the use of wide range of parameters, which calls for more data for learning. Nevertheless, video-based approaches, rest on the segmentation of the iris of the eye PAPER HYPOVIGILANCE DETECTION BASED ON EYELIDS BEHAVIOR STUDY so as to extract the features for the subsequent steps. The segmentation of the iris is calculated from the difference of the images, in the case of using infrared cameras. Still, the drawback of such a method lies in the noise sensitivity of the luminance. In this context, the Hough Circular transform method is used to localize the iris. This method shows sturdiness in the face of the desired shape, an ability to adapt even to images with poor or noisy quality as well as an identification of all directions due to the use of a polar description.
Based on the afore-mentioned remarks, we propose in this paper, an approach that determines the state of drowsiness by analyzing the behavior of the driver's eyes from a video.
This paper presents an approach for driver drowsiness detection [22] by studying the behavior of conductor eyes in real time by an RGB camera (figure 1). This approach requires a critical step presumed through the automatic face detection, first, and the detection of the box that encompasses both eyes, in the step that follows. In order to come to delight the face and the eyes, our approach exploits the object detector of Viola and Jones that is about a learning technique based on Haar features. This method [15] uses three concepts: the rapid extraction of features using an integral image, a classifier based on Adaboost and the implementation of a cascade structure. Figure 2 shows the detection result of the face and the eyes. The face and the eyes located by the Haar features are then used to initialize the tracking module. This module is based on the CAMSHIFT [18] algorithm. The choice of this algorithm is argued by its robustness for real-time systems, the invariance to scale and rotation, treatment of occlusions and finally by its insensitivity to object distortions. The CAMSHIFT algorithm steps can be summarized as follows.

CAMSHIFT Algorithm
1 Select the interest region that contains the object to follow. In our case the face or eyes.
2 Calculate the color histogram of interest region.
3 Calculate the distribution probability of the color histogram.
4 Find the gravity center of the image probability by the Mean-Shift algorithm.
5 Center the search window from the point calculated in step 4 and adjust its size.
6 Return to step 4 until the convergence of the system.
CAMSHIFT algorithm is based on the Mean-Shift algorithm [19]. The Mean-Shift algorithm calculates the center position of interest region. In addition, the CAMSHIFT algorithm calculates the center, the size and orientation of the object. Figure 3 shows some results of detection and face tracking in different positions and lighting conditions.

Iris and both eyelids Detection B.
With reference to the observation of the eye, we note that human eyes are characterized by horizontal contours representing the eyelids and the wrinkles or vertical contours as the ones of the iris. The application of twoscale Haar wavelet allows extracting the vertical, horizontal and diagonal contours. The vertical contours are used in the localization of the iris following application of the Circular Hough Transform. The use of the wavelet allows us to highlight the contours that we want to spot frequently. In our case, the scale of the second rate improves the contours of the iris and the two lids which are going to be detected.
Edge extraction based on 2D Haar wavelet 1) The Haar wavelet [1] allows us to split the image to find the vertical and horizontal details for the detection of the iris and both eyelids. The wavelet transform is characterized by its multi resolution analysis. It is a very effective tool for noise reduction in digital image. We can keep only the most representative edges. This type of PAPER HYPOVIGILANCE DETECTION BASED ON EYELIDS BEHAVIOR STUDY analysis is allowed by the multi-resolution. Figure 4 shows the result comparison between the Haar wavelet decomposition and the edge detection with Sobel for the eye image.

Iris and the two eyelids detection based on 2) Circular Hough Transform
Generally, the Hough transform [2] has two spaces. The XY space and parameter space which varied according to the detected object. The parameter space of the circle is represented by equation 1.
Were A and B are the coordinates of the center of the circle and R is the radius. For reasons of simplification, we can consider that the radius varies between two values in order to reduce the complexity of the algorithm. The equation of the circle can be written in the polar system as follows: The process of the research for circles by the application of the Hough transform follows several stages [2]. First, we apply a contour detection algorithm for the decomposition of the Haar wavelet of second rate. For each detected contour pixel, we draw a circle with the desired radius and we suppose that this point is the center of this circle (figure 5). This circle is drawn in the space parameter such as the X axis represents the values of a, Y represents the values of b and Z represents the rays. In the case where a point has coordinates belonging to the circle drawn, we raise its position in the accumulator by one. This accumulator has already the same size parameter spaces. Each coordinate (x, y) of the image becomes a cone (figure 6) in parameter space (a, b, r). Geometric features extraction C.
With reference to the detection of the iris, the upper and lower eyelid, we can extract geometric features able to characterize the driver drowsiness state.

PAPER HYPOVIGILANCE DETECTION BASED ON EYELIDS BEHAVIOR STUDY
We note that, ! ! !and ! ! are calculated from the intersection between the straight line which passes through ! and perpendicular to the line (! ! ! ! ) and the two circles ! ! !and!! ! , where the equation of one of the two circles is follow as: And the perpendicular to line (! ! ! ! ) is calculated by applying the equation 5.
By replacing ! ! into ! ! after development we obtain the equation 6.
The straight line (! ! ! ! ) is determined after the detection of two points ! ! !and ! ! (Figure 8) which represent the intersection of two circles ! ! !and ! ! such as the squared And the squared equation of circle ! ! is written as follow: The difference between ! ! ! and ! ! ! allows to determinate the tow points ! ! and ! ! by the equation 9.
Such us: While: And: The use of the point !! as a reference calculation, not the point ! of the center of the iris, allows us to avoid the uncertainties of calculation. These uncertainties are due to the positioning of the point ! above the upper eyelid of the eye, resulting in miscalculations of the feature ! !! ! ( Figure  9). Experimental study, analysis of eye closure and result D.
of blink detection In this section, we describe the experimental studies we conducted to validate the two features !! !! and ! !" previously proposed. In order to produce realistic data, a human subject is placed in front of our system to simulate different possible movements of the head, the eyelids and the positions of the iris, probably related to different states of fatigue. This experiment consists of studying the temporal variation of both features and normalized the initial state of the eye (Equation 13).
The initial value ! ! is calculated at the beginning of the algorithm when the eyes were open by 75% ( Figure 10).   Experimentally and after analyzing the video recordings, we found that the derivative of the signal requires a low pass filter to eliminate noise. The selecting of a period considered as blink, is located in the case where !! ! ! falls below (closure) and above (opening) of a well-defined threshold. Sharabaty [20] showed that the maximum duration of a normal blink is 0.5 seconds whereas above this value is considered prolonged closure. In our case the normal or long blink is validated if it satisfies the two previous conditions and its period must exceed 0.15 seconds. Figure 13 shows blink validated with interrupted lines and not validated (between frames 479 and 490) for the signal of !! ! ! . The purpose of the second experiment is to study the difference in closing speed of a person's eyes in a normal state and in another drowsy one. Figure 14 shows the average time of closing of the eyes of a normal person and of a sleepy one. Generally, for a tired individual, the eye closure is slower than that of a vigilant person. This measure can be used as a factor to determinate the level of fatigue. The cumulative change (Equation 14) allows calculation from the period when the individual has their eyes closed.
Fatigue states are characterized by a continuous segment because the evolution of !! ! ! !is too low ( Figure  15). In brief, the three conducted experiments show the difference between an alert individual and a sleepy one by analyzing the variation of the features ! !! and!! !" . Generally, the drowsiness states detection is processed by the location of blinking first, then by studying the speed of blinking and finally by calculating the duration of eye closure. The driver is considered in a state of drowsiness if the speed of eye closure exceeds 1 second or the duration of eye closure is greater than 2 seconds. We tested our approach on 10 subjects to determine the detection results of the iris by the Circular Hough Transform (CHT) as shown in Table 1. The result of the correct average rate is calculated from the average of correct rate of detection (CRT) by circular Hough transform (Equation 15). This value reached 96.45% as a percentage of detection.
This paper presents an approach to the detection of reduced alertness, based on video analysis. Our system uses a study of the eyes by analyzing the video of several topics.
The steps of detecting drowsiness consist firstly of locating a driver face and eyes by applying Haar features. The circular Hough transform allows the detection of the