A Real Time Intelligent Driver Fatigue Alarm System Based On Video Sequences

P.Ratnakar.et al. Int. Journal of Engineering Research and Applications www.ijera.com
ISSN: 2248-9622, Vol. 6, Issue 4, (Part - 3) April 2016, pp.53-59
www.ijera.com 53|P a g e
A Real Time Intelligent Driver Fatigue Alarm System Based On
Video Sequences
1
P.Ratnakar, 2
K.S.N.Raju, 3
M.Satyanarayana
1,2,3
Department of Electronics and Communication and Engineering, M.V.G.R.C.O.E., A.P., India
ABSTRACT
In automobiles advanced controllers are equipped to control all the data. In this work a new technology is
considered as driver fatigue detection system. Developing intelligent system to prevent car accidents and can be
very effective in minimizing accident death toll. One of the factors which play an important role in accidents is
the human errors including driving fatigue. Relying on new smart techniques; this system detects the signs of
fatigue and sleepiness in the face of the person at the time of driving by capturing the video sequences of the
driver. Then, the frames are transformed from YUV space into RBG spaces. It is one of the inexpensive and
unobtrusive method where face, eyes are detected and edge detection and histogram normalization are
performed on the captured frames using MATLAB as a tool.The face area is separated from other parts with
high accuracy in segmentation, low error rate and quick processing of input data distinguishes this system from
similar ones.
Key Words Used: Eye Detection, Edge Detection, Face Posture, Face Detection, Viola Jones Algorithm
I. INTRODUCTION
Image processing usually refers to digital
image processing. Image processing is any form of
signal processing for which the input is an image,
such as a photograph or video frame; the output of
image processing may be either an image or, a set
of characteristics or parameters related to the
image.Most image-processing techniques involve
treating the image as a two-dimensional signal and
applying standard signal-processing techniques to
it.
Driving is a complex task where the
driver is responsible of watching the road, taking
the correct decisions on time and finally
responding to otherdrivers’ actions and different
road conditions.
Vigilance is the state of wakefulness and
ability to effectively respond to external stimuli. It
is crucial for safe driving. Among all fatigue
related accidents, crashes caused by fell-asleep-
drivers are common and serious in terms of injury
severity. According to recent statistics driver
fatigue or vigilance degradation is the main cause
of 17.9% of fatalities and 26.4% of injuries on
roads. Vigilance levels degrade mainly because of
sleep deprivation, long monotonous driving on
highways and other medical conditions and brain
disorders such as narcolepsy. The study states that
the cause of an accident falls into one of the
following main categories:
(1) Human
(2) Vehicular
(3) Environmental.
The driver’s error accounted for 93% of
Decision errors refer to those that occur as
a result of a driver’s improper course of action or
failure to take action. A recognition error may
occur if the driver does not properly perceive or
comprehend a situation. To perform all these
activities in time and accurately its necessary that
driver must be vigilant. The aim of this paperis to
develop a computer vision method able to detect
and track the face of a driver in a robust fashion,
also determine the status of the eyes, and with the
highest precision possible. It is to serve as the
bases of an automatic driver fatigue monitoring
system
II. USES AND APPLICATIONS
 It is non intrusive system
 Used to reduce death troll caused by driver
fatigueness or negligence
RESEARCH ARTICLE OPEN ACCESS
the crashes.The other two categories of causative
factors were cited as 13% for the vehicle factor and
3% for environmental factors. It is important to
note that in some cases; more than one factor was
assigned as a causal factor. The three main
categories are related among each other, and
human error can be caused by improper vehicle or
highway design characteristics. The recognized
three major types of errors within the human error
category:
(1) Recognition
(2) Decision
(3) Performance

III. TECHNIQUES
Previously this method of fatigue
detection was done through medical techniques by
placing electrodes on the driver body which
receives the signals from the brain based on which
it is determined whether the driver is feeling
drowsy or not. This method proved to be very
inconvenient for the person driving because of the
wires.
In those techniques some traditional techniques
which are adopted from the medical stream are:
 Electroencephalograms(EEG)
 Electrooculography (EOG)
 Electromyogram(EMG)
ExistedReal Time Systems
A real-time system is one in which the
correctness of the computations not only depends
on their logical correctness, but also on the time at
which the result is produced. That is, a late answer
is a wrong answer. For example, many embedded
systems are referred to as real-time systems. Cruise
control, telecommunications, flight control and
electronic engines are some of the popular real-
time system applications where as computer
simulation, user interface and Internet video are
categorized as non-real time applications.
Electronic Engineis a real time system
Consider a computer-controlled machine on the
production line at a bottling plant. The machine's
function is simply to cap each bottle as it passes
within the machine's field of motion on a
continuously moving conveyor belt. If the machine
operates too quickly, the bottle won't be there yet.
If the machine operates too slowly, the bottle will
be too far along for the machine to reach it.
Stopping the conveyor belt is acostly operation as
the entire production will come to halt. Thus the
range of motion of the machine coupled with the
speed of the conveyor belt establishes a window of
opportunity for the machine to put the cap on the
bottle. This window of opportunity imposes timing
constraints on the operation of the machine.
Software applications with these kinds of timing
constraints are termed as real-time applications.
Here, the timing constraints are in the form of a
period and deadline.
IV. FACE DETECTION
In geometric or feature based methods,
facial features such eyes, nose, mouth and chin are
detected. Properties and relations such as areas,
distances, and angles between the features are used
as descriptors of faces. Although this class of
economical and efficient in achieving data
reduction and is insensitive to variations in
illumination and viewpoint, it relies heavily on the
extraction and measurement of facial features.
Unfortunately, feature extraction and measurement
techniques and algorithms developed to data have
not been reliable enough to cater to this need.
Presently available face detection methods
mainly rely on two approaches. The first one is
local face recognition system which uses facial
features of a face e.g. nose, mouth, eyes etc. to
associate the face with a person. The second
approach or global face recognition system use the
whole face to identify a person.
A facial recognition system is a computer
application for automatically identifying or
verifying a person from a digital image or a video
frame from a video source. One of the ways to do
this is by comparing selected facial features from
the image and a facial database.
Fig 1: Detecting of Face
V. EYE DETECTION
The use of template matching is necessary
for the desired accuracy in analysing the user’s
blinking since it allows the user some freedom to
move around slightly. Though the primary purpose
of such a system is to serve people with paralysis,
it is a desirable feature to allow for some slight
movement by the user or the camera that would not
be feasible if motion analysis were used alone. The
normalized correlation coefficient, also
implemented in the system proposed by is used to
accomplish the tracking
Fig 2: Detection of Eye

VI. EYE POSITION DETECTION
Naturally the first step is analysing the
blinking of the user is to locate the eyes. To
accomplish this the difference in image of each
frame and the previous frame is created and then
thresholded, resulting in a binary image showing
the regions of movement that occurred between the
two frames.
Next, a 3x3 star-shaped convolution
kernel is passed over the binary difference image in
an Opening morphological operation. This
functions to eliminate a great deal of noise and
naturally-occurring jitter that is present around the
user in the frame due to the lighting conditions and
the camera resolution, as well as the possibility of
background movement. In addition, this Opening
operation also produces fewer and larger connected
components in the vicinity of the eyes (when a
blink happens to occur), which is crucial for the
efficiency and accuracy of the next phase.
A recursive labelling procedure is applied
next to recover the number of connected
components in the resultant binary image. Under
the circumstances in which this system was
optimally designed to function, in which the users
are for the most part paralyzed, this procedure
yields only a few connected components, with the
ideal number being two (the left eye and the right
eye). In the case that other movement has occurred,
producing a much larger number of components,
the system discards the current binary image and
waits to process the next involuntary blink in order
to maintain efficiency and accuracy in locating the
eyes.
Given an image with a small number of
connected components output from the previous
processing steps, the system is able to proceed
efficiently by considering each pair of components
as a possible match for the user’s left and right
eyes. The filtering of unlikely eye pair matches is
based on the computation of six parameters for
each component pair: the width and height of each
of the two components and the horizontal and
vertical distance between the centroids of the two
components. A number of experimentally-derived
heuristics are applied to these statistics to pinpoint
the exact pair that most likely represents
the user’s eyes.
VII. HISTOGRAM EQUALIZATION:
Fig 3: Detection of Open Eyes In Histogram
Fig 4: Detection of closed eyes in Histogram
The histogram equalized image g will be
defined bygi,j = floor((L − 1)∑fi,j
=0 ) ∶ (2)
Where floor() rounds down to the nearest
integer. This is equivalent to transforming the pixel
intensities, k, of f by the function
The motivation for this transformation
comes from thinking of the intensities of f and g as
continuous random variables X, Y on [0, L − 1]
with Y defined by
Y = T(X) = (L − 1)∫0 px(x)dx ∶(3)
Where pX is the probability density
Histogram equalization is a technique for
adjusting image intensities to enhance contrast. Let
f be a given image represented as a mr by mc
matrix of integer pixel intensities ranging from 0 to
L − 1. L is the number of possible intensity values,
often 256. Let p denote the normalized histogram
of fwith a bin for each possible intensity. So
𝑛=
number of pixels with intensity n
total number of pixels
wheren = 0, 1...L −1 : (1)
𝑝
=0 ),

Our discrete histogram is an
approximation of pX(x) and the transformation in
Equation1 approximates the one in Equation 2.
While the discrete version won’t result in exactly
flat histograms, it will flatten them and in doing so
enhance the contrast in the image.
VIII. EDGE DECTION
Edge detection is the name for a set of
mathematical methods which aim at identifying
points in a digital image at which the image
brightness changes sharply or, more formally, has
discontinuities. The points at which image
brightness changes sharply are typically organized
into a set of curved line segments termed edges.
The same problem of finding discontinuities in 1D
signal is known as step detection and the problem
of finding signal discontinuities over time is known
as change detection. Edge detection is a
fundamental tool in image
IX. METHODOLOGY
In our proposed system we use MATLAB
software and a web cam. MATLAB is a
proprietary language developed by Math Works.
MATLAB is a high-performance language for
technical computing. It integrates computation,
visualization, and programming in an easy-to-use
environment where problems and solutions are
explained in familiar mathematical
notation.Typical uses include Math and
computation.In our proposed system MATLAB
software is used to compare the images in the
machine language to warn the driver about his
fatigueless. Camera captures the video sequences
and images of the driver to check if the driver is in
fatigue position or not. Thus recognize the fatigue
in a driver provided by the camera. An efficient
algorithm is introduced for the same. This
algorithm detects the fatigue in three ways firstly
by detecting the face, then next by detecting the
eyes from the detected face, finally by detecting
the position of the eyes whether it is in an open or
in closure position using this algorithm. Then it
gives the output as an alarm to warn the driver
which avoids the occurrence of accident.
Fig 5: Block diagram
Fig 6: Flow Chart
The algorithm used is Viola jones
algorithm. The problem to be solved is detection of
faces in an image. A human can do this easily, but
a computer needs precise instructions and
constraints. To make the task more manageable,
Viola–Jones requires full view frontal upright
faces. Thus in order to be detected, the entire face
must point towards the camera and should not be
tilted to either side. While it seems these
constraints could diminish the algorithm's utility
somewhat, because the detection step is most often
followed by a recognition step, in practice these
limits on pose are quite acceptable.
function of f. T is the cumulative distributive
function of X multiplied by (L−1). Assume for
simplicity that T is differentiable and invertible. It
can then be shown that Y defined by T(X) is
uniformly distributed on
[0, L−1], namely that pY (y) =L−
1
1
.

All human faces share some similar
properties. These regularities may be matched
using HaarFeatures
A few properties common to human faces:
The eye region is darker than the upper-
cheeks. The nose bridge region is brighter than the
eyes.
Composition of properties forming matchable
facial features:
1. Location and size: eyes, mouth, bridge of nose
2. Value: oriented gradients of pixel intensities
Rectangle features:
3. Value = Σ (pixels in black area) - Σ (pixels in
white area)
4. Three types: two-, three-, four-rectangles,
Viola & Jones used two-rectangle features
9.2. Creating an Integral Image
AdaBoost short for "Adaptive Boosting"
It can be used in conjunction with many other
types of learning algorithms to improve their
performance. The output of the other learning
algorithms ('weak learners') is combined into a
weighted sum that represents the final output of the
boosted classifier.
Fig 11: Edge Detected Eyes Part
3.Adaboost Training
4.Cascading Classifiers
9.1. Haar Features
The above flow chart determines the
process of fatigue detection. This algorithm detects
the fatigue in three ways firstly by detecting the
face, then next by detecting the eyes from the
detected face, finally by detecting the position of
the eyes whether it is in an open or in closure
position using this algorithm. Then it gives the
output as an alarm to warn the driver which avoids
the occurrence of accident. The algorithm has four
stages:
1.Haar Feature Selection
2.Creating an internal image
An image representation called the integral
image evaluates rectangular features in constant
time, which gives them a considerable speed
advantage over more sophisticated alternative
features. Because each feature's rectangular area is
always adjacent to at least one other rectangle,
it follows that any two-rectangle feature can be
computed in six array references, any
three-rectangle feature in eight, and any
four-rectangle feature in nine.
9.3. cascading classifiers
X. RESULTS
The cascade classifier consists of stages,
where each stage is an ensemble of weak learners.
The weak learners are simple classifiers called
decision stumps. Each stage is trained using a
technique called boosting.
9.4 Adaboost training

XI. CONCLUSION
Face detection systems are being
introduced now a days in many vehicles.The
automatic initialization phase (involving the
motion analysis work) is greatly simplified in this
system, with no loss of accuracy in locating the
user’s eyes and choosing a suitable open eye
template. Given the reasonable assumption that the
user is positioned anywhere from about 1 to 2 feet
away from the camera, the eyes are detected within
moments. As the distance increases beyond this
amount, the eyes can still be detected in some
cases, but it may take a longer time to occur since
the candidate pairs are much smaller and start to
fail the tests designed to pick out the likely
components that represent the user’s eyes. In all of
the experiments in which the subjects were seated
between 1 and 2 feet from the camera, it never took
more than three involuntary blinks by the user
before the eyes were located successfully. Another
improvement is this system’s compatibility with
inexpensive USB cameras. These USB cameras are
more affordable and portable, and perhaps most
importantly, support a higher real-time frame rate
of 30 frames per second.
Future scope:
This technique is very useful in
recognizing the driver fatigue using the viola Jones
algorithm and haar features. It can be highly used
in vehicles of the drivers who travel for long
distances for a long time by taking video sequences
and taking the images from them to compare with
default saved image.
In the future we can also make the vehicle to slow
down and warn the driver with an alarm which can
reduce the accidents more than just about warning
the driver.
REFERENCES
[1]. P. Viola and M. J. Jones, Robust real-time
face detection, International Journal of
Computer Vision, 57 (2004), pp.
137{154}.
[2]. F. Rosenblatt, The perceptron: a
probabilistic model for information
storage and organization in the brain.,
Psychological review, 65 (1958), pp.
386{408(3). R. E. Schapire, Y. Freund, P.
Bartlett, and W. S. Lee, Boosting the
margin: A new explanation for the
effectiveness of voting methods, The
Annals of Statistics, 26 (1998)
[3]. L. Shapiro and G.C. Stockman, Computer
vision, 2001. ISBN 0130307963.
[4]. G. Tkacik, P. Garrigan, C. Ratliff, G.
Milcinski, J. M. Klein, L. H. Seyfarth, P.
Sterling, D. H. Brainard, and V.
Balasubramanian, Natural images from
the birth-place of the human eye, Public
Library of Science One, 6 (2011), p.
e20409.
[5]. J. Friedman, T. Hastie, and R. Tibshirani,
The Elements of Statistical Learning, vol.
1, Springer Series in Statistics, 2001.
[6]. H. J_egou, M. Douze, and C. Schmid,
Hamming embedding and weak geometric
consistency for large scale image search,
in European Conference on Computer
Vision, vol. I of Lecture Notes in
Computer Science, Springer, 2008.
[7]. A. Olmos, A biologically inspired
algorithm for the recovery of shading and
reactance images., Perception, 33 (2004)
[8]. Y. Freund and R. Schapire, A decision-
theoretic generalization of on-line
learning and an application to boosting,
Journal of Computer and System
Sciences, 55 (1997), pp. 119{139}.
[9]. Y. Freund, R. Schapire, and N. Abe, A
short introduction to boosting, Journal of
Japanese Society for Artificial
Intelligence, 14 (1999), pp.771{780}.
P. Ratnakar pursuing
B.TECH final year from
department ECE at
MVGR college of
engineering in 2016. He
is a member of IEEE
student chapter

K. Satyanarayana Raju
completed his B.TECH from
MVGR College of engineering
in 2004 and M.TECH degree
from JNTUK in 2009 now he is
working as assistant professor,
Dept. of ECE, MVGRCE,
Dr. Moturi Satyanarayana
completed his, B.Tech from
Nagarjuna University in
2001, &M. Tech. degree from
Andhra University in 2004 and
Ph. D. degree from 2012. He is
now working as Associate Professor, Dept. of
ECE, MVGRCE, Vijayanagaram. He has published
22 journal papers, 30 National and International
Conference papers. Guided 04 Ph.D Scholars, 10
M. Tech students, delivered guest lecturers: 04. He
is a member of IEEE, IETE, IE, SEMCE, ISTE,
and SIOS. He is presently coordinator for R&D
and IEEE Student chapter. Research Areas: 1.
Antennas for wireless applications 2. VLSI 3.
EMI/EMC Applications.
Vijayanagaram. He has 7 years of experience in
teaching. He has published 2 journal papers.
Guided 4 M.TECH students. He is a member of IE.
He is presently co-ordinator for IEEE student
chapter. Research areas: 1. Digital Electronics and
Communication Systems.

A Real Time Intelligent Driver Fatigue Alarm System Based On Video Sequences

More Related Content

What's hot (19)

Similar to A Real Time Intelligent Driver Fatigue Alarm System Based On Video Sequences (20)

Recently uploaded (20)

A Real Time Intelligent Driver Fatigue Alarm System Based On Video Sequences