An eye gaze detection using low resolution web camera in desktop environment
1 like121 views
This document presents a method for detecting eye gaze using a low-resolution webcam. The method uses OpenCV and the Viola-Jones algorithm to detect faces and eyes. It then calculates the eye centers and maps them to X-Y coordinates on the screen. The algorithm achieves 69-74% accuracy in detecting gaze positions. It can be used for applications like controlling interfaces for blind users or in military cockpits. The method provides an economical way to perform eye gaze detection using regular webcams and open-source tools.
![IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308
_______________________________________________________________________________________
Volume: 05 Issue: 04 | Apr-2016, Available @ http://ijret.esatjournals.org 291
AN EYE GAZE DETECTION USING LOW RESOLUTION WEB
CAMERA IN DESKTOP ENVIRONMENT
Mahadak Gopal Mahadeo1
, Landage Vijay Bapurao2
, Lohare Nitin Panditrao3
, Solankar Rasika
Sharad4
1
Student, Computer Engineering Department, DYPCOE, Ambi, Maharashtra, India
2
Student, Computer Engineering Department, DYPCOE, Ambi, Maharashtra, India
3
Student, Computer Engineering Department, DYPCOE, Ambi, Maharashtra, India
4
Student, Computer Engineering Department, DYPCOE, Ambi, Maharashtra, India
Abstract
Purpose of this paper is to detect the focus of the eye on monitor in X and Y coordinates with the help of regular low resolution
web camera. We are using regular low resolution web camera to get the results of system which benefits in economical way for
development. To achieve the systems goal we are using OpenCV (Open Source Computer Vision) library which is open source
library which makes system very economical. System is implemented with the help of Viola Jones algorithm which help to
maximize the accuracy. This system helps in human-computer interaction. This system helps the blind peoples to control the
various systems.
Keywords: Gaze Detection, Voila Jones Algorithm, Haar-Like Features.
--------------------------------------------------------------------***----------------------------------------------------------------------
1. INTRODUCTION
The gaze detection is to detect the center of the eyes on the
monitor. The goal is achieved with OpenCV library.
OpenCV is open source and image processing library which
is compatible with various languages like Python, C and
Java. The system accesses the functions of OpenCV to
achieve various tasks.
The system is developed for various applications like space
mouse controlling and blind person mouse controlling etc.
The systems main focus is to detect users focus on monitor
and map it in the X and Y coordinates in system. The system
is divided into various different steps and these are as
follows:
Face extraction
Eye detection
Eye center calculation
Eye center mapping
The face extraction and eye detection uses the Voila Jones
technique which is widely used because of its faster
processing and accuracy. Voila Jones is having different
modules like haar classifier, intragral image, adaboost and
cascade classifier. Voila Jones extracts the face from
background video and calculates whole face area. Same
technique is used for eye calculation which divides the face
into two parts. From the two parts it divides the upper part
into two parts. From main upper part it calculates the center
of lower part and started applying the cascade classifier
algorithm. For calculating eye center it finds the center of
the square which implemented by haar classifier algorithm.
After finding the center it compares with world coordinates
and windows coordinates. At last it applies the camshift
means algorithm while moving head and eyes. The system
runs the all processing in loop.
Fig-1: Basic Idea of System
The system uses the multithreading concept because the
position of eyes and head changes always runtime and to
calculate it requires large processing time on single core.
The system takes the help of low resolution web camera to
achieve the main goal of system. The camera is of VGA or
0.5-2 megapixels which is having night vision functionality
and software with canny edge detector algorithm
implemented. The system uses the haar classifier face and
eye XML files who is having functions to calculate all
extraction and detections.
To track the pilot’s eye and develop the correct visual
interfaces this system is used in military applications for
cockpit [1]. Emotions of person such as anxiety, deceit or
hostility can be examined also mental states can be tracked
by using eye gaze detection in homeland security
applications [2]-[3].](https://image.slidesharecdn.com/aneyegazedetectionusinglowresolutionwebcameraindesktopenvironment-160922110454/85/An-eye-gaze-detection-using-low-resolution-web-camera-in-desktop-environment-1-320.jpg)
![IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308
_______________________________________________________________________________________
Volume: 05 Issue: 04 | Apr-2016, Available @ http://ijret.esatjournals.org 292
1.1 Existing Methods
The method of gaze detection on monitor is called
Oculography. There are two different methods for gaze
estimation [4].
a) Feature-based Gaze Estimation:
This method explores the various characteristics of the
human eye to identify the various distinctive features of the
eyes like eye corner, eye pupils and corner reflections [5].It
has the two methods model based and interpolation based
[6].
b) Appearance-based Gaze Estimation:
Based on photometric appearance the appearance based gaze
methods detect the gaze position and eyes. It uses the image
content to estimate the gaze direction by mapping image
data to screen coordinates [7]-[8]. They do not require the
calibration of cameras.
2. PROPOSED METHOD
Eye gaze detection method contains the eye tracking process
which tracks eyes movements with respect to head
movements. One of the method capture the eye frames
through the web camera 20-30 frames per second and gives
it to computer for processing, after produces the results. The
video frames help the algorithm for faster processing. Our
system uses the OpenCV library which used for HCI
(human computer interaction) and mainly meant for real
time image processing that contains the functions used in
our algorithms.
We use the windows platform with net beans software. First
we use the algorithm for face detection then we use it for
eyes detection. The algorithm uses haar cascade classifier
for face and eyes detection [9]-[10]. Overview of the system
is represented by following block diagram:
Fig-2: System Architecture
3. IMPLEMENTATION
Voila Jones Algorithm:
This algorithm is used for face and eye detection and it is
based on haar features [11]. Haar feature uses the 25*25
rectangular block for feature detection. In it the white
portion is subtracted from the grey portion [12].
Following are the haar like features which are commonly
used.
Fig-3: Haar- Like Features
At every stage the face candidates are scanned. The weight
of the every feature is calculated by AdaBoost algorithm.
Every feature generates the particular value after processing
the every rectangle. Each stage has number of values of
features in it so AdaBoost algorithm compares the number
of stages with values. No face stages values are avoided &
others are forwarded to next stage.
Fig-4: Cascading Stages
From the cascade classifier the features are detected. In this
system the following classifiers are used:
haarcascade_frontalface_atl2.xml
haarcascade_eye.xml](https://image.slidesharecdn.com/aneyegazedetectionusinglowresolutionwebcameraindesktopenvironment-160922110454/85/An-eye-gaze-detection-using-low-resolution-web-camera-in-desktop-environment-2-320.jpg)
![IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308
_______________________________________________________________________________________
Volume: 05 Issue: 04 | Apr-2016, Available @ http://ijret.esatjournals.org 293
In system first the face is detected & then eyes are detected
with the help of the area of the face as reference. In
OpenCV, there are applications which support the training
of a cascade classifier: opencv_haartraining &
opencv_traincascade. The program has following functions
main() : In the main function we load the XML file &
classifier file.
detectEye(): In the detectEye function the multiscale
eyes & face are detected.
trackeye(): In the trackeye function the mapping of
coordinates is performed.
3. CONCLUSIONS AND RESULT:
By using the Voila Jones technique the accuracy of the
system is achieved & users focus calculated. The system
generates the output in the X & Y coordinates. Following is
the desired system which is having the circle at eyes centers.
Fig-5: Actual System
The result of the system is represented in the X and Y
coordinates. Following is the table which shows the
accuracy level in the left and right eyes.
Sr.
No.
Left Eye
Coordinates
Right Eye
Coordinates
Accuracy
Achieved
1 {325,125} {374,164} 69
2 {423,162} {421,136} 72
3 {546,215} {521,231} 71.45
4 {432,300} {478,321} 70.85
5 {601,120} {621,125} 74.32
By using Voila Jones algorithm we can achieve the good
accuracy to detect the user’s focus on monitor in X and Y
coordinates. Voila Jones computes the image processing
very faster than other techniques.
ACKNOWLEDGEMENT
For completing this project Prof. Anupkumar Bongal gave
us important guidance. We are currently working on
template matching of eye gaze detection using Voila Jones
technique.
REFERENCES
[1]. W. Haibo, X. Chengqi, L. Qing, “The Eye Movement
Experiment and the Usability Evaluation of the Fighter
Cockpit Digital Interface,” 2nd International Conf.
Information Engineering and Computer Science
(ICIECS) 2010, pp. 1-4, 2010.
(10.1109/ICIECS.2010.5678205).
[2]. S. Milekic, “The more you look the more you get:
Intention-based interface using gaze-tracking,” in
Trant, J.(Des.) Museums and the Web2002: Selected
Papers from an Int. Conf., Archives and Museum
Informatics, 2002, pp. 1–27.
[3]. J. Orozco, F. X. Roca, and J. Gonzàlez, “Real-time
gaze tracking with appearance-based models,”
Machine Vision and Applications, 20(6): 353-364,
2009.
[4]. COGAIN: “D2.1 Survey of De-Facto Standards in Eye
Tracking” by the European Commission within the
Sixth Framework Programme, 2005.
[5]. Iannizzotto L. and La Rosa F., (2011) Competitive
combination of Multiple Eye detection and Tracking
Techniques, IEEE Transactions on Industrial
Electronics, 58(8): 3151-3159.
[6]. Hansen D. W., Agustin J. S. and Villanueva A, (2010)
Homography Normalization for Robust Gaze
Estimation in Uncalibrated setups Proc. of the
Symposium on Eye Tracking Research and
Applications, pp. 13-20.
[7]. [7]. Javier Orozco, F. Xavier Roca and Jordi Gonzalez.
(2009) Real time gaze tracking with appearance based
models, Machine Vision and Applications, 20(6), 353-
364.
[8]. Lu F., Sugano Y., Takahiro O. and Sato Y., (2011) A
head pose-free approach for appearance-based gaze
estimation, in Proc. of the 22nd British Machine Vision
Conf.
[9]. Zhu, Zhaomin, et al. "Multi-view face detection and
recognition using haar-like features." Proceedings of
COE Workshop. 2006.
[10].Dabhade, Sarala A., and Mrunal S. Bewoor. "Real
Time Face Detection and Recognition using Haar-
Based Cascade Classifier and Principal Component
Analysis." International Journal of Computer Science
and Management Research 1.1 (2012).
[11].Padmaja, K., and T. N. Prabakar. "FPGA based real
time face detection using Adaboost and histogram
equalization." In Advances in Engineering, Science and
Management (ICAESM), 2012 International
Conference on, pp. 111-115. IEEE, 2012.
[12].Majumder, Anima, Laxmidhar Behera, and Venkatesh
K. Subramanian. "Automatic and robust detection of
facial features in frontal face images." InComputer
Modelling and Simulation (UKSim), 2011 UkSim 13th
international Conference on, pp. 331-336. IEEE, 2011.](https://image.slidesharecdn.com/aneyegazedetectionusinglowresolutionwebcameraindesktopenvironment-160922110454/85/An-eye-gaze-detection-using-low-resolution-web-camera-in-desktop-environment-3-320.jpg)
An eye gaze detection using low resolution web camera in desktop environment
- 1. IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308 _______________________________________________________________________________________ Volume: 05 Issue: 04 | Apr-2016, Available @ http://ijret.esatjournals.org 291 AN EYE GAZE DETECTION USING LOW RESOLUTION WEB CAMERA IN DESKTOP ENVIRONMENT Mahadak Gopal Mahadeo1 , Landage Vijay Bapurao2 , Lohare Nitin Panditrao3 , Solankar Rasika Sharad4 1 Student, Computer Engineering Department, DYPCOE, Ambi, Maharashtra, India 2 Student, Computer Engineering Department, DYPCOE, Ambi, Maharashtra, India 3 Student, Computer Engineering Department, DYPCOE, Ambi, Maharashtra, India 4 Student, Computer Engineering Department, DYPCOE, Ambi, Maharashtra, India Abstract Purpose of this paper is to detect the focus of the eye on monitor in X and Y coordinates with the help of regular low resolution web camera. We are using regular low resolution web camera to get the results of system which benefits in economical way for development. To achieve the systems goal we are using OpenCV (Open Source Computer Vision) library which is open source library which makes system very economical. System is implemented with the help of Viola Jones algorithm which help to maximize the accuracy. This system helps in human-computer interaction. This system helps the blind peoples to control the various systems. Keywords: Gaze Detection, Voila Jones Algorithm, Haar-Like Features. --------------------------------------------------------------------***---------------------------------------------------------------------- 1. INTRODUCTION The gaze detection is to detect the center of the eyes on the monitor. The goal is achieved with OpenCV library. OpenCV is open source and image processing library which is compatible with various languages like Python, C and Java. The system accesses the functions of OpenCV to achieve various tasks. The system is developed for various applications like space mouse controlling and blind person mouse controlling etc. The systems main focus is to detect users focus on monitor and map it in the X and Y coordinates in system. The system is divided into various different steps and these are as follows: Face extraction Eye detection Eye center calculation Eye center mapping The face extraction and eye detection uses the Voila Jones technique which is widely used because of its faster processing and accuracy. Voila Jones is having different modules like haar classifier, intragral image, adaboost and cascade classifier. Voila Jones extracts the face from background video and calculates whole face area. Same technique is used for eye calculation which divides the face into two parts. From the two parts it divides the upper part into two parts. From main upper part it calculates the center of lower part and started applying the cascade classifier algorithm. For calculating eye center it finds the center of the square which implemented by haar classifier algorithm. After finding the center it compares with world coordinates and windows coordinates. At last it applies the camshift means algorithm while moving head and eyes. The system runs the all processing in loop. Fig-1: Basic Idea of System The system uses the multithreading concept because the position of eyes and head changes always runtime and to calculate it requires large processing time on single core. The system takes the help of low resolution web camera to achieve the main goal of system. The camera is of VGA or 0.5-2 megapixels which is having night vision functionality and software with canny edge detector algorithm implemented. The system uses the haar classifier face and eye XML files who is having functions to calculate all extraction and detections. To track the pilot’s eye and develop the correct visual interfaces this system is used in military applications for cockpit [1]. Emotions of person such as anxiety, deceit or hostility can be examined also mental states can be tracked by using eye gaze detection in homeland security applications [2]-[3].
- 2. IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308 _______________________________________________________________________________________ Volume: 05 Issue: 04 | Apr-2016, Available @ http://ijret.esatjournals.org 292 1.1 Existing Methods The method of gaze detection on monitor is called Oculography. There are two different methods for gaze estimation [4]. a) Feature-based Gaze Estimation: This method explores the various characteristics of the human eye to identify the various distinctive features of the eyes like eye corner, eye pupils and corner reflections [5].It has the two methods model based and interpolation based [6]. b) Appearance-based Gaze Estimation: Based on photometric appearance the appearance based gaze methods detect the gaze position and eyes. It uses the image content to estimate the gaze direction by mapping image data to screen coordinates [7]-[8]. They do not require the calibration of cameras. 2. PROPOSED METHOD Eye gaze detection method contains the eye tracking process which tracks eyes movements with respect to head movements. One of the method capture the eye frames through the web camera 20-30 frames per second and gives it to computer for processing, after produces the results. The video frames help the algorithm for faster processing. Our system uses the OpenCV library which used for HCI (human computer interaction) and mainly meant for real time image processing that contains the functions used in our algorithms. We use the windows platform with net beans software. First we use the algorithm for face detection then we use it for eyes detection. The algorithm uses haar cascade classifier for face and eyes detection [9]-[10]. Overview of the system is represented by following block diagram: Fig-2: System Architecture 3. IMPLEMENTATION Voila Jones Algorithm: This algorithm is used for face and eye detection and it is based on haar features [11]. Haar feature uses the 25*25 rectangular block for feature detection. In it the white portion is subtracted from the grey portion [12]. Following are the haar like features which are commonly used. Fig-3: Haar- Like Features At every stage the face candidates are scanned. The weight of the every feature is calculated by AdaBoost algorithm. Every feature generates the particular value after processing the every rectangle. Each stage has number of values of features in it so AdaBoost algorithm compares the number of stages with values. No face stages values are avoided & others are forwarded to next stage. Fig-4: Cascading Stages From the cascade classifier the features are detected. In this system the following classifiers are used: haarcascade_frontalface_atl2.xml haarcascade_eye.xml
- 3. IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308 _______________________________________________________________________________________ Volume: 05 Issue: 04 | Apr-2016, Available @ http://ijret.esatjournals.org 293 In system first the face is detected & then eyes are detected with the help of the area of the face as reference. In OpenCV, there are applications which support the training of a cascade classifier: opencv_haartraining & opencv_traincascade. The program has following functions main() : In the main function we load the XML file & classifier file. detectEye(): In the detectEye function the multiscale eyes & face are detected. trackeye(): In the trackeye function the mapping of coordinates is performed. 3. CONCLUSIONS AND RESULT: By using the Voila Jones technique the accuracy of the system is achieved & users focus calculated. The system generates the output in the X & Y coordinates. Following is the desired system which is having the circle at eyes centers. Fig-5: Actual System The result of the system is represented in the X and Y coordinates. Following is the table which shows the accuracy level in the left and right eyes. Sr. No. Left Eye Coordinates Right Eye Coordinates Accuracy Achieved 1 {325,125} {374,164} 69 2 {423,162} {421,136} 72 3 {546,215} {521,231} 71.45 4 {432,300} {478,321} 70.85 5 {601,120} {621,125} 74.32 By using Voila Jones algorithm we can achieve the good accuracy to detect the user’s focus on monitor in X and Y coordinates. Voila Jones computes the image processing very faster than other techniques. ACKNOWLEDGEMENT For completing this project Prof. Anupkumar Bongal gave us important guidance. We are currently working on template matching of eye gaze detection using Voila Jones technique. REFERENCES [1]. W. Haibo, X. Chengqi, L. Qing, “The Eye Movement Experiment and the Usability Evaluation of the Fighter Cockpit Digital Interface,” 2nd International Conf. Information Engineering and Computer Science (ICIECS) 2010, pp. 1-4, 2010. (10.1109/ICIECS.2010.5678205). [2]. S. Milekic, “The more you look the more you get: Intention-based interface using gaze-tracking,” in Trant, J.(Des.) Museums and the Web2002: Selected Papers from an Int. Conf., Archives and Museum Informatics, 2002, pp. 1–27. [3]. J. Orozco, F. X. Roca, and J. Gonzàlez, “Real-time gaze tracking with appearance-based models,” Machine Vision and Applications, 20(6): 353-364, 2009. [4]. COGAIN: “D2.1 Survey of De-Facto Standards in Eye Tracking” by the European Commission within the Sixth Framework Programme, 2005. [5]. Iannizzotto L. and La Rosa F., (2011) Competitive combination of Multiple Eye detection and Tracking Techniques, IEEE Transactions on Industrial Electronics, 58(8): 3151-3159. [6]. Hansen D. W., Agustin J. S. and Villanueva A, (2010) Homography Normalization for Robust Gaze Estimation in Uncalibrated setups Proc. of the Symposium on Eye Tracking Research and Applications, pp. 13-20. [7]. [7]. Javier Orozco, F. Xavier Roca and Jordi Gonzalez. (2009) Real time gaze tracking with appearance based models, Machine Vision and Applications, 20(6), 353- 364. [8]. Lu F., Sugano Y., Takahiro O. and Sato Y., (2011) A head pose-free approach for appearance-based gaze estimation, in Proc. of the 22nd British Machine Vision Conf. [9]. Zhu, Zhaomin, et al. "Multi-view face detection and recognition using haar-like features." Proceedings of COE Workshop. 2006. [10].Dabhade, Sarala A., and Mrunal S. Bewoor. "Real Time Face Detection and Recognition using Haar- Based Cascade Classifier and Principal Component Analysis." International Journal of Computer Science and Management Research 1.1 (2012). [11].Padmaja, K., and T. N. Prabakar. "FPGA based real time face detection using Adaboost and histogram equalization." In Advances in Engineering, Science and Management (ICAESM), 2012 International Conference on, pp. 111-115. IEEE, 2012. [12].Majumder, Anima, Laxmidhar Behera, and Venkatesh K. Subramanian. "Automatic and robust detection of facial features in frontal face images." InComputer Modelling and Simulation (UKSim), 2011 UkSim 13th international Conference on, pp. 331-336. IEEE, 2011.
- 4. IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308 _______________________________________________________________________________________ Volume: 05 Issue: 04 | Apr-2016, Available @ http://ijret.esatjournals.org 294 BIOGRAPHIES Mahadak Gopal Mahadeo Gopalmahadak188@gmail.com(Bachelor of Engineering Student, Computer Engineering Department, DY Patil College of Engineering, Ambi, Pune.) Landage Vijay Bapurao Vijlandge1@gmail.com(Bachelor of Engineering Student, Computer Engineering Department,DYPCOE, Ambi, Pune.) Solankar Rasika Sharad solankarrs@gmail.com(Bachelor of Engineering Student, Computer Engineering Department, DY Patil College of Engineering, Ambi, Pune.) Lohare Nitin Panditaro lohare45@gmail.com(Bachelor of Engineering Student, Computer Engineering Department, DY Patil College of Engineering, Ambi, Pune.)