Gesture recognition using artificial neural network,a technology for identifying gestures commonly originating from hand and face.

GESTURE RECOGNITION USING
ARTIFICIAL NEURAL NETWORK
By:
NIDHINRAJ.P.P
1

Introduction
Communication
Gestures
Gesture Recognition
Sign Language
Seminar Presentation 2014 College of Engineering Adoor
2

Gesture Recognition
 Gestures are expressive, meaningful body motions i.e.
physical movements of the fingers, hands, arms, head, face,
or body with the intent to convey information or interact
with the environment.
 Gesture Recognition is the act of interpreting motions to
determine such intent.
 It is the mathematical Interpretation of a human motion by
a computing device
3

•Gesture recognition, along with facial recognition, voice
recognition, eye tracking and lip movement recognition are
components of what developers refer to as a perceptual user
interface.
•Gestures can be static (the user assumes a certain pose or
configuration) or dynamic (with pre-stroke, stroke, and post-
stroke phases).
4

Types Of Gesture Recognition
1. Hand and Arm Gesture Recognition: Hand gesture recognition
consists of hand poses and sign languages. Hand gesture
technology allows operations of complex machines using only a
series of fingers and hand movements, eliminating the need for
physical contact between operator and machine.
2. Body Gesture Recognition: Body gesture involves full body motion.
Recognizing body gestures, and recognizing human activity. Such as
tracking movement of two people interacting outdoors, recognizing
human gaits for medical rehabilitation and athletic training.
5

3. Head and Face Gesture Recognition:Face gesture recognition
creates an effective non-contact interface between users and their
machines. They are direct, naturally eminent means for humans to
communicate their emotions.
• The goal of face gesture recognition is to make machines effectively
understand human emotion, regardless of the countless physical
differences between individuals.
• Facial expressions involve extracting sensitive features from facial
landmarks such as regions surrounding the mouth, nose, and eyes of
a normalized image.
6

Face Gesture Recognition
7

Hand Gesture Recognition
Hand Gesture Recognition Techniques:
1. Glove - based hand gesture recognition
2. Vision - based hand gesture recognition
Glove - based hand gesture recognition: Requires the user to be
connected to the computer. It require the user to wear a
cumbersome device and carry a load of cables connecting the device
to a computer. The user has to wear a glove and to make gestures in
front of the camera.
8

Hand Gesture Recognition
Examples of Data Glove
9

Vision - based hand gesture recognition: It uses one or more
camera to record images of human hand gestures and lighting
conditions that enhance gesture classification accuracy. It is fast
and can easily detect movements of the fingers when the user’s
hand is moving. A vision-based device can handle properties like
texture and color of gestures.
It will at best get a general sense of the type of finger motion. In
order to create the database for gesture system, the gestures
should be selected with their relevant meaning and each gesture
may contain multi-samples for increasing the accuracy of the
system.
10

Vision based system depends upon the:
• Number of camera used.
• Their Speed and latency.
• 2-D or 3-D.
• User requirements.
• Time.
11

Artificial Neural Network (ANN)
• Human brain has many incredible characteristics such as
massive parallelism, distributed representation and
computation, learning ability, generalization ability,
adaptability, which seems simple but is really complicated.
• It has been always a dream for computer scientist to create a
computer which could solve complex perceptual problems
this fast.
• ANN models was an effort to apply the same method as
human brain uses to solve perceptual problems
12

• Similar to human brain artificial neural networks consist of
artificial neurons called Perceptrons that receive numerical
value and then the inputs are weighted and added, the
result is then transformed into the output by a transfer
function.
• Today neural networks can be trained to solve problems that
are difficult for conventional computers or human beings.
13

Artificial Neuron
Ƒ=Ʃ(Xi * Wi)+bias
14

Methodology
• The presented system is based on one powerful hand feature
in combination with a feed-forward multi-layer neural
network based classifier.
• Generally speaking this method contains 4 main steps:
1. Gesture modeling
2. Segmentation
3. Feature Extraction
4. Classification
15

Recognition System
16

Gesture Modelling
• By gesture modeling, one means selection and formation of
proper gesture. This forms an essential aspect to best design
appropriate gesture vocabulary for Human-Computer
Interaction. One purpose of Human-Computer Interaction is
to make computer tasks controlled by a set of commands in
the form of hand gestures.
17

Segmentation
• Segmentation is based on the skin color. It is used to
separate the skin area from the background. The effect of
luminosity should be segregated from the color components.
This makes HSI color model a better choice than RGB.
• The input RGB gesture is converted to HSI form to reduce the
burden on the network and also for accuracy. After
segmenting, the hand region is assigned a white color and
other areas are assigned black.
18

Feature Extraction
• The feature extraction aspect of image analysis seeks to
identify inherent characteristics, or features of objects found
within an image. These characteristics are used to describe
the object, or attribute of the object.
• Feature extraction produces a list of descriptions or a feature
vector. For static hand gestures features such as fingertips,
finger directions and hand‘s contours can be extracted.
Feature extraction is a complex problem, and often the
whole image or transformed image is taken as input.
Features are thus selected implicitly and automatically .
19

Classification
• After the phase of extraction, is the classification phase
where the extracted features are fed into the neural network
to recognize the particular character.
20

Conclusion
• Gesture Recognition provides the most important means for
non-verbal interaction among people especially for impaired
people (i.e. deaf-dumb). ANN is one of the most effective
technique of software computing for hand gesture
recognition problem.
• Neural Network is efficient as long as the data sets are small
and no further improvement is expected. Another advantage
of using neural networks in our research is that you can draw
conclusions from the network output.
21

• Gesture could be identified from the input hand gesture
video by identifying the fingers and their postures. The
segmentation of the hand and the fingers play a crucial role
in such process. Accuracy was increased when neural
networks were used.
• The detection capability of the system could be expanded to
body gestures as well.
22

References
[1].Rajesh Mapari,, Dr. Govind Kharat, “Hand Gesture
Recognition using Neural Network”(IJCSN)
[2].Ms. Shweta K. Yewale, Mr. Pankaj K. Bharne, “Artificial
Neural Network approach for Hand Gesture Recognition”,
(IJEST)
[3]. Prateem Chakraborty, Prashant Sarawgi, Ankit Mehrotra,
Gaurav Agarwal, Ratika Pradhan, “Hand Gesture Recognition:
A Comparative Study”,(IMECS)
23

24

Thank You

Gesture recognition using artificial neural network,a technology for identifying gestures commonly originating from hand and face.

More Related Content

What's hot (14)

Viewers also liked (20)

Similar to Gesture recognition using artificial neural network,a technology for identifying gestures commonly originating from hand and face. (20)

Recently uploaded (20)

Gesture recognition using artificial neural network,a technology for identifying gestures commonly originating from hand and face.