IRJET - Android based Portable Hand Sign Recognition System

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 07 Issue: 02 | Feb 2020 www.irjet.net p-ISSN: 2395-0072
© 2020, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 2103
ANDROID BASED PORTABLE HAND SIGN RECOGNITION SYSTEM
Ponmandhira Muthu I1, Mohanapriya S2, Revathi M3
1Student, Computer Science and Science, Agni College of Technology, Chennai, India
2Student, Computer Science and Science, Agni College of Technology, Chennai, India
3Assistant Professor, Dept. of Computer Science and Engineering, Agni College of Technology, Chennai, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - When thinking about how deaf and dumb
people live, most of us often wonder how they communicate.
One of the best methods of communication between them is
the sign language. Using the sign language, they can share
their thoughts and knowledge with the normal people. In
this paper we have presented an approach that gives a
technique for improving Sign Language Recognition System
using glove. It consists of flex sensors that sense the hand
gestures, a microcontroller to convert input analogue data
to digital data and power supply is to provide voltages to
specific units and finally Bluetooth module to send the data
from micro controller to android mobile.
Key Words: Flex sensor, Microcontroller, Gesture
Recognition, Android, Sign Language Translation, and
Bluetooth.
1. INTRODUCTION
The number of deaf-mutes in the world are
roughly calculated to be from 700,000 to 900, 000 and of
these 63 percent, are said to be born deaf,theotherslosing
their hearing by different accident. Sign Language is a
language of gestures and symbols that is used not only by
deaf and dumb people but also by others with normal
hearing to communicate with them. It is based on ideas
rather than words. When deaf and dumb people
communicate with each other, they commonly use a
standard sign language such as AmericanSignLanguageor
British Sign Language developed in USA and UK
respectively.
Fig -1: The letters of the English alphabet in ASL
There is a connection among human languages
such as gestures, facial expressions and emotions which
are inseparable. Hence communication will become
meaningless and incomplete if they are uttered
individually. So, to understand these links use of more
complex processes becomes necessary than thoseusedfor
simple peripheral preprocessing of signals. To understand
how human communication exploits information from
several channels it is necessary to know about the
constitution of cross-cultural database comprising verbal
and non-verbal (gaze, facial expressions and gestures)
data. Gathering data across a spectrum of disciplines in a
structured manner will reinforce the ability to portray the
underlying meta-structure and will lead to new
mathematical models and approaches. Though scientific
study of speech, gestures, gaze and emotions are of
integrated system appears to have great strides in the last
two decades, analysis has been at snail’s pace due to the
limitation in determination of boundaries among them.
This opens door for automation of these processes.
According to medical science, a child brain
develops drastically during early stages i.e.1-5years.Itisa
crucial period during which a child acquires most of its
knowledge including languages and other skills. Hence
early introduction to sign language becomes inevitable in
developing linguistic skills among deaf children. Gestures
used for sign language recognition is useful for processing
information from man which is not conveyed through
speech or type. There are various types of gestures which
can be identified by computers just as speech recognition,
speech to text, gesture recognition represented through
sign language into text or voice. The letters of English
alphabet in American Sign Language (ASL) is as shown in
Fig -1.
Here, a custom-designed data-glove is developed
using android database with the help of hardware such as
flex sensors, microcontroller and Bluetooth module.
2. EARLIER WORK
The previous works mainly focused on a third
person perspective or also known as vision-based
technique for hand gesture recognition using image
processing [3][4][5]. We have referred these papers for
understanding their limitations and learning the basic
gesture detection techniques. But currently, due to
availability of variety of sensors andscopeofapplicationin

real life, many of the researchers have started using more
portable techniques for gesture recognition [1][2].
P. Subha Rajam[3] had proposed a system that
recognizes hand sign gesturesusingimageprocessing. The
gesture is captured using a camera and an outline of the
image is generated. According to the finger positions, the
system generates a binary code of the gesture. The binary
code is mapped to a text and the text is shown on a
monitor. In [4] this work author uses Hidden Markov
Models (HMM) as an approach for recognizing and
modeling dynamic hand gestures for the User Interface of
in-vehicle information. The proposed method in [5] had
recognized signs for all alphabets using Scale Invariant
Feature Transform which extracts the featuresofimageby
finding and Describing key points and then converts them
into text.
In [1] the project decodes the captured signs of
American Sign Language into simple English sentences
using a sensor glove. In order to recognize the sensor
values, form the sensor glove artificial neural networks
were used. These values received were categorizedinto26
alphabets of the English language and two punctuation
symbols were introduced by the author. In [2], a similar
kind of approach was proposedbytheauthorthatincluded
flex sensors and an accelerometer to detect the bend in
fingers and the gesture of the hands and Arduino to
convert. The flex sensors and accelerometer send the
different voltage readings to thePICmicrocontroller which
will be mapped to the text.
Celestine Preetham [6] had proposed a system to
make the gloves communicate wirelessly with another
Bluetooth device. The device has a speech synthesizing
software that will produce the translated speech for the
hand gesture.
By referring to all these works, we have tried to
provide our contribution towards society. We have
extracted the best of the techniques from each of these
papers, combined it under a common framework and
extended it for multiple applications.
3. GESTURE DETECTION SYSTEM
3.1 System Overview
The gesture detection system is divided into following
sections:
Fig -2: System overview
3.2 Block Diagram
The block diagram of the gesture detection system is
given below:
Fig -3: Block Diagram
3.3 Working Principle
The working principle of the various major blocks or
sections used in the gesture detection system is given
below

3.3.1 Flex sensor
Fig -4: Flex Sensor
A flex sensor is a sensor that measurestheamount
of deflection and bending. It changes in resistance
depending upon the angle of bend on the sensor.
They convert the changes in bend to electrical
resistance, more the change more the value of resistance.
Flex sensors are usually is in the form ofstrips.Theycanbe
made unidirectional and bidirectional. The unidirectional
changes its resistance, when it is bent in only one direction
and the bidirectional change its resistance for both
directions. Here, bidirectional flex sensor is used.
Carbon resistive elements within a thin flexible
substrate are inside the flex sensors. The sensor produces
a resistance output relative to the radius, when the
substrate is bent. In typical flex sensors, a flex of degree
with 90 will give a resistance of 20 – 30 K ohms. They
require 5 – volt input and output between 0 – 5 V. The
sensors are connected to the device through 3 pin
connectors. In that device, sensors are activated in sleep
mode and when not in use, it enables them to power down
mode.
Fig -5: Variation of resistance with degree in bend
Fig -6: Basic Flex Circuit
Figure 3 shows circuit of basic flex sensor. By
voltage divider rule, output voltage is given by = *
R1/ (R1+R2). Here, R1 is the other input resistor to the
non-inverting terminal. Fig -7 shows the characteristics:
(a) Resistance V/S Bending (b) Voltage V/S Resistance
Fig -7: Characteristics of Flex Sensor

Table: Resistance and corelated Bending
Resistance in
KΩ
Voltage in
volts
Angle in
degree
26.18 0.82 90
40.26 0.67 75
48.74 0.45 60
52.92 0.40 45
57.10 0.37 30
60.40 0.32 15
63.50 0.30 00
3.3.2 PIC Microcontroller
Fig -8: PIC16f877a Microcontroller
The PIC microcontroller PIC16f877a is one of the most
reowned microcontrollers.This microcontroller is very
practical to use, the programming of this controller is also
easier.
Fig -9: PIC Microcontroller Pin Configuration
The advantage of this microcontroller is that it can be
write and erase as many times as possible due to the useof
FLASH memory technology. Fig -9 shows the pin
configuration of PIC Microcontroller. It contains total
number of 40 pins and in that 33 pins for input and
output.The Microcontroller convert inputanaloguedata to
digital data and for further processing.
3.3.3 Bluetooth Module
Fig -10: Bluetooth Module
HC-05 is a Bluetooth module which is designed for
wireless communication for transferringDigital signal that
is being converted by microcontroller. This module can be
used as a model of asymmetric communication which is a
master or slave configuration. The digital signal is sent to
android phone by using Bluetooth module.
3.3.3 Android Smart Phone
At server side the android smartphone takes the input
from the microcontroller through Bluetooth and based on
those input it will match the pattern with the alphabets
pattern which have already fed into the database. The
database is created for 26 alphabets of American Sign
Language in the android smartphone. Then the text of the
corresponding sign language is being displayed. The
pattern that does not match with database will respond
with error message.
4. ADVANTAGES & APPLICATIONS
 The normal people don’t need to learn sign language to
understand what the dumb people are trying to say.
 It is a portable device and can be easily carried
anywhere.

 The cost of the device is low.
 Power consumption for this system is also low.
 Easy interpretation
 Flexible to user
5.CONCLUSION
Sign language is the way of conveyinginformation
and thoughts of deaf and dumb people. This system will
helpful for thosepeople who communicatewiththenormal
people. The sign language recognition system convertsthe
hand gestures into text with the help of flex sensors which
is connected to the gloves.In order to enhance and
facilitate the different hand gestures recognition,we have
added the option to include more hand gestures into the
database.
6. FUTURE WORK
In this system, more sensors can be include to
recognize the hand gestures easily with more perfection
and accuracy. The system can also translate the words
from one language to another using some advanced
technologies and it will become wireless to transmit data
to a smart phone. This should eliminate the use of an LCD
display and it will have the advantage of text to voice
applications which is available in the smart phones.
Using centralized IoT hub,the glovescanalsoused
for interacting with a set of electronic devices across
house. We can teach the glove to understand the gestures
by introducing the concept of machine learning. By using
the different software development strategies and various
programming techniques the systems efficiency can be
increased.
7. REFERENCES
[1] S. A. Mehdi and Y. N. Khan, "Sign language recognition
using sensor gloves," Neural InformationProcessing,2002.
ICONIP ' 02. Proceedings of the 9th International
Conference on, 2002, pp. 2204-2206 vol.5.
[2] C. Preetham, G. Ramakrishnan, S. Kumar, A. Tamse and
N. Krishnapura, "Hand Talk-Implementation of a Gesture
Recognizing Glove," India Educators' Conference (TIIEC),
2013 Texas Instruments, Bangalore, 2013, pp. 328-331.
[3] P. S. Rajam and G. Balakrishnan, "Real time Indian Sign
Language Recognition System to aid deaf -dumb people,"
2011 IEEE 13th International Conference on
Communication Technology, Jinan, 2011, pp. 737-742
[4] N. Deo, A. Rangesh and M. Trivedi, "In-vehicle Hand
Gesture Recognition using Hidden Markov models," 2016
IEEE 19th International Conference on Intelligent
Transportation Systems (ITSC), Rio de Janeiro, Brazil,
2016, pp. 2179-2184.
[5] S. P. More and A. Sattar, "Hand gesture recognition
system using image processing," 2016 International
Conference on Electrical, Electronics, and Optimization
Techniques (ICEEOT), Chennai, 2016, pp. 671-675.
[6] N. Harish and S. Poonguzhali, "Designanddevelopment
of hand gesture recognition system for speech impaired
people," 2015 International Conference on Industrial
Instrumentation and Control (ICIC), Pune, 2015, pp. 1129-
1133
[7] Subash chandar, Vignesh Kumar and WillsonAmalraj―
Real Time Data Acquisition and Actuation via Network for
Different Hand gestures using Cyber glove‖, International
Journal of Engineering Research and Development ISSN,
Vol. 2, No.1, pp. 50-54, July 2012.
[8] Pławiak P., Sośnicki T., Niedźwiecki M., Tabor Z.,
Rzecki K. Hand body language gesture recognition based
on signals from specialized glove and machine learning
algorithms. IEEE Trans. Ind. Inform. 2016; 12:1104–1113.
doi: 10.1109/TII.2016.2550528.
[9] Karami A., Zanj B., Sarkaleh A.K. Persian sign language
(PSL) recognition using wavelet transform and neural
networks. Exp. Syst. Appl. 2011; 38:2661–2667. doi:
10.1016/j.eswa.2010.08.056.
[10] Munib Q., Habeeb M., Takruri B., Al-Malik H.A.
American sign language (ASL) recognitionbasedonHough
transform and neural networks. Exp. Syst. Appl. 2007;
32:24–37. doi: 10.1016/j.eswa.2005.11.018.
[11] Ahmed S.F., Ali S.M.B., Qureshi S.S.M. Electronic
speaking glove for speechless patients,a tonguetoa dumb;
Proceedings of the 2010 IEEE Conference on Sustainable
Utilization and Development in Engineering and
Technology (STUDENT); Petaling Jaya, Malaysia. 20–21
November 2010; pp. 56–60.
[12] Adnan N.H., Wan K., Shahriman A., Zaaba S., Basah
S.N., Razlan Z.M., Hazry D., Ayob M.N., Rudzuan M.N., Aziz
A.A. Measurement of the flexible bendingforceoftheindex
and middle fingers for virtual interaction. Procedia
Eng. 2012; 41:388–394.doi:10.1016

IRJET - Android based Portable Hand Sign Recognition System

More Related Content

What's hot (20)

Similar to IRJET - Android based Portable Hand Sign Recognition System (20)

More from IRJET Journal (20)

Recently uploaded (20)

IRJET - Android based Portable Hand Sign Recognition System