conference1final
Download as DOC, PDF0 likes217 views
This document describes a digital glove system that translates sign language gestures into voice to enable communication between deaf/mute communities and others. The glove uses flex sensors that detect finger bending and sends signals to an Arduino Uno microcontroller. The Arduino compares the signals to pre-programmed gestures and outputs the corresponding word as text on an LCD display and audio from a speaker. The system was able to recognize 32 different words or phrases through unique finger bending patterns detected by the flex sensors. This digital glove provides a low-cost way to facilitate communication for deaf/mute individuals.
![Abstract— Dumb people mostly use sign language for
communication but they find difficulty in communicating with
others who don’t understand sign language. This paper aims to
bridge this barrier in communication. It is based on the need of
developing an electronic device that can translate sign
language into voice in order to make the communication
possible between the mute communities with the general public
possible[7]. A Digital glove is used which is normal cloth of
driving gloves attached with Flex sensors along the length of
each finger and the thumb.
Mute people can use this glove to perform hand gesture
and it will be converted into a particular pattern so that normal
people can understand their meaning. A gesture in a sign
language is a particular movement of the hands with a specific
shape made out of them. In this project Flex Sensor plays the
important role, Flex sensors are sensors that change in
resistance depending on the amount of bend on the sensor and
corresponding output is generated in form of text displayed on
LCD and in form of audio played through speaker[4].
Index Terms—Flex Sensor, Arduino Uno, LM386.
I. INTRODUCTION
In our daily life, we have to interact with different kinds
of the people through regional or global language. This
communication is effective because both people know the
language, but in case of dumb, mute people problem is
arrived as maximum people are unaware of sign language.
So we are going to design the system through which
everyone can able to understand sign language without
extra efforts using Arduino Uno and Flex sensor. This
system will be helpful in malls, hospitals and mute schools.
A. Technical Background
The gesture recognition can be done either by image
processing technique or using sensor based network. In
image processing, expensive camera is used as input and
captured image taken as variable parameter with more
complex computation as it involves more data.
In previously sensor based technique, tri-axial
accelerometer along with electromyogram (EMG) sensor
was used [5]. Due to cost of tri-axial accelerometer and
EMG, was not affordable by laymen. As the finale module
was also not much handy and recognized limited gestures,
made to design more handy and wide system. Our project
employs a similar kind of hand gloves with Flex sensor but
much more handy and can precisely determine many hand
gestures.
B. Proposed Solution
We kept a solution for above mentioned problem in three
phase i.e sensing phase, processing phase and output
phase. In sensing phase, Flex sensor senses the physical
characteristic and converts it into electrical signal which
passed to processing unit. The controller is used here is
Arduino Uno which gives an digital output in form of text
displayed on LCD and in form of audio played through a
speaker.
Figure 1.1 Block diagram for proposed solution
.
C. Organisation
The next session describes about the proposed solution. It
explains a feasible solution, can implement using
hardware. The third section provides the various hardware
and ICs used in the project. A brief explanation of software
implementation is elaborated in section 3.2. The fourth
section discusses the result which is obtained. The last
section is deals with conclusion along with the strength,
limitation and future scope.
II. PROPOSED SOLUTION
Mute people use hand sign to convey the message to
other. A glove fitted with flex sensor is used to translate
the signs made by the person into speech. The gloves is
continuously in operating mode whenever, a particular
gesture pattern is made by user, an equivalent electrical
signals are given by the sensor. These signals are passed to
analog i/p ports of Arduino Uno controller. For 32 different
gestures, signals are different. A range of 10-bit ADC values
are assigned for each gesture. The corresponding word is
stored in sd card memory. The role of controller is
comparing the received signals with stored values and
displaying the corresponding word on LCD. The
corresponding word is also played through speaker. The
block diagram of proposed solution is shown in fig 2.1
Figure 2.1 Block diagram
1
Digital Glove For Mute People
Abhijit A. Kathwate, Prof.C.Y.Patil
Instrumentation and Control Dept.
jeetkathwate@gmail.com, cypatil@gmail.com](https://image.slidesharecdn.com/af1082f8-d6e5-4b27-9d3b-dc6a7c39c050-150607072314-lva1-app6891/85/conference1final-1-320.jpg)
![III. HARDWARE IMPLEMENTATION
A. Gloves
Any kind of cotton glove can be used for the purpose of
our project. A special kind of nylon gloves would be better
as they increase the easiness of movement of fingers.
B. Flex Sensor
Flex sensors are attached to the glove by using needle
and thread. It requires 5-volt input and output between 0
and 5 V, the resistivity varies with the sensor’s angle of
bend and the voltage output changes accordingly. The
device can activate the sensors from sleep mode, enables
them to power down when not in use and greatly
decreasing power consumption. It will only change
resistance in one direction. An unflexed sensor has a
resistance of about 10 kilo ohms. As the flex sensor is bent,
the resistance increases to 18- 25 kilo ohms at 90 degrees.
The sensor measures 0.25 inch wide, 4.5 inches long and
0.18 inches thick[1].
Figure 3.1 Flex sensor
C. Arduino Uno
The reason behind selecting Arduino uno as controller:
The Arduino Uno is a microcontroller board based on
ATmega328. It has 14 digital input/output pins, out of
which 6 can be used as PWM outputs, 6 analog inputs, a 16
MHz ceramic resonator, a reset button. It contains
everything needed to support the microcontroller, can
simply connected with a USB power cable with a AC-to-DC
adapter or battery for power source. The LCD, SD card and
a speaker can easily interfaces with Arduino Uno. Since the
size of Arduino Uno is small, makes the circuit more
compatible for user to handle.
D. SD Card
While interfacing SD card with Arduino Uno, following
connections need to be done [8]. Connect GND to ground,
3.3v to 3.3v, CLK to Pin 13 on your Arduino, MISO to pin
12, MISI to pin 11, and CS to pin 10. You can use a different
pin, as long as you have to remember to change the pin in
SD.begin().
Figure 3.2 Schematic of SD card interfacing
E. Speaker
The project accesses a series of .WAV files on an SD Card
and plays them according to input gesture when pin 9 of
the Uno is pulled high ( as speaker is directly triggered
using LM386 ). We tried to play 31 different .WAV files [9]
using Arduino TMRpcm.h library.
Figure 3.3 Speaker Interfacing using LM386
IV. SOFTWARE IMPLEMENTATION
A. Algorithm
The device is powered up through 5V and hand gesture
is fetched by user. Signals are generated and ADC
conversion is started, controller is made to wait till
conversion gets over. This process is continued till
maximum accuracy achieved by system. The generated
ADC values define the angle to which the joint are bent
like straight or 90 degree bending. Thus a series of new
digital values, either digital 1 or digital 0 obtained which
compared with previously stored values. This enables the
device in both static and dynamic hand gesture recognition
in the form of text visualized on LCD and audio played
through a speaker. The flowchart of the algorithm is
shown in figure 4.1
2](https://image.slidesharecdn.com/af1082f8-d6e5-4b27-9d3b-dc6a7c39c050-150607072314-lva1-app6891/85/conference1final-2-320.jpg)
![VI. CONCLUSION
This digital glove is a handy module which facilitates easy
communication for mute/deaf people with everyone. This
module communicates exactly as designed version in the
form of words/phrases. These phrases are heard as voice
and can also be read on the LCD16*2. This module does
not impose any extra effort for the knowledge of any
technical details for its use. This module built a foundation
of a more robust base and portable module to
communicate as sentences.
A. Advantages
The cost of final module is quit affordable i.e. costing just
7000 INR. The power consumption of the module is very
low, measuring the precisely the movements of finger.
Since the gesture is translated into both voice and text, a
conversion can be held between anyone even in between
a mute and deaf or in between a blind and mute.
B. Limitations
This module only works with words/phrases, does not
communicate in the form of sentences. The device
performs quit slowly as compared with normal speaking
people. This module is only dedicated for those people
who knows configuration of words with gestures. Few
standard gestures are slightly differs causes
misinterpretation.
C. Future Scope
Implementation of a digital glove for both hands will
expand the library of words that can be used for
communication. Addition of camera in the system can port
facial expression in the more appropriate meaning of
words. Noise reduction can be improved using noise
removal technique.
ACKNOWLEDGEMENT
We would like to thank ''Adhar Muk Badhir Vidyalaya,
Pune'' for providing an information related to deaf mute
people. This information is related with different
commands which we targeted it as output with respect to
different gestures. We would also like to thank HOD
INSTRU, COEP for providing us space and lab facility. We
would like to thank our mentor Prof Dr.C Y Patil.
REFERENCES
[1] Saggio, Giovanni. "Bend sensor arrays for hand movement
tracking in biomedical systems." In Advances in Sensors and
Interfaces (IWASI), 2011 4th IEEE International Workshop on, pp.
51-54. IEEE, 2011
[2] Tanyawiwat, Netchanok, and Surapa Thiemjarus. "Design of
an assistive communication glove using combined sensory
channels." In Wearable and Implantable Body Sensor Networks
(BSN), 2012 Ninth International Conference on, pp. 34-39. IEEE,
2012.
[3]Dipietro, Laura, Angelo M. Sabatini, and Paolo Dario. "A survey
of glove-based systems and their applications." Systems, Man,
and Cybernetics, Part C: Applications and Reviews, IEEE
Transactions on 38, no. 4 (2008): 461-482
[4]Praveenkumar S Havalagi, Shruthi Urf Nivedita.”The amazing
digital gloves that give voice to voiceless.”International journal of
advance in engineering and technology(March
2013),vol.6,issue.1,pp 471-480.
[5]Masaguppi, Shruti C., and D. R. Ambika. "Virtual Talk for Deaf,
Mute, Blind and Normal Humans." in India Educators'
Conference (TIIEC), 2013 Texas Instruments, pp. 316-320. IEEE,
2013.
[6]Delliraj, M., and S. Vijayakumar. "Design of Smart e-Tongue for
the physically challenged people." in Recent Trends in
Information Technology (ICRTIT), 2013 International Conference
on, pp. 306-311. IEEE, 2013.
[7]Prapat Parab, Sanika Kinalekar, Rohit Chavan, Deep Sharan,
Shubhadha Deshpande."Hand Gesture Recognition using
Microcontroller & Flex Sensor." in International Journal Of
Scientific Research And Education , pp. 518-522. March 2014.
[8] http://arduino.cc/en/Reference/SD
[9] http://playground.arduino.cc/SmartWAV/SmartWAV
4](https://image.slidesharecdn.com/af1082f8-d6e5-4b27-9d3b-dc6a7c39c050-150607072314-lva1-app6891/85/conference1final-4-320.jpg)
conference1final
- 1. Abstract— Dumb people mostly use sign language for communication but they find difficulty in communicating with others who don’t understand sign language. This paper aims to bridge this barrier in communication. It is based on the need of developing an electronic device that can translate sign language into voice in order to make the communication possible between the mute communities with the general public possible[7]. A Digital glove is used which is normal cloth of driving gloves attached with Flex sensors along the length of each finger and the thumb. Mute people can use this glove to perform hand gesture and it will be converted into a particular pattern so that normal people can understand their meaning. A gesture in a sign language is a particular movement of the hands with a specific shape made out of them. In this project Flex Sensor plays the important role, Flex sensors are sensors that change in resistance depending on the amount of bend on the sensor and corresponding output is generated in form of text displayed on LCD and in form of audio played through speaker[4]. Index Terms—Flex Sensor, Arduino Uno, LM386. I. INTRODUCTION In our daily life, we have to interact with different kinds of the people through regional or global language. This communication is effective because both people know the language, but in case of dumb, mute people problem is arrived as maximum people are unaware of sign language. So we are going to design the system through which everyone can able to understand sign language without extra efforts using Arduino Uno and Flex sensor. This system will be helpful in malls, hospitals and mute schools. A. Technical Background The gesture recognition can be done either by image processing technique or using sensor based network. In image processing, expensive camera is used as input and captured image taken as variable parameter with more complex computation as it involves more data. In previously sensor based technique, tri-axial accelerometer along with electromyogram (EMG) sensor was used [5]. Due to cost of tri-axial accelerometer and EMG, was not affordable by laymen. As the finale module was also not much handy and recognized limited gestures, made to design more handy and wide system. Our project employs a similar kind of hand gloves with Flex sensor but much more handy and can precisely determine many hand gestures. B. Proposed Solution We kept a solution for above mentioned problem in three phase i.e sensing phase, processing phase and output phase. In sensing phase, Flex sensor senses the physical characteristic and converts it into electrical signal which passed to processing unit. The controller is used here is Arduino Uno which gives an digital output in form of text displayed on LCD and in form of audio played through a speaker. Figure 1.1 Block diagram for proposed solution . C. Organisation The next session describes about the proposed solution. It explains a feasible solution, can implement using hardware. The third section provides the various hardware and ICs used in the project. A brief explanation of software implementation is elaborated in section 3.2. The fourth section discusses the result which is obtained. The last section is deals with conclusion along with the strength, limitation and future scope. II. PROPOSED SOLUTION Mute people use hand sign to convey the message to other. A glove fitted with flex sensor is used to translate the signs made by the person into speech. The gloves is continuously in operating mode whenever, a particular gesture pattern is made by user, an equivalent electrical signals are given by the sensor. These signals are passed to analog i/p ports of Arduino Uno controller. For 32 different gestures, signals are different. A range of 10-bit ADC values are assigned for each gesture. The corresponding word is stored in sd card memory. The role of controller is comparing the received signals with stored values and displaying the corresponding word on LCD. The corresponding word is also played through speaker. The block diagram of proposed solution is shown in fig 2.1 Figure 2.1 Block diagram 1 Digital Glove For Mute People Abhijit A. Kathwate, Prof.C.Y.Patil Instrumentation and Control Dept. jeetkathwate@gmail.com, cypatil@gmail.com
- 2. III. HARDWARE IMPLEMENTATION A. Gloves Any kind of cotton glove can be used for the purpose of our project. A special kind of nylon gloves would be better as they increase the easiness of movement of fingers. B. Flex Sensor Flex sensors are attached to the glove by using needle and thread. It requires 5-volt input and output between 0 and 5 V, the resistivity varies with the sensor’s angle of bend and the voltage output changes accordingly. The device can activate the sensors from sleep mode, enables them to power down when not in use and greatly decreasing power consumption. It will only change resistance in one direction. An unflexed sensor has a resistance of about 10 kilo ohms. As the flex sensor is bent, the resistance increases to 18- 25 kilo ohms at 90 degrees. The sensor measures 0.25 inch wide, 4.5 inches long and 0.18 inches thick[1]. Figure 3.1 Flex sensor C. Arduino Uno The reason behind selecting Arduino uno as controller: The Arduino Uno is a microcontroller board based on ATmega328. It has 14 digital input/output pins, out of which 6 can be used as PWM outputs, 6 analog inputs, a 16 MHz ceramic resonator, a reset button. It contains everything needed to support the microcontroller, can simply connected with a USB power cable with a AC-to-DC adapter or battery for power source. The LCD, SD card and a speaker can easily interfaces with Arduino Uno. Since the size of Arduino Uno is small, makes the circuit more compatible for user to handle. D. SD Card While interfacing SD card with Arduino Uno, following connections need to be done [8]. Connect GND to ground, 3.3v to 3.3v, CLK to Pin 13 on your Arduino, MISO to pin 12, MISI to pin 11, and CS to pin 10. You can use a different pin, as long as you have to remember to change the pin in SD.begin(). Figure 3.2 Schematic of SD card interfacing E. Speaker The project accesses a series of .WAV files on an SD Card and plays them according to input gesture when pin 9 of the Uno is pulled high ( as speaker is directly triggered using LM386 ). We tried to play 31 different .WAV files [9] using Arduino TMRpcm.h library. Figure 3.3 Speaker Interfacing using LM386 IV. SOFTWARE IMPLEMENTATION A. Algorithm The device is powered up through 5V and hand gesture is fetched by user. Signals are generated and ADC conversion is started, controller is made to wait till conversion gets over. This process is continued till maximum accuracy achieved by system. The generated ADC values define the angle to which the joint are bent like straight or 90 degree bending. Thus a series of new digital values, either digital 1 or digital 0 obtained which compared with previously stored values. This enables the device in both static and dynamic hand gesture recognition in the form of text visualized on LCD and audio played through a speaker. The flowchart of the algorithm is shown in figure 4.1 2
- 3. V. RESULTS The values of ADC without tuning the 5 Flex sensors are obtained are mapped with logic 0 and logic 1 is shown in following table: Logic 0 ADC values less than count of -200 Logic 1 ADC values greater than count of -200 Table 5.1 Mapping of ADC values Since the output voltage is taken across the potentiometer, when the resistance of the sensor increases, the voltage drop across potentiometer increases. Hence the ADC value should increase accordingly, resulting shifting to positive count. Observing the table 5.2, the logic values are shown with respect to each word with each finger. Table 5.2 Configuration of OUTPUT Sr No Words Thum b Fore Middle Ring Sm all 1 Well Done 0 0 0 0 0 2 Miss You 0 0 0 0 1 3 Love You 0 0 0 1 0 4 Sad 0 0 0 1 1 5 Happy 0 0 1 0 0 6 Run 0 0 1 0 1 7 Keep Quit 0 0 1 1 0 8 Stop 0 0 1 1 1 9 Start 0 1 0 0 0 10 Ready 0 1 0 0 1 11 Take Care 0 1 0 1 0 12 Gn 0 1 0 1 1 13 Good Morning 0 1 1 0 0 14 All The Best 0 1 1 0 1 15 Congrat 0 1 1 1 0 16 Beautiful 0 1 1 1 1 17 Bye 1 0 0 0 0 18 Help 1 0 0 0 1 19 Thank You 1 0 0 1 0 20 Fine 1 0 0 1 1 21 No 1 0 1 0 0 22 Yes 1 0 1 0 1 23 Good 1 0 1 1 0 24 Bad 1 0 1 1 1 25 Welcome 1 1 0 0 0 26 Go 1 1 0 0 1 27 Nice 1 1 0 1 0 28 Up 1 1 0 1 1 29 Down 1 1 1 0 0 30 Sorry 1 1 1 0 1 31 Hello 1 1 1 1 0 Few pictures of project are shown in Figure 5.1. Figure 5.1a shows the tuning of a digital glove with LED. As there is a bending of Flex sensor, logic 0 is assigned to LED, resulting LED gets off and reverse action occurred when Flex sensor remains unfetched. Figure 5.1a Tuning with LED Figure 5.1b shows LCD module of the project and also the PCB of the same is been created. Figure 5.1b LCD module 3
- 4. VI. CONCLUSION This digital glove is a handy module which facilitates easy communication for mute/deaf people with everyone. This module communicates exactly as designed version in the form of words/phrases. These phrases are heard as voice and can also be read on the LCD16*2. This module does not impose any extra effort for the knowledge of any technical details for its use. This module built a foundation of a more robust base and portable module to communicate as sentences. A. Advantages The cost of final module is quit affordable i.e. costing just 7000 INR. The power consumption of the module is very low, measuring the precisely the movements of finger. Since the gesture is translated into both voice and text, a conversion can be held between anyone even in between a mute and deaf or in between a blind and mute. B. Limitations This module only works with words/phrases, does not communicate in the form of sentences. The device performs quit slowly as compared with normal speaking people. This module is only dedicated for those people who knows configuration of words with gestures. Few standard gestures are slightly differs causes misinterpretation. C. Future Scope Implementation of a digital glove for both hands will expand the library of words that can be used for communication. Addition of camera in the system can port facial expression in the more appropriate meaning of words. Noise reduction can be improved using noise removal technique. ACKNOWLEDGEMENT We would like to thank ''Adhar Muk Badhir Vidyalaya, Pune'' for providing an information related to deaf mute people. This information is related with different commands which we targeted it as output with respect to different gestures. We would also like to thank HOD INSTRU, COEP for providing us space and lab facility. We would like to thank our mentor Prof Dr.C Y Patil. REFERENCES [1] Saggio, Giovanni. "Bend sensor arrays for hand movement tracking in biomedical systems." In Advances in Sensors and Interfaces (IWASI), 2011 4th IEEE International Workshop on, pp. 51-54. IEEE, 2011 [2] Tanyawiwat, Netchanok, and Surapa Thiemjarus. "Design of an assistive communication glove using combined sensory channels." In Wearable and Implantable Body Sensor Networks (BSN), 2012 Ninth International Conference on, pp. 34-39. IEEE, 2012. [3]Dipietro, Laura, Angelo M. Sabatini, and Paolo Dario. "A survey of glove-based systems and their applications." Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on 38, no. 4 (2008): 461-482 [4]Praveenkumar S Havalagi, Shruthi Urf Nivedita.”The amazing digital gloves that give voice to voiceless.”International journal of advance in engineering and technology(March 2013),vol.6,issue.1,pp 471-480. [5]Masaguppi, Shruti C., and D. R. Ambika. "Virtual Talk for Deaf, Mute, Blind and Normal Humans." in India Educators' Conference (TIIEC), 2013 Texas Instruments, pp. 316-320. IEEE, 2013. [6]Delliraj, M., and S. Vijayakumar. "Design of Smart e-Tongue for the physically challenged people." in Recent Trends in Information Technology (ICRTIT), 2013 International Conference on, pp. 306-311. IEEE, 2013. [7]Prapat Parab, Sanika Kinalekar, Rohit Chavan, Deep Sharan, Shubhadha Deshpande."Hand Gesture Recognition using Microcontroller & Flex Sensor." in International Journal Of Scientific Research And Education , pp. 518-522. March 2014. [8] http://arduino.cc/en/Reference/SD [9] http://playground.arduino.cc/SmartWAV/SmartWAV 4