A Review on Gesture Recognition Techniques for Human-Robot Collaboration

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 10 | Oct 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 632
A Review on Gesture Recognition Techniques for Human-Robot
Collaboration
Sunil G. Deshmukh1, Shekhar M. Jagade2
1 Associate Professor, Department of Electronics and Computer Engineering, Maharashtra Institute of Technology,
Aurangabad, Maharashtra-431010, India
2 Professor, Department of Electronics and Telecommunication, N B Navale Sinhgad College of Engineering,
Kaegaon-Solapur-413255, Maharashtra, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - The sole means of communication for the deaf
and dumb is sign language. These people with physical
disabilities use sign language to express their thoughts and
emotions. A kind of nonverbal communication called a hand
gesture may be used in a number of situations, such as deaf-
mute communication, robot control, human-computer
interface (HCI), home automation, and medical applications.
In hand gesture-based research publications, a variety of
approaches have been used; including those based on
computer vision and instrumented sensortechnologies. Toput
it another way, the hand sign may be classified as a posture,
gesture, dynamic, static, or a mix of the previous two. This
research focuses on a review of the literature on hand gesture
tactics and examines their benefits and drawbacksin different
contexts. Four key technical elements make up the gesture
recognition for human-robot interaction concept: sensor
technologies, gesture detection, gesture tracking, and gesture
classification. The four essential technical components of the
reviewed approaches are used to group them. After the
technical analysis is presented, statistical analysis is also
provided. This paper offers a thorough analysis of hand
gesture systems as well as a quick evaluation of some of its
possible uses. The study's conclusion discusses potential
directions for further research.
Key Words: Hand gesture, Sensor based gesture
recognition, Vision-based gesture recognition, Human-
robot collaboration, Sensor gloves.
1. INTRODUCTION
People with speaking and listening impairments utilize sign
language as a means of transmitting data. Essentially, sign
language is made up of body parts moving to convey
information, emotion, and knowledge. Body language, facial
emotions, and a system manual are all employed in sign
language as a form of communication.Themany expressions
are employed in sign languages as a more natural medium of
interaction, and for those who are deaf or hard of hearing,
they are the only available form. Deaf and dumb persons
who use sign language may communicate without using
words. The Sign language is not a universal languagelikethe
various spoken languages spoken around the globe since it
may vary by region and it is feasible that different nations
may use unique or several sign languages [1].
There may be many sign languages in certain nations,
including India, the United States, the UK, and Belgium.
People utilize it as a kind of nonverbal communication to
convey their feelings and ideas. The non-impaired
individuals found it challenging to communicate with the
impaired people, and in order to do so during any training
related to medical or educational fields. A certified sign
language interpreter is needed [2]. The need for translators
has been steadily rising over time in order to facilitate
efficient communication with persons who are deaf or have
speech impairments. Hand gestures are the major method
used to display the Sign. The Hand Gesture may be further
divided into the global motion and the local motion. While
just the fingers move in a local motion, the whole hand
moves in a global motion [3]. Many implementations of
recognizing of hand gestures technology are being used
today, including surveillance cameras, human-robot
communication, smart TV, controller-free videogames, and
sign language detection, among others. The appearance of a
bodily component that incorporates non-verbal
communication is referred to as a gesture. The Hand
gestures are further divided into a number of types,
including Oral communication, Manipulative, and
Communicative signs [4] [5].
The American Sign Language, the Indian Sign Language,
and the Japanese Sign Language are only a few examples of
the many sign languages that exist today. Whilethesemantic
interpretations of the language might change from place to
region, the overall structure of sign language often stays the
same. A basic "hello" or "bye" hand motion, for instance, can
be the same everywhere and practically in every sign
language [6][7]. When using sign language, gestures are
expressed using the hands, fingers,andoccasionallytheface,
depending on the native languages. Because signlanguage is
a significant form of communication and has a significant
impact on society, it is widely recognized. Without theuse of
an interpreter, hearing-impaired personsmaycommunicate
effectively with non-signers thanks to a strong functional
sign language detection approach [8]. Due to regional
differences in sign language, each nation's sign languagehas
its own rules and syntax as well assomevisuallycomparable
characteristics. For illustration, without the necessary
understanding of Indian Sign language, American Sign
Language users in America could not comprehend Indian
Sign Language [9][10].

1.1 Sign language understanding
The use of language is an essential component of our daily
lives. Without understanding each other's language,
communication might be challenging.Differentcommunities
employ a variety of languages for communication. The
visually challenged utilize sign language as a means of
communication [11]. The primary means of communicating
in sign language is visual, and signs are used to express
sound patterns and meanings. The development of various
sign language standards based on regional and ethnic
differences has been facilitated by manycountries,including
Korean Sign Language (KSL), Chinese Sign Language (CSL),
Indian Sign Language (ISL), American Sign Language (ASL),
British Sign Language (BSL), and Persian Sign Language
(PSL) [12]. The hearing and speech challenged may
effectively communicate with others who are not affected
with the use of sign language recognition technology. The
Sign language recognition systems is enhancing societal,
private, and academic aspects of communication between
those who are impaired and those who are not in a disability
in a nation like India [13].
The main issue with ISL is that there isn't a standardized
database for the language, no common formatting for ISL,
and that there are several variants throughout the nation.
Translating certain words becomestoocomplicatedbecause
they have numerous meanings, such as the word "dear,"
which is particularly tough to deal with since it cannot be
directly translated into the specific set of sign language [14].
The research also demonstrates that English'ssyntax,vocab,
and structure are all well established in Indian Sign
Language. The solitary hand gesture and the double hand
gesture, which represent the tasks that need the most
stretching, are two of the several gestures employed in ISL
[15].
1.2 Barriers to sign language recognition
The identification in sign language is more difficultsinceitis
multimodal. The handshape, movement, and position are
connected with manual aspects whereas the facial
appearance, head position, and lip stance are related with
non-manual elements in the identification process of sign
language. How sign language connects to English words is
one of the primary obstacles to sign language recognition
[16]. To locate English terms with two distinct meanings isa
straightforward test. There would be a symbol with the
identical two senses as the English term if ISL signs
represented English words. For instance, the English word
"right" may signify either "left" or"wrong," dependingonthe
context. There isn't an ISL sign that has these two meanings,
however. In ISL, they are conveyed by two distinct signs,
exactly as they are in French, Spanish,Russian,Japanese,and
the majority of other languages, where they are stated by
two distinct words [17].
1.3 Importance of Indian sign language
We all know that language is the primary means of
intercultural communication. Languages like Hindi, English,
Spanish, and French may be used dependingonthenationor
area that a person is from. Currently, sign language is
utilized as a means of communication by those who are
unable to listen and speak. Hand gestures areusedtoconvey
ideas and what they want to say [18]. Today, it is impossible
to envisage a world without sign language owing to how
challenging it would be for individuals to exist without it.
Without sign language, individuals will become helpless.
People who are unable to listen or speak now have the
chance to live better lives in our world because to sign
language. Now that they can interpret sign language, they
can converse with others. We created an Indian sign
language recognitionsystem,whichcanbedeployedinmany
locations, to enable people to comprehend the language of
the Dumb and the Deaf and make their lives more
comfortable [19].
1.4 Scope of the Indian sign language
Since we are aware that deaf individuals find it challenging
to thrive in environments without sign language
interpreters, we have developed this system to help ussolve
the issues these people face. The ISL detection method may
now be used in banks, where a different system can be
established for the deaf and dumb [20]. A separate screen
can be mounted on the wall and will be visible to all bank
employees while they are at work. Now, when a person
enters a bank, they may go straight to thespecial systemthat
has been established for them and use gestures to present
their questions to the ISL recognition system. The motions
will be recognized by the ISL recognition system, and the
relevant output will appear on the screen. The public's
inquiries may be answered in this manner. The ISL may be
used in grocery shops where consumers can place orders
using gestures and the ISL recognition system, and the
shopkeeper can also place orders using thisframework [21].
1.5 Indian sign language recognition system
applications
There are many ways to use sign language, and the practice
of sign language interpretation is very crucial for the group
of people with disabilities. Through the use of computers,
this group may communicate with other hearing
communities. We may install our computer equipment in
banks, railroads, airports, and other public and private
locations so that dumb and deaf people can communicate
with hearing people using these computers. We may utilize
computers as an interaction between these two groups as
sign language is solely understood by the deaf and dumb
society, but it may be understood by thehearingcommunity.
These computers may provide assistance to such a
community. It allows for the transcription of motions into

text or speech. Our warriors are able to communicate with
one another silently by employing sign language [22][23].
The following are a few uses of the Indian sign language
recognition system:
1. With the use of this method, hearing impaired and dumb
individuals may converse with one other.
2. It aids in the development of a sign language lexicon.
3. It also aids in the empowerment of the Deaf and Deaf
people.
Fig - 1: American sign languages
Fig - 2: Indian sign languages
Fig - 3: Some popular sign languages
One of the most useful forms of communication for those
who cannot hear or communicate is sign language.
Additionally, it helps those who can hear but cannot talk or
vice versa. Deaf and dumb human beings can be benefited
from sign language. Sign language is a collection of various
hand gestures, body postures, and facial and body
movements. This unique gesture is used by deaf individuals
to convey their ideas [24]. Each hand gesture, face
expression, and body movement have a distinct meaning
that is attributed to it [25]. Different sign languages are
utilized in various regions of the globe. Verbal
communication language and culture of any given location
have an impact on the sign language there. For instance,
American Sign Language (ASL) is used in the USA, but
research on Indian sign language has recently begun with
the standardization of ISL[26].Thereareover6000gestures
for words that are often used in finger spelling in American
Sign Language, which also has itsownsyntax.26movements
and single hand are utilized in fingers pronunciations to
represent the 26 English letters. The project's goal is to
create a system that can translate American Sign Language
into text. The sign language gloves are an effective tool for
ensuring that conversations have significance [27]. A
schematic of the ASL, ISL and common sign languages has
been given Fig. 1, 2, and 3, respectively.
Humans often utilizehandgesturestocommunicatetheir
intentions while interacting with otherpeopleortechnology.
A computer and a person are communicating with one
another nonverbally via this medium. An alternate means of
communication for voice technology is this. It is a
contemporary method that allow computercommandstobe
executed after being correctly recorded and understood by
hand gesture. Real-time applications like automated
television controlling [28], automation [29], intelligent
monitoring [30], virtual and augmented reality [31], sign
language identification [32], entertaining, smart interfaces
[33][34], etc. often use human-generated motions or
gestures. Gestures are a continual movement that a human
can easily perceive but that a computer has a hard time
picking up on. Human gestures include motions of the head,
hand, arms, fingers, etc. Relative to other human anatomical
parts, hands are the best understandable and practical
mechanism for Human-Machine Interaction (HMI).
It is separated into two categories: static gestures and
dynamic movements. A static gesture is a certain
arrangement and stance that is shown by a single picture.
And, A dynamic gesture is a mix of stance and a time-related
video sequence. A single frame with minimal metadata and
low processing effort is used for every static gesture.
Dynamic gesture, on the other hand, is better suited forreal-
time application since it is in the format of video and
includes more complicated and significant information.
Additionally, this gesture needed high-end gear for training
and testing. To focus on hand gesture detection, several
researchers have attempted to apply various machine

learning techniques [35]. Hands with fingers, palms, and
thumbs are included in this scientific discipline of hand
gestures in order to detect, identify, and recognize the
gesture. Because of the tiny size relative to other body parts,
increased identification difficulty, and frequent self- or
object occlusions, hand estimation is particularly
challenging. Additionally, it is challenging to build an
efficient hand motion detection system. Since numerous
investigations have been conducted, handgesturedetection,
especially from 2-dimensional video streams using digital
cameras, has been reported.
These methods, however, run into issues such color
sensitivities, opacity, cluttered background, lighting shifts,
and light variance. Investigators can now take into
consideration the 2D scene details thanks to the recently
released version of the Microsoft Kinect camera that has a
realistic and affordable depth sensor. Furthermore, because
of the advanced technology used to create it, the depth
sensor is more resistant to variations in light. The
recognition method has traditionally been carried out by
machine learning algorithms after explicit extraction of the
spatial-temporal hand gesture characteristics [36]. Many of
the elements were manually built or "hand-crafted" by the
user in accordance with the framework toaddressparticular
issues like hand size variation and variableillumination. The
method of feature extraction in the hand-crafted
methodology is problem-oriented. Finding the ideal balance
between effectiveness and precision in computers is also
necessary for the production of hand-craftedcharacteristics.
Deep learning has brought about a paradigm change in
computer vision, however, because to recent advancements
in data and processing resources with strong hardware.
These deep learning-based algorithms have recently been
able to infer meaningful information from movies.
Processing this data doesn't need an outside algorithm.
Interesting results have also been found in a number of
tasks, notably recognition of hand gestures [37]. On the use
of hand gesture recognition inourdailylives,a recentreview
work has shed some light. Some of the frequent uses of hand
gesture recognition include robot control, gaming
surveillance, and sign language understanding. The main
goal of this study is to offer a thorough analysis of hand
gesture recognition, which is the core of sign language
recognition [38].
There are several hardware methods that are utilized to
get data about body location. The majority of trackers are
device- or image-based. Gathering the motions is the first
phase, and then identifying them is the next. This present
research makes an effort to analyze data from flex sensors
installed on a glove in order to recognize the motions that
are done. Despite a number of flaws in the system, such as
uncertainty and noise, the predictionaccuracyforthegadget
is relatively high. The majority of people who are speech-
and hearing-impaired has drastically increased in recent
years. To communicate, these folks use their own language.
However, we must comprehend their language,whichmight
be challenging at times. By employing flex sensors, whose
output is sent to the microcontroller, the project seeks to
solve this problem and enable two-way communication.The
signals are processed by the microcontroller, which
transforms analog impulsesintodigital signals.Theintended
output is also presented with the spoken output when the
gesture has been further analyzed [39].
In consideration of the academic community's growing
interest in gesture recognizing in general and hand gesture
identification in particular during the last two decades, a
multitude of relevant approaches and tools have evolved.
Although several secondary studies haveattemptedtocover
these answers, there are no comprehensive studies in this
field of study. The fundamental contribution of this paper is
the thorough analysis and compilationofsignificantfindings
on hand gesture recognition, supporting technologies, and
vision-basedhandgesture recognitionmethods.Theprimary
methodsandalgorithmsfor vision-basedgesture recognition
are also explored in this paper. This paper offers a thorough
examination of hand gesture analysis methods, taxonomies
for hand gesture identification methods, methodologiesand
programs, implications, unresolved problems, and possible
future research areas.
The literature reviews, polls, andstudiesonhandgesture
recognition are briefly discussedinthissubsection.Areview
of the research on vision-based human motion tracking was
done by Moeslund et al. [40]. The review gives a general
overview of the four motion capture system procedures of
identification, tracking, initiation, and motion analysis. The
authors also analyzed and addressed a variety of system
functioning features and suggested a number of potential
future research avenues to develop this field. Derpanis et al.
[41] discussion on hand gesture recognition included a
variety of angles. The study focuses on a number of ground-
breaking papers in the subjects under discussion, including
feature types, classificationstrategies,gestureset(linguistic)
encoding, and hand gesture recognition implementations.
Common issues in the field of vision-based hand gesture
identification are discussed in the study's conclusion. The
survey by Mitra et al. [42] is an analysis of the prior
literature with a primary emphasis on hand gestures and
facial reactions. Techniques forrecognizingfacial movement
and hand gestures based on vision are presented. This
important paper also identifies current difficulties and
potential research ramifications. The corpus of information
on hand gesture recognition has been reviewed in a
literature study by Chaudhary et al. [43]. The primary
emphasis of this review is on artificial neural networks,
genetic algorithms, and other soft computing-based
methodologies. It was discovered that the majority of
researchers recognize hands in appearance-basedmodeling
using their fingers. Various sensor and vision-based hand
gesture detection algorithms were examined by Corera etal.
[44] in addition to current tools. The article covers the

design issues and logical difficulties associated with hand
gesture recognition methods. The authors also assess the
advantages and disadvantages of sensor- and vision-based
approaches.
2. RESEARCH STRATEGIES
This section of the study highlights the present research
needs by specifying the investigation topics and associated
keywords.
2.1 Questions from literature reviews
The issue of identifying and demonstrating hand motion
recognition techniques is not new. Different strategies,
procedures, and techniques have been created throughout
time to demonstrate the components of hand gesture
recognition. The following SLR research issues will be
addressed in this study:
The following research questions will be addressed by this
SLR:
Q1. What current techniques and supporting
technologies are available forhand gesture recognition?
Findings from addressing the research questions. This part
should address the research queries created to achieve the
objectives of the study. Interacting and communicating with
people may be done via gestures. Recognizing gestures
demonstrates one's capacity to discern the messageanother
person is trying to convey. Gesture recognition is the
practice of identifying human gestures by computers. The
physical movement of one's fingers, arms, hands, or torso
may convey a gesture and convey a message [45].
Gesture recognitionhasbeen usedincomputerscienceto
understand and analyzehumangesturesusingmathematical
algorithms. Although they may be made with the whole
body, gestures usually start with the hands or the face.
Gestures are expressive, meaningful bodily motions that
include physical actions of the fingers, hands, arms, head,
face, or torso, according to research by Mitra and Acharya
[46]. In comparison to a machine's capacity for information
recognition, the human perception of body motions is
distinct from and more spontaneous. Through the use of
senses including hearing and vision, humans are better able
to detect gestures on the spot.
2.2 Hand gestures
Compared to other body parts, the hand is the one that is
used for gesturing the most. As a result, the hand is the ideal
communication tool and candidate for HCI integration.
Natural and efficient non-verbal communication using a
computer interface is made possible by human hand
movements. Hand gestures are the expressive gesture signs
of the hands, arms, or fingers. Hand gesture recognition
includes both static motions with complicated backgrounds
and dynamic gestures that interact withbothcomputersand
people. The hand serves as the machine's input directly; no
intermediary medium is required for the communication
function of gesture recognition [47].
2.2.1 Methods to analyze hand gesture
Sensor-based analysis and vision-based analysisarethetwo
main kinds of hand gesture analysis methodologies that are
utilized to recognize hand gestures. In order to make hand
gestures, the first solution calls for the integration of visual
or mechanical sensors in gloves that also carries electrical
signals incorporated in the finger’s flexor muscles. Using an
additional sensor, the hand position may be determined.
Then, a specific glove with signal-receiving capabilities is
attached to an auditory or magnetic sensor. To transmit
information pertinent to hand position identification, this
glove is linked to a software toolbox [48]. The second
category of methods, referred to as “vision-based analysis,”
rely on the way in which individuals pick up on cues from
their surroundings.Thisapproachextractsseveral attributes
generated from specific photos using a variety of extraction
methods. Most methods for recognizing hand gestures use
mechanical sensors, which is often employed to regulate
signals from the environment and surrounds.
Communication systems that merely utilize signals include
mechanical senses as well. The application of mechanical
sensors for hand gesture identification, on the other hand,is
associated with a few problems with the accuracy and
dependability of the data acquired as well as potential noise
from electromagnetic equipment. The process of gesture
engagement may be facilitated by visual perception [49].
2.2.2 Technology Required for Recognizing Hand
Gestures
Gesture Recognition is a term used todescribetheprocessof
analyzing and translating hand motions into meaningful
instructions. Therefore, the major objective of hand gesture
recognition is to develop a platform that can detect human
hand gestures as inputs and that can then utilize these
gestures to understand and examine information in order to
generate output data that is in accordance with specific
criteria supplied in the input. There are two categories of
hand gesture recognition technologies: 1. Sensor-based and
2. Vision-based methods exist [50].
i. Sensor-based technologies
Sensor-based technologies depend on tools for user
interactions and other real-world objects that relies on
technological advancements. A schematic of varioussensors
for gesture recognition has been used in Fig. 4.

Fig - 4: Different types of gesture recognition sensors
a. Various sensors for gesture recognition
The glove technique makes use of sensors to record the
location and movements of the hands. The motion sensors
on the hand are responsible for detecting the user's hand,
while the sensors on the gloves are responsible for
determining thepropercoordinatesof wheretheuser'spalm
and fingers are located. This method is laborious since the
user must be physically linked to the computer. An
illustration of a sensor-based technique is the data glove
[51]. The advantages of this strategy include highly accurate
and quick response times. In Fig. 5 a glove consisting of
sensors has been shown.
Fig - 5: Sensors based data gloves
The drawback of this strategy is that data gloves are
prohibitively costly, need the use of the whole hardware
toolbox, and are rigid in character. Therefore, the vision-
based technique was developed in order to get over this
restriction.
ii. Marked coloured glove approach
A schematic has been added in Fig. 6 related to the coloured
marked hand glove recognition system. According to this
method, the glove that the human hand will wear is
coloured-coded to guide the operation of monitoring the
hand and finding the palm and fingers that will supply the
precise geometric characteristics that lead to thefabrication
of the prosthetic hand [52].
Fig - 6: Colour marker based hand gesture recognition.
iii. Vision-based technology
For a recognition system to function naturally,a camera isall
that is needed to capture a picture. In real-time applications,
that is more beneficial [53]. Although this approach seems
straightforward, there are several obstacles to overcome
when using it, including a complicated backdrop, varying
lighting, and other objects with different skin tones holding
the hand item. Technologies that rely on vision deals with
the aspects of an image such as texture and colour that are
necessary for gesture recognition. Following picture pre-
processing, various strategies have developed for
recognizing hand objects. Steps for the execution of
programs have been shown in Fig. 7, which shows initiation
from the image capturing to the gesture recognition step.
Fig.-7: Vision based hand gesture recognition system

iv. Appearance based approaches
This method involves modelling the aesthetic appeal of the
input hand pictureutilizing morphological operations,which
is then matched to the extraction of features of the stored
image. Especially in comparison to a 3D model-based
technique, this method is simpler and offers real-time
performance. This strategy's general aim is to look for areas
in a picture that are skin-coloured. Many studies have been
conducted recently thatuseinvariantcharacteristics,suchas
the Ad boost learning method rather of modelling the whole
hand, one may identify specific locations and regions on the
hand by using this invariant property. The advantage of this
approach is that it solves the occlusion issue [54].
v. 3D model-based approaches
Fig - 8: 3D model-based approaches.
This method uses a 3D model to analyze and simulate the
geometry of the hand as depicted in Fig. 8. A depth variable
is provided to the 3D model to increase accuracy since some
data is lost during the 2D projection [3]. Volumetric and
skeleton models are the two types of 3D models. Those two
models had limitations while they were being designed.
Because it deals with the 3D model's foundational look ofthe
user's hand, volumetric models are employed in real world
applications. This system's enormous dimensionality
parameters, which force the designer to operate in three
dimensions, are the fundamental design challenge [55].
vi. Colour based recognition
Based on several coloured indicators as shown in Fig. 9, this
design is used to monitor the mobility of the body or specific
body components. Users may magnify, rotate, sketch, and
scribble on onscreen keyboards bymountingtheRed,Green,
Blue, and yellow-coloured markers, which are used to
communicate with the virtual prototypes. This type offers
more versatility [56].
Fig -9: Colour based recognition system
vii. Skeletal based Recognition
Fig - 10: 3D Skeletal based Model
By restricting the variables, the skeletal modeling fixes the
issues with the volumetric model. The sparse erosion will
result in more efficiency. The feature optimization is
intricate. Compressive sense is employed to retrieve the
sparse signals from a small number of observations, which
helps to save resources [57]. A schematic of 3-D skeletal
based has been depicted in Fig. 10.
viii. Representations techniques of vision-based gesture
The motions of diverse human body gestures may be
described by a variety of paradigms and depictions of
gestures. The 3D Prototype and Appearance-based
techniques are the two primary methods for representing
gestures, as shown in Fig. 11.

Fig -11: Representation of vision-based hand recognition
techniques
Fig -12: Statistical data of the use of 3D prototype and
appearance-based techniques
A statistical study of the use of 3D prototype and
appearance-based techniques in vision-based gesture
identification is shown in Fig. 12. Here, it appears that the
scientificcommunity favors appearance-based methodsover
3D model approach and vision-based methods for gesture
detection. 38.1% of the academic publications discussed 3D
model-based techniques, while 61.9% of the articles
discussed appearance-based techniques [58].
Q2. What are the many situations in which hand
gestures for interactivity are used commercially
available programs, equipment, and application
domains?
Systems for recognizing hand gestures using vision are
crucial for several everyday tasks.
2.3 Domains of applications
This subsection addresses some disciplines that require
gesture interactions since gesture recognition is crucial in
many different fields [59]. Fig. 13 shows the possible
application of hand gesture interaction system.
Fig -13: Application of hand gesture interaction system
a) Virtual reality: Increasing real-world applications and
virtual gestures both have helped to drivesignificantgrowth
for computer hardware.
b) Surgical Clinic: The Kinect technology is oftenutilizedin
hospitals to streamline operations owing to increasing
workload. The many functionalities, such as move, modify,
and zoom in on Ct scanners, MRIs, and other diagnostic
pictures, will be operated by the physicians using the hand
gesture system.
c) Programs for desktop and tablet computers: Gestures
are utilized in these applications to provide an alternative
kind of interactivity to that produced by the keyboard and
mice.
d) Sign Language: The use of signs as a means of
communications is seen as an essential component of
interpersonal communication. This is due to the fact that
sign language is seen to have a stronger structural
foundation and is better suited for test beds used in
algorithmic computer vision. Furthermore, it may be said
that using such algorithms will help persons with
impairments communicate with one another. The literature
has paid a lot of attention to how sign language for the deaf
encourages the utilization of gestures [60].
2.4 Primary shortcomingsanddifficultiesinvision-
based gesture recognition
According to the current assessment, investigation on
gesture recognition and vision-based approaches typically
encounters a number of open problems and difficulties. The
research of hand gesture identification is now confronting a
number of difficulties, including the identification of
invariant characteristics, transition models between
gestures, minimum sign language recognition units,
automated recognition unit segments, recognition

approaches with scalabilityregardingvocabulary,secondary
information, signer independence and mixed gestures
verification, etc.
The static gesture recognition based on vision is thus the
current trend in hand gesture recognition and primarily has
the following two technological challenges:
1. Difficulties detecting targets Target detection involves
extracting the item of interest from the visual stream by
capturing the target against a complicated backdrop. The
separation of the human hand region from other backdrop
areas in a picture is always a challengeforvision-basedhand
gesture identification systems, mostly because of the
variability of the background and unanticipated
environmental elements.
2. Challenges with target identification Understanding hand
gestures helps to clarify the broad implications related to
hand posture and movement. The essential technique for
recognizing hand gestures is to extract geometric invariant
aspects in light of the hand gesture's subsequent properties.
3. Since the hand is a flexible entity, comparable movements
may vary greatly from one another while yet sharing a great
deal of similarities. The human hand has more than 20
degrees of freedom, and it may move in a variety of intricate
ways. As a result, similar gestures done by different persons
may differ from those produced by the same person at a
different time or location.
4. Since identifying finger characteristics is a crucial
component of hand gesture recognition, the hand includes a
lot of duplicate information. One such redundant
information is the palm feature.
3. FUTURE RESEARCH DIRECTIONS AND TRENDS
We were able to identify a number of potential study
avenues and research subjects that may be investigated
further based on our analysis and inspection of more than
150 papers. The main developments and directions in
research for gesture recognition are shown in Fig. 14. First,
SLs contain unique grammatical rules that have not been
well studied in relation totheirequivalentspokenlanguages.
Although there have been relatively few research efforts in
this area, it may be a future avenue for hand gesture
detection problems. Therefore, research into and
incorporation of sign language morphological norms into
spoken expressions may be beneficial for prospective sign
language recognition systems. Second, creating hybrid
techniques is a current trend in many domains that may be
researched and used in hand gesture detection.
Development of hybrid systems incorporating various
algorithms and approaches, as well as non-homogeneous
sensors, such as Kinect, innovative gloves, image sensors,
detectors, etc., is thus urgently needed. This is regarded as
another worthwhile study topic in the area of recognition of
hand gestures.
Fig -14: Future research trends and directions
4. CONCLUSION
The general public, people with physical disabilities, those
playing 3D games, and people communicating nonverbally
with computers or other people may all use hand gesture
detection systems. There is a need for study since hand
gesture recognition systems are being used more and more.
This article provides a critical analysis of the many
approaches employed nowadays. The hand gesture
recognition system's generickeyelementsandtechnique are
detailed. The categorization, monitoring, extraction of
features, and gesture detection methods are briefly
compared. Developing highly effective and intelligent
Human-Computer Interfaces (HCI) requires the use of
vision-based gesture recognition algorithms.Techniquesfor
recognizing hand gestures have become a popular kind of
HCI.
There has been a noticeablechangeinstudyfocustoward
the understanding of sign language. This research also
outlined prospective uses for hand gesture recognition
technologies as well as the difficulties in this fieldfromthree
angles: system, environmental, and gesture-specific
difficulties. The findings of this systematic review also
suggested several possible future study topics, includingthe
urgent need for a focus on linguistic interpretations, hybrid
methodologies, smart phones, standardization, and real-
world systems.
REFERENCES
[1] G Adithya V., Vinod P. R., Usha Gopalakrishnan,
“Artificial Neural Network Based MethodforIndianSign
Language Recognition,”IEEE ConferenceonInformation

and CommunicationTechnologies(ICT),2013,pp.1080-
1085.
[2] Rajam, P. Subha and G Balakrishnan, "Real Time Indian
Sign Language Recognition System to aid Deaf and
Dumb people," 13th International Conference on
Communication Technology (ICCT), 2011, pp 737-742.
[3] Deepika Tewari, Sanjay Kumar Srivastava, “A Visual
Recognition of Static Hand Gestures in Indian Sign
Language based on Kohonen Self- Organizing
MapAlgorithm,” International Journal of Engineering
and Advanced Technology (IJEAT), Vol.2, Dec 2012, pp.
165-170.
[4] G. R. S. Murthy, R. S. Jadon, “A Review of Vision Based
Hand Gestures Recognition,” International Journal of
Information Technology and Knowledge Management,
vol. 2(2), 2009, pp. 405- 410.
[5] P. Garg, N. Aggarwal and S. Sofat. “Vision Based Hand
Gesture Recognition,” World Academy of Science,
Engineering and Technology, Vol.49, 2009,pp.972-977.
[6] FakhreddineKarray,MiladAlemzadeh,Jamil AbouSaleh,
Mo Nours Arab, “Human Computer Interaction:
Overview on State of the Art,” International Journal on
Smart Sensing and Intelligent Systems, Vol. 1(1), 2008,
[7] Appenrodt J, Handrich S, Al-Hamadi A, Michaelis B ,
“Multi stereo camera data fusion for fingertip detection
in gesture recognition systems,” International
conference of soft computing and pattern recognition,”
2010, pp. 35–40.
[8] Auephanwiriyakul S, Phitakwinai S,Suttapak W,Chanda
P, Theera-Umpon N,“Thai sign language translation
using scale invariant feature transform and hidden
markov models,” Pattern Recogn Lett 34(11), 2013, pp.
1291–1298
[9] Aujeszky T, Eid M, “A gesture recognition architecture
for Arabic sign language communication system,”
Multimed Tools Appl 75(14), 2016, pp.8493–8511.
[10] Bao J, Song A, Guo Y, Tang H, “Dynamic hand gesture
recognition based on SURF tracking,” international
conference on electric information and control
engineering, 2011, pp 338–341.
[11]Bellarbi A, Benbelkacem S, Zenati-Henda N,BelhocineM
“Hand gesture interaction using colorbased method for
tabletop interfaces,” 7th international symposium on
intelligent signal processing, 2011, pp. 1–6.
[12]Dhruva N. and Sudhir Rao Rupanagudi, Sachin S.K.,
Sthuthi B., Pavithra R. and Raghavendra, “Novel
Segmentation Algorithm for HandGestureRecognition,”
IEEE, 2013, pp. 383-388.
[13]Gaolin Fang, Wen Gao, and Debin Zhao, “Large
Vocabulary Continuous Sign Language Recognition
Based on Transition-Movement Models,” IEEE
transactions on systems, man, and cybernetics—part a:
systems and humans, Vol. 37(1), 2007, pp.1-9.
[14]Sulochana M. Nadgeri,Dr. S.D.Sawarkar,Mr.A.D.Gaw
ande, “Hand gesture recognition using CAMSHIFT
Algorithm,” ThirdInternational ConferenceonEmerging
Trends in Engineering and Technology, IEEE ,2010,
pp.37- 41.
[15] Vasiliki E. Kosmidou, and Leontios J. Hadjileontiadis,
“Sign Language Recognition Using Intrinsic-Mode
Sample Entropy onsEMGandAccelerometerData,”IEEE
transactions on biomedical engineering, Vol. 56 (12),
2009, pp. 2879-2890.
[16] Joyeeta Singha, Karen Das, “Indian Sign Language
Recognition Using Eigen Value Weighted Euclidean
Distance Based Classification Technique,” International
Journal of AdvancedComputerScienceandApplications,
Vol. 4, No. 2 , 2013, pp. 188-195.
[17]Dipak Kumar Ghosh, Samit Ari, “A Static Hand Gesture
Recognition AlgorithmUsing K-MeanBasedRadial Basis
Function Neural Network,” IEEE, 2011, pp. 1-5.
[18]N. D. Binh, T. Ejima, “Real-Time Hand Gesture
Recognition Using Pseudo 3-D Hidden Markov Model,”
5th IEEE International Conference on Cognitive
Informatics, 2, 2006, pp. 820-824.
[19] Y. Han, “A Low-Cost Visual Motion Data Glove as an
Input Device to Interpret Human Hand Gestures,” IEEE
Transactions on Consumer Electronics, vol. 56, 2010, pp.
501-509.
[20]L. Dipietro, A. M. Sabatini, P. Dario, “Survey of Glove
Based Systems and Their Applications, “ IEEE
Transaction on Systems, Man and Cybernetics, Part C:
Applications and Reviews, vol. 38, 2008, pp. 461-482.
[21]X. Y. Wang, G. Z. Dai, X. W. Zhang, “Recognition of
Complex Dynamic Gesture Based on HMM-FNN Model,”
Journal of Software, vol. 19, 2008, pp. 2302-2312.
[22]F. Quek, T. Mysliwiec, M. Zhao, “Finger Mouse: a Free-
hand Pointing Interface,” IEEE International Conference
on Automatic Face and Gesture Recognition-FGR, 1995,
pp. 372-377.
[23]J. Triesch, C. V. D. Malsburg, “A System for Person-
independent HandPostureRecognitionagainstComplex

Backgrounds,” IEEE Transaction on Pattern Analysisand
Machine Intelligence, vol. 23, 2001, pp. 1449-1453.
[24] B. Stenger, “Template Based Hand Pose Recognition
Using Multiple Cues,” Lecture Notes in ComputerScience,
Vol. 3852, 2006, pp. 551-560.
[25] C. Keskin, F. Kirac and Y. E. Kara, Lale Akarun, “Real
Time Hand Pose Estimation Using Depth Sensors,” In
Proceedings of the IEEE International Conference on
Computer Vision Workshops, 2011, pp. 1228–1234.
[26] J. Davis, M. Shah, “Visual Gesture Recognition,” IEEE
Proceedings on Vision, Image and Signal Processing, vol.
141 1994, pp. 101-106.
[27]Jerome M. Allen, Pierre K. Asselin, Richard Foulds,
“American Sign Language Finger Spelling Recognition
System,” IEEE, 2003, pp. 285-286.
[28]Nimitha N, Ranjitha R, Santhosh Kumar V, Steffin
Franklin, “Implementation of Real Time Health
Monitoring System Using Zigbee For Military Soldiers,”
International Journal for Research in Applied Science &
Engineering Technology (IJRASET), Volume 3, Issue IV,
April 2015, pp. 1056-1059.
[29]A. S. Ghotkar , G. K. Kharate, “Hand Segmentation
Techniques to Hand Gesture Recognition for Natural
Human Computer Interaction,” International Journal of
Human Computer Interaction(IJHCI), ComputerScience
Journal, Malaysia, Volume 3, no. 1, April 2012,pp.15-25.
[30]U. Rokade, D. Doye, M. Kokare, “Hand Gesture
Recognition Using Object Based Key Frame Selection,”
International Conference on digital Image Processing,
IEEE Computer Society, 2009, pp. 228-291.
[31]N. D. Binh, E. Shuichi, T. Ejima, “Real time HandTracking
and Gesture Recognition System,” ICGST International
Conference on Graphics, Vision and Image Processing,
GVIP 05 Conference, 2005, pp. 362-368.
[32] T. Maung,“Real-Time Hand Tracking and Gesture
Recognition System Using Neural Networks”, PWASET,
Volume 38, 2009, pp. 470-474.
[33]Feng-Sheng Chen, Chih-Ming Fu, Chung-Lin Huang;
“Hand gesture recognition using a real-time tracking
method and hidden Markov models,” Image and Vision
Computing, vol. 21,2003, pp. 745-758.
[34]Yu Bo, Chen Yong Qiang, Huang Ying Shu,Xia Chenjei;
“Static hand gesture recognition algorithm based on
finger angle characteristics,” Proceedings of the 33rd
Chinese Control Conference, 2014, pp. 8239-8242.
[35] Palvi Singh, Munish Rattan, “Hand Gesture Recognition
Using Statistical Analysis of Curvelet Coefficients,”
International Conference on Machine Intelligence
Research and Advancement, IEEE, 2013, pp 435-439.
[36]Liu Yun, Zhang Lifeng, Zhang Shujun, “A Hand Gesture
Recognition Method Based on Multi-Feature Fusionand
Template Matching,” Procedia Engineering, vol. 29,
2012, pp. 1678-1684.
[37] U. Rokade, D. Doye, M. Kokare, “Hand Gesture
Recognition by thinning method,” International
Conference on digital Image Processing, IEEE Computer
Society, 2009, pp. 284-287.
[38] Liling Ma, Jing Zhang, Junzheng Wang ,“Modified CRF
Algorithm for Dynamic Hand Gesture Recognition,”
Proceedings of the 33rd Chinese Control Conference,
IEEE, 2014, pp. 4763-4767.
[39]C. Wang and S. C. Chan; “A new hand gesture recognition
algorithm based on joint color-depth Super-pixel Earth
Mover’s Distance”;International WorkshoponCognitive
Information Processing, IEEE, 2014, pp. 1-6.
[40]Moeslund TB, Granum E, “A survey of computer vision-
based human motion capture”, Comput Vis Image
Underst, vol. 81(3), pp 231–268.
[41]Derpanis KG, “Mean shift clustering,” Lecture notes, vol.
32, 2015, pp 1-4.
[42] Mitra S, Member S, Acharya T, Member S, “Gesture
recognition: a survey,” IEEE Trans SystManCybernPart
C Appl Rev, vol. 37(3), 2007, pp. 311–324
[43]Chaudhary A, Raheja J, Das K, Raheja S, “Intelligent
approaches to interact with machines using hand
gesture recognition in natural way: a survey,” Int J
Comput Sci Eng Surv, vol. 2(1), 2011, pp. 122–133.
[44]Corera S, Krishnarajah N, “Capturing hand gesture
movement: a survey on tools, techniques and logical
considerations,” Proceedings of chi sparks, 2011, pp. 1-
11.
[45]Hu M, Shen F, Zhao J, “Hidden Markov models based
dynamic hand gesture recognition with incremental
learning method,” international joint conference on
neural networks (IJCNN), 2014, pp. 3108–3115.
[46]Huang D-Y, Hu W-C, Chang S-H, “Vision-based hand
gesture recognition using PCA+Gabor filters and SVM,”
fifth international conference on intelligentinformation
hiding and multimedia signal processing, 2009, pp 1–4.

[47]Huang D-Y, Hu W-C, Chang S-H , “Gabor filter-based
hand-pose angleestimationforhandgesturerecognition
under varying illumination,” ExpertSystAppl,vol.38(5),
2011, pp. 6031–6042
[48]Huang D, Tang W, Ding Y, Wan T, Wu X, Chen Y, “Motion
capture of hand movements using stereo vision for
minimally invasive vascular interventions,” sixth
international conference on image and graphics, 2011 ,
pp. 737–742
[49] Ibarguren A, Maurtua I, Sierra B, “Layered architecture
for real time sign recognition: Hand gesture and
movement,” Eng Appl Artif Intell, vol. 23(7), 2010, pp.
1216–122.
[50]Ionescu B, Coquin D, Lambert P, Buzuloiu V, “Dynamic
hand gesture recognition using the skeleton of the
hand,” EURASIP J Adv Signal Process , vol. 13, 2005, pp.
2101–2109.
[51]Ionescu D, Ionescu B, Gadea C, Islam S, “An intelligent
gesture interface for controlling TV sets and set-top
boxes,” 6th IEEE international symposium on applied
computational intelligenceandinformatics(SACI), 2011,
pp. 159–164.
[52] Ionescu D, Ionescu B, Gadea C, Islam S, “A multimodal
interaction method that combinesgesturesandphysical
game controllers,” proceeding-international conference
computer.Communicationnetworks,ICCCN,2011,pp1–
6.
[53]Jain AK, “Data clustering: 50 years beyond K-means,”
Pattern Recogn Lett, vol. 31(8), 2010, pp. 651–666.
[54] Ju SX, Black MJ, Minneman S, Kimber D, “Analysis of
gesture and action in technical talks for video indexing,”
Proceedings of IEEE computer society conference on
computer vision and pattern recognition, 1997,pp595–
601.
[55]Just A, “Two-handed gestures for human-computer
interaction,” Research report IDIAP, 2006, pp. 31-43.
[56]Kaâniche M,“Gesturerecognitionfromvideosequences,”
Diss. Université Nice Sophia Antipolis, 2009, pp. 57-69.
[57]Kanungo T, Mount DM, Netanyahu NS, Piatko CD,
Silverman R, Wu AY, “An efficient k-means clustering
algorithm: analysis and implementation,” IEEE Trans
Pattern Anal Mach Intell, vol. 24(7), 2002, pp. 881–892.
[58]Karam M, “A framework for research and design of
gesture-based human-computer interactions,” Doctoral
dissertation, University of Southampton, 2006, pp. 130-
138.
[59]Kausar S, Javed MY, “A survey on sign language
recognition,” Front Inf Technol, 2011, pp. 95–98
[60]Kılıboz NÇ, Güdükbay U, “A hand gesture recognition
technique for human–computer interaction,” J Vis
Commun Image Represent, vol. 28, 2015, pp. 97–104.
BIOGRAPHIES
Sunil G. Deshmukh acquired the
B.E. degree in Electronics
Engineering from Dr. Babasaheb
AmbedkarMarathwada University,
Aurangabad in 1991 and M.E.
degree in Electronics Engineering
from Dr. Babasaheb Ambedkar
Marathwada University (BAMU),
Aurangabad in 2007. Currently, he
is pursuing Ph.D. from Dr.
BabasahebAmbedkarMarathwada
University, Aurangabad, India. He
is the professional member of
ISTE, His areas of interests include
image processing, Human-
computer interaction.
Shekhar M. Jagade secured the
B.E. degree in Electronics and
Telecommunication Engineering
from Government Engineering
College, Aurangabad in 1990 and
M.E. degree in Electronics and
Communication Engineering from
Swami Ramanand Teerth
Marathwada University,Nandedin
1999 and Ph.D from Swami
Ramanand Teerth Marathwada
University, Nanded in2008.Hehas
published nine paper in
international journals and three
papers in international
conferences. He is working as a
vice-principal in the N. B. Navale
Sinhgad College of Engineering,
Kegaon-Solapur, Maharashtra,
India. He is the professional
member of ISTE, ComputerSociety
of India, and Institution of
Engineers India. His field of
interests includes image
processing, micro-electronics,
Human-computer interaction.

A Review on Gesture Recognition Techniques for Human-Robot Collaboration

More Related Content

Similar to A Review on Gesture Recognition Techniques for Human-Robot Collaboration (20)

More from IRJET Journal (20)

Recently uploaded (20)

A Review on Gesture Recognition Techniques for Human-Robot Collaboration