Smart Robotic Assistant using IOT

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 06 | June -2017 www.irjet.net p-ISSN: 2395-0072
© 2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 857
Smart Robotic Assistant Using IoT
Mr. Ananthapadmanabhan J1, Ms. Annu Mariam Abraham 2, Mr. Libin M George3 ,
Ms. Vineetha Anna Saji4 , Prof. Anil A R5
1234Student, Dept. of Computer Science &Engineering, Sree Buddha College of Engineering, Kerala, India
5Professor, Dept. of Computer Science &Engineering, Sree Buddha College of Engineering, Kerala, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - A Smart Robotic Assistant (RA) that will help
human being a lot in multiple ways is developed. The area
used is Internet of Things (IoT). The RA operates on human
voice commands and gestures commands, given remotely by
using an Android platform based smart IoT device like smart
phone. The voice and gesture commands are convertedtotext
form and is the communicated to the RA over a Wi Fi network.
The RA is provided with mechanical armstopickanobjectand
place it in another location. An Arduino microcontrollerbased
platform is used to develop this smart RA. The RA is capable of
performing different operations like start/stop, move
left/right, forward/backward, pickingupobjectandplacingit
in another location. These peculiarities are really helpful for
humans especially for elderly people. The applications of
robotic assistant including homes , hospitals, industries etc.
Key Words: smart robotic assistant, voice commands,
gesture commands, Wi Fi network, Android based smart
IoT devices.
1. INTRODUCTION
A robot is a machine especially one programmable bya
computer .the robot is capable of carrying out a complex
series of actions automatically. They are guided by an
external control device. The branch of technology that deals
with the design, construction, operation, and application of
robots, as well as computer systems for their control,
sensory feedback, and information processing is robotics.
Robots have replaced humans in performing repetitive and
dangerous tasks.
An internet is an interconnected network providing a
variety of information and communication facilities. The
main advantage of internet is that it helps us to send
messages quickly between computersaroundtheworld.The
Internet of Things (IoT) is a concept in which surrounding
objects are interconnected through wired and wireless
networks without user intervention.
The smart RA developed here will perform its operations
according to both gesture and voice commands. The
operations performed are move forward, backward,leftand
right. The RA can pick and release objects too. This will help
the elderly people very much. A 3d printed arm, an Arduino
module and Wi-Fi module is used. A total of 4 dc motors are
used 2 for movement of RA, 1 for up and down movement
and 1 for picking and releasing object. Rechargeable lithium
battery is used for carrying out these functions.
2. PROPOSED SYSTEM
Smart robotic assistants help human beingsinreducing
the manual efforts in day-to-day tasks and the risk to
precious human lives in hazardous situations. In this paper,
we discuss about a smart robotic assistant that operates on
human voice and gesture commands, given remotely by
using an Android platform based smart IoT device. The real-
time signal processing of the voice commands is carried out
using a MQTT (Message Queue Telemetry Transport)server.
The speech command signal converted to text form is then
communicated to the robotic assistantovera Wi-Finetwork.
The robotic assistant is developed on an Arduino micro-
controller based platform.
2.1 Microcontroller
The first requirement for the design of the robot is the
microcontroller. In this project, Intel Edison Arduino
microcontroller has been used. The Intel Edison module is a
SoC(System on Chip) that includes an Intel Atom 500MHz
dual-core, dual-threaded CPU and an Intel Quark 100MHz
microcontroller. Intel Edison Kit for Arduino provides the
Arduino 1.0 pinout and standard connectors such as a micro
USB connected to a UART, a USB OTG port that can be
switched between a second micro USB device connector, a
standard size USB host Type- A connector, a SD card holder,
and a DC power jack. The Intel Edison Kit for Arduino makes
possible to have 20 digital input/output pins, of which 6 can
be used as analogue inputs.
3. WORKING
In this work, movements of the smartRAarecontrolled
by voice and gesture commands given through a smart
mobile phone using Android OS based platform.
The voice commands signal is converted to the text
form. Using a smart IoT device, this text command is then
transmitted to the Wi Fi module on-board the robot via
MQTT server. The Wi Fi module receives the signal and then
sends commands to the micro-controller, on-board the RA’s
body. The micro-controller in turn processes the commands

and controls all the 4 DC motors used for various
movements. Voice commands make the control of RA easier
for the humans, thus providing a better and user friendly
interface. The accuracy of voice to text transformation
depends upon the distance between the mouth and the
smart IoT device, and the signal strength.
The schematic block diagram of the operation of RA is
shown in Fig. 1, with the movements of RA’sbody,handsand
arms. Human voice or gesture commands are given using an
android based smart IoT device using Google’s voice and
gesture recognition system.
Fig -1: Schematic block diagram of smart robotic assistant
(RA)
Smart IoT device is used here for wireless
communication. A smart phone is used as an IoT device that
uses Wi Fi. In the receiver side, the micro-controller ports
are used as interface between the Wi Fi receiver and the RA.
The movements of the RA’s body, hands and arm are
independent of each other and all can be controlled at the
same time, since each function is controlled by motors. The
main modules in the voice recognition system are:
GUI: It is used for interacting with the user. Basic structure
of application is reflected in GUI.
Voice Recognition: For intelligent voice assistant
application is done using Google Server. The conversion of
acoustic speech into a set of words using a software
component is the process done here. For a speech
recognition system, its accuracy depends on differ in
vocabulary size and confusability, modality of speech
(isolated, discontinuous, or continuous speech, read or
spontaneous speech), task and language constraints. The
system consists of five modules: feature extraction, phone
model training, dictionarypreparation,grammarestimation,
and sentence decoding.
TTS Engine: A text-to-speech (TTS) system converts
normal language text into speech. Creation of synthesized
speech can be done using the concatenation of pieces of
recorded speech that are stored in a database. The output is
given in the form of speech
The operations done by the smart RA is as follows, the RA
moves forward when the voice command is forward, moves
backwards when voice command is backward, moves right
when voice command is right, moves left when voice
command is left, arm moves up when voice command is up,
arm moves down when command is down, RA pick object
when command is pick, drops the object when command is
drop.
Gestures have been an important means of
communication in the physical world from ancient times,
even before the invention of any language. In this era of
machines taking control of every complex works,
interactions with machines have become more important
than ever. Robots are classified into two types: Autonomous
robots like Line sensing or edge sensingrobots,andRemote-
controlled robots like Gesture controlled Robots.
The gesture commands like pinch, un pinch, swipe up,
swipe down, single tap, double taps are given through
mobile application and is converted using inbuilt gesture
libraries. The commands obtained is transmitted to a robot
via a Wi-Fi module of smartphone using an android app.
Further, it is processed by a Arduino microcontroller
embedded on the robot for its desirable motions.Robotscan
be used for performing various tasks that the humans are
unable to perform due to size limitation, physical disability
etc.
An android operated phone is used as the platformforthe
Java application to execute. An Intel Edison Arduino
microcontroller is incorporated in the robot for the main
computation and the main communication between all the
modules. Then there is a motor driver that deals with the
computation and functioning of the motors to turn the
wheels essential for the movement of the robot. Last,butnot
least, a Wi Fi module is incorporated in the robot that serves
as the means of receiving the data from the Smartphone
which is processed in the Arduino to detect the direction of
movement of the user’s hand and move the robot
accordingly.
The gesture commands drawn in the android platform
and corresponding working is as follows swipe up: move
forward, swipe down: move backwards,swipeleft:move left,
swipe right: move right, single tap: arm moves up, double
tap: arm moves down, pinch: picks object, un-pinch: release
object.

Fig -2: Smart Robotic Assistant
4. RESULT
From the observation, the working of RA depends upon:
 The bandwidth of Wi Fi module.
 Accuracy of Google’s speech to text conversion.
The Smart RA has a wide range of applications in:
Helping elderly peoples, Peoples having severe disabilities,
Can be used as a pick and place robot in industries and
laboratories.
5. CONCLUSION
In our proposed system, we are using a smart robotic
assistant that will work according to voice command and
gesture commands. The voice and gesture commands are
converted to text form and are transmitted to the board.
From there it is moved to the corresponding motors. Thus,
the wheels are rotated. The RA will perform movement
operation like move right, left, forward, and backward. The
arm will move up and down also able to pick and release
objects.
6. FUTURE WORKS
Future works that can exploit are the effect of the
distance between the mouth and the smart IoT deviceonthe
performance of the robot, and the effect of noise on the
speech to text conversion. The cost of the RA can possibly be
reduced by using renewable source of energy. The RA
provides for more development of applications based on
android operating system. Such as, Application based on
sensors. This opens door for wide range of possible similar
applications such as automation of household tasks and
remote starter for car, military applications, dangerous
exploration environments. The weight carrying capacity of
the robot can be increased by using a stronger metal of the
arms, and the motors with higher torque. Also precision of
its movement can be improved by using encoders in the
error feedback path.
ACKNOWLEDGEMENT
We are deeply expressing our sincere gratitude to our
project guide Prof. Anil A R who always supported and
guided us with valuablecomments.Weexpressourimmense
pleasure and thankfulness to all faculty members of the
Department of Computer Science & Engineering of Sree
Buddha College of Engineering, Alappuzha.
REFERENCES
[1] Wong Guan Hao, Y. Y. Leck and Lim Chot Hun, “6-DOF
PC-Based Robotic Arm PC-ROBOARM with efficient
trajectory planning and speed control”, in Proc. 4th
International Conference Mechatronics (ICOM), Kuala
Lumpur, pp. 1-7, April 2011, ISBN:978-1-61284-435-0.
[2] H. Uehara, H. Higa and T. Soken, “A Mobile Robotic Arm
for people with severe disabilities”, in Proc. 3rd IEEE
International Conference on Biomedical Robotics and
Biomechatronics (BioRob), RAS and EMBS , Tokyo, pp.
126-129, September 2010, ISSN:2155-1774.
[3] David Orenstein, “People with paralysis control robotic
armsusingbrain”,https://news.brown.edu/articles/201
2/05/braingate2 (Last viewed on October 23, 2014).
[4] Lin. H. C, Lee. S. T, Wu. C. T, Lee. W. Y and Lin. C. C,
“Robotic Arm drilling surgical navigation system”, in
Proc. International conference on Advanced Robotics
and Intelligent Systems (ARIS),Taipei,pp.144-147, June
2014.
[5] Rong-Jyue Wang, Jun-Wei Zhang, Jia-Ming Xu and Hsin-
Yu Liu, “The Multiple-functionIntelligentRoboticArms”,
in Proc. IEEE International Conference on Fuzzy
Systems, FUZZ-IEEE, Jeju Island,pp.1995 -2000,August
2009, ISSN:1098-7584
[6] S. Karamchandani, S. Sinari, A. Aurora and D. Ruparel,
“The Gesture Replicating Robotic Arm”, International
Symposium Computational and Business Intelligence
(ISCBI), New Delhi, pp. 15-19, August 2013
[7] M. Balaganesh, E. Logashanmugam, C. S. Aadhitya andR.
Manikandan, “Robotic Arm showing writing skills by
speech recognition”, InternationalConference on
Emerging Trends in Robotics and Communication
Technologies (INTERACT), Chennai, pp. 12 - 15,
December 2010.
[8] M. Wongphati, Y. Matsuda, H. Osawa and M. Imai,
“Where do you want to use a robotic arm? And what do
you want from the robot?”, IEEE RO-MAN, Paris, pp.
322-327, September 2012, ISSN:1944-9445.
[9] P. Krasnansky, F. Toth, V. V. Huertas, Rohal’-Ilkiv, “Basic
laboratory experiments with an educational robotic
arm”, Process Control (PC), Strbske Pleso, pp. 510 - 515,
June 2013.
[10] Karmoker, S. Polash, Zakir Hossan, “Design of a low cost
PC interface Six DOF robotic arm utilizing recycled
materials”, Electrical Engineering and Information &
Communication Technology (ICEEICT), Dhaka, pp. 1 - 5,
April 2014.

Smart Robotic Assistant using IOT

More Related Content

What's hot (20)

Similar to Smart Robotic Assistant using IOT (20)

More from IRJET Journal (20)

Recently uploaded (20)

Smart Robotic Assistant using IOT