IRJET- Smart Home Automation using IoT and Deep Learning

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 04 | Apr 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 1
Smart Home Automation using IoT and Deep Learning
Rishabh Dev Manu1, Sourav Kumar2, Sanchit Snehashish3, K.S. Rekha4
123B.E, Computer Science & Engineering, The National Institute of Engineering, Mysuru, Karnataka, India
4 Assistant Professor, B.E, Computer Science & Engineering, The National Institute of Engineering, Mysuru,
Karnataka, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract – Human Activity Recognition and Home
Automation have been important topics of interest to
many researchers in recent years. Due to its numerous
benefits, home automation is becoming popular. Home
automation refers to control of home appliances and
domestic features through local networking or remote
control. Artificial Intelligence provides us with the real-
time decision-making and automation framework for
Internet of Things (IoT).Theworkfocusesontheconcept
of home automation by using their smartphone data to
recognize the residents ' human activities. In this paper,
we propose a smart home system thatrecognizeshuman
activities through a Long Short Term Memory (LSTM)
Deep Learning algorithm and then performs pre-
determined tasks based on the recognized activity.
Key Words: Smart Home,HomeAutomation,DeepLearning,
Long Short Term Memory, LSTM, Smartphone,
accelerometer, sensors, Human Activity Recognition,
Raspberry Pi.
1. INTRODUCTION
For many researchers, there has been a topic of interest in
recognition of human activities in recent years. In this
project, we are proposing a smart home system that bydeep
learning algorithm recognizes human activities. We use the
WISDM (Wireless Sensor Data Mining) dataset [1], where
data is collected in a controlled laboratory setting using the
smartphone's accelerometer. With these data, we will train
our deep learning model to predict human activities
performed inside the home and use these predictions to
react to various human activities inside the smart home.
A convergence of technologies in machine learning and
omnipresent computing as well as the development of
robust sensors and actuators has brought interest in the
development of smart environments to emerge and support
valuable functions in DailyLivingActivities(ADLs). Theneed
for such technologies to be developed is underlined by
population aging, the cost of formal health care, and the
importance individuals place on remaining independent in
their own homes. Individuals need to be able to complete
daily living activities such as eating, dressing, cooking,
drinking, reading, taking medicine, sleeping, to function
independently at home. Automating activity recognition is a
crucial step towards monitoring a smart home resident's
functional health and helping them perform these activities
effectively.
Before smart home technologies can be deployed for these
older people, several challenges should be resolved,
including data collection, algorithms foractivityrecognition,
etc. This technology can be used wildly in the future if the
accuracy is sufficiently higher. There is a research project
called the Advanced Studies Center in Adaptive Systems
(CASAS) where only passive, non-intrusive sensors [2] are
deployed at Washington State University to create an
intelligent home environment.
First, we should design a suitable algorithm. And then the
actual data is used to test the algorithm that we choose. We
will use data collected from accelerometersensors.Virtually
every modern smartphone has a tri-axial accelerometerthat
measures acceleration in all three spatial dimensions.
Additionally, accelerometers can detect device orientation.
We will train an LSTM Neural Network for Human Activity
Recognition (HAR) from accelerometer data. The trained
model will be used to predict different activities by the
resident of the home. After that we will Implement different
IoT based applications using the result of the different
recognized activities.
We are not just limiting the control with the trained
model, but the smart home will also be designed in such a
way that the users can control and get all the details of the
smart homes in their smartphones in case of emergency or
the failure of the above system.
2. LITERATURE SURVEY
Deep learning algorithm is an effective way for recognizing
human activities in smart homes [3]. They used a network
having 4 hidden layers and this was pre-trained layer by
layer using the algorithm called Restricted Boltzmann
Machine (RBM). Then the fine-tuning work started using CG

algorithm. Their deep learning model is used to solving the
problem about recognizing human activities,theresultswas
compared with hidden Markov model and naïve Bayes
classifier. But there are still some challenges we must
resolve, such as the number of the units in each layer, and
the value of the epoch. Ultimately, they found deep learning
to be more effective in terms of activity recognition. The
performance evaluated withthereal data thatwerecollected
from smart homes showed great significance in this aspect.
The results of their deep learning model are better than
those traditional approaches, such as HMM and NBC.
S. Szewcyzk, Dwan, Minor, Swedlove, and
Swedlove, and D. Cook[4] investigated four alternative
mechanisms with a corresponding activity label to annotate
sensor data. They used sensor data collected in a real smart
apartment to evaluate alternative methods along the
dimensions of annotation time, resident burden, and
accuracy. Motion sensors, Burner sensors, hot water sensor
in the kitchen, and cold water sensor in the kitchen are the
sensors used for data collection. The labeling activities
include sleeping, eating, personal hygiene, preparinga meal,
working on a computer, watching television, and others.
The home automation system that uses Wi-Fi
technology system consistsofthreemaincomponents;a web
server that presents system core that controls and monitors
the home and hardware interfacemoduleofusers,providing
the appropriate interface for home automation system
sensors and actuators. The system is better than the
commercially available home automation systems from the
point of view of scalability and flexibility. The user can use
the same technology toconnecttothe web-basedapplication
on the server. If server is connected to the internet, so
remote users can access server web-based application
through the internet using compatible web browser.
Many home automation systems are now using the
protocol Message Queuing Telemetry Transport (MQTT) to
communicate with the devices. MQTT's popularity can be
attributed to the fact that it is an easy to implement
lightweight protocol. Although MQTT messaging uses an
unsecured Transmission Control Protocol (TCP), when
implementing mission-critical business, wecanencryptdata
with TLS / SSL Internet security to make it robust. A client
can subscribe on the basis of a pattern to all published
topics. MQTT defines three QoS which, based on the
importance of each message and the repetitiveness of the
messages in the environment, can cater to the client.
Arun Cyril Jose and Reza Malekian [5] show how
modern homes have changed the concept of securityand the
meaning of the word "intruder." The paper highlights the
weaknesses in identifying and preventing sophisticated
intruders in a home environment from existing home
automation systems. For future work in the field of home
automation security, the researchers are encouraged to
consider a home automation system and develop behavior
prediction and advancedsensingparametersthatcanhelpto
identify and prevent skilled and sophisticated intruders.
Security is vital for the proper implementation and
development of the home automation systems.
The methods which utilizedinhabitantfeedback not
only increased the accuracy ofthemodelsbutalsodecreased
the annotation time, since the annotators had a much
smaller set of possible activities to associate with each half
hour of sensor data. In addition, the visualizer provided
better results than the raw data because the annotator got a
better sense of what was happening in the smart apartment.
3. METHODOLOGY
This section describes the activity recognition task and the
process for performing this task. We trained a Deep Neural
Network (DNN) to recognize the type of movement
(Walking, Running, Jogging, etc.) based on a given set of
accelerometer data from a mobile device carried around a
person’s waist. The recognized activity was then used to
carry out various control operations inside the smart home
using Raspberry Pi.
3.1 Proposed Algorithm
The block scheme of the proposed algorithm for LSTM
Human Activity Recognition is shown in figure.
Fig-1: Steps in the proposed algorithm
The algorithm is performed predominantly in the following
steps:

a) Pre-processing the data and changing its dimension to
54901x200x3.
b) Building a model which contains 2 fully-connected and 2
LSTM layers (stacked on each other) with 64 units each.
c) Training the Model for 50 epochs.
d) Evaluating the model and predicting the human activity
by taking input from the mobile phone’s accelerometer.
e) Performing the pre-determined actions through
Raspberry Pi inside the smart home basedontherecognized
activity.
3.2 The Data
We used data provided by the Wireless Sensor Data
Mining (WISDM) Lab. The dataset was collected in a
controlled, laboratory setting. The dataset contains
1,098,207 rows and 6 columns. There are no missing values.
This dataset is a collectionofaccelerometerdatatakenfroma
smartphone that various people carried with them while
conducting six different activities i.e., Jogging, Sitting,
Standing, Walking, Upstairs, and Downstairs.
For each activity, the acceleration for the x, y, and z axis was
measured and captured with a timestamp and person ID.
Fig-2: Accelerometer Data (WISDM Lab Dataset)
The columns weweremoreinterestedinwereactivity,x-axis,
y-axis, and z-axis. The number of training examples by
activity type and by user are shown.
Chart-1: Training Examples by Activity Type
Chart-2: Training Examples by User
As we can see, the datasetcontainsmoredata forwalking
and jogging activities than the others. Also, it is visible from
the above chart that 36 persons have participated in the
experiment. The accelerometer data for each of the three
axes for all six possible activities is recorded at a sampling
rate of 20 Hz (20 values per second). Since we show the first
180 records, each chart shows a 9 second interval for each of
the six activities (calculation: 0.05 * 180 = 9 seconds).
Fig-3: Jogging Accelerometer Data
Fig-4: Downstairs Accelerometer Data
3.3 Data Preprocessing
Since our LSTM model expected fixed length sequences
as training data, so we generated sequences each containing
200 training examples. After this step, the training data size
was drastically reduced. We took the most common activity

and assigned it as a label for the sequence. Thedata wasthen
transformed into sequences of 200 rows, each containing x,
y, and z. A one-hot encoding was also applied to the labels.
Finally, the data was split into training(80%)andtest(20%)
set.
3.4 Building Model and Training
The model contains 2 full-connected and 2 LSTM layers
(stacked on each other) with 64 units each. We used an
optimizer with a learning rate of 0.0025 and trained the
model for 50 epochs keeping track of accuracy and error.
4. RESULTS
Our model seems to learn well with accuracyreachingabove
97% and loss hovering around 0.2.
Fig-5: Training Session’s progress over iteration
The confusion matrix for model’s prediction is shown in
figure:
Fig-6: Confusion Matrix for Model’s Prediction
Again, it looks like our model performed good. Somenotable
exceptions include the misclassification of Upstairs for
downstairs and vice versa.
5. CONCLUSIONS
We have built an LSTM model that can predict human
activity from 200 time-step sequence with over 97%
accuracy on the test set. The identified smart home project
utilizes a wide range of technologies serving different goals.
The integration of Bluetooth and Wi-Fi technology in the
control of home appliances can help and improve the
lifestyle of all user groups in terms of safety and comfort,
especially for the disabled and the elderly. In terms of
recognition of activity, deep learningisquite effective.Inthis
aspect, the performance assessed with the actual data
collected from smart homes shows great significance.
REFERENCES
[1] Jennifer R. Kwapisz, Gary M. Weiss and Samuel A.Moore
(2010). Activity Recognition using Cell Phone
Accelerometers, ProceedingsoftheFourthInternational
Workshop on Knowledge Discovery from Sensor Data
(at KDD-10), Washington DC.
[2] Rashidi, P., & Cook, D. J. (2008). Adapting to resident
preferences in smart environments.In AAAI Workshop
on Preference Handling (pp. 78-84).
[3] Fang, Hongqing, and Chen Hu. "Recognizing human
activity in smart home using deep learning algorithm."
In Control Conference (CCC), 2014 33rd Chinese, pp.
4716-4720. IEEE, 2014.
[4] S. Szewcyzk, K. Dwan, B. Minor, B. Swedlove, and D.
Cook, Annotating smart environment sensor data for
activity learning. Technology and Health Care, special
issue on Smart Environments: Technology to support
health care, 2009.
[5] Arun Cyril Jose and Reza Malekian, “Smart Home
Automation Security: A Literature Review”, Smart
Computing Review, vol. 5, no. 4, August 2015

IRJET- Smart Home Automation using IoT and Deep Learning

More Related Content

What's hot (20)

Similar to IRJET- Smart Home Automation using IoT and Deep Learning (20)

More from IRJET Journal (20)

Recently uploaded (20)

IRJET- Smart Home Automation using IoT and Deep Learning