Infant Care Assistant with Emotion Detection-Using Machine Learning, Image Processing & IOT Sensor Network

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 07 | July 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 461
Infant Care Assistant with Emotion Detection-Using Machine Learning,
Image Processing & IOT Sensor Network
Prerana Rout1, Lucky Upadhyay2, Sai Abhishek Babu3, Faloudeep Gupta4, M Sindhu Sree5
12345Dept. of ECE, Dayananda Sagar College of Engineering, Bangalore, Karnataka, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - With evolving technology and busy working
culture, there is a need of smart device for working parents to
help them in monitoring and assisting their child. Infant Care
Assistant is such a smart device consisting of IOT Sensor
Network, a microcontroller and Raspberry pi to collect
information on the current state of the child and its
surroundings and soothe it accordingly with automated
techniques. Addition to it, an emotion recognitionmodelusing
machine learning has also been included to detect the face of
the child and predict its emotion. Moreover, an app called
Blynk has been used for graphical user interface. this model
could help reducing workload of busy working parents in
taking utmost care of their child.
Key Words: Automation, Sensors, Machine Learning,
IOT, Infant Care
1. INTRODUCTION
In the growing era of technology where everything around
us is getting advanced and intelligent day by day by the
means of artificial intelligence, internetofthingsandvarious
smart devices such as smartphones, smart TVs, smart
watches, smart home appliances and many other devices,
then why not a smart assistant??
In this busy world where now a days both the parents are
working and have little infants or first-timeparentswhojust
entered parenthood and don’t have enough experience are
facing difficulties in providing sufficient time to the infants
provided infants need proper attention and care maximum
of the time.so in order to cope with this a solutionora device
is need of the time.
A smart assistant is needed to provide proper assistance to
the infants in the absence of parents as in monitors the
infants all the time, acquire information related to them,
send notifications if any attention is required and perform
real time interactions between the parents and their child.
2. LITERATURE SURVEY
A Method for Face Segmentation, Facial Feature Extraction
and Tracking (Samir K. Bandyopadhyay) [1] presents the
comparison betweenthemethodologiesusedforhumanface
segmentation from face images based on textural analysis
and KNN classifier. In Automatic E-Baby CradleSwingBased
on Baby Cry (Misha Goyal and Dilip Kumar)[2]theycame up
with an initial low-cost E-Baby Cradle that would rock on its
own when the baby cries where the speed of the cradle can
be regulated. The system consists of an alarm which specify
wet mattress of the baby and its loud cry. Baby cry detection
in domestic environment using deep learning (Yizhar
Lavner, Rami Cohen, Dima Ruinskiy, Hans Ijzerman) [3]
includes the use of two machine-learning algorithms for
automatic detection of baby cry one is logistic regression
classifier & the other is CNN classifier. In Image Processing
Techniques to Recognize Facial Emotions (A. Mercy Rani, R.
Durgadevi) [4] they proposed an emotion recognition
system which included face detection and morphological
processing using viola jones algorithm.
3. BLOCK DIAGRAM AND WORKING PRINCIPLE
There are 3 key features of the infant care Assistant i.e., Data
acquiring, Infant soothing andemotion recognition. Thedata
acquiring unit consists of IOT Sensor network comprising of
various sensors and a node micro controllerunitESP8266to
acquire data related to the child and its surroundings. Here
ESP8266 mcu is an open-source software and hardware
development environment built on low-cost chip, designed
and manufactured by espressif containing its own CPU, ram
and wi-fi. This mcu is integrated with various sensors such
as a dht11 sensor which is used to detect the humidity and
temperature of the infant surroundings, a noise sensor used
for detecting infant cry noise, a moisturesensorfordetecting
the presence of wetness in the child’s bed. all the readings of
these sensors are then reflected on a freely available
android/iOS app called blynk using which parents can
monitor their child.
Infant soothing unit comprises of a baby cradle made of
metal which would rock by the help of a servo motor and 2
channels relay, when the noise sensor is activated. if the
humidity crosses a certain threshold set by the user, the fan
would automatically get on. If the moisture sensor senses
moisture, then it would reflect on the blynk app saying
moisture detected. hence these automated techniques
ensure the calming of a troubled child.
The emotion recognition feature includes a raspberry pi
module with an attached pi camera.Amachinelearningcode
has been developed based on python which on running
would trigger the camera to get on and detect the real-time
image of the child and predict the emotion whether it is
happy or sad or angry or yawning etc. on the window of the
terminal.

Fig -1: Block diagram of the entire setup.
All these 3 units or features are integrated together as a
single set up and power supply is provided to them from a
single source and connected to the same wi-fi address.
4. WORKING OF EMOTION RECOGNITION
Fig -2: Algorithm of emotion recognition
The flowchart states that when we start or run the code, the
camera is triggered to get on. If the camera gets on it would
perform further proceedings, if not the algorithm ends. Ifthe
camera gets on, it would capture and read the live frame or
image.it will then convert the frameorimageintograyimage
internally and load the haar cascade classifier algorithm
which is already ready in the code. It is a ML based approach
where a cascade function is trained from a lot of +ve and –ve
images initially. Later features are extracted from it just like
convolutional kernel where a single value is obtained by
subtracting sum of pixels of white rectangle from black one.
All possible sizes and kernels are used to calculate features
which is around 160000 out of which only relevant features
are extracted and compared to the data set and emotion is
predicted. Although, there is a scope of misprediction and
less accuracy. The classifier scans the frame for facial
features in order to detect face. Once detected it woulddraw
a rectangle around the face and if not the steps from2ndone
starts again. If it is done successfully then it returns the
detected face with the emotion recognized on the terminal
window.
5. ARCHITECTURE
The architecture can be divided into 2 parts i.e., hardware
and software.
In the hardware section, the following tools were used:
 Power Supply(12V)
 ESP-32 microcontroller
 DHT11 Sensor
 Moisture Sensor
 Noise Sensor
 Motor and relays
 Fan and Cradle
 Connector
 USB cables & connecting wires
 Raspberry Pi
 Pi Camera
In the Software section, the following tools were used:
 Arduino IDE
 Blynk app
 Library: Blynk, wifi.h
 Board- ESP 8266
 Python libraries-Pandas, CV2, NumPy etc.

6. RESULTS AND DISCUSSIONS
The 3 features implemented in Infant Care Assistant Namely
Data acquisition, infant soothing and emotion recognitionof
the child has been tested. The prototype was able to acquire
information of the child and its surroundingsand wasableto
soothe it with the designated automated techniques. the
emotion recognition unit was successfully able to detect the
child face and predict the emotion out of it. Hence the entire
set up proved to be very helpful for the parents in taking
care of their child in their absence or during situations like
pandemic which required minimal contact.
Fig -3: Cradle developed with attached fan and motor.
Fig -4: IOT Sensor network with mcu, raspberry pi and
attached pi camera.
Fig -5: result displayed on blynk app
Fig -6: real time face detection and emotion prediction
(Prediction of yawning emotion)
7. CONCLUSIONS
The idea revolves around automating all the process which
can result in saving time and effort of the parents and
ensuring proper monitoring and soothing of the child. the
implementation will result in time saving and cost
effective.in future this project can be further enhanced by
using advanced hardware and software tools such as
Bluetooth, robotics components, other sensors etc.
The enhanced prototype can be used in hospitals for
paralyzed patients or neo natal intensive care units etc.
Thus, it has a very larger scope in future and way more use
cases can be introduced.
REFERENCES
[1] Samir K. Bandyopadhyay proposed, "A Method for Face
Segmentation, Facial Feature Extraction and Tracking"
IJCSET | April 2011 | vol.no:1, issue 3,137-139 [8]
Gonzales and Woods, ―Digital Image Processing‖,
Pearson Education, India, Third Edition.
[2] Misha Goyal and Dilip Kumar, “AutomaticE-BabyCradle
Swing based on Baby Cry”, International Journal of
Computer Applications 71(21):39-43, June 2013.
[3] Y. Lavner, R. Cohen, D. Ruinskiy and H. Ijzerman, "Baby
cry detection in domestic environment using deep
learning," 2016 IEEE International Conference on the
Science of Electrical Engineering(ICSEE),Eilat,2016,pp.
1-5.
[4] A. Mercy Rani, R. Durgadevi, “Image Processing
Techniques to RecognizeFacial Emotions”,International
Journal of Engineering and Advanced Technology
(IJEAT) ISSN: 2249 – 8958, Volume-6 Issue-6, August
2017.

Infant Care Assistant with Emotion Detection-Using Machine Learning, Image Processing & IOT Sensor Network

More Related Content

Similar to Infant Care Assistant with Emotion Detection-Using Machine Learning, Image Processing & IOT Sensor Network (20)

More from IRJET Journal (20)

Recently uploaded (20)

Infant Care Assistant with Emotion Detection-Using Machine Learning, Image Processing & IOT Sensor Network