Using Neural Net Algorithms to Classify Human Activity, with Applications in Providing Assistance to the Elderly

1
Using Neural Net
Algorithms to Classify
Human Activity, with
Applications in
Providing Assistance
to the Elderly
Category: Mechanical Engineering
Project ID: HS-MENG-031-T3

2
Table of Contents
Abstract……………………………………………………………………………………………3
Background Research………………………………………………………………………...…4-6
Engineering Goal and Purpose…………………………………………………………………….7
Materials……………………………………………………………………………………...…8-9
Design Criteria, Constraints, and Test Plan……………………………………………………...10
Engineering Design…………………………………………………………………………..11-13
Prototype……………………………………………………………………………………...14-15
Initial Testing and Evaluation………………………………………………………………..…..16
Interpretation of Results and Design Results……………………………………………….……17
Final Testing and Evaluation………………………………………………………………...…..18
Conclusion………………………………………………………………………………….……19
Acknowledgements…………………………………………………………………………..…..20
Bibliography…………………………………………………………………………………..…21

3
HS-MENG-031-T3
Using Neural Net Algorithms to Classify Human Activity,
with Applications in Providing Assistance to the Elderly
According to U.S. Census data, of the approximately 15.3 million senior citizens facing
disabilities, 2/3 reported difficulty with mobility. Currently, there are no convenient solutions
that provide a portable and affordable option to both classify human activity, and then actively
assist depending on what the circumstance demands. Our project set out to utilize Neural
Network Algorithms to detect a walking or falling pattern, depending on gyroscopic data
collected from sensors along the waist of the individual. The use of a Neural Network allows for
improvement by learning over time, and can adapt itself depending on the individual wearing the
belt. We created a prototype to simulate human motion using a 3D Printed gimbal, and collected
several datasets to train our Neural Network. The Neural Network was able to classify either a
walking, or falling action with over 88% accuracy. Such classification technology has many
uses, we further demonstrated one such use to help elderly people balance. If the Neural Network
detects a falling pattern, it will automatically deploy 4 reaction wheels to begin stabilizing the
individual, by using brushless motors spinning along the axis of rotation to counteract the
direction which the individual is falling. Using PWM signals, the motors dynamically changes
speed to bring the individual back to a stable orientation. Unlike traditional self-balancing
algorithms, the use of a Neural Network to deploy the system makes the reaction wheels only
turn on when abnormal activity is detected, and therefore not be intrusive during normal motion.

4
Background Research
What technology will we use?
Neural Nets:
Our project primarily revolved around the development of an accurate neural net. Neural Nets
are primarily used to emulate the human thought process, and are used for applications in
machine learning.
When developing a neural net, the first step is to "train" the neural net, by giving it known
inputs, and their corresponding outputs. This gives the neural net a baseline for future inputs, and
provides the foundational characteristics used for distinguishing activity in the future.
Python Code:
Python is a widely used high-level programming language for general-purpose programming,
created by Guido van Rossum and first released in 1991. An interpreted language, Python has a
design philosophy that emphasizes code readability (notably using whitespace indentation to
delimit code blocks rather than curly brackets or keywords), and a syntax that allows
programmers to express concepts in fewer lines of code than might be used in languages such
as C++ or Java. It provides constructs that enable clear programming on both small and large
scales.
3D Printer (Flashforge Dreamer):

5
3D printing, also known as additive manufacturing, refers to processes used to create a three-
dimensional object in which layers of material are formed under computer control to create an
object. We used the Flashforge Dreamer printer in specific to print out our materials. (Source:
Google)
MPU6050 Gyroscope:
The MPU6050 Gyro is a gyroscope that tracks in 3 axis, and also an accelerometer that can track
3 axis. The gyroscope records angular velocity in X, Y, and Z, and communicates through an I2C
bus. It is powered by a standard 5V pin on a microcontroller.
Raspberry Pi:
The Raspberry Pi is an affordable, palm-sized computer. It contain multiple ports that allow it to
act like a desktop computer (USB, Ethernet, etc), but also contains pins similar to a
microcontroller, like an Arduino, that allow programmers to also use it for unique applications,
that need to be run with more computing power than a standard Arduino.

7
Engineering Goal and Purpose
Engineering Goal:
The Engineering Goal of this project is to design, and evaluate a neural net that can monitor the
activity, and movements of humans, specifically elderly, in order to accurately classify behavior
and provide assistance when applicable.
Purpose:
According to the US Census Bureau, " Nearly 40 percent of people age 65 and older had at least
one disability, according to a U.S. Census Bureau report that covered the period 2008 to 2012.
Of those 15.7 million people, two-thirds of them say they had difficulty in walking or climbing".
Motor impairments is a widespread problem that impacts millions of our elderly population, and
current consumer-available products provide no active solution to solving such problems. The
objective of our project was to create a portable system that can help to fill this gap, by first
having the ability to classify the actions of humans, and secondly, be able to prove the
applications of such system, by providing active assistance through the reaction-wheel belt.

8
Materials
The Materials used to build and design our prototype were as follows:
• Hardware:
o Raspberry Pi
o Jumper Wires
o Gyroscopes MPU6050
o Screws
o 3D Printer + Filament
• Software:
o Python
o Anaconda Library Package
The additional Materials used to create the Reaction Wheel Belt were as follows:
• A2212 Brushless Motors
• 7 Cell NiMH batteries

10
Design Criteria, Constraints, and Test Plan
Design Criteria:
The Design Criteria for this project will be to create a neural net that can improve within 1%
accuracy per iteration, and be able to classify the motion of the user with greater than a 75%
accuracy.
Constraints:
The constraints that can limit our ability to perform this project are computer processing power
and speed, cost, time allotted to our science fair, and wearability of the product. The neural net
should be able to function on a wearable computer like a Raspberry Pi. Additionally, the cost
cannot be greater than $500, and we cannot spend more than 6 months designing our prototype
because of deadline constraints. Finally, the product should be easy to wear and use by the user,
and should be a reasonable size.
Test Plan:
To test our design, we will first need to establish what is a “correct” versus “incorrect”
classification. We will create 2 datasets, one that simulates a human walking, and the other that
simulates a human falling. We will then classify each respective dataset as walking or falling,
and store it in the neural net, so that it has something it can check against. We will then begin
feeding gyroscopic data points in real time into the neural net, as we move the gimbal around in
either a walking, or falling pattern, and begin to appraise the success rate of the classifications.
We will complete the procedure over 100 trials, and test “per event” the success rate of the
neural net. The neural net will continue to be improved, until we begin achieving higher than 75
correct classifications per 100, which is our goal for the experiment.

11
Engineering Designs
Our Engineering Designs mainly centered on which type of input to use as the basis for training
the neural net. The gyroscope used is equipped with sensors that could determine angular
velocity and acceleration.
The 2 types of inputs tested for the training dataset were position or velocity, as the main values
used to train the neural net. The position design would integrate the velocity values, and use the
change in position to determine if a person was walking/falling. The velocity design would take
the angular velocity at a given time, and make a determination based on the magnitude of the
vector, if the person was walking or falling.
When investigated, the position design turned out to be relatively high in false positives and
inaccuracy. Because we were using position, and alternative actions that the user may have been
performing were included in the determination of walking or falling. For example, sitting in a
chair may have been viewed as falling, because of the change in the Z axis.
The final design that was determined to be the most efficient for this experiment was the velocity
design. The actions of falling is essentially increasing angular velocity from roughly 0 to some
final velocity, it was more precise for determining and classify the actions the user was
performing. Additionally, users could perform other actions, like sitting, without the angular
velocity increasing substantially, and thus the accuracy of classifications was significantly
increased when using this model.

14
Prototype
Procedure:
1. Design and Assemble the Gimbal to simulate the human body/waist. The gimbal should
contain at least 1 gyroscope, that can transmit data about the angular velocity of the gimbal at
any 1 time.
2. Gather a preliminary training dataset for the neural net. For 5 minutes, simulate a walking
pattern on the gimbal, and record the angular velocities, and repeat this step for a falling pattern
on the gimbal.
3. In Python, train the Neural Net using the 2 datasets, by labelling the walking dataset as
"walking" and the falling dataset as "falling".
4. After training, write a program that can take the inputs of the real-time gyroscope values, and
make a determinate about whether the pattern is walking or falling.
5. To improve accuracy, continue to retrain the neural net to ensure improvement overtime.

16
Initial Testing and Evaluation
To test the Neural Net’s accuracy, we tested the accuracy of classification for 100 events. Our
goal was for the neural net to classify greater the 75% of the events correctly, and continues to
learn per each trial that we conducted. This way in the future, the neural net can improve overall
accuracy. We simulated a walking or falling action on the gimbal, and ran the neural net. For live
motor values, at any given second, the neural net either provides a 0 or a 1 response, with the 0
representing walking, and a 1 representing falling. We conducted each action for 10 seconds
each, and recorded how many 1s and 0s were produced by the neural net during that time
duration. We repeated this process 100 times, and recorded the classifications. The results are
shown below.
Additionally within each 10 second trial, we wanted the neural net to record more than 75% of
each of the individual numbers correctly. Within each 10 second interval, 10 numbers were
recorded, and we wanted to see that greater than 75% of those numbers were also the correct
classification. Any trials that were less than 75% accurate, were automatically classified as an
“inaccurate classifications”, and sorted accordingly in the data tables.
Number of Tests Performed: Number of Tests Performed:
50 50
Percentage Expected: Percentage Expected:
75%+ 75%+
Percentage in Testing: Percentage in Testing:
88% 90%

17
Interpretation of Results and Design Results
The results and data collected throughout this project reflected several trends and interpretations
that were interesting to observe and reflect upon. First, the accuracy of the neural net during the
first initial trials were much higher than originally anticipated. Our minimum target we wanted to
reach was 75%, however we were able to attain close to a 90% accuracy for both the walking
and falling trials.
When trying to explain this high accuracy rate, we looked to our previous decisions along the
way that could have potentially played a factor. One such decision was for our inputs to be
converted to angular velocity, as opposed to positional change through integration. This is
because, while an individual is walking, small motions or stumbles were classified as “falling”,
because the person’s position was actively changing. Additionally, if the person was bending or
picking something up, positional changes were also detected and classified as falling, therefore
leading to several errors in the classification, and drastically reducing the percentage of
accurately classified.
When we switched to angular velocity, the new “parameter” that the neural net was looking for,
was very fast movements, as opposed to positional changes. This allowed the person to continue
to do their tasks normally, and only be classified as falling, when the individual is quickly
moving in a certain direction. This improved the accuracy of our classifications overall, and are
the percentages recorded for the results portion of the experimentation.

18
Final Testing and Evaluation
To prove the concept of the Neural Net, and further prove applications of the this technology, we
created a reaction wheel belt system that could actively assist an elderly individual if instability
was detected.
When finalizing the design for implementation with the reaction wheel belt system, we settled on
a lowered motor mount, such that the brushless motors were in line with their respective axis of
rotation. This allowed for maximum leverage when spinning the wheels, and improved the
reaction wheel stability performance.
The reaction wheel belt system was made to deploy only if the Neural Net made the
determination that the individual was falling. If a falling pattern was detected, the reaction wheel
belt system would begin actively spinning brushless motors at high velocities to try to counteract
the direction that the individual was falling. The brushless motors could also dynamically vary
speed to provide stability assistance in the most effective manner possible.
Brushless motors utilize a signal knows as PWM, which stands for Pulse Width Modulation.
Essentially depending on the width of the signal that is transmitted to the motor, the speed of the
brushless motors will be varies. The images below depict the gimbal design optimized for the
reaction wheel belt system, and an image of the PWM signal detected from a Tektronics Scope.
https://youtu.be/cFS9z11R7ZU

19
Conclusion
The overall outcomes of this project included the successful creation of a Neural Net algorithm
that could classify human motion with a high level of accuracy. Additionally, we were
able to further prove the applications of this technology, and show potential in this sector
through our reaction wheel belt demonstration.
Applications of this technology go much farther than just the reaction wheel belt system. It is
possible to classify a wide array of technology, simply by creating more datasets. The
Neural Net will learn to adapt to each individuals walking style. This would make it
possible for all actions that a senior citizens are performing to be classified with a high
level of accuracy, specific to that indidvidual.
Additionally, it is possible that the Neural Net does not even need to run on a microcontroller
hardware like a Raspberry Pi, rather, it can be uploaded to a smart phone, where no
additional hardware would be required; the Neural Net would simply operate off the
gyroscope built into the phone.
Finally, it is possible that the Neural Net can alert emergency services in the event that it detects
a potentially dangers action like a fall. Currently solutions require external hardware be
worn, or that the senior citizen manually dial emergency services.
Follow-up Project:
In the future, some improvements to our project are overall classifying more actions and
increasing the database of actions that we have stored in the neural net. Additionally,
adding more gyroscope sensors to determine orientation in 3D can be helpful in
increasing precision of the classifications.

20
Acknowledgements
We would like to thank our parents for supporting us through the project, despite the difficulties
due to the pandemic. We would also like to thank Mr. Tobler, who oversaw our design
and prototyping, and provided valuable insight into our project. We would finally, like to
thank ACSEF and Ms. Carothers, for continuing the fair this year and working hard to
support local students, despite the pandemic.

21
Bibliography
“Please Enable Cookies.” StackPath, www.vision-systems.com/boards-
software/article/14037858/fundamentals-of-deep-neuralnetworks.
Https://Developers.google.com/Machine-Learning/Crash-Course/Introduction-to-Neural-
Networks/an Atomy.
Https://Openreview.net/Pdf?Id=S1c2cvqee.
Https://Lib.dr.iastate.edu/Cgi/Viewcontent.cgi?Article=1476&Context=Rtd.
“NNDesign.” Hagan.okstate.edu.

Using Neural Net Algorithms to Classify Human Activity, with Applications in Providing Assistance to the Elderly

More Related Content

What's hot (20)

Similar to Using Neural Net Algorithms to Classify Human Activity, with Applications in Providing Assistance to the Elderly (20)

Recently uploaded (20)

Using Neural Net Algorithms to Classify Human Activity, with Applications in Providing Assistance to the Elderly