Vision-based Fall Detection System - How it Works.pdf
0 likes95 views
The document outlines an AI-driven posture analysis and fall detection system designed for elderly individuals, detailing its implementation and methodology. It features stationary camera setups for monitoring various activities and static postures, employing techniques like frame differencing and pose estimation to classify states such as standing, sitting, or lying. Results from evaluations indicate high accuracy in detecting falls and recognizing various postures, while acknowledging some limitations in heavily occluded environments.
Vision-based Fall Detection System - How it Works.pdf
- 1. AI Driven Posture Analysis Fall Detection System for the Elderly By: Patrick Ogbuitepu Supervised By: Adrian Clark MSc. Artificial Intelligence and Its Applications CSEE Department University of Essex December 2024 How it Works
- 2. Contents Justification 1. Overview of the Solution 2. Implementation Approach 3. Demonstration 4. Static Posture Classification 5. Observations 6. Fall Detection 7. Results 8.
- 3. Justification Source: https://ageing-better.org.uk/our-ageing-population-state-ageing-2023-4 Source: The Centre for Ageing Better
- 6. Overview of the Solution
- 7. Activity Recognition Computer Vision Sensor Based Wearables Environment Monitors Measured Values: Heart Rate Acceleration Angular Velocity Magnetic Fields Measured Values: Sound Motion Pressure Monochrome 3D (RGB-depth) Infrared Thermal 2D (RGB) Frame Sequencing Background Subtraction Data Capture Methods Activity Recognition Methods Pose Detection Optical Flow Histogram of Oriented Gradients Tech Utilised
- 8. Motion Detection (frame diff + bg subtraction) Human Pose Estimation Static Posture Classification Activity Recognition Fall Detection 2D RGB Video Feed Fall No Fall Static Posture Classification Stand, Sit, Lie, or Absent and Bounding Box Aspect Ratio Fall Detection AI-Driven Posture Analysis Fall Detection System for the Elderly
- 9. Design Constraints Stationary High Resolution Stationary 2D RGB Camera 1. Camera must have full view of the person from head to toe 2. Designed for indoor purpose 3. Only supports single individual living alone 4. Room must be well lit 5.
- 10. Implementation Approach Proof of Concept Static Posture Classification Posture Classification for Video Fall Detection Image Processing with OpenCV for Motion Detection Pose Estimation with MediaPipe Analysis of Static Poses Data Collection and Labeling (113 Images and 9 videos) Implement Prediction Logic on Static Images Improve Accuracy of Motion Detection Statistical Analysis of Training Data Performance Evaluation on Training Videos Recognize State Transition Design finite state machine model to detect falls Performance Evaluation on the Validation Videos
- 11. Position in Frame 1 Position in Frame 2 1 2 3 4 5 Value = 0 Value = 255 1 2 3 4 5 Value = 255 Value = 0 1 2 3 4 5 1 2 3 4 5 Frame 2 Frame 1 Frame Difference Value = -255 Value = 255 Value = 0 1 2 3 4 5 Bounding Box Region of Detected Motion Result of Frame Differencing Minus Equals Motion Detection
- 12. Frame Difference After Denoising After Background Subtraction Denoising Frame Differencing Snapping Bounding Box to a Fixed Grid
- 13. Overlap and Retained Previous Region of Interest Before: This frame was skipped After: The frame was recognized despite lack of movement
- 14. MediaPipe Detected Pose on the Chair Before: Entire frame was fed into MediaPipe After: Only the region of interest was fed into MediaPipe
- 15. Creation of My Dataset in 5 Different Venues static posture training 113 Images static posture validation fall detection training 3 Videos fall detection validation 6 Videos
- 16. Categories of Static Posture& Inspiration Lying Hip to Shoulder is not Upright Standing Hip to Shoulder or Hip to ankle is in a vertical position Sitting Hip to Shoulder is Upright & Not Standing
- 17. Key Joints of Focus Key Joints of Focus
- 18. THIGH BONE SHIN BONE BACK BONE 12 11 23 24 26 25 27 28 Analysis of Static Poses& Upright Test X X X X Upright Test: P1 is far greater than P2 P1 P2 P1 P2 Upright Not upright Important criteria to distinguish sitting and lying Also, increases the confidence of predicting standing
- 19. Standing vs Non-Standing Classification Rules Standing: % Threshold ≤ 16% 1. Squatting: 16% < % Threshold ≤ 38% 2. Percentage Difference= ∣(2D Thigh Bone+2D Shin Bone)−(Y-axis Distance from Hip to Ankle)∣ _________________________________________________________ 2D Thigh Bone+2D Shin Bone
- 20. Sitting vs Non-Sitting ( Y position of knee ≥ Y position of hips OR Y position of knee < Y position of hips & Y distance btw hip and ankle ≤ Shin bone ) & Difference in Z of Hip and Z of Knee > 0.5 thigh length in 3D & Upright Test
- 21. YES NO Predict Stand, Sit, Lie Are Legs in Standing Pose? Stand Is Upright? YES NO Both or either Lying? Lie YES NO Both Standing? Both Squatting? Sitting, Lying and Low % Upright? Standing and Squatting? Both Sitting? Max % Standing is high and Min % Sit is Low? Either Standing? Either Squatting? Both Lying? Either Sitting? Stand Stand Lie Sit Stand Stand START START PREDICTION Uncertain and Certain? Standing? Squatting? Sitting? Both Uncertain? Stand Stand Sit Lie Stand Stand Sit Standing? Squatting? Sitting? Stand Stand Sit None Prediction Logic full list of feature vectors include is_upright 1. percent_upright 2. stand_left 3. stand_right 4. percent_stand_left 5. percent_stand_right 6. sit_left 7. sit_right 8. percent_sit_left 9. percent_sit_right 10. lie_left 11. lie_right 12.
- 22. Aspect Ratio to Predict Static Pose
- 23. Aspect Ratio to Predict Static Pose KMEANS Clustering No of Clusters: 9 Histogram Plot of Aspect Ratio No of Bins: 30 Stand Sit Lie
- 24. Evaluation Metric Bounding Box Aspect Ratio First iteration Second Iteration % Improvement Accuracy 84 73 94 22.34 Precision 87.18 63.38 93.75 32.39 Recall 65.69 63.34 96.77 34.55 F1 Score 70.55 63.03 94.85 33.55 Number of Prediction Rules 10 32 Training Results of Static Pose Classification
- 25. Activity Recognition Analysis of Manual Labels Analysis of Manual Labels with Focus on 4 States Only
- 26. Observations on Static Pose Classification Misclassified as sit Misclassified as sit After incorporating 3D coordinates it was correctly classified Bone lengths are not always to scale / proportional in lying position, consider incorporating anglesor using head/feet orientation in future Updated standing logic to account for both 2d and 3d coordinates, while giving more importance to 2d coordinates
- 27. Temporal Smoothing To compensate for occasionally wrong pose estimation
- 28. Before and After Fine-tuning Before: Mis-classified as sit After: Classified as stand
- 29. Video File hr_fall_detection_1 hr_fall_detection_2 hr_fall_detection_3 Main Characteristics Close-up shot High level of Occlusion Wide Angle Modal Class 53.85% 62.5% 41.07% Pre Pre- Smooth Post Pre Pre- Smooth Post Pre Pre- Smooth Post Accuracy 95.04 83.02 85.63 75.18 88.83 90.00 78.51 95.80 95.95 Precision 87.11 88.01 89.67 84.5 85.32 85.31 89.85 96.12 96.02 Recall 81.76 88.86 91.29 76.26 75.35 73.84 86.46 91.87 93.57 F1 Score 81.88 87.31 89.45 75.67 77.26 75.92 85.25 93.19 94.27 Test Results of Static Pose Classification on Unseen Data
- 30. Falling Sit/Stand 100 audio files Stand Sit Lie Absent Lying Recovery Recovery States Transitions Finite State Machine Model State transition must occur Stand-Lie (State Transition) Sit-Lie (Rate of Change of Bounding Box Aspect Ratio) Delay to see if the subject remains in the new state for a minimum period
- 32. Train Video File hr_fall_detection_1 hr_fall_detection_2 hr_fall_detection_3 Main Characteri stics Close-up shot High level of Occlusion Wide Angle Modal Class 50.00% 57.14% 57.14% Pre Post Pre Post Pre Post Accuracy 91.82 100 97.94 85.71 98.61 85.71 Precision 95.38 100 98.94 90.00 99.30 90.00 Recall 79.11 100 77.90 83.33 74.69 83.33 F1 Score 84.38 100 85.28 84.44 82.70 84.44 Fall Detection Training Results Location of video recording: CSEE Labs Other Approaches Considered: 1. Reinforcement Learning with Q-tables 2. Long Short-Term Memory (LSTM) Networks 3. Gated Recurrent Unit (GRU) Networks
- 33. Fall Detection Training Results Plot
- 34. Train Video File 4 5 6 8 9 10 Main Characteristic s Close-up shot Close-up shot + High level of Occlusion Wide Angle Modal Class 57.14% 55.55% 60.00% 53.85% 60.00% 54.55% Length of Video (seconds) 207 146 83 435 67 243 Accuracy 100 66.67 60.00 84.61 100 81.81 Precision 100 81.25 30.00 88.89 100 87.5 Recall 100 62.50 50.00 83.33 100 80.00 F1 Score 100 58.46 37.50 83.75 100 80.35 Fall Detection Training Results Plot MediaPipe was unable to detect pose in heavily occluded videos especially when the head was hidden Image from Video 6, dimly lit and occluded
- 35. Fall Detection Training Results Plot
- 36. Fall Detection Training Results Plot
- 37. End