Session 48 Johan Karlsson
1 like359 views
The document discusses a research project that aims to improve driver drowsiness and distraction detection through sensor fusion. It describes collecting data from drivers on test routes to train and optimize a classifier. Multiple indicators of drowsiness are measured, like blink duration and lane keeping variability. A support vector machine is used to fuse the sensor data and find the optimal detection threshold. The goal is to enhance overall detection performance by combining data from different sensors and reducing false alerts. Evaluation involves calculating the sensitivity and specificity of detection compared to a ground truth scale.
Session 48 Johan Karlsson
- 1. Driver Drowsiness and Distraction Detection by Sensor Fusion D4SF Johan Karlsson, Autoliv Transportforum 2011 Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 1 Copyright Autoliv Inc., All Rights Reserved
- 2. Overview Background Goals Drowsiness detection (Distraction detection) Method Data collection Training/optimization of classifier Sensor fusion Results Reference – ground truth Improvement by (f)using parallel detectors Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 2 Copyright Autoliv Inc., All Rights Reserved
- 3. Driver Drowsiness detection Drowsy driving is a road safety problem - drowsiness contributing in 10-30% of accidents (Anund & Patten 2010) What can be done? Commercial fleet traffic Fatigue Risk Management Work time regulation Detection and warning Privately owned vehicles Detection and warning Detection? Detection systems offered as option from several OEMs So far, performance is far from ideal... Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 3 Copyright Autoliv Inc., All Rights Reserved
- 4. Target and Goals Sensitivity Different indicators exist Specificity Availability - ’Physiology’ A Indicator measures+ blink duration etc. - – + - Driving performance measures - lane keeping measures - EnvironmentBmeasures - time of day, traffic, road type Indicator – (previous sleep possible in commercial fleet vehicles??) + + Indicator C + + – Various indicators have different strengths and weaknesses Improve performance by fusing data from multiple indicators+ Fusion ++ ++ + The fusion algorithm shall show an improvement in: - Overall performance - Reduced number of faulty detections - Increased number of correct detections Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 4 Copyright Autoliv Inc., All Rights Reserved
- 5. Data collection On-road tests were conducted with Data collection governmental approval (N2007/5326/TR) and ethical approval by Regional ethics Relevant vehicle data approval board (EPN 142-07 T34-09). Speed, lane position, SW angle, pedals etc. Video based gaze direction, eyelid opening, head pos KSS value every 5 minute EEG, EOG and EMG Video recordings (road scenery and cabin) In total: 43 drivers have completed 3 drives each Procedure: Each driver drove three times during one day (day, evening and night). Trip duration 80-90 minutes Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 5 Copyright Autoliv Inc., All Rights Reserved
- 6. Test Route Road RV34 Mostly 9 m width Driving lane width 3,75 m Speed limit - mostly 90 km/h Numbers on map are Yearly day traffic volume in January 2002 We know of only a few similar studies performed on public roads 90 minute driving, approx 115 km distance Rested safety driver – dual command Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 6 Copyright Autoliv Inc., All Rights Reserved
- 7. Ground Truth – KSS KSS Description in Swedish Verbal description 1 extremt pigg extremely alert 2 mycket pigg very alert 3 pigg alert 4 ganska pigg rather alert 5 varken pigg eller sömnig neither alert nor sleepy 6 första tecknen på sömnighet some signs of sleepiness 7 sömnig, ej ansträngande vara vaken sleepy, but no effort to keep alert 8 sömnig, viss ansträngning vara vaken sleepy, some effort to keep alert 9 mycket sömnig, ansträngande vara vaken, very sleepy, great effort to keep alert, kämpar mot sömnen fighting sleep + Simple to collect + Simple to understand – immediately ready for analysis - Training needed for participants - Some offset for inexperienced participants? Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 7 Copyright Autoliv Inc., All Rights Reserved
- 8. Example indicators of driver sleepiness Closing Opening Open 200 ms Closed Blink duration (AS): Amplitude 400 ms Closed Mean blink duration Short Blink Long Blink GVI (Sandberg 2008) Lane keeping variability (Lane): G= 1 N ∑ w(zi ) | zi |k N i=1 Variability in Steering wheel position or zi = xi − (δ x + (1 − δ ) p) Lane Position. e.g. using Generic cL cR Variability Indicator (Sandberg 2008) . w( z) = −α L ( z −β L ) + −α R ( z −β R ) 1+ e 1+ e Time-of-day (TPM): Expected drowsiness with regard to time of day (circadian rythm) * Each indicators has several parameters that needs to be tuned for optimal performance Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 8 Copyright Autoliv Inc., All Rights Reserved
- 9. Video examples Video examples Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 9 Copyright Autoliv Inc., All Rights Reserved
- 10. Sensor fusion SVM (Support Vector Machine): Machine learning method using data from field tests to calculate “best fit” function between indicator values and ground truth (KSS rating scale) Indicator parameters optimized simultaneously with training of SVM Data sets for SVM training and validation are from separate drivers. Thus, validation is done on truly “never-before-seen” data. Drowsy data Goal: Find the maximum margin hyperplane Indicator B Alert data Indicator A Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 10 Copyright Autoliv Inc., All Rights Reserved
- 11. Evaluation Method Assuming a binary classification, A alert or = sensitivitydrowsy A+C Ground truth Performance is the mean value of D Non- specificity = sensitivity and specificity Drowsy Performance + D Drowsy B is related to the Algorithm output A B sensitivity + specificity proportion of the time where the Detect performancis = algorithm e correct (hit) (false hit) 2 Non- C D Detect (miss) (correct reject) KSS = ground truth Ground truth cutoff Sum A+C B+D Binary Algo output KSS Time Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 13 Copyright Autoliv Inc., All Rights Reserved
- 12. Example of results from sensor fusion Model Fitness Sens Spec Blink 0.66 (0.64) 0.36 (0.32) 0.96 (0.95) Blink + Circadian 0.80 (0.83) 0.77 (0.79) 0.83 (0.87) Blink + Lane + Circ. 0.80 (0.78) 0.68 (0.68) 0.92 (0.88) Blink + Steer + Circ. 0.80 (0.85) 0.76 (0.81) 0.84 (0.89) First figure is training data performance second figure is test data performance Decision every 1 minute KSS >= 7 drowsy KSS < 7 alert Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 14 Copyright Autoliv Inc., All Rights Reserved
- 13. Fulfillment of goals The fusion algorithm shall show an improvement in: - Improved performance true - Increased number of correct detections true - Reduced number of faulty detections (?) Clearly improved overall performance – Minor differences between different combinations Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 15 Copyright Autoliv Inc., All Rights Reserved
- 14. Summary Controlled experiment on public roads 43 drivers so far What is ideal performance? Method developed with focus on mathematical performance Most important goal is to have relevant warnings More data is needed: Different road types Different conditions (weather, drive duration etc.) Different driver types (age, cultural differences etc.) Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 16 Copyright Autoliv Inc., All Rights Reserved
- 15. Thank you for you attention! Fordonsstrategisk Forskning och Innovation FFI – D4SF ALR-JKAR/Jan2011/Transportforum - 17 Copyright Autoliv Inc., All Rights Reserved