Tablet tools and micro tasks
- 3. resilient procedure use study 2014 video: https://vimeo.com/98273509
- 4. products
- 6. EOP flowchart Process overview display Micro-task tablet tool Eye tracking simulator plug-in
- 11. operation training validation research
- 12. operation training validation research EOP tool ✔ ✔ ✔ ✔ Overview display ✔ ✔ ✔ Micro-task tablet app ✔ ✔ ✔ Eye tracking plugin ✔ ✔ ✔
- 13. micro tasks s w i p e t o s t a r t
- 15. problem
- 16. How to test performance benefits of new interface solutions
- 17. How to increase confidence in HRA results and safety cases
- 18. solution
- 19. benefits
- 20. Generate quantitative results about performance benefits (time and reliability) of digital/innovative CR interfaces Highly efficient, objective method Compare to existing reference data Qualitative insights via eye tracking Data directly supports HRA Highly customisable and precise Generate exactly the data needed for the HRA
- 21. big picture
- 22. Micro task tablet app Standardized question sets Database training hra interface design gamification evaluation of training validation a mobile companion for HRA / HF specialists benchmarking within / across organisations
- 23. example
- 24. 2014 micro task study
- 25. innovative vs conventional displays
- 26. innovative vs conventional displays
- 28. mass balance conventional innovative Do the innovative displays lead to faster, more reliable identifications and decisions?
- 29. mass balance conventional innovative Is performance (time and reliability) with innovative displays at least as good as with conventional?
- 30. method
- 31. Large number of questions Related to systems, components, procedures, etc. Varying levels of difficulty Including higher-level decision making Different display conditions E.g. innovative vs. conventional displays We measure response time / accuracy Compare data between conditions If needed, review eye tracking recordings to understand anomalies detection/decision tasks under time pressure
- 32. example
- 33. 4 operators working in the simulator individually (no communication)
- 34. We control which displays are available (eg innovative, conventional)
- 35. Instructions: ”Please answer the questions correctly, but also as quickly as possible. It is very important that you work as fast as you can.”
- 36. Run 1 s w i p e t o s t a r t
- 37. How many condensate pumps are running on turbine 31? 1 2 3 s w i p e t o c o n t i n u e
- 38. “How many condensate pumps are running on turbine 31?” Average identification time Conventional 8 sec Innovative 6 sec
- 39. Is the subcooling margin sufficient? Yes s w i p e t o c o n t i n u e No
- 40. Which steam generators are faulty? SG-1 s w i p e t o c o n t i n u e SG-2 SG-3
- 41. Should safety injection be stopped? Yes s w i p e t o c o n t i n u e No
- 42. What is the narrow-range level in steam generator 1? s w i p e t o c o n t i n u e % 1 2 3 4 5 6 7 8 9 0 37
- 43. up to 200 questions per hour with 4 operators in the simulator working individually, that means we can run up to 800 questions per hour
- 44. results
- 45. data source where does this data come from?
- 46. 6840 data points 3420 response time measures 3420 accuracy measures
- 47. 20 operators
- 48. 5 hours simulator time in total for the whole data collection
- 49. innovative displays are superior
- 50. HSI mean performance times Current effect: F(2, 8,003 )=21,889, p=,00057 Vertical bars denote 0,95 confidence intervals LSD (Innovative) OWD (Conventional) LSD and OWD 15 16 17 18 19 20 21 22 23 24 25 26 Performancetime(sec) Operators were faster with innovative displays than with conventional displays innovative conventional HSI mean performance times Current effect: F(2, 8,003 )=21,889, p=,00057 Vertical bars denote 0,95 confidence intervals 21 22 23 24 25 26 etime(sec) Highly statistically significant difference
- 51. Slightly more accurate with innovative displays than conventional displays Percentage of correct answers innovative conventional innovative
- 53. conclusion
- 54. As reliable as conventional displays Faster to read innovative displays in the 2014 study are
- 55. what next?
- 57. Compare analog CR to HAMMLAB
- 58. Obtain reference data about analog CR
- 59. Interested parties / collaborators AREVA Germany Idaho National Labs Virginia Tech University GSE Systems U.S. NRC
- 60. features
- 61. MT app trigger simulator events Simulator events trigger MT app Integration with WL methods Database prototype
Editor's Notes
- #50: Obviously this is not a general statement, but specific to the findings in this study
- #51: There were 3 conditions: - Conventional displays only. Only the 30” operators screens were turned on (data in the middle) Innovative symbols only. Only the large screen was turned on (data on the left) Both. All screens were turned on, operators could choose which screens they use (data on the right)