How to plan and execute an ecologically valid NeuroIS study
2 likes312 views
This document provides an overview of how to plan and execute an ecologically valid NeuroIS study. It discusses key elements to consider such as tasks and scenarios, measures, interfaces, protocols, instruments, recruiting subjects, and compensation. Examples of case studies and insights from NeuroIS research are also presented. The document emphasizes developing a complete test plan that addresses objectives, teams, interfaces, subjects, and other important details.
How to plan and execute an ecologically valid NeuroIS study
- 1. PIERRE-MAJORIQUE LÉGER, Ph.D. NSERC-Prompt Industrial Research Chair in User Experience Co-director Tech3Lab, Director ERPsim Lab June 4 2019, Vienna, Austria. How to plan and execute an ecologically valid NeuroIS study
- 2. © Copyright Léger (2019) Pierre-Majorique Léger, Ph.D. Lab Serious games to learn enterprise systems and business analytics https://www.researchgate.net/profile/Pierre-Majorique_Leger NSERC-Prompt Industrial Research Chair in User Experience Director ERPsim Lab Co-direct0r Tech3Lab pml@hec.ca GRADUATE STUDIES POST-DOCTORATE LABORATORIES FINANCIAL SUPPORT GUEST PROFESSOR
- 3. Powered by: © Copyright Léger et Fredette (2019)
- 4. Marc Fredette, Ph.D. Data Sciences Sylvain Sénécal, Ph.D. Marketing Pierre-Majorique Léger, Ph.D. Information technologies
- 5. 2018-2019 50Scientific articles and conference proceedings 8Postdoc researchers 5PhD Students 21Master students 10summer interns 420human subjects 20research projects
- 6. Industrial research partners from a wide range of industries INSURANCE ONLINE GROCERY MEDIA RAILWAYS BANKING AERONAUTICS LOGISTICS FINANCE r partenaire de PLUSIEURS BOURSES DE RECHERCHE À TOUS LES NIVEAUX : chaire_ux.hec.ca INSCRIVEZ-VOUS COMME PARTICIPANTS À NOS ÉTUDES : panel.hec.ca Organismes subventionnaires : partenaire de
- 7. est fier partenaire de SEVERAL RESEARCH GRANTS FOR ALL ACADEMICS LEVELS: chaire_ux.hec.ca SUBSCRIBE NOW AS PARTICIPANTS IN OUR RESEARCH PROJECTS: panel.hec.ca Proud Partner of Powered by: Funding Organisations:
- 8. Tech3Lab: A Real UX Playground 183 3000PROJECTS RESEARCH PARTICIPANTS 6YEARS
- 9. © Copyright Léger (2019) A Bvs How to plan a NeuroIS study SCHEDULING TASKS AND SCENARIOS MESURES INTERFACES PROTOCOL INSTRUMENTS RECRUITING SUBJECTS COMPENSATION
- 10. © Copyright Léger (2019) How dynamic is the interface? 1 2 3 4- +IMAGE WEB STATIQUE MOBILE STATIQUE DYNAMIQUE
- 11. © Copyright Léger (2019) Detection of events AUTOMATIC MARKERS MANUAL MARKERS WEB DYNAMIC STIMULI = 30 FPS $ $ $ $
- 12. © Copyright Léger (2019) Ecologically valid NeuroIS study TASK SUCCESS OR FAILURE? SCENARIO SCREEN 1 EVENT EVENT EVENT EVENT SCREEN 2 SCREEN 3 SCREEN 4
- 13. © Copyright Léger (2019) Complete test plan TASK 1 TASK 2 ... INTERVIEW SURVEY DEBRIEFING INSTRUMENTS PROTOCOL COMPENSATION
- 14. © Copyright Léger (2019) RANDO- MISATION INSTRUMENTS Within subject research design PROTOCOL TASK A TASK B
- 15. © Copyright Léger (2019) Between subject research design INSTRUMENTS PROTOCOL
- 16. © Copyright Léger (2019) What is the research question? What are the hypothesis? What is the interface? On which platform? How dynamic is the interface? What are the event markers? What are the characteristics of the research subjects? How many subjects are needed for the experiment? What are the time constraints? Who are the researchers involved and what are the responsibilities? OBJECTIVES TEAM INTERFACE SUBJECTS Questions to ask to define the project
- 17. © Copyright Léger (2019) What are the tasks? What is the scenario? What are the main subtasks? What information is required to perform the task? How will the subjects be recruited? What will be the compensation? When and where will the experiment take place? Which version of the interface will be used ? Which instruments will be used / are available ? Questions to ask to define the test plan A Bvs A Bvs A Bvs A Bvs A Bvs A Bvs Bvs SCENARIOS AND TASKS MESURES SCHEDULE INTERFACESPROTOCOL INSTRUMENTSRECRUTEMENT COMPENSATION
- 18. © Copyright Léger (2019) Case studies
- 19. © Copyright Léger (2019) Which of these two ads creates the most optimal emotional reaction?
- 20. © Copyright Léger (2019) Presentation study EMOTIONAL AROUSAL (Biopac) VISUAL ATTENTION (Tobii) EMOTIONAL VALENCE (Facereader) UX HEATMAP + ... 5 seconds 5 seconds 5 seconds Powered by: or or or Happy Sad Calm Excited
- 21. © Copyright Léger (2019) Ad #1 Ad #2 Ad Emotional Response: Results and Insights INSIGHT #1 For more than half the participants, Ad #1 generated only positive emotion. As the UX heatmaps show, the face and faded lights in the background seem to have contributed to the positive emotional response for this Ad. INSIGHT #2 Even when participants preferred this Ad, it generated mostly negative emotions. The backward-facing person in this Ad contributes to the negative emotion felt by participants. Positive valence (Facereader) Negative valence (Facereader) UX Heatmap powered by CubeHX
- 22. © Copyright Léger (2019) Ecommerce experience (1/2) EMOTIONAL AROUSAL (Biopac) VISUAL ATTENTION (Tobii) EMOTIONAL VALENCE (Facereader) EMOTIONAL AND COGNITIVE CIRCUMPLEX Powered by: RANDOMIZED Checkout AROUSAL(STANDARDIZEDAROUSAL(STANDARDIZED PUPIL SIZE VALENCE (FACEREADER) Calm Excited Easy Difficult Happy Sad
- 23. © Copyright Léger (2019) Emotional Circumplex METHODOLOGICAL NOTES EMOTIONAL VALENCE EMOTIONAL INTENSITY Sad Happy Calm Excited UX No Man Land Excitement SerenityTolereble HOW TO READ THIS CIRCUMPLEX On the vertical axis, we find the emotional intensity experienced by consumers; the higher they are, the more emotionally intense is the experience. Horizon- tally, emotional valence is represented; on the right, experience generates pleasure, while on the left, the experience generates displeasure. Finally, the size of the circle represents the mental workload; a large circle represents a significant mental effort of con- sumers.
- 24. © Copyright Léger (2019) WALMART, THE LAGGER. Low prices is not sufficient for a bet- ter experience. Walmart is a generalist and must find a way to compete against websites that can afford to carve a unique experience for users. A microsite can be a way to compete on the emo- tional front. Walmart has successfully used this strategy in the past for niche products. INSIGHT #1 Experience on Walmart is as emotionally engaging on these two pages. EBGames and BestBuy are able to carry over to the search page the excitement generated on the homepage. EXAMPLE 2 Homepage Product Search VALENCE (FACEREADER)VALENCE (FACEREADER) AROUSAL(STANDARDIZED VALENCE (FACEREADER)VALENCE (FACEREADER) AROUSAL(STANDARDIZED Product pages: Insights & Recommendations (1/2) AROUSAL(STANDARDIZEDEDA)
- 25. © Copyright Léger (2019) Product Specs Checkout AROUSAL(STANDARDIZED AROUSAL(STANDARDIZED AROUSAL(STANDARDIZED AROUSAL(STANDARDIZED PUPIL SIZE PUPIL SIZE THE ULTIMATE STEP IN THE CONVERSION FUNNEL: THE PRODUCT PAGE The product page is the last page before consumers make a purchase. The experience on EBGames appears to be the gold standard. It performs significantly better than Walmart and BestBuy. By having the best product page the EBGames website, is making the most of their efforts to convert consumers to paying customers on their website. Product pages: Insights & Recommendations (2/2) INSIGHT #2 EBGames product page generates emotions that are positive and intense, while maintaining a low mental workload. VALENCE (FACEREADER) VALENCE (FACEREADER)VALENCE (FACEREADER) EXAMPLE 2
- 26. © Copyright Léger (2019) Journey maps METHODOLOGICAL NOTES 1 2 3 ... + - ACTIVATION: (StandardizedEDA-Biopac) Time STEPS Friction Point EMOTIONAL VALENCE (Facereader) + - HOW TO READ THIS JOURNEY MAP Colors of tasks are presented on the right, the red squares representing moments of physiological pain points along the journey. On the vertical axis, we find the emotional intensity experienced by consumers; the higher they are, the more emotionally intense is the experience. Horizontally, the time is presented in seconds. The size of the circle represents the emotional valence; a large circle represents negative emotions. In other words, pain points are high and large dots on the journey map!
- 27. © Copyright Léger (2019) Ecommerce experience (2/2) EMOTIONAL AROUSAL (Biopac) VISUAL ATTENTION (Tobii) EMOTIONAL VALENCE (Facereader) PAIN POINT IDENTIFICATIONS Happy Sad Powered by: Calm Excited Easy Difficult VALENCE (FACEREADER) INSIGHT #1 Even after filtering for the price, consumers cannot find the product he is looking for, which creates frustration. INSIGHT #2 The total order amount does not adjust when an item is removed from the cart, creating confusion.