4. Drop outs in Living Labs
Download as PPTX, PDF0 likes109 views
This webinar will discuss drop-out in living lab field tests. A living lab field test involves testing innovations in real-life contexts with voluntary participants. Key issues include ensuring participation continues throughout the study. Drop-out occurs when participants do not complete all tasks by the deadline. Factors influencing drop-out include problems with the innovation, how tasks are designed, and participants' circumstances. Recommendations to reduce drop-out include clearly communicating expectations and ensuring the test remains flexible and beneficial for participants. Ethical considerations include informed consent and protecting participants' interests.
4. Drop outs in Living Labs
- 1. Living Labbers Webinar #1: Etat de Genève (GE-LAB / Genève Lab) WELCOME to the Living Labbers webinar! The presentation will start shortly, we’re waiting a few minutes for all attendees to arrive. Please, remain muted until the Q&A session at the end. You will be given the opportunity to ask questions later. If you have questions arising during the presentation please enter them in the chat box, we will go through these questions first before opening the floor to others! This webinar is recorded and will be shared online. However, there is no need to be shy - you can ask questions at the end & these will not be included in the recording without your permission!
- 2. Living Labbers WEBINAR 4: DROP-OUT IN LIVING LAB FIELD TEST
- 3. DROP-OUT IN LIVING LAB FIELD TEST BOTNIA LIVING LAB By Abdolrasoul Habibipour
- 4. Presentation Outline ● What is a Living Lab Field Test ○ Potential problems ○ Living Lab Field Test setting ○ Key Components and Key principles ● The challenge of Drop-out in Living Lab Field Test ○ Defining Drop-out ○ Taxonomy of Drop-out in Living Lab field Test ● Recommendations on how to keep participants motivated ● Some ethical considerations
- 5. What is a Living Lab field test? ● Definition of field test ○ To test (a procedure, a product, etc.) in actual situations reflecting intended use. (Merriam-Webster Dictionary, 2018) ○ Experiment, research, or trial conducted under actual use conditions, instead of under controlled conditions in a laboratory. (BusinessDictionary.com, 2018) ● Living Lab field test ○ A user study in which test users interact with a (digital) innovation in their real-life everyday use context while testing and evaluating it
- 6. Key differences between Living Lab field test and traditional tests ● Living Lab field tests are usually conducted in an open environment, in contrast to traditional tests, where the testing is undertaken within a controlled situation. ● A key feature of Living Lab field tests is that users are under no obligation to test the innovation or to follow the structure on how to use it. ● The commercial maturity of the prototyped product, service, or system in Living Lab setting is lower than in traditional tests. ● Living Lab field tests are usually long-term user studies involving relevant stakeholders
- 7. Potential problems of Living Lab field tests ● Ensuring that the setting is representative ● The Hawthorne effect ● Extra work for end-users ● Recruitment and resource allocation challenges ● Potential legal and ethical challenges ● The lack of control over the situation and over the innovation ● The lack of control over who is actually testing the innovation
- 8. Key components of Living Lab field tests ● Innovation ● Approach ● participants ● ICT-Infrastructure ● Governance and management ● Partners
- 9. Key principles of Living Lab field tests ● Societal and value creating ● Open and inclusive ● Explorative ● Responsible and sustainable ● Real life context
- 10. Living Lab field test setting ● Field test environment ○ People’s home ○ Public area (train station, airport, shopping centers...) ○ People’s workplace ● Data collection methods ○ Participant diaries ○ Participants observation ○ Individual and focus group interviews ○ Post test Questionnaire ○ Email communication
- 11. Key questions in a Living Lab Field Test ● First evaluation ● Method selection ● Resource allocation ● User recruitment ● Setting ● Analysis ● Final evaluation
- 12. Field test process in Living Lab setting ● Develop a test story line for the test process: ○ Cleary define the aim of the test ○ Test the innovation and develop a user journey – scenario ○ Identify technology requirements ○ Define assignments ○ Develop clear instructions ○ Identify methods ○ Share feedback ○ Iterate the test and evaluation
- 13. CHALLENGE: DROP-OUT!!! ● The participants’ motivations to involve in the living lab field test at the beginning of the project are usually higher than once the activity is underway. ● Therefore: ○ The participants who voluntarily participate in the living lab field tests tend to drop-out of user studies before the project or activity has ended. ● Why the issue of user drop-out is important? ○ Time efficiency ○ Cost efficiency ○ Quality assurance ○ The value of an established mutual trust ○ The participants' deep understanding about the project
- 14. “A drop-out during a living lab field test is when someone who signed up to participate in the field test, does not complete all the assigned tasks within the specified deadline” What is drop-out in living lab field tests?
- 15. Taxonomy of drop-out factors in Living Lab Field Test Drop-outinliving labfieldtest INNOVATION-RELATED Technological problems Perceived usefulness Perceived ease of use PARTICIPANT-RELATED Everyday context Participants’ attitude Participants’ resources PROCESS-RELATED Task design Interaction Timing
- 16. Taxonomy of drop-out factors in Living Lab Field Test
- 17. Recommendations on how to keep participants motivated ● Spending enough time to investigate the innovation’s functionality before engaging users ● A clear and on time communication and interaction with the users ● Giving the participants this feeling that their contribution is important ● Managing users’ expectation ● Flexible and appropriate timing of the Living Lab field test ● Avoiding to prolong the Living Lab field test ● Considering an appropriate financial reward for participants
- 18. Ethical considerations in Living Lab field test ● Voluntariness of user engagement ● Unwitting participation ● Informed consent ● Overlooking the participants’ interests ● Costs and benefits of their participation ● The ethical interaction with the research participants
- 19. Questions? 1/ questions from the chat are introduced by moderators 2/ questions from the audience? Abdolrasoul Habibipour Information Systems, Luleå Tekniska Universitet User Engagement Expert, Botnia Living Lab Abdolrasoul.Habibipour@ltu.se
- 20. Thank you! The next webinars in the series: XXX NEXT WEBINAR DETAILS XXX www.bit,ly/LLmaterials : sign-up for future webinars
Editor's Notes
- #2: (ENoLL will facilitate this slide, and give a very brief few sentence introduction to start the webinar - usually a few minutes late, to allow participants to join)
- #3: (After ENoLL introduction, you can begin with this slide)
- #4: Introduction: what is this presentation about? (suggested: “experiences with LLs for public administration” - feel free to change this, but please keep us in the loop via email so we update dissemination materials, too) > Think about adding a personal touch! Maybe, introduce a question you’d like the participants to think about during the presentation and to come back to during the Q&A, or tell a personal story about yourself/the experiences of your LL that relates to this topic. Or maybe take the opportunity to share a lesson learnt, achievement, or challenge you’ve faced in this area!
- #5: TABLE OF CONTENTS?
- #6: Definition of field test The aim of field test is to test (a procedure, a product, service, etc.) in actual situations reflecting intended use. (Merriam-Webster Dictionary, 2018) Experiment, research, or trial conducted under actual use conditions, instead of under controlled conditions in a laboratory. (BusinessDictionary.com, 2018) Living Lab field test In a living lab setting, a field test is a user study in which the interaction of test users with an innovation in the context of use is tested and evaluated in real life setting
- #7: What are the key differences between a LLFT and traditional test? Test and experiment with new innovations in real world contexts Carry out exploratory studies with voluntary contributors The level of functional maturity in LLFT is usually lower compared to traditional FT that the innovation is highly mature Provide insights into unexpected uses of products and services to identify opportunities for innovation
- #8: Its not easy to ensure that the setting is representative The Hawthorne effect: (the observer effect): is the tendency for humans to improve their performance if they know it is being observed Extra work for end-users because the field test is in progress in LLFT usually there are also other challenges such as Recruitment and resource allocation Potential legal and ethical challenges because users are involved in their real life situation Next to this, The lack of control over the situation and over the innovation is also a challenge And finally, Its not also easy to understand who is actually testing the innovation
- #9: So, what are the Key components of Living Lab field tests? Like any other LL activity, there are some key components while conducting a LLFT First the Innovation. A (digital) innovation is tested and evaluated by different stakeholders and participants in the Living Lab context Approach. Is a mixture of data collection methods and engagement of different stakeholders in Living Lab field test process Citizens. The participants are co-testers with the power of decision-making ICT-Infrastructure. Different technologies that support different tasks and activities in Living Lab field test in real-life experimentation (e.g., IoT deployments, sensor networks, distributed tools) Governance and management. The way that a Living Lab field test in the strategic or operational level is managed and organized Partners. Various stakeholders who participate in the Living Lab field test including public actors such as cities, researchers and government as well as private actors such as citizens, visitors, inhabitants, etc
- #10: When it comes to key principles of a LLFT, it should be Societal and value creating. Influencing societal challenges and providing innovative solutions by a shared vision in a long-term period Open and inclusive. Multi-directional flow of knowledge and experience between all stakeholders, including research centers, universities, Living Lab organizations, companies, developers and citizens Explorative. The Living Lab field test is conducted in an explorative environment, driven by curiosity and difficult to predict Responsible and sustainable. Evaluative and anticipate the impact of the innovation on individual, organizational and societal level both now and in the future Real life context. Experimentation, engagement and interaction of stakeholders with the innovation in real life environment
- #11: Real-life environment case in more detail > Think about including pictures here!
- #12: Several questions must be answered before and while conducting a LLFT First evaluation: Should we conduct tests for this innovation now? Method selection: How should we conduct the field tests? Resource allocation: What resources do we need and how do we use them? User recruitment: How many users should be involved and how do we motivate them? Setting: In what environment or situation should we conduct the field tests? Analysis: What can we learn from the feedback we have received? Final evaluation: What can we do to improve this innovation?
- #13: Develop a test story line for the test process: Cleary define the aim of the test Test the innovation and develop a test scenario for use and experience Identify technology requirements – what is needed from end-users and context? Define tasks (and split them to micro-tasks if necessary) – make sure that the innovation as a whole is tested Developing a clear instructions for tasks, data collection methods and tools (how to capture insights) Identify methods for recruiting, motivating and interacting with end-users Share feedback with relevant actors in appropriate form Iterate the test and evaluation by improving, testing again and validating the field test results
- #14: Motivations and expectations are changed over time >50% drop-out
- #18: Spending enough time to investigate the innovation’s functionality before engaging users by having a pre-test in small scale A clear and on time communication and interaction with the users having a clear contact points, etc Giving the participants this feeling that their contribution is important Managing users’ expectation by telling them that it is a prototype not a final version Flexible and appropriate timing of the Living Lab field test, so they can participate at their own pace is possible Avoiding to prolong the Living Lab field test Considering an appropriate financial reward for participants
- #19: Voluntariness of user engagement (due to group pressure, received technology) Unwitting participation (sometimes it’s not possible to opt-out of the LLFT because the setting is an airport, a train station, a city hall) Informed consent (It is not enough to only get permission, but also tell them about the potential costs and risks of their participation, also at homes it’s tricky because of presence of children, etc.) Overlooking the participants’ interests (don’t consider them as cheap or unpaid contributors, or like giny pigs) Costs and benefits of their participation (costs of participation are real, but the benefits are uncertain, having a reward to secures pay-back) The ethical interaction with the research participants
- #20: (ENoLL will facilitate the Q&A section: first, going through the questions entered in the chat, then opening the floor for the audience to ask questions. You will be the one answering all questions - except, if someone is asking about when the next webinar will be held etc. about the webinar series, rather than content wise)
- #21: (ENoLL will conclude with this slide)