Implementing Ethics for a Mobile App Deployment
1 like579 views
The document discusses the complexities of implementing ethics in deployment-based research, particularly in the context of a mobile app called Quped designed to promote walking. It highlights the challenges of informed consent, communication between researchers and participants, and the review processes that vary across institutions and platforms. The conclusion emphasizes the need for HCI to adapt ethical practices to keep pace with the evolving nature of technology design.
Implementing Ethics for a Mobile App Deployment
- 1. IMPLEMENTING ETHICS FOR A MOBILE APP DEPLOYMENT JOHN ROOKSBY, PARVIN ASADZADEH, ALISTAIR MORRISON, CLAIRE MCCALLUM, CINDY GRAY, MATTHEW CHALMERS
- 2. These are slides for a “GIST” seminar at the University of Glasgow (17 / 11 / 2016). Pre-conference talk about OzCHI paper. Full paper here: johnrooksby.org/publications.html
- 3. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
- 4. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 IN THIS TALK ▸ Case study of “implementing” or “operationalising” ethics in a deployment-based study. ▸ No claim that our work is perfect. Instead, an exploration of the problem of ethics: ▸ There is uncertainty and complexity in research ethics for deployment studies. ▸ Research ethics are enmeshed with system design in deployment studies. ▸ I will conclude that a “in action” view of research ethics is appropriate.
- 5. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 IN-ACTION ETHICS ▸ “The realities of conducting the study can unexpectedly differ from what the researchers have planned for” Munteanu et al. Situational Ethics: Re-thinking Approaches to Formal Ethics Requirements for Human- Computer Interaction. Proc. CHI ’15. ▸ “While technology design has become explorative, situated and responsive, the accompanying ethics processes largely remain static and anticipatory.” Frauenberger et al. In Action Ethics. Interacting With Computers 2016. ▸ Responsible HCI research “is not an absolute virtue but an attitude and practice” Grimpe et al. Towards a closer dialogue between policy and practice: responsible design in HCI. Proc. CHI ’14.
- 6. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 DEPLOYMENT BASED RESEARCH ▸ Characteristics: ▸ A technology released “in the wild” (outside of a controlled environment), over a prolonged period. ▸ Beta/advanced prototype. ▸ Uptake is not necessarily via formal recruitment, and often there is little interaction between participants and researchers. ▸ Data is collected during use of the system.
- 7. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 DEPLOYMENT BASED RESEARCH ▸ Problems: ▸ People may be unaware they are participating in a study. ▸ Limited opportunity for communication between participants and researchers. ▸ It is difficult to anticipate and manage problems outside of a controlled environment.
- 8. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016
- 9. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 QUPED ▸ Designed to encourage walking. Uses three behavioural change strategies: ▸ Self monitoring (step counts) ▸ Automated goal setting ▸ Social comparison
- 10. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 ETHICS OF QUPED ▸ Formal review processes ▸ Institutional review and Apple review ▸ Supporting autonomy ▸ Informed consent and withdrawal ▸ There are other ethical issues ▸ Risk - covered in the paper but not the talk ▸ Multiple other issues in ethics and responsible research
- 11. FORMAL REVIEWS
- 12. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016TEXT INSTITUTIONAL REVIEWS ▸ Until recently, adherence to ‘codes of ethics’ was the norm. ▸ Now, many institutions and funding bodies mandate a formal ethical review for any project involving human subjects. ▸ “Ethical Creep” Brown et al. Five Provocations for Ethical HCI Research. Proc. CHI ’16.
- 13. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016TEXT INSTITUTIONAL REVIEWS ▸ ACM has long had a Code of Ethics ▸ ACM Conferences (such as CHI) moving towards mandating formal, institutional reviews:
- 14. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 INSTITUTIONAL REVIEW ▸ At the University of Glasgow, research with (adult, non-vulnerable) human participants is reviewed by a college ethics committee. There are four colleges, including: ▸ College of Science and Engineering. ▸ College of Social Science. ▸ The application forms/processes for the colleges are different. ▸ Science and Engineering has an orientation to lab studies, Social Science to field studies. Neither directly accommodates deployment research. ▸ Neither specifies what the ethical issues are.
- 15. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 INSTITUTIONAL REVIEW ▸ For Quped, as a multi-disciplinary team we had a choice of committee. We chose Social Science: ▸ Deployments more akin to fieldwork. ▸ Previous experience with applications showed an application to social science would be easier. ▸ We told Claire it would be a “good experience” for her to apply.
- 16. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 APPLE REVIEW ▸ App Store Apps are also reviewed by Apple, with their own criteria, e.g.: ▸ “Apps conducting health-related human subject research must obtain consent from participants or, in the case of minors, their parent or guardian. Such consent must include the (a) nature, purpose, and duration of the research; (b) procedures, risks, and benefits to the participant; (c) information about confidentiality and handling of data (including any sharing with third parties); (d) a point of contact for participant questions; and (e) the withdrawal process.” ▸ Two separate, independent and overlapping review processes.
- 17. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 FORMAL REVIEW: SUMMARY ▸ Multiple, partially conflicting guidelines and processes. ▸ Guidelines must be translated to specific study. ▸ Reviews are late in the day - the objective is to “pass” the review, not to consider ethical issues at this point. ▸ Different organisations have jurisdiction over ethics.
- 18. AUTONOMY
- 19. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 AUTONOMY ▸ The term autonomy comes from an ancient Greek term translating as “self law” (i.e. to be a law to oneself). ▸ In the context of research ethics, it means that an individual should be able to make an informed and free decision to participate in a study. ▸ Often seen as key for research ethics, however: ▸ Consent as severance from data. Luger and Rodden Luger, E. and Rodden, T. An Informed View on Consent for UbiComp. UbiComp ’13. ▸ Consent as limitation of legal liability. Cairns, P. and Thimbleby, H. The diversity and ethics of HCI. Computer and Information Science ’03.
- 20. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 CONSENT ▸ For Quped, main recruitment mechanism is the app store. It is not possible to ensure people are informed about the study before the install. ▸ Cannot frame app as a research study in the app store. ▸ Cannot specify the app is for adults. ▸ Installation cannot be considered consent. ▸ Therefore, we need in-app mechanisms to gain consent.
- 21. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 CONSENT ▸ Consent integrated with on-boarding process
- 22. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 CONSENT ▸ In app information sheet appears after description. ▸ No data uploaded until after ‘accept’ ▸ A ‘decline’ button (just sends you back) ▸ Scrollable in textarea to log how far people view the text ▸ Under 18 switch ▸ Changed in an update to be ‘switch off is you are under 18’.
- 23. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 CONSENT ▸ In app information sheet appears after description. ▸ No data uploaded until after ‘accept’ ▸ A ‘decline’ button (just sends you back) ▸ Scrollable in textarea to log how far people view the text ▸ Under 18 switch ▸ Changed in an update to be ‘switch off is you are under 18’.
- 24. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 CONSENT ▸ Apple also requires the app to ask for permissions. ▸ Pop up, ‘just in time’ permission requests.
- 25. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 CONSENT ▸ Information sheet also available in-app.
- 26. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 CONSENT ▸ Scaffolded approach to collecting demographic data ▸ This data not required to use the app, only for social comparison. ▸ Second opportunity for identifying non-adults ▸ If a user specifies under 18, they are excluded from the study.
- 27. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 WITHDRAWAL ▸ People should also be able to freely withdraw from a research study ▸ Withdrawing from a deployment somewhat different from with drawing from a lab study. ▸ Stopping using the app: people can do this whenever they wish. ▸ Pulling out of the study (i.e. data not to be included): we need to support this in-app.
- 28. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 WITHDRAWAL
- 29. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 DEBRIEFING ▸ Debriefing is seen as good practice in research studies ▸ Another opportunity to give information and ask questions ▸ However, with a deployment such as Quped there is no end. ▸ If a user deletes the app, there is no way to contact them (unless we collected extra data such as email address). ▸ Left debriefing as a design problem to address later. ▸ We can turn off logging after some point (12 months?) ▸ We could create a strategy based on whether the intervention does/ doesn’t work.
- 30. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 AUTONOMY: SUMMARY ▸ Standard approaches to autonomy problematised in deployment research. ▸ Automation, with limited opportunity for discussion ▸ Changing nature of withdrawal and debriefing ▸ Consent and withdrawal part of the app design. ▸ Design changes were made after release, and some decisions put off until future.
- 31. DISCUSSION
- 32. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 UNCERTAINTY IN ETHICS ▸ No clear set of ethical rules and procedures, but multiple, diverging review processes and guidelines for us to navigate. ▸ The deployment is built upon opaque infrastructure that sits between the participants and us, which constrains and to an extent dictates what we can do. ▸ As HCI researchers we have a concern for naturalistic, ecologically valid data. This produces tensions with ethical concerns for informing people about the research. ▸ For long term deployments, some issues are clarified during the deployment. Therefore it can be appropriate to revisit ethical decisions.
- 33. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 ETHICS AS DESIGN ▸ Research ethics had to be worked out in anticipation of the ethical reviews during the design process. Problems with reviews may have entailed major changes to our design. ▸ We needed to create mechanisms for consent, withdrawal and data management that were a coherent part of an overall user experience. These needed to be present in the app, easy to understand and not off-putting. ▸ App design is iterative, and updates and changes can have implications for our ethics procedures.
- 34. JOHN ROOKSBY | IMPLEMENTING ETHICS | OZCHI 2016 CONCLUSION ▸ “While technology design has become explorative, situated and responsive, the accompanying ethics processes largely remain static and anticipatory.” Frauenberger et al. In Action Ethics. Interacting With Computers 2016. ▸ Implications ▸ HCI should share and discuss design knowledge relevant to research ethics ▸ Ongoing support research ethics in the planning/design phases and post- release ▸ Reviews are valuable (in principle) but better support needed for deployment based research