Preventing failed interventions

Editor's Notes

• #2: Good afternoon, I am Orla Mooney and welcome to my presentation on developing my skill for intervention design
• #3: The overall aim of my PhD (subject to change) is to develop a self-management intervention for teenagers living with Epilepsy. When working on my systematic review, other reviews in this area for children and young people have identified mixed support for the small number of interventions that have been developed and this is why I wanted to develop my skills in the area of intervention design
• #4: This is a diagram of my skills assessment and from my work on a student internship across 3 summers I noticed gaps in interventions being underpinned by theory, a lack of tools online for intervention design and seemingly a mismatch between what participants want and need from interventions (that relates back to the comment on slide one, where an intervention designer explained sometimes we go into we know best mode. So I really wanted to hon my skills and learn more about intervention design as understanding possible reasons of failure, can help me from phase 2 not just at the end at the design phase.
• #5: To develop this skill of intervention design, I took a 5 day course, ‘Preventing Failed Interventions in Behavioural Research’ in Erasmus MC in Feb which involved 40 hours of class attendance, in class group sessions and a group presentation on the final day
• #6: My expectations were that I would be informed of best practice in intervention design and implement what I learnt into the next phases of my research and also for research I may do, after my PhD
• #7: This course elaborates on intervention development, implementation and evaluation in the field of medical psychology. The focus is on learning from encountered difficulties, mistakes and failures from previous research and researchers. Some of the topics addressed were: exampled of failed interventions that the trainers were involve in, why we need to do sample size calculations for example, for grants and for power, however we often do it the other way around based on availability of participants, money, time restraints and personnel available. We looked at the methods of administration in relation to questionnaires (for baseline and evaluations) and how to improve response, for example, incentives, reminders, involving clinicians and also discussed time spent on missing data and drop outs. The intervention mapping I will discuss in more detail next. Monitoring interventions was about the importance of the provider/therapist and how we assume all providers provide the same intervention. Benchmarking therapists was also important and an interesting idea of recruiting less experienced ones, as they are more likely to follow your exact instructions and not feel that they know better and make their own adjustments and monitoring such as some type of recordings of sessions. For biases we explored topics such as social desirability, framing effects and issues around likert scales such as how many labels and should you have a middle label or reverse scales. We also looked at the use of SPIRIT and CONSORT checklists which help you check you have all the relevant information to report on interventions. Finally we had 24 hours to work within groups to develop an intervention protocol and present it to the class.
• #9: For example, an intervention for Hep C was to run across 14 centres, it required 300 participants, 150 in each arm and ended up with 33 recruits and only 5 completed. Some of the reasons they identified: changes of personnel, buy in from consultants however it was nurses who would be involved, there were only 400 eligible participants to recruit from their inclusion/exclusion criteria, they initially excluded people with HIV but had to remove that, most participants were ex-drug addicts who can have issues with adherence
• #10: For example, an intervention for Hep C was to run across 14 centres, it required 300 participants, 150 in each arm and ended up with 33 recruits and only 5 completed. Some of the reasons they identified: changes of personnel, buy in from consultants however it was nurses who would be involved, there were only 400 eligible participants to recruit from their inclusion/exclusion criteria, they initially excluded people with HIV but had to remove that, most participants were ex-drug addicts who can have issues with adherence
• #11: Intervention mapping is a protocol for developing theory-based and evidence-based health promotion programs. Intervention Mapping describes the process of health promotion program planning in six steps: the needs assessment based on the PRECEDE-PROCEED model. I found this an excellent tool to learn about and essential for the next phases of my project. It provides 6 clear steps to follow, for example, beginning with needs assessments which I will implement with a qualitative phase where I will involve stakeholders (through interview or online focus groups) to identify what self-management they currently do and what their self-management needs are – what they feel is not being met (not what I think they are). From this I can then identify the relevant behaviour and environmental change, performance objectives and change determinants. With this in place, I can identify which change theory will best underpin the intervention. For the design of the intervention I have learnt from the course that it is not just enough to involve participants only at the qualitative stage and that it is important that they get to have an input in the design also, and any persons who will be involved in the delivery. This will further map the actual needs with the most suitable self-management tools/modules for the program. Moving on to implementation and evaluation. The evaluation is an important step as from reading much literature it is an area that is not reported regularly on and can contribute to the mixed results.