A Decade of Interapy Research

Editor's Notes

• #2: I am a PHD student at the Department of clinical psychology at the University of Amsterdam. In this presentation, I would like to give you an overview of a research program of this department which we call Interapy, in which we examine the feasibility of Internet-based psychotherapy. I will briefly describe Interapy, to give you a basic understanding to our approach to online treatment, and I will then discuss the efficacy of Interapy as established in a series of controlled clinical trials, and its effectiveness in routine clinical practice.
• #3: Interapy is online CBT: cognitive behavior therapy which is delivered over the Internet. It is completely online: it does not involve any face to face contact. Patients can access treatment anywhere, as long as they can access the Internet with a common webbrowser. Interapy should not be confused for telepsychiatry. It is not. It does not involve videoconferencing. Instead, patients interact with their therapists through asynchronous text-messages. It is perhaps best to think of the therapeutic dialogue as an e-mail conversation. This conversation, however, is highly standardized: Interapy treatments are computerized manuals that specify the who, what and when of every step of treatment. Clients complete a predetermined sequence of homework assignments, in which they are supported by regular and scheduled therapist feedback.
• #4: Here you see a screenshot of the system that we used in most of our research. As you can see, this is very simple webappplication. The steps of the treatment are listed on the left, in the yellow bar, and in the center pane users complete the tasks that are assigned to them as specified by the treatment manual. Now, I admit it, this is not really what you would call an immersive, engaging therapeutical environment. However, in this case, looks are deceiving. Although the program may look simple, our research so far shows that its effects are promising.
• #5: Over the years, Interapy has been evaluated in nine controlled efficacy trials across a variety of mental health disorders. Anxiety disorders, such as panic disorder and posttraumatic stress, but also depression, burnout and bulimia nervosa. These trials included a total of 840 participants. The majority of studies were randomized controlled trials, in which participants were randomly assigned to Interapy or to a waiting list control condition.
• #6: In each of our studies, we assessed treatment effects through the repeated administration of self-report measures of symptom severity, such as the Beck Depression Inventory, the impact of event scale, and the self-report versions of the Panic Disorder Severity Scale and the Eating Disorder Dxamination. These are reliable and well-established instruments, which are often used in clinical outcome research because of their psychometric characteristics. Relevant to our research, these measures were also validated for online assessment.
• #7: So, what did we find? Did online CBT induce significant reductions in primary symptoms? The answer is yes. Across the efficacy trials, we found that symptoms reduced more with Interapy, in comparison to the experimental controls. The plot shows between-group effect sizes for the nine Interapy trials. The boxes represent the point estimate of the standardized mean difference in improvement between Interapy and the control group. The horizontal lines, crossing the boxes, represent the 95% confidence interval. The plot tells us that the standardized mean difference ranged from .5 to 1.3 in favor of Interapy, and that these differences were significant, since none of the confidence intervals cross zero. At the bottom of the figure, the black diamond marks the pooled effect size, the overall effect of Interapy, which is .9. This represents a large effect, roughly equivalent to the effect of face to face CBT.
• #8: Now, it is one thing to show that a treatment is effective for the average patient, but that does not tell us much about the clinical relevance of the improvements experienced by individual patients. For this, we contrasted the treatments and the experimental controls in terms of the number of patients meeting the conservative criteria of reliable recovery. According to these criteria, patients are considered recovered only if their change scores represent improvents that are large enough to rule out measurement error, and if they score below the clinical cut-off at posttest. Using these criteria, we found that patients were more likely to recover with Interapy. Across the trials, the average recovery rate with Interapy was 60% while this was only 23% in the experimental comparison groups. If you translate that to an odds ratio, as we did in this effect plot, we see that the overall difference in recovery equates to a significant odds ratio of 6.1, which represents a moderate-to-large effect size.
• #9: So, the controlled trials support the efficacy of Interapy. What about its effectiveness in the real world? In 2001, members of the Interapy research group founded the Interapy clinic, with the aim of implementing online CBT in the public health system. This clinic has been in operation since, not the least because of an important milestone which was reached in 2005, when health regulatory bodies officially recognized Interapy as an alternative to face to face treatment. Since then, costs of online CBT are reimbursed through public health assurance to all Dutch citizens with a GP-referral for psychotherapy. The programs and procedures that are used in this clinic are identical to the ones we used in the clinical trials. Therefore, to assess the effectiveness of online CBT in routine clinical practice, we could export the relevant data from the electronic patient records of the database of the clinic. In this effectiveness study, which we will submit for publication this summer, we examined treatment outcome of 1500 patients.
• #10: This plot shows mean standardized changes in symptom severity as observed in clinical practice. Data suggest that improvements are immediate and stable up to one year after treatment. On the short-term (at post-test and six weeks follow-up), patients report significant reductions in symptom severity. One year after treatment, these gains are maintained. In terms of effect size, the improvements are large. The overall effect size is 1.4, which is higher than the .9 effect size of the controlled trials, but that was to be expected, because there was no control group in this study.
• #11: In this plot, the dashed lines represent scores of treated participants of the clinical trials and the solid lines represent scores of patients of the clinic. As can be seen, patients of the clinic gain more because their symptom levels are higher at pretest.
• #12: This table shows the results of the clinical significance analysis of the effectiveness data. About half of the patients, 49%, experience a clinically significant change, and recover from their symptoms. If we also include reliable improvement as a the overall response rate is 71%: 7 out of ten of the patients experience a reliable improvement.
• #13: Online treatments are commonly associated with high drop-out. This problem appears to be related to the lack of therapist support. When therapist assistance is provided, as in Interapy, adherence rates are much better. In our clinical trials, an unweighted average of 82% of the patients complete every step of treatment. In routine clinical practice, treatment adherence is somewhat less, 71%, but that percentage still comparabes well to known dropout rates in Dutch mental healthcare.
• #14: Of course, we should consider the limitations of the present evidence. - In the clinical trials, Interapy has been evaluated primarily against waiting-list comparison groups. Any reference to the equivalence of online CBT and face to face therapy is based on comparisons of effect sizes that were collected in different contexts. Firm conclusions require equivalence trials or superiority trials. a second limitation is that Interapy has been evaluated mostly by researchers who developed the program. Only two of the nine controlled trials were conducted by independent research groups. Although this is not uncommon in emerging fields, we need more independent evaluations. Finally, I should mention that long-term assessments were uncontrolled, and that study attrition in the clinical practice study was high at one-year follow-up. The present evidence with regard to the long-term outcome of online CBT is encouraging, but needs to be corroborated.
• #15: In sum, the Interapy research of the past decade has taught us that Online CBT can be efficious, across a variety of mental health disorders, and that effects can be large and clinically significant. That the lack of face to face contact does not necessarily imply high dropout. In our studies, treatment adherence was acceptable. Online CBT is effective in routine practice. Results of the clinical trials generalize well to routine clinical practice. Considering the evidence, I hope you agree with me that online CBT appears to be feasable alternative to face to face treatment.
• #16: Thank you for your attention.