Flaskdata.io automated monitoring for clinical trials
0 likes232 views
The document discusses the evolution and challenges of clinical trial monitoring, particularly in the context of virtual trials during the COVID-19 pandemic. It highlights the complexity of virtual trials as distributed systems that require new monitoring strategies, emphasizing the need for automation in data collection, detection of anomalies, and real-time responses. The author calls for a reevaluation of clinical trial design and operations to address the inefficiencies and risks associated with traditional monitoring methods.
Flaskdata.io automated monitoring for clinical trials
- 1. Danny Lieberman, Founder and CEO Clinical Trial Monitoring in a COVID-19 era The language of automation for clinical trials June 2020 flaskdata.io
- 2. • To understand how automation can be used to speed delivery of valid data to decision makers in clinical trials, we must first understand where we are today and how we got there. • Today, virtual trials are popular because of COVID-19. • We’ll show that virtual trials are complex distributed systems with a new set of problems. • We’ll review the history of clinical trials and show that little has changed since the big streptomycin trial in 1948 • We’ll devote the last part of the talk to how monitoring automation works. This talk
- 3. The language of automation • Collect - collect data from patients, investigators and devices. The study designers should be able to choose the right model for their therapeutic and patient population. Whether fully virtual or site-centric or a hybrid - the study designers should have reliable tools to collect data. Data collection can be from a phone, a desktop, a Web browser or an API. It should not matter. • Detect - automated detection of protocol violations uses real-time data and user behavior to detect anomalies and validate data streams. • Act - automated detection should be able to trigger an automated response; whether a push notification to a site coordinator phone or a Web hook callback.
- 5. 3 Virtual trials A brief history We're not in Kansas anymore Monitoring distributed systems 4 Outline 1 2 3 4
- 6. Virtual trials Virtual Clinical Trials (VCTs), also called remote or decentralized trials, are a relatively new and yet underutilized method of conducting clinical research taking full advantage of technologies such as apps, electronic monitoring devices, and online social engagement platforms. https://leoinnovationlab.com/2020/04/02/virtual-clinical-trials-create-new-possibilities-for-patients-
- 7. Virtual trials: Complex distributed systems Virtual clinical trials may save time and money. John Robert Zibert - LEO Innovation Lab 4/2020 eCOA ePRO eSource eConsent EDC
- 8. 3 Virtual trials A brief history We're not in Kansas anymore Monitoring distributed systems 41 2 3 4
- 9. A brief history of time. Daniel 1 526BCE Avicenna Canon of Medicine James Lind and the scurvy trial 1025 1747 Streptomycin RCT 1946 FDA Guidance on RBM 2013
- 10. Daniel 1 Please test your servants for ten days: Give us nothing but vegetables to eat and water to drink. Then compare our appearance with that of the young men who eat the royal food, and treat your servants in accordance with what you see.” So he agreed to this and tested them for ten days. At the end of the ten days they looked healthier and better nourished than any of the young men who ate the royal food. So the guard took away their choice food and the wine they were to drink and gave them vegetables instead.
- 11. Avicenna - Drug testing rules ‣ Disease without complications ‣ 2 contrary cases ‣ At time of action and reproducibility
- 12. James Lind and Scurvy trial ‣ Cohort 1 Gruel and mutton-broth ‣ Cohort 2 Quart of cyder a day ‣ 50 years later British Navy made lime compulsory
- 13. Clinical trial monitoring basics (1948) ‣ Communicate with PI and site staff ‣ Review site processes, procedures ‣ Verify data accuracy
- 14. ICH E6 and ISO 14155:2011 ‣ Monitoring plan ‣ Design, complexity, size, and endpoints ‣ Consistent with FDA Guidance
- 15. FDA Guidance for RBM (2013) ‣ Centralized monitoring can replace onsite visits (1) ‣ Focus on site, not patient oversight ‣ Response: 2-3 days to 5-7 weeks after events ‣ Mitigate safety, quality risk ‣ Too slow for virtual/hybrid trials in a COVID-19 era (1) Bakobaki et al. The Potential for Central Monitoring Techniques to Replace On-Site Monitoring:(2012)
- 16. Over-monitoring ‣ Monitoring is non-invasive and seems harmless (1) ‣ 72–99% of ICU alarms clinically insignificant ‣ Cry wolf syndrome (1) Feder and Funk, Over-monitoring and alarm fatigue: For home the bells toll? https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4926996/
- 17. Onsite monitoring with CRAs ‣ Onsite monitoring 20-35% of study cost ‣ Log deviations 7-13 weeks after data event ‣ SDV updates < 3% of data ‣ Creates data cleaning bottleneck at study close
- 18. Monitoring with Dropbox (2020) ‣ Remote monitoring with PDF ‣ More paper ‣ Less site visits. Not good.
- 19. 3 A brief history We're not in Kansas anymore Monitoring distributed systems 41 2 3 4 Virtual trials
- 20. Regulatory and reimbursement conflicts Generalizable data from near-real-life usage Well-designed controlled clinical studies Security and privacy vulnerabilities
- 21. Vintage monitoring of virtual trials does not work Sites Study monitors Doing what they feel like Things happening fast Patients are anywhere 10x
- 22. Submission by December 17 Need to revisit clinical trial design and operations ‣ Wearables, devices, apps ‣ Cloud services and APIs ‣ Real-time data & response for real-life usage
- 23. 3 A brief history We're not in Kansas anymore 41 2 3 4 Virtual trials Monitoring distributed systems
- 24. Monitoring by alerts ‣ Alerts urgent, important, actionable and real ‣ Symptoms better than causes ‣ Visualize trends
- 25. What are alerts? ‣ A metric over / under a threshold ‣ GCP, safety, common sense ‣ Rate of data acquisition
- 26. In your monitoring plan ‣ Monitor for you ‣ Symptom-based ‣ Latency. Fast. Fast. Fast. 10x
- 27. Alerts require 4 things ‣ Detection ‣ Enable mitigation ‣ Share knowledge ‣ Route to decision maker
- 28. Writing alert rules ‣ Metric and threshold ‣ Over-monitoring harder than under-monitoring ‣ Classify: Data latency, Protocol violation, AE/SAE, Recruitment
- 29. Taking action ‣ Playbook is important ‣ Rule or family of rules ‣ What it means and how to mitigate
- 30. Alert V&V ‣ Validation: Are we calculating the right metric? ‣ Verification: Are we calculating the metric right?
- 31. Alerts, issues and email ‣ Don't send mail ‣ Issue-tracking systems may help ‣ Goal is simplicity and speed 10x
- 32. Tracking & accountability ‣ Track and periodically review alerts ‣ If WFM remove it ‣ If alert < 50% accurate its broken
- 33. Over-optimism ‣ Causes beneath noise ‣ Symptoms may arrive late ‣ Rules may be more complex than issues
- 34. Automated detection & response is now a requirement Wrap-up ‣ Patient-centric/virtual/hybrid clinical trials are complex distributed systems with new challenges. ‣ The old people and process methods used by CROs are too slow in a COVID-19 era ‣ Lessons can be learned from complex system monitoring
- 35. Thank you’s. ‣ Jenya, Rivka, Eugene, Alex, Oleg, Anat, Batya and Dan for joining me on the flaskdata.io journey ‣ Sergey and his DevOps soldiers for playing in the mud with us ‣ Tim and Gene for allowing me to join their adventure at Fidelis Security ‣ Rob Ewaschuk’s observations while a Site Reliability engineer at Google ‣ Lana’s insight on conflicts