SlideShare a Scribd company logo
Investigating Integration of Computerized Decision Support into Workflow at 3 Benchmark InstitutionsFebruary 17th, 2011Doebbeling B.N., Haggstrom, D.A., Militello, L.G., Flanagan, M.E., Arbuckle, C.L., Kiess, C.L., Saleem, J.J.VA HSR&D Center on Implementing Evidence-Based Practice;Regenstrief Institute; IU School of Medicine and Purdue School of Engineering
Acknowledgements2Supported by:Agency for Healthcare Quality and Research (AHRQ)HSA2902006000131Department of Veterans Affairs, VHA HSR&D  CDA 09-024-1
Background3Colorectal cancer screeningLow screening rates; evidence for screening effectivenessClinical decision support (CDS) has been associated with improved qualityHowever, the design and workflow integration of CDS may limit its impactRecent IOM Committee calling for new paradigm in cognitive support
Study Question4Barriers to colorectal cancer (CRC) screening and follow-up?Factors influencing integration into clinical workflow
Rationale for Study Sites“Benchmark institutions” for CDS  Regenstrief Institute, Partners Healthcare, Veterans Health Administration, (Intermountain Health Care)Early-adopter institutions that implemented internally developed health information technology systems, including computerized CDSWidely implemented CDS in these institutionsIdeal settings to study integration of CDS in workflow(Chaudhry et al, Ann Intern Med, 2006)Chaudhry et al., Ann Intern Med, 2006
MethodsCognitive Field ResearchEthnographic observationsOpportunistic interviewsStudy:  CDS for colorectal cancer (CRC) screening in Primary care clinics2 VAMCs2 teaching hospitals (Regenstrief, Partners) 
Form of CRC Screening CDS at Study Sites7VA Medical Center 1Computerized clinical reminder for CRC screeningVA Medical Center 2Suite of computerized CDS for risk stratification, screening, follow-up, and surveillanceRegenstrief InstitutePaper encounter form reminder for CRC screeningPartners HealthcareElectronic, template health maintenance list
Study Sites and Participants8Multiple outpatient clinics2-3 observers experienced in ethnographic observationParticipants: Observation & opportunistic interviews:   120 providers (physicians, NPs, PAs)118 patient encounters observedKey informant interviews: 11 providers2 Focus groups: 11 physicians
Organizations, settings, providers, patients9
Analyses10Coding template based on the sociotechnical modelSocial, technical, and external subsystemsQualitative analyses: top-down vs. bottom-up codingSummary and integrative findingsFindings integrated across sites
Barriers to colorectal cancer screening and follow-up11Lack of communication of “outside” exam resultsPoor data organization & presentationOmission of provider, patient education in CDSLack of interface flexibilityLack of coordination between primary care and GINeeded technological enhancementsUnclear role assignments
% of Coded Segments in the Technical Sub-Section by Themes & Sites12Implementing and Improving the Integration of Decision Support into Outpatient Clinical Workflow, AHRQ ACTION HSA2902006000131
13
Sociotechnical System Primary Code Frequency By Sites14
% of Coded Segments in the Technical Sub-Section by Themes & Sites15
Practices and design features161) organizational priorities; 2) contextual structure and process;3) team role assignments & workflow;4) coordination and communication between clinics and other services; 5) integrating outside results; 6) improved data organization, presentation; 7) just-in time patient education and provider cognitive support; 8) interface and user interaction; 9) technological enhancements.
ConclusionsDespite differences between health systems, barriers were quite consistent. New CDS prototypes are needed which: 1) improve data organization and presentation; 2) integrate outside results and 3) provide just-in time education and cognitive support.  Workflow variations, user-centered design and usability key to an information system that works in practice.Effective design and integration of new technologies requires mindful iteration.  
Thank-you!18Haggstrom DA, Saleem JJ, Militello LG, Arbuckle N, Flanagan M, Doebbeling BN.  Examining the relationship between clinical decision support and performance measurement.  Proc AMIA Symp 2009; 223-7.Saleem JJ, Militello LG, Arbuckle N, Flanagan M, Haggstrom DA, Linder JA, Doebbeling BN.  Provider perceptions of colorectal cancer screening decision support at three benchmark institutions. Proc AMIA Symp 2009; 558-62.
VA Computerized Patient Record SystemVAMC 119
Form of Colorectal Cancer Screening CDS at Study Sites20VA Medical Center 1Computerized clinical reminder for CRC screeningVA Medical Center 2Set of computerized clinical reminders for screening, follow-up, and surveillanceRegenstrief InstitutePaper encounter form reminder for CRC screeningPartners HealthcareElectronic, template health maintenance list
Form of Colorectal Cancer Screening CDS at Study Sites21VA Medical Center 1Computerized clinical reminder for CRC screeningVA Medical Center 2Set of computerized clinical reminders for screening, follow-up, and surveillanceRegenstrief InstitutePaper encounter form reminder for CRC screeningPartners HealthcareElectronic, template health maintenance list
Regenstrief Medical Record System22Printed paper encounter form with clinical reminders at the bottom.  Paper reminders automatically generated by the CDS rules.* Annual FOBT and periodic sigmoidoscopy are recommended for all persons aged 50 or over to screen for colorectal cancer.  If screening FOBT is positive, colonoscopy is recommended.* HEMOCCULT 1)Pt refused  2) Done Today (results: ___________________)
Form of Colorectal Cancer Screening CDS at Study Sites23VA Medical Center 1Computerized clinical reminder for CRC screeningVA Medical Center 2Set of computerized clinical reminders for screening, follow-up, and surveillanceRegenstrief InstitutePaper encounter form reminder for CRC screeningPartners HealthcareElectronic, template health maintenance list
Partners – Longitudinal Medical Record24
Receiving and documenting “outside” exam results25Physician: “In the [CRC] clinical reminder [dialog] box you cannot easily document that a colonoscopy was done outside of the VA. Say the patient had an outside colonoscopy done 5 years ago- you need to enter the exact date, time, location. But the patient may only remember that he had a colonoscopy about 5 years ago.”
Colorectal cancer screening CDS     not accurate26Physician: “One patient was sent to GI three times for a colonoscopy. Each time they told him he wasn’t due. But the reminder keeps coming up. He had a colonoscopy recently, so I don’t know why the reminder doesn’t turn off.”
Compliance issues27Physician: “They did it wrong up front – they completed the colorectal cancer screening reminder wrong.  It [the reminder] says the cards were given to the patient but she [health tech] did not give him the cards.  Every system has weak links.  This is one of them for us.”
Poor EHR or CDS usability28Physician AssistantPaper spreadsheet to track date and results for colonoscopiesNurse PractitionerNeed to repeat screening
Lack of coordination between primary care and GI29Physician: “GI should be able to clear out the [computerized clinical] reminder. For example, the patient we just saw…it took me a while to go through and satisfy it [the CRC clinical reminder]. The patients see lots of different people in the hospital and they all have their hands in the patient’s care. They should be satisfying some of the reminders as well.”
Acute vs. preventive care30Physician: “If I have to choose between chest pain and hemoccult [fecal occult blood test], I am going to choose chest pain.”

More Related Content

PPT
HL7: Clinical Decision Support
PDF
Web application for clinicians - SidekickCV
PDF
The Business of Genomic Testing by James Crawford
PPT
Govindarajan cer saem 2011
PDF
The De-Identification of a Large Electronic Medical Records Database for Seco...
PDF
Best Practices for a Data-driven Approach to Test Utilization
PDF
Advanced Laboratory Analytics — A Disruptive Solution for Health Systems
PPT
Kaiser permanente transition care
HL7: Clinical Decision Support
Web application for clinicians - SidekickCV
The Business of Genomic Testing by James Crawford
Govindarajan cer saem 2011
The De-Identification of a Large Electronic Medical Records Database for Seco...
Best Practices for a Data-driven Approach to Test Utilization
Advanced Laboratory Analytics — A Disruptive Solution for Health Systems
Kaiser permanente transition care

What's hot (20)

PDF
Test Utilization White Paper
PDF
Tackling the U.S. Healthcare System’s Infectious Disease Management Problem
DOCX
Christie 2015 Resume
PDF
Noninvasive prenatal testing_for_fetal_aneuploidy
PPT
Clinical Decision Support Systems
PDF
Revolutionizing Renal Care With Predictive Analytics for CKD
PDF
Patient Centered Medical Home (PCMH) is not a pill Kevin Grumbach 2013
PPTX
Clinical decision support system
PDF
Confronting Diagnostic Error-Employer
PDF
Advanced Lab Analytics for Patient Blood Management Programs
DOC
Updated Improved Licensed Professional Cytotechnologist Resume Short form
PDF
Hydroxyurea for the treatment of sickle cell disease2
PPTX
Asra defining value may 2015
PPTX
How Clinical Decision Support Systems (CDSS) is the right tool for physicians?
PPT
PreNatal Module, HughesRiskApps
PDF
Clinical Decision Support Systems
PDF
Bigtown simulation model
PDF
A Clinical Decision Support System For Alzheimer´s Disease and Other Related ...
PPT
Decision Support System for clinical practice created on the basis of the Un...
PDF
SVMPharma Real World Evidence - Randomised controlled trials were never desig...
Test Utilization White Paper
Tackling the U.S. Healthcare System’s Infectious Disease Management Problem
Christie 2015 Resume
Noninvasive prenatal testing_for_fetal_aneuploidy
Clinical Decision Support Systems
Revolutionizing Renal Care With Predictive Analytics for CKD
Patient Centered Medical Home (PCMH) is not a pill Kevin Grumbach 2013
Clinical decision support system
Confronting Diagnostic Error-Employer
Advanced Lab Analytics for Patient Blood Management Programs
Updated Improved Licensed Professional Cytotechnologist Resume Short form
Hydroxyurea for the treatment of sickle cell disease2
Asra defining value may 2015
How Clinical Decision Support Systems (CDSS) is the right tool for physicians?
PreNatal Module, HughesRiskApps
Clinical Decision Support Systems
Bigtown simulation model
A Clinical Decision Support System For Alzheimer´s Disease and Other Related ...
Decision Support System for clinical practice created on the basis of the Un...
SVMPharma Real World Evidence - Randomised controlled trials were never desig...
Ad

Viewers also liked (9)

PPT
Kho Amia2008 Demo Final
PPTX
Chir F2 F Mrsa 2011 10 18
PPTX
Taking Transformational Change To Scale.Doebbeling.3.9.10.Final
PPTX
Advancing Transformational Science Doebbeling Cpmrc 1.22.11
PPTX
Programs to Reduce MRSA Infections.8.28.10.Symposium on Healthcare Associated...
PPTX
Iu Ahrq Hai Assessment Ctr Presentation Feb 22 2010 Final
PPTX
Brad Mrsa For CDC Meeting Oct. 20 2009
PPTX
Ahrq Hai Collab Meeting 7.13 14.2009
PPTX
Chir Sc Mrsa.6.21.10
Kho Amia2008 Demo Final
Chir F2 F Mrsa 2011 10 18
Taking Transformational Change To Scale.Doebbeling.3.9.10.Final
Advancing Transformational Science Doebbeling Cpmrc 1.22.11
Programs to Reduce MRSA Infections.8.28.10.Symposium on Healthcare Associated...
Iu Ahrq Hai Assessment Ctr Presentation Feb 22 2010 Final
Brad Mrsa For CDC Meeting Oct. 20 2009
Ahrq Hai Collab Meeting 7.13 14.2009
Chir Sc Mrsa.6.21.10
Ad

Similar to Investigating Integration Of Computerized Decision Support Into Workflow Hsr&D Pres Feb 17 2011 (20)

PPTX
Amia Pres Oct 26 2011 Final
PPT
Brad Doebbeling Slides for AHRQ Kick-Off Event
PDF
Computer-Based Clinical Decision Support for Colorectal Diseases: An Overview...
PPT
David Haggstrom Slides from AHRQ Kick-Off Event
PDF
ACP 2018 Early Career Finalist Presentation
PPT
Jason Saleem Slides from AHRQ Kick-Off
PPTX
CDC Call-On Congress 2012
PPT
Heather Woodward Slides from AHRQ Kick-Off
DOCX
Module 5 (week 9) - InterventionAs you continue to work on your .docx
PPTX
IHE / RSNA Image Sharing Project - IHE Colombia Workshop (12/2014) Module 5c
PDF
10.e John Rayner - " Por un modelo de evaluación para la implantación de aten...
PPTX
Implementing Clinical Decision
PDF
Clinical Decision Support System Essay
PPTX
Clinical decision support systems
PDF
ACR Select: Clinical Decision Support Imaging Guidelines
PDF
Maine’s CRC Policy Story
PPTX
Participation of the population in decisions about their health and in the pr...
PDF
HIMSS EMRAM US 2008
PPT
Talat Ashraf Catch It Presentation
PPT
Talat Ashraf Catch It Presentation
Amia Pres Oct 26 2011 Final
Brad Doebbeling Slides for AHRQ Kick-Off Event
Computer-Based Clinical Decision Support for Colorectal Diseases: An Overview...
David Haggstrom Slides from AHRQ Kick-Off Event
ACP 2018 Early Career Finalist Presentation
Jason Saleem Slides from AHRQ Kick-Off
CDC Call-On Congress 2012
Heather Woodward Slides from AHRQ Kick-Off
Module 5 (week 9) - InterventionAs you continue to work on your .docx
IHE / RSNA Image Sharing Project - IHE Colombia Workshop (12/2014) Module 5c
10.e John Rayner - " Por un modelo de evaluación para la implantación de aten...
Implementing Clinical Decision
Clinical Decision Support System Essay
Clinical decision support systems
ACR Select: Clinical Decision Support Imaging Guidelines
Maine’s CRC Policy Story
Participation of the population in decisions about their health and in the pr...
HIMSS EMRAM US 2008
Talat Ashraf Catch It Presentation
Talat Ashraf Catch It Presentation

Investigating Integration Of Computerized Decision Support Into Workflow Hsr&D Pres Feb 17 2011

  • 1. Investigating Integration of Computerized Decision Support into Workflow at 3 Benchmark InstitutionsFebruary 17th, 2011Doebbeling B.N., Haggstrom, D.A., Militello, L.G., Flanagan, M.E., Arbuckle, C.L., Kiess, C.L., Saleem, J.J.VA HSR&D Center on Implementing Evidence-Based Practice;Regenstrief Institute; IU School of Medicine and Purdue School of Engineering
  • 2. Acknowledgements2Supported by:Agency for Healthcare Quality and Research (AHRQ)HSA2902006000131Department of Veterans Affairs, VHA HSR&D CDA 09-024-1
  • 3. Background3Colorectal cancer screeningLow screening rates; evidence for screening effectivenessClinical decision support (CDS) has been associated with improved qualityHowever, the design and workflow integration of CDS may limit its impactRecent IOM Committee calling for new paradigm in cognitive support
  • 4. Study Question4Barriers to colorectal cancer (CRC) screening and follow-up?Factors influencing integration into clinical workflow
  • 5. Rationale for Study Sites“Benchmark institutions” for CDS Regenstrief Institute, Partners Healthcare, Veterans Health Administration, (Intermountain Health Care)Early-adopter institutions that implemented internally developed health information technology systems, including computerized CDSWidely implemented CDS in these institutionsIdeal settings to study integration of CDS in workflow(Chaudhry et al, Ann Intern Med, 2006)Chaudhry et al., Ann Intern Med, 2006
  • 6. MethodsCognitive Field ResearchEthnographic observationsOpportunistic interviewsStudy: CDS for colorectal cancer (CRC) screening in Primary care clinics2 VAMCs2 teaching hospitals (Regenstrief, Partners) 
  • 7. Form of CRC Screening CDS at Study Sites7VA Medical Center 1Computerized clinical reminder for CRC screeningVA Medical Center 2Suite of computerized CDS for risk stratification, screening, follow-up, and surveillanceRegenstrief InstitutePaper encounter form reminder for CRC screeningPartners HealthcareElectronic, template health maintenance list
  • 8. Study Sites and Participants8Multiple outpatient clinics2-3 observers experienced in ethnographic observationParticipants: Observation & opportunistic interviews: 120 providers (physicians, NPs, PAs)118 patient encounters observedKey informant interviews: 11 providers2 Focus groups: 11 physicians
  • 10. Analyses10Coding template based on the sociotechnical modelSocial, technical, and external subsystemsQualitative analyses: top-down vs. bottom-up codingSummary and integrative findingsFindings integrated across sites
  • 11. Barriers to colorectal cancer screening and follow-up11Lack of communication of “outside” exam resultsPoor data organization & presentationOmission of provider, patient education in CDSLack of interface flexibilityLack of coordination between primary care and GINeeded technological enhancementsUnclear role assignments
  • 12. % of Coded Segments in the Technical Sub-Section by Themes & Sites12Implementing and Improving the Integration of Decision Support into Outpatient Clinical Workflow, AHRQ ACTION HSA2902006000131
  • 13. 13
  • 14. Sociotechnical System Primary Code Frequency By Sites14
  • 15. % of Coded Segments in the Technical Sub-Section by Themes & Sites15
  • 16. Practices and design features161) organizational priorities; 2) contextual structure and process;3) team role assignments & workflow;4) coordination and communication between clinics and other services; 5) integrating outside results; 6) improved data organization, presentation; 7) just-in time patient education and provider cognitive support; 8) interface and user interaction; 9) technological enhancements.
  • 17. ConclusionsDespite differences between health systems, barriers were quite consistent. New CDS prototypes are needed which: 1) improve data organization and presentation; 2) integrate outside results and 3) provide just-in time education and cognitive support.  Workflow variations, user-centered design and usability key to an information system that works in practice.Effective design and integration of new technologies requires mindful iteration.  
  • 18. Thank-you!18Haggstrom DA, Saleem JJ, Militello LG, Arbuckle N, Flanagan M, Doebbeling BN. Examining the relationship between clinical decision support and performance measurement. Proc AMIA Symp 2009; 223-7.Saleem JJ, Militello LG, Arbuckle N, Flanagan M, Haggstrom DA, Linder JA, Doebbeling BN. Provider perceptions of colorectal cancer screening decision support at three benchmark institutions. Proc AMIA Symp 2009; 558-62.
  • 19. VA Computerized Patient Record SystemVAMC 119
  • 20. Form of Colorectal Cancer Screening CDS at Study Sites20VA Medical Center 1Computerized clinical reminder for CRC screeningVA Medical Center 2Set of computerized clinical reminders for screening, follow-up, and surveillanceRegenstrief InstitutePaper encounter form reminder for CRC screeningPartners HealthcareElectronic, template health maintenance list
  • 21. Form of Colorectal Cancer Screening CDS at Study Sites21VA Medical Center 1Computerized clinical reminder for CRC screeningVA Medical Center 2Set of computerized clinical reminders for screening, follow-up, and surveillanceRegenstrief InstitutePaper encounter form reminder for CRC screeningPartners HealthcareElectronic, template health maintenance list
  • 22. Regenstrief Medical Record System22Printed paper encounter form with clinical reminders at the bottom. Paper reminders automatically generated by the CDS rules.* Annual FOBT and periodic sigmoidoscopy are recommended for all persons aged 50 or over to screen for colorectal cancer. If screening FOBT is positive, colonoscopy is recommended.* HEMOCCULT 1)Pt refused 2) Done Today (results: ___________________)
  • 23. Form of Colorectal Cancer Screening CDS at Study Sites23VA Medical Center 1Computerized clinical reminder for CRC screeningVA Medical Center 2Set of computerized clinical reminders for screening, follow-up, and surveillanceRegenstrief InstitutePaper encounter form reminder for CRC screeningPartners HealthcareElectronic, template health maintenance list
  • 24. Partners – Longitudinal Medical Record24
  • 25. Receiving and documenting “outside” exam results25Physician: “In the [CRC] clinical reminder [dialog] box you cannot easily document that a colonoscopy was done outside of the VA. Say the patient had an outside colonoscopy done 5 years ago- you need to enter the exact date, time, location. But the patient may only remember that he had a colonoscopy about 5 years ago.”
  • 26. Colorectal cancer screening CDS not accurate26Physician: “One patient was sent to GI three times for a colonoscopy. Each time they told him he wasn’t due. But the reminder keeps coming up. He had a colonoscopy recently, so I don’t know why the reminder doesn’t turn off.”
  • 27. Compliance issues27Physician: “They did it wrong up front – they completed the colorectal cancer screening reminder wrong. It [the reminder] says the cards were given to the patient but she [health tech] did not give him the cards. Every system has weak links. This is one of them for us.”
  • 28. Poor EHR or CDS usability28Physician AssistantPaper spreadsheet to track date and results for colonoscopiesNurse PractitionerNeed to repeat screening
  • 29. Lack of coordination between primary care and GI29Physician: “GI should be able to clear out the [computerized clinical] reminder. For example, the patient we just saw…it took me a while to go through and satisfy it [the CRC clinical reminder]. The patients see lots of different people in the hospital and they all have their hands in the patient’s care. They should be satisfying some of the reminders as well.”
  • 30. Acute vs. preventive care30Physician: “If I have to choose between chest pain and hemoccult [fecal occult blood test], I am going to choose chest pain.”

Editor's Notes

  • #7: Two studies of the integration of clinical decision support tools Two days of observation at each site2-3 observers at each siteEach observer stayed with one provider through an average of two CDS interactions before observing another providerData analyzed using upward abstraction
  • #13: Site 3 had a tool focused on coordination of care between primary and specialty careSite 2 had much on paper-electronic blend—had greatest use of paperSite 4 (PH) had more comments about functionality—was that related to positive or negative comments—comparable in advanced IT development to VA, but smaller system—huge IT research dept colocated with themUsability comments comparable across the sites similar even though very different systems—most were negativeLowest site on usability (RI) had templates, fewest negative commetns on usability probably related to using less for their jobRigidity—computer system, electronic medical record—forced you to do things, computer systemSite 1 and 3 had highest comments about rigidity (VA’s) and centrally controlled development strategyCoordination between specialty and primary care—tool developed as joint effort between primary and specialty care to foster scheduling, intake and provider and specialty and provider. Here coordination between intake and provider removed.Make a list of interpretations—do this again after we create figure from card sort and recommendations.
  • #15: Interesting predominance of comments on technical system—is there an imbalance on technologically driven system, or was that the focus of the observations?Site 1 West HavenSite 2 RISite 3 Columbia VASite 4 PartnersCould this reflect frequency due to additional capturing of codesSite 1 had 2 peopleSite 2 had 3 peopleBig other data so detailed and made a point of recording all data regarding other stuff observed—for example, any time CRC mentioned, coded as other.Revised slide with only observations that were coded end of FTF session…1 page list of most important results and themesCreate graph of that card sort…4 recommendations that fell under the card sort, show proportion by siteMade up of codes under each of those strategies
  • #16: Site 3 had a tool focused on coordination of care between primary and specialty careSite 2 had much on paper-electronic blend—had greatest use of paperSite 4 (PH) had more comments about functionality—was that related to positive or negative comments—comparable in advanced IT development to VA, but smaller system—huge IT research dept colocated with themUsability comments comparable across the sites similar even though very different systems—most were negativeLowest site on usability (RI) had templates, fewest negative commetns on usability probably related to using less for their jobRigidity—computer system, electronic medical record—forced you to do things, computer systemSite 1 and 3 had highest comments about rigidity (VA’s) and centrally controlled development strategyCoordination between specialty and primary care—tool developed as joint effort between primary and specialty care to foster scheduling, intake and provider and specialty and provider. Here coordination between intake and provider removed.Make a list of interpretations—do this again after we create figure from card sort and recommendations.