Evaluating opportunities to optimize learning and economic impact: Applying system dynamics to model training deployment in a medical center
- 1. Evaluating opportunities to optimize learning and economic impact: Applying system dynamics to model training deployment in a medical center Daniel McLinden, EdD Assistant Vice President Education & Learning Department Cincinnati Children’s Hospital Medical Center Assistant Professor Department of Pediatrics University of Cincinnati College of Medicine Stacey Farber, PhD Manager, Education Research and Measurement Education & Learning Department Cincinnati Children’s Hospital Medical Center
- 2. Our Vision: To be the leader in improving child health Historical Highlights • Established in 1883 • Development of the Sabin oral polio vaccine • Creation of the first heart-lung machine • Rotavirus vaccine developed and tested by Children's researcher National Recognition • U.S. News & World Report (2008) ranked Cincinnati Children’s Hospital Medical Center (CCHMC) the third best children’s hospital in the United States • Child magazine named CCHMC one of the top five best children's hospital in the United States by. In addition, cancer care and neonatal care at Cincinnati Children's were named among the top five subspecialty programs in the United States. • Recipient of the 2006 American Hospital Association-McKesson Quest for Quality Prize™ honoring leadership and innovation in quality, safety and commitment to patient care Full-service, not-for- profit pediatric academic medical center with 475 registered inpatient beds and 36 registered residential psychiatry beds
- 3. • 10,000+ person organization • 1000+ new hires each year • Multiple training requirements driven by external agencies • Risk associated with training compliance • Cost association with training compliance The situation
- 4. Engage Stakeholders to make explicit the features, assumptions and outcomes. Improved employee morale will make this a better place to work but will not affect our business either positively or negatively. The bottom line will improve if we improve employee morale and this program is just what we need to increase morale.
- 5. A logic model is… • A tool for telling your program’s story – McLaughlin, J.A. & Jordan, G.B. (1999). Logic models: A tool for telling your program's performance story. Evaluation and Program Planning, 22, 65-72. • A connection of theory with implementation and outcomes – Frechtling, J.A. (2007). Logic Modeling in Program Evaluation, San Francisco: Jossey Bass. • A tool to enable collaborative planning for program impact – Kraigier, K., McLinden, D., & Casper, W.J. (2004). Collaborative planning for training impact. Human Resource Management, 43(4), 337-351.
- 6. Required Training – Logic model Newly hired staff Scheduling of programs OSHA requirements met Requirements for Institutional initiatives met Instructors for classroom programs INPUTS OUTCOMES Joint Commision requirements met Organization is compliant with requirements, Personnel are aware of norms and expectations of them. Etc. OUTPUTS Completion of required training Context • Participants are required to locate course in Learning Management System • Participants are required to manage their compliance • Participants can enroll at any time during and after the required timing New employees are aware of 30 day requirements New employees are aware of 365 day requirements
- 7. What are the Economics: A Spreadsheet model CARES Orientation Discrimination and Harassment Awareness in the Workplace Collaborative IRB Training Initiative (CITI) Diversity Orientation (Step 1) Diversity Awareness (Step 2) Diversity for Managers HIPAA Training New Employee Orientation (NEO) New Employee Safety Test NIMS for NEO NIMS HICS 100 2007 Patient Safety Education Safety College Etc. What is the cost of participant time Training time Number of training participants
- 8. What are the economics: A system model • System models simulate change over time in a system. • Incorporate nonlinearities, feedback, and delays. • Define the linkages between cause and effect to test assumptions. • Discover how to adjust cause and effect relationships to improve outcomes • Quickly simulate multiple scenarios to test new assumptions • Offer and experiential way to share the logic
- 9. Steps in the modeling process* *Excerpted from Sterman, J.D. (2000). Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston: Irwin McGraw-Hill, (p.86). 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation
- 10. What are the implications of the current approach to required training 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation 1,000+ • Multiple program requirements • Multiple time requirements • Self selection
- 11. What are the implications of the current approach to required training Start End of year one 1,000+ Self selection 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation
- 12. Change the model of required training 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation Start End of year one 1,000+ Orient Onboard Optimize
- 13. Cost is constant (almost) Productivity is variable Start End of year one 1,000+ Orient Onboard Optimize $ $ $ Cost 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation
- 14. 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation Assume time to competence is not linear
- 15. 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation Orient Onboard Optimize New employees transit through the growth curve
- 16. 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation Orient Onboard Optimize Value is a function of productivity relative to cost $
- 17. Stocks and Flows 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation 1,000+ Orient Onboard Optimize 100 1000 + Month 1 Months 2 -10 Months 11-12
- 18. Convert time and people to cost 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation 1,000+ Orient Onboard Optimize 100 1000 + Month 1 Months 2 -10 Months 11-12 Cost of time Transit time
- 19. Accumulate costs over time 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation 1,000+ Orient Onboard Optimize 100 1000 + Month 1 Months 2 -10 Months 11-12 Cost of time Total compensation
- 20. Accumulate productive value of time 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation 1,000+ Orient Onboard Optimize 100 1000 + Month 1 Months 2 -10 Months 11-12 Cost of time Total compensation Productive Value of time
- 21. Accumulate productive value of time 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation 1,000+ Orient Onboard Optimize 100 1000 + Month 1 Months 2 -10 Months 11-12 Cost of time Total compensation Productive Value of time Training & Development accessing productive time Orient Onboard Optimize Training & Development allocated across year 1.
- 22. 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation week one productiv e hour s deploy ed month 2 to 10 deploy ingwork for ce ~ growth ofc ompetenc y ov er time total hour s compens ated tr ansitioningto month 11 hours per per son per week hours per per son per week deploy ed month 1 total time compens ated acc umulating compens ated time c ounting time acc umulating value ofpr oduc tiv e time tr ansitioningpeopletomonth 2 productiv e valueoftime final weeks of year fully competent tottr ain hr mo 1 Year one tr aininghours tottr ain hr mo 2to10 Training Hour s per per sonweek 1 Training Hour s per per son week 2through 4 Training hour s per pers on month 2thr ough 10 Training hour s per pers on months 11 and12 productiv ity s lac k training productiv ity s lac k tottr ain hr finalweeks productiv e hour s tottr ain hrs week 1 total cost of training as func tionofproductivity total hour s compens ated acc umulating oppty c os toftraining tottr ain hrs
- 23. 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation Training Hours per person week 1 40 Training Hours per person week 2 through 4 40 Training hours per person month 2 through 10 4 Training hours per person months 11 and 12 4 U Timing and Hours of training 88.0 95.0 154,045 208,000.0 52 Total hours compensated Total producitive hours Cost: Total training hours taken during productive time Enter hours in the table. Press "run... Press "reset..." to clear all values. Year one total training hours per person Week Runthe simulation Reset all values Close application
- 24. • New Personnel become more productive over time and this can be modeled in three phases: – Orienting new hires – Onboarding recent hires – Optimizing the value of experienced hires • Assumes there is SLACK in the time of the person who is not productive and training and other development is using the slack • Training delivered in the OPTIMIZE phase should be high value and enhance productivity. • Training delivered to meet organizational requirements is not a best use of optimize phase 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation Findings
- 25. Policy Implications • Implications for policy: Engaging stakeholders – Test assumptions. – Adjust cause and effect relationships to improve outcomes – Quickly simulate multiple scenarios to test new assumptions – An experiential way to share the logic. • Change the practice – Front load training – Take control of completion of required training – Re-think the timing of requirements 2. Dynamic Hypothesis 1. Problem Definition 3. Formulation of model 4. Testing 5. Policy Formulation