MEASURING THE BUSINESS IMPACT OF LEARNING: WHAT WE’VE LEARNED
1 like382 views
The document outlines details for a webinar, including audio streaming instructions, access to slides, and certification information. It addresses common questions and outlines the agenda focused on measuring the business impact of learning, along with strategies to overcome barriers in measurement. Key takeaways encourage participants to start small and engage stakeholders to refine measurement processes.
MEASURING THE BUSINESS IMPACT OF LEARNING: WHAT WE’VE LEARNED
- 1. #CLOwebinar The presentation will begin at the top of the hour. A dial in number will not be provided. Listen to today’s webinar using your computer’s speakers or headphones. Welcome to the webinar!
- 2. #CLOwebinar Tools You Can Use Audio Control – A dial in number will not be provided. – The audio will stream through your headphones or computer speakers. – Also check your computer’s volume for external speakers or headsets.
- 4. #CLOwebinar 1. May I receive a copy of the slides? YES! Click on the handouts located on the right portion of your screen. 2. May I review the webinar recording at a later date? YES! You may log in again using today’s link to review the presentation on-demand. 3. Is this webinar HRCI or SHRM certified? YES! The HRCI and SHRM certification codes will be sent in the follow up email. Frequently Asked Questions
- 5. Learning Design Director LEO Learning Measuring the business impact of learning: What we’ve learned Ella Richardson Director of Strategy Watershed Analytics Tim Dickinson Chief Strategy Officer Learning Technologies Group Piers Lea
- 6. Agenda The state of measurement Biggest barriers to measurement Practical steps to overcome barriers Takeaways in a nutshell
- 7. Over 1000 respondents over 3 years
- 8. ? Survey
- 13. ? What are your results?
- 14. Barriers to measurement: Getting Started 1 Getting buy in Getting Data 2 3
- 15. But First: Remember measurement is ongoing & iterative Watershed Method for Learning Analytics Categories of analytics + Complexities of analytics
- 16. Start with what you can control. Barrier 1 I don’t know how to get started.
- 17. Have they learned it? What do people need to learn? Are people doing that? What do people need to do? Did they complete it? What training is required? Was the goal achieved? Business goal Learning Design Measurement Getting Started: Link design to measurement
- 18. How In Practice: Get your team to understand Develop a bespoke measurement model and processes Upskill the trainers to operate as performance consultants Put measurement first in the design and delivery process Building internal measurement capability
- 19. Evaluate Analysis Consult Measurement Process Action Design DevelopImplement Evaluate Analyse Consult • Clarify the business goals • Define the business objectives • Create a business impact action plan • Identify the business data sources • Write the learning objectives • Specify the learning outcomes • Identify the learning data sources • Refine the measurement action plan • Identify whether the required data can be captured • Baseline the data • Design how the data will be displayed • Pilot the learning • Gather the business and learning data • Check the quality of the data and move to full roll-out • Explore the business and learning data for insights • Report back on initial findings and improve training • Develop a deep understanding of the business performance gaps • Create a draft improvement plan • Modify programme / manage learners as required A bespoke measurement process
- 20. Getting Started: Map out your ecosystem Identify your current tools and resources Question your current ecosystem Choose an area to test your new measurement It’s easier than you think 1 2 3
- 21. Getting Started: Test it out
- 24. 8% behavioral change within month of launch
- 25. xAPI cohort ● Learn from peers ● Team-based projects Free LRS ● Essentials LRS ● SCORM cloud ● Learning Locker Getting Started: DIY
- 26. xAPI IPK API LRS Behr built it themselves.
- 27. Outcome There was a 2% sales lift during the four-week period when video and assessment were used as planned.
- 28. Show them what it could look like. Barrier 2 Competing Priorities: How to get budget and time?
- 29. Getting Buy-In: Define a strategy Key components of a measurement strategy No.1 Co-create measurement project No.2 Establish the business drivers No.3 Map out stakeholders No.4 Document chain of evidence No.5 Identify data types and source No.6 Define, develop tech: Integration & tools No.7 Pilot the solution No.8 Document strategy and business case No.9 Define the change programme No.10 Measure impact of measurement strategy
- 30. The chain of evidence Engagement Learning Behaviour Results
- 31. Chain of Evidence Engagement Knowledge, skills & attitudes Performance Behaviors Business Change Has the learning landed? Has learning taken place? Have the performance behaviors changed? Has business performance improved? Learner Change & Performance2 Business impact3Learning Delivery 1 © LEO 2019
- 32. Getting Buy-In: Share the ecosystem vision
- 33. Getting Buy-In: Share the ecosystem vision Read more: bit.ly/2NCbMkk
- 34. Get aligned and don’t be afraid to ask for help. Barrier 3 How do you get access to the data?
- 35. Getting Data: Vendor Alignment Share the vision and how your vendor fits in Build a partnership based on collaboration Validate interoperability before taking out the checkbook
- 36. Getting Data: Vendor Alignment
- 37. Measurement – sources of evidence Engagement • Portal/learning visits and behaviour data (which courses, by whom, for how long) • Star rating • User Surveys • Learner voice • User acceptance testing • Interviews • Annual staff survey • Individual 360 • Survey of departmental satisfaction K, S, A • Upfront diagnostics (including 360) • Post intervention Online assessments (immediate & after 3 months) • Marked work (essays, reports) • Peer assessment • Demonstration • Confidence measure taken after every intervention Performance Behaviours • User surveys • Manager surveys • Interviews (in manageable numbers) • Surveys of coaches and mentors • User observation • Service user feedback (net promoter score) • Activity monitoring software • Use of learning journals Business Change • Business Metrics & KPIs • Profitability, Growth, Productivity etc. • Improved staff retention • Faster turnaround times • Money saved by new approach • Money saved by outcomes
- 38. Getting Data: Ask for help Build alliances within the organisation, early Get the best team round the table
- 39. Key Takeaways Get Started ○ Design for data ○ Start small, test and learn ○ DIY: Learn about it yourself Get Buy in ○ Define a measurement strategy ○ Map out your ecosystem/what’s available to you already Get Data ○ Align vendors and tools for long term success ○ Ask for help
- 40. ? Q&A
- 41. Leolearning.com | watershedlrs.com Access free tools and resources at: bit.ly/2Q6ssSt
- 42. #CLOwebinar Register for the next webinar! Register for all upcoming Chief Learning Officer Webinars at chieflearningofficer.com/webinars