Does your learning solution solve business performance problems?
Download as PPTX, PDF1 like682 views
This document discusses how to better measure and demonstrate the value of training programs. It suggests moving beyond simply measuring inputs and reactions to training, and instead focusing on outcomes like learning, application of skills on the job, impact on business metrics, and financial return on investment. The document provides examples of different levels of evaluation for training programs from reaction to ROI. It emphasizes the importance of aligning training objectives and evaluation with organizational needs and goals to demonstrate how training contributes to business performance.
Does your learning solution solve business performance problems?
- 1. Does learning solve business performance problems? By Javier Amaro
- 2. What is value? • Think about a training program that you recently implemented. • How would you define the value delivered by the program?
- 3. What is the value of training?
- 4. New definition of value Be balanced, with qualitative and quantitative data Contain financial and nonfinancial perspectives Reflect strategic and tactical issues Satisfy all key stakeholders Be consistent in collection and analysis Be grounded in conservative standards Come from credible sources
- 5. Do we have a place in the table?
- 6. Do we have a place in the table? • Do training and development professionals participate in business decisions? • Do training and development professionals participate in government strategic decisions?
- 7. Shifting Paradigms Activity Based Results Based No business need for the program Program linked to specific business needs No assessment of performance issues Assessment of performance effectiveness No specific measurable objectives Specific objectives for behavior and business impact No effort to prepare program participants to achieve results Results expectations communicated to participants No effort to prepare the work environment to support transfer Environment prepared to support transfer No efforts to build partnerships with key managers Partnerships established with key managers and clients No measurement of results or benefit-cost analysis Measurement of results and benefit-cost analysis Planning and reporting is input focused Planning and reporting is outcome focused
- 9. Become a business partner
- 10. Levels of Evaluation 10 Levels of Evaluation Measurement Focus 0. Inputs and Indicators The input into the project in terms of scope, volume, efficiencies, costs Participants, Hours, Costs, Timing 1. Reaction & Perceived Value Reaction to the project or program, including the perceived value Relevance, Importance, Usefulness, Appropriateness, Intent to use, Motivation to take action 2. Learning Learning to use the content and materials, including the confidence to use what was learned Skills, Knowledge, Capacity, Competencies, Confidence, Contacts 3. Application & Implementation Use of content and materials in the work environment, including progress with actual items and implementation Extent of use, Task completion, Frequency of use, Actions completed, Success with use, Barriers to use, Enablers to use 4. Business Impact The consequences of the use of the content and materials expressed as business impact measures Productivity, Revenue, Quality, Time, Efficiency, Customer Satisfaction, Employee Engagement Typical Measures 5. ROI Comparison of monetary benefits from program to program costs Benefit-Cost Ratio (BCR), ROI%, Payback Period
- 11. Executive view – the report Measure We currently measure this We should measure this in the future My ranking of the importance of this measure 1. Inputs: “Last year, 78,000 employees received formal learning.” 94% 2. Efficiency: “Formal learning costs $2.15 per hour of learning consumed.” 78% 3. Reaction: “Employees rated our training very high, averaging 4.2 out of 5.” 53% 4. Learning: “92% of participants increased knowledge and skills” 32% 5. Application: “At least 78% of employees are using the skills on the job.” 11% 6. Impact: “Our programs are driving our top 5 business measures in the organizations.” 8% 7. ROI: “Five ROI studies were conducted on major programs yielding an average of 68% ROI.” 4% 8. Awards: “Our learning and development program won an award from American Society of Training and Development.” 40%
- 12. Executive view – the need Measure We currently measure this We should measure this in the future My ranking of the importance of this measure 1. Inputs: “Last year, 78,000 employees received formal learning.” 94% 85% 2. Efficiency: “Formal learning costs $2.15 per hour of learning consumed.” 78% 82% 3. Reaction: “Employees rated our training very high, averaging 4.2 out of 5.” 53% 22% 4. Learning: “92% of participants increased knowledge and skills” 32% 28% 5. Application: “At least 78% of employees are using the skills on the job.” 11% 61% 6. Impact: “Our programs are driving our top 5 business measures in the organizations.” 8% 96% 7. ROI: “Five ROI studies were conducted on major programs yielding an average of 68% ROI.” 4% 74% 8. Awards: “Our learning and development program won an award from American Society of Training and Development.” 40% 44%
- 13. Executive view – the gap Measure We currently measure this We should measure this in the future My ranking of the importance of this measure 1. Inputs: “Last year, 78,000 employees received formal learning.” 94% 86% 6 2. Efficiency: “Formal learning costs $2.15 per hour of learning consumed.” 78% 82% 7 3. Reaction: “Employees rated our training very high, averaging 4.2 out of 5.” 53% 22% 8 4. Learning: “92% of participants increased knowledge and skills” 32% 28% 5 5. Application: “At least 78% of employees are using the skills on the job.” 11% 61% 4 6. Impact: “Our programs are driving our top 5 business measures in the organizations.” 8% 96% 1 7. ROI: “Five ROI studies were conducted on major programs yielding an average of 68% ROI.” 4% 74% 2 8. Awards: “Our learning and development program won an award from American Society of Training and Development.” 40% 44% 3
- 14. Fear of negative ROI
- 15. Chain of Impact Reaction & Planned Action Learning Application & Implementation Isolate the Effects of the Program Intangible Benefits Impact
- 16. Program Alignment: The V Model
- 17. Matching evaluation levels Improve work group productivity by 20% in 6 months 4 Initiate at least three cost reduction projects in 15 days 3 Achieve an average cost reduction of $20,000 per project 4 Use counseling skills in 90% of situations where work habits are unacceptable 3 Be able to describe the four elements of the employee assistance program 2 Provide a 4 out of 5 rating on relevance of course content 1 Achieve a 2:1 benefit-to-cost ratio one year after the new leadership program is implemented 5 For each objective listed below, indicate the level of evaluation at which the objective is aimed. 1. Reaction and planned action 2. Learning 3. Application 4. Impact 5. Return on Investment (ROI)
- 18. Program Alignment V Model Decrease absenteeism by 5%
- 19. Key Alignment Questions Potential ROI ROI Payoffs Objectives5 5 Is this a problem worth solving? Is there a potential pay off? What is the actual ROI? What is the BCR? Needs Program Assessment Objectives Evaluation
- 20. Needs Program Assessment Objectives Evaluation Business Impact Business Needs Objectives Impact Key Alignment Questions 4 4 What is the specific measure? What happens if we do nothing? Which business measure improved? How much is related to the program?
- 21. Needs Program Assessment Objectives Evaluation Job Performance Application Application Needs Objectives Key Alignment Questions 3 3 What is occurring or not occurring on the job that influences the business measure? What has changed? Which skills/knowledge has been applied?
- 22. Skills/Knowledge Learning Learning Needs Objectives Needs Program Assessment Objectives Evaluation Key Alignment Questions 2 2 What skills or knowledge is needed to support the job performance need? What did they learn? What are the takeaways?
- 23. Preferences Reaction Reaction & Objectives Planned Action Needs Program Assessment Objectives Evaluation Key Alignment Questions 1 1 How should the solution be structured? What was the reaction to the program? How will the program be implemented?
- 24. Business Alignment process 1. Determine payoff needs 2. Identify business needs 3. Identify performance needs 4. Identify learning needs 5. Set Objectives 6. Design for results 7. Measure the business impact 8. Isolate the effects of the program
- 25. Training with the end in mind • What is the value of your training? – ROI – Impact – Application – Learning – Reaction – Input • Report the whole picture
- 26. Build up Capabilities in ROI • ROI Certification program – Sydney from 7th to 11th December 2015 – http://insources.com.au/index.php/roi/roi- certification
- 27. Feedback Counts! Your feedback helps ATD 2015 APC continue to provide top-notch educational programs that help you stay on top of a changing profession. Evaluation forms for this session are available via the mobile app and at the following link: https://zh.surveymonkey.com/r/ATD2015APC