Learning's Big Data Problem: Measuring & Analyzing Impact
Download as PPTX, PDF2 likes1,297 views
The document discusses the challenges and advancements in measuring learning impact through big data, emphasizing the importance of moving beyond traditional formal learning methods. It highlights the growing trend of xAPI (Experience API) adoption as a means to connect various learning technologies and improve organizational learning effectiveness. Key takeaways stress that learning should be integrated into daily work processes and that the right tools, like Learning Record Stores (LRS), are essential for tracking and analyzing learning outcomes.
Learning's Big Data Problem: Measuring & Analyzing Impact
- 1. Research Spotlight Learning’s Big Data Problem: Measuring & Analyzing Impact David Wentworth, Principal Analyst, Brandon Hall Group @DavidMWentworth Mike Rustici, CEO, Watershed @mikerustici Michael Rochelle, Chief Strategy Officer Brandon Hall Group
- 2. HCM Research and Advisory Firm © Brandon Hall Group 2016 2 FOUNDED IN 1993 Community of 300,000+ 10,000 Global Clients Leading Independent HCM Research & Analyst Firm Publishing 2+ Pieces of Research Each Day Sample of Our Clients
- 3. How We Help You © Brandon Hall Group 2016 3 MEMBERSHIP STRATEGIC CONSULTING EXCELLENCE AWARDS PROFESSIONAL DEVELOPMENT Learning & Development Talent Management Leadership Development Talent Acquisition Workforce Management
- 4. Open Surveys © Brandon Hall Group 2017 4 Visit www.brandonhall.com click Open Surveys If you would like to join a panel of survey takers, please contact us at success@brandonhall.com • Recruitment Marketing Practices 2017 • Extended Enterprise Learning 2017 • State of Talent Acquisition 2017 • HCM Technology 2017
- 5. How To Ask Questions © Brandon Hall Group 2016 5 • Submit your questions or comments about the discussion to our presenters using the Questions tab on your control panel. • Presentation slides and giveaways can be found in the Handouts tab on your control panel. Recording & Slides will also be sent out after the webinar.
- 6. Agenda The State of Learning Measurement Moving and Measuring Beyond Formal How Do We Get Better? The Role of Learning Standards
- 7. 7 23% 27% 38% 40% 53% 66% 78% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% To more strongly link learning and reduced turnover To provide input to the performance management process To determine the ROI of learning programs To more strongly link learning and employee engagement To more strongly link learning and individual performance To more strongly link learning and organizational performance To improve the effectiveness of learning programs Drivers of Learning Measurement
- 8. 14% 30% 37% 19% Formal 8 51% 30% 17% 2% Informal 43% 32% 16% 7% Experiential Not at all Somewhat Moderately Highly Effectiveness of Learning Measurement
- 9. • Formal Learning/Catalog • Certification & Compliance • Courses • Groups Associated with Classes • Impressions • Expertise Location • Idea Engine • Real-Time Collaboration • Groups & Workspaces 9 Very Little Learning is Formal
- 10. The Blending Effect 4.9% 29.4% 46.1% 19.6% Not at all Small extent Moderate extent Great extent Improved Link to Individual Performance 9.1% 37.4% 37.4% 16.2% Not at all Small extent Moderate extent Great extent Improved Link to Organizational Performance 95% 91% Source: Brandon Hall Group HCM Outlook 2017
- 11. How Do We Get Better? © Brandon Hall Group 2016 11
- 12. “L&D is responsible for improving our most important asset, our people. If we can improve L&D, we can improve the world” me, I said that
- 13. We can do better.
- 15. Why connect with xAPI? To transfer learning records Example: when you get a new learning platform For products with different functions to work together Example: your LMS and a social learning platform To back up learning records Example: in case your LMS breaks down xAPI is a technical specification that makes it easier and cheaper to connect learning technologies.
- 16. How far has xAPI adoption come?
- 18. Standards define and confine an industry.
- 19. New model Old model eLearning LMS 2000 SCORM eLearning LMS Mobile Social Informal Simulations Games MOOCs ILT Adaptive Personalized LRS Learning Experience Platform Many more…… 2013 xAPI
- 20. xAPI deployments pilots 2015 – 16 prototypes 2013 – 14 production 2017 –
- 21. Why do I have to go beyond CMI- 5?
- 22. SCORM: Not evolving, but not dead yet.
- 23. What’s a Learning Record Store?
- 24. Three types of LRS An Embedded LRS is typically part of another system. It often adds xAPI functionality to an LMS or a sophisticated learning experience provider. (Example: Rustici Software’s Engine) 01 Embedded LRS The main role of this LRS aligns with the technical definition from the xAPI spec; it receives, stores, and provides access to learning records via xAPI. These systems are primarily technical tools with an interface designed for technical people. (Example: Learning Locker) 02 Data Storage LRS A Learning Analytics Platform (LAP) is a tool that contains an LRS, but layers a rich reporting and analytics toolset on top of it. It’s designed for learning professionals and facilitates robust analysis of all the data collected by xAPI. (Example: Watershed) 03 Learning Analytics Platform
- 25. Tips for choosing an LRS 01 Be on the same page. Make sure everyone involved in the process shares a common definition for “learning record store” and an understanding of the type of LRS you need. 04 Verify it meets expectations. Ask what you can do with the data in your LRS. Many legacy LMS vendors have implemented an embedded LRS to say they have xAPI, but, most don’t have out-of-the-box ways to access that data for reporting once it’s in the system. 02 Ensure its xAPI compliant. ADL hasn’t released an xAPI certification program, but there is a test suite coming soon that you can run against any LRS to see how it stacks up. 05 Ensure unfettered access to data. One of the core tenants of xAPI is that data should be freely portable across systems. Make sure your LRS vendor will allow you to retrieve all the data you store in your system. 03 Join the xAPI community. The xAPI community is friendly and eager to help. Get involved in discussions and ask more about each provider. 06 Speak with client references. Make sure your vendor has deployed its product at a scale and scope similar to your situation and speak to an actual client to find out how the process and deployment went.
- 26. Key Takeaways • Learning does not just exist within the confines of the LMS. • If we want learning to impact behavior and performance, we have to bring it closer to the point of need. • Don’t let fear of measurement and tracking prevent you from getting started. • xAPI is a standard that can help pull all the pieces of the puzzle together. • AN LRS is the tool that lets you see the whole puzzle once it’s together.
- 27. Questions? Submit your questions to the presenters via the questions tab on your control panel 27 Type your questions here
- 28. Thank you for joining us today! If you have any additional questions, please email us at success@brandonhall.com.
- 29. Learn More About Us Products and services that we offer © Brandon Hall Group 2016 29
- 30. Membership Offers Tailored Support RESEARCH ACCESS & EVENTS • Reports • Case Studies, Frameworks & Tools • DataNow® & TotalTech® • Webinars and Research Spotlights • Annual HCM Conference ADVISORY SUPPORT • Ask the Expert • 1 on 1 Consultations • Research Briefings • Benchmarking CLIENT SUCCESS PLAN • Your Priorities • Executive Sponsor • Client Associate • Monthly Meetings © Brandon Hall Group 2016 30 To learn more about becoming a member or to sign up for a free trial, please email us at success@brandonhall.com
- 31. Strategic Consulting Offers Expert Solution Development BENCHMARKING • Competitive/Comparative • Maturity Model • Custom Research STRATEGY • Business Case • Planning • Organization & Governance TECHNOLOGY SELECTION • Vendor Selection • Architecture Design • Systems Evaluation DEVELOPMENT & INTEGRATION • Program Design • Assessment • Survey • Process Integration © Brandon Hall Group 2016 31 To learn more about consulting, please email us at success@brandonhall.com
- 32. Why Brandon Hall Group © Brandon Hall Group 2016 32 1 2 3 4 5 We offer flexible, high-value memberships that offer everything you need at a cost you can afford. Our research is better – more targeted, easily digestible, focused on business results, predictive, prescriptive. Our solutions gets better results – research-driven, focused on the business, simple, targeted, flexible approach. Our case studies are unique for their diversity and focus on business results. Our professional development is practical, tailored to your needs, and equips you to take action.
- 33. We Do This Through… Research and Analysis that Define New HCM Standards Success Stories from Global Organizations that Show those Standards in Action Professional Development that Assesses Current State and Enables Innovation Flexible Services to Design and Implement Transformative Solutions Recognition that Validates the Transformation © Brandon Hall Group 2016 33
- 34. What Our Clients Have to Say Brandon Hall Group’s Excellence Awards allows you to take a deeper dive and evaluate the programs that you’ve created at a level that you might not have the opportunity to do… They have always been a cornerstone for me. - Katrina Williams, Director, Sales Capability CDW https://youtu.be/OgdA_tpM55U © Brandon Hall Group 2016 34
- 35. What Our Clients Have to Say I describe the attention and that customer service that I’ve experienced as being excellent. They are very attentive and wherever I need to be connected to whomever, they are there to help me. - Joanne Veech, Global Talent Leader PwC https://youtu.be/YkY-v8gfW5k © Brandon Hall Group 2016 35
- 36. HCMx Radio © Brandon Hall Group 2016 36 THE ONLY PODCAST IN THE HCM ARENA THAT WEAVES CURRENT MARKET RESEARCH, HR TECHNOLOGY AND INDUSTRY LEADERS INTO EACH EPISODE To catch up on the latest episodes, visit www.brandonhall.com/hcmxradio or subscribe on iTunes