Training Works! Says the Science!

==============

Author's Note:

This article combines ideas from several chapters of my recently published book, The CEO's Guide to Training, eLearning & Work: Empowering Learning for a Competitive Advantage. You can learn more about the book at the book's website (https://www.ceosguide.net/) or on Amazon (https://amzn.to/4674JGS).

=================

Preface

In the book, I write as if I'm writing to a CEO, letting him/her/they know how to manage their learning function to get a competitive advantage. I tell CEOs how they might manage us better and I tell them how they can get the most out of our good work.

The book is not just intended for CEOs and other senior leaders. It's also intended for us, learning and performance professionals, so we can empower ourselves to our full potential.

The book has received advance praise from leaders in the workplace learning field, including by the following people: 𝗖𝗹𝗮𝗿𝗸 𝗤𝘂𝗶𝗻𝗻, 𝗗𝗼𝗻𝗮𝗹𝗱 𝗖𝗹𝗮𝗿𝗸, 𝗘𝗹𝗵𝗮𝗺 𝗔𝗿𝗮𝗯𝗶, 𝗚𝘂𝘆 𝗪𝗮𝗹𝗹𝗮𝗰𝗲, 𝗝𝗲𝗿𝗿𝘆 𝗛𝗮𝗺𝗯𝘂𝗿𝗴, 𝗝𝗼𝘀 𝗔𝗿𝗲𝘁𝘀, 𝗞𝗮𝗿𝗹 𝗞𝗮𝗽𝗽, 𝗠𝗮𝘁𝘁 𝗥𝗶𝗰𝗵𝘁𝗲𝗿, 𝗠𝗲𝗴𝗮𝗻 𝗧𝗼𝗿𝗿𝗲𝗻𝗰𝗲, 𝗠𝗶𝗰𝗵𝗮𝗲𝗹 𝗔𝗹𝗹𝗲𝗻, 𝗠𝗶𝗰𝗵𝗲𝗹𝗹𝗲 𝗢𝗰𝗸𝗲𝗿𝘀, 𝗡𝗶𝗸𝗸𝗶 𝗩𝗮𝘀𝘀𝗮𝗹𝗹𝗼, 𝗥𝗶𝗰𝗵𝗮𝗿𝗱 𝗘. 𝗖𝗹𝗮𝗿𝗸, 𝗥𝗼𝗯 𝗕𝗿𝗶𝗻𝗸𝗲𝗿𝗵𝗼𝗳𝗳, 𝗥𝗼𝘆 𝗣𝗼𝗹𝗹𝗼𝗰𝗸, 𝗥𝘂𝘁𝗵 𝗖𝗹𝗮𝗿𝗸: 𝗦𝘁𝗲𝗹𝗹𝗮 𝗟𝗲𝗲.

The Pointed Questions

Does Training Work? Does it produce positive benefits for organizations, employees, communities, society? Do employees learn? Do they remember? Do they apply what they've learned to their work? Are they successful in changing their behavior? Does the training help employees make a difference?

In addition to all the learning design work I do and all the time I spend teaching others about research-inspired learning design, I also do a ton of learning evaluation. In doing learning-evaluation work, I hear a theme over and over. Chief Learning Officers and other learning leaders ask, "Hey Will, can you help us measure learning so that we can show that our training works?" I say, "Yes! I can help. Learning evaluation is complex; we should talk in depth about it, but tell me, why now? Why do you want to prove your training works?" They give two reasons: (1) they're getting pressure from senior management to show that training is helping the organization, or (2) they are worried that they might get pressure in the future.

What they're asking is whether the training at their organization is working? To answer that question, they need to hire me to help them do an evaluation study.

In this article, let's ask a different question. Let's ask, "Does training work in general?" Has anyone ever demonstrated that training works most of the time? Is there scientific research—not just anecdotal evidence or weakly-researched white papers?

The short answer is yes! Training works! And there's a ton of research on this! I will share many research findings below.

Does Training Create Business Results?

Decade after decade, researchers who study the impact of training find that training is positively related to organizational performance. In a recent meta-analysis—a scientific study of many other scientific studies—the researchers reported the following:

• “0ur findings indicate that investment in training was associated with increased organizational performance. Specifically, a one standard deviation increase in training was associated with 0.25 standard deviation increase in organizational performance. This result indicates that organizations can achieve significant performance gains through investing in training.” (Garavan, et al., 2021).
• Garavan, T., McCarthy, A., Lai, Y., Murphy, K., Sheehan, M., & Carbery, R. (2021). Training and organisational performance: A meta‐analysis of temporal, institutional and organisational context moderators. Human Resource Management Journal, 31(1), 93–119.

While the above quote is from a recent scientific article, a decade ago a separate group of researchers echoed the same findings.

• “Meta-analyses integrating a large number of empirical studies across various training topics from manager training to team training, cross-cultural training, and all forms of employee training consistently show that when training is designed systematically and based on the science of learning and training, it yields positive results.” (Salas, et al., 2012).
• Salas, E., Tannenbaum, S. I., Kraiger, K., & Smith-Jentsch, K. A. (2012). The science of training and development in organizations: What matters in practice. Psychological Science in the Public Interest, 13(2), 74–101.

I’ve got more good news! The research data shows that the learning-and-development field is getting better over time! Decade to decade, my fellow L&D compatriots are producing additional organizational impacts! As the most robust review of research findings concluded:

• “One of our significant and novel findings is that the strength of the relationship between training and organizational performance has increased year on year over the past three decades.” (Garavan, et al., 2021).

Correlation is Not Necessarily Causation

There’s some skepticism deserved for the glowing results reported above. The researchers themselves advocate for better research practices. They decry the large percentage of self-report data. They admit that despite their efforts to temporalize the data—finding training data that comes before results data—the correlational nature of their analyses might leave us wondering what is causing what. Maybe as we hope, training causes improved organizational results. But maybe the opposite is true. Maybe organizations who are doing well are more likely to spend money on training than organizations who are not doing well.

Here is a recent research review that advocates for improved methods for researching the relationship between training and organizational performance.

• Garavan, T., McCarthy, A., Sheehan, M., Lai, Y., Saunders, M. N. K., Clarke, N., Carbery, R., & Shanahan, V. (2019). Measuring the organizational impact of training: The need for greater methodological rigor. Human Resource Development Quarterly, 30(3), 291–309.

Let's dig deeper.

More Research on the Benefits of Training

Here is another recent meta-analysis that demonstrates the positive impact of training. These researchers focused on the benefits of training on attitudes and motivations.

• Uslu, D., Marcus, J., & Kisbu-Sakarya, Y. (2022). Toward optimized effectiveness of employee training programs: A meta-analysis. Journal of Personnel Psychology, 21(2), 49–65.

In addition to training’s impact on organizational performance, researchers who take care in investigating specific types of training have found that training typically creates robust benefits.

In a recent meta-analysis that covered 335 studies, leadership development training led to strong benefits as rated by participants and demonstrated in learning results, behavior change, and organizational results.

• Lacerenza, C. N., Reyes, D. L., Marlow, S. L., Joseph, D. L., & Salas, E. (2017). Leadership training design, delivery, and implementation: A meta-analysis. Journal of Applied Psychology, 102(12), 1686–1718.

Team training—that is, training teams to work together more effectively—has been found to produce robust positive outcomes.

• Hughes, A. M., Gregory, M. E., Joseph, D. L., Sonesh, S. C., Marlow, S. L., Lacerenza, C. N., Benishek, L. E., King, H. B., & Salas, E. (2016). Saving lives: A meta-analysis of team training in healthcare. Journal of Applied Psychology, 101(9), 1266–1304.

In addition to these results, researchers continue to examine factors that promote training improvements. For example, a recent meta-analysis showed that the work context impacts training results, with key factors being the learners’ motivation to transfer, and peer and supervisor support for training application.

• Hughes, A. M., Zajac, S., Woods, A. L., & Salas, E. (2020). The role of work environment in training sustainment: A meta-analysis. Human Factors, 62(1), 166–183.

Researchers have also examined learner motivation to tease out the factors that are most important in creating the benefits that such motivation elicits.

• Chung, S., Zhan, Y., Noe, R. A., & Jiang, K. (2022). Is it time to update and expand training motivation theory? A meta-analytic review of training motivation research in the 21st century. Journal of Applied Psychology, 107(7), 1150–1179.

Researchers have looked at pre-training interventions and found which approaches create more benefits.

• Mesmer-Magnus, J., & Viswesvaran, C. (2010). The role of pre-training interventions in learning: A meta-analysis and integrative review. Human Resource Management Review, 20(4), 261–282.

Researchers have looked at after-action reviews to find out which components are most important in creating AAR’s benefits.

• Keiser, N. L., & Arthur, W. (2022). A meta-analysis of task and training characteristics that contribute to or attenuate the effectiveness of the after-action review (or debrief). Journal of Business and Psychology. Advance online publication.

Researchers have looked at online training to determine what works to keep learners engaged.

• Lee, J., Sanders, T., Antczak, D., Parker, R., Noetel, M., Parker, P., & Lonsdale, C. (2021). Influences on user engagement in online professional learning: A narrative synthesis and meta-analysis. Review of Educational Research, 91(4), 518–576.

The conclusion from these many examples—and these represent only a few of the vast numbers of studies conducted on training and workplace learning—is that (1) there is strong evidence that training works and (2) some specific learning methods work better than other methods.

Research on Specific Learning-Design Factors

There is also a ton of research on learning itself—and the design elements that create the most effective learning. I have been researching learning factors for several decades—I enjoy it—and there is indisputable evidence that some learning factors make a huge difference in terms of learning success. I will detail a few that are especially valuable. They not only produce large learning benefits, but they are easy and practical to use.

Retrieval Practice

Retrieval Practice is one of the most studied phenomena in the learning research and universally touted as one of the most important methods to support people in remembering. There are many recent research reviews on retrieval practice, all of them acknowledging its benefits. In each of these research reviews, dozens and dozens of scientific studies are examined. Here is a list of reviews going back a few decades.

• Agarwal, P. K., Nunes, L. D., & Blunt, J. R. (2021). Retrieval practice consistently benefits student learning: A systematic review of applied research in schools and classrooms. Educational Psychology Review, 33(4), 1409–1453.
• Latimier, A., Peyre, H., & Ramus, F. (2021). A meta-analytic review of the benefit of spacing out retrieval practice episodes on retention. Educational Psychology Review, 33(3), 959–987.
• Roediger, H. L. III, & Karpicke, J. D. (2018). Reflections on the resurgence of interest in the testing effect. Perspectives on Psychological Science, 13(2), 236–241.
• Roediger, H. L. III, & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27.
• Roediger, H. L. III, & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181–210.
• Bjork, R. A. (1988). Retrieval practice and the maintenance of knowledge. In M. M. Gruneberg, P. E. Morris, R. N. Sykes (Eds.), Practical Aspects of Memory: Current Research and Issues, Vol. 1., Memory in Everyday Life (pp. 396-401). NY: Wiley.

In my forthcoming book—CEO's Guide to Training, eLearning & Work: Reshaping Learning into a Competitive Advantage—in the chapter notes for the chapter on retrieval practice, I detail ten different research studies, which show average improvements due to retrieval practice at over 100%. Let me say that again. They used retrieval practice and on average it more than doubled the learning results!! More than doubled the amount people learned!

As I say in the book, be careful! The studies I reviewed were picked randomly and willy-nilly based on the research I had studied. Learning teams who use retrieval practice are likely to see significant improvements in learning, but they may not double their results!

Context Alignment

Context Alignment—known by different names within the learning research—is a fundamental aspect of human cognition. There is no doubt that contextual stimuli trigger thoughts, actions, and emotions. Researchers are still teasing apart some of the subtleties, but in general, learning that is designed to link contextual cues with desired thoughts and actions is likely to produce performance benefits. Using simulations to mimic workplace context is the prototypical way learning designers can utilize context alignment, but they can also use scenarios, case studies, hands-on practice, and other context-informed learning approaches.

There is a ton of great research on context alignment. Here are a few classics plus some more recent research:

• Pan, S. (1926). The influence of context upon learning and recall. Journal of Experimental Psychology, 9(6), 468–491.
• Weiss, W., & Margolis, G. (1954). The effect of context stimuli on learning and retention. Journal of Experimental Psychology, 48, 318-322.
• Smith, S. M., Glenberg, A., and Bjork, R. A. (1978). Environmental context and human memory. Memory and Cognition, 6, 342-353.
• Smith, S. M. (1979). Remembering in and out of context. Journal of Experimental Psychology: Human Learning and Memory, 5, 460-471.
• Godden, D. R., and Baddeley, A. D. (1975). Context dependency in two natural environments: on land and underwater. British Journal of Psychology, 91, 99-104.
• Smith, S. M. (1982). Enhancement of recall using multiple environmental contexts during learning. Memory & Cognition, 10, 405-412.
• Grant, H. M., Bredahl, L. C., Clay, J., Ferrie, J., Groves, J. E., McDorman, T. A., & Dark, V. J. (1998). Context-dependent memory for meaningful material: Information for students. Applied Cognitive Psychology, 12, 617-623.
• Marian, V., & Neisser, E. (2000). Language-dependent recall of autobiographical memories. Journal of Experimental Psychology: General, 129, 361-368.
• Hatala, R., Cook, D. A., Zendejas, B., Hamstra, S. J., & Brydges, R. (2014). Feedback for simulation-based procedural skills training: A meta-analysis and critical narrative synthesis. Advances in Health Sciences Education, 19(2), 251–272.
• Oliveira Silva, G., Oliveira, F. S. e, Coelho, A. S. G., Cavalcante, A. M. R. Z., Vieira, F. V. M., Fonseca, L. M. M., Campbell, S. H., & Aredes, N. D. A. (2022). Effect of simulation on stress, anxiety, and self-confidence in nursing students: Systematic review with meta-analysis and meta-regression. International Journal of Nursing Studies, 133, 1–15.
• Paulus, P. C., Dabas, A., Felber, A., & Benoit, R. G. (2022). Simulation-based learning influences real-life attitudes. Cognition, 227, 1–6.

Context alignment has not demonstrated the astonishing 100% or more improvements that my retrieval-practice review found—nor would we expect it to—but strong results have been found nonetheless. The reason the researchers find weaker effects than we might expect is that they most often focus on incidental background context—like the look and feel of a room—rather than more focal contextual features of the kind learners will pay the most attention.

Spacing Repetitions Over Time

The spacing effect is the fascinating finding that if you repeat concepts over time—repetition is already a powerful learning technique—that more learning is created when the repetitions are spaced farther apart in time than if they are not spaced or are spaced at shorter time spans. A similar practice, called "interleaving," has also been shown to create powerful benefits.

There have been many meta-analyses and research reviews spanning decades that have confirmed the power of spaced repetitions.

• Kim, S. K., & Webb, S. (2022). The effects of spaced practice on second language learning: A meta‐analysis. Language Learning. Advance online publication.
• Maddox, G. B. (2016). Understanding the underlying mechanism of the spacing effect in verbal learning: A case for encoding variability and study-phase retrieval. Journal of Cognitive Psychology, 28(6), 684–706.
• Küpper-Tetzel, C. E. (2014). Understanding the distributed practice effect: Strong effects on weak theoretical grounds. Zeitschrift für Psychologie, 222(2), 71–81.
• Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380.
• Bahrick, H. P., & Hall, L. K. (2005). The importance of retrieval failures to long-term retention: A metacognitive explanation of the spacing effect. Journal of Memory and Language, 52(4), 566–577.
• Dempster, F. N., & Farris, R. (1990). The spacing effect: Research and practice. Journal of Research & Development in Education, 23(2), 97–101.
• Underwood, B. J. (1961). Ten years of massed practice on distributed practice. Psychological Review, 68(4), 229–247.

In 2005, Lynda Hall and Harry Bahrick wrote: "The spacing effect is one of the oldest and best documented phenomena in the history of learning and memory research."

In 2016, after reviewing the latest spacing research, Geoffrey Maddox wrote: “Because of its robustness, the spacing effect has the potential to be applied across a variety of contexts as a way of improving learning and memory.”

Research-Proven Learning Factors

The three learning-research factors I outlined above are just a few of the many factors critical to improving learning results. I picked these three because each of them is particularly potent in supporting long-term remembering and minimizing how much people forget.

Summary — Does Training Work?

Here's what we've seen from the research:

1. Research shows that training is related to organizational performance. Investments in training are related to positive organizational outcomes.
2. If that was our only evidence, it would be fair to have some skepticism. It's difficult to tease out whether training improves performance or successful organizations have more resources to devote to training.
3. Additional research shows that specific types of training produce robust benefits. Benefits were created with online training, team training, leadership training, pre-training interventions, etc.
4. Additional research on specific learning factors shows large improvements in learning when research-aligned learning practices are used (for example, retrieval practice, context-alignment, spacing, feedback, reoetition, guiding attention, etc.).

Can Training Still Be Improved?

Of course training can be improved! Improvements can be made in all complex fields of endeavor.

In their excellent review of the training industry, Eduardo Salas, Scott Tannenbaum, Kurt Kraiger, and Kimberly Smith-Jentsch found the following problems in the way training is designed and delivered:

• “Unfortunately, systematic training needs analysis, including task analysis, is often skipped or replaced by rudimentary questions.” (p. 81)
• “Recent reports suggest that information and demonstrations (i.e., workbooks, lectures, and videos) remain the strategies of choice in industry. And this is a problem [because] we know from the body of research that learning occurs through the practice and feedback components.” (p. 86)
• “It has long been recognized that traditional, stand-up lectures are an inefficient and unengaging strategy for imparting new knowledge and skills.” (p. 86)
• Researchers have “noted that trainee errors are typically avoided in training, but because errors often occur on the job, there is value in training people to cope with errors both strategically and on an emotional level.” (p. 86)
• “Historically, organizations and training researchers have relied on Kirkpatrick’s [4-Level] hierarchy as a framework for evaluating training programs…[Unfortunately,] The Kirkpatrick framework has a number of theoretical and practical shortcomings. [It] is antithetical to nearly 40 years of research on human learning, leads to a checklist approach to evaluation (e.g., ‘we are measuring Levels 1 and 2, so we need to measure Level 3’), and, by ignoring the actual purpose for evaluation, risks providing no information of value to stakeholders…" (p. 91)

There are other well-known issues in the learning-and-development field.

• Sometimes training isn't needed, but we train anyway.
• Sometimes we train people well before they'll need the information they learn, and they, as expected, forget much of the information.
• Sometimes there are better alternatives—besides training—to improve work performance, but we don't use them. For example, sometimes people just need a simple job aid or checklist, or the ability to look up information or watch a video, or be supported by an intelligent tool designed to assist performance.
• Often training is viewed as able to maximize work performance on its own, as if no other efforts or supports are needed.
• Our learning evaluation practices are too often ineffective.
• Often organizational stakeholders outside of learning-and-development don't know how best to work with L&D teams to maximize work performance.
• Too often myths and misconceptions guide learning design and other learning decisions and thus create inadequate results.
• Too often learning teams are under-resourced and are not allowed the time it takes to create the most effective learning interventions.

Final Conclusions

1. Training generally creates positive benefits.
2. Many training programs could be made much more effective.
3. Leverage points for improvement include:

• More use of research-aligned learning methods.
• An improved process for determining needs and solutions.
• More and better analysis of the work-performance context.
• More and better exploration of non-training supports.
• Improved practices for evaluating learning.
• More investment in professional development for learning professionals.
• Periodic outside reviews of learning designs and practices.

How To Learn More

THE BOOK. You can learn more about the book at the book's website (https://www.ceosguide.net/) or on Amazon (https://amzn.to/4674JGS).

LTEM. (The Learning-Transfer Evaluation Model) is rapidly replacing older evaluation models. I invite you to join me in the LTEM Boot Camp open-enrollment workshop or contact me to arrange a private Boot Camp for your organization. LTEM Boot Camp LEARN MORE.

My Website. To access my research-to-practice reports, my blog, job aids, and get an introduction to my consulting services. WorkLearning.com/.

Coaching. I'm available as a coach, and offer a pay-what-you-can pricing option. Check out my coaching options.

Book Chapters Available on LinkedIn:

• Training Works! Says the Science.
• Learning Data is Often Crap Data.
• Learning Strategy for Learning Teams.
• Full-Factor Learning-Request Process.
• Separating Good Research from Bad in Workplace Learning.
• Your Experts Aren't Always Great at Teaching.
• Your Learners Don’t Always Know Learning.
• Why I Wrote the Book (not a chapter).

Other Articles Available on LinkedIn:

• LTEM Five Years Later.
• The Danger of Creating Learner Surveys Alone.
• Your CEO is an AI and has Advice for You as a Learning Professional.

Sign Up For This Newsletter

You can sign up for this LinkedIn newsletter if you like. There's a link somewhere here where you can do that.