Strategies for Implementing Personalized Learning While Evidence and Resources Are Underdeveloped

Perspective C O R P O R AT I O N Expert insights on a timely policy issue October 2018 Strategies for Implementing Personalized Learning While Evidence and Resources Are Underdeveloped John F. Pane I nnovators are exploring new designs for the primary and second- guidance draws on theory, basic principles of learning science, and ary education system under the umbrella of personalized learning. the limited research that does exist on personalized learning and its The overarching set of practices being explored in the space is component parts. quite broad, but consensus is lacking on a precise definition of I begin this Perspective with a high-level definition of per- personalized learning or on which component practices are essen- sonalized learning and a description of the opportunities it has to tial. Practitioners and policymakers seeking to implement person- address long-standing challenges in the U.S. education system. I alized learning, lacking clearly defined evidence-based models to then summarize the limited evidence on efficacy, along with some adopt, are creating custom designs for their specific contexts. Those of the implementation challenges identified by the early research. I who want to use rigorous research evidence to guide their designs offer a set of principles that implementers might use to guide their will find many gaps and will be left with important unanswered designs in the absence of proven-effective models. Finally, I illus- questions about which practices or combinations of practices are trate the application of these principles to mastery-based skill devel- effective. It will likely take many years of research to fill these gaps. opment, which is often present in personalized learning models. Despite the lack of evidence, there is considerable enthusiasm about personalized learning among practitioners and policymakers, What Is Personalized Learning? and implementation is spreading. The purpose of this Perspective It is increasingly common for schools serving Kindergarten is to offer strategic guidance for designers of personalized learning through 12th grade (K–12) in the United States to experiment with programs to consider while the evidence base is catching up. This innovative schoolwide strategies that are intended to provide more- customized educational experiences for every student. The specifics ing materials—including curricula, assessments, and technological of personalized learning vary from school to school, incorporating supports—being deployed in these schools. There is not widespread such strategies as: public agreement on this or any other definition of personalized • enabling students to work on content targeted to their individ- learning; some stakeholders define it more narrowly or use different ual levels of achievement rather than synchronized with peers terminology, such as student-centered learning. In this Perspective, I at the same grade level (often referred to as a mastery-based or use the term model to refer to the specific set of personalized learn- competency-based approach) ing strategies and materials adopted by a school or set of schools. • finding ways to make learning activities more relevant to With such a broad and inclusive definition of personalized learn- students and allowing them to be more involved in setting ing, any specific model will likely include only a subset of the many educational goals and in selecting the materials and activities available strategies. to achieve those goals • placing greater emphasis on developing social-emotional skills, Opportunities Addressed by Personalized which, along with academic achievement, are important for Learning postsecondary success Many of the goals and strategies employed for personalized learn- • nurturing stronger relationships with students and their fami- ing are not new. There is a long history of educators striving to lies and learning more about social, family, medical, or other meet students’ individual needs and incorporating their interests situations that might be relevant to students’ performance in and preferences into instruction. These efforts include developing school. individualized education plans for students with special needs, using data to help make instructional decisions for individual stu- In sum, the goal of personalized learning is to make each dents, providing instruction to individual students or small groups student’s educational experience responsive to his or her talents, of students, providing tutors or support teachers to supplement interests, and needs. In this Perspective, I use personalized learning the classroom teacher’s instruction, and offering diverse elective as a broadly inclusive term for the strategies, practices, and support- courses. Personalized learning can be viewed as a comprehensive schoolwide integration and intensification of these ideas across all grades and subject areas. It has become more feasible recently The goal of personalized learning is to through the availability of technological supports. Specifically, make each student’s educational experience technology has progressed to the point that it can help educators responsive to his or her talents, interests, and orchestrate the complex process of tracking individual students’ learning plans and progress, and it can provide a rich variety of needs. 2 instructional opportunities to students so that teachers can devote age this kind of flexibility to accommodate its increased focus on more of their time to working with individuals or small groups. developing social-emotional skills. The opportunity afforded by technology is emerging in the context of widespread dissatisfaction with how the U.S. K–12 edu- Challenges to Implementation cation system currently performs. The United States ranks lower It is no surprise that the widespread changes aspired to in personal- than desired in such international comparisons as the Organisation ized learning will pose challenges. At the micro level, to continue for Economic Co-operation and Development’s Programme for with the earlier example, a finer granularity of course-taking 1 International Student Assessment. U.S. proficiency rates on the departs from the whole-course criteria currently used to define 2 National Assessment of Educational Progress are too low, dropout the requirements of a high school diploma or college admission. 3 rates are too high, and growing achievement gaps between the Students who took parts of calculus, statistics, and accounting highest-performing and lowest-performing students suggest that will have completed none of those courses. How will they docu- there are substantial inequities in educational opportunity along ment what they accomplished? This will require developing a new 4 the lines of race or ethnicity or socioeconomic status. These prob- consensus for how students demonstrate that they have completed lems have stubbornly persisted despite extensive efforts to address sufficient work, as well as how their accomplishments are commu- them over many decades, and notwithstanding occasional minor nicated to parents, colleges, employers, and others. improvements on some metrics, such as elementary school profi- More importantly, macro-level implementation challenges ciency rates or high school dropout rates. It is possible that making could render personalized learning ineffective for improving stu- more-substantial advancements on these problems requires major dent outcomes. Particular risks at this early stage of development changes in how schools operate. Schoolwide personalized learning include the following: aspires to changes of this magnitude. • Curriculum materials, assessments, and technologies to Personalized learning can also allow for finer granularity in support personalized learning are relatively immature, frag- topic coverage and more-flexible pathways for student success. mented, and of uneven quality, requiring educators to spend Where a student would traditionally take a course in calculus, sta- valuable time and effort assembling the necessary supporting tistics, or accounting, a mastery-based system could allow the stu- materials and making them work together. dent to learn selected parts of each subject, tailored to the student’s • Educators, lacking models that have been proven effective, interests or to meet the demands of their intended career path. might adopt ineffective practices as they attempt to figure out Such an approach can accommodate new topics that gain relevance what personalized learning will look like in their classrooms without overloading the curriculum, helping to avoid such perverse and schools. responses as fitting in computer science by considering it a foreign • Schools—especially those that operate within traditional 5 language. Indeed, personalized learning itself might need to lever- district structures—might struggle as their new personal- 3  ized instruction conflicts with long-standing district or state samples or sufficient data to provide insights into which specific policies. components of personalized learning might be most effective. Despite the positive results on average, the study revealed some Some skeptics argue that these are among the reasons that reasons for concern. Estimated effects for individual schools varied personalized learning will be another in a long series of educational widely, including negative effects for some of the personalized reforms that do not lead to meaningful improvements on the key learning schools. In mathematics, schools were estimated to pro- 6 problems faced by the U.S. education system. duce significant effects ranging from +14 to –17 percentile points. The major challenges that accompany the opportunities of In reading, the range was +16 to –19 percentile points. In both personalized learning drive the central recommendation of this subjects, seven schools (nearly a quarter of the sample) had negative Perspective: to take a strategic approach to personalized learn- effects strong enough to be statistically significant. ing implementation, focusing first on components most likely to If personalized learning were responsible for the modestly be effective while waiting for policies, materials, and evidence to positive average effects, it is clear that it did not work equally well evolve in ways that support more-expansive implementation. Before in every school. There are many possible explanations for this elaborating on this recommendation, I briefly review the existing disparity, including the specific models that each school designed evidence on schoolwide personalized learning. or implementation challenges that we identified (such as intensive demands on teachers’ time, difficulties integrating data from mul- The Research Evidence on Personalized Learning tiple technology systems, and tensions between personalized learn- Thus far, the research evidence on personalized learning as an over- ing practices and state or local policies). Many of the same chal- arching schoolwide model is sparse. A team of RAND Corporation lenges were echoed in a more recent study of personalized learning researchers conducted the largest and most-rigorous studies of stu- implementation conducted by the Center for Reinventing Public 7 dent achievement effects to date. We examined groups of schools Education (which did not examine achievement effects).9 implementing diverse schoolwide models of personalized learning. Implementation challenges and wide variations in achievement Most recently, we reported that 32 such schools produced average results raise key questions about the effectiveness of personalized positive effects of about 3 percentile points in mathematics and learning for improving student outcomes. In a rigorous evaluation 8 reading achievement. However, the quasi-experimental research that enables causal attribution, would personalized learning lead to design we employed leaves open the possibility that factors other meaningfully positive student outcomes? What specific strategies or than personalized learning might have caused the positive effects. models work best? Which are not effective? What contextual fac- Making a more conclusive determination will require follow-up tors are important to the success of personalized learning? Do the studies, likely using more-rigorous methods, such as randomized answers to these questions vary across different student populations controlled trials. Moreover, our studies did not have large enough 4 Educators, policymakers, and advocates are moving forward without the guidance of conclusive research evidence. How can they make wise choices to help ensure that their personalized learning models will benefit students and, importantly, not be harmful? or subgroups? How can we minimize the risk of negative effects? Encyclopedia to distinguish research that meets strong standards These and related questions will take many years to address. of rigor from less-reliable studies that some vendors or advocates While we await the answers to those questions, substantial might promulgate.10 enthusiasm around personalized learning persists. Educators, policymakers, and advocates are moving forward without the Align with Principles of Learning Science guidance of conclusive research evidence. How can they make wise Where strong empirical evidence on a particular personalized choices to help ensure that their personalized learning models will learning component does not yet exist, implementers can turn benefit students and, importantly, not be harmful? Theory and to more-basic research on cognition and learning science. For existing evidence can guide implementers to make wise choices and example, a decade ago, IES sponsored the preparation of a practice help to mitigate risks. guide based on research on learning and memory, which provides seven concrete recommendations on “Organizing Instruction to Principles to Guide Personalized Learning Improve Student Learning.”11 Since that time, the field has contin- Adoption ued to advance, supported by investments of multiple funders in In the absence of comprehensive, rigorous evidence to help select national centers on the science of learning.12 Principles identified the personalized learning components most likely to succeed, what in such research can provide rich fodder for helping to determine is the path forward? I suggest a few guiding principles aimed at which components of personalized learning models are most likely using existing scientific knowledge and the best available resources. to prove effective, and importantly, to rule out strategies that have been proven ineffective. For example, research gives reason to be Embrace Rigorous Empirical Evidence Where It Exists wary of some popular ideas in the personalized learning movement, Innovations that could be suitable as components of personalized such as the idea that today’s learners are digital natives for whom learning models, such as adaptive learning products, might already older methods of teaching no longer work, that learning should be have demonstrated positive effects. Implementers can look to such matched to a student’s learning style, or that students should be clearinghouses as the Institute of Education Sciences’ (IES’s) What given maximum control over what they learn and their learning Works Clearinghouse or Johns Hopkins University’s Best Evidence trajectory.13 5 Successful personalized learning strategies Use Rigorous Instructional Materials Decades of work have gone into developing rigorous academic or models likely will be designed to conserve standards and aligned instructional materials, most recently teachers’ time and effort for activities that through the Common Core initiative.14 But teachers trying to are most directly helpful to students. personalize learning naturally seek out new materials and lessons. These materials often are not carefully evaluated for rigor and can Focus on the Productive Use of Student Time and Attention lack coherence when cobbled together. Before shelving traditional Student time and attention might be the most valuable resources materials, educators should consider how they might be redeployed for a student’s education; learning simply cannot occur without in a personalized learning classroom, supplemented by strategies to them. If a student expends time and attention on activities that meet such personalized learning objectives as increased motivation are distracting or otherwise unproductive, those resources are and agency. Where educators feel that they must step outside the permanently lost. Moreover, even productive activities have an traditional curriculum materials and content, they should apply a opportunity cost: What else could the student have learned while quality rubric for evaluating new material before adoption. expending the same time and attention? Some personalized learn- ing activities might lead to learning while making inefficient use of Monitor Implementation and Be Prepared to Adapt these resources. This is not to say that students cannot gain impor- Although the five principles outlined above can help identify the tant skills through diverse activities beyond pure academics but most-promising strategies to incorporate into a personalized learn- rather that it is important to be attuned to both the benefits and ing design, success is not guaranteed. It is important to monitor the costs of all activities from which skill development is expected. whether the expected effects are emerging and ensure that there are no undesirable side effects. In addition to being useful for internal Maximize the Productive Use of Teacher Skill purposes of continuous improvement, such information can help Teachers are the next-most-valuable resources available to students the whole field of personalized learning improve. when their skills are properly focused on providing instruction and related support to students. Successful personalized learning strate- Applying the Guiding Principles to Mastery-Based gies or models likely will be designed to conserve teachers’ time and Skill Development effort for activities that are most directly helpful to students. One of the central changes in many personalized learning models is finding ways to enable students to work on content targeted to their individual level of achievement, rather than synchronized with peers at the same grade level. Indeed, this is the first major strategy 6 in the definition of personalized learning put forth in this Perspec- ing on material they are ready to learn because they have already tive. In these mastery-based or competency-based approaches, stu- developed prerequisite skills. Optimizing the amount of time that dents are assigned work for which they have satisfied prerequisites students are within that zone can theoretically result in greater and thus have the skills needed to learn the new material, and they achievement. continue working on that material until they demonstrate that they To illustrate, let us first consider students who enter non– have learned it. What do the guiding principles have to say about mastery-based schools performing below expectations. Although this approach? extra resources could be provided to help these students catch up, until they do (and they often do not), they are probably working Embrace Rigorous Empirical Evidence Where It Exists outside their zone of proximal development most of the time. Con- Empirical evidence on the effects of one-on-one human tutoring sequently, they might never fill in certain gaps, struggle to learn that used mastery-based progression is supportive. An often-cited new material, and do poorly on assignments and tests. This is not a 1984 study reported large positive achievement effects compared recipe for success, but one that is likely to exacerbate performance 15 with instruction in traditional classes. Subsequent studies of both gaps. Students who endure this for their entire school career might human and artificial (computer-based) tutors that used mastery- be more likely to become discouraged and to form poor concepts of based approaches have supported this finding, although not at self-efficacy.19 such a large magnitude as the 1984 study.16 Reviewing decades In contrast, personalized learning instruction, which is of research on human and computer-based tutoring systems, one designed to have students work in their zones of proximal develop- researcher hypothesized that “Tutoring does not work due to indi- ment and master material before moving on, could lead to better vidualization alone. It works due to individualization plus nurtur- learning and retention.20 In this approach, students should expe- ing and attention.”17 (This hypothesis resonates with another of the rience greater success, gain confidence in their abilities, and be major strategies of personalized learning: nurturing relationships better prepared to continue experiencing success on the material with students.) Taken together, these findings suggest the value of they move to subsequently. There are also possible benefits for high an approach where students regularly engage with educators, even achievers who might not be adequately challenged in non–mastery- if technology takes responsibility for some individualization of based systems if they are constrained to work at the pace of their content and pacing. peers. Align with Principles of Learning Science Focus on the Productive Use of Student Time and Attention Mastery-based skill development is consistent with the theory Taken together, the learning science and empirical evidence that learning builds on existing knowledge in a zone of proximal suggests that students’ time is well spent when they receive indi- 18 development. That is, students learn most effectively when work- vidualized attention, content at an appropriate level, and pacing 7 based on mastery, along with quality interaction, guidance, and Use Rigorous Instructional Materials nurturing from educators. Having a personal teacher or tutor for Implementing personalized learning is not as simple as picking every student might be one way to accomplish this, although it is some software and adding computers to the classroom. Educators, impractical because of the number of skilled educators necessary principals, and district leaders need to ask hard questions about the and the consequent cost. How can the benefits of individualized, software they are adopting, the quality of its content, and how it mastery-based tutoring be capitalized on at a large scale? Technol- will be deployed in classrooms. ogy has been the obvious answer. Key questions include how tech- nology can get us closer to individualized, mastery-based tutoring Monitor Implementation and Be Prepared to Adapt and whether it leads to better student outcomes than the system of Mastery-based approaches, however promising, are not without mainly whole-class instruction. Some of the most-promising tech- risks. For example, it is not yet known how such approaches will nology solutions do not move all instructional responsibility from play out over the long term. It seems plausible that a student who teachers to technology but use the two in tandem, consistent with experiences the hypothesized benefits of the approach—better both learning science and evidence to date. learning, retention, and motivation—could reach adulthood with a much higher level of achievement than they would have under the Maximize the Productive Use of Teacher Skill more traditional approach. Even if students benefit in a mastery- A key aspect of this approach is that teachers can be freed from based system, will they learn faster and catch up, or will they their responsibility to serve all students simultaneously, allowing perhaps learn more—and learn better—but never get an opportu- them to focus their skills on individuals or small groups while nity to learn some of the higher-level concepts that their peers are other students are actively engaged with technology. Furthermore, exposed to? This is known as the coverage versus mastery dilemma.21 the technology can use the information it gathers to advise teach- Whether a long-term mastery-based approach is better for a stu- ers on how best to use their time with the students. In practice, dent’s future success in life than the current system of primarily these benefits will only be realized if the system functions well and age-based progression has not yet been tested empirically. teachers are not burdened with technical-support issues or other Mastery-based approaches also raise complex questions of distractions. equity that echo concerns about tracking (i.e., grouping students into academic or vocational tracks).22 Pending concrete evidence Educators, principals, and district leaders need to ask hard questions about the software they are adopting, the quality of its content, and how it will be deployed in classrooms. 8 that mastery-based approaches are beneficial for adult outcomes, grade-based targets of attainment toward systems that focus on it seems particularly important to be sensitive to equity concerns. growth in achievement. Careful monitoring might be necessary to ensure that students who Overall, applying these guiding principles reveals many argu- are working on more-basic material in relation to their grade-level ments in favor of implementing mastery-based skill development peers are not somehow excluded from learning higher-level mate- approaches, along with risks that warrant diligent monitoring. rial. Educators must have an unrelenting focus on advancing these students’ achievement and mastery of complex concepts to the The Path Forward greatest extent possible. This requirement reinforces the impor- In conclusion, there are aspects of personalized learning that seem tance of teacher involvement. It is also important to distinguish to hold promise for improving the U.S. K–12 education system, the concept of flexible pacing, which is intrinsic to the mastery- based on some limited research. However, more work is necessary based approach, from self-pacing. Although self-pacing might help to establish causal evidence that the concept leads to improved out- to build student agency skills, it is unlikely to be a good general comes for students. And because personalized learning is composed strategy to ensure that students learn academic content at an of so many interrelated strategies, considerable additional research 23 appropriate breadth and depth. A mastery-based system must be will be needed to sort out the fine details of which strategies, and designed with awareness of the inclination of learners of all abilities in which combinations, are most effective for which students. to sometimes avoid the hard work of learning. Presently, early implementers of personalized learning are working Finally, mastery-based approaches can conflict with stan- with imperfect evidence, underdeveloped curricular resources, and dards, assessment, and accountability policies that have not yet policies that might hinder their efforts. As personalized learning adapted accordingly. Students in most states are still required to approaches become widespread, there are risks that these condi- take assessments based on grade-level standards, and teachers and tions could cause early implementations to fail. This could lead to schools often are evaluated on how well students do on those assess- the larger concept being abandoned before it can be tested under ments. In this context, teachers who are willing to experiment with more-favorable conditions. As a protection against these risks, mastery-based approaches early in the year might become increas- implementers should use some guiding principles to help discern ingly uncomfortable with the practice as year-end assessments the aspects of personalized learning that are most likely to lead to approach and students have not covered the material on which they success. Following these principles could increase the chance that will be tested. States, school districts, and school leaders who want early efforts are productive, which will help to spur momentum to allow mastery-based approaches to play out over multiple years for the development of the tools necessary to sustain personalized will need to rethink how systems of standards, assessments, and learning and put it on a path toward meeting its full potential as a accountability operate. This might require a shift away from age- or major reform of the K–12 education system. 9  9 Endnotes B. Gross, and M. DeArmond, Personalized Learning at a Crossroads: Early Lessons 1 National Center for Education Statistics, Program for International Student from the Next Generation Systems Initiative and the Regional Funds for Breakthrough Assessment (PISA), “Selected Findings from PISA 2015,” webpage, undated. As of Schools Initiative, Seattle, Wash.: Center for Reinventing Public Education, 2018. July 29, 2018: https://nces.ed.gov/surveys/pisa/pisa2015/pisa2015highlights_1.asp 10 IES, “What Works Clearinghouse,” homepage, undated. As of July 29, 2018: 2 The Nation’s Report Card, “NAEP Mathematics and Reading Assessments: https://ies.ed.gov/ncee/wwc/; Johns Hopkins University School of Education, Highlighted Results at Grades 4 and 8 for the Nation, States, and Districts,” “Best Evidence Encyclopedia,” webpage, undated. As of July 29, 2018: webpage, 2017. As of July 29, 2018: https://www.nationsreportcard.gov/ http://www.bestevidence.org reading_math_2017_highlights 11 Harold Pashler, Patrice M. Bain, Brian A. Bottge, Arthur Graesser, Kenneth 3 National Center for Education Statistics, “Trends in High School Dropout and Koedinger, Mark McDaniel, and Janet Metcalfe, Organizing Instruction and Study Completion Rates in the United States: Selected Findings,” webpage, Washington, to Improve Student Learning: A Practice Guide, Washington, D.C.: U.S. Depart- D.C.: U.S. Department of Education, 2014. As of July 29, 2018: ment of Education, September 2007. As of September 11, 2018: https://nces.ed.gov/programs/dropout/findings.asp https://ies.ed.gov/ncee/wwc/Docs/PracticeGuide/20072004.pdf 12 4 S. F. Reardon, “The Widening Academic Achievement Gap Between the Rich Johns Hopkins University Science of Learning Institute, “Who Is and the Poor: New Evidence and Possible Explanations,” in G. J. Duncan and R. Investing in the Science of Learning?” webpage, undated. As of July 29, J. Murnane, eds., Whither Opportunity? Rising Inequality, Schools, and Children’s 2018: http://scienceoflearning.jhu.edu/science-to-practice/resources/ Life Chances, New York: Russell Sage Foundation, 2011, pp. 91–116. who-is-investing-in-the-science-of-learning 13 5 David J. Ferrero, “Are Schools Trying to Teach Too Much?” Education Week, Paul A. Kirschner and Jeroen J. G. van Merriënboer, “Do Learners Really Know July 20, 2018. As of July 30, 2018: https://www.edweek.org/ew/ Best? Urban Legends in Education,” Educational Psychologist, Vol. 48, No. 3, articles/2018/07/20/are-schools-trying-to-teach-too-much.html 2013, pp. 169–183. 14 6 Frederick Hess, “What We’ve Forgotten About School Reform: Cour- Allan Golston, “What’s Working: Getting Teachers More Common Core tesy of Messrs. Tyack, Cuban, and Payne,” Education Next, Septem- Aligned Materials,” Huffpost Blog, April 14, 2015. As of September 5, 2018: ber 22, 2017. As of August 21, 2018: https://www.educationnext.org/ https://www.huffingtonpost.com/allan-golston-/ what-weve-forgotten-about-school-reform-courtesy-tyack-cuban-payne whats-working-getting-tea_b_9692866.html 15 7 John F. Pane, Elizabeth D. Steiner, Matthew D. Baird, Laura S. Hamilton, and B. S. Bloom, “The 2 Sigma Problem: The Search for Methods of Group Instruc- Joseph D. Pane, Informing Progress: Insights on Personalized Learning Implementa- tion as Effective as One-to-One Tutoring,” Educational Researcher, Vol. 13, 1984, tion and Effects, Santa Monica, Calif.: RAND Corporation, RR-2042-BMGF, pp. 4–16. 2017. As of September 11, 2018: https://www.rand.org/pubs/research_reports/ 16 K. VanLehn, “The Relative Effectiveness of Human Tutoring, Intelligent Tutor- RR2042.html; John F. Pane, Elizabeth D. Steiner, Matthew D. Baird, and Laura ing Systems, and Other Tutoring Systems,” Educational Psychologist, Vol. 46, S. Hamilton, Continued Progress: Promising Evidence on Personalized Learning, No. 4, 2011, pp. 197–221. Santa Monica, Calif.: RAND Corporation, RR-1365-BMGF, 2015. As of Septem- 17 ber 11, 2018: https://www.rand.org/pubs/research_reports/RR1365.html Robert Slavin, “New Findings on Tutoring: Four Shockers,” Robert Slavin’s Blog, April 5, 2018. As of July 30, 2018: https://robertslavinsblog.wordpress.com/ 8 To interpret effects in percentile points, consider a student who would have per- 2018/04/05/new-findings-on-tutoring-four-shockers formed at the median (50th percentile) on the outcome measure (e.g., an achieve- 18 ment test) if they had been in the control group. A treatment effect of 3 percentile L. S. Vygotsky, “Interaction Between Learning and Development,” in M. Cole, points means that this student is estimated to surpass 3 percent of students to V. John-Steiner, S. Scribner, and E. Souberman, eds., Mind in Society: The Devel- perform at the 53rd percentile if he or she were instead in a personalized learning opment of Higher Psychological Processes, Cambridge, Mass.: Harvard University school. Thus, these effects are describing the placement of a student’s test score Press, 1978, pp. 79–91. (Original manuscripts [ca. 1930–1934]). in the distribution of all of the students’ scores. A common misperception is to interpret this as a percentage-point difference in the test score itself. 10 19 A. Bandura, “Self-Efficacy,” in V. S. Ramachaudran, ed., Encyclopedia of Human Behavior, New York: Academic Press, Vol. 4, 1994, pp. 71–81. 20 Bandura, 1994. 21 R. E. Slavin, “Mastery Learning Reconsidered,” Review of Educational Research, Vol. 57, No. 2, 1987, pp. 175–213. 22 G. Schütz, H. W. Ursprung, and L. Wößmann, “Education Policy and Equality of Opportunity,” KYKLOS, Vol. 61, No. 2, 2008, pp. 279–308. 23 Kirschner and van Merriënboer, 2013. 11 About This Perspective Acknowledgments Innovators are exploring new designs for the K–12 education system under The author is grateful for the insightful comments and suggestions provided the umbrella of personalized learning, but consensus is lacking on a precise by Betheny Gross, Elizabeth Steiner, and Cathy Stasz. Reader comments definition of personalized learning or on which component practices are and queries are welcome. essential. Practitioners and policymakers seeking to implement personal- ized learning are creating custom designs for their specific contexts. Those who want to use rigorous research evidence to guide their designs will find About the Author many gaps and will be left with important unanswered questions about John F. Pane is a senior scientist who studies technology innovations which practices or combinations of practices are effective. Despite the lack in education using rigorous research methods. His recent work includes of evidence, there is considerable enthusiasm about personalized learn- the first large-scale evaluation of schoolwide personalized learning and ing among practitioners and policymakers, and implementation is spread- several efficacy studies of intelligent tutoring systems. He held RAND’s Dis- ing. Thus, the purpose of this Perspective is to offer strategic guidance for tinguished Chair in Education Innovation from 2015 to 2018. designers of personalized learning programs to consider while the evidence base is catching up. This guidance draws on theory, basic principles from learning science, and the limited research that does exist on personalized learning and its component parts. This research was conducted in RAND Education. RAND Ventures Limited Print and Electronic Distribution Rights The RAND Corporation is a research organization that develops solutions This document and trademark(s) contained herein are protected by law. This representa- to public policy challenges to help make communities throughout the world tion of RAND intellectual property is provided for noncommercial use only. Unauthor- safer and more secure, healthier and more prosperous. RAND is nonprofit, ized posting of this publication online is prohibited. Permission is given to duplicate this nonpartisan, and committed to the public interest. document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of our research docu- RAND Ventures is a vehicle for investing in policy solutions. Philan- ments for commercial use. For information on reprint and linking permissions, please visit thropic contributions support our ability to take the long view, tackle tough www.rand.org/pubs/permissions.html. and often-controversial topics, and share our findings in innovative and The RAND Corporation is a research organization that develops solutions to public compelling ways. RAND’s research findings and recommendations are policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the based on data and evidence, and therefore do not necessarily reflect the public interest. policy preferences or interests of its clients, donors, or supporters. RAND’s publications do not necessarily reflect the opinions of its research clients and Funding for this venture was provided by gifts from RAND supporters sponsors. R® is a registered trademark. and income from operations. For more information on this publication, visit www.rand.org/t/PE314. © Copyright 2018 RAND Corporation C O R P O R AT I O N www.rand.org PE-314-RC (2018)