SlideShare a Scribd company logo
Myths and Misperceptions about Online Learning Jorge Larreamendy-Joerns, Universidad de los Andes,  Mary Tallent Runnels, Texas Tech University Julie A. Thomas, Oklahoma State University Robert Bernard, Phil Abrami, Concordia University Facilitator: Peter Shea, University at Albany, State University of New York
Myths and Misperceptions Longstanding confusion regarding online learning – purpose, history, quality, interaction, outcomes, NSD etc.  Need for high quality research e.g. Review of Educational Research (RER)  #1 scholarly journal in the field of educational research Opportunity for more nuanced views Three kinds of reviews...
Conceptual Review Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education.  Review of Educational Research, 76  (4), 567-606 .
Traditional Review Tallent-Runnels, M. & Thomas, J. et. al. (2006). Teaching courses online: A review of the research.  Review of Educational Research , Vol. 76, No. 1, 93-135.
Empirical/Meta-analytic review Bernard, R.M. & Abrami, P.C. et. al. (2009) A meta-analysis of three types of interaction treatments in distance education.  Review of Educational Research , 79(3): 1243 - 1289. Bernard, R.M., Abrami, P.C., et. al. (2004). How does distance education compare to classroom instruction? A Meta-analysis of the empirical literature.  Review of Educational Research , 74(3), 379-439.
Order Jorge Larreamendy-Joerns: “Online education: Promises and Concerns with a Past”  Mary Tallent-Runnels & Julie Thomas: “Myths and Misperceptions”  Robert Bernard & Philip Abrami: “Distance Education and Online Learning Research: Past, Present and Future”
Online education: Promises and concerns with a past Jorge Larreamendy-Joerns Universidad de los Andes, Colombia
“ Our classroom can be worldwide. Its present bounds are marked by Western China in the far East, Dawson, Fairbanks, and Kodiak Island to the North, Chile to the South, and Senegal and the Union of South Africa in the near East. Our constituency embraces college presidents, government officials, representatives of most of the professions and vocations, and those who by stress of circumstance are debarred from the ordinary means of education.”
A sense of history Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education.  Review of Educational Research, 76  (4), 567-606 . Continuity between past (traditional distance education) and present (online education)
Some questions What to expect from online education? What to be concerned about? Can online education overcome educational limitations? Can it be an instrument of democratization? Does it amount to marketize quality education? What is the goal? Deep learning? Flaring technology? Reaching out to audiences?
Charting promises and concerns Major visions in the current scene from a learning perspective: Presentational view Performance-tutoring view Epistemic engagement view  Need not be exclusive, but signal to emphases Urgency to undertake analyses of underlying conceptions of learning, knowledge, practice, and expertise.
Going back to the past Online education has inherited many of the concerns and promissory notes of distance education. Faculty resistance Pedagogical imperialism Learning and autonomy Democratization Liberal education Educational quality
Democratization Increasing either the access to higher education of populations that would be otherwise excluded, or increasing the range of people who might be served by elite institutions. Anna Ticknor’s  Society to Encourage Studies at Home  (1873). The legacy of William Rainey Harper, president of the UChicago. The Department of Home-Study
 
Concerns Distance education as dispensable.  Misalignments between expectations and actualities. Underestimation and overestimation.  The down of liberal arts education The issue of quality
(Some) lessons learned A non linear path Instructional quality and media limitations The equivalence of faculty
Pentimento
Thanks
Mary Tallent-Runnels and Julie Thomas
Males and females report similar learning benefits with online learning.
Faculty spend the same amount of time teaching online as they do face-to-face.
Online classes are taken by many existing students.
Universities provide support for faculty teaching online. This includes both instructional and technical assistance.
Technical difficulties always lower student achievement.
The more that students experience technical difficulties in online classes the lower they rate their instructors.
Online classes do not have as much interaction between students and their instructors and among students as in face-to-face classes.
Student interest in online classes has leveled off.
Taking online classes is more expensive that taking face-to-face classes.
Synchronous online classes (where students can work together online with the instructor) are more popular than asynchronous online classes.
Formats of online course evaluation are more limited than in face-to-face class evaluation.
Online-students have less access to lectures (and other help from instructors) than do face-to-face students.
Meta-Analysis of Distance Education and Online Learning Research:  Past, Present and Future Robert M. Bernard Philip C. Abrami Centre for the Study of Learning and Performance Concordia University Montreal, Canada 15 th  Sloan-C International Conference on Online Learning 1
Characteristics  of a Systematic Review Systematic—step-by-step   approach Comprehensive —encompasses all relevant research Objective—minimizes bias at each step Transparent —reveals all important details Repeatable—other researchers can duplicate   Integrative —reaches general conclusions Explanatory—relates findings to theory/practice Relevant —adds value to practitioners/policymakers
Overview of   Meta-Analysis Quantitative synthesis of experimental research literature Effect size expresses the standardized difference between treatment and control conditions Estimates the magnitude of the average effect size   and explores variability within effect size distributions Explores moderating influences of coded study features Findings generalize to populations of learners better than any single primary study can
The Past and Present:  DE and Online Learning Compared to Classroom Instruction Between 2000 and the present, 15 meta-analyses   were conducted assessing the difference between DE/Online Learning and Classroom Instruction What have we learned?  All forms of DE/OL   are at least equal to CI Wide variability   surrounds average effect sizes from highly positive to highly negative effect sizes Quality of research studies   is questionable
What About Online Learning?  Results of Meta-Analyses of OL vs. CI
What Does an  ES + of   0.12-0.15  Mean?
What About   Online Learning  Compared to Asynchronous Distance Education? Note: Asynchronous DE studies are primarily “correspondence, video-based or broadcast” applications
The Future:  Comparisons Between  Different DE/OL Conditions “… the next form of progress to advance theory will be made as researchers begin to examine how structural and technological treatments differ between DE conditions, not between DE and CI.” (Bernard, Abrami, Wade, et. al, 2009) “ We need studies clarifying when to use e-learning (studies exploring strengths and weaknesses) and how to use it effectively (head-to-head comparisons of e-learning interventions).” (Cook, 2009)
How Studies   Between DE/OL and DE/OL  Might be Meta-Analyzed Problem:  Which condition is the “treatment” and which is the “control”? One solution:  Choose a  construct  (e.g., interactivity) and establish “greater presence” as the treatment and “lesser presence” as the control Effect sizes with a (+) valence  favor the treatment  (greater presence) and those with a (–) valence  favor the control  (lesser presence)
We Chose  Interaction  as a  Researchable Construct  Interaction is commonly understood as actions between/among individuals but here it includes individual or collective interactions with curricular content   (Moore, 1989) Student-student (S-S) interaction refers to interaction among individual students or among students working in small groups Student–teacher (S-T) interaction focuses  on dialogue between students and the instructor (e.g., clarification, feedback) Student-content (S-C) interaction refers to students interacting with the subject matter to construct meaning, relate it to personal knowledge, and apply it to problem solving
What is an  “Interaction Treatment”? Definition:  ITs are instructional or technology conditions implemented to enable S-S, S-T or  S-C interaction S-S Example:  students working online collaboratively to complete a project S-T Example:  availability of teacher-provided feedback to students S-C Example:  students watching streamed video of lectures or any other content
Meta-Analysis of   Three Interaction Treatments  in Distance Education From Bernard, Abrami, Borokhovski, Wade et al. (2009).  RER.
Summary of  Results Overall treatment strength was   g + = 0.38 ( k  = 74) S-S  and  S-C  treatments were higher than  S-T Higher  and  medium  strength ITs  better  than  low  strength Levels of strength differed  (higher > lower)  across combinations  S-S + SC  and  S-T + SC Synchronous, Asynchronous and Mixed DE  not different 11
One Big   Caveat  in this Research We examined the   nature of interaction treatments   “instructional arrangements intended to foster interaction” We did not examine   levels of interaction   directly (how much actual interaction the treatments produced) Measuring interaction   represents a challenge for the next generation of primary researchers interested in interaction in DE/OL
Evidence-Based Principles for   Increasing Achievement Through Interactivity First Principle:   Increasing interaction of all kinds (S-S, S-T, S-C) leads to higher achievement gains Second Principles:   Support the content; increase learners’ access to and engagement with whatever is intended to be learned Increase access to the teacher or other students depending on the orientation to teacher- or student-centeredness and desired learning activities Provide tertiary levels of support as resources (e.g., personnel, time, technology) allow
What We  Need  to Push the Field Forward More research   that compares at least one OL treatment to another OL treatment (no need to get too fancy) Better research   designs (if not RCTs, designs that at least control for selection bias) More studies   across the grade levels (k-12) and in higher education settings of all types Better-quality   descriptions of treatments Better-quality   measures Better-quality reporting ,   preferably full descriptive statistics
References Anderson, T. (2003).  Getting the mix right again: An updated and theoretical rationale for interaction.  International Review of Research in Open and Distance Learning, 4(2),  9–14. Bernard, R.M., Abrami, P.C., Lou, Y., Borokhovski, E., Wade, A. et al. (2004).  How does distance education compare to classroom instruction? A Meta-analysis of the empirical literature.  Review of Educational Research, 74 (3), 379-439. Bernard, R.M., Abrami, P.C., Borokhovski, E., Wade, A., Tamim, R. et al. (2009).  A meta-analysis of three interaction treatments in distance education.  Review of Educational Research, 79 , 1243-1289. (doi:10.3102/0034654309333844v1). Cook, D.A. (2009).  The failure of e-learning research to inform educational practice, and what we can do about it.  Medical Teacher ,  31 (2), 158-162. Cook, D.A., Levinson, A.J., Garside, S., Dupras et al. (2008).  Internet-based learning in the health professions: A meta-analysis.  Journal of the American Medical Association,  300(10), 1181-1196 (doi:10.1001/jama.300.10.1181). U.S Department of Education, Office of Planning, Evaluation, and Policy Development (2009).  Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies,  Washington, D.C. Moore, M.G. (1989).  Three types of interaction.  American Journal of Distance Education, 3(2), 1–6. Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006).  The comparative Effectiveness of web-based and classroom instruction: A meta-analysis . Personnel Psychology, 59,  623-664.
Examples of Categorization Decisions:   Student-Student Interaction   Bell, Hudson and Heinan (2004)  provided two methods of online learning to Physician Assistant students in a medical terminology course.  Both versions of the course used the same materials, but some students worked independently on the Web  (Group B) , while others  (Group A)   received 12 case studies in an online conference setting, which they then discussed through the use of asynchronous messaging.  We applied the comparative outcomes to “student-student interaction,” counting the case-based discussion participants  (Group A)   as the experimental group.
Examples of Categorization Decisions:   Student-Teacher Interaction  Rovai (2001)  compared fully online instruction in education, delivered via an asynchronous learning network  (Group A)  with blended instruction which included monthly face-to-face meetings with the instructor  ( Group B ) .  Students in  Group B   were given extra chances to communicate (interact) with their teacher (as the means for communication online were equal in both conditions).  Group B   was designated as the experimental group for “student–teacher interaction.”
Examples of Categorization Decisions:   Student-Content Interaction  Bernard and Naidu (1994)  offered nursing students two different strategies for studying in a DE course on Community Mental Health. Participants received either created and studied concept maps  (Group A)  or answered predesigned study post-questions  (Group B) .  Group A   interacted with the course materials to a greater extent than  Group B  and, thus, constituted the experimental condition.

More Related Content

PPT
Sabbatical (Massey University) - An Introduction to a New Research Paradigm: ...
PPT
Design- Based Research: New Research Paradigm
PPTX
Social media and scholarly research
PPTX
PhD research proposal presentation Sonia Saddiqui 28 Nov 2013
PPT
A Qualitative Phenomenological Study on Prison Volunteers in California’s Cor...
PPTX
E-Research Open Learning Conference Unisa 2018
PPTX
Understanding Online Reading Comprehension, Collaboration, and Digital Inquir...
PPT
PhD proposal presentation
Sabbatical (Massey University) - An Introduction to a New Research Paradigm: ...
Design- Based Research: New Research Paradigm
Social media and scholarly research
PhD research proposal presentation Sonia Saddiqui 28 Nov 2013
A Qualitative Phenomenological Study on Prison Volunteers in California’s Cor...
E-Research Open Learning Conference Unisa 2018
Understanding Online Reading Comprehension, Collaboration, and Digital Inquir...
PhD proposal presentation

What's hot (18)

PPT
My Dissertation Proposal Defense
PDF
CWS WRIL presentation - Christine Wihak
PPTX
Improving young people's HE decision-making: Exploring the role of IAG
PDF
DPARK Ed D
PDF
Counternarratives and HBCU Student Success - NASPA 3.24.15
PDF
Halcon dissertation final defense powerpoint 11-08-10
PDF
Learning analytics in higher education: Promising practices and lessons learned
PPT
Basic principles of interaction for learning in web based environment
PDF
Information Literacy and Student Engagement: Cultivating Student Learning Th...
PPT
Researching e-portfolios: The current state of play
PDF
Towards a philosophy of lifelong learning 
with eportfolios
PPTX
2012 BERA IAG paper_KV
PDF
Oumh1103 pdf factor influencing adult learners' decicion to drop out or per...
PPTX
Research on and Practice of Social Media in higher-ed teaching
PPTX
Oral defense power point
PPTX
Reflection, Integration, Identity, and Institutional Change
PPT
Slides presentation example for proposal defence
My Dissertation Proposal Defense
CWS WRIL presentation - Christine Wihak
Improving young people's HE decision-making: Exploring the role of IAG
DPARK Ed D
Counternarratives and HBCU Student Success - NASPA 3.24.15
Halcon dissertation final defense powerpoint 11-08-10
Learning analytics in higher education: Promising practices and lessons learned
Basic principles of interaction for learning in web based environment
Information Literacy and Student Engagement: Cultivating Student Learning Th...
Researching e-portfolios: The current state of play
Towards a philosophy of lifelong learning 
with eportfolios
2012 BERA IAG paper_KV
Oumh1103 pdf factor influencing adult learners' decicion to drop out or per...
Research on and Practice of Social Media in higher-ed teaching
Oral defense power point
Reflection, Integration, Identity, and Institutional Change
Slides presentation example for proposal defence
Ad

Viewers also liked (18)

PPTX
Programming A Robot Using
PPT
Credentials Q4
PPT
Qatar Airways. Летайте с роскошью арабских шейхов!
PPT
Credentials Q4.09
PPT
What Is Social Media
PDF
Презентация компании Okamura
PPTX
Vicoustic
PPTX
Mj Acoustic
PPTX
Loksatta for new members
PPTX
Upadesa Saram - my notes
PPTX
Computational Intelligence Unconference Jul 2014 Data to Knowledge
ODP
Niatalk24jan10
PPS
Have a nice weekend
PPT
Content Marketing for Leads, Fame and Revenue
PDF
Loksatta's Roadmap to Universal Healthcare
PDF
Speidel brew master brewing system
PPTX
Microcontroller
PPTX
Microprocessors
Programming A Robot Using
Credentials Q4
Qatar Airways. Летайте с роскошью арабских шейхов!
Credentials Q4.09
What Is Social Media
Презентация компании Okamura
Vicoustic
Mj Acoustic
Loksatta for new members
Upadesa Saram - my notes
Computational Intelligence Unconference Jul 2014 Data to Knowledge
Niatalk24jan10
Have a nice weekend
Content Marketing for Leads, Fame and Revenue
Loksatta's Roadmap to Universal Healthcare
Speidel brew master brewing system
Microcontroller
Microprocessors
Ad

Similar to Myths And Misperceptions About Online Learning2 (20)

PDF
What does effective online/blended teaching look like?
PDF
intro to online tools for teaching and learning.pdf
PPTX
Online Teaching and Learning - Basic Concepts
DOC
Nasrin Nazemzadeh, DissertationTitle page, Abstract, and Table of Contents, D...
PPTX
Agonquin College Keynote
PPTX
Distance Education theorists 2011
PPT
Evaluation Of Evidence Based Practices In Online Learning
PPTX
Opportunities and Challenges in Open,Distance and e-Learning -ODEL
PPTX
Online or Face to Face?
PDF
Linda Harasim on Online Collaborative Learning
PPT
Online Learning Theory
PPTX
Week2 islearningdifferentfinal
PPTX
Online learning
PPTX
theadvantagesanddisadvantagesofonlinelearning-121020185318-phpapp01.pptx
KEY
Bcuenr310
PPT
Online Learning and courses
PDF
Distance Learning – A new way of learning
PPT
Design based for lisbon 2011
PPTX
Bunnell ELT7008-4
PPTX
online laerning fsfs fsdf ffe rwtewt twtwt twweew
What does effective online/blended teaching look like?
intro to online tools for teaching and learning.pdf
Online Teaching and Learning - Basic Concepts
Nasrin Nazemzadeh, DissertationTitle page, Abstract, and Table of Contents, D...
Agonquin College Keynote
Distance Education theorists 2011
Evaluation Of Evidence Based Practices In Online Learning
Opportunities and Challenges in Open,Distance and e-Learning -ODEL
Online or Face to Face?
Linda Harasim on Online Collaborative Learning
Online Learning Theory
Week2 islearningdifferentfinal
Online learning
theadvantagesanddisadvantagesofonlinelearning-121020185318-phpapp01.pptx
Bcuenr310
Online Learning and courses
Distance Learning – A new way of learning
Design based for lisbon 2011
Bunnell ELT7008-4
online laerning fsfs fsdf ffe rwtewt twtwt twweew

Recently uploaded (17)

PDF
Cyberagent_For New Investors_EN_250808.pdf
DOC
École毕业证学历认证,劳伦森大学毕业证毕业证文凭
PDF
Corporate Finance, 12th Edition, Stephen Ross, Randolph Westerfield, Jeffrey ...
PPTX
opinion fact prediction, value judgement
PPTX
Chemistry.pptxjhghjgghgyughgyghhhvhbhghjbjb
PPT
275505080-Excitation-System FRWEFAAG.ppt
PPTX
TTL1_LMS-Presenfdufgdfgdgduhfudftation.pptx
PDF
North Arrow Corporate and Kraaipan Gold Project Update
PDF
How to Analyze Market Trends in Precious Metal.pdf
PPTX
investment-opportunities-in-rajasthan.pptx
PPTX
opinion fact prediction, biasness, vested interest
PDF
Deutsche EuroShop | Company Presentation | 08/25
PDF
Buy Verified Chime Accounts - Lori Donato's blo.pdf
PDF
The-Importance-of-Mutual-Funds-in-Your-Financial-Life (1).pdf
PDF
How Foreign Investment in Nepal Makes a Difference.pdf
PDF
Top Investment Opportunities in Nepal (1).pdf
PPTX
Individual report of global perspective.
Cyberagent_For New Investors_EN_250808.pdf
École毕业证学历认证,劳伦森大学毕业证毕业证文凭
Corporate Finance, 12th Edition, Stephen Ross, Randolph Westerfield, Jeffrey ...
opinion fact prediction, value judgement
Chemistry.pptxjhghjgghgyughgyghhhvhbhghjbjb
275505080-Excitation-System FRWEFAAG.ppt
TTL1_LMS-Presenfdufgdfgdgduhfudftation.pptx
North Arrow Corporate and Kraaipan Gold Project Update
How to Analyze Market Trends in Precious Metal.pdf
investment-opportunities-in-rajasthan.pptx
opinion fact prediction, biasness, vested interest
Deutsche EuroShop | Company Presentation | 08/25
Buy Verified Chime Accounts - Lori Donato's blo.pdf
The-Importance-of-Mutual-Funds-in-Your-Financial-Life (1).pdf
How Foreign Investment in Nepal Makes a Difference.pdf
Top Investment Opportunities in Nepal (1).pdf
Individual report of global perspective.

Myths And Misperceptions About Online Learning2

  • 1. Myths and Misperceptions about Online Learning Jorge Larreamendy-Joerns, Universidad de los Andes, Mary Tallent Runnels, Texas Tech University Julie A. Thomas, Oklahoma State University Robert Bernard, Phil Abrami, Concordia University Facilitator: Peter Shea, University at Albany, State University of New York
  • 2. Myths and Misperceptions Longstanding confusion regarding online learning – purpose, history, quality, interaction, outcomes, NSD etc. Need for high quality research e.g. Review of Educational Research (RER) #1 scholarly journal in the field of educational research Opportunity for more nuanced views Three kinds of reviews...
  • 3. Conceptual Review Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76 (4), 567-606 .
  • 4. Traditional Review Tallent-Runnels, M. & Thomas, J. et. al. (2006). Teaching courses online: A review of the research. Review of Educational Research , Vol. 76, No. 1, 93-135.
  • 5. Empirical/Meta-analytic review Bernard, R.M. & Abrami, P.C. et. al. (2009) A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research , 79(3): 1243 - 1289. Bernard, R.M., Abrami, P.C., et. al. (2004). How does distance education compare to classroom instruction? A Meta-analysis of the empirical literature. Review of Educational Research , 74(3), 379-439.
  • 6. Order Jorge Larreamendy-Joerns: “Online education: Promises and Concerns with a Past” Mary Tallent-Runnels & Julie Thomas: “Myths and Misperceptions” Robert Bernard & Philip Abrami: “Distance Education and Online Learning Research: Past, Present and Future”
  • 7. Online education: Promises and concerns with a past Jorge Larreamendy-Joerns Universidad de los Andes, Colombia
  • 8. “ Our classroom can be worldwide. Its present bounds are marked by Western China in the far East, Dawson, Fairbanks, and Kodiak Island to the North, Chile to the South, and Senegal and the Union of South Africa in the near East. Our constituency embraces college presidents, government officials, representatives of most of the professions and vocations, and those who by stress of circumstance are debarred from the ordinary means of education.”
  • 9. A sense of history Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76 (4), 567-606 . Continuity between past (traditional distance education) and present (online education)
  • 10. Some questions What to expect from online education? What to be concerned about? Can online education overcome educational limitations? Can it be an instrument of democratization? Does it amount to marketize quality education? What is the goal? Deep learning? Flaring technology? Reaching out to audiences?
  • 11. Charting promises and concerns Major visions in the current scene from a learning perspective: Presentational view Performance-tutoring view Epistemic engagement view Need not be exclusive, but signal to emphases Urgency to undertake analyses of underlying conceptions of learning, knowledge, practice, and expertise.
  • 12. Going back to the past Online education has inherited many of the concerns and promissory notes of distance education. Faculty resistance Pedagogical imperialism Learning and autonomy Democratization Liberal education Educational quality
  • 13. Democratization Increasing either the access to higher education of populations that would be otherwise excluded, or increasing the range of people who might be served by elite institutions. Anna Ticknor’s Society to Encourage Studies at Home (1873). The legacy of William Rainey Harper, president of the UChicago. The Department of Home-Study
  • 14.  
  • 15. Concerns Distance education as dispensable. Misalignments between expectations and actualities. Underestimation and overestimation. The down of liberal arts education The issue of quality
  • 16. (Some) lessons learned A non linear path Instructional quality and media limitations The equivalence of faculty
  • 19. Mary Tallent-Runnels and Julie Thomas
  • 20. Males and females report similar learning benefits with online learning.
  • 21. Faculty spend the same amount of time teaching online as they do face-to-face.
  • 22. Online classes are taken by many existing students.
  • 23. Universities provide support for faculty teaching online. This includes both instructional and technical assistance.
  • 24. Technical difficulties always lower student achievement.
  • 25. The more that students experience technical difficulties in online classes the lower they rate their instructors.
  • 26. Online classes do not have as much interaction between students and their instructors and among students as in face-to-face classes.
  • 27. Student interest in online classes has leveled off.
  • 28. Taking online classes is more expensive that taking face-to-face classes.
  • 29. Synchronous online classes (where students can work together online with the instructor) are more popular than asynchronous online classes.
  • 30. Formats of online course evaluation are more limited than in face-to-face class evaluation.
  • 31. Online-students have less access to lectures (and other help from instructors) than do face-to-face students.
  • 32. Meta-Analysis of Distance Education and Online Learning Research: Past, Present and Future Robert M. Bernard Philip C. Abrami Centre for the Study of Learning and Performance Concordia University Montreal, Canada 15 th Sloan-C International Conference on Online Learning 1
  • 33. Characteristics of a Systematic Review Systematic—step-by-step approach Comprehensive —encompasses all relevant research Objective—minimizes bias at each step Transparent —reveals all important details Repeatable—other researchers can duplicate Integrative —reaches general conclusions Explanatory—relates findings to theory/practice Relevant —adds value to practitioners/policymakers
  • 34. Overview of Meta-Analysis Quantitative synthesis of experimental research literature Effect size expresses the standardized difference between treatment and control conditions Estimates the magnitude of the average effect size and explores variability within effect size distributions Explores moderating influences of coded study features Findings generalize to populations of learners better than any single primary study can
  • 35. The Past and Present: DE and Online Learning Compared to Classroom Instruction Between 2000 and the present, 15 meta-analyses were conducted assessing the difference between DE/Online Learning and Classroom Instruction What have we learned? All forms of DE/OL are at least equal to CI Wide variability surrounds average effect sizes from highly positive to highly negative effect sizes Quality of research studies is questionable
  • 36. What About Online Learning? Results of Meta-Analyses of OL vs. CI
  • 37. What Does an ES + of 0.12-0.15 Mean?
  • 38. What About Online Learning Compared to Asynchronous Distance Education? Note: Asynchronous DE studies are primarily “correspondence, video-based or broadcast” applications
  • 39. The Future: Comparisons Between Different DE/OL Conditions “… the next form of progress to advance theory will be made as researchers begin to examine how structural and technological treatments differ between DE conditions, not between DE and CI.” (Bernard, Abrami, Wade, et. al, 2009) “ We need studies clarifying when to use e-learning (studies exploring strengths and weaknesses) and how to use it effectively (head-to-head comparisons of e-learning interventions).” (Cook, 2009)
  • 40. How Studies Between DE/OL and DE/OL Might be Meta-Analyzed Problem: Which condition is the “treatment” and which is the “control”? One solution: Choose a construct (e.g., interactivity) and establish “greater presence” as the treatment and “lesser presence” as the control Effect sizes with a (+) valence favor the treatment (greater presence) and those with a (–) valence favor the control (lesser presence)
  • 41. We Chose Interaction as a Researchable Construct Interaction is commonly understood as actions between/among individuals but here it includes individual or collective interactions with curricular content (Moore, 1989) Student-student (S-S) interaction refers to interaction among individual students or among students working in small groups Student–teacher (S-T) interaction focuses on dialogue between students and the instructor (e.g., clarification, feedback) Student-content (S-C) interaction refers to students interacting with the subject matter to construct meaning, relate it to personal knowledge, and apply it to problem solving
  • 42. What is an “Interaction Treatment”? Definition: ITs are instructional or technology conditions implemented to enable S-S, S-T or S-C interaction S-S Example: students working online collaboratively to complete a project S-T Example: availability of teacher-provided feedback to students S-C Example: students watching streamed video of lectures or any other content
  • 43. Meta-Analysis of Three Interaction Treatments in Distance Education From Bernard, Abrami, Borokhovski, Wade et al. (2009). RER.
  • 44. Summary of Results Overall treatment strength was g + = 0.38 ( k = 74) S-S and S-C treatments were higher than S-T Higher and medium strength ITs better than low strength Levels of strength differed (higher > lower) across combinations S-S + SC and S-T + SC Synchronous, Asynchronous and Mixed DE not different 11
  • 45. One Big Caveat in this Research We examined the nature of interaction treatments “instructional arrangements intended to foster interaction” We did not examine levels of interaction directly (how much actual interaction the treatments produced) Measuring interaction represents a challenge for the next generation of primary researchers interested in interaction in DE/OL
  • 46. Evidence-Based Principles for Increasing Achievement Through Interactivity First Principle: Increasing interaction of all kinds (S-S, S-T, S-C) leads to higher achievement gains Second Principles: Support the content; increase learners’ access to and engagement with whatever is intended to be learned Increase access to the teacher or other students depending on the orientation to teacher- or student-centeredness and desired learning activities Provide tertiary levels of support as resources (e.g., personnel, time, technology) allow
  • 47. What We Need to Push the Field Forward More research that compares at least one OL treatment to another OL treatment (no need to get too fancy) Better research designs (if not RCTs, designs that at least control for selection bias) More studies across the grade levels (k-12) and in higher education settings of all types Better-quality descriptions of treatments Better-quality measures Better-quality reporting , preferably full descriptive statistics
  • 48. References Anderson, T. (2003). Getting the mix right again: An updated and theoretical rationale for interaction. International Review of Research in Open and Distance Learning, 4(2), 9–14. Bernard, R.M., Abrami, P.C., Lou, Y., Borokhovski, E., Wade, A. et al. (2004). How does distance education compare to classroom instruction? A Meta-analysis of the empirical literature. Review of Educational Research, 74 (3), 379-439. Bernard, R.M., Abrami, P.C., Borokhovski, E., Wade, A., Tamim, R. et al. (2009). A meta-analysis of three interaction treatments in distance education. Review of Educational Research, 79 , 1243-1289. (doi:10.3102/0034654309333844v1). Cook, D.A. (2009). The failure of e-learning research to inform educational practice, and what we can do about it. Medical Teacher , 31 (2), 158-162. Cook, D.A., Levinson, A.J., Garside, S., Dupras et al. (2008). Internet-based learning in the health professions: A meta-analysis. Journal of the American Medical Association, 300(10), 1181-1196 (doi:10.1001/jama.300.10.1181). U.S Department of Education, Office of Planning, Evaluation, and Policy Development (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies, Washington, D.C. Moore, M.G. (1989). Three types of interaction. American Journal of Distance Education, 3(2), 1–6. Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). The comparative Effectiveness of web-based and classroom instruction: A meta-analysis . Personnel Psychology, 59, 623-664.
  • 49. Examples of Categorization Decisions: Student-Student Interaction Bell, Hudson and Heinan (2004) provided two methods of online learning to Physician Assistant students in a medical terminology course. Both versions of the course used the same materials, but some students worked independently on the Web (Group B) , while others (Group A) received 12 case studies in an online conference setting, which they then discussed through the use of asynchronous messaging. We applied the comparative outcomes to “student-student interaction,” counting the case-based discussion participants (Group A) as the experimental group.
  • 50. Examples of Categorization Decisions: Student-Teacher Interaction Rovai (2001) compared fully online instruction in education, delivered via an asynchronous learning network (Group A) with blended instruction which included monthly face-to-face meetings with the instructor ( Group B ) . Students in Group B were given extra chances to communicate (interact) with their teacher (as the means for communication online were equal in both conditions). Group B was designated as the experimental group for “student–teacher interaction.”
  • 51. Examples of Categorization Decisions: Student-Content Interaction Bernard and Naidu (1994) offered nursing students two different strategies for studying in a DE course on Community Mental Health. Participants received either created and studied concept maps (Group A) or answered predesigned study post-questions (Group B) . Group A interacted with the course materials to a greater extent than Group B and, thus, constituted the experimental condition.