The tricky relationship between research and practice
Download as PPTX, PDF1 like368 views
This document summarizes key points from a presentation on the relationship between educational research and practice. It notes that while research aims to inform practice, there are challenges to directly applying findings due to differences between controlled studies and real-world classrooms. Specifically, it discusses issues like varying contexts, the difficulty of measurement and quantification in education, and how research findings can become oversimplified as they spread. It advocates for stronger collaboration between researchers and teachers to help address these issues and ensure research has practical relevance and application.
The tricky relationship between research and practice
- 1. The tricky relationship between research and practice 29 June 2018 Christian Bokhove HCS conference
- 2. Evidence informed education • Evidence-based medicine • Evidence-based practice • Evidence-informed education
- 3. "Evidence-based practice is about making decisions through the conscientious, explicit and judicious use of the best available evidence from multiple sources by … Asking: Translating a practical issue or problem into an answerable question Acquiring: Systematically searching and retrieving the evidence Appraising: Critically judging the trustworthiness and relevance of the evidence Aggregating: Weighing and and pulling together the evidence Applying: Incorporating the evidence in the decision-making process Assessing: Evaluating the outcome of the decision taken . … to increase the likelihood of a favorable outcome." https://www.cebma.org/a-definition-of-evidence-based-management/
- 4. Evidence-informed still means ‘based on scientific research’ but we’re in the field of learning sciences, which is an interdisciplinary science • Sociology • Cognitive psychology • Etc.. “dealing with so many variables that are extremely hard to (all) control.” https://3starlearningexperiences.wordpress.com/2018/06/26/working-in-an-evidence-informed-way/
- 5. Some age-old issues 1 • Cause and effect • Hume’s Skepticism and Kant • Correlation and causation • Falsification • Popper and Kuhn • Does one study falsify a body of research? • Context • Berliner: ‘the hardest science of them all’
- 6. Some age-old issues 2 • A lesser form of knowledge? • Labaree: soft knowledge. • Negative: lower status, weaker authority, push quantification. • Positive: knowledge for practice, free from consumer pressures, no disciplinary boundaries, general public.
- 7. Some age-old issues 3 • Quantification and measurement • To improve credibility, origins ‘statistics’ • But could destroy local practical knowledge • What are we measuring (fMRI, SES, mindset, load) This requires a vast knowledge base…
- 8. But there are tensions with this
- 10. “Modal models for prospective teachers should use folk psychological terms whenever doing so will not mislead. For example, the core properties of working memory can be conveyed with the intuitive observations that there is a mental “space” for thinking, that this space is limited, and that it can be occupied by things perceived in the environment and/or things from the long-term memory.” But the key question then becomes: when misleading? (Willingham, 2017)
- 12. A thought-provoking example • Everyone loves Cognitive Load Theory • It is good science not bad science • So how can you be critical of it? • Being a Skeptic • Being skeptical does not mean I disagree with it • It forces people to reflect on their thinking • Let me ‘strip it and flip it’ (Willingham, 2012) REDUCE COGNITIVE LOAD This is often interpreted as to ‘take away’ something
- 13. Load from words R I S D E S Add a P and I can make SPIDERS A unit with meaning Load interacts with prior knowledge and schemas.
- 14. Load from picturebooks research (Flack & Horst, 2017)
- 15. Rest of slide Intentionally left blank to not impose too much load. Scale originates from Paas (1992)
- 16. Feedback might reduce load “Across the three experiments–with different problem-solving tasks and participant populations– we found that subjective ratings of effort investment were significantly higher after negative than after positive feedback; ratings given without feedback fell in between. These findings show that feedback valence alters perceived effort investment (possibly via task perceptions or affect), which can be problematic when effort is measured as an indicator of cognitive load.” (Raaijmakers et al., 2017)
- 17. Moment of measuring… “The findings from Experiment 1 (between-subjects) and 2 (within-subjects), using different arrangements of simple and complex tasks, showed that a single rating after a series of tasks resulted in a higher mental effort score than the average of ratings provided immediately after every task. A similar result was obtained in Experiment 3 with series of complex tasks, but not with simple tasks. Experiment 4 showed that knowing beforehand that mental effort rating will be required after completing all tasks results in lower scores, but average retrospective ratings per task still differed from a single retrospective rating.” (Van Gog et al, 2012)
- 18. Message • NOT that we can’t get something out of this, but… • “It’s complex” • Messages get distorted as they propagate through the system. • So even a statement like “REDUCE COGNITIVE LOAD” must be unpicked.
- 19. How do misconceptions start? “examples of cases in which entrepreneurs have knowingly set out to mislead educators are difficult to find.” (Howard-Jones, 2014, p. 817) “more likely that such interventions originate from uninformed interpretations of genuine scientific facts and are promoted by victims of their own wishful thinking who hold a “sincere but deluded fixation on some eccentric theory that the holder is absolutely sure will revolutionize science and society” (Howard- Jones, 2014, p. 817)
- 20. Perpetuated • Cultural conditions e.g. differences in terminology and language • Also check Lilienfeld et al. (2015, 2017) for lists with psychological terms to avoid and pairs of confusion • Counter-evidence difficult to access • Untestable • Biases • Complex
- 21. Robin-Garcia et al. (2017) • Dentistry • “tweeting about scholarly articles represents curating and informing about state-of-the-art appears not to be realized in practice.” • “Simplistic and naïve use of social media data risks damaging the scientific enterprise, misleading both authors and consumers of scientific literature.” • (Now check the paper, it says much more, and it has its own limitations)
- 22. Do teachers need research literacy skills? • Ideally, yes, as much as possible • But it is clear this can be quite challenging for all kinds of reasons, ranging from time, knowledge, skills constraints. So why not *work* with Academics?
- 23. Academia interested in links? • This is very likely • Universities seek impact • Researchers seek research participants • It is good for ecological validity that schools and classrooms participate. • But because of all kinds of privacy and safegaurding reasons increasingly difficult. • Societal responsiblity
- 25. PhD work • Finished in 2011 • Worked with 12 teachers in 9 schools in the Netherlands • Algebraic skills training with technology (Bokhove & Drijvers, 2010)
- 26. MC-squared project • Design and develop a new genre of authorable e-book, which we call 'the c-book' (c for creative) • Creative Mathematical Thinking (CMT) • Initiate a ‘Community of Interest’ (CoI) • A community of interest consists of several stakeholders from various ‘Communities of Practice. • England, Spain, Greece, France • Within these, teachers who co-design and use resources for teaching, can contribute to their own professional development. • Social Creativity • UK CoI: learning analytics and feedback (e.g. Fischer, 2001; Wenger, 1998) www.mc2-project.eu
- 28. Building blocks
- 29. Building blocks
- 30. Summary - chances • So despite the challenges it can be done… • Needs action from both sides (I know: workload! But benefits could outweigh the ‘cost’) • But we can benefit from each other… schools gain a research culture, universities gain societal impact
- 31. In the meantime • Try to follow up sources as much as possible • Refrain from too firm a position, until you feel you have reviewed a fair amount of material, from different actors, might be a good strategy. Sorry – this is just hard work, and I understand that practitioners do not always have this time. • Be mindful of over-simplifications. • Follow the facts and, if one simplifies, be aware of the limitations or what it leaves out. See what I said about Willingham (2017). • Be cautious about developing policy based on new claims. • Research findings should be accompanied by a clear scope and disclaimer with regard to claims. No silver bullet. • Look up a researcher • They don’t bite.
- 32. Thank you • C.Bokhove@soton.ac.uk • Twitter: @cbokhove • Website: www.bokhove.net • Want to collaborate? • Please let me know. There are only two types of people in the world: those who believe in false dichotomies, and penguins.
- 33. Selected references Bokhove, C., & Drijvers, P. (2010). Digital tools for algebra education: criteria and evaluation. International Journal of Computers for Mathematical Learning, 15(1), 45-62. Online first. Fischer, G. (2001). Communities of interest: learning through the interaction of multiple knowledge systems. In the Proceedings of the 24th IRIS Conference. S. Bjornestad, R. Moe, A. Morch, A. Opdahl (Eds.) (pp. 1-14). August 2001, Ulvik, Department of Information Science, Bergen, Norway. Flack, Z. M., & Horst, J.S. (2017). Two sides to every story: Children learn words better from one storybook page at a time. Infant and Child Development, 27(1) Howard-Jones, P. (2014). Neuroscience and education: myths and messages. Nature Reviews Neuroscience, 15(12), 817-824. Mintrop, R. (2016). Design-Based School Improvement: A Practical Guide for Education Leaders. Cambridge. Harvard Education Press. Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84(4), 429-434. Raaijmakers, S. F., Baars, M., Schaap, L., Paas, F. & Van Gog, T. (2017). Effects of performance feedback valence on perceptions of invested mental effort. Learning and Instruction, 51 36-46. Robinson-Garcia, N., Costas, R., Isett, K., Melkers, J., & Hicks, D. (2017). The unbearable emptiness of tweeting—About journal articles. PLOS one. https://doi.org/10.1371/journal.pone.0183551 Van Gog, T., Kirschner, F., Kester, L., & Paas, F. (2012). Timing and frequency of mental effort measurement: Evidence in favour of repeated measures. Applied Cognitive Psychology, 26, 833-839. Wenger, E. (1998). Communities of Practice: Learning, Meaning, Identity. Cambridge University Press. Willingham, D. T. (2012). When can you trust the experts: How to tell good science from bad in education . Willingham, D. T. (2017). A Mental Model of the Learner: Teaching the Basic Science of Educational Psychology to Future Teachers. Mind, Brain and Education, 11(4), 166-175.
Editor's Notes
- #2: The recent move towards more 'evidence informed teaching' has seemingly increased the interest of schools in researchers and academia, and of academia in schools. This development presents us with several questions. How can we best organise this relationship? Do teachers need research literacy skills? Why would academia be interested in cultivating teacher-researcher links? This talk will explore these questions and give several examples of research projects, most concerning mathematics in secondary education, where fruitful collaboration between me as researcher and classroom teachers, brought benefits to the school and higher education. I will try to convey what opportunities and chances there are in working together, but also -in light of workload discussions- some of the risks.