Where’s the evidence of evidence-based practice? Exposing the ruinous twins of evidence and policy in teacher education

Editor's Notes

• #2: In this talk I examine how ideas associated with what are commonly termed evidence-based practice (EBP) and evidence based teaching (EBT) have been re-formed and interpreted by governments and state funded gateways for teachers in the lifelong learning sector in England (such as LLUK, SVUK, LSIS, DFE, Ofsted). I chart the relationship between interpretations of educational research and EBP/T and teacher education policy and practice in the sector from the early 1990’s until the most recent reviews of vocational education (the Wolf Review) and Professionalism in Further Education (Lingfield Report 2012). Links are made between notions of routinised practices (Iredale 2012) and the ‘ruinous twins’ of evidence and policy. The conclusion will caution against the influence of both simplistic ‘evidence-based’ approaches on teachers, systematic review, and the rising tendency for policymakers and managers in the sector to lose interest in wider more critical educational research.
• #3: Literature suggests that the process of reification of situational knowledge formed out of EBP/T may lead to the endorsement by teacher educators of routinised ‘safe’ teaching methods and the avoidance of risk in professional practice.Furthermore that EBP/T is being used to value what is measured, not to measure what is valued in education.Kemmis cautions against research that changes practice from without and ‘spectator research’ that names and judges – Kemmis. Much of this sort of research relates to school teaching rather than our sector, but it is reasonable to assume that with the new common mandatory content for the level 5 Diploma in Education and Training (the new qualifications), the emphasis of research and scholarship including EBP) will increase research in and for our sector.
• #4: EBT is a Disputed concept. A little like the pshmepullu it faces in opposite directions and is both castigated and celebrated in the same breath in the research.Thomas (2004) distinguishes the criteria for evidence in science and philosophy. He relates evidence in scientific research with Levi-Strauss’s metaphor of the bricoleur and the engineer. The bricoleur uses inductive reasoning and is not limited in his search for what might constitute evidence for his arguments, whereas the engineer begins with a theoretical proposition, applying deductive reasoning to verify empirical sources. Thomas goes on to clarify the crossover in technique for both types, concluding that the common problems in educational research for both the teacher as bricoleur and the teacher as engineer are those of veracity, sufficiency and generalisability. Similarly in philosophical research the emphasis is on the sufficiency of evidence in the quest for knowledge about the world.For further reading on this you might want to read:What works – Biesta 2007Deterministic top-down state driven managerialism – Avis 2010EBT marshalled to promote routinised or safe practices and to marginalise risk taking (Iredale 2012).
• #9: From the EBTNLearning styles (Learning preferences)There is not significant evidence to support the idea that different students have different learning styles and that teaching them in that style improves learning.  The brain has three main 'inputs': vision, hearing and touch.  All three should be used and any weak ones developed.Left brain, right brainLike 'learning styles', this myth builds on some truth.  The brain does have 2 halves.  Different activities take place in different halves.  However, most thinking requires several different areas, in different parts of the brain, to work together.  Weak areas need practice, not avoidance.HydrationThe belief that students need water on their desks in order to keep hydrated, is not proven.  It requires significant dehydration to affect the brain.Brain GymThe claims that certain exercises stimulate certain parts of the brain are pseudo-science.  However, gentle exercise wakes students up and  routines develop self-control.  Keep the exercise, but ditch the theory.Neuro-Linguistic ProgrammingCreated in the 1970's, this scientific-sounding approach is largely pseudo-science.  No evidence has been found for theories about eye-movements etc and the majority of studies show it to be ineffective.Fish OilWhile it is true to say that the Omega 3 fatty acids in fish-oil are used in the brain, they are also available from non-fish sources.  No evidence exists which shows that giving fish-oil to pupils, in general, improves thinking/learning.  However, if their diet is low in Omega 3, then any source will help.
• #12: The final sense of “folk psychology” is closely associated with the work of David Lewis. On this view, folk psychology is a psychological theory constituted by the platitudes about the mind ordinary people are inclined to endorse.
• #13: Until the late 1990’s ITE for further education was voluntary, relying on college management attitudes. Government fee grants and bursaries helped to maintain a relatively healthy ITE provision across the country. 59% of FE teachers held a teaching qualification beyond what was then the assessor award.European comparison: Most ITE programmes were, and still are in the main, formed from diverse subject specialisms. This is different from most of Europe.Vocational teachers are expected to train at masters level. Higher status of vocational education
• #14: The context for ITE LLS has been the focus of research that consistently identifies neglect by all political parties and constituencies (see Avis 2002, Lucas 2004, Gleeson et al, Keep 2006). Paradoxically it has faced levels of control beyond that of most of the public sector. Since the early 1990’s the prevalent accusation is that of structural regimes characterized by managerialism, audit and control (see Avis 2002:79). Avis traces managerialism back to the scientific movement and Taylorism. Ball (1990) and Enteman (1993), provide reductionist definitions where bureaucratic society is replaced and becomes a function of the sum of the application of organizational practices. Citing Child (1969) Fitzsimons (1999) characterizes managerialsim as both technical and social, binding practices and processes together in a system that maintains itself through authoritative, rather than bureaucratic governance. The relationship between managerialism and the LLS is illustrated well by Pusey (1991: 22), albeit from an Australian perspective, arguing that “There can be no quarrel with the notion of efficiency as such. The inherent problem lies instead at another level – with the criteria that define what count as costs and benefits; with the loss of social intelligence; and with the number and range of potentially constructive discourses that have been suppressed”.The push towards managerialism could be said to have begun at state level In 1992 following the introduction of the Further and Higher Education Act. Clow, commenting about the effects of the 1992 Act noted that many FE teachers did not have pedagogic knowledge, with only 59% holding a recognized teaching qualification beyond the Training and Development lead Body units D32/33 (Clow 2001:409). She argued that much of what might be considered professional about teaching in FE was restricted, rather than extended (after Hoyle 1974), “there seems to be no evidence for arguing that FE teachers belong to a profession” (2001:409).
• #15: It can be argued that an outcome driven instrumental approach to criteria at all levels in the LLS has suppressed the ‘social intelligence’ and ‘constructive discourses’ provided by teachers and allied professionals in the sector. Regulation and control structures have replaced professional autonomy, particularly in the LLS where observable processes (after Foucault 1979) are rewarded as norms above subtler, less visible reflective activities. Lucas et al (2012:693) argued that:“after a decade of reform, successive standards and regulatory frameworks have not brought about coherence and in many respects have fragmented the system even further. The overwhelming message from those who have had to design ITT programmes in response to quickly changing standards and assessment requirements is that being forced to play a game of complying with external standards and regulations has diverted attention from addressing more fundamental weaknesses such as developing stronger mentoring support and achieving a better synergy between the taught and practice elements of courses.” They further concluded that the effects of regulation reform fail to impact on those intended as recipients, suggesting: “that a more flexible, less prescriptive approach is required that allows for specifications to be interpreted within different contexts, actively encouraging variation and innovation to meet the diverse needs of trainees”.
• #17: Lucas (2007) describes the learning environment for student teachers within the Lifelong Learning Sector as both “expansive” and “restrictive”. He describes an expansive learning environment as a place where there are opportunities to engage in ‘multiple communities of practice at and beyond the workplace, access to a multidimensional approach to the acquisition of expertise, and the opportunity to pursue knowledge-based courses and qualifications’ (2007:99). There are opportunities for student teachers to participate beyond their narrow subject range, particularly with regard to functional skills teaching and support, tutorial and extra curricula activities, but more commonly student teachers are located with their subject specialist mentor, restricted to the classes offered by the subject team. They are also restricted by a teacher education curriculum that transfers largely uncontroversial professional standards (Simmons and Thompson 2007). Whilst the standards referred to by Simmons and Thompson were the FENTO standards, their replacement, the LLUK New Overarching Professional Standards (LLUK 2006) continue to restrict student teachers to a set of criteria aimed at experienced teachers, with no accommodation for levels of study, inexperienced teachers and student teachers. Furthermore Ellis (2010), referring to schoolteacher education, sees the landscape of teacher education as a process of acculturation to the ‘existing practices of the setting with an emphasis on the reproduction of routinisedbehaviours and the development of bureaucratic virtues such as compliance and the collection of evidence’ (2010:106).
• #18: Thomas (2004) distinguishes the criteria for evidence in science and philosophy. He relates evidence in scientific research with Levi-Strauss’s metaphor of the bricoleur and the engineer. The bricoleur uses inductive reasoning and is not limited in his search for what might constitute evidence for his arguments, whereas the engineer begins with a theoretical proposition, applying deductive reasoning to verify empirical sources. Thomas goes on to clarify the crossover in technique for both types, concluding that the common problems in educational research for both the teacher as bricoleur and the teacher as engineer are those of veracity, sufficiency and generalisability. Similarly in philosophical research the emphasis is on the sufficiency of evidence in the quest for knowledge about the world. Historically the movement towards EBP began with Cochrane (1972) who advocated a move towards evidence resulting from practice to be available to the general public, rather than a narrow field of researchers. He believed that the public needed to be reassured that decisions made by health professionals were based on sound evidence and not just personal beliefs about the efficacy of decisions. The model required that practitioners should be able to show that their actions and decisions were effective, efficient and equitable. Several groups began to collaborate from the 1970’s onwards, mainly in the scientific field but with the shift in Government emphasis on policy making in the late 1990’s social science research began to establish large-scale data analyses. Of the three principles outlined by Cochrane in the 1970’s, effectiveness, efficiently and equity, it is only recently that the third (more challenging to measure) one has come to the fore in the drive to contextualize and localize practice decisions. Whereas the scientific and social scientific research community has developed EBP for over 50 years educational research, perhaps because of the difficulties associated with measuring effectiveness, efficiency and equity, has been slower to develop.
• #19: To focus on educational research and EBP I take as my starting point the flurry of contributions and rejoinders that came initially from a lecture by David Hargreaves to the Teacher Training Agency in 1996 (Hargreaves 1996). In this lecture Hargreaves compares research within the teaching profession with that in the medical profession – both claimed as ‘people-centred professions’ (2007:44). His main argument is that what counts as research in the educational research community has failed to serve the needs of teachers. This is based upon a premise that improvements are required in the quality of teaching in schools. In this sense his premise is bounded up in technicist notions of efficiency and effectiveness across a large and diverse sector (albeit less so than the LLS). His view resonates with Kemmis’s discussion on spectator research (2012) and its effect on the ‘conversational space’ (2012:894). Hargreaves’ comparison with the medical profession assumes similarities in that they both use their respective professional knowledge and skills to respond to individual needs. It also points to fundamental differences in that whereas traditional western medicine can claim to possess an agreed professional knowledge base teaching does not. For every educational researcher who claims a sociological basis to professional knowledge, there will be another who attests to psychological and therapeutic bases (see Avis, Biesta, Dweck and Ecclestone). Whatever the perspective there is a growing emphasis in the LLS on knowledge derived from practice, skills and attitudinal knowledge, and personalised learning for employability. It is useful to review some recent developments in EBP in the LLS, starting with the then Department for Education and Skills (DfES) who produced ‘Equipping our Teachers for the Future’ (Ofsted 2004)which, amongst reforms to initial teacher training, required all new teachers and trainers from to be registered with IFL. Government regulation followed in the Further Education Teachers’ Continuing Professional Development and Registration (England) Regulations 2007, and the Further Education Teachers’ Qualifications (England) Regulations 2007. Alongside these moves to regulate the teaching workforce the Government produced the White Paper Further Education: Raising Skills, Improving Life Chances (DES 2006)which sought to drive forward reforms to the sector in terms of a set of objectives, amongst which was the “Improved use of a range of practices and techniques, including e-learning, use of new technologies and target-setting to provide a personalised learning experience” (DES 2006).Evidence from a recent evaluation report found that for this objective:There is evidence from interviews with teachers, department and faculty heads and team leaders, that new staff systematically being enrolled on and obtaining the ITE qualification equips staff with increased confidence, the ability to use different teaching methods to support learners with varying needs and learning preferences, and increased reflective practice (DBIS 2012).The systematic approach to teacher development, deriving from regulatory activity is, in this quotation, linked to confidence, technical proficiency and reflective practice. It is in the area of technical proficiency - the use of teaching methods, the ability to respond to learner needs, and use of e-learning - that EBP/T has found the most purchase
• #20: The final report by Lord Lingfield into professionalism in further education (DBIS 2012), charts the plethora of changes in the national policy landscape with six different Government departments having had a hand in regulating the sector since 2004 (DBIS 2012:16). In addition since Success for All (2002) and theOfsted survey of Initial Training of FE Teachers – (2003) no fewer than twenty one governmental and sector-related publications have provided commentary on aspects of teacher development. One specific example of the morphing of improvement levers was the Standards Unit in 2003 (, set up as part of a Government initiative to improve the sector, which then became subsumed into the Excellence Gateway in its earlier incarnation, and, more recently as part of the Learning and Skills Improvement Service (LSIS). Since this time the coalition government (2011 – present) has at least recognised the faulty premise of deficit policy, as it acknowledges that much of the sector’s provision is judged to be good or outstanding by successive Ofsted inspections (DBIS 2012:). Coffield and Edward (2009:386) encourage researchers and practitioners to restore the notion of complexity and connectedness when considering how to interpret and apply good, best or excellent practice in the classroom. They urge that where regulatory influence seeks to homogenise practice into measurable standards, teachers should develop their practice by forming questions within a shared community against a topology of dimensions of good practice. Derived from Alexander (1997) Coffield and Edward argue for professional autonomy and stability, as a counterbalance to the ‘ratchet screwdriver’ representation of successive government policy (2009:386).