Echo presentation hierarhical process modelling case study
1 like1,043 views
The document discusses the Oasis Echo Project, which aims to improve educational outcomes by fostering learning communities across three Oasis academies. It emphasizes the importance of systems thinking, resilience, and stakeholder engagement in evaluating school performance through a comprehensive data collection approach. The project presents a perimeta model to analyze educational data, enhance leadership decision-making, and encourage collaborative learning among educators, students, and parents.
Echo presentation hierarhical process modelling case study
- 1. Evaluating the wider outcomes of schooling: the Oasis ECHO project Raising standards through creating learning communities Ruth Deakin Crick, Stephen Barr, Howard Green
- 2. ! Background • The Education Charter • The Echo Project • Three Oasis Academies
- 3. ! What to do? "Here is Edward Bear, coming downstairs, bump, bump, bump, on the back of his head, behind Christopher Robin. It is, as far as he knows, the only way of coming downstairs, but sometimes he feels that there really is another way, if only he could stop bumping for a moment and think of it" Alan Alexander Milne The opening paragraph of “Winnie-the-Pooh”, 1926
- 4. • Systems thinking – managing complexity • Developing resilience – becoming learning organisations • Responding to emergence – unpredictable outcomes • Taking responsibility for our own development • Engaging all stakeholders in a networked improvement community
- 5. ! Key Questions • How can we measure the full range of what we value in schools? • How can collect and manage evidence to understand the big picture as well as the detail? • How can we lead and manage complexity? • How can we used data for real-time leadership decisioning as well as for evaluating performance
- 6. …… Learning is a core process at multi-levels in learning Macro Layers and feedback loops organisations Community Learning Leaders Learning Teachers Learning Students Learning Blockley, D. (2010) The Importance of Being Process, Civil Engineering and Environmental Systems, 27(3)189-199
- 7. ! The complex problem of creating learning communities
- 8. ! Imagine ……
- 9. What sort of design should we use to model our learning communities?
- 10. A Hierarchical Model: Defined by Purpose
- 11. How data 3. Inputs Sources of evidence Students Teachers 2.3 Developing the learning of leaders so that they lead the learning of teachers and students effectively Leaders 3.11 Contributing actively to a community that 3.12 Feeling included and involved best and learns together, shares what works with my children’s education to do Academy and knowing knows what it needs at the to improve that there is an open door for contact when I need it 3.13 Being effective role models as leaders of 3.10 Providing evidence that training and development opportunities have helped me to improve my classroom practice and/or my effectiveness as a leader and manager 3.9 Contributing actively to a community where shared leadership is promoted 2.2 Developing the learning of teachers so that they realise their full potential 3.7 Feeling that my successes are recognised 3.8 Contributing actively to a community which focuses on service to others 2. Outputs 2.1 Developing the learning of students so that they realise their full potential 3.6 Achieving results that meet my aspirations and expectations 3.4 Knowing that we are being helped to reach our full potential 3.5 Maintaining a good rate of progress at key transitions, for example, primary to secondary school 3.3 Knowing that I am becoming an effective learner 3.2 Feeling part of a community that focuses its activities on learning and achievement 3.1 Feeling valued and included in our community PURPOSE Outcome 1.1 Establishing and sustaining a group of high achieving learning communities that enables everyone to realise their full potential and refuses to put limits on achievement 2.4 Engaging parents/ carers effectively in the learning activities of the Academy and in supporting the learning of their children Parents/ carers
- 12. What type of data do we need to collect to measure the core processes which fulfil our purpose?
- 13. Stakeholders Type of data Focus 1 Y7 students Numerical scores per dimension Learning power 2 Y8 & 10 Students Numerical values for each question students as learners and values & citizenship 3 Y9 students Stories rated by researchers Personal transformation 4 Y11 leavers Numerical values for each question Aspirations, trajectories 5 Parents/carers Numerical values for each question Participation 6 Teachers Numerical values for each question Impact of professional learning on student outcomes 7 Leaders Numerical values for each question Leaders as learners 8 KPI s Numerical values of performance Behavioural indicators 9 Y11 Numerical values of performance Attainment
- 14. ! The nature of uncertainty Fuzziness Incompleteness Randomness
- 15. ! .. combined with performance measures that cope explicitly with uncertain evidence Evidence that A is successful Lack of Evidence Evidence that A is not successful 0 1 Performing well with little uncertainty Performing poorly with some uncertainty We don’t really know what is going on!
- 16. Data collected to evaluate our core processes Means & SDs Yes Uncertain Perimeta model No A Leadership Ownership and Decisioning P D A S Improvement Protocols Rapid Prototyping Harnessing collective intelligence across the system
- 17. So what does our Perimeta model tell us about Parents/carers ..… an example
- 19. ! Perimeta model results
- 20. ! Drilling down into the evidence to understand key issues Outcome Outputs Inputs Establishing and sustaining learning communities Developing the learning of students so that they realise their full potential Feeling valued and Feeling part of a Knowing that I am Knowing that we are included in our community focused on becoming an effective being helped to reach community learning & achievement learner our full potential Statement I was helped and supported when I struggled My teachers taught me well The Academy made good use of technologies I was valued and included I was often challenged to do better There were great facilities I felt part of a learning community I developed responsibility towards others I had a wide range of opportunities for learning I felt safe POS 0.98 0.90 0.90 0.89 0.89 0.89 0.89 0.89 0.89 0.89 Statement I am looking forward to the move to Sixth Form, Further Education, training or work I led activities during my time at the Academy No others above 10% NEG 0.18 0.17 Statement I supported activities that would improve things for other people I led activities I am looking forward to the move to Sixth Form, FE, training or work My parents/carers felt involved Everyone worked very hard to make it a great place to learn and to do well I enjoyed my time I could not have gone to a better school I was offered opportunities for leadership I never felt that anyone was putting limits on what I could achieve My teachers accurately predicted my GCSEs UNC 0.38 0.34 0.33 0.15 0.15 0.15 0.14 0.14 0.13 0.13
- 21. ! The PerimetA toolkit supports collaborative action to improve learning Focusing on PERFORMANCE vs broad outcomes KEY FEATURE: “Italian Flag” evidence model & rich uncertainty calculus KEY FEATURE: Excel interface for linked data and predictive models Understanding COMPLEX PROCESSES Weighing up uncertainty of EVIDENCE Informing robust CHOICES Enabling robust & collaborative ACTION KEY FEATURE: Hierarchy of interactions focused on outcomes
- 22. So how do we respond to what Perimeta is telling us as a learning community?
- 23. Professional Enquiry through Rapid Prototyping to address key challenges highlighted by Perimeta…. Plan Act Do Study …..within a disciplined evaluation framework
- 24. Measurable System Improvement: New Knowledge and Know How P D A S P D A S P D A S P D A S P D A S P D A S P D A S P D A S P D P D A S A S P D P D A S A S Experiential knowing, hunches, ideas Bryk, A. Gomez L. Gunrow A. (2010) Getting Ideas into Action: Building Networked Improvement Communities in Education, Carnegie Foundation for the Advancement of Teacher Education, Carnegie, Stanford.
- 25. How can we use technology to enhance our ability to develop organisational learning through Perimeta
- 26. ! An academy with systemic analytics X Academy Learner Dispositions Teacher Dispositions Reflective Enquiry Blogs Strategic Progress Stories of Change & Evidence
- 27. ! An academy network with systemic analytics Evidence Hub Data Data Data Analytics Integration Data 2 7
- 28. ! Systems Learning & Leadership Evidence Hub http://sysll.evidence-hub.net Share and debate evidence… About…
- 29. ! Systems Learning & Leadership Evidence Hub http://sysll.evidence-hub.net Replay Oasis/Univ Bristol/Open Uni event: http://learningemergence.net/2013/07/17/deed-elli-ai-ci-systemic-school-learning 2 9
- 30. ! Learning Analytics Hub (3.0) Online Tools designed to measure core processes and to stimulate self-directed change Learning Analytics System Simulations Recommendation Engines Data Sets + Data Streams Buckingham Shum, S. & Deakin Crick, R. (2012) Learning Dispositions and Transferable Competencies: Pedagogy, Modelling, and Learning Analytics, Proceedings of LAK12: 2nd International Conference on Learning Analytics & Knowledge, April 29 - May 2, 2012, Vancouver, BC. ACM: New York. ISBN 978-1-4503-1111-3.
- 31. ! Conclusions • The ECHO Project as a proof of concept: The ECHO Project has been an attempt to weave all the evaluation strands together in order to create a whole and more balanced snapshot of the school’s performance for improvement planning … initial analyses suggest that conceptually, professionally, practically and technically there is real promise in this approach … use of the Perimeta model enables links to be established between extended self-evaluation, performance management and school improvement planning.
- 32. ! Conclusions • Further research is needed, plus drilling down into the pilot project data, particularly the relationship between qualitative and quantitative data, e.g. what is the relationship between Year 11 leaver responses (Strand 4 questionnaire) and GCSE results (Strand 9 data)? • Traditional statistical analysis has shown that the questionnaires and interview schedules can be refined and reduced in length
- 33. ! Conclusions • Evaluation of ‘teachers as learners’ and ‘leaders as learners’ is relatively new and would benefit from review and further development, e.g. addition of a 360degree perspective to the processes • Approaches and processes developed through ECHO offer the potential to extend and enrich the insights of senior leadership teams (SLTs) into next steps for improvement and for the Oasis group as it seeks to identify shared issues that would benefit from coordinated resourcing
- 34. ! Conclusions • Perimeta feedback offers an opportunity to celebrate and share what’s going well, to address areas of weakness and, uniquely, to explore further aspects of the school where there is uncertainty from the evaluation evidence. Perimeta is designed to draw people into exploring and managing their own data. It is essentially a decision-making tool. It measures for the purpose of stimulating positive change according to locally defined need.
- 35. ! Next steps • Reporting to and feedback from 3 pilot Academies • ECHO questionnaires and interview schedules to be refined, put online and rolled out across the group • New questionnaire for Academy Councillors to be piloted in 2013/14 • Questionnaire for wider community involvement to be developed and piloted in 2013/14
- 36. ! Next steps • Further development of Perimeta as a model directly available to schools • Further research on issues arising from the ECHO Project • SLTs encouraged to develop the rigorous and systematic use of qualitative as well as quantitative evaluation evidence for performance review and improvement planning • OCL to build similar approaches into performance review and improvement planning across the group
- 37. ! Next steps and, finally, our thanks • Embedding of the ECHO work within Oasis, via OCL and, possibly, OCHE/CLICT • Wider dissemination of the ECHO results
- 38. With thanks to Students, staff and parents/carers at the ECHO especially, students, staff and parents/ Academies carers at the 3 ECHO Academies; staff in the Graduate School of Education at Bristol; ‘Perimeta people’; University Graduate School of Education – staff at OCL/ of Oasis Centre; Jerusalem Trust for funding Systems Centre – University of Bristol University of Bristol Research Committee Oasis Community Learning /Oasis Centre The Jerusalem Trust Bristol