Patrick Raven and Matt Wingfield - Using Controlled Digital Frameworks to Assess Practical Synoptic Tasks | VocTEL 2015
Download as PPTX, PDF1 like416 views
The document discusses advancements in e-assessment, highlighting the use of digital frameworks for assessing practical tasks and presenting various assessment types, including adaptive testing and evidence-based assessments. It introduces the concept of Adaptive Comparative Judgement (ACJ) as an alternative assessment approach that reduces subjectivity by ranking paired student work. The benefits of structured synoptic e-assessment include enhanced visibility of the learning process, improved readiness indicators, and increased transparency in tracking learner progression.
Patrick Raven and Matt Wingfield - Using Controlled Digital Frameworks to Assess Practical Synoptic Tasks | VocTEL 2015
- 1. PATRICK CRAVEN – DIRECTOR ASSESSMENT POLICY & RESEARCH VocTEL 2015 Using Controlled Digital Frameworks to Assess Practical Synoptic Tasks Presentation – 17th June MATT WINGFIELD – CHIEF BUSINESS DEVELOPMENT OFFICER
- 3. ASSESSMENT: AN INEXACT SCIENCE RELIABILITYTrust & Regulation Agenda Rigour and Quality Assurance VALIDITY Fit for Purpose and Authentic Industry & Assessment Agenda VIABILITY Robust and Sustainable Service & Business Agenda
- 4. THE SCOPE FOR INNOVATION Convergent Assessment Divergent Assessment Multiple Choice, Hot-Spot, Drag & Drop, Interactive ... OBJECTIVE SUBJECTIVE ASSESSMENT TYPES E-ASSESSMENTCOMPLEXITY Single Word/Value Response Explanatory Phrase/Sentence Formulae Explanatory Paragraph Technical Factual Essays Creative Essays Skills/Functions Tests Technical Performance Creative Professional Performance AUTOMATED ASSESSMENT HUMAN MARKING
- 5. THE EVOLUTION OF E-ASSESSMENT • E-assessment is no longer just on-screen MCQ: • Adaptive Testing • Simulations • Evidence-based assessments • All are robust, scalable, proven and auditable • There are great examples out there that... • Support self-reflective learning • Evidence the learning process • Assess the application of knowledge & understanding
- 6. THE WISDOM OF ROUTINE
- 7. THE THEORY OF EVOLUTION In the struggle for survival, the fittest win out at the expense of their rivals because they succeed in adapting themselves best to their environment. (Charles Darwin)
- 9. • A leading provider of evidence-based assessment tools to support personalised assessment for learning; • Used by institutions around the world to track and assess learner progression and to provide feedback; • Used by professional/accreditation bodies since 2004 to support formal assessment of knowledge & skill application. DIGITAL ASSESS
- 10. SO, WHAT DOES THIS LOOK LIKE?
- 12. ILLUSTRATE UNDERSTANDING OF SKILLS
- 13. CAPTURE PLANNING & PREPARATION
- 15. CAPTURE REVIEW & ANALYSIS
- 16. CAPTURE THE LEARNER JOURNEY
- 17. • People are good at: • Relative Judgements; • Ranking work. • People are less good at: • Absolute Judgements, e.g. Scoring. • In addition, when assessing student work we: • are influenced by past experience; • work independently – magnifying the effect of subjectivity. THE ASSESSMENT CHALLENGE
- 18. Comparative Judgement: Perhaps we should take a different approach? • Measuring without subjectivity – adaptive scaling without needing scores; • Presents a pair of items and asks the observer to make a value judgement – which is ‘better’? • Repeat this through all possible combinations, and record the results to create a rank order; • The Law of Comparative Judgement (Thurstone - 1927). THE ASSESSMENT CHALLENGE
- 19. We have created a web-based ADAPTIVE comparative judgement solution for assessing student work...presents successive pairs of student work to assessors and asks them to say which is best at meeting assessment criteria: ACJ – Adaptive Comparative Judgement: HOW IT WORKS
- 20. • The system dynamically organises the work into a scaled rank order as the students complete their judgements. • Which can produce a grade/score if required. HOW IT WORKS
- 21. • The benefits of structured synoptic e-assessment: • Enables quicker work-readiness through use of authentic tasks; • Provides a legitimate place for technology in assessment; • Facilitates richer, stronger formative support of learning through improved visibility of process; • Provides ‘forward indicators of readiness’: The right skills; The right attitude; In the right time; • Provides greater transparency: • extending learning to address weaknesses. • of collaboration and teamwork • of progress, decision making, critique and explanation WHY DOES THIS MATTER?
- 22. The eAssessment Assocation Join the eAA for FREE @ www.e-assessment.com
- 23. THANK YOU FOR LISTENING Patrick Craven Patrick.Craven@cityandguilds.com www.cityandguilds.com Matt Wingfield Matt.Wingfield@digitalassess.com www.digitalassess.com
Editor's Notes
- #3: Slide two But that’s enough about H&W, I’d like to tell you a little bit about City & Guilds and why I am here today. We were founded by the Livery Companies out of the second industrial revolution to help Britain get the skills it needed to grow and compete in a changing world. We partner with training providers, employers and governments to help individuals get the skills they need to get into a job, progress on the job, and move into the next one. We’re growing and for more than two decades we’ve been operating internationally with a presence in 81 countries. Like H&W, Our rich heritage has taught us to be as agile and responsive as the organisations we help to grow. That’s why we are dedicated to developing new products and services to help people learn whenever and wherever they can. Essentially, we exist to connect skills and jobs so people and organisations can progress. Our purpose is to enable people and organisations to develop their skills for personal and economic growth.
- #6: Hundreds of thousands of users – stress tested to millions of users Feltag targets: 10% online learning by 2016 50% online assessment by 2018