Reall: rubric for the evaluation of apps in language learning
1 like2,438 views
This document outlines a research study conducted to develop a rubric for evaluating mobile apps for language learning (REALL). The study involved three stages: 1) analyzing and categorizing existing English language learning apps, 2) designing an evaluation rubric based on pedagogical criteria, and 3) creating a parallel rubric to assess apps based on linguistic criteria defined by the Common European Framework of Reference for Languages. The rubrics were then used to evaluate the top five apps identified in stage one. The study aims to provide a comprehensive framework for assessing the pedagogical quality, technical features, and linguistic suitability of apps for teaching and learning English as a foreign language.
Reall: rubric for the evaluation of apps in language learning
- 1. REALL: Rubric for the Evaluation of Apps in Language Learning Elena Martín Monje (UNED), Jorge Arús Hita (UCM), Pilar Rodríguez Arancón (UNED) & Cristina Calle Martínez (UCM) ATLAS Research Group Logroño, 2-3 May 2013
- 2. Table of contents Introduction Pedagogic assessment of mobile learning apps for EFL ◦ Stage 1: Analysis and categorization of EFL apps available ◦ Stage 2: Design of an evaluation rubric for the pedagogic assessment of EFL apps ◦ Stage 3: Rubric for the evaluation of apps in language learning (REALL) Conclusion References
- 3. Introduction Mobile learning Mobile-Assisted Language Learning (MALL) SO-CALL-ME project Stages in this research study: Stage 1: Analysis and categorization of EFL apps available Stage 2: Design of rubric for pedagogic assessment of EFL apps Stage 3: Creation of REALL (Rubric for the Evaluation of Apps in Language Learning
- 4. Previous rubrics & categorizations Taken from http://langwitches.org/blog/2012/03/31/ipad-apps-and-blooms-taxonomy/
- 5. Taken from Tony Vincent’s learninginhand.com
- 6. Stage 1: Apps available 3 criteria: apps’ cognitive value, similarity/ complementarity with SO-CALL-ME 67 apps assessed Frequent technical problems downloading/ starting apps Vast majority for Apple devices Most expensive: dictionaries, grammar Many for 2-3€ or with initial free sample pack Categories: ◦ Games ◦ App versions of printed materials (e.g. coursebooks) ◦ Apps for pronunciation/ grammar/vocabulary
- 7. Stage 2: Rubric for pedagogic assessment Quality guide for digital learning objects (Fernández- Pampillón et al., 2011) • Cognitive value and pedagogic coherence (1) • Content quality (2) • Capacity to generate learning (3) • Interactivity and adaptability (4) • Motivation (5) Pedagogical criteria • Format and layout (6) • Usability (7) • Accessibility (8) • Visibility (9) • Compatibility (10) Technical criteria
- 8. Rubric with descriptors and scale 1-5. E.g. Criterion 3 in rubric for pedagogic assessment: The 5 top EFL apps of Stage 1 assessed with this rubric: 1 2 3 4 5 3. Capacity to generate learning Contents do not help to achieve learning goals or autonomous learning Contents help autonomous learning but not clearly the achievement of the initial learning goals Contents help to achieve the learning goals but neither autonomous learning nor relating old knowledge to new knowledge Contents help to achieve the learning goals but not autonomous learning OR not relating old knowledge to new knowledge Contents help to achieve the learning goals, autonomous learning and relating old knowledge to new knowledge
- 10. Stage 3: Rubric for linguistic assessment Common European Framework of Reference for Languages (Council of Europe, 2001) Focus on levels A2-B2 Initial rubric for oral comprehension Categories: Level Type of texts Topics Delivery E.g. Delivery in rubric for linguistic assessment: 1 2 3 4 5 N/A Delivery Language difficulty, clarity and speed mix different levels. If adaptive, delivery is clearly not well adapted Language difficulty, clarity and speed rarely belong to the same level. If adaptive, delivery rarely corresponds to the right level Language difficulty, clarity and speed tend to belong to the same level. If adaptive, delivery more often than not corresponds to the right level Language difficulty, clarity and speed usually belong to the same level. If adaptive, delivery usually corresponds to the right level Language difficulty, clarity and speed belong to the same level. If adaptive, delivery corresponds to the right level
- 11. Parallel process in stage 2 & stage 3: 1 rubric, 2 evaluators • Stage 2: 91/100 • Stage 3: 29/40 • Stage 2: 89/100 • Stage 3: 37/40 • Stage 2: 82/100 • Stage 3: 13/40 • Stage 2: 81/100 • Stage 3: 19/40 • Stage 2: 64/100 • Stage 3: 14/40
- 12. The most comprehensive apps assessed were Speakingpal and Learn English Elementary Podcasts
- 13. Conclusions Pedagogic & technical quality ≠ linguistic value & adequacy for EFL teaching & learning Importance of all three dimensions: ◦ Pedagogical ◦ Technical ◦ Linguistic Piloting of stage 2 & stage 3 rubrics enables a future study on a larger scale Future theoretical framework for successful, pedagogically & linguistically sound EFL apps
- 14. References Arús-Hita, Jorge; Pilar Rodríguez-Arancón and Cristina Calle- Martínez (in press) “A pedagogic assessment of mobile learning applications”. In Proceedings of ICDE 2013, Mobilizing Distance Education, UNED, Madrid. Council of Europe (2001). Common European Framework of Reference for Languages. Cambridge: Cambridge University Press. Fernández-Pampillón Cesteros, Ana; Elena Domínguez Romero and Isabel de Armas Ranero (2011) Herramienta para la revisión de la Calidad de Objetos de Aprendizaje Universitarios (COdA): guía del usuario. v.1.1. Madrid: e-prints Complutense. Retrieved from http://eprints.ucm.es/12533/ (accessed 18/04/2013).