Cmap with Errors: Why not? Comparing Two Cmap-Based Assessment Tasks to Evaluate Conceptual Understanding
0 likes308 views
The document compares two assessment tasks using concept maps (cmaps) to evaluate the conceptual understanding of 110 first-year undergraduate students in natural science. It highlights the effectiveness of cmaps for learning assessment, emphasizing their advantages in time efficiency and the inclusion of challenging conceptual relationships. Results showed that students performed better in judging tasks than in finding errors, indicating the impact of task format on assessment outcomes.
Cmap with Errors: Why not? Comparing Two Cmap-Based Assessment Tasks to Evaluate Conceptual Understanding
- 1. Cmaps with Errors: Why not? Comparing Two Cmap-Based Assessment Tasks to Evaluate Conceptual Understanding Paulo Correia, Gisele Cabral & Joana Aguiar prmc@usp.br 06.09.16
- 2. Teaching & learning Assessment plays a key role Information exchange between teacher & students Checkpoint to regulate the process Alignment of expectations and duties Special moment requires preparation
- 3. Cmaps are a great alternative Knowledge structures become visible Feedback can easily address learning obstacles Continuous support to meaningful learning Constructivist atmosphere reaches the classroom Pedagogic resonance (the bridge between teacher knowledge and student learning) is fostered
- 4. Reality Requirements for systemic use of Cmaps The preparedness of teachers The motivation of students Conditions in which teachers and students communicate effectively as partners … and the development of straightforward tasks If concept mapping is so helpful to learning biology, why aren’t we all doing it? Kinchin, I. M. (2011). International Journal of Science Education, 23(2), 1257-1269.
- 5. Who is the mapper?
- 6. Aims Explore Cmap with errors under classroom conditions Two assessment tasks Find the errors Judge the selected propositions Two topics Different level of difficulty (students’ background)
- 7. Context Natural Science Course Period of study: class #11 to #14 (100 min each) Topics Creationism vs Evolutionism | background Molecular biology | background Formal assessment: class #15 Questionnaire Cmap with errors
- 8. Participants & Groups 110 1st-year undergraduate students 2 X 2 (Assessment task format X Topic) quasi-experimental design
- 9. C=19 | P=24 | E=8 | J=6E+4C
- 10. C=26 | P=32 | E=9 | J=7E+4C
- 11. Data collection Answer the questionnaire 10 statements abour Creationism vs Evolutionism 10 statements about Molecular Biology 10 min | 1pt for each correct answer (max. 10pt)
- 12. Answer the Cmap task (Find or Judge) 20 min | students marked the time on task (max. 10pt) Data collection
- 13. Student’s performance Find the errors 𝑷 = 𝑰𝒅𝒆𝒏𝒕𝒊𝒇𝒊𝒆𝒅 𝒆𝒓𝒓𝒐𝒓𝒔 𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒆𝒓𝒓𝒐𝒓𝒔 − 𝑾𝒓𝒐𝒏𝒈 𝒊𝒅𝒆𝒏𝒕𝒊𝒇𝒊𝒄𝒂𝒕𝒊𝒐𝒏𝒔 𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒄𝒐𝒓𝒓𝒆𝒄𝒕 𝒑𝒓𝒐𝒑 Example 20 propositions | 5 errors | 4 found | 2 misjudgments 𝑷 = 𝟒 𝟓 − 𝟐 𝟏𝟓 × 𝟏𝟎 = 𝟔. 𝟔𝟕
- 14. Student’s performance Judge the selected propositions 𝑷 = 𝑪𝒐𝒓𝒓𝒆𝒄𝒕 𝒋𝒖𝒅𝒈𝒆𝒎𝒆𝒏𝒕𝒔 𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇 𝒑𝒓𝒐𝒑 𝒕𝒐 𝒃𝒆 𝒋𝒖𝒅𝒈𝒆 × 𝟏𝟎 Example 7 propositions to be judge | 4 correct judgements 𝑷 = 𝟒 𝟕 × 𝟏𝟎 = 𝟓. 𝟕𝟏
- 15. Data analysis Two-way ANOVA considering Questionnaire performance (max. 10pt) Time on Cmap task (max. 20min) Cmap task performance (max. 10pt) Explored comparisons Different Cmap tasks within the same topic Different topics within the same Cmap task
- 16. Results Main effect for the topic Students’ background are different (CrEv > MoBi) CrEv: revise & refine their previous schemas MoBi: conceptual assimilation & schema construction No effect for the Cmap task format
- 17. Results No effect for the topic No effect for the Cmap task format Average time in task: 13min Lower than the time to create & revise a Cmap (~60min) Difficult conceptual relationships were not avoided
- 18. Results Main effect for the Cmap task format Judge groups presented better performances Only Find task penalized type 1 error (false positive) No effect for the topic
- 19. Task X Topic interaction BG CrEv BG MoBi
- 20. Level of Difficulty Adjustments from students’ right answers (RA) Very easy 𝑹𝑨 ≥ 𝟖𝟎% Easy 𝟔𝟎% ≤ 𝑹𝑨 ≥ 𝟖𝟎% Moderate 𝟒𝟎% ≤ 𝑹𝑨 ≥ 𝟔𝟎% Difficult 𝟐𝟎% ≤ 𝑹𝑨 ≥ 𝟒𝟎% Very difficult 𝑹𝑨 < 𝟐𝟎%
- 24. Conclusions Cmap with errors is useful for learning assessment 1. Short time to administer 2. Inclusion of difficult conceptual relationships 3. At least two task formats 4. Ease adjustment of difficult level 5. Short time to correct and provide precise feedback