Developing Diagnostic Tests
Download as PPTX, PDF0 likes68 views
The document discusses developing diagnostic tests to identify student misconceptions. It outlines a process that includes: 1. Identifying the key concept and possible alternative conceptions students may have. 2. Refining questions to address ambiguities and ensure they accurately assess the target concept. 3. Analyzing student responses to determine the frequency of alternative conceptions and refine teaching.
![New A-levels for Old?
Mean
(A-level Entry)
Mean
(All Entry)
Mean
(Other Entry)
2016/17 59% [80] 58% [108] 55% [28]
2017/18 57% [32] 53% [58] 48% [26]
No significant difference in means for A-level entry](https://image.slidesharecdn.com/micer18kjhaxton-200728122158/85/Developing-Diagnostic-Tests-8-320.jpg)
Developing Diagnostic Tests
- 1. It’s the Keele difference. Developing Diagnostic Tests Dr Katherine J. Haxton @kjhaxton
- 2. Think Like a Chemist
- 3. Test Structure (with optional bits) 1. Methane is best represented as A CH4 B -CH3 C -Me Please rate how confident you are in your answer: 1. Not at all confident 2. 3. 4. 5. 6. 7.Very confident Please explain your answer: Methane is the simplest alkane, and has the formula CH4. Carbon forms four bonds to four hydrogen atoms. Answers B and C are the methyl group that can be found in other compounds and the carbon only has three bonds so there is one missing. Have you studied this topic before? Yes No Don’t Recall
- 4. Dunning-Kruger Effect High Confidence, Wrong Answer Low Confidence, Correct Answer Increasing Confidence, Correct AnswerLearning? https://www.psychologytoday.com/us/blog/head-in-the-cloud/201701/the-dunning-kruger-president
- 5. Refine List Underlying Conceptions Previous Teaching Agency Analysis of Answers WHY was the answer incorrect? Coding Errors Sifting of Errors Misunderstanding Systematic Error Alternative Conception Random Error Ambiguous Answer Development: Analysis of Exams
- 6. CONCEPT & alternative conceptions REFINE Ambiguity of wording/format QUESTION WRITING Free Text Responses MCQs with justification TESTING MCQs MCQs with justification ANALYSIS OF RESPONSES Frequency of alternative conceptions Developing a Diagnostic Question Refining: setting it loose on students
- 7. Which of the following travels slowest through space?
- 8. New A-levels for Old? Mean (A-level Entry) Mean (All Entry) Mean (Other Entry) 2016/17 59% [80] 58% [108] 55% [28] 2017/18 57% [32] 53% [58] 48% [26] No significant difference in means for A-level entry
Editor's Notes
- #2: Abstract: Through analysis of examination and class test questions and answers, I have developed diagnostic test questions in IR, NMR, UV-Vis and equations of light that fit broadly with pre-university and 1st year study in English institutions. These questions are multiple choice questions designed to root out key misconceptions to inform subsequent teaching, and to evaluate knowledge acquired prior to university study. I will discuss the approach I have taken to developing and refining multiple choice diagnostic test questions, and share some preliminary data analysis comparing cohorts in 2016 and 2017 studying the old and new style A-levels. Pre-reading: Rather than pre-reading, I have two challenges: Try a sample of diagnostic test questions: https://goo.gl/forms/Cny0xZS6bpW4NJHE3 Consider a concept that you know very well and have taught for some time. List the common errors students make in dealing with this topic. Try to formulate it into an MCQ with good distractors related to the most common errors. How would you go about gathering better evidence for common errors?
- #3: Understanding chemical reactions is a key skill for a chemist. But what does that mean? - Knowledge of types of reactions and reagents - Knowledge of reaction conditions and rate - Knowledge of characterisation techniques and applications - Knowledge of the mechanism of the reaction on the molecular/atomic level My teaching interests and ped res currently focus on the idea of ‘think like a chemist’, that is to gain a conceptual understanding of the synthesis, characterisation and properties of molecules and materials, and to look at teaching methods that promote conceptual understanding. The hard bit - you have to cut content to do it. The good bit - students seem more able to tackle advanced problem solving if they have a solid grasp of the basics. Traditional teaching in chemistry is a bit like Mary Berry’s cook book (the title, no reflection on the author): memorise 100 recipes and call yourself a baker. Teaching to promote conceptual understanding is more like Paul Hollywood’s title: draw out the underlying concepts and understand them fully and you don’t need to memorise recipes, you can construct your own because you understand the principles. There is no consensus on what the fundamental chemistry concepts are but I am working on identifying them in areas relevant to my teaching. Not only must we identify the concepts, we must consider the misconceptions that students hold around them and act to address them: meet the students where they are at, not where their A-level marks say they should be!
- #4: ︎Questions to probe specific alternative conceptions Confidence scale to rate how strongly held concept is (vs. guessing) {Whether topic has been studied before} [Free text answer space for working or explanation] Use to alert students and lecturer to potential issues prior to teaching and direct students to additional learning resources Analysis: statistical analysis to establish test validity (although questionable)
- #7: Pentacle here - developing a good diagnostic test question. There are other ways to go about it.