A presentation on Principles of Assessment.ppt
- 1. Principles and types of Assessment PROF. LUBNA ANSARI BAIG MBBS, MPH, MME, FCPS, PHD
- 2. Objectives By the end of the session the participants will be able to: •Describe the principles of assessment •Differentiate between summative and formative assessment
- 4. Learning Is the process through which we acquire an attribute not previously possessed or further develop an existing attribute. (Bloom 1956, 1964) The evidence of which is the change in the behavior. The change may be in any of the three domains of learning, which are: ◦ Cognition (knowledge) ◦ Psychomotor (skills) ◦ Affective (attitudes)
- 5. Assessment Assessment refers to student’s appraisal or measurement of their competency at various levels. Assessment is an integrated process for determining the nature and extent of students' learning and development (Grunlund 1980). Students’ learning is directed by the methodology and content of evaluation forming their grade (Roth 2002, Cunnington 2002)
- 6. Students’ Assessment 1. Formative assessment It is used for monitoring the learning progress during the entire course/training of students. The goal is to provide continuous feedback to the students, parents and teachers concerning the performance and the learning of the students 2. Summative assessment Typically it comes at the end of term, unit, block or semester and is designed to determine the extent to which the instructional objectives have been achieved. The purpose primarily is to assign grades to the students and also to judge the appropriateness of the course objectives and the effectiveness of teaching.
- 7. Matching learning with appropriate assesment tools...... Knows Shows how Knows how Does Knows Factual tests: MCQ, essay type, oral….. Knows how (Clinical) Context based tests: MCQ, essay type, oral….. Shows how Performance assessment in vitro: OSCE, SP-based test….. Does Performance assessment in vivo: Undercover SPs, Video,Logs,Portfolios Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.
- 9. Reliability: Classical Test Theory X = T + E (error is 1-reliability) Where: X= obtained score T=True score An obtained score represents a sample from some larger set of scores 9
- 10. Reliability in Assessment Is the assessment sufficiently accurate for the purpose regarding: pass / fail decisions? Agreement between different examiners? Consistency in standard across different sites of the examination?
- 11. Reliability Reliability Reliability is essentially concerned with restricting measurement errors within acceptable limits
- 12. Reliability: What are we talking about? Reliability ◦ Precision of test scores ◦ Identifying the sources of error Measures of Reliability ◦ Internal reliability ◦ Test re-test reliability ◦ Equivalent and parallel forms ◦ Inter-rater reliability 12 Lee J. Cronbach and Richard J. Shavelson. Educational and Psychological Measurement June 2004 64: 391-418
- 13. Reliability improves with…….. Longer testing time optical / computer marking good item design item-banking and re-use of proven psychometric quality examiner training
- 14. What is Cronbach’s Alpha? It is the method of estimating reliability. The ideal Alpha value is 0.8 atleast. Though 0.7 is also acceptable.
- 15. Testing Time and Testing Time and Reliability for MCQs Reliability for MCQs Similar for both formats (one best type and T/F questions) optimum time: *2.5 hours for reasonable reliability (about Cronbach’s alpha = 0.7) *3 hours for good reliability (about Cronbach’s alpha = .075) *4 hours for excellent reliability ( about Cronbachs’ alpha = 0.8)
- 16. Reliability as a Function of Testing time Cronbach’s Alpha used as a measure Testing time (Hrs) MCQ1 Short essay2 Oral Exam3 OSCE4 Mini CEX5 1 .62 .68 .50 .54 .73 2 .76 .73 .69 .69 .84 4 .93 .84 .82 .82 .92 8 .93 .82 .90 .90 .96 1 Norcini et al. 1985 2 Stalenhoef-Halling et al 1990 3 Swanson 1987 4 Van der Vleuten 1999 5 Norcini et al 1999
- 17. Reliability of an Oral examination (Swanson 1987) Testing time in Hours No. of cases Same examiner for all cases New examiner for each case Two new examiners for each case 1 2 .31 .50 .61 2 4 .47 .69 .76 4 8 .47 .82 .86 8 12 .48 .90 .93
- 18. Validity – what are we talking about? •Face (looks ok, experts agree) •Content (matches course objectives) •Criterion –Predictive (Predicts success on subsequent exams) –Concurrent (contemporary) •Construct • Measuring what the theme/construct that it intends to measure Accuracy
- 19. VAILIDITY VAILIDITY Validity is essentially concerned with: the content of the assessment ( are the right things being tested?)
- 20. VAILIDITY ( contd.) VAILIDITY ( contd.) the effect of learning ( are the students learning what we want them to learn?) the assessment methods( are we using the most appropriate methods for the purpose?)
- 21. Validity improves with……... Defining the curriculum/ syllabus mapping exam content against the curriculum emphasis on essential and important curriculum content selecting appropriate methods
- 22. FEASIBILITY Feasibility is essentially concerned with the resources available ( time, examiners, administration, space, etc.)
- 23. Feasibility improves with making good use of resources: Making a priority list and being ready to make trade off decisions Using a variety of test methods selecting high performing test methods ( I.e. those with the best reliability and / or validity characteristics)
- 29. Characteristics of an Exam Reliability - Extent to which an assessment technique yields consistent results S Standardization - Extent to which assessment tasks and procedures are similar for all students V Validity - Extent to which an assessment technique measures what it is intended to measure P Practicality - Extent to which an assessment technique is easy and inexpensive to use Impact on Learner – Extent to which the assessment tools affects students’