Assessment_basics_ictcm. in educationppt
- 1. 1 1 ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 Assessment - overview Assessment - overview G. Donald Allen G. Donald Allen Department of Mathematics Department of Mathematics Texas A&M University Texas A&M University ICTCM – October 28, 2004
- 2. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 2 2 Speakers Speakers • Don Allen • Mike Speed • Dave Thomas • Mike Pilant • Diem Nguyen • Greg Tobin
- 3. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 3 3 What is a “Student Learning What is a “Student Learning Outcome? Outcome? Student learning outcomes are properly defined in terms of the knowledge,skills, and abilities that a student has attained at the end (or as a result) of his or her engagement in a particular set of education experiences. Council for Higher Education Accreditation
- 4. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 4 4 The Purpose of an Assessment The Purpose of an Assessment Is: Is: • to provide information about skills in a content area • to guide instruction for the individual and for groups • to provide feedback or information • to plan further instruction • to improve instructional practices • to focus on what the student can do • to guide decisions about instruction (Formative)
- 5. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 5 5 The Purpose of a Test/Exam Is: The Purpose of a Test/Exam Is: • to provide feedback about skills learned or obtained via instruction • to determine the areas in which an individual needs re-teaching • to provide grades • to focus on what the student cannot do • to indicate attainment of skills (Summative)
- 6. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 6 6 Characteristics of Educational Characteristics of Educational Measures Measures • Educational measures are – Indirect – Incomplete – Relative – Used as classification • Educational measures have errors
- 7. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 7 7 Desirable test attributes Desirable test attributes • Reliability • Validity • Objectivity • Balance • Fairness • Practicality Test length should be 1. Long enough to be reliable 2. Short enough to be administered
- 8. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 8 8 Goals for making tests Goals for making tests • Be concise • Be singular – focus on one aspect • Be realistic • Use definite terms • Delineate expected behavior
- 9. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 9 9 Blooms taxonomy Blooms taxonomy 1. Knowledge (basic) 2. Comprehension 3. Applications 4. Analysis 5. Synthesis 6. Evaluation (advanced)
- 10. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 10 10 Simplified Bloom’s for testing Simplified Bloom’s for testing I. Knowledge (basic) II. Understanding – ability to restate, interpret III. Applications – beyond restatement to include analysis, synthesis, and evaluation
- 11. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 11 11 Function of a test Function of a test • Advancement (criterion referenced) • Achievement of objectives (norm referenced, items should discriminate well)
- 12. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 12 12 Cognitive testing – another view Cognitive testing – another view • Recall • Recognition • Differentiation • Constructed response • Problem solving
- 13. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 13 13 Capacity to assess Capacity to assess • Ability • Achievement • Personality • Skill • Opinion • Attitude
- 14. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 14 14 Assessment Assessment What can it do? What can it do? • Provide – to students a chance to reflect on what they have learned and need to learn – to teachers a chance to reflect on what they have taught well and what needs further attention
- 15. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 15 15 Assessment Assessment What can it do? What can it do? • Provide feedback – to the teacher on the clarity of given assessments – to teachers about the progress of learning
- 16. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 16 16 Assessment Assessment What can it do? What can it do? • Providing – clarity to students on the type of mathematics knowledge valued – clarity to students on the type of mathematics proficiency valued
- 17. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 17 17 Assessment Assessment What can it do? What can it do? • Become a routine part of ongoing classroom activity • Promote regular class attendance • Promote multiple solutions and approaches, giving a well rounded picture that allows each student to show his/her best strengths
- 18. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 18 18 What Counts as Evidence What Counts as Evidence for Student Learning for Student Learning Outcomes? Outcomes? “Evidence” refers to the kinds of information about student learning outcomes that is most appropriate to accreditation settings. Council for Higher Education Accreditation
- 19. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 19 19 Evidence vs. Anecdotes Evidence vs. Anecdotes However… the plural of anecdote is not evidence! “Evidence” can simultaneously embrace the results of quantitative and qualitative approaches to gathering information
- 20. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 20 20 Types of evidence Types of evidence •Comprehensive examinations and assignments. • Performance on licensing examinations. • Professionally judged performances. • Portfolios of student work. • Samples of representative student work.
- 21. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 21 21 What is a “Standard” of Student What is a “Standard” of Student Learning? Learning? “Standard" refers to a specific expectation or level of performance that an institution or program establishes for student learning. Standards are the point of comparison against which to judge the actual evidence of student learning once it is collected.
- 22. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 22 22 Assessment Assessment Capacity to Assess Capacity to Assess • Achievement • Personality • Skill • Opinion • Attitude
- 23. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 23 23 Testing for what? Testing for what? • PREDICTION • EVALUATION • DIAGNOSIS • SCREENING
- 24. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 24 24 Assessment Weaknessess Assessment Weaknessess • Lack of content validity • Lack of scoring economy • Scorer unreliability (leniency, severity) • Halo effect • Informed judgement • Extraneous factors (quality of hand-writing, pen vs. pencil, spelling errors)
- 25. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 25 25 Essay questions Essay questions • Item types – Short answer – Extended answer • Item Construction – Focus the direction of the response
- 26. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 26 26 Guidelines for essay questions Guidelines for essay questions • Allow adequate time • Should have a problem to be solved – Problem should be defined explicitly – Problem should be limited • Directions should be explicit • Don’t use optional questions • Construct detailed key for each question
- 27. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 27 27 Assessment – It is… Assessment – It is… • Broadly defined and used • Complex • Detailed • Wrought with unfairness • Essential • Different in different contexts
- 28. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 28 28 Engines Engines • ALTEC – online tools for teachers http://altec.org/index.php – QuizStar – Rubistar – Profiler Pro • Survey Monkey - http://www.surveymonkey.com/ • Blackboard, WebCT
- 29. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 29 29 Forms of Assessment Forms of Assessment • Collaborative/group projects • Direct observation Essays • Examinations (unseen), Examinations (seen/open book) • Multiple-choice tests, Oral questioning after observation • Performance projects, Portfolios • Practical projects, Presentations • Problem sheets, Projects • Self-assessment, Case-studies • Short-answer questions
- 30. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 30 30 Two simple assessments Two simple assessments
- 31. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 31 31 The Muddiest Point The Muddiest Point • What point was least clear during this session? • What point needs further clarification? • What point prevented me from learning fully the contents of this session?
- 32. ICTCM Pre session, Oct 2004 ICTCM Pre session, Oct 2004 32 32 The one minute paper The one minute paper 1. What was the most useful or meaningful thing you learned during this session? 2. What questions remain uppermost in your mind as we end this session? Answer each question in one or two sentences.