Designing and Conducting Summative Evaluations
- 1. Designing and Conducting Summative Evaluations By Teri Miles MCOM 530 ~ East Stroudsburg University
- 2. The Purpose of Summative Evaluation • Make “go-no-go” decisions • Keep current materials? • Look for something better suited to meet organization’s specific instruction needs? • Summative evaluations are used to judge the impact of a plan of instruction on the organization’s initial problem.
- 3. Evaluators – Why they should be External • The primary evaluator in a summative evaluation is rarely the designer or developer of the instruction. • Evaluator is often unfamiliar with the materials, the organization requesting the evaluation, or the setting in which the materials are evaluated. • Preferred because they have no personal investment and will likely be more objective
- 4. Two Main Phases of Summative Evaluation 2.Expert Judgment Phase 2. Field Trial Phase
- 5. Expert Judgment Phase (Table 12.1) Purpose: Do the materials have the potential for meeting this organization’s needs? 1. Congruence Analysis: Are the needs and goals of the organization congruent with those in the instruction? 2. Content Analysis: Are the materials complete, accurate, and current? 3. Design Analysis: Are the principles of learning, instruction, and motivation clearly evident in the materials? 4. Feasibility Analysis: Are the materials convenient, durable, cost-effective, and satisfactory for current users?
- 6. Field Trial Phase Purpose: Are the materials effective with target learners in the prescribed setting? Outcomes Analysis: 1.Impact on Learners 2.Impact on Job 3.Impact on Organization
- 7. Field Trial Phase (continued) Management Analysis: 2. Are instructor and manager attitudes satisfactory? 3. Are recommended implementation procedures feasible? 4. Are costs related to time, personnel, equipment, and resources reasonable?
- 8. Comparison of Formative and Summative Evaluation (Table 12.3) Formative Evaluation Summative Evaluation Document strengths and Locate weaknesses in weaknesses in instruction in Purpose instructor in order to revise it order to decide whether to maintain or adopt it On-to-one Expert judgment Phases or Stages Small group Field trial Field Trial Systematically designed in- Produced in-house or Instructional house and tailored to the elsewhere not necessarily Development History needs of the organization following a systems approach One set of materials or several Materials One set of materials competing sets Member of design and Position of Evaluator development team Typically an external evaluator A report documenting the A prescription for revising design, procedures, results, Outcomes instruction recommendations, and rationale
- 9. See Rubric for Evaluating Summative Evaluations Then, take the quiz to test your knowledge!