Designing and conducting formative evaluations
Download as PPTX, PDF4 likes3,467 views
This presentation discusses formative evaluations and how to design and conduct them. Formative evaluations are used during instructional design to improve effectiveness and identify problems. They involve subject matter experts, learning specialists, and learners. One-to-one and small group evaluations identify obvious errors. Field trials test the instruction in its intended context. Data is collected to evaluate clarity, impact, feasibility, and whether objectives are met. Concerns that influence formative evaluations include the context, learners, and outcomes. Problem solving is used to improve weak units based on evaluation data.
Designing and conducting formative evaluations
- 1. DESIGNING AND CONDUCTING FORMATIVE EVALUATIONS Presented by Larry F. Cobb April 30, 2014
- 2. BACKGROUND In this presentation, we will address formative evaluations, how they should be used, and how to design and conduct them
- 3. OBJECTIVE To discuss and give clarity to the following subjects: Formative Evaluation Designs Role of Subject-Matter, Learning, and Learner Specialists in Formative Evaluation One-to-One Evaluation with Learners Small-Group Evaluation Field Trial Formative Evaluation in the Performance Context Formative Evaluation of Selected Materials Formative Evaluation of Instructor-Led instruction Data Collection for Selected Materials and Instructor-Led instruction Concerns Influencing Formative Evaluation Problem Solving during Instructional Design
- 4. DICK AND CAREY’S MODEL
- 5. WHAT IS FORMATIVE EVALUATION? A formative evaluation is the collection of data and information during the development of instruction that can be used to improve the effectiveness of the instruction. The process is used by designers to make their instructions more effective and effiecient. They were originally used as a process to improve instruction after the first draft of instruction was developed. It was later found better to try out earlier components of the design process, thereby avoiding a lot of problems that would otherwise not be discovered until after the draft of the instruction was complete.
- 6. FORMATIVE EVALUATION DESIGNS The evaluation design needs to yield information about the location of and the reasons for any problems. This prevents the evaluation that is too narrow or too broad. The instructional strategy is the framework for the design. The components will help you create questions that will allow you to evaluation instruction better.
- 7. FORMATIVE EVALUATION DESIGNS CON’T These are questions that can be asked to evaluate the components of the instructional strategy: 1. Are the materials appropriate for the type of learning outcome? 2. Do the materials include adequate instruction on the subordinate skills, and are these skills sequenced and clustered logically? 3. Are the materials clear and readily understood by representative members of the target group? 4. What is the motivational value of the materials? Do the learners find the materials relevant to their needs and interests? Are they confident as they work through the materials? Are they satisfied with what they have learned? 5. Can the materials be managed efficiently in the manner they are mediated?
- 8. ROLE OF SUBJECT-MATTER, LEARNING, AND LEARNER SPECIALISTS IN FORMATIVE EVALUATIONIt’s important to have the instruction reviewed by specialists. SMEs may be able to comment on the accuracy and currency of the instruction. Learning specialist may be able to critique your instruction related to what is known about enhancing that particular type of learning. Learner specialist may be able to provide insights into the appropriateness of the material for the eventual performance context.
- 9. ONE-TO-ONE EVALUATION WITH LEARNERS The purpose of one-to-one evaluation is to identify and remove the most obvious errors in the instruction and to obtain initial performance indications and reactions to the content by learners There are three criteria: 1. Clarity – Is the message, or what is being presented, clear to individual target learners 2. Impact – What is the impact of the instruction on individual learner’s attitude and achievement of the objectives and goals 3. Feasibility – How feasible is the instruction given the available resources (time/context)
- 10. SMALL GROUP EVALUATION Small group evaluation have two primary purposes. The first one is to determine the effectiveness of changes made following the one-to- one evaluation and identify any remaining learning problems that learners may have. Secondly, small group evaluation is used to determine whether learners can use the instruction without interacting with the instructor.
- 11. FIELD TRIAL The field trial is the third stage in formative evaluation, referring to the evaluations of the program and product in the setting in which it is intended to be used. It is used to determine whether the changes/revisions in the instruction made after the small group stage were effective and to also see whether the instruction can be used in the context for which it was intended.
- 12. FORMATIVE EVALUATION IN THE PERFORMANCE CONTEXT The purpose of the performance context formative evaluation is to make three fundamental determinations 1. Do the learners find that it is appropriate to use their new skills in the workplace and have they been doing so 2. If they have been used, what has been the impact on the organi_ation 3. What suggestions do the learners and other that they work with have for improving the instruction
- 13. FORMATIVE EVALUATION OF SELECTED MATERIALS Selected materials formative evaluation concludes that the instructor chooses existing materials be there is seldom time to or resources to conduct all faces of the evaluation. In this case, the instructor should proceed directly to the field trial. The instructor who conducts a field trial is able to observe the progress and attitudes of learners using a set of adopted or adapted materials.
- 14. FORMATIVE EVALUATION OF INSTRUCTOR-LED INSTRUCTION Formative evaluation of instructor-led instruction is done to determine whether the instruction is effective and decide how to improve it. It is functions just like the field trial. To prepare, the instructor must be concerned with the entry skills and prior knowledge, the posttest knowledge, and that attitudes of learners. This is an opportunity to evaluate instructional procedures.
- 15. DATA COLLECTION FOR SELECTED MATERIALS AND INSTRUCTOR-LED INSTRUCTION Information dealing with collection of data in a field trial of original instructional materials applies equally well to date collection procedures used in the evaluation of selected materials and instructional procedures.
- 16. CONCERNS INFLUENCING FORMATIVE EVALUATION There are several concerns about the formative evaluation context and the learners who participate in the evaluation that the designer should keep in mind when planning and implementing data-collection procedures 1. Context concerns 2. Concerns about Learners 3. Concerns about Formative Evaluation Outcomes 4. Concerns with Implementing Formative Evaluation
- 17. PROBLEM SOLVING DURING INSTRUCTIONAL DESIGN The purpose is to make a decision about what to do with a particular unit of instruction. The designer could collect enough date in the formative evaluation about the illustrations to make at least a tentative decision about their continued use in instructoin.
- 18. LIFELONG LEARNER SUMMARY As a change agent, formative evaluations must be done regularly to ensure that information and content is kept current.
- 19. LARRY F. COBB Email: lfcobb@alasu.edu