Running Head INSTRUCTIONAL DESIGN MODELS .docx

Running Head: INSTRUCTIONAL DESIGN MODELS
1
INSTRUCTIONAL DESIGN MODELS
7
Instructional Design
Models
Introduction to Instructional Design Models
Instructional design models are used in e-learning where
various sources are applied to the benefit of the user. It is often
regarded as a framework where instructional materials are often
developed. It's an online tool used by instructional designers to
give both meanings as well as structure to the reading material
(Karger & Stoesz, 1998). It is common that any learning course
is usually broad and requires breaking down the entire process

into stages that are separately handled to create efficiency. The
main goal of instructional design models is to see to it that the
anticipated learning objectives, as well as the desired
expectations, are met to the letter.
Reasons for using instructional design models
The core factor as to why instructional design models are
used is to promote a systematic learning process and also save
on the time used to reach the desired goals. Since they are
mostly used in online courses, most of the users are from
different regions in ten worlds and have different needs and
capabilities (Karger & Stoesz, 1998). Online learning is
efficient in that it does not limit people on common grounds
like religion, race, geographical location or any other variable.
The first step as to why instructional design models are required
is the concept of whether there is a need to develop the training.
Once this question is answered, then the need for such a model
arises immediately (Karger & Stoesz, 1998). The other variable
to be considered is the amount of content that is desired to
achieve the desired objectives. In all sectors, models save the
money used for expenditure and also helps in filling in the
content gaps in between. Some of the examples of instructional
models include ADDIE, SAM, Dick and Carey, Kemp design
model, ASSURE and also Instructional Design System.
1. ADDIE
ADDIE was the first instructional design model to be used
in many areas since the instructional models began. However,
there are many concerns as to the efficiency of the model,
owing to the fact that there are many changes that have occurred
in the past five decades (Faryadi, 2007). Every decade witnesses
a new chapter of a technological revolution which turns most of
the old things obsolete. ADDIE is an acronym for Analysis,
Develop, Design, Implement as well as Evaluate. Each process
is a stage with different tasks to achieve the desired outcomes.
Here is a description of each of the steps:
Variable 1: Analysis
Analysis answers the question of why the training is

required in the first place. This follows after comprehensive
data has been collected and evaluated. However. To remain on
the right track, the designers must also understand the
expectations as well as the needs of the firm in question. This
phase drives the development as well as the design of the entire
process.
Variable 2: Design
This is the stage where the instructional designers chose
the instructional strategy which to follow after which they put
down the desired objectives (Faryadi, 2007). To finalize the
process, they have distinct delivery methods and also
appropriate media to accomplish their relevant tasks.
Variable 3: Development
This entails developing all the necessary elements of the
course material. This is done following the design phase of the
model
Variable 4: Implementation
This stage involves rolling out the course, delivering it to the
relevant learners and also monitoring the effects of the course.
Variable 5: Evaluation
The course provides the anticipated results. Instructional
designers work with the clients and come up with the effects of
the course.
2. Merrill’s Principles of Instruction (MPI)
This principle of instruction was developed by David
Merril in 2002, and it is comprised of five learning principles
which include:
a) Activation principle
b) Integration principle
c) Task-centred principle
d) Application framework
e) Demonstration framework
3. Gagne’s Nine Events of Instruction
This model is based on the behavioural model of learning
that is composed of multiple framework events that enable
instructional designers to follow distinct learning situations

(Thompson, 2001). It is the most common eLearning framework
that is necessary for administering practical learning objectives.
These nine steps include:
1) Gaining the attention of students.
2) We are informing the students about the underlying
objectives.
3) I am stimulating the recall of previous learning objectives.
4) I am presenting the content.
5) I am providing learners with guidance.
6) We are assessing the overall performance.
7) Elicit performance.
8) I am providing the necessary feedback.
9) I am enhancing job promotion as well as retention.
4. Bloom’s Taxonomy
This model was advanced by Benjamin Bloom with the aim
of organizing the distinct levels of cognitive learning in the
long run. A more detailed version, Revised Taxonomy, was
developed by Anderson alongside his friend Krathwohl in 2001
(Thompson, 2001). The following is a description of the model.
5. Situated Cognition theory
It was initially published in 1989 where the core principles
are still practised until today. The core element of the law is
that no one is capable of separating doing from knowing. Its
area of emphasis is that no one is in a position to do what they
do not understand. Every learning curve is based on a particular
concept that is often shown in action (Thompson, 2001).
Learning is also described as a social endeavour which provides
a common platform for knowledge expansion as well as wisdom
gathering. This is often done through numerous discussions as
well as problem-solving characteristics that are applicable in
the long run.
6. Sociocultural Learning Theory
The initial script that described this theory was published
in the 1930s, but it took longer to become popular because of
the political turmoil under Starlin by then. There were also

challenges in translating the original script to other scripts in
different languages (Lantolf, 2017). Social, cultural learning
theory centres around three core variables which include
communication, the zone of proximal development and also
culture.
7. Individualized construction
This is more focused on how people gain knowledge and
also how people respond to their external environment.
Although it is meant for those who can grasp aspects quickly, it
also allows those who are slow in the process to move at their
own pace (Lantolf, 2017). In addition to these adjustments, it
also includes the learners who have distinct learning
preferences and with different response to the learning curve.
The critical principles focused on are:
a) Autonomous completion of work by the learners. This gives
them the opportunity to focus on individual strengths as well as
opportunities.
b) Written learning materials which are preferred over numerous
presentations.
c) I am supporting the learners as well as adding an entirely
new level of social interactivity as well as experience.
d) I am assessing the learners after they have learned a
particular lesson to gauge their learning progress in the long
run.
8. The SAM Model
This gives the researcher the room to make the desired
changes through performing well-distributed tasks over a long
period of time. The researchers are also in a position to perform
multiple iterations (Lantolf, 2017). It has three processes that
ensure that the changes are well distributed and that the
eLearning project moves to completion. The stages in this
model are:
a) Presentation phase where enough information regarding the
project is collected.
b) Iterative design where the project designs are formulated.
c) Iterative development where the research design is not only

created, but also reviewed.
References
Karger, H. J., & Stoesz, D. (1998). American social welfare
policy: a pluralist approach. New York: Longman.
Faryadi, Q. (2007). Instructional Design Models What a
Revolution! Place of publication not identified: Distributed by
ERIC Clearinghouse.
Lantolf, J. P. (2017). Sociocultural theory and second language
learning. Oxford: Oxford University Press.
Thompson, L. (2001). Addie. Waterville, Me.: Five Star.
Running Head: PROFESSIONAL DEVELOPMENT
1
PROFESSIONAL DEVELOPMENT
8
Professional Development

What is the task that individuals should be able to accomplish
or perform?
In any institutional setting, there exists a theoretical flame
work that is represented by the school system composed of
different ranks. It is for this reason that professionalism is a
significant ingredient in the administration of the institution.
Each person is required to stretch beyond his or her limits in
order to achieve the anticipated objectives. Even though
professional development is primarily focused on the theory
part, it is absolutely clear that they are the significant advisers
responsible for linking all the stakeholders in the institution
(Turner, J, 2004). High-quality delivery of services. In
accordance with Cheng, is a factor of professionalism? The
more the teachers are skilled, the better the services and the
brighter the future. As Gallagher explained, improving the
schools can only be done by enhancing the skills as well as the
abilities that the teachers possess.
Professional development is a crucial sector not only in the
lives of the stakeholders but also to the economy as a whole. It
is the knowledge that is gained and passed down from the
instructors to the students that are used to run the day to day
life of tomorrow. I agree with researchers that the most
significant impact in student learning, as well as achievement,
is the teachers around them (Turner, J, 2004). The nation's
future in the hands of distinct professionals, and this arises the
question of what is expected from these professionals. As a

matter of fact, although there are many requirements to gain
professional knowledge, the primary task is to lay down a
strong foundation of careers (Turner, J, 2004). Jobs are in many
fields ranging from management, teaching, accounting,
leadership, and on and on. Therefore, the most crucial task of
individuals is to pass professional knowledge from one group to
the other clearly.
For competent professional development, in the long run,
it is required that every stakeholder should participate to ensure
that the project runs to completion. The teachers are expected to
deliver qualitative and quantitative data that will enable the
learners to understand the key concepts within the given time
period. On the other hand, the learners are expected to ensure
that they avail themselves to the respected venues of learning
(Turner, J, 2004). Apart from education and delivering
information, following up is another crucial task that is
necessary for professional development. Following up, people
ensure that they act in accordance with the set principles and
also are in line with the laws of the state (Turner, J, 2004). In
addition to this compliance, they also ensure that the
environment is conducive for offering the necessary information
to the target group. Nevertheless, this can only be accomplished
through joint efforts by the community, schools, and teachers,
as well as the government.
What are the critical components of this task?
The critical components of professional development tasks
are the teachers, institutions, and the roles that are expected to
be performed in the face of professional development. The
primary functions that are performed at this stage are:
a) Stating professional activities and features. This is where a
sophomore illustrates his or her passion after graduating in a
particular field. This is the specialization in the areas that one
is best at and joining the respective memberships (Cheng, Y. C,
1996). There are four developmental ranks at this stage, which
include: instructional, organizational, career, and discipline
development.

b) They are assessing the knowledge as well as the skills. This
entails identifying the strengths as well as the weaknesses of the
course that the students have acquired in the long run (Cheng,
Y. C, 1996). In short, the students look for those areas that they
can primarily add more knowledge as well as gain more skills.
c) Set goals. Goals include both professional as well as
individual objectives. This is done by reflecting at the past as
well as the future of careers, and the individuals involved
ensure that the short-term goals, as well as the long-term goals,
are considered.
d) It is creating an action plan. This entails the direction and the
anticipated position within a given period of time (Cheng, Y. C,
1996). An action plan also involves the skills as well as the
knowledge that is required to develop new skills as well as the
timeline that is necessary to achieve the desired goals.
e) I am documenting the development. This is the output that is
produced after it has undergone some series of advisors and
reflecting on personal statements. It also entails revisiting
personal goals regularly to ensure that the skills, as well as the
knowledge required to get the anticipated outcome, is gained.
What is the sequence through which a task is accomplished and
performed and should be learned and taught?
For any task to be learned and taught, there is a timeline
that needs to be followed, and the desired actions have done to
ensure that the course is effectively mastered. The principles
that lie behind the collection, as well as the gathering of the
required tasks, are essential in the literary assessment of
learners. However, the sequence of learning these tasks varies
according to the geographical region as well as the core values
of the underlying state (Eisner, E, 1991). Every job requires
ample time for active constituents to be learned. This is done by
allocating the necessary time needed for each task. The figure
below illustrates the time that is required for each job as well as
the collection of data. Owing to the massive technological,
economic, and political as well as social changed to the world,
there are multiple changes that are required in the learning

sequence.
Unlike the earlier years where there were few competing firms,
today, there are thousands of firms that compete in the same
industry. It is for this reason that every entity needs to ensure
that they are in line with the latest learning lessons. Earlier on,
there were very few components that required to be learned, and
there were also limited opportunities where people would work
to ensure that they earned their living. The mode of learning is
divided into phases that are required to provide that, at most,
every concept is not only taught but also understood. These
phases include:
1. Stage 1: Collection of baseline data. This is the phase that
involves the collection of baseline data with the aim of
gathering the necessary data that will be used in the study
schedule. The students familiarize themselves with the required
information and also understand what is required of them.
2. Phase 2: Collaboratively doing the required tests in the long
term. This phase occurs in the second, third, and fourth years of
study, where the students dig deeper into the required subjects.
It is concerned with carrying out the required tasks, doing
various research, and also coming up with the desired
conclusions.
3. Phase 3: Insight into professional knowledge. There are
multiple avenues that require the desired conclusions. There are
numerous scenarios that aim at expanding the understanding of
the students so that they can further their knowledge.
How can you determine whether the individual is able to
accomplish the task?
Determining whether an individual is accomplished for the
task is not a complicated process. The first method that can be
used to assess the efficiency of a person is through analysing
the years and the disciplines of study (Eisner, E, 1991). If there
exists no correlation between the subjects studied and the task
in question, then the person is not qualified for the job. In

addition to academic qualifications, integrity, as well as the
willingness to work, is required to ensure that the task will be
efficiently done. Finally, the minimum requirements set for the
job can also be used to determine the qualifications of a person.
In the 21st century, there is an increased demand for skills
that are technologically oriented in the marketplace. For the last
two decades, there has been increased use of artificial
intelligence in the production of goods and services. The more
the use of technology, the more the demand for these skills. It is
in this era that there has been the rise of technology companies
has been witnessed. More and more companies have risen, and
leverage is be9ng used more than ever today. Although there are
some of the tasks which cannot be automated, it is clear that the
demand for the relevant skills is increasing on a daily basis.
Skills can thus be used as avenues for determining the
qualification of a person for a specific job.
References
Abadiano, H. L. & Turner, J. (2004). Professional staff
development: What works? The NERA Journal, 40(2), 87-91.

Cheng, Y. C. (1996). The relation between teachers'
professionalism and job attitudes, educational outcomes, and
organizational factors. The Journal of Educational Research, 89
(3), 163-171.
Eisner, E. (1991). The enlightened eye: Qualitative inquiry and
the enhancement of educational practice. Englewood Cliffs, NJ:
Prentice-Hall.
Chapter 10
Determining the Success of the
Instructional Design Product
and Process
Tobias is the lead instructional designer for a company that is
developing training materials for
web-based software. Although his team is making progress
toward completing the project, he feels
there are some issues that need to be resolved within the design
and production team if the beta
version of the training is going to be released on time. He
wonders whether he should conduct a
formative evaluation to determine issues that might be resolved
to help improve the group’s work-
ing relationship.
Elizabeth works for a faculty development center at a
university. Her university has spent signifi-

cant amounts of money over the past two years on two web-
based synchronous communication and
learning tools. Elizabeth was asked to create and deliver a
three-part workshop on integrating the
tools in order to increase faculty use of them. After conducting
the workshop over two semesters,
she was asked to evaluate the workshop’s effectiveness.
Guiding Questions
• What information can formative and summative evaluations
provide to improve the instruc-
tional design process?
• Why is it important to include various types of evaluation in
the instructional design process?
• How can the success of formative and summative evaluations
be judged?
• How can an instructional designer use evaluation to improve
an instructional design team’s
effectiveness?
Source: Shutterstock 195817664.
01/27/2020 - RS0000000000000000000002052606 (May
Katitar-Thompson) - The
Essentials of Instructional Design
Determining the Success of the ID Product 163
Key Terms
formative evaluation (page 163)

group processing (page 176)
program evaluators (page 171)
rapid prototyping (page 168)
summative evaluation (page 170)
usability testing (page 168)
Chapter Overview
Evaluation is a crucial part of instructional design. As we
discussed in Chapter 9, evalua-
tion is a process that allows for data to be gathered that help
determine the level of success
of someone or the worth of something. In instructional design, a
major purpose of evalu-
ation is to help determine how successful a learner has been as a
result of participating in
instruction. This type of evaluation is called “learner
evaluation.” Although learner evalu-
ation is extremely important, there are two other important
evaluation types: formative
and summative. These two types concentrate on the
instructional design process and the
instruction developed as a result of the process. This chapter’s
focus is on defining forma-
tive and summative evaluations, describing specific approaches
of instructional design
experts, and how formative and summative evaluations can be
designed and implemented.
Formative and Summative Evaluation
It is a common misconception of beginning instructional
designers that evaluation only
takes place at the conclusion of instruction to evaluate learners.
Evaluating the learners is
only one of several types of evaluation that instructional

designers must understand how
to design and implement. In reality, evaluation should also take
place at various stages of
the instructional design process through the use of formative
and summative evaluations.
Knowing when and how to use various evaluation types helps
instructional designers
develop instructional interventions that are efficient and
effective.
Formative Evaluation
Formative evaluation is used throughout the instructional design
process to gather data
that can be used to provide feedback on how the process is
going. It is especially useful
during the early stages of the instructional design process. The
feedback allows an
instructional designer to make improvements to the instruction
before it is completely
developed. This helps ensure that high-quality instruction is
developed. In addition to
helping improve the instruction, data gathered through a
formative evaluation can be
shared with a client to indicate how the project is progressing.
Periodic communication
with your client about how the project is progressing helps
make certain that project goals
and expectations will be met.
Approaches to Formative Evaluation
Instructional design experts take a variety of approaches to
formative evaluation. Despite
their differences, they all have in common the goal of helping
improve the instruction

that is being developed. Let us begin by looking at three
different approaches to formative
evaluation.
Smith and Ragan (2004) write that formative evaluation is
conducted to “determine
the weakness in the instruction so that revisions can be made to
make them more effective
01/27/2020 - RS0000000000000000000002052606 (May
164 Determining the Effect of the Intervention
and efficient.” This helps an instructional designer know
“whether the instructional mate-
rials are ‘there’ yet, or whether she needs to continue the design
process” (p. 388). The
stages that Smith and Ragan advocate for formative evaluation
are design reviews, expert
reviews, learner validation, and ongoing evaluation.
The first stage—the design review—is completed prior to the
development of instruc-
tion. Stage two—the expert review—typically occurs after the
instruction is completed but
before it is used with learners. The final two phases include the
use of the actual instruc-
tion with learners who represent the intended learners.
Design reviews are conducted after various phases of the
instructional design process, such
as the needs analysis, task analysis, goals and objective

analysis, and learner analysis. Design
reviews help to verify the accuracy of information at each stage
of the instructional design
process before instruction is developed. Smith and Ragan
(2004) advocate revisiting the data
gathered during these instructional design phases to determine
how accurate they are.
Expert reviews are conducted to gather information about the
instruction to determine
if it is accurate and current. Various experts—such as content
experts, instructional design
experts, pedagogical experts, and experts on the learners—can
be used to provide various
perspectives on the instruction. The instruction provided for
expert reviews is typically at
the draft stage. Smith and Ragan (2004) suggest that expert
reviewer comments should be
divided into three categories: (1) revisions that need to be made
immediately, (2) suggested
revisions that require additional data that can be gathered
during the final two stages of
the formative evaluation, and (3) suggested revisions that
should be ignored.
Learner validation includes three levels: one-on-one evaluation,
small-group evalua-
tion, and field trials. The three levels are identical to those
promoted by Dick, Carey, and
Carey (2011), discussed later in this chapter.
The final stage—ongoing evaluation—includes gathering data
on the long-term effec-
tiveness of instruction. Smith and Ragan (2004) state that if
instruction is meant to be
used multiple times, then provisions need to be made in a

formative evaluation plan to
collect effectiveness data. These data, which are similar to data
collected during a field
trial, provide feedback on what revisions needed to be made to
instruction based on its
actual use and whether the revisions made have been effective.
Morrison, Ross, and Kemp (2007) advocate a basic model for
formative evaluation
based on the work of Gooler (1980). Gooler’s approach follows
these eight steps:
1 purpose
2 audience
3 issues
4 resources
5 evidence
6 data gathering techniques
7 analysis
8 reporting.
There are three main phases to this approach: planning,
conducting, and reporting. Phase
one includes steps one through five, while phase two includes
steps six and seven. Phase three
is the eighth and final step, reporting the results.
Phase one includes determining the evaluation purpose, the
primary audience the
results will be disseminated to, the issues that need to be
addressed, the resources that are
needed to address the issues, and the types of evidence that will
be acceptable to address
these issues. It is extremely important to work closely with the
client during phase one in
order to clearly articulate and manage expectations.

01/27/2020 - RS0000000000000000000002052606 (May
The first step—determining the purpose or purposes of the
evaluation—is done in con-
sultation with the client. The two most common purposes are to
improve the instruction
that is being developed and to satisfy administration
requirements of the client you are
working for. Many corporations and institutions require
evaluations to be conducted.
Most state agencies are required to include an evaluation
component to help ensure that
project goals are being met and to report how monies have been
spent.
The audience of the evaluation is important to determine
because it will establish the
types of information that need to be collected and reported. The
client will be able to help
determine who the intended audience will be. Conducting an
evaluation for multiple audi-
ences should be avoided because it will be difficult to satisfy
varying needs within a single
report. It is best to try to narrow the audience down as much as
possible.
After determining the purposes of the evaluation and the
intended audience, it is time to
determine the issues that need to be addressed. The issues are

generally stated as questions
that need to be answered. For example, in the case of an
interactive, instructional kiosk:
• Are learners able to successfully utilize the kiosk after
completing the instruction?
• What elements of the kiosk do learners find most
understandable and least
understandable?
Or in the case of web-based instruction:
• What is the average time spent on the summary exercises?
• Do the learners find the web-based self-instructional
materials helpful in learning the
content?
Once questions have been developed based on the evaluation
issues, resources should be
identified that are needed to answer the questions. People,
artifacts, and equipment are com-
mon types of resources that are often needed to address
evaluation issues. It may be difficult
to actually secure all the resources needed to address the
evaluation issues. If this situation
occurs, the instructional designer needs to communicate this to
his or her client. Issues may
need to be adjusted if the necessary resources cannot be
secured. The final step in phase one
is to identify the types of evidence that are needed to satisfy the
evaluation issues.
Phase two—conducting the evaluation—includes determining
the data collection tech-

niques that will be used, gathering the data, and analyzing the
data. When determining the
data collection techniques to be used, Morrison et al. (2007)
state: “Two key, and often
opposing, factors need to be weighed: precise measurement
versus feasible or practical
measurement” (p. 310). For example, during the planning stage
of a formative evaluation
of a nurse-training program, an instructional designer
determined that all participants
would have specific skills evaluated by using hospital patients.
However, due to logis-
tical impracticalities (e.g., amount of time required), the nurse’s
skills were tested on
other nurses in the training program. An instructional designer
must often make a choice
between what is ideal and what is practical when gathering data.
The various data collection techniques that can be used are the
same as those presented
in Chapter 9. They include observations, questionnaires,
interviews, paper-and-pencil
tests, and performance tests—to name a few. It is highly
recommended to use a variety of
techniques to gather multiple sources of data. This can help to
triangulate the findings.
Step seven—analyzing the data—should reflect the purpose of a
formative evaluation:
to provide usable and useful information that helps the
instructional designer improve
instruction. Complex statistical analyses are typically not
required. Morrison et al. (2007)
recommend the following typical analysis procedures:
01/27/2020 - RS0000000000000000000002052606 (May

• frequency distributions
• frequency graphs or histograms
• descriptive statistics, such as percentages, means, and
medians
• listing of actual comments made by respondents.
(p. 311)
The final phase—step eight—is reporting the results of the
evaluation to the primary audi-
ence. This is typically done as an evaluation report, with the
format of the report tailored
to the audience the report will be disseminated to. A typical
evaluation report is formatted
in the following way:
1 Executive summary: an abstract that outlines the major
findings, conclusions, and
recommendations.
2 Evaluation purpose:
• evaluation issues—stated as questions
• description of the instruction being evaluated.
3 Methodology used:
• participants
• instruments used to gather data.

4 Results:
• analysis
• findings.
5 Conclusions and recommendations.
It is not uncommon that, in addition to a written report, an oral
report of the project will
be requested. The oral report can either be conducted one-on-
one with the client or more
formally with a larger group. It is important to remember that
the purpose of the evalu-
ation is to provide recommendations based on the data
collected. When disseminating a
final report, be certain to highlight the conclusions and
recommendations.
Dick et al. (2011) write: “The emphasis in formative evaluation
is on the collection and
analysis of data and the revision of instruction” (p. 258). They
provide three basic phases
of formative evaluation: (1) one-on-one or clinical, (2) small
group, and (3) field trial. It is
important to note that the type of formative evaluation Dick et
al. advocate is designed to
work specifically with self-instructional materials. Despite this,
elements of their approach
can be used with other types of instruction and materials.
During phase one, the instructional designer works with
individual learners to gather
data that are used to revise the instruction. The purpose of the
phase is to remove obvious
errors and omissions in the instruction, gauge initial learner

performance, and determine
learner reactions to the content. Dick et al. (2011) suggest that
the instructional designer
work with at least three learners (with varying abilities,
attitudes, and/or experiences)
who represent the target population the instruction is intended
for. However, only one
learner at a time should be worked with during a one-on-one
evaluation. The instructional
designer should be actively involved during this phase.
After selecting the learners who will participate in the one-on-
one evaluation, the steps
to this phase are:
1 Explain to the learner that you would like his or her reactions
to new instructional
materials that have been developed.
2 Give a pretest on the content.
01/27/2020 - RS0000000000000000000002052606 (May
3 Encourage the learner to talk about the materials as he or she
goes through the instruc-
tion and to be candid about what he or she likes or does not
like, what makes sense,
and what does not make sense.
4 Give a post-test; discuss the results with the learner by asking

him or her to explain
the responses made.
5 Note the time it takes the learner to go through the
instruction.
Phase two has two primary purposes: (1) to determine how
effective the changes are that
were made as a result of phase one and (2) to determine if
learners can go through the
instruction without the assistance of the instructor. For this
phase, eight or more learners
should be selected who are a representative sample of the entire
intended group of
learners. After selecting the learners who will participate in the
small-group evaluation,
the steps to this phase are:
1 Explain to the group of learners that you would like reactions
to new instructional
materials that have been developed; you want them to work
through the material on
their own as best they can.
2 Conduct the instruction as it is intended to be used in its final
form; give a pretest if
that is part of the instruction.
3 Observe what is taking place without involvement unless the
learners are unable to
continue the instruction; make note of the difficulties learners
have in completing the
instruction.
4 Conduct a post-test once the instruction is finished.
5 Administer an attitude questionnaire.

6 Debrief the group of learners.
The final phase—the field-trial evaluation—is intended to help
determine how effective
the changes made during the small-group evaluation are and
whether the instruction is
capable of being used in its intended environment. A
representative group of 30 learners
should be used for this phase. The environment chosen for the
field-trial evaluation should
be similar to an environment for which the instruction is being
developed.
The steps to this phase are similar to those of the small-group
evaluation. The major
difference is that an instructor, not the instructional designer,
should administer the
instruction. The role of the instructional designer is to observe
and take notes of what
takes place. The outcome of a field-trial evaluation is to
determine the attitudes of the
learners toward the instruction as well as the learners’
achievement.
In addition to these three phases, Dick et al. (2011) advise that
the instruction should
be given to subject matter experts and/or instructional experts
to note whether it is
accurate and current. This information—along with the data
gathered during the three
phases—should provide an instructional designer with enough
information to make neces-
sary revisions to the instruction being developed.
Professionals in Practice

Design and development occur in parallel—an existing course
does not require the
preliminary design steps that a new course typically requires.
Loosely, the format is
lecture-discussion-lab with an emphasis on avoiding PPT
overkill. All courseware is first
implemented in a beta environment so that the course flow can
be observed, and new labs
(continued)
01/27/2020 - RS0000000000000000000002052606 (May
(continued)
can be tested on live systems. It can be a stressful exercise. In
some cases, “secret beta”
classes are conducted (typically by the instructional designer) in
which students don’t real-
ize that what they are experiencing is new. Formative
evaluation is conducted every step
of the way until a class is released—warts and all. After a beta
class, student feedback is
collected via questionnaire, and post-mortems are conducted
with any instructors who
were in attendance. Everything related to instruction is fair
game. Here are some recent
examples:
• Are fonts in the PPT too small to be seen from the last

row in the room?
• Does a lab work with one web browser but not another?
• Are numbered steps in a lab out of order?
• Should number ordering restart after each section in a
lab?
• Should a PPT slide which introduces a lab include
starting *and* ending pages?
• Is the lab so simple that a monkey could do it by just
blindly following instructions
(remember the whole “why” issue)?
If courseware issues are too complex to fix within a short
timeframe, we try to fix them
the next time around. It’s very much an iterative process.
Erik Novak
Technical Training Developer
F5 Networks
Rapid Prototyping and Usability Testing
In addition to the three approaches we described in the previous
section, two additional—
and related—approaches to formative evaluation are important
for instructional designers
to understand and have as part of the instructional design skill
repertoire: rapid prototyp-
ing and usability testing. Both are often overlooked as
formative evaluation approaches
because they are traditionally thought of as “product processes”
rather than formative
evaluation processes. However, built in to each approach is
formative evaluation, which
allows the instructional designer to determine how well the

instructional intervention is
being designed.
As we discuss in Chapter 1, the essential idea behind rapid
prototyping is to arrive at a
final product through the creation of a series of prototypes.
Each prototype is evaluated
by some combination of experts and end users; each successive
prototype is more like
the final product—that is, “fidelity” of the prototypes increases
with each new one until
a working product is achieved. A rapid prototyping approach
requires that the design
environment allow for the relatively quick and easy creation of
instructional materials
(Tripp & Bichelmeyer, 1990).
Rapid prototyping suggests that each member of the
design/development team is a
“co-inquirer” (Rathburn, Saito, & Goodrum, 1997) and that
there is a social process of
design and development in which everyone offers feedback and
criticism to each other.
With rapid prototyping, each time a new version of the product
is tested, it provides a way
to critically reflect on what the final product should actually
look like (Rathburn et al.,
1997). The analysis of learners’ needs and the content of the
final product depend in part
01/27/2020 - RS0000000000000000000002052606 (May

on the knowledge that is gained by actually building and trying
out a prototype (Tripp &
Bichelmeyer, 1990).
Usability testing is a concept originally borrowed from
engineering that has gained
popularity among software developers, multimedia producers,
and instructional designers.
Usability testing is a type of formative evaluation that consists
of evaluating the product
by observing it in action with potential end users. Typically, a
usability test consists of
evaluators observing a user or users interacting with the product
under controlled condi-
tions. The user(s) interact with the product for a specified
amount of time, having been
asked to complete a set of tasks determined in advance by the
evaluation team. The tasks
are based on the goals and objectives the product is designed to
meet. Results of a usability
test inform the production team of the problems inherent in the
design (e.g., the graphic
for a button is not well understood or the instructions are
worded in a manner that is
confusing) and suggest—through observation of end users in
action—ways to effectively
eliminate those problems.
Usability testing is key. There’s no way to know whether your
user interface is actually
usable, unless you test it with real users. This may sound
pedantic, but you’d be amazed

how frequently this step is overlooked in user experience
design. Validating design with
fellow designers, developers, managers, or any other internal
people, is simply not enough
to ensure that end users will be able to interact with your UI
efficiently.
Usability test early and often, WITH USERS. Test your idea,
your paper prototypes,
your interactive prototype, and developed product. Early testing
gives you time to fix
things before development is underway (a much less costly
timeframe to make major
changes to a workflow or page layout). Later testing helps to
confirm that your design
survived its translation into completed code.
Usability testing resolves arguments. Design sessions can get
heated, and rival opinions
emerge, on the “right” or “best” or “most usable” solution.
Users cut right through that
argument. They don’t care who designed something, or how
much time you spent on a
particular design. Either they can use it or they can’t. If you
have more time, you can test
multiple ideas. If you have less time, you can test and refine
one design (and hope you
had a good starting point). Either way, the user is your
objective third party to shut down
disagreements.
Usability testing doesn’t have to be complicated. A one-on-one
qualitative usability test
can be completed with as few as five users (one at a time, that
is) and will give you a high
level of confidence that you have identified key issues. Large-

scale tests (100+ users) can
be done online and there are many third-party providers of
systems to support this. And
don’t hesitate to invite your team to observe a test! There’s a
huge difference between
you reporting the results of a user test, versus a developer or a
designer, with his own
eyes, watching a user struggle with a “perfect” design.
There’s an art and a science to usability testing. Identifying and
creating tasks for users
to perform is a critical part of it. “Use the website” or “Give us
your opinions” are not
tasks! A good task includes a goal, such as “Send an email to
the instructor that includes
(continued)
01/27/2020 - RS0000000000000000000002052606 (May
(continued)
an attachment” or “Where can you find information about office
hours?” that involves
observable interaction with the UI.
At Ipswitch, we build complex applications for network
administrators. Our software
is data-heavy and designed to be used by people with deep
technical skills. Even so, we

can’t assume that just because our users are technically savvy,
they’re able to interpret
our user interfaces. What might seem blindingly simple to us
(since we’ve been staring at
our designs for months on end) will be baffling to these
technical adepts. So, we test. And
change our designs. And test again. And again, until we get to a
point of confidence that
our user interface is acceptable to the people we are asking to
use it.
Lisa Hansen
Senior Manager, User Experience
Ipswitch, Inc.
Tobias realizes, after talking separately to various members of
his instructional design
team, that they need to formally engage in some group
processing. There seem to
be some misconceptions regarding team member
responsibilities. Although the group
processing will most likely be initially uncomfortable for
everyone and it will take time
away from working on their product, Tobias believes it will
help refocus the team.
Elizabeth believes that she needs to conduct a formative
evaluation to determine if
the workshops had a positive impact on participants. She is
specifically interested in
determining if and how faculty are using the web-based
synchronous communication
and learning tools in their courses.
Designing and Conducting a Formative Evaluation

Seasoned instructional designers understand the importance of
formative evaluation in
helping create efficient and effective instruction. Because of
this understanding, they con-
sistently include plans for formative evaluation when beginning
ID projects. The approach
to formative evaluation an instructional designer will take is
dictated by the scope of the
ID project. It may not be feasible for every ID project to
conduct a complete formative
evaluation by using one of the three approaches described in
this chapter. A less robust
formative evaluation may need to be developed and carried out
for smaller ID projects.
No matter what the scale of an ID project is, it is extremely
important that some form of
formative evaluation is carried out.
Summative Evaluation
As part of the instructional design process, summative
evaluation takes place after an
instructional intervention has been implemented. The major goal
of a summative evalua-
tion is to gather data that allow for its effectiveness to be
determined. Did the instruction
bring about the desired changes? Were the goals of the client
met? These are two major
questions that summative evaluations help to answer.
The results of a summative evaluation are described in a formal
report outlining the
impact of the instructional intervention. The evaluation report
will typically outline who

01/27/2020 - RS0000000000000000000002052606 (May
participated in the instructional intervention, what activities
impacted them, and what
changes occurred from their participation. The evaluation report
will often include the
costs and benefits of implementation, descriptions of the
essential conditions necessary to
continue the program or reproduce it, and recommendations on
whether the instructional
intervention should be modified, discontinued, or continued as
is.
Approaches to Summative Evaluation
For the beginning instructional designer, summative evaluation
can be a daunting task.
It is a complex process that takes a great deal of skill and
experience to successfully
carry out. There are evaluation experts whose sole professional
activity is to conduct
summative evaluations; they are typically referred to as
program evaluators. A program
evaluator uses various approaches to conduct a summative
evaluation. In Program evalu-
ation: Alternative approaches and practical guidelines (4th
edition), Fitzpatrick, Sanders,
and Worthen (2004) describe six such approaches (see Figure
10.1). It is beyond the scope
of this book to describe the intricacies of these approaches and

how to become a program
evaluator. If you wish to learn more about summative evaluation
than what is presented
in this chapter, we suggest that you start by reading their book
and by taking a course on
program evaluation. The knowledge and skills gained as a result
will certainly help you as
an instructional designer.
Criterion Formative Evaluation Summative Evaluation
Purpose To determine value or quality To determine value or
quality
Use To improve a program or
instruction
To make decisions about the
instruction’s future or adoption
Audience Program administrators and staff Program
administrators and/or potential
consumer or funding agency
By Whom Primarily internal evaluators,
supported by external evaluators
External evaluators, supported by
internal evaluators in unique cases
Major Characteristics Provides feedback so program
personnel can improve it
Provides information to enable program
personnel to decide whether to continue
it, or consumers to adopt it

Design Constraints What information is needed?
When?
What evidence is needed for major
decisions?
Purpose of Data Collection Diagnostic Judgmental
Measures Sometimes Informal Valid and Reliable
Frequency of Data Collection Frequent Infrequent
Sample Size Often Small Usually Large
Questions Asked What is working? What needs to be
improved? How can it be improved
What results occur? With whom? Under
what conditions? With what cost?
Figure 10.1 Differences between formative and summative
evaluation.
Source: Fitzpatrick, J., Sanders, J., & Worthen, B. Program
evaluation: Alternative approaches and practical guidelines, 4th
edition. 2004. Printed and electronically adapted by permission
of Pearson Education, Inc., Upper Saddle River, New Jersey.
01/27/2020 - RS0000000000000000000002052606 (May

We focus our discussion of summative evaluation on four
different approaches that
instructional design experts advocate. The purpose is to
introduce how summative
evaluation is approached in instructional design; it is not to
make you an expert.
One of the most cited approaches to summative evaluation in ID
is Kirkpatrick’s
Four Levels of Evaluation. Kirkpatrick developed this model to
evaluate the effective-
ness of training programs—specifically, training programs in
industry. The four levels
of his model are: (1) reactions, (2) learning, (3) transfer, and (4)
results. According to
Kirkpatrick (1994), evaluation should always begin with level
one and then progress
through the remaining levels as time and the budget allows.
However, to truly evaluate
the effectiveness of a training program, all four levels should be
used.
Each of Kirkpatrick’s levels requires more time and rigor than
previous levels. Data col-
lected from each level serve as a base for the next level and the
evaluation that takes place
at that level. As each level is conducted, more precise measures
of the effectiveness of the
training program are gathered.
Level 1—reactions—attempts to provide data on how
participants reacted to the train-
ing. Did participants enjoy the training? Was the training
relevant to the participants?
Kirkpatrick (1994) indicates that all training programs should at

least include this mini-
mum level of evaluation in order to provide data that will help
improve the training. A
typical method used to gather data at this level is an attitude
survey, in which participants
indicate how satisfied they were with the training. If students
indicate negative feelings
toward the training, this typically shows that little to no
learning took place (however,
positive reactions do not necessarily indicate that learning took
place).
Level 2—learning—is conducted to determine whether
participants’ skills, knowledge,
or attitudes changed as a result of the training. Determining this
is much more laborious
than Level 1 because it requires gathering data at multiple
times. Typically, pretests and
post-tests are used to measure these changes.
Level 3—transfer—attempts to answer the question of whether
the newly acquired
skills, knowledge, or attitudes are being used by participants in
their real-world environ-
ments. In other words, have participants transferred what they
learned in the training into
their everyday environments? This is often considered to be the
truest measure of a train-
ing program’s effectiveness. Evaluating at this level is
complicated because it is difficult
to determine when a participant will actually display what he or
she learned. Therefore,
decisions will need to be made on when, how often, and how
long the evaluation will take
place in order to determine if transfer has taken place.

Level 4—results—attempts to evaluate a training program’s
effectiveness in business
measures, such as increased sales, improved product quality,
fewer on-the-job accidents,
and so forth. Evaluation at this level must be long term in
nature in order to determine
trends that have taken place. Data that help support the
effectiveness of training at this
level are often gathered through methods such as interviews
with managers, focus group
meetings with customers, and post-training surveys. Smith and
Ragan (1999) write:
Within the context of instructional design, the purpose of
summative evaluation is to
collect, analyze, and summarize data to present to decision
makers in the client orga-
nization so that they can make a judgment regarding the
effectiveness, and perhaps
appeal and efficiency, of the instruction.
(p. 352)
The judgment that is being made is whether the use of the
instruction should be continued.
The specific question being asked is, “Does the instruction
adequately solve the ‘problem’
that was identified in the needs assessment and that resulted in
the development of the
01/27/2020 - RS0000000000000000000002052606 (May

instructional materials?” (p. 353). They provide an eight-step
process for conducting a
summative evaluation:
1 Determine the goals of evaluation: Identify the questions that
should be answered.
The questions should be developed in consultation with the
client. Typically, more
questions will be developed than can be answered during the
course of the evaluation.
It is important to come to an agreement on the exact questions
that are feasible to
answer given the available resources.
2 Select indicators of success: Where will data be gathered?
What needs to be looked at
in order to answer the evaluation questions? Again, in
consultation with the client,
these issues need to be addressed and agreed on.
3 Select the orientation of evaluation: Will the evaluation be
objective or subjective? An
objective orientation focuses on answering the evaluation
questions based on data
collected through quantitative methods (e.g., questionnaires,
paper-and-pencil tests, per-
formance tests). The advantage of this orientation is that results
are generally replicable.
A subjective orientation is based on the perspective of the
individual conducting the
evaluation. Qualitative methods (e.g., interviews and
observations) are used to gather
data that describe the impact of the instructional intervention.
The advantage of this ori-

entation is that a rich description is provided that describes the
impact the instructional
intervention has had. The downside to this orientation is that
the results can be influ-
enced by the biases of the evaluator. Most summative
evaluations do not strictly follow
one or the other orientation; typically, a combination of the two
orientations is used.
4 Select design of evaluation: Determine how the data will be
collected, when they will
be collected, and under what conditions.
5 Design or select evaluation measures: Decide what measures
will be looked at to
determine effectiveness of the instructional intervention.
Learning transfer, learning
outcomes, attitudes, level of implementation, and costs are all
measures that can be
used to determine effectiveness.
6 Collect data.
7 Analyze data.
8 Report results: A summative evaluation report should include
the following sections:
summary, background information, description of the evaluation
study, results, and
conclusion and recommendations. The background information
will come from the
analyses conducted early on in the ID process (e.g., needs,
learner). The description of
the evaluation study includes the purpose of the evaluation, the
evaluation design, and
the outcome measures. The results will include a discussion on
how well the outcomes

were met.
The approach taken by Morrison et al. is similar in many ways
to the approach described
by Smith and Ragan. Morrison et al. (2007) write: “A
summative evaluation permits a
designer or instructor to reach unbiased objective answers to
evaluation questions con-
cerning expected program outcomes and to then decide whether
the program is achieving
those outcomes” (p. 318). A summative evaluation can help
examine the following issues:
• Effectiveness of learner or trainee learning.
• Efficiency of learner or trainee learning.
• Cost of program development and continuing expenses in
relation to effectiveness
and efficiency.
• Attitudes and reactions to the program by learners,
instructors, and staff.
• Long-term benefits of the instructional program.
(p. 320)
01/27/2020 - RS0000000000000000000002052606 (May
These issues fall within three major areas: program
effectiveness, program efficiency,
and program costs. A program is referring to the instructional

intervention that has been
designed and implemented. Evaluating program effectiveness is
the major issue that is
dealt with by a summative evaluation. The program
effectiveness helps answer this ques-
tion: “How successful have learners been in meeting the
learning objectives?” Specifically,
what is being determined is whether change has taken place
with the learners. For exam-
ple, are employees able to do their jobs more efficiently and
effectively after participating
in an instructional intervention?
Morrison et al. (2007) list the major steps of a summative
evaluation process as:
1 Specifying program objectives: Revisit the instructional goals
and objectives of the
instructional intervention that was developed.
2 Determining the evaluation design for each objective: How
will data be collected that
will help determine if the learning goals and objectives have
been met? Determine
what types of data are needed.
3 Developing data collection instruments and procedures for
each objective: Appropriate
data collection instruments and procedures were discussed
earlier in this chapter.
Pretests, post-tests, questionnaires, and observations are all
examples of data collec-
tion instruments or procedures.
4 Carrying out the evaluation: It is advised that data are
collected from the beginning

stages of the project. This will ensure that the necessary data
are collected, especially
data regarding costs and time involvement. Data collection may
need to be scheduled.
5 Analyzing the results from each instrument.
6 Interpreting the results.
7 Disseminating the results and conclusions: Develop a
summative evaluation report
(refer to the previous section on Smith and Ragan (1999) to see
how an evaluation
report can be formatted). Individual discussions and group
presentations are often
useful (and required by the client) to disseminate evaluation
findings.
Dick et al. (2011) advocate a very different approach from the
two previous approaches
we described. Despite the different look and how it is
conducted, the fundamental goal is
the same as the other two approaches: to determine the
effectiveness of the instructional
intervention. Dick et al. (2011) define summative evaluation as
“the design of evaluation
studies and the collection of data to verify the effectiveness of
instructional materials with
target learners” (p. 320). Its major purpose is to decide whether
currently used instruc-
tional materials should continue to be used or whether new
instructional materials need
to be adopted that have the potential to meet the instructional
needs of an organization.
They state that a summative evaluation has two main phases: an
expert judgment and

a field trial. The purpose of the expert judgment phase is to
determine whether instruction
that is currently being used or instruction that is being
considered for use has the potential
to meet the instructional needs of an organization. The field-
trial phase has the purpose
of documenting the effectiveness of “promising instruction”
given to learners from the
intended target group given in the actual setting. There is a
series of steps that takes place
during each phase that helps reach these purposes.
Once I have built all of my instructional media and developed
the course, the implementa-
tion piece is simply to run it. According to the ADDIE model,
evaluation comes at the end.
01/27/2020 - RS0000000000000000000002052606 (May
I disagree. For me the evaluation piece is constant and
necessary. Your users are always
very quick at letting you know what is working and what is not.
You may find that you
need to make changes to both content and design long before
any assessment is given or
the course is completed. This is a good thing! By making your
evaluation constant, you
are making your course instructionally sound and solid now and

not later, giving current
users the best experience.
Kara Andrew
Manager, Instructional Design
University Extended Education
California State University, Fullerton
Designing and Conducting a Summative Evaluation
In an ideal situation, the instructional designer would be
directly involved in designing the
summative evaluation but would not be responsible for
conducting it. The evaluation would
be conducted by an evaluator outside of the organization that
contracted the ID project.
Having the instructional designer responsible for designing and
conducting the summative
evaluation is often seen as a conflict of interest because the
instructional designer is biased
and has a tremendous stake in the project’s success. However,
this is not to imply that an
instructional designer cannot be unbiased; it only complicates
the situation.
As we mentioned, conducting a successful summative
evaluation is a complex process
that takes a great deal of skill and experience. If at all possible,
we advise that begin-
ning instructional designers do not take on the role of primary
evaluator for a summative
evaluation. However, the reality is that, as an instructional
designer, you will be required to
conduct a summative evaluation. If this is the case, it is
important to plan for a summative

evaluation at the beginning stages of an ID project. Discuss
with the client what the expec-
tations are for the summative evaluation. Especially important
to discuss are the summative
evaluation goals. Although these may change once the
summative evaluation is formally
developed, the goals will help an instructional designer know
what the client expectations
are for the ID project. In addition, planning in advance will
allow for data to be gathered
through the formative evaluation that can be used in the
summative evaluation.
The evaluation goals will dictate the remaining steps that are to
be taken in the
summative evaluation. A process, such as the one advocated by
Smith and Ragan, can
be used as a guiding model for a beginning instructional
designer. As with successful
ID projects, continual and consistent communication with the
client is a key element
in determining the level of success an instructional designer
will have in conducting a
summative evaluation.
Group Processing: Evaluating the Instructional
Design Team
It is rare for an instructional designer to work alone. Most
projects require that instruc-
tional designers work in teams that often consist of individuals
with various roles, such
as subject matter expert, programmer, graphic artist, content
editor, web developer, or
media developer—to name a few. In addition to working with
these individuals on a team,

the instructional designer is often called on to be the project
manager and manage the
team. Being the project manager requires that the instructional
designer is able to manage
all aspects of the ID process, including how the team works
together.
01/27/2020 - RS0000000000000000000002052606 (May
Although project management is formally dealt with in Chapter
11, one aspect we will
focus on now deals with evaluating how well a team is working
together. A technique that
can be used to accomplish this is group processing. Group
processing can be considered a
type of formative evaluation. Group processing refers to the
reflections that group mem-
bers have on their work and interactions with team members.
The reflections help the
team focus on improving the working relationship of the group
to ensure that the team
effectively meets project goals. It allows team members to voice
concerns, share successes,
and provide feedback that will help the team to complete
projects successfully.
Johnson, Johnson, and Holubec (1998) have written much about
group processing.
Although their work focuses on teams in a classroom
environment, what they have writ-

ten is appropriate for instructional design teams. We have often
used group processing in
the instructional design projects we have worked on. It has
allowed us to form positive
working relationships within our teams. According to Johnson
et al. (1998), group
processing helps:
• improve the quality of how the team approaches tasks;
• increase individual accountability of team members by
focusing attention on the tasks
a team member must complete;
• the team to learn from each other by spending time
discussing the tasks individuals are
completing;
• eliminate problems that may be occurring.
Group processing is a time when team members can share
information. Four different
parts of processing should be included: feedback, reflection,
goal improvement, and
celebration. Feedback should be given to team members about
how well they are com-
pleting their tasks and how well the team is working together.
Team members should
then have time to reflect on the feedback they have received. If
improvement needs to be
made on how the team is working together, then the team should
discuss ways to improve
and come up with a plan of action. Additionally, the team
members should help each
other and set goals for improving the quality of their work.
Finally, the team should cel-
ebrate the hard work of the team members and the team’s

success (Johnson et al., 1998).
Tobias decided to approach group processing with his team by
using a multipronged
approach. The initial phase was carried out during a weekly
team meeting, where
Tobias described the situation as he currently saw it. He
allowed team members to
voice concerns about how the project was progressing and how
the team was work-
ing together. The second phase consisted of individual meetings
with team members,
where Tobias provided direct feedback to each member on his
or her work and the
contributions each was making. During these meetings, Tobias
clarified team member
duties and expectations. The third phase took place at a team
meeting. He clarified
to the group the various team member roles and expectations.
Tobias also provided
a time for praises and wishes. The praises included successes of
the team and team
members. The wishes consisted of constructive criticism of how
the team needed to
progress and what was expected for the team during the
upcoming week.
Elizabeth settled on Kirkpatrick’s Four Levels of Evaluation as
a framework to guide
her evaluation. This framework would allow Elizabeth to gather
data on multiple
levels to provide a holistic picture of how effective her
workshops were in influencing
participants to use the synchronous tools in their courses.
Elizabeth hoped to answer
the following questions as part of the evaluation:

01/27/2020 - RS0000000000000000000002052606 (May
• Did the workshops Elizabeth conducted provide faculty
with the incentives,
knowledge, and skills faculty needed?
• Did faculty members take what they learned and apply it
to their own courses?
• If so, did the use of these tools positively influence the
teaching and learning pro-
cesses in the courses?
The data gathered from the evaluation will allow Elizabeth to
determine the success
of her workshops. She will be able to use the data to improve
her workshops.
Product and Process Evaluation and the Instructional
Design Process
Determining the success of the instructional design product and
process is a significant
task that can help lead to an improved ID product and process.
Although there are various
approaches to conducting an evaluation, the questions an
instructional designer will need
to answer before beginning an evaluation are the same. These
are:

• What is the goal of the evaluation?
• Who is asking for the evaluation?
• What type of evaluation is needed?
• When does the evaluation need to take place?
• What are the questions that need to be answered?
• What types of data need to be gathered?
• Do the data currently exist?
• What instruments are needed to gather the data?
• How will the data be gathered?
• How will the data be analyzed?
• How will the data be reported?
• How will the results be used to improve the instructional
design product or process
or both?
Summary
Sustained and varied evaluation is a crucial element of the
instructional design process.
Evaluation provides information that allows decisions to be
made about learners, instruc-
tion, and the ID process. Formative, summative, and learner
evaluations are three types of
evaluation that instructional designers commonly employ.
Formative evaluation is used throughout the ID process to
gather data that can be
used to provide feedback on how the process is going.
Typically, formative evaluation is
used to provide feedback on the instruction being developed.
Feedback obtained during
a formative evaluation can help instructional designers make
changes to the instruction
that makes it more efficient and effective. Formative evaluation

should be built into an ID
project from the beginning stages.
Although ID experts conduct formative evaluation for a
common reason, the approaches
they take are different. We described three approaches to
formative evaluation. Dick et al.
(2011) advocate three basic phases of formative evaluation: (1)
one-on-one or clinical,
(2) small group, and (3) field trial. Smith and Ragan (1999)
provide a four-step approach to
formative evaluation: design reviews, expert reviews, learner
validation, and ongoing eval-
uation. Morrison et al. (2007) recommend Gooler’s (1980)
eight-step process: purpose,
audience, issues, resources, evidence, data gathering techniques,
analysis, and reporting.
01/27/2020 - RS0000000000000000000002052606 (May
Summative evaluation takes place after an instructional
intervention has been imple-
mented. The major goal of a summative evaluation is to gather
data that allow for its
effectiveness to be determined. Two main questions are to be
answered: “Did the instruc-
tion bring about the desired changes?” and “Were the client’s
goals met?”
It is often considered a conflict of interest to have the

instructional designer conduct
a summative evaluation on an ID project he or she has worked
on. If at all possible, an
instructional designer should not be the primary evaluator. He
or she should act in a sup-
porting role by helping design the evaluation and provide
necessary data. However, reality
dictates that instructional designers are asked to conduct
summative evaluations on proj-
ects they have worked on. To ensure success of a formative
evaluation, the instructional
designer should meet with the client early on in the ID project
to discuss expectations of
the evaluation, especially the evaluation goals. A process such
as the one advocated by
Smith and Ragan is a good model to be used by a beginning
instructional designer.
An additional type of formative evaluation called “group
processing” should be included
as part of the instructional design process. Group processing
allows for instructional design
team members to regularly reflect on and communicate how
well the team is working to meet
project goals. It consists of four areas: feedback, reflection,
goal improvement, and celebra-
tion. Group processing helps to build healthy and productive
instructional design teams.
Connecting Process to Practice Activities
1 How would you describe to your client the differences
between learner, formative,
and summative evaluations?
2 What decisions are being made with each type of evaluation?

3 What are the major issues associated with conducting an
evaluation on an instruc-
tional design product?
4 What are the major issues associated with conducting an
evaluation on the instruc-
tional design process that an instructional design team has gone
through (or is
currently going through)?
5 Consider Elizabeth’s scenario. What issues do you see with
the approach she advo-
cated? Explain.
6 Could Tobias have taken a different approach to examining
the situation with his
instructional design team? Explain the approach.
7 Your client asks you to justify why you have suggested
multiple evaluations through-
out the instructional design process. What do you tell your
client?
8 You have been contracted by a large law firm (approximately
50 employees) to design
its intranet. The firm would like the following initial
functionalities built into the
intranet: client billing, document uploading and retrieval, and a
calendar of firm
events (e.g., court and deposition dates and times, scheduled
vacations). As part of
the contract, you have been asked to provide training to the
paralegals and legal sec-
retaries working at the firm. What type of evaluation(s) do you
include in your plan?

What do you anticipate using, and how do you communicate this
to your client?
9 You are the lead instructional designer on a project team that
has six members. Why
is it important to include group processing as part of this
instructional design project?
Describe how you would incorporate group processing.
Recommended Reading
Fitzpatrick, J., Sanders, J., & Worthen, B. (2004). Program
evaluation: Alternative approaches and
practical guidelines (4th ed.). Columbus, OH: Pearson.
Kirkpatrick, D. L. (1994). Evaluating training programs: The
four levels. San Francisco: Berrett-
Koehler.
01/27/2020 - RS0000000000000000000002052606 (May
Rathburn, G. A., Saito, R. S., & Goodrum, D. A. (1997).
Reconceiving ISD: Three perspec-
tives on rapid prototyping as a paradigm shift. In Proceedings of
Selected Research and
Development Presentations at the 1997 National Convention of
the Association for Educational
Communications and Technology (pp. 291–296). Washington,
DC: AECT.

Tripp, S. D., & Bichelmeyer, B. (1990). Rapid prototyping: An
alternative instructional design strat-
egy. Educational Technology Research and Development, 38(1),
31–44.
References
Dick, W., Carey, L., & Carey, J. (2011). The systematic design
of instruction. New York: Pearson.
Fitzpatrick, J., Sanders, J., & Worthen, B. (2004). Program
evaluation: Alternative approaches and
practical guidelines (4th ed.). Upper Saddle River, NJ: Pearson
Education.
Gooler, D. (1980). Formative evaluation strategies for major
instructional projects. Journal of
Instructional Development, 3(3), 7–11.
Johnson, D., Johnson, R., & Holubec, E. (1998). Cooperation in
the classroom. Boston: Allyn &
Bacon.
Kirkpatrick, D. L. (1994). Evaluating training programs: The
four levels. San Francisco: Berrett-
Koehler.
Morrison, G. R., Ross, S. M., & Kemp, J. E. (2007). Designing
effective instruction (5th ed.). New
York: John Wiley & Sons.
Rathburn, G. A., Saito, R. S., & Goodrum, D. A. (1997).
Reconceiving ISD: Three perspec-
tives on rapid prototyping as a paradigm shift. In Proceedings of
Selected Research and
Development Presentations at the 1997 National Convention of

the Association for Educational
Communications and Technology (pp. 291–296). Washington,
DC: AECT.
Smith, P. L., & Ragan, T. J. (1999). Instructional design.
Hoboken, NJ: John Wiley & Sons.
Smith, P. L., & Ragan, T. J. (2004). Instructional design (2nd
ed.). Hoboken, NJ: John Wiley &
Sons.
Tripp, S. D., & Bichelmeyer, B. (1990). Rapid prototyping: An
alternative instructional design
strategy. Educational Technology Research and Development,
38(1), 31–44.
01/27/2020 - RS0000000000000000000002052606 (May

Running Head INSTRUCTIONAL DESIGN MODELS .docx

More Related Content

Similar to Running Head INSTRUCTIONAL DESIGN MODELS .docx (20)

More from jeanettehully (20)

Recently uploaded (20)

Running Head INSTRUCTIONAL DESIGN MODELS .docx