TSLB3143 Topic 1c Experimental Research

Research Design:
Experimental & Quasi-
experimental
Lecturer: Yee Bee Choo
IPGKTHO
Topic 1c

What is Experimental Design
(ED)
• Experimental design (also called intervention
studies or group comparison studies) are
procedures in quantitative research in which
researcher determines whether an activity make
a difference in results for participants.
• You assess this impact by giving one group one
set of activities (called an intervention) and
withholding the set from another group.
(Creswell, 2012)

What is an Experimental
Design
• Experimental researchers test an idea (or
practice or procedure) to determine its effect on
an outcome.
• Researchers decide on an idea with which to
“experiment,” assign individuals to experience it
(and have some individuals experience
something different), and then determine
whether those who experienced the idea or
practice performed better on some outcome
than those who did not experience it.

When do Use Experimental
design
• You use an experiment when you want to
establish possible cause and effect between your
independent and dependent variables.
• This means that you attempt to control all
variables that influence the outcome except for
the independent variable.
• Then, when the independent variable (IV)
influences the dependent variable (DV), we can
say the independent variable “caused” or
“probably caused” the dependent variable.

Independent Vs Dependent
Variables
• The IV is the predictor variable whereas the DV is
the outcome variable.
• Researchers manipulate and control the IV to
study it's effect on the DV.

Variables
Independent Variable
• an activity or characteristic believed to make a
difference with respect to some behavior
• (syn.) experimental variable, cause, treatment
Dependent Variables
• the change or difference occurring as a result of
the independent variable
• (syn.) criterion variable, effect, outcome,
posttest

Variables
Examples of IV and DV:
• If a scientist conducts an experiment to test the
theory that a vitamin could extend a person’s life.
• IV: (the variable controlled by the experimenting
scientist)
• the amount of vitamin that is given to the
subjects within the experiment.
• DV: (the variable being affected by the
independent variable)
• life span.

Variables
More Examples of IV and DV:
1. A scientist studies the impact of a drug on cancer.
IV: the administration of the drug - the dosage and the timing.
DV: the impact the drug has on cancer.
2. A scientist studies the impact of withholding affection on
rats.
IV: the amount of affection.
DV: the reaction of the rats.
3. A scientist studies how many days people can eat soup until
they get sick.
IV: the number of days of consuming soup.
DV: the onset of illness.

Types of ED
• A “true” experiment involves random
assignment of participants to groups or units.
This form of an experiment is the most rigorous
and controlled of all types.
• A quasi-experimental design involves the use of
an intervention, but not random assignment of
participants to groups.

Types of ED
• Quasi-experimental design is used when the
researchers need to use intact groups.
• This might happen because of the availability of
the participants or because the setting prohibits
artificial groups.
• It includes assignment, but not random
assignment of participants to groups.
• It is done to avoid disrupting classroom
learning.

Types of ED
• With an experimental research study, the
participants in both the treatment and control
groups are randomly assigned.
• Quasi-experimental research designs do
not randomly assign participants to treatment
or control groups for comparison.
• Quasi-experimental designs are often used in
educational research since students and
teachers are not typically able to be randomly
assigned to districts, schools, or classrooms for
research purposes.

Characteristics of ED
1. Random assignment
2. Control over extraneous variables
3. Manipulation of the treatment conditions
4. Outcome measures
5. Group comparisons
6. Threats to validity

1. Random Assignment
• Random assignment is the process of assigning
individuals at random to groups or to different
groups in an experiment.
• It is used to avoid any bias so that the personal
characteristics of individuals in the experiment is
distributed equally among the groups.
• Random assignment is a decision made by the
researcher before the experiment begins.

• By randomisation, you provide control for
extraneous characteristics of the participants that
might influence the outcome (e.g., student ability,
attention span, motivation).
• The experimental term for this process is “equating”
the groups.
• Equating the groups means that the researcher
randomly assigns individuals to groups and equally
distributes any variability of individuals between or
among the groups or conditions in the experiment.

• Other control procedures you can use both before
and during the experiment are:
1. Pre-tests (e.g. tests given before a treatment)
2. Covariates (e.g. scores on pre-test)
3. Matching of participants (e.g. 10 boys and girls)
4. Homogeneous samples (select people who vary
very little in their personal characteristics, e.g.
academic performance, racial group)
5. Blocking variables (e.g. age: equal number of 16
and 17 years old)

• The researchers physically manipulate the
treatment conditions for one or more groups.
• Before beginning the experiment, the researcher
(randomly) assigns his/her sample to two
different groups: the control group and the
experimental (treatment group or clinical group).
• The control group receives no manipulation of
the IV (no treatment), whereas the experimental
group receives the manipulation of the IV.

• In experimental treatment, the researcher physically
intervenes to alter the conditions experienced by
the experimental unit.
Example of procedure:
1. Identify a treatment variable: type of classroom
instruction in the civics class
2. Identify the conditions (or levels) of the variable:
classroom instruction can be (a) regular topics or
(b) topics related to the health hazards of smoking
3. Manipulate the treatment conditions: provide
special activities on health hazards of smoking to
one class and withhold them from another class

(Creswell, 2012, p.302)

4. Outcome measures
• In experiments, the outcome (or response,
criterion, or posttest) is the dependent variable
that is the presumed effect of the treatment
variable.
• It is also the effect predicted in a hypothesis in
the cause-and-effect equation.
• Examples of DV in experiments might be:
• Achievement scores on a criterion-referenced test
• Test scores on an aptitude test

• A group comparison is the process of a
researcher obtaining scores for individuals or
groups on the dependent variable and
comparing the means and variance both within
the group and between the groups.
• This is accomplished by statistically comparing
the groups.

https://cft.vanderbilt.edu/guides-sub-pages/test-enhanced-
learning-using-retrieval-practice-to-help-students-learn/
1. students who studied
the passages four times
for five minutes each
(SSSS group);
the passages three
times and completed
one recall test in which
they were given a blank
sheet of paper and
asked to recall as much
of the passage as they
could (SSST group);
the passages one time
and then performed the
recall practice three
times (STTT group).

• Threats to validity refer to specific reasons for
why we can be wrong when we make an
inference in an experiment because of
covariance, causation constructs, or whether the
causal relationship holds over variations in
persons, setting, treatments, and outcomes
(Shadish, Cook, & Campbell, cited in Creswell,
2012).

Internal Validity (Participants)
• History
• Maturation
• Regression
• Selection
• Mortality
• Interaction with selection
External Validity
• Interaction of selection and treatment
• Interaction of setting and treatment
• Interaction of history and treatment

Advantages of ED
• Gains insight into methods of instruction.
• Researcher can have control over variables.
• Use to determine what is best for population.
• Internal validity is ensured.

Disadvantages of ED
• Subject to human error.
• Can produce artificial results.
• Can take an extensive amount of time to do full
research.
• Personal bias of researcher may intrude.
• Sample may not be representative.
• Groups may not be comparable.
• Participants can be influenced by environment.

Advantages of Quasi-ED
• More feasible because it often does not have the
time and logistical constraints associated with
many true experimental designs.
• It may reduce the time and resources required
because extensive pre-screening and
randomisation is not required or utilized.
• Reactions of test subjects are more likely to be
genuine because it is not an artificial research
environment.

Disadvantages of Quasi-ED
• Lack of internal validity
• The lack of random assignment into test groups
leads to non-equivalent test groups which can
limit the generalisability of the results to a larger
population.
• Statistical analysis may not be meaningful due to
the lack of randomisation and the threats to
internal validity.
• Pre-existing factors and other influences are not
taken into account because variables are less
controlled
• Subject to human error.

Steps in Conducting ED
1. Decide if an experiment address your research problem
2. Form hypotheses to test Cause-and-Effect relationships
3. Select an experimental unit and identify study
participants. The researchers might randomly assign
individuals to groups.
4. Select an Experimental Treatment and Introduce It
5. Choose a type of experimental design
6. Conduct the experiment
7. Organise and analyse the data
8. Report the results

A Good ED
1. The experiment has a powerful intervention.
2. The treatment groups are few in number.
3. Participants will gain from the intervention.
4. The researcher derives the number of participants per
group in some systematic way.
5. An adequate number of participants were used in the
study.
6. The researcher uses measures and observations that
are valid, reliable, and sensitive.
7. The researcher controls for extraneous factors that
might influence the outcome.
8. The researcher addresses threats to internal and
external validity.

Let’s do an ED
Topic
• Testing effect on retention of memory among the
undergraduates
Background of Study
• Tests are used as a process to measure what students know
or what they have learned.
• Tests can also be used as learning materials. By exposing to
test immediately, the students can retain the information
longer due to their short term memory.
Statement of Problem
• Tests can enhance retention of previously learned information
even if no additional study or feedback was provided.
• However, it has been stressed that there is a need for
additional studies of test-enhanced learning using
educationally relevant materials during a course.

Let’s do an ED
Purpose of Study
• To examine the testing effect on one’s memory
Research Objective
• To find out the difference of testing effect on one’s memory in
an immediate retention test and a delayed retention test
Research Question
• Do students who are given immediate retention test
remember information longer than students who are given a
delayed retention test?
Research Hypotheses
• Students who are given immediate retention test will
remember information longer than students who are given
delayed retention test

Let’s do an ED
Scope of Study
• Two groups of undergraduates at IPGKTHO. They are majoring
in TESL and will prepare a research proposal in a course.
Limitations of Study
• The tests are taken at two different time.
• Only two tests are taken for the experimental group.
• The constraint of time.
Significance of Study
• For students: Students could learn effectively through self-
testing or other test-related items that could enhance their
memory.
• For teachers: Students’ performance on test is often
influenced by the evaluation practices of the classroom
teacher. Therefore, teacher should give test appropriately in
the class.

Let’s do an ED
Operational Definitions
1. Testing effect
• While taking a test, one may learn or retrieve previously
learned information via some mental processes that
work on the memory. This phenomenon is called the
“testing effect” (Atabek Yiğit, Balkan Kıyıcı, & Çetinkaya,
2014).
• Therefore, testing effect in this study means the retrieval
of information when a test is given in a time that is
immediate or delayed.

Let’s do an ED
Operational Definitions
2. Retention of memory
• Memory retention and recalling are key memory
processes. Retention is the capability to hold
information, and retrieval is the recollection of held
information in the mind in response to external stimuli
(Amin & Malik, 2013).
• Therefore, retention of memory refers to students’ recall
of specific facts and concepts from the lecturer’s
instruction in class as well as their ability to answer
questions within specific time.

Tutorial 1c (Pair Work)
• Read the article from
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4235419/
• Based on the article, identify the following:
• Title
• Author
• Problem/Issue
• Suggestion(s) to overcome the problem/issue
• Research objectives/questions
• Research hypotheses
• Sample
• Research methodology
• Research instruments
• Findings/ Discussion of findings
• You can type or handwrite your answer.
• Write your answers in not more than two sentences.

Main Reference
• Creswell, J.W. (2012). Educational research:
planning, conducting and evaluating quantitative
and qualitative research. (4th Ed.). Boston, MA:
Pearson Education Inc.

Answers
• Title: Strengthening concept learning by
repeated testing
• Authors:Carola Wiklund-Hörnqvist, Bert
Jonsson, and Lars Nyberg
• Issue: Another aspect of testing that has largely
been neglected by educationalists is its potential
to serve as a way of facilitating learning
• Suggestions: (1) combining a lecture with
computer-assisted learning of key concepts
might further improve students’ knowledge
level; (2) provide feedback

Answers
• Objectives: (1) to examine whether repeated
testing with feedback, using SA questions,
promotes long-term retention relative to
rereading of key concepts during the progression
of an introductory university course. (2) to
consider how individual differences in working
memory capacity (WMC) relate to learning
ability
• Hypothesis: Testing with feedback should lead to
better performance compared to the restudy
condition both in the immediate test and at the
delayed tests.

Answers
• Sample: Eighty-three undergraduate students
registered on a cognitive psychology course aged
from 19–44 years
• Research Methodology: Experiment
• Research Instruments: Tests and questionnaire
• Findings: (1) repeated testing with feedback
significantly enhanced learning compared to
rereading at all delays, demonstrating that repeated
retrieval enhances retention compared to repeated
encoding in the short- and the long-term. (2) the
effect of repeated testing was beneficial for students
irrespectively of working memory capacity.

TSLB3143 Topic 1c Experimental Research

More Related Content

What's hot (20)

Similar to TSLB3143 Topic 1c Experimental Research (20)

More from Yee Bee Choo (20)

Recently uploaded (20)

TSLB3143 Topic 1c Experimental Research

Editor's Notes