Research Design and Methodology, Dr. W.A. Kritsonis

RESEARCH DESIGN AND METHODS

Research Design

William Allan Kritsonis, PhD

Educational research is the application of the scientific and disciplined

inquiry approach to the study of educational problems and the primary goal

of educational research is to explain and help understand educational issues,

questions, and processes. Secondary goals of educational research are to

help others understand, predict future outcomes, improve future research and

practice, and raise new questions to research.

Scientific and disciplined inquiry approach is distinguished by four

crucial steps. These are: Recognize and identify a question or a problem to

be studied, Describe and execute procedures to collect information about the

questions and problems being studied, Analyze the collected information,

and state the results or implications based on the analysis of the information.

Basic research is conducted to develop or refine theory, not to solve

immediate practical problems. Applied research is conducted to find

solutions to current practical problems. Most of the educational research

would be conducted for and classified towards the applied end of the basic-

applied research continuum. The purpose of evaluation research is to aid

decision making about educational programs and practices.

The general purpose of Quantitative research is to generalize about or

control phenomena. Quantitative research has been conducted since the

early 1920s and is the primary way educational researchers have sought to

bring about an understanding of educational issues. Quantitative methods

involve the collection and analysis of numerical data that is obtained from

test, questionnaires, checklist, and surveys. Important features of

quantitative research are: defining the problem or questions to be studied and

developing a hypotheses that predicts the results of the research before the

study begins; controlling contextual factors that might influence the results

of the study; collecting data from samples of participants; and using

numerical, statistical approaches to analyze the collected data. An important

assumption that underlines the quantitative approach is that the world that

we live in and carry out our research in is relatively stable, uniform, and

coherent; therefore, it can be measured, understood, and classified.

Qualitative research methods involve the collection and analysis of

primarily nonnumerical data obtained from observation, interviews, taped

information, documents. The important points of qualitative research are the

following: defining the problem or question to be studied; the

aforementioned point does not have to be stated at the start of the study. The

researcher also takes into account contextual factors of the research

participants; data is collected from a small number of purposely selected

research participants; and using nonnumerical, interpretive approaches to

provide narrative descriptions of the participants and their contexts. The

underlining belief of qualitative research is that the world is neither stable,

coherent, nor uniform, and therefore, "truth" as sought by quantitative

researchers cannot be obtained because perspectives and understandings

differ from group to group.

Descriptive research and survey research is synonymous in that they

both collect numerical data to answer questions. Most descriptive studies

obtain information about the preferences, attitudes, practices, concerns, or

interest of some group. The data are collected by a self-administered

instruments and or telephone polls. The difficulty in conducting descriptive

research is constructing clear, consistent descriptive instruments and trying

to acquire instruments from interviewees along with cooperation in

telephone interviews.

Correlational research examines the degree of relationship that exist

between two or more variables. A correlation is a quantitative measure of

the degree of correspondence between two or more variables. The degree of

relationship is measured by a correlation coefficient of .00 indicating no

relationship between the variables. The researcher must keep in mind that if

there is a high relationship between variables, it does not mean that one is a

cause of the other.

Causal-Comparative research seeks to investigate cause and effect

relationships between two or more programs, methods, or groups. The

activity thought to make a difference; program, method, or group is called

the causal factor, treatment, or independent variable. The "effect" is called

the dependent variable. In most casual-comparative research studies the

researcher does not have control over the causal factor because it has already

occurred or cannot be manipulated. This makes cause-effect conclusions

tenuous and tentative. Causal-comparative research is useful in those

circumstances when it is impossible or unethical to manipulate the causal

factor.

Experimental research investigates the cause-effect relationship by

selecting participants from a single pool and assigning them at random to

different causal treatments. The researcher controls contextual variables that

might interfere with the study and because of the randomness of selecting

and assigning participants into different treatments cause-effect statements

are considered to be true statements.

Historical research is a form of qualitative research that involves

interpreting past events of individuals, important social issues, links between

the old and the new, and reinterpretations of prior historical works. The

primary sources of data are first-person eyewitnesses or authors. Secondary

sources are non-first person accounts. Historians use external criticism to

assess the authenticity of their data and use internal criticism to assess the

truthfulness of their data.

Qualitative research may include a number of specific methods such

as ethnology, ethnomethodology, case study phenomenology, and symbolic

interaction. These methods focus on the deep descriptions of aspects of

people's everyday perspectives and context. Qualitative research provides

field-focused, interpretive, detailed descriptions and interpretations of

participants and their settings. Long-term immersion into the research

setting is also very common. Common methods of data collection include

observation, interviewing, tape and video recording, examining artifacts, and

participant observation. Data analysis is based on categorizing and

interpreting the observations, conversations with participants, documents,

tape recordings, and interviews collected to provide a description and

explanation of the participants and their experiences. The qualitative

researcher writes from the perspective of the participants.

There are guidelines that a researcher can follow to determine the

research method needed to be used for a particular study. The same general

problem can be investigated using many types of research. The more

information available about the nature and procedures of a study, the easier

to classify it. A structured way to classify a study is to first determine if it is

a quantitative or qualitative study. If a study is quantitative, try to identify

the purpose of the study to determine whether it is descriptive, correlational,

causal-comparative, or experimental. If the study is qualitative, determine

whether it is historical or qualitative. Look for key words in the title of the

study: survey, description, relationship, historical, culture, and the like.

Identifying a research topic may come from three main sources:

theory may be composed of generalizations and concepts, personal

experience, and replication of an existing study. A basic characteristic of a

good research problem is that it is researchable.

The stating of a research topic for a quantitative study generally

indicates variables and a well-written topic that defines relevant variables,

either directly or operationally. The topic statement is the first component in

the introductory section and will provide the direction for the study.

The review of literature involves the systematic identification,

location, and analysis of documents containing information related to the

research topic. The major reasons for the review of literature are to put your

study in context with what information already exist and it also points out

research strategies, procedures, and instruments that may be used.

Identifying your sources of information is the first step in a

comprehensive research project. Most libraries use a computer catalog

system and computer databases such as: ERIC, Educational Index,

Psychological Abstract, and Dissertation Abstracts. The Internet is a

computer link to the world -wide web that can be utilized through search

engines such as Yahoo or Lycos.

Abstracting involves locating, reviewing, summarizing, and

classifying your references. For each reference write a complete

bibliographic record including authors name, date of publication, title,

journal name or book title, volume number, issue number, page number, and

library call number.

Formulation of a statement of a hypothesis is based on the theory or

review of literature. A characteristic of a good hypothesis is that it is based

on sound rational, a reasoned prediction. A good hypothesis states as clearly

and concisely as possible the expected relationship (or difference) between

two variables and defines those variables in measurable terms. A well-stated

and well-defined hypothesis must be testable.

The six types of hypothesis discussed in the Educational Research

book are as follows: An inductive hypothesis is a generalization made from

a number of observations, a deductive hypothesis is derived from theory and

is aimed at providing evidence that supports, expands, or contradicts aspects

of a given theory, A research hypothesis states the expected relationship (or

difference) between two variables, a nondirectional hypothesis indicates that

a relationship or difference exists but does not indicate the difference, a

directional hypothesis indicates that a relationship or difference exist and

indicates the direction of the difference, and a null hypothesis states that

there will be no significant relationship (or difference) between variables.

A research plan is a detailed description of a proposed study. It

includes justification for the study, description of the steps that will be

followed in the study, and information about the analysis of the collected

data. They are always ethical considerations that must be thought of and

there are several sources that a researcher can go to for information. The

National Research Act of 1974 and The Ethical Principles of Psychologists

are two documents that have guidelines as to the rules and regulations that

one should follow while conducting research.

A design is a general strategy for conducting a research study, steps

that will take place and the order that they will take place.

Sampling is the process of selecting a number of individuals for a

study in such a way that the individuals represent the larger group from

which they were selected.

Random sampling is the process by which all individuals in the

defined population have an equal and independent chance of being selected

for the sample. Stratified Sampling is the process of selecting a sample in

such a way that the sample represents an identified subgroup in the same

proportion as they exist in the general population. Cluster Sampling is a

process in which groups, not individuals, are randomly selected. Systematic

Sampling is a process of sampling by which individuals are selected from a

list by taking every Kth name. That is K= number of individuals on the list

divided by the number of participant's need for the sample.

Instruments of measurement are as follows: Constructs are mental

abstractions such as personality, creativity and intelligence that in most

instances cannot be measured directly. Constructs become variables when

different levels or scores can be used to measure the construct.

Measurement scales describe four different levels of measurement: nominal,

which categorize, ordinal which rank, interval variables have equal intervals,

and ratio variables have a defined zero point.

Cognitive Test are as follows: achievement test that measure the

current status of an individual on a particular subject matter, Aptitude test

are used to predict how well an individual is likely to do in future test or

occupations, Affective test measure characteristics such as interest, values,

attitude, and personality. The five basic types of scales used to measure

attitudes are Likert, Semantic, Differential, Rating, Thurstone, and Guttman.

Projective test takes into account an ambiguous situation and is preformed

through the use of association.

Validity is the most important quality of a test. It refers to the degree

to which a test measures what it is suppose to measure. Validation is also a

matter of degree: highly valid, moderately valid, or generally valid. Content

validity is the degree to which a test measures an intended content area.

There is also Criterion -Related Validity that is divided into two forms:

Concurrent validity is the degree to which scores on a test are related to

scores on another test administered at the same time and Predictive validity

which refers to the degree to which scores on a test are related to scores on

another test to be administered in the future.

Reliability refers to the degree to which a test consistently measures

whatever it is suppose to measure. The five general approaches to reliability

are stability, equivalence, equivalence and stability, internal consistency, and

score/rater.

The Standard Error of Measurement is an estimate of how often one

can expect test score errors of a given size.

Qualitative research has characteristics that are common features in

this type of research. There is reliance on interpretation involving intensive

field participation, rich data collected in the field, and the researcher as the

primary synthesizing and interpretive agent of data analysis. Data analysis

typically involves a six-step process. They data managing;

reading/memoing; describing; classifying interpreting; and representing the

results.

Descriptive research involves collecting data in order to test

hypotheses or answer questions about opinions. This can be done through

self-report or through observation and is categorized in terms of cross-

sectional or longitudinal. Categories to consider are: statement of the

problem, selection of participants, construction of the questionnaire,

preparation of the cover letter, pre-testing the questionnaire, follow-up to the

questionnaire, dealing with non-response, and analysis of the results.

Correctional research involves collecting data to determine whether

and to what degree a relationship exists between two or more variables. The

degree of the relationship is expressed as a correlation coefficient. The

process is composed of the following: problem selection, participant and

instrument selection, design and procedure, data analysis and interpretation

Causal-comparative research or ex post facto research is used to

determine the cause or reason for existing differences in the behavior in a

group. The basic design involves selecting two groups differing on some

independent variable and comparing them on some dependent variable.

Data analysis involves a variety of descriptive and inferential statistics.

Experimental research utilizes the manipulation of at least one

independent variable, controls the other independent variables, and observes

the effect on one or more independent variables. The steps in the

experimental research are basically the same as the steps for other types of

research: selection and definition of a problem, selection of participants and

measuring instruments, selection of a design, execution of a procedure,

analysis of data, and formulations of conclusions.

Inferential statistics deals with the inferences about populations based

on the results of samples. This type of research allows the researcher to

generalize to a population of individuals based on the information obtained

from a limited number of research participants. There is an expected

standard error of chance variation among the means. The Null Hypothesis

says that there is no true difference or relationship between parameters in the

population. The levels of significance are four fold and along with this there

are Type I and Type II errors. When the null hypothesis is true and the

researcher rejects it and says there is a difference, he/she makes an incorrect

decision, this is a type I error. If the null hypothesis is false and the

researcher says it is true and does not reject it, the researcher also makes an

incorrect decision referred to as a type II error.

Probably the most foremost rule of research report writing is to relate

aspects of the study in a manner that accurately reflects what was done and

what was found.

GLOSSARY OF RESEARCH-RELATED TERMS

A-B design A single-subject design in which baseline measurements are repeatedly made
until stability is presumably established, treatment is introduced, and an appropriate
number of measurements are made during treatment.
A-B-A design A single-subject design in which baseline measurements are repeatedly
made until stability is presumably established, treatment is introduced, and an appropriate
number of measurements are made, and the treatment phase is followed by a second
baseline phase.
A-B-A-B design A single-subject design in which baseline measurements are repeatedly
made until stability is presumably established, treatment is introduced, and an appropriate
number of measurements are made, and the treatment phase is followed by a second
baseline phase, which is followed by a second treatment phase.
abstract A summary of a study, which appears at the beginning of the report and
describes the most important aspects of the study, including major results and
conclusions.
accessible population Refers to the population from which the researcher can
realistically select participants.
accidental sampling See convenience sampling.
achievement test An instrument that measures the current status of individuals with
respect to proficiency in given areas of knowledge or skill.
action research An approach in which teachers study their own problems or concerns in
their own classrooms.

additive designs Refers to variations of the AB design which involve the addition of
another phase or phases in which the experimental treatment is supplemented with
another treatment.
alternating treatments design A variation of a multiple-baseline design which involves
the relatively rapid alternation of treatments for a single participant.
analysis of covariance A statistical method of equating groups on one or more variables
and for increasing the power of a statistical test; adjusts scores on a dependent variable
for initial differences on some variable such as pretest performance or IQ.
analytic induction A method of identifying regularities in qualitative data, determining
their explanation, and finding other contexts to determine whether the explanations hold
up.
applied research Research conducted for the purpose of applying, or testing, theory and
evaluating its usefulness in solving problems.
aptitude test A measure of potential used to predict how well someone is likely to
perform in a future situation.
artificial categories Categories which are operationally defined by the researcher.
assumption Any important "fact" presumed to be true but not actually verified;
assumptions should be described in the procedures section of a research plan or report.

attenuation Refers to the principle that correlation coefficients tend to be lowered
because less-than-perfectly reliable measures are used.
basic research Research conducted for the purpose of theory development or refinement.
case study The in-depth investigation of one "unit," e.g., individual, group, institution,
organization, program, or document.
category The classification of ideas and concepts in qualitative data analysis.
causal-comparative research Research that attempts to determine the cause, or reason,
for existing differences in the behavior or status of groups of individuals; also referred to
as ex post facto research.
census survey Descriptive research that attempts to acquire data from each and every
member of a population.
changing criterion design A variation of the A-B-A design in which the baseline phase
is followed by successive treatment phases, each of which has a more stringent criterion
for acceptable behavior level.
chi square A nonparametric test of significance appropriate when the data are in the form
of frequency counts; it compares proportions actually observed in a study with
proportions expected to see if they are significantly different.
clinical replication Refers to the development and application of a treatment package,
composed of two or more interventions which have been found to be effective
individually, designed for persons with complex behavior disorders.
cluster sampling Sampling in which intact groups, not individuals, are randomly
selected.
coefficient alpha (a) See Cronbach's alpha.
common variance The variation in one variable that is attributable to its tendency to vary
with another variable.
concurrent validity The degree to which the scores on a test are related to the scores on
another, already established test administered at the same time, or to some other valid
criterion available at the same time.
constant comparison A qualitative method for identifying similarities and differences by
comparing new evidence to prior evidence.
construct validity The degree to which a test measures an intended hypothetical
construct, or nonobservable trait, which explains behavior.
contamination The situation that exists when the researcher's familiarity with the
participants affects the outcome of the study.
content analysis The systematic, quantitative description of the composition of the object
of the study.
content validity The degree to which a test measures an intended content area; it is
determined by expert judgment and requires both item validity and sampling validity.
control Efforts on the part of the researcher to remove the influence of any variable
other than the independent variable that might affect performance on a dependent
variable.
control group The group in a research study that either receives a different treatment
than the experimental group or is treated as usual.
control variable A nonmanipulated variable, usually a physical or mental characteristic
of the participants (such as IQ).

convenience sampling The process of using as the sample whoever happens to be
available, e.g., volunteers. (Also referred to as accidental sampling and haphazard
sampling.) correlational research Research that involves collecting data in order to
determine whether, and to what degree, a relationship exists between two or more
quantifiable variables.
correlation coefficient A decimal number between .00 and :!:1.00 that indicates the
degree to which two variables are related.
counterbalanced design A quasi-experimental design in which all groups receive all
treatments, each group receives the treatments in a different order, the number of groups
equals the number of treatments, and all groups are post tested after each treatment.
credibility A term used in qualitative research to indicate that the topic was accurately
identified and described.
criterion In a prediction study, the variable that is predicted.
criterion-related validity Validity which is determined by relating performance on a test
to performance on another criterion; includes concurrent and predictive validity.
Cronbach's alpha (a) The general formula for estimating internal consistency based on a
determination of how all items on a test relate to all other items and to the total test.
(Also referred to as coefficient alpha and Cronbach's coefficient alpha.) cross-validation
Validation of a prediction equation with at least one group other than the group on which
it was based; variables that are no longer found to be related to the criterion measure are
removed from the equation.
curvilinear relationship A relationship in which increase in one variable is associated
with a corresponding increase in another variable to a point, at which point further
increase in the first variable is associated with a corresponding decrease in the other
variable (or vice versa).
data saturation A point in qualitative research when so much data are collected that it is
very unlikely that additional data will add to what is already collected.
deductive hypothesis A hypothesis derived from theory which provides evidence which
supports, expands, or contradicts the theory.
dependent variable The change or difference in behavior that occurs as a result of the
independent variable; also referred to as the criterion variable, the effect, the outcome, or
the post test.
descriptive statistics Data analysis techniques enabling the researcher to meaningfully
describe many scores with a small number of numerical indices.
developmental studies Studies concerned with behavior variables that differentiate
children at different levels of age, growth, or maturation.
diagnostic test A type of achievement test yielding multiple scores for each area of
achievement measured that facilitate identification of specific areas of deficiency.
differential selection of participants Refers to the fact that groups may be different
before a study even begins, and this initial difference may at least partially account for
post test differences.
direct replication Refers to the replication of a study by the same investigator, with the
same participants or with different participants, in a specific setting.
ecological validity The degree to which results can be generalized to environments
outside of the experimental setting.

educational research The formal, systematic application of the scientific and disciplined
inquiry approach to the study of educational problems.
environmental variable A variable in the setting in which a study is conducted that
might cause unwanted differences between groups (e.g., learning materials).
equivalent forms Two tests identical in every way except for the actual items included.
equivalent-forms reliability Indicates score variation that occurs from form to form of a
test; also referred to as alternate-forms reliability.
ethnographic research A qualitative approach that studies the cultural patterns and
perspectives of participants in their natural setting.
ethnomethodology A qualitatiTe approach that studies how participants make sense of
their everyday activities to act in a social way.
evaluation The systematic process of collecting and analyzing data in order to make
decislons.
experimental group The group in a research study that typically receives a new, or
novel, treatment, a treatment under investigation.
experimental research Research in which at least one independent variable is
manipulated, other relevant variables are controlled, and the effect on one or more
dependent variables is observed.
experimenter bias A situation in which the researcher's expectations concerning the
outcomes of the study actually contribute to producing various outcomes.
ex post facto research See causal-comparative research.
external criticism The analysis of data to determine their authenticity.
external validity The degree to which results are generalizable, or applicable, to groups
and environments outside of the experimental setting.
factorial analysis of variance The appropriate statistical analysis if a study is based on a
factorial design and investigates two or more independent variables and the interactions
between them; yields a separate F ratio for each independent variable and one for each
interaction.
factorial design An experimental design that involves two or more dependent variables
(at least one of which is manipulated) in order to studv the effects of the variables
individually a~d in interaction with each other.
fieldwork A qualitative research strategy that involves spending considerable time in the
setting under study, immersing oneself in this setting, and collecting as much relevant
information as possible as unobtrusively as possible.
follow-up study A study conducted to determine the status of a group of interest after
some period of time.
generosity error The tendency to give an individual the benefit of the doubt whenever
there is insufficient knowledge to make an objective judgment.
grounded theory Theory based on data collected in real-world settings which reflect
what naturally occurred over an extended period of time.
halo effect The phenomenon whereby initial impressions concerning an individual
(positive or negative) affect subsequent measurements.
haphazard sampling See convenience sampling.
hardcopy Refers to computer output that is printed out on paper.
hardware Refers to the actual equipment, the computer itself and related accessories
such as printers.

Hawthorne effect A type of reactive arrangement resulting from the participants'
knowledge that they are involved in an experiment, or their feeling that they are in some
way receiving "special" attention.
historical research The systematic collection and evaluation of data related to past
occurrences in order to describe causes, effects, or trends of those events which may help
to explain present events and anticipate future events.
history Any event which is not part of the experimental treatment but which may affect
performance on the dependent variable.
hypothesis A tentative, reasonable, testable explanation for the occurrence of certain
behaviors, phenomena, or events.
independent variable An activity or characteristic believed to make a difference with
respect to some behavior; also referred to as the experimental variable, the cause, and the
treatment.
inductive hypothesis A generalization based on observation.
inferential statistics Data analysis techniques for determining how likely it is that results
based on a sample or samples are the same results that would have been obtained for an
entire population.
instrumentation Unreliability in measuring instruments that may result in invalid
assessment of participants' performance.
interaction Refers to the situation in which different values of the independent variable
are differentially effective depending upon the level of the control variable.
interiudge reliability The consistency of two (or more) independent scorers, raters, or
observers.
internal criticism The analysis of data to determine their accuracy which takes into
consideration the knowledge and competence of the author, the time delay between the
occurrence and recording of events, biased motives of the author, and consistency of the
data.
internal validity The degree to ,hich obse!'ed differences on the dependent variable are
a direct result of manipulation of the independent variable, not some other variable.
interval scale A measurement scale that classifies and ranks participants, is based upon
predetermined equal intervals, but does not have a true zero point.
intervening variable A variable which intervenes between, or alters the relationship
between, an independent variable and a dependent variable, which cannot be directly
observed or controlled (e.g., anxiety) but which can be controlled for.
intrajudge reliability The consistency of the scoring, rating, or observing of an
individual.
item validity The degree to which test items represent measurement in the intended
content area.
John Henry effect The phenomenon whereby if for any reason members of a control
groupfeel threatened or by being in competition with an experimental group, they may
out do themselves and perform way beyond what would normally be expected.
judgment sampling The process of selecting a sample which is believed to be
representative of a given population. (Also referred to as purposive sampling.) Likert
scale An instrument that asks an individual to respond to a series of statements by
indicating whether she or he strongly agrees (SA), agrees (A), is undecided (U), disagrees
(D), or strongly disagrees (SD) with each statement.

limitation An aspect of a study which the researcher knows may negatively affect the
results or generalizability of the results, but over which he or she has no control.
linear relationship The situation in which an increase (or decrease) in one variable is
associated with a corresponding increase (or decrease) in another variable.
logical validity Validity which is determined primarily through judgment; includes
content validity.
matching A technique for equating groups on one or more variables, resulting in each
member of one group having a direct counterpart in another group.
maturation Physical or mental changes which occur within participants over a period of
time and which may affect their performance on a measure of the dependent variable.
mean The arithmetic average of a set of scores.
measures of Indices representing the average or typical score attained by a group of
participants.
measures of variability Indices indicating how spread out the scores are in a
distribution.
median That point in a distribution above and below which are 50% of the scores.
menu-driven Refers to computer programs which allow the user to select desired
analyses from a list, or menu, of options.
meta-analysis A statistical approach to summarizing the results of many studies which
have investigated basically the same problem.
mode The score that is attained by more pa;..ticipants in a group than any other score.
modem A device which permits telephone communication between two computers by
converting computer language to audiotones.
mortality Refers to the fact that participants who drop out of a study may share a
characteristic such that their absence has a significant effect on the results of the study.
multiple-baseline design A single-subject design in which baseline data are collected on
several behaviors for one participant or one behavior for several participants and
treatment is applied systematically over a period of time to each behavior (or each
participant) one at a time until all behaviors (or participants) are under treatment.
multiple comparisons Procedures used following application of analysis of variance to
deterrnine which means are significantly different from which other means.
multiple regression equation A prediction equation using two or more variables that
individually predict a criterion to make a more accurate prediction.
multiple time-series design A variation of the time-series design that involves the
addition of a control group to the basic design.
multiple-treatment interference Refers to the carry-over effects from an earlier
treatment that make it difficult to assess the effectiveness of a later treatment.
naturalistic observation Observation in which the observer purposely controls or
manipulates nothing, and in fact works very hard at not affecting the observed situation in
any way.
negatively skewed distribution A distribution in which there are more extreme scores at
the lower end than at the upper, or higher, end.
nominal scale The lowest level of measurement which classifies persons or objects into
two or more categories; a person can only be in one category, and members of a category
have a common set of characteristics.

nonequivalent control group design A quasiexperimental design involving at least two
groups, both of which are pretested; one group receives the experimental treatment, and
both groups are posttested.
nonparametric test A test of significance appropriate when the data represent an ordinal
or nominal scale, when a parametric assumption has been greatly violated, or when the
nature of the distribution is not known.
nonparticipant observation Observation in which the observer is not directly involved
in the situation to be observed, i.e., the observer does not intentionally interact with or
affect the object of the observation.
nonprobability sampling The process of selecting a sample using a technique which
does not permit the researcher to specify the probability, or chance, that each member of
a population has of being selected for the sample.
novelty effect A type of reactive arrangement resulting from increased interest,
motivation, or participation on the part of participants simply because they are doing
something different.
null hypothesis States that there is no relationship (or difference) between variables and
that any relationship found will be a chance relationship, the result of sampling error, not
a true one.
observational research Descriptive research in which the desired data is obtained not by
asking individuals for it but through such means as direct observation.
observer bias The phenomenon whereby an observer does not observe objectively and
accurately, thus producing invalid observations.
observer effeds The phenomenon whereby persons being observed behave atypically
simply because they are being observed, thus producing invalid observations.
one-group pretest-posttest design A pre-experimental design involving one group
which is pretested, exposed to a treatment, and post tested.
one-shot case study A pre-experimental design involving one group which is exposed to
a treatment and then post-tested.
operational definition One which defines concepts in terms of processes, or operations.
ordinal scale A measurement scale that classifies participants and ranks them in terms of
the degree to which they possess a characteristic of interest.
organismic variable A characteristic of a participant, or organism (e.g., sex), which can-
not be directly controlled but which can be controlled for.
parameter A numerical index describing the behavior of a population.
parametric test A test of significance appropriate when the data represent an interval or
ratio scale of measurement and other assumptions have been met.
participant A person who provided data for a research study.
participant observation Observation in which the observer actually becomes a part of, a
participant in, the situation to be observed.
participant variable A variable on which participants in different groups in a study
might differ, e.g., intelligence.
pattern The connection of categories in qualitative analysis.
Pearson r A measure of correlation appropriate when the data represent either inter'al or
ratio scales; it takes into account each and every score and produces a coefficient between
.00 and ::::1.00.

percentile rank A measure of relative position indicating the percentage of scores that
fall at or below a given score.
phenomenology The experience of an activity or concept from participants' perspectives
pilot study A small-scale study conducted prior to the conducting of the actual study;
the entire study is conducted, every procedure is followed, and the resulting data are
analyzed-all according to the research plan.
placebo effect Refers to the discovery in medical research that any "medication" could
make participants feel better, even sugar and water.
population The group to which the researcher would like the results of a study to be
generalizable.
positively skewed distribution A distribution in which there are more extreme scores at
the upper, or higher, end than at the lower end.
posttest-only control group design A true experimental design involving at least two
randomly formed groups; one group receives a new, or unusual, treatment and both
groups are post tested.
power The ability of a significance test to avoid making a Type II error.
prediction study An attempt to determine which of a number of variables are most highly
related to a criterion variable, a complex variable to be predicted.
predictive validity The degree to which a test is able to predict how well an individual
will do in a future situation.
predictor In a prediction study, the variable upon which the prediction is based.
pretest-postest control group design A true experimental design which involves at least
two randomly formed groups; both groups are pretested, one group receives a new, or
unusual treatment, and both groups are post tested.
pretest sensitization See testing.
pretest-treatment interaction Refers to the fact that participants may respond or react
differently to a treatment because they have been pretested.
primary source Firsthand information such as the testimony of an eyewitness, an
original document, a relic, or a description of a study written by the person who
conducted it.
probability sampling The process of selecting a sample using a sampling technique
which permits the researcher to specify the probability, or chance, that each member of a
defined population has of being selected for the sample.
problem statement A statement which indicates the variables of interest to the
researcher and the specific relationship behveen those variables which is to be, or was,
investigated.
prospective causal-comparative research A variation of the basic approach to
causalcomparative research which involves starting with the causes and investigating
effects.
purposive sampling See judgment sampling.
qualitative approach The collection of extensive narrative data in order to gain insights
into phenomena of interest.
qualitative research The collection of extensive narrative data on many variables over
an extended period of time, in a naturalistic setting, in order to gain insights not possible
using other types of research.

quantitative research The collection of numerical data in order to explain, predict and/
or control phenomena of interest.
quartile deviation One-half of the difference between the upper quartile (the 75th
percentile) and the lower quartile (the 25th percentile) in a distribution.
quota sampling The process of selecting a sample based on required, exact numbers, or
quotas, of persons of varying characteristics.
random sampling The process of selecting a sample in such a way that all individuals in
the defined population have an equal and independent chance of being selected for the
sample.
range The difference between the highest and lowest score in a distribution.
rationale equivalence reliability An estimate of internal consistency based on a
determination of how all items on a test relate to all other items and to the total test.
ratio scale The highest level of measurement that classifies participants, ranks
participants, is based upon predetermined equal intervals, and has a true zero point.
reactive arrangements Threats to the external validity of a study associated with the
way in which a study is conducted and the feelings and attitudes of the participants
involved.
readiness test A test administered prior to instruction or training in a specific area in
order to determine whether and to what degree a student is ready for, or will profit from,
instruction.
relationship study An attempt to gain insight into the variables, or factors, that are
related to a complex variable such as academic achievement, motivation, and self-
concept.
reliability The degree to which a test consistently measures whatever it measures.
replication Refers to when a study is done again; the second study may be a repetition of
the original study, using different participants, or it may represent an alternative approach
to testing the same hypothesis.
research The formal, systematic application of the scientific and disciplined inquiry
approach to the study of problems.
research hypothesis A statement of the expected relationship (or difference) between
two variables.
research plan A detailed description of a proposed study designed to investigate a given
problem.
response set The tendency of an observer to rate the majority of observees the same
regardless of the observees' actual behavior.
retrospective causal-c:omparative research The basic approach to causal-<:omparative
research which involves starting with effects and investigating causes.
review of literature The systematic identification, location, and analysis of documents
containing information related to a research problem.
sample A number of individuals selected from a population for a study, preferably in
such a way that they represent the larger group from which they were selected.
sample survey Research in which information about a population is inferred based on the
responses of a sample selected from that population.
sampling The process of selecting a number of individuals (a sample) from a population,
preferably in such a way that the individuals selected represent the larger group from
which they were selected.

sampling bias Systematic sampling error; two major sources of sampling bias are the use
of volunteers and the use of available groups.
sampling error Expected, chance variation in variables that occurs when a sample is
selected from a population.
sampling validity The degree to which a test samples the total intended content area.
Scheffe test A conservative multiple comparison technique appropriate for making any
and all possible comparisons involving a set of means.
secondary source Secondhand information, such as a brief description of a study written
by someone other than the person who conducted it.
selection-maturation interaction Refers to the fact that if already-formed groups are
used in a study, one group may profit more (or less) from treatment or have an initial
advantage (or disadvantage) because of maturation factors; selection may also interact
with factors such as history and testing.
selection-treatment interaction Refers to the fact that if nonrepresentative groups are
used in a study the results of the study may hold only for the groups involved and maynot
be representative of the treatment effect in the population.
self-report research Descriptive research in which information is solicited from
individuals using, for example, questionnaires or interviews.
semantic differential scale An instrument that asks an individual to give a quantitative
rating to the participant of the attitude scale on a number of bipolar adjectives such as
goodbad, friendly-unfriendly, positive-negative. shrinkage Refers to the tendency of a
prediction equation to become less accurate when used with a different group, a group
other than the one on which the equation was originally formulated.
simple analysis of variance (ANOVA) A parametric test of significance used to
determine whether there is significant difference between or among two or more means at
a selected probability level.
simulation observation Observation in which the researcher creates the situation to be
observed and tells the participant what activities they are to engage in.
simultaneous replication Refers to when replication is done on a number of participants
with the same problem, at the same location, at the same time.
single-subject experimental designs Designs applied when the sample size is one; used
to study the behavior change which an individual exhibits as a result of some
intervention, or treatment.
single-variable designs A class of experimental designs involving only one independent
variable (which is manipulated).
single variable rule An important principle of single-subject research which states that
only one variable should be manipulated at a time.
skewed distribution A nonsymmetrical distribution in which there are more extreme
scores at one end of the distribution than the other.
sociometric study A study that assesses and analyzes the interpersonal relationships
within a group of individuals.
software Refers to the programs which give instructions to the computer concerning
desired operations
Solomon four-group design A true experimental design that involves random
assignment of participants to one of four groups; two groups are pretested, two are not;

one of the pretested groups and one of the unpretested groups receive the experimental
treatment, and all four groups are post tested.
Spearman rho A measure of correlation appropriate when the data for at least one of the
variables are expressed as ranks; it produces a coefficient between .00 and :!:1.00.
specificity of variables Refers to the fact that a given study is conducted with a specific
kind of participant, using specific measuring instruments, at a specific time, under a
specific set of circumstances, factors that affect the generalizability of the results.
split-half reliability A type of reliability that is based on the internal consistency of a test
and is estimated by dividing a test into two equivalent halves and correlating the scores
on the two halves.
standard deviation The most stable measure of variability which takes into account each
and every score in a distribution.
standard error of the mean The standard deviation of sample means which indicates by
how much the sample means can be expected to differ if other samples from the same
population are used.
standard error of measurement An estimate of how often one can expect errors of a
given size in an individual's test score.
standard score A derived score that expresses how far a given raw score is from some
reference point, typically the mean, in terms of standard deviation units.
stanines Standard scores that divide a distribution into nine parts.
static group comparison A pre-experimental design that involves at least two
nonrandomly formed groups; one receives a new, or unusual, treatment and both are
posttested.
statistic A numerical index describing the behavior of a sample or samples.
statistical regression The tendency of participants who score highest on a pretest to
score lower on a post test, and of participants who score lowest on a pretest to score
higher on a post test.
statistical significance The conclusion that results are unlikely to have occurred by
chance; the observed relationship or difference is probably a real one.
stratified sampling The process of selecting a sample in such a way that identified
subgroups in the population are represented in the sample in the same proportion that
they exist in the population or in equal proportion.
structured interview Interview questions that provide options for participants to select
from.
structured item A question and a list of alternative responses from which the responder
selects; also referred to as a closed-form item.
survey An attempt to collect data from members of a population in order to determine the
current status of that population 'ith respect to one or more variables.
systematic replication Refers to replication which follows direct replication, and which
involves different investigators, behaviors, or settings.
systematic sampling Sampling in which individuals are selected from a list by taking
every Kth name, where K equals the number of individuals on the list divided by the
number of participants desired for the sample.
T score A standard score derived from a z score by multiplying the z score by 10 and
adding 50.

t test for independent samples A parametric test of significance used to determine
whether there is a significant difference between the means of two independent samples
at a selected probability level.
t test for nonindependent samples A parametric test of significance used to determine
whether there is a significant difference between the means of two matched, or
nonindependent, samples at a selected probability level.
target population Refers to the population to which the researcher would ideally like to
generalize results.
terminal A device for communicating with a computer which consists of a display screen
and a keyboard
test A means of measuring the knowledge, skill, feelings, intelligence, or aptitude of an
individual or group.
testing A threat to experimental validity which refers to improved scores on a post test
which are a result of participants having taken a pretest. (Also referred to as pretest
sensitization.) test objectivity Refers to a situation in which an individual's score is the
same, or essentially the same, regardless of who is doing the scoring.
test of significance A statistical test used to determine whether or not there is a
significant difference between or among two or more means at a selected probability
level.
test-retest reliability The degree to which scores on a test are consistent, or stable, over
time.
time-series design A quasi-experimental design involving one group which is repeatedly
pretested, exposed to an experimental treatment, and repeatedly post tested.
triangulation The use of multiple methods, data collection strategies, and/ or data
sources, in order to get a more complete picture and to cross-check information.
true categories Categories into which persons or objects naturally fall, independently of
the research study.
Type I error The rejection by the researcher of a null hypothesis which is actually true.
Type II error The failure of a researcher to reject a null hypothesis which is really false.
unobtrusive measures Inanimate objects (such as school suspension lists) which can be
observed in order to obtain desired information.
munstructured ite A question giving the responder complete freedom of response.
validity The degree to which a test measures what it is intended to measure; a test is valid
for a particular purpose for a particular group.
variable A concept that can assume anyone of a range of values, e.g., intelligence,
height, aptitude.
z score The most basic standard score that expresses how far a score is from a mean m
terms of standard deviation.
Z score See T score.

Link: http://www.artsci-ccwin.Concordia.ca/edtech/et664.html
http://www.ed.gov/offices/oeri/nerppb/
http://www.oeri/research.html
http://www./resCtr.html
http://www.crede.ucsc.edu/

Research Design and Methodology, Dr. W.A. Kritsonis

More Related Content

What's hot (20)

Similar to Research Design and Methodology, Dr. W.A. Kritsonis (20)

More from guestcc1ebaf (20)

Recently uploaded (20)

Research Design and Methodology, Dr. W.A. Kritsonis