Improving test items

I. Choosing Between Objective and Subjective Test Items
II. Suggestions For Using and Writing Test Items
2/12/2019 @reinafranco13 1

Test Item Quiz (circle the correct answer)
1. Essay exams are easier to construct than are objective exams. T F ?
2.
Essay exams require more thorough student preparation and study time
than objective exams.
T F ?
3. Essay exams require writing skills where objective exams do not. T F ?
4. Essay exams teach a person how to write. T F ?
5. Essay exams are more subjective in nature than are objective exams. T F ?
6. Objective exams encourage guessing more so than essay exams. T F ?
7. Essay exams limit the extent of content covered. T F ?
8.
Essay and objective exams can be used to measure the same content or
ability.
T F ?
9.
Essay and objective exams are both good ways to evaluate a student's
level of knowledge.
T F ?

Choosing
Between
Objective and
Subjective Test
Items

Subjective Test
This kind of tests are especially appropriate when:
•the group to be tested is small and the test is not to be
reused.
•you wish to encourage and reward the development of
student skill in writing.
•you are more interested in exploring the student's attitudes
than in measuring his/her achievement.
•you are more confident of your ability as a critical and fair
reader than as an imaginative writer of good objective test
items.

Objective tests
Objective tests are especially appropriate when:
• the group to be tested is large and the test may be reused.
• highly reliable test scores must be obtained as efficiently as
possible.
• impartiality of evaluation, absolute fairness, and freedom from
possible test scoring influences (e.g., fatigue, lack of anonymity)
are essential.
• you are more confident of your ability to express objective test
items clearly than of your ability to judge essay test answers
correctly.
• there is more pressure for speedy reporting of scores than for
speedy test preparation.2/12/2019 @reinafranco13 5

Either essay or objective tests can be used to:
•measure almost any important educational
achievement a written test can measure.
•test understanding and ability to apply principles.
•test ability to think critically.
•test ability to solve problems.
•test ability to select relevant facts and principles
and to integrate them toward the solution of
complex problems.

Learning Objectives Most Suitable Test Item
The student will be able to categorize and name
the parts of the human skeletal system.
Objective Test Item (M-C, T-F, Matching)
The student will be able to critique and appraise
another student's English composition on the
basis of its organization.
Essay Test Item (Extended-Response)
The student will demonstrate safe laboratory
skills.
Performance Test Item
The student will be able to cite four examples of
satire that Twain uses in Huckleberry Finn.
Essay Test Item (Short-Answer)

Suggestions
For Using
and Writing
Test Items

Multiple-choice test items
The multiple-choice item consists of two
parts:
(a) the stem, which identifies the question or
problem and
(b) the response alternatives. Students are
asked to select the one alternative that
best completes the statement or answers
the question.2/12/2019 @reinafranco13 9

Sample Multiple-Choice Item
(a) Item Stem: Which of the following is a chemical change?
(b) Response Alternatives: a.Evaporation of alcohol
b.Freezing of water
*c.Burning of oil
d.Melting of wax
*correct response

•Advantages In Using Multiple-Choice Items
Multiple-choice items can provide ...
•versatility in measuring all levels of cognitive ability.
•highly reliable test scores.
•scoring efficiency and accuracy.
•objective measurement of student achievement or ability.
•a wide sampling of content or objectives.
•a reduced guessing factor when compared to true-false
items.
•different response alternatives which can provide
diagnostic feedback.

Limitations In Using Multiple-Choice Items
Multiple-choice items ...
•are difficult and time consuming to construct.
•lead an instructor to favor simple recall of facts.
•place a high degree of dependence on the
student's reading ability and instructor's writing
ability.

Suggestions For
Writing Multiple-
Choice Test Items

1.When possible, state the stem as a direct question rather than as an
incomplete statement.
2.Present a definite, explicit and singular question or problem in the
stem.
3.Eliminate excessive verbiage or irrelevant information from the stem.
4.Include in the stem any word(s) that might otherwise be repeated in
each alternative.
5.Use negatively stated stems sparingly. When used, underline and/or
capitalize the negative word.
6.Make all alternatives plausible and attractive to the less knowledgeable
or skillful student.
7.Make the alternatives grammatically parallel with each other, and
consistent with the stem.
8.Make the alternatives mutually exclusive.2/12/2019 @reinafranco13 14

9. When possible, present alternatives in some logical order (e.g.,
chronological, most to least, alphabetical).
10.Be sure there is only one correct or best response to the item.
11.Make alternatives approximately equal in length.
12.Avoid irrelevant clues such as grammatical structure, well known verbal
associations or connections between stem and answer.
13.Use at least four alternatives for each item to lower the probability of
getting the item correct by guessing.
14.Randomly distribute the correct response among the alternative positions
throughout the test having approximately the same proportion of
alternatives a, b, c, d and e as the correct response.
15.Use the alternatives "none of the above" and "all of the above" sparingly.
When used, such alternatives should occasionally be used as the correct
response.2/12/2019 @reinafranco13 15

True-False Test Items
A true-false item can be written in one of three
forms: simple, complex, or compound. Answers
can consist of only two choices (simple), more
than two choices (complex), or two choices plus
a conditional completion response (compound).

Sample True-False Item: Simple
The acquisition of morality is a developmental process. True False
Sample True-False Item: Complex
The acquisition of morality is a developmental
process.
TrueFalse Opinion
Sample True-False Item: Compound
The acquisition of morality is a developmental process.
If this statement is false, what makes it false? True False

Advantages In Using True-False Items
True-False items can provide ...
 the widest sampling of content or objectives per unit of
testing time.
 scoring efficiency and accuracy.
 versatility in measuring all levels of cognitive ability.
 highly reliable test scores.
 an objective measurement of student achievement or
ability.2/12/2019 @reinafranco13 18

Limitations In Using True-False Items
True-false items ...
 incorporate an extremely high guessing factor. For simple true-false items,
each student has a 50/50 chance of correctly answering the item without any
knowledge of the item's content.
 can often lead an instructor to write ambiguous statements due to the
difficulty of writing statements which are unequivocally true or false.
 do not discriminate between students of varying ability as well as other item
types.
 can often include more irrelevant clues than do other item types.
 can often lead an instructor to favor testing of trivial knowledge.

Suggestions For
Writing True-
False Test Items

1.Base true-false items upon statements that are absolutely true or false,
without qualifications or exceptions.
2.Express the item statement as simply and as clearly as possible.
3.Express a single idea in each test item.
4.Include enough background information and qualifications so that the
ability to respond correctly to the item does not depend on some
special, uncommon knowledge.
5.Avoid lifting statements from the text, lecture or other materials so
that memory alone will not permit a correct answer.
6.Avoid using negatively stated item statements.
7.Avoid the use of unfamiliar vocabulary.

8. Avoid the use of specific determiners which would permit a test-wise
but unprepared examinee to respond correctly. Specific determiners
refer to sweeping terms like "all," "always," "none," "never,"
"impossible," "inevitable," etc. Statements including such terms are
likely to be false. On the other hand, statements using qualifying
determiners such as "usually," "sometimes," "often," etc., are likely
to be true. When statements do require the use of specific
determiners, make sure they appear in both true and false items.
9. False items tend to discriminate more highly than true items.
Therefore, use more false items than true items (but no more than
15% additional false items).

Matching Test Items
In general, matching items consist of a
column of stimuli presented on the left side
of the exam page and a column of
responses placed on the right side of the
page. Students are required to match the
response associated with a given stimulus.

Directions:
On the line to the left of each factual statement, write the letter of the principle which bests explains the
statement's occurrence. Each principle may be used more than once.

Advantages In Using Matching Items
- require short periods of reading and response time, allowing you to
cover more content.
 provide objective measurement of student achievement or ability.
 provide highly reliable test scores.
 provide scoring efficiency and accuracy.
Limitations In Using Matching Items
• have difficulty measuring learning objectives requiring more than
simple recall of information.
• are difficult to construct due to the problem of selecting a common
set of stimuli and responses.2/12/2019 @reinafranco13 25

Suggestions
For Writing
Matching Test
Items2/12/2019 @reinafranco13 26

1.Explain whether or not a response can be used more than once and
indicate where to write the answer.
2.Use only homogeneous material in matching items.
3.Arrange the list of responses in some systematic order if possible (e.g.,
chronological, alphabetical).
4.Avoid grammatical or other clues to the correct response.
5.Keep matching items brief, limiting the list of stimuli to under 10.
6.Include more responses than stimuli to help prevent answering through
the process of elimination.
7.When possible, reduce the amount of reading time by including only
short phrases or single words in the response list.

Completion Test Items
The completion item requires the student to answer a question or to
finish an incomplete statement by filling in a blank with the correct word
or phrase. For example,
Sample Completion Item
According to Freud, personality is made up of three major systems, the
_________, the ________ and the ________.

Advantages In Using Completion Items
 can provide a wide sampling of content.
 can efficiently measure lower levels of cognitive ability.
 can minimize guessing as compared to multiple-choice
or true-false items.
 can usually provide an objective measure of student
achievement or ability.

Limitations In Using Completion Items
• are difficult to construct so that the desired response is clearly
indicated.
• have difficulty measuring learning objectives requiring more than
simple recall of information.
• can often include more irrelevant clues than do other item types.
• are more time consuming to score when compared to multiple-
choice or true-false items.
• are more difficult to score since more than one answer may have to
be considered correct if the item was not properly prepared.

Suggestions For
Writing Completion
Test Items

1.Omit only significant words from the statement.
2.Do not omit so many words from the statement that the
intended meaning is lost.
3.Avoid grammatical or other clues to the correct response
4.Be sure there is only one correct response.
5.Make the blanks of equal length.
6.When possible, delete words at the end of the statement after
the student has been presented a clearly defined problem.
7.Avoid lifting statements directly from the text, lecture or other
sources.
8.Limit the required response to a single word or phrase.2/12/2019 @reinafranco13 32

Essay Test Items
The essay test is probably the most popular of all types of teacher-made
tests. In general, a classroom essay test consists of a small number of
questions to which the student is expected to demonstrate his/her ability
to
(a) recall factual knowledge,
(b)organize this knowledge and
(c) present the knowledge in a logical, integrated answer to the
question. An essay test item can be classified as either an extended-
response essay item or a short-answer essay item. The latter calls for
a more restricted or limited answer in terms of form or scope.

Sample Extended-Response Essay Item
Explain the difference between the S-R (Stimulus-Response) and
the S-O-R (Stimulus-Organism-Response) theories of
personality. Include in your answer (a) brief descriptions of
both theories, (b) supporters of both theories and (c) research
methods used to study each of the two theories. (10 pts. 20
minutes)
Sample Short-Answer Essay Item
Identify research methods used to study the S-R (Stimulus-
Response) and S-O-R (Stimulus-Organism-Response) theories of
personality. (5 pts. 10 minutes)2/12/2019 @reinafranco13 34

Advantages In Using Essay Item
 are easier and less time consuming to construct
than are most other item types.
 provide a means for testing student's ability to
compose an answer and present it in a logical
manner.
 can efficiently measure higher order cognitive
objectives (e.g., analysis, synthesis, evaluation).2/12/2019 @reinafranco13 35

Limitations In Using Essay Items
 cannot measure a large amount of content or objectives.
 generally provide low test and test scorer reliability.
 require an extensive amount of instructor's time to read
and grade.
 generally do not provide an objective measure of
student achievement or ability (subject to bias on the
part of the grader).

Suggestions For
Writing Essay Test
Items

1.Prepare essay items that elicit the type of behavior you want to
measure.
2.Phrase each item so that the student's task is clearly indicated.
3.Indicate for each item a point value or weight and an estimated time
limit for answering.
4.Ask questions that will elicit responses on which experts could agree
that one answer is better than another.
5.Avoid giving the student a choice among optional items as this greatly
reduces the reliability of the test.
6.It is generally recommended for classroom examinations to administer
several short-answer items rather than only one or two extended-
response items.2/12/2019 @reinafranco13 38

Suggestions For
Scoring Essay
Items2/12/2019 @reinafranco13 39

1.Choose a scoring model. Two of the more common scoring
models are ANALYTICAL SCORING and GLOBAL QUALITY.
 ANALYTICAL SCORING: Each answer is compared to an
ideal answer and points are assigned for the inclusion of
necessary elements. Grades are based on the number of
accumulated points either absolutely (i.e., A=10 or more
points, B=6-9 pts., etc.) or relatively (A=top 15% scores,
B=next 30% of scores, etc.)
 GLOBAL QUALITY: Each answer is read and assigned a
score (e.g., grade, total points) based either on the total quality
of the response or on the total quality of the response relative
to other student answers.2/12/2019 @reinafranco13 40

Examples Essay Item and Grading Models
"Americans are a mixed-up people with no sense of ethical
values. Everyone knows that baseball is far less necessary than
food and steel, yet they pay ball players a lot more than farmers
and steelworkers."
WHY? Use 3-4 sentences to indicate how an economist would
explain the above situation.

Necessary Elements to be Included in Response Points
Salaries are based on demand relative to supply of such
services.
3
Excellent ball players are rare. 2
Ball clubs have a high demand for excellent players. 2
Clarity of Response 2
____
ANALYTICAL SCORING

Global Quality
Assign scores or grades on the overall quality of the written
response as compared to an ideal answer. Or, compare the
overall quality of a response to other student responses by
sorting the papers into three stacks:
Below Average I Average I Above Average

2.Try not to allow factors which are irrelevant to the learning outcomes
being measured affect your grading (i.e., handwriting, spelling,
neatness).
3.Read and grade all class answers to one item before going on to the
next item.
4.Read and grade the answers without looking at the students' names to
avoid possible preferential treatment.
5.Occasionally shuffle papers during the reading of answers to help
avoid any systematic order effects (i.e., Sally's "B" work always
followed Jim's "A: work thus it looked more like "C" work).
6.When possible, ask another instructor to read and grade your
students' responses.2/12/2019 @reinafranco13 44

Problem Solving Test Items
 Another form of a subjective test item is the problem
solving or computational exam question. Such items
present the student with a problem situation or task and
require a demonstration of work procedures and a correct
solution, or just a correct solution.
 This kind of test item is classified as a subjective type of
item due to the procedures used to score item responses.
Instructors can assign full or partial credit to either correct
or incorrect solutions depending on the quality and kind of
work procedures presented2/12/2019 @reinafranco13 45

Example Problem Solving Test Item
It was calculated that 75 men could complete a
strip on a new highway in 70 days. When work was
scheduled to commence, it was found necessary to
send 25 men on another road project. How many
days longer will it take to complete the strip? Show
your work for full or partial credit.

Advantages In Using Problem Solving Items
o minimize guessing by requiring the students to provide an
original response rather than to select from several
alternatives.
o are easier to construct than are multiple-choice or matching
items.
o can most appropriately measure learning objectives which
focus on the ability to apply skills or knowledge in the
solution of problems.
o can measure an extensive amount of content or objectives.2/12/2019 @reinafranco13 47

Limitations In Using Problem Solving Items
o generally provide low test and test scorer reliability.
o require an extensive amount of instructor time to
read and grade.
o generally do not provide an objective measure of
student achievement or ability (subject to bias on
the part of the grader when partial credit is given).

1.Clearly identify and explain the problem.
2.Provide directions which clearly inform the student of the type of
response called for.
3.State in the directions whether or not the student must show his/her
work procedures for full or partial credit.
4.Clearly separate item parts and indicate their point values.
5.Use figures, conditions and situations which create a realistic
problem.
6.Ask questions that elicit responses on which experts could agree that
one solution and one or more work procedures are better than others.
7.Work through each problem before classroom administration to
double-check accuracy.2/12/2019 @reinafranco13 49

Performance Test Items
 A performance test item is designed to assess the ability of a student
to perform correctly in a simulated situation (i.e., a situation in
which the student will be ultimately expected to apply his/her
learning). The concept of simulation is central in performance
testing; a performance test will simulate to some degree a real life
situation to accomplish the assessment.
 In theory, a performance test could be constructed for any skill and
real life situation. In practice, most performance tests have been
developed for the assessment of vocational, managerial,
administrative, leadership, communication, interpersonal and
physical education skills in various simulated situations.

Advantages In Using Performance Test Items
o can most appropriately measure learning objectives
which focus on the ability of the students to apply
skills or knowledge in real life situations.
o usually provide a degree of test validity not possible
with standard paper and pencil test items.
o are useful for measuring learning objectives in the
psychomotor domain.

Limitations In Using Performance Test Items
o are difficulty and time consuming to construct.
o are primarily used for testing students individually and not
for testing groups. Consequently, they are relatively costly,
time consuming, and inconvenient forms of testing.
o generally provide low test and test scorer reliability.
o generally do not provide an objective measure of student
achievement or ability (subject to bias on the part of the
observer/grader).

Suggestions For Writing Performance Test Items
 Prepare items that elicit the type of behavior you want to measure.
 Clearly identify and explain the simulated situation to the student.
 Make the simulated situation as "life-like" as possible.
 Provide directions which clearly inform the students of the type of
response called for.
 When appropriate, clearly state time and activity limitations in the
directions.
 Adequately train the observer(s)/scorer(s) to ensure that they are fair
in scoring the appropriate behaviors.2/12/2019 @reinafranco13 53

REFERENCES FOR FURTHER READING
• Ebel, Robert L. Measuring educational achievement. Englewood Cliffs, New Jersey: Prentice-Hall, 1965, Chapters
4-6.
• Ebel, Robert L. Essentials of educational measurement. Englewood Cliffs, New Jersey: Prentice-Hall, 1972,
Chapters 5-8.
• Gronlund, N. E. Measurement and evaluation in teaching. New York: Macmillan Publishing Co., 1976, Chapters 6-
9.
• Mehrens, W. A. & Lehmann, I. J. Measurement and evaluation in education and psychology. New York: Holt,
Rinehart & Winston, Inc., 1973, Chapters 7-10.
• Nelson, C. H. Measurement and evaluation in the classroom. New York: Macmillan Publishing Co., 1970, Chapters
5-8. Measurement and Evaluation Division, 247 Armory Building. Especially useful for science instruction.
• Payne, David A. The assessment of learning. Lexington, Mass.: D.C. Heath and Co., 1974, Chapters 4-7.
• Scannell, D. P. & Tracy, D. B. Testing and measurement in the classroom. New York: Houghton-Mifflin Co., 1975,
Chapters 4-6.
• Thorndike, R. L. (Ed.). Educational measurement (2nd ed.). Washington, D.C.: American Council on Education,
1971, Chapter 9 (Performance testing) and Chapter 10 (Essay exams).

THANK YOU
FOR
LISTENING!

Essay items are generally easier and less time consuming to construct
than are most objective test items. Technically correct and content
appropriate multiple-choice and true-false test items require an
extensive amount of time to write and revise. For example, a
professional item writer produces only 9-10 good multiple-choice
items in a day's time.
TRU
E

According to research findings it is still
undetermined whether or not essay tests require
or facilitate more thorough (or even different)
student study preparation.
Undetermi
ned

Writing skills do affect a student's ability to
communicate the correct "factual" information
through an essay response. Consequently,
students with good writing skills have an
advantage over students who have difficulty
expressing themselves through writing
TRUE

Essays do not teach a student how to write but
they can emphasize the importance of being able
to communicate through writing. constant use of
essay tests may encourage the knowledgeable but
poor writing student to improve his/her writing
ability in order to improve performance.
FALSE

Essays are more subjective in nature due to their susceptibility to
scoring influences. Different readers can rate identical responses
differently, the same reader can rate the same paper differently over
time, the handwriting, neatness or punctuation can unintentionally
affect a paper's grade and the lack of anonymity can affect the
grading process. While impossible to eliminate, scoring influences or
biases can be minimized through procedures discussed later in this
booklet.
TRUE

Both item types encourage some form of guessing.
Multiple-choice, true-false and matching items can be
correctly answered through blind guessing, yet essay
items can be responded to satisfactorily through well
written bluffing.
Undetermi
ned

Due to the extent of time required by the student to
respond to an essay question, only a few essay
questions can be included on a classroom exam.
Consequently, a larger number of objective items can
be tested in the same amount of time, thus enabling
the test to cover more content.
TRUE

Both item types can measure similar content
or learning objectives. Research has shown
that students respond almost identically to
essay and objective test items covering the
same content.
TRUE

Both objective and essay test items are good
devices for measuring student achievement.
However, as seen in the previous quiz answers,
there are particular measurement situations
where one item type is more appropriate than the
other.
TRUE

Improving test items

More Related Content

What's hot (20)

Similar to Improving test items (20)

More from Reina Antonette Franco (17)

Recently uploaded (20)

Improving test items

Editor's Notes