General Framework for Setting Examination Papers and Test Papers
- 1. GENERAL FRAMEWORK FOR SETTING EXAMINATION PAPERS: TEST SPECIFICATIONS PAPER PRESENTED BY W. M. Kapambwe Principal - TEVET Examinations Council of Zambia
- 2. INTRODUCTION You are a teacher, one day a colleague phones you and she tells you that she is ill. According to the programme she has to develop a test. Due to the illness he/she is not able to develop that test. Of course you are willing to help your colleague. What kind of information do you need to be able to develop the test?
- 3. According to The Standards for Educational and Psychological Testing : “… the validity of an intended interpretation of test scores relies on all the available evidence relevant to the technical quality of a testing system. This includes evidence of careful test construction…” Content Definition and Mapping What content is being measured? How is the measurement of the content domain(s) being operationalised?
- 4. Test Specifications Attainment targets Assessment objectives (Behaviour or content) Taxonomy Test format Test plan Test matrix or test grid
- 5. Assessment Options for Different Learning Domains All human behavior can be divided into one of three categories or domains: Cognitive Affective Psychomotor
- 6. The three categories promote different learning behaviors or objectives. The most commonly assessed behavior domain is the cognitive. Behaviours in the cognitive have been organized into general categories. Such an organization is called taxonomy (system of classification). Owing to the different learning behaviours, different assessment techniques are utilized.
- 7. Bloom’s Taxonomy The Blooms taxonomy is the main taxonomy used Test Matrix or Grid Total Evaluation Synthesis Application Comp Know Behaviours
- 8. 1. Knowledge – define, recognize 2. Comprehension – explain, compare 3. Application – Calculate, apply 4. Analysis – Analyze, Plan 5. Synthesize – Design, schematize 6. Evaluation – Criticize, Choose, judge Taxonomy: Bloom
- 9. Bloom Taxonomy Verbs for the cognitive domain Appraise Conclude Critique Critcize Grade Judge Justify Interpret Support Recommend Arrange Combine Construct Create Design Formulate Genera- lise Generate Group Integrate Organise summarise Breakdown Differentiate Investigate Relate Separate Subdivide Change Compute Construct Solve Demonst-rate Illustrate Predict Classify Compare Convert Discuss Distinguish Estimate Explain Generalise Give Examples Infer Interpret Para Rewrite Summarize translate Count Define Identify Label list Match Name Outline Point out Quote Recite Repeat Reproduce Select state trace Behaviours Evaluation Synthesis Analysis App. Comp. Know. Cog. Area
- 10. Old & New Bloom’s Taxonomy The six levels of Blooms taxonomy were changed from nouns to verbs as shown below.
- 11. The New Bloom’s Taxonomy incorporates not only the cognitive domain but also the affective and the psychomotor. This represents a holistic treatment of all the learning domains. The New Blooms Taxonomy has the following domains : The cognitive - knowledge based domain, consisting of six levels The affective - attitudinal based domain, consisting of five levels, and The psychomotor - skills based domain, consisting of six levels.
- 12. New Terms Explained Remembering : Retrieving, recognizing, and recalling relevant knowledge from long-term memory. Understanding : Constructing meaning from oral, written, and graphic messages through interpreting, exemplifying, classifying, summarizing, inferring, comparing, and explaining. Applying : Carrying out or using a procedure through executing, or implementing
- 13. Analyzing : Breaking material into constituent parts, determining how the parts relate to each other and to an overall structure or purpose through differentiating, organizing, and attributing. Evaluating : Making judgments based on criteria and standards through checking and critiquing. Creating : Putting elements together to form a coherent or functional whole; reorganizing elements into a new pattern or structure through generating, planning or producing.
- 14. Assessment Options for the Affective Domain Difficult to assess due to the personal and internal qualities of affect. Inquiry/problem-solving commonly used. Receiving: observations of learners’discussions and questionnaires. Responding: observation of learners’ participation and interviews. Valuing: Interviews, questionaiires and essays. Organisation: Observation of learners’choices. Characterisation: Learners’ responsibilities,projects and debates.
- 15. Assessment Options for the Psychomotor Hierarchy of difficulty levels ranging from reflex movements to skilled movements. Levels generally assessed by observation of either behaviour or performance. Observational data can be recorded as an anecdotal record, or by using ckecklists or rating scales.
- 16. Romiszowski Taxonomy Knowledge Facts Concepts Skills Reproduction Production Cognitive (RC) Interactive (Ri) Psychomotor (Rpm) Reactive (Rr) Cognitive (PC) Interactive (Pi) Psychomotor (Ppm) Reactive (Pr)
- 17. Romiszowski : Knowledge and Skills (a) Knowledge: Factual knowledge : recognizing and remembering of facts and action prescription Conceptual knowledge : understanding principles of strategies actions (b) Skills Reproductive skills : Routines and vocational activities based on standard procedures executed according to a clear prescription or protocol. Productive skills : Appeals to the creativity and problem solving skills of the student. Application of principles and strategies in new situations.
- 18. (c ) Sub-skills for Reproductive and Productive Cognitive skills: Application of knowledge, eg. Interpretation, analysis, deciding. Psychomotor skills: Execution of physical, motoric actions using knowledge and comprehensive. Reactive skills: Attitudes according to a value system. Interactive skills: Application of social and communicative aspects in daily life in relation to others, communicating, collaborating.
- 19. Romiszowki Taxonomy: Verbs Model Design Plan Analyze Argue Discuss Demonstrate Construct Apply (in concrete situations) Explain Interpret Classify Compare Put in order Combine Distinguish Describe Give examples Recognize Repeat List Name Define Point out Underline Productive skill Reproductive skills Conceptual knowledge Factual Knowledge
- 20. Romiscowski Test Matrix Skills Skills Knowledge Knowledge Content (Summarized/End Term) Test Goal/Sub- domain Status: Draft Final Test Format Test Number: Period of Testing: Author: Year/Class School Type Subject
- 21. Conclusion Test construction requires knowledge of the syllabus/course outline on which an examination syllabus is based. There should be congruency or agreement between the learning outcome or objective and the testing technique used to assess a learning outcome.
- 22. The taxonomy used should be determined by the nature of learning outcome. Not all learning outcomes can be assessed by pencil and paper and within a short time. Most learning outcomes in the affective and psychomotor can be observed by observations and recording over a long time. Performance tasks are also assessed by observation. A mismatch between learning outcomes and assessment procedures leads to what is called curriculum distortion. This distortion could lead to lower learning achievement due to wrong enforcement from the assessment process.
- 23. Bibliography American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999). Standards for Educational and Psychological Testing . Washington, DC: American Educational Research Association Anderson, L. W. and David R. Krathwohl, D. R., et al (2000) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Allyn & Bacon
- 24. Bloom, B.S. and Krathwohl, D. R. (1956) Taxonomy of Educational Objectives: The Classification of Educational Goals, by a committee of college and university Examiners. Handbook I: Cognitive Domain. NY, NY: Longmans, Green. Barnes, J. (1976) 'Introduction' to Aristotle The Nicomachean Ethics ('Ethics'), Harmondsworth: Penguin.
- 25. Barrow, R. (1984) Giving Teaching back to Teachers. A critical introduction to curriculum theory, Brighton: Wheatsheaf Books. MoE, 2003: The Zambia Basic Education Syllabi. Kelly, A.V. (1999) The Curriculum – Theory and Practice, Paul Chapman Publishing Limited, London.
- 26. Cito International(2009) Theory and Practice of Test Construction, Netherlands Fellowship Programme, Arnheim Kapambwe, W.M. (2006) The Formative Evaluation of the Continuous Assessment Pilot Programme at Basic School Level in Zambia, Unpublished M.Ed Thesis, University College Dublin
- 27. Pickard M.J. New Bloom’s Taxonomy: An Overview for Family and Consumer Sciences. Journal of Family and Consumer Sciences Education, Vol.25, No.1, Spring/Summer 2007. Stenhouse, L. (1975) An Introduction to Curriculum Research and Development, Heinemann Educational Books, London.