RBT and DOK Alignment for Practitioners

Practical Alignment for Item / Test Developers: Revised Bloom’s Taxonomy (RBT) and Depth of Knowledge Levels (DOK) Staff Trainings, 2009 Jerrie W. Brown, Content Lead for Social Studies Assessments North Carolina State University Center for Urban & Community Services Technical Outreach for Public Schools (TOPS)

“ Lack of excellence in American schools is not caused by ineffective teaching, but mostly by misaligning what teachers teach, what they intend to teach, and what they assess as having been taught.” -Prof. S. Alan Cohen, 1987 University of San Francisco “ Alignment also helps ensure the validity of test results. If standards and tests are not aligned, results might provide a misleading picture of how well students are attaining standards.” -Robert Rothman, 2003 Annenberg Institute for School Reform

What is alignment? The No Child Left Behind Act of 2001 (NCLB) requires that each state align assessments to content standards. Once the enacted (instruction), intended (state standards), and assessed (state tests) curricula have been measured, questions can be asked about the extent to which content is similar across them. To the extent content is the same, they are said to be aligned. J. Brown, NCSU, 2009

Defining Alignment If the content assessed is exactly the same as the content represented in the standards, alignment is perfect. There are two ways in which alignment can be less than perfect: (1) Content in the standards may not be assessed, and (2) content assessed may not be in the standards. J. Brown, NCSU, 2009

Alignment Procedure Review alignment criteria of Revised Bloom’s Taxonomy (RBT) and Depth of Knowledge (DOK) levels Rate standards Rate test items Aggregate individual ratings Determine degree of RBT-based alignment Determine degree of DOK-based alignment J. Brown, NCSU, 2009

Alignment Dimensions DOK – Two Dimensions Depth of Knowledge Levels Content Topics or Strands (Webb, 2002) J. Brown, NCSU, 2009 RBT – Two Dimensions Cognitive Processes Knowledge Types (Anderson, 2001) 6. Create 5. Evaluate 4. Analyze 3. Apply 2. Understand 1. Remember Content Topic/Strand A 4. Extended Thinking 3. Strategic Thinking 2. Skill/Concept 1. Recall A. Factual B. Conceptual C. Procedural D. Meta-Cognitive Content Topic/Strand B Content Topic/Strand C (Note: achieves content validity) (Note: achieves content validity and instructional validity)

Revised Bloom’s Taxonomy (RBT) Table for _____________________________ (Subject / Grade Level) Adapted from Anderson, Lorin W. and David R. Krathwohl, et al., (2001). J. Brown, NCSU, 2009 Knowledge Dimension (Lower Order = 0% ) Cognitive Process Dimension (Higher Order = 0% ) 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge (0%) B. Conceptual Knowledge (0%) C. Procedural Knowledge (0%) D. Meta- Cognitive Knowledge (0%)

Depth of Knowledge (DOK) Levels for _____________________________ (Subject / Grade Level) Adapted from Norm Webb, (2002). J. Brown, NCSU, 2009 Content Categories Depth of Knowledge Levels Categorical Concurrence Yes / No 1. Recall 2. Skill/Concept 3. Strategic Thinking 4. Extended Thinking A. Topic/Strand __________ (Specify) B. Topic/Strand __________ (Specify) C. Topic/Strand __________ (Specify)

RBT’s Knowledge Dimension: J. Brown, NCSU, 2009 Major Types Definitions A Factual The basic elements students must know to be acquainted with a discipline or solve problems in it B Conceptual The interrelationships among the basic elements within a larger structure that enable them to function together C Procedural How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods D Metacognitive Knowledge of cognition in general as well as awareness and knowledge of one’s own cognition

RBT’s Knowledge Dimension: J. Brown, NCSU, 2009 Major Types Subtypes A Factual AA. Knowledge of terminology AB. Knowledge of specific details and elements B Conceptual BA. Knowledge of classifications and categories BB. Knowledge of principles and generalizations BC. Knowledge of theories, models, and structures C Procedural CA. Knowledge of subject-specific skills and algorithms CB. Knowledge of subject-specific techniques and methods CC. Knowledge of criteria for determining when to use appropriate procedures D Metacognitive DA. Strategic knowledge DB. Knowledge about cognitive tasks, including appropriate contextual and conditional knowledge DC. Self-knowledge

RBT’s Cognitive Process Dimension: J. Brown, NCSU, 2009 Cognitive Process Categories Definitions 1 Remember Retrieve relevant knowledge from long-term memory 2 Understand Construct meaning from instructional messages, including oral, written, and graphic communication 3 Apply Carry out or use a procedure in a given situation 4 Analyze Break material down into its constituent parts and determine how the parts relate to one another and to an overall structure or purpose 5 Evaluate Make judgments based on criteria and standards 6 Create Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure

RBT’s Cognitive Process Dimension: J. Brown, NCSU, 2009 Cognitive Process Categories Cognitive Processes 1 Remember 1.1 Recognizing 1.2 Recalling 2 Understand 2.1 Interpreting 2.2 Exemplifying 2.3 Classifying 2.4 Summarizing 2.5 Inferring 2.6 Comparing 2.7 Explaining 3 Apply 3.1 Executing 3.2 Implementing 4 Analyze 4.1 Differentiating 4.2 Organizing 4.3 Attributing 5 Evaluate 5.1 Checking 5.2 Critiquing 6 Create 6.1 Generating 6.2 Planning 6.3 Producing

Norm Webb’s Specific Alignment Criteria Content Focus A. Categorical Concurrence: between goals and assessment is met if the same or consistent categories of content appear in both documents B. Depth-of-Knowledge Consistency: at least 50% of the items corresponding to a goal had to be at or above the level of knowledge of the goal C. Range-of-Knowledge Correspondence: fifty percent of the objectives for a goal must have at least one related assessment item D. Balance of Representation: the degree to which one objective is given more emphasis on the assessment than another Formal Alignment Studies and Depth of Knowledge Levels (DOKs) J. Brown, NCSU, 2009

Depth of Knowledge Levels Norm Webb (2002) Generic DOKs Level 1: Recall Recall of a fact, information, or procedure. Level 2: Skill/Concept Use information or conceptual knowledge, two or more steps, etc. Level 3: Strategic Thinking Requires reasoning, developing plan or a sequence of steps, some complexity, more than one possible answer. Level 4: Extended Thinking Requires an investigation, time to think and process multiple conditions of the problem. J. Brown, NCSU, 2009

Subject-Specific DOKs J. Brown, NCSU, 2009 Science Social Studies Math Level 1 Recall and Reproduction Recall of Information Recall Level 2 Skills and Concepts Basic Reasoning Skill/ Concept Level 3 Strategic Thinking Complex Reasoning Strategic Thinking Level 4 Extended Thinking Extended Reasoning Extended Thinking

Reading DOKs J. Brown, NCSU, 2009 Level 1 : Support ideas by reference to details in the text. Use a dictionary to find the meaning of words. Identify figurative language in a reading passage. Level 2: Use context cues to identify the meaning of unfamiliar words. Predict a logical outcome based on information in a reading selection. Identify and summarize the major events in a narrative. Level 3: Determine the author’s purpose and describe how it affects the interpretation of a reading selection. Summarize information from multiple sources to address a specific topic. Analyze and describe the characteristics of various types of literature. Level 4: Analyze and synthesize information from multiple sources. Examine and explain alternative perspectives across a variety of sources. Describe and illustrate how common themes are found across texts from different cultures.

Getting Started Resources needed: Knowledge types (a-d) Cognitive processes (1-6) Depth of knowledge levels (1-4) Standards ratings forms Test items ratings forms RBT blank taxonomy tables J. Brown, NCSU, 2009

Rating WA State Standards J. Brown, NCSU, 2009

Plotting Standards J. Brown, NCSU, 2009 Knowledge Dimension (Lower Order = 0% ) Cognitive Process Dimension (Higher Order = 0% ) 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge (0%) 10 1, 5 9 B. Conceptual Knowledge (0%) 2, 3 8 6 C. Procedural Knowledge (0%) 7 4 D. Meta- Cognitive Knowledge (0%)

Rating ‘WASL’ Test Items J. Brown, NCSU, 2009

Plotting Test Items J. Brown, NCSU, 2009 Knowledge Dimension (Lower Order = 0% ) Cognitive Process Dimension (Higher Order = 0% ) 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create A. Factual Knowledge (0%) 10 1, 5 9 B. Conceptual Knowledge (0%) 2, 3 8 6 C. Procedural Knowledge (0%) 7 4 D. Meta- Cognitive Knowledge (0%)

Thank You for Participating! FOR MORE INFORMATION : Mr. Jerrie W. Brown, Consultant North Carolina State University Center for Urban and Community Services Technical Outreach for Public Schools 1500 Blue Ridge Road Raleigh, N. Carolina 27607 Tel: (919) 515-1125 Email: [email_address] Website: http://www.cuacs.ncsu.edu

RBT and DOK Alignment for Practitioners

More Related Content

What's hot (20)

Viewers also liked (11)

Similar to RBT and DOK Alignment for Practitioners (20)

Recently uploaded (20)

RBT and DOK Alignment for Practitioners