A Pair-Wise Analysis Of The Cognitive Demand Levels Of Mathematical Tasks Used During Classroom Instruction And Those Assigned For Homework

CANADIAN JOURNAL OF SCIENCE, MATHEMATICS
AND TECHNOLOGY EDUCATION, 11(4), 348–364, 2011
Copyright C
OISE
ISSN: 1492-6156 print / 1942-4051 online
DOI: 10.1080/14926156.2011.624819
A Pair-Wise Analysis of the Cognitive Demand Levels
of Mathematical Tasks Used During Classroom
Instruction and Those Assigned for Homework
Donna Kotsopoulos
Faculty of Education, Wilfrid Laurier University, Waterloo, Ontario, Canada
Joanne Lee
Department of Psychology, Faculty of Science, Wilfrid Laurier University,
Waterloo, Ontario, Canada
Duane Heide
Waterloo Region District School Board, Kitchener, Ontario, Canada
Abstract: This research compared the cognitive demand levels of mathematical tasks engaged in
during classroom instruction to paired mathematical tasks assigned for homework. The research
took place in an eighth-grade classroom over the course of one school year. In total, the cognitive
demand levels of 66 mathematical tasks were evaluated using the IQA Academic Rigor: Mathematics
Rubric for the Potential of the Task (Boston Smith, 2009). Results from this research showed
that approximately two thirds of the time the mathematical tasks assigned for homework differed in
levels from the tasks used during classroom instruction. Implications for student learning, classroom
instruction, homework, and further research are discussed.
Résumé: Cette étude compare le niveau d’exigence cognitive des tâches mathématiques accomplies
en classe et celui des tâches mathématiques données en devoir à faire à la maison. L’étude a été
réalisée au cours d’une année scolaire, dans une classe de huitième année. Au total, le niveau
d’exigence cognitive de 66 tâches mathématiques a été évalué au moyen du test d’évaluation de la
qualité de l’enseignement de Boston et Smith (IQA Academic Rigor: Mathematics Rubric for the
Potential of the Task, 2009). Les résultats montrent que, dans environ les deux tiers des cas, les tâches
mathématiques données en devoir étaient d’un niveau différent de celui des tâches accomplies en
classe pendant les cours. Les implications de cet état de fait sur l’apprentissage, l’enseignement en
classe, les devoirs et la recherche future sont ensuite analysées.
From very early on during formal schooling, most students are asked to do some form of
homework. According to the recent Trends in International Mathematics and Science Study
(TIMMS; Mullis, Martin, Foy, 2008), the amount of time students spend on mathematics
This research was funded by the Social Sciences and Humanities Research Council.
Address correspondence to Donna Kotsopoulos, Faculty of Education, Wilfrid Laurier University, 75 University
Avenue West, Waterloo, ON N2C 3L5, Canada. E-mail: dkotsopo@wlu.ca

COGNITIVE DEMAND LEVELS OF MATHEMATICAL TASKS 349
homework varies significantly between countries and grades from two to three times per week to
three to four times a week for approximately 30 minutes at any one time. Despite such disparate
homework practices, these results suggest that many students spend a considerable amount of
time completing mathematics homework.
According to Trautwein, Niggli, Schnyder, and Lüdtke (2009), there are three primary functions
of homework. First, homework permits teachers to extend time spent on learning beyond the time
available during regular school hours. Second, homework enables students to rehearse what
they have learned during classroom instruction. Third, homework potentially enhances student
motivation and promotes self-regulation.
Homework has been examined from a variety of perspectives, including relationship to achieve-
ment (Cooper, Robinson, Patall, 2006; Dettmers, Trautwein, Lüdtke, 2009), parental involve-
ment (Cancio, West, Young, 2004; Patall, Cooper, Robinson, 2008; Pezdek, Berry, Renno,
2002; Tam Chan, 2009; Trautwein Lüdtke, 2009), teacher implementation and beliefs (i.e.,
assessed versus not assessed, answers taken up in class as a whole class, collected by the teacher,
etc.; Epstein Van Voorhis, 2001; Simplicio, 2005), student motivation and self-regulation
(Hong, Peng, Rowell, 2009; Stoeger Ziegler, 2008), types of homework (i.e., drill, problem
sets, problem solving, data collection, data analysis, online versus textbook, computer generated,
etc.; Fife, 2009; Kodippili Senaratne, 2008), and the efficacy of certain types of homework
over others (i.e., problem sets, textbook questions, computer generated; Epstein Van Voorhis;
Fife; Kodippili Senaratne; Simplicio; Trautwein et al., 2009).
Though homework has been examined broadly from a variety of perspectives, the cognitive
alignment of in-class mathematical tasks to those assigned for homework has not been well
researched. Yet, this may be an important consideration in facilitating and extending learning to
time spent on homework. Thus, reflecting upon the cognitive (mis)alignment of mathematical
tasks across settings through a pair-wise analysis may be useful. Therefore, in this research we
examined the cognitive demand levels between mathematical tasks engaged in during classroom
instruction and those assigned for homework in one class setting.
Cognitive demand levels, simply put, are the processes in which students engage during task
completion that are embedded in the task and may be varied in terms of levels of complexity
and demand (Stein, Grover, Henningsen, 1996). We elaborate more fully on cognitive demand
levels in the upcoming Theoretical Frameworks section.
The mathematical tasks examined in this research were corelated in that the classroom teacher
(third author), Duane, explicitly attempted to connect a mathematical task engaged in during
classroom instruction to one assigned for homework using the provincially-based curriculum
as his guide. Corelated, according to Duane, reflected mathematical tasks that used similar
mathematical processes, within a similar mathematical strand, with similar levels of challenge
and complexity (e.g., reasoning and proving, communicating, connecting, reflecting, selecting
tools, computational strategies, etc.). Duane’s motives in the corelating of the mathematical tasks
were to (a) provide opportunities for students to rehearse learning from the classroom and (b)
provide students with opportunities during homework to continue to engage in what he perceived
to be mathematical tasks of high cognitive demand. Though this latter motive may not be the
appropriate for structuring all instructional environments, this was nevertheless his goal. We
problematize the latter motive in the Theoretical Frameworks section.
With Trautwein et al.’s (2009) functions of homework in mind, the goal of our research was to
engage in a pair-wise analysis of the cognitive demand levels of mathematical tasks explored in this

350 KOTSOPOULOS ET AL.
eighth-grade classroom and the corelated mathematical task assigned for homework. We examined
the extent to which corelated mathematical tasks’ cognitive demand levels across both settings
were either in accord (i.e., similarly aligned in terms of cognitive demand level) or in discord
(i.e., a higher cognitive demand level of a task in one setting versus the other and vice versa).
THEORETICAL FRAMEWORKS
Mathematical Tasks
Watson and Sullivan (2008) described mathematical tasks generally as those “questions, situa-
tions and instructions teachers might use when teaching students” (p. 109). According to Stein
et al. (1996), a mathematical task is defined as a classroom activity (or sets of activities) in-
tended to focus students’ attention toward a particular mathematical idea. The cognitive-based
definition of a mathematical task by Stein and colleagues is different than other definitions that
consider characteristics of a mathematical task (i.e., nonroutine versus routine), levels of student
engagement, and so forth (Sullivan, Clarke, Clarke, 2009).
For the purpose of this research, we adopt Stein et al.’s (1996) definition of a mathematical
task. Mathematical tasks in this research are used during problem solving in the classroom and
then again assigned for homework. They can be viewed as synonymous with the term problem.
Evidence from empirical studies suggests that similarities in mathematical content or surface-level
features (i.e., graphs), textbook and curricular materials, and the teacher’s levels of mathematical
content knowledge most significantly influence a teacher’s selection of mathematical tasks rather
than an explicit consideration of cognitive demand (Epstein Van Voorhis, 2001; Kodippili
Senaratne, 2008; Lloyd, 1999; Lloyd Wilson, 1998; Remillard, 2000; Remillard Bryans,
2004; Sullivan et al., 2009).
Stein and colleagues (1996) suggested that a mathematical task may be (and likely is) com-
prised of multiple related activities rather than just one singular activity and that the multiple
related activities are ongoing with the explicit intent of occasioning cognitive change on behalf of
the student. Duane’s corelation of the mathematical tasks is consistent with Stein et al.’s definition
of a mathematical task as potentially a set of activities aimed at facilitating cognitive development
of a particular mathematical idea. Therefore, the corelated mathematical tasks can be viewed as
a unitary instructional environment that crosses two settings, that of the school and home.
High-Level Instructional Environment
As Boston and Smith (2009) have pointed out, over a decade of research has demonstrated that
high-quality learning environments that are sustained throughout instruction are most effective in
occasioning increased student achievement (Boaler Staples, 2008; Hiebert et al., 2004; Stigler
Hiebert, 1999). Yet no studies were found that analyzed the extent to which high-level instructional
environments are sustained beyond the classroom to the time spent on homework. Though the
types, duration, and relation to achievement of homework are interesting and indeed helpful
information for teachers, an understanding of why and how high-level instructional environments
from the classroom can or should be aligned to students’ experiences during homework may be
more useful for informing classroom instruction and improving student learning.

For this research, we define a high-level instructional environment as one in which students en-
gage with mathematical ideas that challenge and extend their own thinking through mathematical
modeling, argumentation, exploration, conjecturing, and so forth. We propose that an important
aspect of a high-level instructional environment is mathematical tasks of high cognitive demand
that are engaged in during classroom instruction and homework.
Cognitive Demand
Cognitive demand is defined “as the cognitive processes in which students actually engage as
they go about working on the task” (Stein et al., 1996, p. 461). The authors made the distinction
between mathematical “tasks that engage students at a surface level and tasks that engage students
at a deeper level by demanding interpretation, flexibility, the shepherding of resources and the
construction of meaning” (p. 459).
In their research, Stein and colleagues (1996; Stein, Smith, Henningsen, Silver, 2000)
distinguished between mathematical tasks of low and high cognitive demand levels. Mathematical
tasks with a low cognitive demand level are those that predominantly involve memorization and/or
engagement in mathematical processes in the absence of connections between mathematical ideas
(see Table 1). In contrast, mathematical tasks with a high cognitive demand level are those that
facilitate connections between mathematical ideas and require “doing mathematics” such that
students are engaging in self-reflection and self-regulation (see Table 1).
Recently, Boston and Smith (2009) elaborated on the two levels of cognitive demand proposed
by Stein et al. (2000) to include five levels (see Table 2). The five-level rubric was used by Boston
and Smith to investigate mathematical task implementation of secondary school mathematics
teachers following professional development. As Boston and Smith explained, Stein et al.’s
(2000) framework overall was used to formulate each of the levels within their own rubric.
The lowest level of Boston and Smith’s (2009) rubric, zero, is reserved for tasks that do not
have any mathematical activity (i.e., Internet searches, typing a report, etc.). Levels 1 and 2 are
somewhat aligned to Stein and colleagues’ (1996, 2000) low level in that these levels are also
representative of mathematical tasks that are largely limited to engaging students in memorizing
or reproducing facts, rules, formulae, or definitions (level 1) or do not require students to make
connections to the concepts or meaning underlying the procedure being used (level 2).
In contrast, levels 3 and 4 of Boston and Smith’s (2009) rubric aligns with Stein and colleagues’
(1996, 2000) high level. Level 3 requires students to engage in complex thinking and level 4
requires going beyond engagement with the mathematical task to a fairly sophisticated level of
exploration and understanding. The elaboration of the instrument, according to Boston and Smith,
allows for a more nuanced analysis of cognitive demand levels.
For the purpose of this research, and evolving from the definition presented earlier, a high-level
instructional environment is one that consistently stimulates learning and involves mathematical
artifacts that are at a level 4 as defined by Boston and Smith (2009). To facilitate a high-level
instructional environment across settings (i.e., from school to home), it could be hypothesized
that high cognitive demand levels during homework tasks may also be important.
To be clear, we do no not suggest that classroom tasks and homework tasks should always be
of a high cognitive demand level. This assertion only addresses the aim of achieving a high-level
instructional environment.

TABLE 1
The Task Analysis Guide From Stein et al. (2000)
Low-level cognitive demands High-level cognitive demands
Memorization: Procedures with connections tasks:
• Involves either producing previously learned facts,
rules, formulae, or definitions or committing facts,
rules, formulae, or definitions to memory.
• Focus students’ attention on the use of procedures for
the purpose of developing deeper levels of
understanding of mathematical concepts and ideas.
• Cannot be solved using procedures because a procedure
does not exist or because the time frame in which the
task is being completed is too short to use a procedure.
• Are not ambiguous—such tasks involve exact
reproduction of previously seen material and what is to
be reproduced is clearly and directly stated.
• Has no connection to the concepts or meaning that
underlie the facts, rules, formulae, or definitions being
learned or reproduced.
• Suggest pathways to follow (explicitly or implicitly)
that are broad, general procedures that have close
connections to underlying conceptual ideas as opposed
to narrow algorithms that are opaque with respect to
underlying concepts.
• Usually are represented in multiple ways (e.g., visual
diagrams, manipulatives, symbols, problem situations).
Making connections among multiple representations
helps develop meaning.
• Require some degree of cognitive effort. Although
general procedures may be followed, they cannot be
followed mindlessly. Students need to engage with the
conceptual ideas that underlie the procedures in order to
successfully complete the task and develop
understanding.
Processes without connections: Doing mathematics tasks:
• Are algorithmic. Use of the procedure is either
specifically called for or its use is evident based on prior
instruction, experience, or placement of the task.
• Require limited cognitive demand for successful
completion. There is little ambiguity about what needs
to be done and how to do it.
• Requires complex and nonalgorithmic thinking (i.e.,
there is not a predictable, well-rehearsed approach or
pathway explicitly suggested by the task, task
instructions, or worked-out example).
• Requires students to explore and understand the nature
of mathematical concepts, processes, or relationships.
• Have no connection to the concepts or meaning that
underlies the procedure being used.
• Demands self-monitoring or self-regulation of one’s
own cognitive processes.
• Are focused on producing correct answers rather than
developing mathematical understanding.
• Requires students to access relevant knowledge in
working through the task.
• Require no explanations or explanations that focus
solely on describing the procedure that was used.
• Requires students to analyze the task and actively
examine task constraints that may limit possible
solution strategies and solutions.
• Requires considerable cognitive effort and may involve
some level of anxiety for the student due to the
unpredictable nature of the solution process required.
Note. Reprinted with permission from Teachers College Record, copyright 2000 by the Teachers College, Columbia
University. All rights reserved.
There are some instructional environments whereby a mismatch in cognitive demand levels
between the classroom and the homework task may be pedagogically appropriate. As Trautwein
et al. (2009) stated, mathematical tasks may have one or more of three functions: extending
learning, rehearsal, and enhancement of motivation and self-regulation. Depending on the function

TABLE 2
IQA Academic Rigor: Mathematic Rubric for the Potential of the Task
4 The task has the potential to engage students in exploring and understanding the nature of mathematical concepts,
procedures, and/or relationships, such as:
• Doing mathematics: using complex and nonalgorithmic thinking (i.e., there is not a predictable, well-rehearsed
approach or pathway explicitly suggested by the task, task instructions, or a worked-out example);
or
• Apply the procedures with connections: applying a broad general procedure that remains closely connected to
mathematical concepts.
The task must explicitly prompt for evidence of students’ reasoning and understanding.
For example, the task may require student to:
• Solve a genuine, challenging problem for which students’ reasoning is evident in their work on the task;
• Develop an explanation for why formulae or procedures work;
• Identify patterns and form generalizations based on these patterns;
• Make conjectures and support conclusions with mathematical evidence;
• Make explicit connections among representations, strategies, or mathematical concepts and procedures;
• Follow a prescribed procedure in order to explain/illustrate a mathematical concept, process, or relationship.
3 The task has the potential to engage students in complex thinking or in creating meanings for mathematical
concepts, procedure, and/or relationships. However, the task does not warrant a level 4 because:
• It does not explicitly prompt for evidence of students’ reasoning and understanding;
• Students may be asked to engage in doing mathematics or procedures with connections, but the underlying
mathematics in the task is not appropriate for the specific grouping of students (i.e., too easy or too hard to
promote engagement with high-level cognitive demand);
• Students may need to identify patterns but are not pressed for generalizations;
• Students may be asked to use multiple strategies or representations, but the task does not explicitly prompt
students to develop connections between them; and
• Students may be asked to make conjectures but are not asked to provide mathematics evidence or explanations
to support conclusions.
2 The potential for the task is limited to engaging students in using a procedure that is either specifically called for
or its use is evident based on prior instruction, experience, or placement of the task. There is little ambiguity
about what needs to be done and how to do it. The task does not require students to make connections to the
concepts or meaning underlying the procedure being used. The focus of the task appears to be on producing
correct answers rather than developing mathematical understanding (e.g., applying a specific problem-solving
strategy, practicing a computational algorithm);
or
The task does not require students to engage in cognitively challenging work; the task is too easy to solve.
1 The potential of the task is limited to engaging students in memorizing or reproducing facts, rules, formulae, or
definitions. The task does not require students to make connections to the concepts or meanings that underlie
the facts, rules, formulae, or definitions being memorized or reproduced.
0 The task requires no mathematical activity.
Note. Boston and Smith (2009). Reprinted with permission from Journal for Research in Mathematics Education,
copyright 2009 by the National Council of Teachers of Mathematics. All rights reserved.
of the homework, it may be appropriate to have mathematical tasks of differing levels of cognitive
demand.
For example, if the pedagogical goal of the homework is to extend classroom learning, then
perhaps a higher cognitive demand level of the mathematical task assigned for homework may be
more appropriate. However, a higher cognitive demand level in the homework mathematical task

than that of the classroom mathematical task could present additional challenges for a student
who may not be able to independently work through a more difficult mathematical task and may
not have home support to rely upon (Kotsopoulos Lavigne, 2008; Patall et al., 2008; Pezdek
et al., 2002). The incompatibility of cognitive demand levels, with the homework being higher,
could jeopardize a student’s motivation to continue with the homework if the level is beyond the
student’s ability.
Likewise, if the pedagogical goal of homework is to extend the instructional time from the
classroom to home so that students can rehearse learning that occurred during class time, then
perhaps the same cognitive demand level between the classroom mathematical task and that
assigned for homework may be appropriate. In such cases, it might be more useful to have the
mathematical task from the classroom corelated with more than one homework question. The
second (or third) question may provide a higher cognitive demand level that would permit the
student to engage in an extension of the high-level instructional environment from the classroom.
This additional corelating of mathematical tasks in the homework could be useful in encouraging
motivation and self-regulation. For this to occur, however, the corelating of the questions, as well
as extension questions, should be made explicit to students in order for the students to recognize
how their engagement with particular questions is shaping their own learning (Adler, 1999; Kang
Kilpatrick, 1992).
We contemplated whether there would be any case where the cognitive demand level of the
homework should be lower than the level of the classroom mathematical task. Predominantly,
we only saw such a situation as a plausible approach for those students who may have required a
different cognitive demand level from the onset because of learning challenges. In the absence of
such a case, we hypothesize that cognitive demand levels of homework that are consistently lower
than those of the classroom may adversely affect all three goals of homework (i.e., extension
of learning time, rehearsal, enhanced motivation and self-regulation) outlined by Trautwein
et al. (2009). In such a case, the lower level homework task may not be perceived by the
student as productive use of time and may lead to decreased motivation. Additionally, such
a misalignment may give the student a false sense of thinking that he achieved mastery or
understanding of a concept when he may have not. Finally, a lower cognitive demand level of
mathematical tasks during homework may limit the potential for the rehearsal of more complex
processes.
RESEARCH QUESTION
Our pair-wise analysis of cognitive demand levels of mathematical tasks engaged in during class-
room instruction and those assigned for homework examined the extent to which (mis)alignment
of cognitive demand levels occurred. Consequently, our research question is as follows: How do
the cognitive demand levels of tasks assigned for homework compare to related tasks engaged in
during classroom instruction? The results of this research may be useful for contemplating the
structuring of instructional environment including those of a high level; thus, the results could be
important for student learning.
We make clear at this time that this research is a study of the practices of one classroom.
The results, though not generalizable, are intended to be exploratory and question generating.
Our research is limited to a mathematical task-level analysis exclusively. Given the dearth of

research in this area, the results are recognizably preliminary. A preliminary focus at the task
level is nevertheless fitting because mathematical tasks are arguably a beginning point for con-
ceptualizing and constructing instructional environments in mathematics; thus, they are a central
consideration.
METHOD
Participants
This research took place in an eighth-grade classroom during the course of one school year. There
were 14 male and 14 female students in the class. All students were between 13 and 14 years of
age. The school in which the research took place was located in an economically, socially, and
culturally diverse urban setting.
Duane, the classroom teacher in this classroom, had been teaching for 11 years at the time. He
had completed a master’s degree in education. In previous years, Duane had attended numerous
professional development sessions held through his school board and provincial mathematics
associations where he had learned about teaching mathematics using problem-solving tasks
in order to develop deeper thinking and understanding about mathematics. Consequently, his
lessons were predominantly structured around one or more mathematical tasks (see examples in
Table 3).
Duane was also considered a master teacher in his school board and thus was selected to host a
mathematics “demonstration” classroom where other teachers came to observe and contemplate
real-time mathematics instruction as a form of professional development.
Data Sources
The data set consisted of 66 mathematical tasks (33 paired mathematical tasks). Data for this
research were drawn from a year-long study exploring student thinking during mathematics
homework. Our role as researchers was strictly as that of researcher–observer. At no point during
the year-long study did we provide any input on directions for classroom instruction or feedback
of results from student-level data analysis.
Duane’s mathematics classes were video taped daily by a trained research assistant. Over
the course of the school year, Duane used 33 mathematical tasks during class (approximately
one per week) to explore five mathematical strands: data management, geometry, measurement,
patterning and algebra, and number sense and numeration. No researcher input was made into
the selection of the mathematical tasks throughout the research.
All of the classroom mathematical tasks analyzed in this research took place in small group
settings (i.e., two to four students). Some of these mathematical tasks were structured around a
real-world context; others were not. Often the exploration of the mathematical task took place
over two or more consecutive days. Mathematical tasks used in the classroom and for homework
were drawn from a variety of sources. These sources included the textbook, other curriculum
guides, Internet sites, and other teachers. Additionally, some tasks were developed independently
by Duane.

TABLE 3
Samples of Coded Mathematical Tasks
Mathematical
task No difference Class higher Homework higher
Class Regular tetrahedron task:
A regular tetrahedron is rolled and
the color on its face down (i.e., face
touching the flat surface) is recorded.
The colors on the faces are red, pink,
blue, and yellow. A spinner has
numbers 1 to 5, as shown below.
What is the chance of landing on a
pink and the number 4? Explain your
reasoning. (Level 4)
Summer camp task:
How much of a sub? Given the
information below, answer this
question: Did some of the
campers get more of a sub than
others or did all of the campers
receive the same amount of
sub?
(Level 4)
Cabin # Campers/# Subs
1 4/3
2 9/8
3 8/7
4 6/5
5 5/4
Circle task:
What can you measure in these
circles? (Level 2)
Homework Product 24 game task:
Fran and Aidan each design a game
called Product 24 game.
Fran designs a spinner with four
equal sections labeled with the
sections 3, 4, 6, and 8. The pointer on
the spinner is spun twice. To win the
spinner game, a person must spin two
numbers whose product is 24.
Aidan designs a game with a set of
four cards with the numbers 2, 3, 8
and 12. The cards are placed
downwards. One card is selected at
random and then replaced. The
second card is selected at random. To
win this game, a player must select
two cards whose product is 24.
Using the information about each
game provided, determine which
game a player has a better chance of
winning. Explain your answer.
(Level 4)
Lawn mowing task:
Bill had two thirds of the lawn
left to cut. After lunch he cut
three quarters of the lawn that
was left. How much of the
whole lawn did Bill cut after
lunch? (Level 3)
Plate task:
Circular plates with diameter
12 cm are placed side by side
on the top of a table. The table
measures 2.4 m by 1.2 m. How
many plates can fit side by side
on the tabletop with no part of
a plate extending over the edge
of the table? (Level 3)
Note. Color table available online.
At the beginning of each week, Duane would distribute a homework sheet to each student. The
homework sheet consisted of at least one mathematical task, which he had indicated was intended
to be corelated to the mathematical task explored during class, along with other questions related
to the mathematical strand. The other questions were primarily computation and predominantly
explored procedural knowledge.

The homework sheets were created by Duane. The mathematical task in the homework core-
lated to the task done in class and often involved the engagement of numerous mathematical
processes. Students had the autonomy to choose how much homework would be completed daily,
with the expectation that all homework was to be completed by the end of the week. The home-
work was taken up at the end of the week as a whole class. Every other week, the homework was
collected and evaluated for completion.
Duane was observed in the classroom video data as approaching problem solving in a fairly
consistent way. This is not to suggest that there were no subtle shifts in his pedagogy on a day-to-
day basis. Regardless of Duane’s consistency, uptake of the mathematical tasks may have varied
or shifted across students and even across mathematical strands. Our research is intentionally
focused at the task level and not on how the tasks were ultimately implemented by the teacher or
taken up by the students either during class or during homework (cf. Lithner, 2004). The way in
which the tasks may have changed as a result of implementation or uptake is beyond the current
research, which is focused exclusively at the task level.
Instrument and Coding
All coding was done by the first author and the teacher in this research following the conclusion
of the school year. Our interest in exploring the cognitive demand levels of the classroom
mathematical tasks and the homework mathematical task emerged at the conclusion of the
classroom data collection phase, following an interview with Duane during which he explained
his corelating of mathematical tasks. Therefore, there was no opportunity for Duane to change
his approach to selecting mathematical tasks as the year progressed as a result of the focus of this
aspect of the research.
Though it may be perceived that including Duane in the analysis may bias the results, our
own view is that including Duane in the coding of the mathematical tasks provided important
reliability and ensured that the pairing of the tasks was accurate. In addition, it was, as Duane
explained, a tremendous opportunity for him to reflect upon the mathematical tasks he used in
class and assigned for homework.
The first step of the data coding involved pairing the corelating homework question to the
mathematical task explored during class using Duane’s informal guidelines: mathematical tasks
that used similar mathematical processes with similar levels of challenge and complexity. Duane
did not use a formal instrument or guide for evaluating the mathematical tasks. Rather, he used
as an informal guide the Ontario Achievement Chart, which outlines levels of achievement in
mathematics rather than cognitive levels specifically (included in, Ontario Ministry of Education
and Training [OMET], 2005).
The achievement chart (OMET, 2005) provides a framework for assessment of learning through
four levels of achievement ranging from one as the lowest to four as the highest. Level 2 usually
reflects learning that may involve only limited conceptual reasoning and limited evidence of
computational fluency (i.e., only one mathematical operation). Level 4 is understood to represent
advanced thinking, reasoning, and analytical skills that extend beyond computational fluency
which is generally adequately captured with a level 3 within the achievement chart. According to
Duane, he aimed to select mathematical tasks that would stimulate level 4 processes.

The pairing process was done independently and then checked for interrater agreement between
the coders. For two classroom mathematical tasks, there was an uncertainty as to which homework
question matched the task. These two mathematical tasks were discussed and coded accordingly
following Duane’s explanation of his pairing.
Following the pairing of the mathematical tasks, we independently evaluated each mathe-
matical task using as our instrument, a modified IQA Academic Rigor: Mathematics Rubric for
the Potential of the Task from Boston and Smith (2009; see Table 2) which “assesses the level
of cognitive demand necessary for students to produce the best possible response to the task”
(Boston Smith, p. 133). The rubric has five different cognitive demand levels ranging from
zero to four, with four being the highest possible level.
Examples of the mathematical tasks coded as levels 2, 3, and 4 are provided in Table 3. An
example of a level 2 mathematical task is the circle task in which students are asked to identify
the ways in which a circle can be measured. The mathematical task is limited in that it does not
require students to make connections to the concepts or meaning underlying the measurements
being identified. In contrast, the circle task could have had a higher cognitive demand level had
students been asked to, perhaps, explore the relationship between the radius and the circumference.
The focus of the circle task does not appear to be on developing mathematical understanding.
Likewise, the circle task is not a level 1 because the task is not simply using a formula to determine
a measurement.
In contrast to the circle task, Table 3 also illustrates the lawn cutting task identified as a level 3.
In this task students are required to use proportional reasoning twice, first to determine how much
of the lawn is left to cut and then again to determine how much of the lawn left is subsequently
cut. This mathematical task has the potential to engage students in creating meanings about
proportional relationships. However, the lawn cutting task does not warrant a level 4 because it
does not explicitly prompt for evidence of students’ reasoning and understanding, nor does it ask
student to use multiple strategies or representations. Students are not asked to provide evidence
or explanations to support conclusions.
Table 3 also highlights the level 4 task called the regular tetrahedron task. In this mathematical
task, students are required to apply procedures with connections about theoretical and experi-
mental probability to determine the chance of rolling a certain number or colored face. As such,
the task has the potential to engage students in exploring the underlying nature of probability.
The task explicitly prompts students to explain their reasoning, therefore requiring the students
to make conjectures and support conclusions with mathematical evidence. Students would have
had to connect to some previous knowledge about experimental probability in order to conjec-
ture about theoretical probability. In making connections, some students may have conducted an
experiment to solve the task and gained understanding of overall probability given two separate
events.
There is a significant amount of subjectivity at play when assigning cognitive demand levels to
mathematical tasks, notwithstanding the clarity of the instrument being used, which is why two
independent coders were necessary. At the conclusion of our independent coding of the paired
corelated mathematical tasks, we compared our results and discussed differences. Differences oc-
curred in only 3 of the 66 pairs analyzed. Duane’s active involvement in the coding was extremely
beneficial in considering the discrepancies in the coding. His pedagogical intent and perspective
on the mathematical tasks were used as the ultimate guide for both the initial interpretation of the
pairings and any subsequent discrepancies across corelated mathematical tasks.

Data Analysis
Descriptive statistics and nonparametric tests were used to examine and compare cognitive
demand levels of paired mathematics tasks used during classroom instruction and those assigned
for homework. Wilcoxon signed-rank tests for nonparametric paired data were used to explore
the pair-wise relationship between mathematical tasks used in the classroom and the related task
assigned for homework. Spearman’s rho correlation coefficient was calculated to examine the
predictive relationship between class and homework mathematical task levels. Qualitative data are
provided to illustrate mathematical tasks from class and homework that had similar and different
levels.
In the Results section, we report on our research goal: to engage in a pair-wise analysis of
the cognitive demand levels of mathematical tasks from an eighth-grade classroom and related
mathematical tasks assigned for homework. In the Discussion section we reflect upon the potential
implications of the accords and discords of the pairs on student learning. Finally, in the Conclusion
section we identify potential research directions in light of our analysis.
RESULTS
The mean level of cognitive demand for both classroom and homework mathematical tasks was
3.3 (SD = 0.847 and 0.585 respectively), with more variation between cognitive demand levels
for the classroom mathematical tasks. However, no significant relationships were found between
the median levels of the mathematical tasks in class and those assigned for homework (Wilcoxon,
n = 33, Z = −0.052, p = .958, two-tailed). Corelated mathematical tasks (i.e., classroom and
the related homework task) were found to be independent of one another through a pair-wise
analysis.
Our analysis revealed that 75.7% of mathematical tasks selected and implemented in class
were at a cognitive demand level 3 or 4. In comparison, 94% of mathematical tasks assigned for
homework were at a cognitive demand level 3 or 4 (see Table 4).
However, mathematical tasks at a cognitive demand level 4 that were engaged in during
classroom instruction occurred 18% more than those assigned for homework. Examples of math-
ematical tasks that were coded as the same level or different levels are illustrated in Table 3.
Noteworthy is the finding that 30.3% of mathematical tasks assigned for homework had a
lower cognitive demand level than the corelated mathematical tasks from the class (see Table 5).
None of the mathematical tasks from either the classroom or homework were found to have a
TABLE 4
Cognitive Demand Level of Mathematical Task (%)
Class Homework
Level 2 8 (24.2) 2 (6.0)
Level 3 7 (21.3) 19 (57.6)
Level 4 18 (54.5) 12 (36.4)
Total 33 (100) 33 (100)

TABLE 5
Frequency of Differences Between Levels of Paired Mathematical Tasks, n = 33 (%)
No difference Class higher Homework higher
11 (33.3) 12 (36.4) 10 (30.3)
cognitive demand level of zero. In contrast, 36.4% of homework mathematical tasks were higher
than the classroom mathematical tasks. These results suggest that, at least for this data set, almost
two thirds of the homework questions did not represent a sustained cognitive demand level in
the homework phase, although approximately one third did represent an increase in cognitive
demand level.
Spearman’s rho correlation coefficients were calculated to examine the predictive relationship
between classroom and homework mathematical tasks. The correlation between cognitive demand
levels of classroom mathematical tasks and homework mathematical tasks was negative and not
significant (r = −0.03). These results suggest that there does not appear to be a predictive
relationship between the cognitive demand levels and the mathematical tasks; that is, a classroom
mathematical task of level 3 cognitive demand could not predict a similar cognitive demand level
in the mathematical task assigned for homework.
DISCUSSION
In this research we engaged in a pair-wise analysis of corelated mathematical tasks engaged in
during class and then during homework. The pair-wise analysis of the mathematical tasks was
appropriate and fitting given that the two settings (classroom and home) could be conceived of
as one instructional environment across two settings (Stein et al., 1996, 2000).
We are clear at the onset of our discussions about the tentativeness of our observations. Our
observations are exclusively at the mathematical task level. Given that the mathematical task
is often at the forefront of instructional planning, the task-level analysis of the instructional
environment is an important and relevant beginning point.
According to Duane, his intent in his corelating of the mathematical tasks from the classroom
and those assigned for homework was to engage students in opportunities to rehearse their
classroom learning. In selecting all mathematical tasks, he aimed to select mathematical tasks
that would be of a high cognitive demand level. He was motivated to provide students with a
high-level instructional environment. He was surprised when he discovered that the cognitive
demand levels were aligned only approximately one third of the time and that at least one third
of the tasks he selected were at a low cognitive demand level.
The findings from this research suggest that though the mean cognitive demand levels were
found to be consistent for both classroom and homework mathematical tasks, approximately two
thirds of the time the homework assigned in this class had either a higher or lower cognitive
demand level than the paired mathematical task from class. Consequently, at least two thirds of
the time Duane’s goal of establishing a high-level instructional environment may not have been
sustained during homework.

Regardless of Duane’s intention to corelate the mathematical tasks, our pair-wise analysis
revealed a nonsignificant relationship between the tasks from the two settings. This finding is
noteworthy given his explicit intent to corelate. It suggests that the corelation of mathematical tasks
and hence the structuring of instructional environments may be significantly more complex—even
for an expert teacher.
Duane, who was viewed to be a master teacher and who had received extensive professional
development training, had difficulty corelating classroom and homework tasks despite intention-
ally trying to do so. Duane used the Achievement Chart in the provincial curriculum documents
(OMET, 2005) as his informal guide in selecting tasks and in developing the instructional envi-
ronment. The Achievement Chart is used for assessment purposes and therefore may not have
provided an appropriate lens for evaluating mathematical tasks, which might explain some of the
misalignment that was observed. In order to afford the desired gains from homework, it is impor-
tant for teachers not only to be intentional in their efforts at corelating classroom and homework
mathematical tasks but to use tools that may help them in establishing these corelationships in
predictable ways.
Duane was very motivated to provide a high-level instructional environment and felt that his
efforts could have seriously benefited from the use of the IQA Academic Rigor: Mathematic Rubric
for the Potential of the Task by Boston and Smith (2009) used in this research. He also expressed
that in the future he would be using this instrument as a guide for evaluating mathematical tasks
for use in his classroom. Additionally, the idea of having a three-way pairing of mathematical
tasks whereby one mathematical task in the homework provides rehearsal and the other provides
an extended learning opportunity—each based upon the classroom mathematical task—was seen
as worth building into his future mathematics pedagogy.
Numerous studies have shown that a sustained high level of instruction is the most effective
for facilitating student learning (Boaler Staples, 2008; Hiebert et al., 2004; Mullis et al., 2008;
Stigler Hiebert, 1999). According to other studies, teachers typically use similarities between
mathematical tasks as a guide for selecting tasks (Epstein Van Voorhis, 2001; Kodippili
Senaratne, 2008; Lloyd, 1999; Lloyd Wilson, 1998; Remillard, 2000; Remillard Bryans,
2004; Sullivan et al., 2009). Thus, Duane’s attempt to corelate the mathematical tasks by com-
plexity and challenge using his provincial guide for achievement may be seen as a positive
pedagogical strategy.
It is important to note that the interpretation of what constitutes a high-level instructional
environment can and may vary across different learning settings. In this research, a high-level
instructional environment was intended to imply mathematical tasks that aim to provide oppor-
tunities for students to engage with mathematics at cognitive demand level 4 based on Boston
and Smith’s (2009) rubric. Furthermore, selection of a mathematical task that is deemed to have
a high level of cognitive demand may not result in sustained high levels of cognitive demand
during classroom implementation (Boston Smith). A mathematical task determined to have an
appropriately high-level cognitive demand for most students may have a cognitive demand level
that is either too low or too high for different students.
There are numerous pedagogical considerations that teachers must consider in selecting math-
ematical tasks, which include curricular goals, individual student needs, prior classroom learning,
and so forth. Each of these pedagogical considerations may, however, be biased by the teacher’s
own level of pedagogical and mathematical content knowledge (Adler Davis, 2006; Ball, Bass,
Sleep, Thames, 2005; Kotsopoulos Lavigne, 2008). Consideration of the cognitive demand

levels of mathematical tasks chosen by teachers should be tailored to the unique needs and learn-
ing trajectories of students and ultimately to the pedagogical intent of assigned homework. There
may be instances in which, for example, a misalignment of cognitive demand levels is appropriate
and necessary to support and facilitate learning.
Classroom instruction and learning cannot be divorced from what occurs later, in the home,
when students engage in mathematics homework. It may be useful and necessary to utilize a
cognitive demands rubric, such as the one used in this research by Boston and Smith (2009), for
structuring classroom learning and related homework.
CONCLUSIONS
This research examined the cognitive demand levels of corelated mathematical tasks engaged in
during classroom instruction and during homework. We found that cognitive demand levels of
paired mathematical tasks from this classroom and those in the homework differed more than two
thirds of the time. We conceptualized what accords or discords in cognitive demand levels across
the corelated mathematical tasks might mean for student learning through a pair-wise analysis.
The extent to which these results are idiosyncratic is unknown. These results cannot be
generalized because of the research design (i.e., study of one classroom setting); they nevertheless
raise important and interesting questions about homework and its relationship to classroom
practices and sustained high-level instructional environments.
Our research is intentionally focused at the task level of the instructional environment and
not on how the tasks were ultimately implemented by the teacher or taken up by the students
either during class or during homework (cf. Lithner, 2004). As Stein and colleagues (1996, 2000)
noted, a mathematical task may be transformed in many ways by many factors once “unleashed.”
The way in which the mathematical tasks may change as a result of implementation or uptake
is beyond the current research, which is focused exclusively at the task level. This remains an
important area of further research—both at the teacher and student levels.
Additionally, it would be also useful to explore the extent to which (mis)alignment of math-
ematical tasks adversely or otherwise influences learning and achievement. Persistent questions
related to learning and achievement include: Do changes in cognitive demand levels across core-
lated mathematical tasks influence learning and achievement? Are specific (mis)alignments more
beneficial for learning and achievement than others?
Further research is also needed to explore whether intentional discords or accords of cogni-
tive demand levels during homework completion support different types of learners. Lingering
questions would be: Are certain (mis)alignments more beneficial to students and if so does this
change across different types of students (i.e., English language learners, cognitively delayed,
etc.? How do (mis)alignments support different types of students? It would be useful to study
instances when students need additional rehearsal to examine whether a mathematical task for
homework with a similar cognitive demand level to the mathematical task from class improves
or enhances understanding.
As Duane articulated during the analysis, the IQA Academic Rigor: Mathematics Rubric for
the Potential of the Task (Boston and Smith, 2009) instrument was a useful guide to support his
thinking about classroom instruction, homework, and sustained high-level instructional environ-
ments. It would also be important to examine whether such an instrument (a) supports teacher

practice and (b) assists in developing teaching content knowledge. Another interesting research
scenario would be to compare student learning in situations in which teachers do not use the tools
outlined in this research with those that do.
Finally, the above outlined areas of further research would, from an instructional perspective, be
useful to test the extent to which corelated mathematical tasks support Trautwein et al.’s (2009)
proposed primary functions of homework (i.e., to extend time spent on learning, to rehearse
what students have learned during classroom instruction, and to enhance student motivation and
promote self-regulation).
ACKNOWLEDGMENTS
Thank you to Ms. Amanda Schell for her work as a research assistant. The authors also thank Dr.
Rina Zazkis and the anonymous reviewers for their valuable comments on earlier drafts of this
article.
REFERENCES
Adler, J. (1999). The dilemma of transparency: Seeing and seeing through talk in the mathematics classroom. Journal
for Research in Mathematics Education, 30(1), 47–64.
Adler, J., Davis, Z. (2006). Opening another black box: Researching mathematics for teaching in mathematics teacher
education. Journal for Research in Mathematics Education, 37(4), 270–296.
Ball, D., Bass, H., Sleep, L., Thames, M. (2005, May). A theory of mathematical knowledge for teaching. Paper
presented at the Fifteenth ICMI Study: The Professional Education and Development of Teachers of Mathematics
Aguas de Lindóia, Brazil.
Boaler, J., Staples, M. (2008). Creating mathematical futures through an equitable teaching approach: The case of
Railside School. Teachers College Record, 110(3), 608–645.
Boston, M. D., Smith, M. S. (2009). Transforming secondary mathematics teaching: Increasing the cognitive demands
of instructional tasks used in teachers’ classrooms. Journal for Research in Mathematics Education, 40(2), 119–
156.
Cancio, E. J., West, R. P., Young, K. R. (2004). Improving mathematics homework completion and accuracy of students
with EBD through self-management and parent participation. Journal of Emotional Behavioral Disorders, 12(1),
9–22.
Cooper, H., Robinson, J. C., Patall, E. A. (2006). Does homework improve academic achievement? A synthesis of
research, 1987–2003. Review of Educational Research, 76(1), 1–62.
Dettmers, S., Trautwein, U., Lüdtke, O. (2009). The relationship between homework time and achievement is not
universal: evidence from multilevel analyses in 40 countries. School Effectiveness School Improvement, 20(4),
375–405.
Epstein, J. L., Van Voorhis, F. L. (2001). More than minutes: Teachers’ roles in designing homework. Educational
Psychologist, 36(3), 181–193.
Fife, S. R. (2009). The relation of homework type to achievement and retention of students enrolled in intermediate
algebra. Houston, TX: University of Houston.
Hiebert, J., Gallimore, R., Garneir, H., Bogard Givvin, K., Hollingsworth, H., Jacobs, J., . . . Stigler, J. (2004). Teaching
mathematics in seven countries: Results from the TIMSS 1999 video study (NCES 2003-13 Revised). Washington,
DC: U.S. Department of Education, National Center for Education Statistics.
Hong, E., Peng, Y., Rowell, L. L. (2009). Homework self-regulation: Grade, gender, and achievement-level differences.
Learning Individual Differences, 19(2), 269–276.
Kang, W., Kilpatrick, J. (1992). Didactic transposition in mathematics textbooks. For the Learning of Mathematics,
12(1), 2–7.

Kodippili, A., Senaratne, D. (2008). Is computer-generated interactive mathematics homework more effective than
traditional instructor-graded homework? British Journal of Educational Technology, 39(5), 928–932.
Kotsopoulos, D., Lavigne, S. (2008). Examining “mathematics for teaching” through an analysis of teachers’ perceptions
of student “learning paths.” International Electronic Journal of Mathematics Education, 3(1), 1–23.
Lithner,J.(2004).Mathematicalreasoning in calculustextbook exercises.Journal ofMathematicalBehavior,23,405–427.
Lloyd, G. M. (1999). Two teachers’ conceptions of a reform-oriented curriculum: Implications for mathematics teacher
development. Journal of Mathematics Teacher Education, 2, 227–252.
Lloyd, G. M., Wilson, M. S. (1998). Supporting innovation: the impact of a teacher’s conceptions of functions on his
implementation of reform curriculum. Journal for Research in Mathematics Education, 29(3), 248–274.
Mullis, I. V. S., Martin, M. O., Foy, P. (2008). TIMSS 2007 international mathematics report: Findings from IEA’s
Trends in International Mathematics and Science Study at the fourth and eighth grades. Chestnut Hill, MA: TIMSS
PIRLS International Study Center, Lynch School of Education, Boston College.
Ontario Ministry of Education and Training. (2005). The Ontario curriculum Grades 1–8 mathematics, revised. Toronto,
Canada: Queen’s Printer for Ontario.
Patall, E. A., Cooper, H., Robinson, J. C. (2008). Parent involvement in homework: A research synthesis. Review of
Educational Research, 78(4), 1039–1101.
Pezdek, K., Berry, T., Renno, P. A. (2002). Children’s mathematics achievement: The role of parents’ perceptions and
their involvement in homework. Journal of Educational Psychology, 94(4), 771.
Remillard, J. T. (2000). Can curricular materials support teachers’ learning? Two fourth-grade teacher’s use of a new
mathematics text. The Elementary School Journal, 100(4), 331–350.
Remillard, J. T., Bryans, M. B. (2004). Teachers’ orientation toward mathematics curriculum materials: Implications
for teacher learning. Journal for Research in Mathematics Education, 35(5), 352–388.
Simplicio, J. S. C. (2005). Homework in the 21st century: The antiquated and ineffectual implementation of a time
honored educational strategy. Education, 126(1), 138–142.
Stein, M. K., Grover, B. W., Henningsen, M. (1996). Building student capacity for mathematical thinking and
reasoning: An analysis of mathematical tasks used in reform classrooms. American Educational Research Journal,
33(2), 455–488.
Stein, M. K., Smith, M. S., Henningsen, M., Silver, E. A. (2000). Implementing standards-based mathematics instruc-
tion: A casebook for professional development (Vol. 33). New York, NY: Teachers College Press.
Stigler, J., Hiebert, J. (1999). The teaching gap. New York, NY: The Free Press.
Stoeger, H., Ziegler, A. (2008). Evaluation of a classroom based training to improve self-regulation in time management
tasks during homework activities with fourth graders. Metacognition Learning, 3(3), 207–230.
Sullivan, P., Clarke, D., Clarke, B. (2009). Converting mathematics tasks to learning opportunities: An important aspect
of knowledge for mathematics teaching. Mathematics Education Research Journal, 21(1), 85–105.
Tam, V. C., Chan, R. M. (2009). Parental involvement in primary children’s homework in Hong Kong. School
Community Journal, 19(2), 81–100.
Trautwein, U., Lüdtke, O. (2009). Predicting homework motivation and homework effort in six school subjects: The
role of person and family characteristics, classroom factors, and school track. Learning Instruction, 19(3), 243–258.
Trautwein, U., Niggli, A., Schnyder, I., Lüdtke, O. (2009). Between-teacher differences in homework assignments
and the development of students’ homework effort, homework emotions. Journal of Educational Psychology, 101(1),
176–189.
Watson, A., Sullivan, P. (2008). Teachers learning about tasks and lessons. In D. Tirosh T. Wood (Eds.), The
international handbook of mathematics teacher education (Vol. 2: Tools and processes in mathematics teacher
education, pp. 109–134). Rotterdam, Netherlands: Sense Publishers.

Copyright of Canadian Journal of Science, Mathematics Technology Education is the property of Routledge
and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright
holder's express written permission. However, users may print, download, or email articles for individual use.

A Pair-Wise Analysis Of The Cognitive Demand Levels Of Mathematical Tasks Used During Classroom Instruction And Those Assigned For Homework

More Related Content

Similar to A Pair-Wise Analysis Of The Cognitive Demand Levels Of Mathematical Tasks Used During Classroom Instruction And Those Assigned For Homework (20)

More from Joe Andelija (20)

Recently uploaded (20)

A Pair-Wise Analysis Of The Cognitive Demand Levels Of Mathematical Tasks Used During Classroom Instruction And Those Assigned For Homework