Paul Cisek Model - No "Decision" "Decision-Making"

The vanishing central executive

Distributed neural mechanisms of decision-making

Paul Cisek
Summer School in Cognitive Sciences
Evolution and Function of Consciousness
July 4, 2012

Our question:

• When, where, how and why — since the origin of life on Earth
about 4 billion years ago — did organisms' input/output
functions become conscious input/output functions?

But first, another question:

• Why “input/output” functions?

What is behavior?

sensory motor
Perception Cognition Action
input output

representation representation
of the world of the motor plan

“The whole neural organism, it will be remembered, is,
physiologically considered, but a machine for converting
stimuli into reactions” (James, 1890, p. 372).

Behavior: An analysis of the world,
followed by deliberation and planning,
followed by execution of the plan. William James

“sense, think, act”

Psychological architecture for behavior

sensory motor
input output

representation representation
of the world of the motor plan

• University courses
• Textbooks Q: From where does this view originate?
• Journals
• Conferences
• Academic departments
• Grant review committees
• Scientists
• Questions we ask
• Theories we propose

“Dualism”

sensory motor
Perception Mind Action
input output

Socrates

Descartes
• Philosophy: The mind is non-physical
– This forces interfaces between non-
physical mind and physical world

• Psychology: Study of the psyche
– Structuralism: The mind is studied
through introspection John Locke

Wilhelm Wundt

Behaviorism

sensory motor
Perception Mind Action
input output

– Stop this metaphysical nonsense…

John Watson

Behaviorism

sensory motor
Perception Action
input output

– Stop this metaphysical nonsense…
– Perception and Action are directly linked
– Subject matter: Learning laws which establish the linkage

John Watson Ivan Pavlov Edward Thorndike B.F. Skinner

Cognitivism

sensory motor
Perception Action
input output

– Internal processes are indispensable Tolman

Cognitivism

sensory motor
input output

– Internal processes are indispensable Tolman
Shannon
– Cognition takes the mind’s place
– A fully physical process – but what
kind?
– “Information processing”
• Definition of “information”
• Definition of “processing”
– Cognition is a computational process
Turing
• Linguistics
Chomsky
• Language of thought
Fodor

Computational view of the brain
• The “computer metaphor”
– Cognition is like computation:
Rule-based manipulation of
representations
(Newell & Simon, Pylyshyn)
– The mind is the software (Block)

Newell & Simon

– Studies of mental phenomena may be conducted independently of
studies of brain physiology
• Less to worry about
• Not so much known (yet) about the brain
• Historical separation between psychology and biology

What kinds of representations?

• “Descriptive” representations
– Capture knowledge about the world
and the organism
– Explicit
– Objective, accurate to external Descriptive representations
reality, uncontaminated by internal delineate the conceptual borders
states input/output
– Examples: between the processes that
•
^
Reconstructed visual image
•
construct them and the
3-D map of the world
input/output
• Labeled objects processes that use them.
• Desired path of the hand in space ^

David Marr

Can we use this to understand the brain?

• Cognitive Neuroscience
– How are psychological /
cognitive functions produced by
the brain?
– Ex: Decision-making
– Based on the concepts of
cognitivism
• Computation, descriptive
representations, working
memory, attentional filters, Michael Gazzaniga
motor programs, etc.

Where is the central representation?

• The visual system
– Two visual processing streams:
• ventral “what”
• where
dorsal “where”
– Separate regions analyze color,
motion, form, etc.
– Separate regions for near and
far space
what
• Binding problem
– How to create the unified
representation of the world that
is needed as input for cognition?

Perception, Cognition, & Action Systems?

• Primary sensory and motor regions
• “Association” regions
– Appear to first encode sensory,
then motor representations Cognition?
Cognition
– Even true at the level of
Cognition
individual neurons

Example: Lateral intraparietal area
– is it “attention”?
(input to cognition)
– or “intention”?
(output of cognition)
– How could it be both?
– Could it be cognition?

Where is the central executive?

• Decision-making
– Neural correlates in prefrontal
and orbitofrontal cortex
– also in parietal cortex
– Premotor cortex
– Supplemental motor area
– Frontal eye fields
– Basal ganglia
– Even the superior colliculus

– Activity reflects decision everywhere at
about the same time (~150ms)

Conceptual challenges

• The binding problem
– How to create the unified representation of the world that is needed
as input for cognition?

• The problem of meaning
– How does a computational process know the meaning of the
representations that it manipulates?
– “Chinese Room” (Searle)
– The “symbol grounding problem” (Harnad)
– Representations are purely syntactic, they have no intrinsic
input/output
semantics, no meaning to the system that uses them
^

Psychological architecture for behavior

sensory motor
input output

• Some observations:
1. This model inherits its structure from
mind-body dualism
2. Was designed to explain the abstract
problem-solving behavior of adult humans
3. Its concepts were developed under the
explicit assumption that the substrate
doesn’t matter

• Perhaps it should not be surprising that this model has difficulty
explaining neural data…

Evolution

• Two key concepts:

– Natural selection
• What is the selective advantage of X?

Darwin
– Descent with modification
• What are the phylogenetic origins of X?

Are brains input/output devices?

• What else could they be?

What kinds of devices are living systems?

• Control systems:
– Ex: Biochemistry
• Suppose there is some substance A necessary A

for survival
• Suppose there’s a catalyst for creating A whose
action is regulated inversely by the concentration of A
• Feedback control system
• Exploits consistencies in the laws of chemistry
• Control loop within the organism: “Physiology”

What kinds of devices are living systems?

• Control systems can extend beyond the skin
– Ex: Kinesis
B

• Suppose substance B cannot be produced A

within the body, must be absorbed from the world
• If the local concentration of substance B falls
below desired levels, move randomly
• Exploits statistics of nutrient distributions
(assumes that there is more elsewhere) Concentration of [B]
• Control loop that extends outside the skin: “Behavior”
– Reliable motor-sensory contingencies exist
• Statistics of food distributions (move → find food)
• Laws of optics and mechanics (contract muscle → arm moves)
• Laws of interaction (you show teeth → I back off)

• Animals are constantly doing whatever brings them to the most
desirable situation (full stomach, safety, etc.)
• “Behavior: The control of perception” (Powers, 1973)

Different ways of looking at behavior

1. Given a perception, produce the best action
“The whole neural organism, it will be
remembered, is, physiologically considered,
but a machine for converting stimuli into
reactions” (James, 1890).
William James

2. Of the possible actions, produce that which
results in the best perception
“What we have is a circuit… the motor
response determines the stimulus, just as
truly as sensory stimulus determines
movement” (Dewey, 1896).
John Dewey

Ethology

• Studies of animal behavior in Von Uexküll

the wild

• Species-specific behavioral
niches

• “Closed-loop” sensorimotor
control

• Key stimuli

Lorenz & Von Holst

Tinbergen

What kinds of representations?

• “Descriptive” representations • “Pragmatic” representations
– Capture knowledge about the world – Used to guide interaction between
and the organism the world and the organism
– Explicit – Implicit
– Objective, accurate to external – Subjective, mix external reality and
reality, uncontaminated by internal internal state, often correlate with
many variables at once
states
– Examples:
– Examples:
• Salience map
• Reconstructed visual image • Motor signals to the limb
• 3-D map of the world • Subject-dependent opportunities for
• Labeled objects action (“affordances”)
• Desired path of the hand in space

David Marr J.J. Gibson

Example: Decision-making

What to do?
Move the queen?
Protect the pawn?
Threaten the
knight?
How to do it?
Which grasp point?
“Selection” What trajectory?
How to avoid
obstacles?

“Specification”

• Classical model:
– First decide what to do (select) then plan the movement (specify)
– Sense, think, act

Decision-making in the wild

• The world presents animals with multiple opportunities for action (“affordances”)
• Cannot perform all actions at the same time
• Real-time activity is constantly modifying affordances, introducing new ones, etc.

Paul Cisek Model - No "Decision" "Decision-Making"

Action specification and selection
must occur in parallel

Sensorimotor contingencies influence
how selection should be done

Specification and selection in parallel

A population of tuned neurons

Distance

Cell activity
Direction

• Action Specification: Activation of parameter regions corresponding
to potential actions
• Action Selection: Competition between distinct regions of activity

What are the neural substrates?

attention Specification in the dorsal visual stream
– Cells sensitive to spatial visual information
(Ungerleider & Mishkin …)
– Involved in action guidance (Milner & Goodale)
– Divergence into separate sub-streams, each
specialized toward different kinds of actions
(Stein; Andersen; Colby & Goldberg; Matelli &
Luppino ...)
– An increasing influence of attentional effects,
enhancing information from particular regions of
interest (Duncan & Desimone; Posner & Gilbert;
Treue; Boynton ...)
– Parietal representation of external world is
“sparse” (Goldberg)

potential actions

attention

Fronto-parietal system
– Activity related to potential motor
actions (Andersen; Georgopoulos;
Kalaska; Wise; Hoshi & Tanji)
– Competition between potential actions
– Various biasing factors
• attention (Goldberg; Steinmetz)
• behavioral relevance
(Mountcastle; Seal & Gross)
• probability (Glimcher; Shadlen)
• reward (Glimcher; Olson)

potential actions

attention

behavioral
biasing

Basal ganglia
– Cortico-striatal-pallido-thalamo-cortical
loops (Alexander; Middleton & Strick)
– Selection of actions from among
alternatives (Mink; Redgrave et al.)
– Reward (Hikosaka; Schultz)

potential actions

attention
cognitive
decision-making

behavioral
biasing

Prefrontal cortex
– High-level decisions
based on knowledge
object about object identity
identity (Fuster; Miller; Tanji…)
– Receives ventral stream
information on object
identity (Sakata…)

potential actions

attention
cognitive
decision-making

predicted behavioral
feedback biasing

Execution
– Fast visual feedback
(Prablanc; Desmurget)
object – Forward models
identity
(Ito; Wolpert; Miall)

visual feedback motor
command

potential actions

attention
cognitive
decision-making

feedback biasing

object
identity

command

“Affordance competition hypothesis”
potential actions

Cisek (2007) Phil.Trans.Royal Soc. B.

•attention
Continuous specification of currently available potential actions
cognitive
decision-making
• Competition between potential action representations in fronto-
parietal regions
• Biasing from frontal and subcortical areas
feedback biasing
• Decision is made through a “distributed consensus”

object
identity

command

Behavior


audition attention propositional
proprioception vision logic forward
inverse
decision planning models
kinematics
making

object vision
trajectory action
recognition of space reinforcement
generation sequencing
learning

Behavior

Action Action
specification selection

decision
grasping reaching running attention making affect

reinforcement
forward vision proprio- inverse arm key stimulus
action learning
kinematics detection
arm models of nearby ception sequencing
space
object
propositional
recognition
logic

Predictions

• Multiple potential actions can be specified simultaneously

• Biased competition between potential actions

• Everything occurs in parallel

Neural activity specifies multiple actions

Classic model:
– Store information, decide,
then plan one action

Affordance competition:
– Specify both actions,
then select one
Time
Ce
Ce

Primary
llll P
PD

Motor
D

Cortex

Caudal
PMd

Rostral
PMd

Cisek & Kalaska (2005) Neuron

Biased choice task

1-TARGET 1 drop Alexandre
CHT 2 drops Pastor-Bernier
DELAY
3 drops
GO

THT

Reward: 1

GO
2-TARGET
FREE THT
CHT
Reward: 3
DELAY 67%

GO

THT

33%

FORCED Reward: 1

Neural activity in premotor cortex
• No effect of value
in 1T task
• However, if another
target is present, then
activity increases with
value of preferred
target
• If value of preferred
target is constant,
activity decreases with
value of other target
• Activity decreases with
distance between
targets

Pastor-Bernier & Cisek (2011) J. Neurosci.

Distance-dependent interactions

• More activity when targets are
closer
• Compare the strength of the
competition as a function of
target distance
– As distance increases, slope is
increasingly negative

• The competition is strongest
between cells with the largest
difference in preferred directions

Why should it matter that distance matters?

• The distance effect suggests that the decision is made
within the sensorimotor system
– If decisions were purely cognitive (“I prefer to get 3 drops of juice
over 1 drop”), then they should be determined in an abstract space
– The dynamics of the competition which determines choice depend
on the spatial relationship between the movements themselves

Timing
potential actions

• An animal is constantly interacting with the world
attention
– Continuous sensorimotor control of ongoing actions cognitive
decision-making
– Continuous specification of alternative actions
– Continuous evaluation of value
– Continuous tradeoffs between persisting in a given activity or
switching to a different, currently available one
feedback biasing
• Specification and selection must normally occur in parallel
• However, if we put the animal in the lab
– Time is broken into discrete “trials”object of which begins with a
each
stimulus and ends with a responseidentity
– The stimulus is deliberately made independent from the response

• What should we see? motor
visual feedback
command

potential actions

Wave 1. Fast feedforward sweep

potential actions

attention
cognitive
decision-making

behavioral
biasing

object
identity

Wave 2. Attentional/Decisional modulation

Two waves of activity
Ledberg et al. (2007) Cerebral Cortex

• Measured LFPs from various regions of cerebral cortex
• Monkeys performed a conditional GO / NOGO task

Two waves of activity

1. Fast feedforward sweep
• Activation in ~50ms
throughout dorsal
stream and frontal
cortex

2. Attentional/Decisional
• About 150ms post-
stimulus,
discrimination of
Go/Nogo throughout
the cortex

Summary 1: Experimental data

• Simultaneous specification of multiple potential actions
– Arm reaching system (PMd, PRR, M1)
– Grasping system (AIP, PMv)
– Saccade system (LIP, FEF, Superior colliculus)
• Biased competition
– Potential actions compete against each other within sensorimotor
maps, influenced by a variety of biasing factors (e.g. reward)
– NOTE: Similar mechanism as attention (Duncan & Desimone)
• “Attention” is selection near sensors, “decision” is selection near
effectors
– Influences depend on geometry – decisions are not simply abstract
• These are “pragmatic” representations, not “descriptive”
– Decision is made through a “distributed consensus”
• Parallel specification and selection systems

Summary 2: Theoretical concepts

• “Affordance competition hypothesis”
– Instead of serial Perception, Cognition, & Action modules, we have parallel
specification and selection systems
– Better match to neural data
– Better suited to the kinds of tasks that dominated animal behavior

• “Pragmatic representations”
– Neural activity aimed not at describing the world, but at mediating
interaction with the world
– Correlation with external and internal variables is necessary, but mixtures
are useful (e.g. spatial direction mixed with reward values)
– Conjecture: Most, but not all, neural activity is of this kind
• “Descriptive” representations (e.g. in the ventral stream) emerged in evolution as
specializations of pragmatic representations for advanced selection

• Cognitive advances evolved through hierarchical elaboration
– Diversification of fronto-parietal loops, cortico-striatal circuits, cortico-
cerebellar circuits, into progressively anterior/abstract systems
– Interaction lays the foundation for cognition (Piaget)

Summary 3: Philosophical implications

• There is no central executive
– Decisions emerge through a distributed consensus

• Classic problems in a different context
– Binding problem:
• Activity of separate streams is coherent by virtue of dealing with the same world
– Symbol grounding problem:
• Interaction has meaning by virtue of influencing the variables critical for life
• Symbols are specializations (“shorthand notation”) that emerged late in evolution,
already within the context of grounded interaction
– The “Hard” problem
• Feeling is different than doing
– Being inside the loop is different than observing it from the outside
– Private language, beetle in box, squirrel in head, 1 st person perspective, the “Umwelt”

• The computer metaphor
– With all due respect to Alan Turing, the computer metaphor is misleading as
a model for the brain
– What matters is control (Wiener, Ashby, Powers, Gibson, Dewey, etc.)

“The great end of life is not
knowledge but action”
– T. H. Huxley
(1825-1895)

“Your head is there to move
you around”
– R.E.M. (1980-2011)

THANK YOU

• Lab members
– Marie-Claude Labonté
– Alexandre Pastor-Bernier
– David Thura
– Ignasi Cos
– Matthew Carland
– Jessica Trung

• Alumni
– Jean-Philippe Thivierge
– Thomas Michelet
– Valeriya Gritsenko

EJLB
THE
FOUNDATION

paul.cisek@gmail.com

Paul Cisek Model - No "Decision" "Decision-Making"

More Related Content

What's hot (20)

Similar to Paul Cisek Model - No "Decision" "Decision-Making" (20)

Recently uploaded (20)

Paul Cisek Model - No "Decision" "Decision-Making"

Editor's Notes