The eye in the control of attention

The Eye in the Control of Attention

Michael Mair

INTRODUCTION

Some years ago, I persuaded two students to talk to each other for half-an-hour while
I was recording their moving faces on videotape, and their voices on audiotape. I
have been working with these records ever since, at first by means of transcriptions of
the movements of their noses and of the fundamental frequency of their voices,
latterly by seeking to understand something of the neurological basis of those
movements. In focusing on something so restricted - just the product of that half-hour
- I would appear to have "caught the lot," to have confronted the entire brain basis of
interactive behavior as it was manifested in those fragments. We do not, of course,
have a complete story, but must skim, hovercraftlike, over the surface of our subject,
dipping down into patches where there is detail known, trying to cover the territory
without losing sight of the two faces on the screen, flashing their eyes in their
interactive dance. They were caught forever just at the advancing edge of time,
leaving behind a story, a text which they had made together, controlling how it turned
out by movements of voice, face, head, and eye; making the future into the past via
the present, albeit the "specious present" of William James's description. Like two
projectors, their brains through their eyes seemed to stab into time, putting form on
the world just ahead, negotiating that form with their movements; leaving it organized
behind them, irretrievably, as their shared text.
Do eyes project vision onto the world? Greek theorists believed just that - that
something streamed out from the eye (Bldkemore, 1977, p. 65). Although we no
longer accept this, we still have no theory to explain the generation of visual
experience. We can say, as might Trevarthen, that we see because we look and that
the movements of the eyes are a manifestation of intention. He might say of my
interactors that what they were doing should be understood in terms of shared
intentionality. The microanalysts of interaction demonstrates the extraordinary detail
and precision of the interlocking of motor output among interacting beings. The

overall thesis of this paper is that these timings are so fast and so coordinated that the
conjoined brains which turn them out are actually modifying each others' output as it
is produced. 1 shall seek to demonstrate that the synchronies that William Condon
(Condon & Ogston, 1967) and others have observed do in fact make sense
neurologically, and 1 do this primarily by describing the visual system and its timings.
1 finish with a speculation about the nature of the faculty of "projection" or
"intention," which mobilizes the biological system.

THE EYE

I will spare the reader the actual dimensions of the eyeball - everybody being aware
that we have two of them, each roughly spherical, with a lens system, a sensory
surface called the retina, and a nerve connecting it to the brain. The lens system
delivers a real, inverted image of the viewed scene to the retina, and this is worth
stressing because some authors (Gibson, 1979, p. 61) occasionally seem to suggest
that a retinal image is somehow not necessary to sight. It is true that what we see has
only a very indirect relationship to the image, but quite a lot is in fact known at least
about the first stages in processing, and it starts with the image. Leonardo was the
first to get direct evidence of the reality of this (See Fig. 7. 1). The illustration shows
how he saw the image by peeling off layers from the back of an ox eye. Blakemore
whimsically suggests that it may have been the discrepancy between this upside-
down, distorted patch and the experience of sight which led him to doubt!
Subsequent knowledge would surely have led him to despair-the retinal image is
almost certainly the last representation of the world in the visual system that is at all
"imagelike."
Each retina has an area of about 10,000 square degrees, and has a complex
structure. Embryologically, the retinae are the pushed in tips of brain stalks (see Fig.
7.2), and this unique origin for a sensory surface has a number of sequelae, one being
that in its layers processing of the image goes on, that is done in the spinal cord or
brain stem for other senses, such as touch, hearing, and taste. The receptor units (rods
and cones) are particularly fine grained at a patch of retina corresponding to the "line
of sight," the fovea (see Fig. 7.3), where they are also limited to one type (cones).
Note the geometricality of the array, and this principle of architectural order is

maintained throughout, although by the time we get to the association areas in the
brain, we have little idea of what the order is doing for function.

FIG. 7.1 Leonardo and the upside down, distorted image.

FIG. 7.2 "The Retinae are the pushed in tips of brain stalks".
Diagramatic representation (Stylised transverse section)

Figure 7.4 shows some dimensions of this crucial patch of the back of the eye
which sees detail. Most of the fibres in the optic nerve serve this patch, and the
primary sensory area in the brain also is disproportionately committed to it, a
phenomenon known as "cortical magnification." But in fact, about 70% of the cells in
the retina make horizontal connections, and the similarity of this set up to the grain of
photographic emulsion ends at the outer receptor layer. In common with other senses,
the unit of sensation is a "center/surround" receptive field, mediated by these
horizontal connections. It is the innermost layer of the retina, the ganglion cells,
whose processes go to the brain, and these are of many types, with many different
receptorfield characteristics. What the brain gets is a very highly coded signal.
Before we leave the retina, it will be instructive to look inside a receptor unit (see Fig.
7.5) where again we see a very geometrical arrangement of membranous sacs. It is
said that a single quantum of light can cause a response in a photoreceptor, because of
an amplifier mechanism afforded by this internal chemical anatomy, and that a mere
half-dozen quanta distributed over a few square millimetres can reliably produce
sensation.

THE CORTEX

Above the retinae, there is a complicated exchange of fibres from the two eyes and the
result is that the visual world is neatly divided by a vertical line down the middle so
that the inputs from the left-hand halves of both retinae go to the left hemisphere, and
from the right-hand halves to the right hemisphere. In describing the visual system,
one must always remember the further inversion produced by the optics of the eye, so
it is the right half of the visual scene which goes to the left hemisphere, and vice
versa.

FIG. 7.3. Foveal cones, cross section.

FIG. 7.4. Macula dimensions (a) Foveola .35 mm (b) Fovea 1.85mm (c) Macula
2.85 mm (The margins of these areas are arbitrarily demarcated.)

From this crossover (the chiasm), the optic tracts, as the nerves are then known,
continue to the lateral geniculate nuclei of the thalamus, but a small side branch has
been given off to noncortical structures in the roof of the mid-brain, the superior
colliculi, which are mentioned later. In the geniculates we encounter a geometrical
array, and a systematic interleaving of the contributions from the two eyes. And in
the visual cortex as well, we have more geometric patterns in the extraordinary system
discovered by Hubel and Wiesel (1979). Whereas up until now the center surround
principle has been maintained in the way single cells are excited, in the cortex the
incoming information is rearranged so that most of its cells respond not to spots of
light but to specifically orientated line segments. It appears that the entire cortex,
including this primary visual cortex, is subdivided functionally by fine vertical
partitions into patches about one millimeter apart (Mountcastle, 1979), and that a
multilayered structure is also present throughout. For most of these systems, little is
known of their anatomical connections or functions in a precise way, but Hubel and
Wiesel have shown that in the visual cortex the columnar arrangement analyses
orientation systematically and also alternates a predominance of control between the
eyes (Hubel & Wiesel, 1979). Figures 7.6 and 7.7 are two of their schematic
diagrams of these columns, and Fig. 7.8 shows how they imagine that a simple line
stimulus to one eye excites the resultant highly ordered matrix.

FIG. 7.5. Single retinal receptors - an artist's impression. (The labeled
structures are cellular organelles 'k" indicates the stacks of membranous sacks.)

FIG. 7.6. Schematic diagram from Hubel and Wiesel (1979) of eye dominance
colums, showing change in preferred orientation of stimuli with depth in the
column. (The alternating 'R' and 'L' strips represent eye dominance columns).

Hubel and Wiesel talk of their columns being like machines, each analysing the
pattern that falls on its patch of retina. "Why," they ask, "should evolution go to the
trouble of designing such an elaborate architecture?" They speculate that it may deal
with the problem of portraying more than two dimensions on a two-dimensional
surface. I return to this dimension problem again at the end of the chapter, when I
speculate on what may be involved in generating and decoding four-dimensional
movement shapes, such as we have in gestures. We must note, however, that beyond
their meticulous work there is no clear picture as yet emerging about what happens
next in visual processing. Even in the visual cortex, we have the claims of many
authors that it is not simple line orientation lures that are being isolated out, but that
instead the retina/cortex system is a frequency analyzer performing a Fourier analysis
of the visual scene (Campbell & Robson, 1968). There is some evidence that it is
spatial frequencies at particular orientations that the single cells are responding to.
The hypothesis has its most daring exponent in Karl Pribram, and he supposes that the
visual system is making holograms, albeit multiplex holograms. It is difficult to
comment on the appropriateness or otherwise of these hypotheses without
considerable mathematical knowledge, but some sort of mathematical transformation
of the visual image does seem likely, and it is true that the basic building block of
perception does seem to be the center-surround field. A hologram hypothesis is
superficially appealing as we confront some aspects of higher visual processing.
Particularly, it is clear that the concept of receptorfield loses out to a looser concept of
image analysis in some of the association areas. Single cells in, for example, the
supero-temporal cortex, appear to respond when the stimulus is anywhere in an area
including the foveal region of both retinae! We seem to be getting such diverse
reports of the repertoire of single cells that there is almost, to pun mercilessly, a
cellfor all reasons. Nobody has tried it with a grandmother's face yet, but particular
faces, even particular configurations on individual faces, have been reported as giving
unique responses (Perratt, Smith, Milner, Jeeves, & Rogers, 1982). Binocular cells
have been found which fire only when something is coming straight at the observer,
not if it will pass him by. There is a great multiplication of visual areas, such as
Zeki's (1977) color areas, areas implicated in producing size constancy, an area in the
posterior parietal cortex linking vision and touch, some cells which fire according to
perceived color (rather than absolute wavelength), etc.

FIG. 7.7. The patterning of monkey visual cortex into eye dominance columns,
those dominated by one eye being black, the other eye white. (This is also a
schematic representation but at a larger scale and shows the interleaving
patterns on a monkey visual cortex, stained to differentiate the alternating eye
dominance.)

So, our brief review of this aspect of the visual system has become anecdotal. How
does all this relate to Attention, which we think of as regulating sensory input? Before
leaving the cortex to consider something of the role that subcortical structures may
play in attention, we must consider two well-documented phenomena which are
clearly related to this. First are the syndromes of Inattention, and second, the
phenomenon of Sensory suppression.

FIG. 7.8. Descartes' despair! What a line might "look" like in terms of the
topography of stimulation it elicits in the cortex. (A line of a particular orientation
is presented to one eye, and excites only those cells sensitive to its orientation.)

INATTENTION AND SUPPRESSION

It is one thing not to see because of damage to the eye or to the visual cortex, but
another to deny it, and to continue to behave as if sighted. Critchley quotes (1979)
what must be the first report of this, from the Roman author Seneca writing of his
wife's old nursemaid. "The silly old woman doesn't even know she is blind. She
keeps asking the house keeper to change her living quarters, saying her apartments are
too dark. "
We saw how a topographical concept had gone over to an image analysis concept
beyond the primary sensory cortex, and it is damage to these higher association areas
which typically gives the inattention or neglect syndrome, particularly the parietal
cortex. When the condition affects body sensation, the opposite side of the body is
just lost to its owner's consciousness, hanging loosely, and such patients may even
insist that it belongs to someone else. When the visual field is affected, we appear to
have a kind of black hole in sight. The defect may be complete or bilateral, or so
subtle as to be only revealed by rigorous testing. In such a test, a patient who may
successfully count the number of fingers that an examiner is holding up on the
affected side may have this ability obliterated by the simultaneous presentation of
fingers to count on the unaffected side-an impairment of attention with preservation of
sight. Such people may protect their disability with circuitous arguments, providing
the speech areas in the left cortex are intact. A similar imperative to make sense of
experience has been noted in split brain experiments, where the right brain has not the
competence to say what it knows, and so the left brain makes something up.
Sometimes the subject will internally complete a missing portion of the visual
field. That we can all do this is easily demonstrated by a simple and well-known
experiment. If we roll up a piece of paper into a tube and look through it with one eye
at, say, the wall, and then hold a hand in front of the other eye without obscuring the
tunnel view of wall seen by the first eye, we will see through the hand. The brain
simply turns off the area of retina obscured by the hand, and completes the hole with
the vision of the wall seen by the other eye. This is suppression and also completion.
Both inattention and suppression are about the interaction of attentional
mechanisms with raw sense data. In both cases, the raw sense data are there at the
retinal level, but lost; in the first case by a damage to association areas, in the second
by a turn off at some as yet unidentified site. It is instructive to discuss a condition in

which the latter almost universally occurs-squint-for the light it throws on the nature
of visual attention.

FIG. 7.9. Confusion and Diplopia. In 'A', both foveas (which are represented by
the intersection of the curved lines at the back of the schematic spheres) are
turned on. Since the eyes are squinting (ie. misaligned) dissimilar images are
superimposed.
In 'B', the fovea of one eye is "turned off" (suppression). Consequently the
object in the line of regard of the squinting eye is not perceived, and instead a
doubled and displaced image of the object in the line of regard of the
nonsquinting eye is seen.

Figure 7.9 is a diagram of eyes out of alignment, and it shows the two possible end
results of this mishap. The first, the superimposition of two images, is called
confusion. Clinician Fells (1979) points out that reports of this are very rare indeed.
The reason for this is that there is suppression of the crucial line of sight patch of
retina. This leaves perception of the line of sight image of the fixing eye doubled by a
fainter and blurred image whose position is very informative to the clinician about the
nature of the neuromuscular defect which has given the squint. But usually, and
especially in the juvenile onset squinter, the hole in sight (scotoma) of the deviating
eye also includes the patch of retina which receives the double image, and then the
subject sees a unitary visual world once more (Pratt-Johnson & MacDonald, 1976).
The explanation usually given for these effects is teleological-and tautological as
well. Suppression occurs to get rid of confusion and diplopia. But what would be
wrong with superimposed images, or doubled images? We have learned something
about visual attention-not only is it singular, but also it somehow regulates its own
input. An electrophysiological correlate of this has been elegantly provided by Arden
(1974), who has shown that the electrical response of the cortex during suppression is
indeed depressed. Among other electrophysiological demonstrations that the turn off

involves the cortical cells, is one in which an after-image effect was induced at some
level (beyond the retina) in a suppressing eye, and its effects were transferred to the
fixing eye (Blake & Lehmkuhle, 1976).
Suppression and Inattention are both cortical in that the cortex is demonstrated to
participate in these phenomena but this participation does not locate the attentional
process here. This point is emphasized at the conclusion of a recent paper on visual
attention (Wurtz, Goldberg, & Robinson, 1982) who admit that an enhancement of
response of cells in the posterior parietal cortex with attention.. . . . . . may accompany
visual attention, just as an eye movement may, but not be part of the neural
mechanism whose product is attention. " These authors, in an ingenious series of
experiments recording from single cells in the brains of conscious monkeys attending
to spots of light, managed to differentiate between cells which had enhanced
responses with visual attention to stimuli in the part of the visual field corresponding
to their receptive field both when that attention was associated with eye movement,
and when it was not. We have already described the fovea and the cortical
magnification of the area dedicated to it, and we shortly discuss the movement of the
eyes which brings the image of an attended object onto it. But nevertheless, line of
sight and visual attention, although very often identical, are not necessarily so, as the
expression through the corner of the eye testifies. It was only some cells in the
posterior parietal cortex which showed enhanced response with visually attended
stimuli and which did not, by their ingenious method, also evoke eye movement.
So now Suppression, Inattention, and Attention itself are all demonstrated to have a
cortical correlate, but it is not the cortex at all which is usually considered in
discussion of neural bases of attention. Subcortical structures are clearly implicated.

SUBCORTICAL STRUCTURES

Nauta and Feirtag (1979) identify in the brain four structures whose input derives in
one way or another from all (or most) of the neocortical expanse. These are the
Limbic System, the Striatum, the Pons (and through it the cerebellum) and the
Superior Colliculus. It is interesting to note that of these four, three infracortical
structures mentioned above are also implicated in the control of eye movement. The
Lymbic system comprises the Hippocampus and the Amygdala, and both output to the

hypothalamus, and perhaps to Septal nuclei. The hippocampus is found at the free
edge of the cortex where that structure is rolled in on itself. Nauta describes it as the
"end station of the neocortical march," the destination for sequential projections that
span the neocortical sheet. The Amygdala has close connections with temporal and
frontal cortex, and also directly from the olfactory cortex. Nauta conceives of the
need for successive cortical stages of visual, auditory, and somaesthetic sensation as
being, because object constancy is necessarily an abstraction, three dimensional,
whereas smell is just intensity gradients. We saw, before, for vision that the
association cortices were doing sophisticated image analyses. Somehow, they get into
attention, here with the Amygdyla and Hippocampus. One should remember also the
results of damage to these structures. The behavioral disturbances they produce are
complex, severe, and intractable.
The Striatum receives projections directly from all parts of the brain, in a
topographical fashion. Parts of its output, which is crucial for the initiation and
patteming of motor programes, curls back on itself to enter the ventro-medial nucleus
of the thalamus.
The pons, comprising fibre tracts linking the cerebellar hemispheres and embedded
reticular nuclei receives inputs from all parts of the neocortex and from there projects
to the cerebellum. The reticular cells situated there and elsewhere in the brainstem
and spinal cord are described by Nauta as "sitting with their dendrites-their cellular
hands-spread across several millimeters, hoping it seems to catch any sort of
message." We can note that in spite of this diffuse arrangement, usually identified
with arousal, very important eye movement control centres are here.
Finally, the Superior Colliculus receives input from many cortical areas, and from
the retina itself by a side branching of the optic tracts. It is implicated also in the
control of eye movement. The remarkable accuracy in localizing visual targets
achieved by some cortically blind people (blindsight) may have its anatomic locus
here.
An economical model of the attentional process is provided by Pribram (1975). He
identifies three classes of attentional systems, "arousal," "activation," and "effort,"
with references to the orienting reaction. He suggests that the fronto-amygdaloid
system is concerned with registering a novel stimulus, and damage leads to
inappropriate orienting. The Striatal system tells you what you latched on to and
damage to it causes the neglect. The effort component he identifies with the

Hippocampus. Figure 7. 10 is a diagrammatic representation of these structures in
situ. We noted before the homogeneous modular structure of the cortex, and can
contrast this to the highly differentiated nature of the subcortical structures (although
some, like the hypothalamus, also have a very homogeneous architecture). We know
also that they are phylogenetically older. All this is compatible with the speculation
that the core-brain structures are running the outfit, and that it is the cortex that is the
data bank and computational matrix. Against this, it is often said that the human is
sightless and senseless without the cortex, but this observation does not refute the
other hypothesis and indeed would be the expected result in such a highly integrated
system.

EYE MOVEMENTS

From Retina, to Cortex, to Core in search of sight; and we have still yet not seen
anything. For that, we have to look at it. There is one decision that we make about
100,000 times each day and that is the decision as to where to look next. The subject
of the control of eye movements has an enormous literature, and there is one class of
these movements, the saccades, which is under voluntary control, and thus can be
considered to be informed by our Intentions. Intention can be economically defined
as the process of organizing motor output, but this definition is not satisfactory in that
it would include those compensatory movements, sometimes classed as reflexes,
which simply maintain a status quo. In fact, there are five identifiable systems of eye
movement control, and even the saccades can be argued to be largely a matter of
reflexes. The literature itself shows a curious neglect phenomenon in regard to the
discussion of attention. For example, Davson (1980), in his textbook on the
physiology of the eye, says of the,fixation reflex merely that if "the eyes are
stimulated by a bright light in the peripheral field, Attention is aroused so that the
eyes move and the images of the object approach the foveae . . . " (p. 428). This
attention that can move the eye does not need to do so however.

FIG. 7.10. Schematic representation of core brain structures (from Nauta,
Scientific American, Vol. 241 No. 3) Note the embedded core brain structures
within the cortical mantle (cf. Fig. 7.12)

It remains a moot point whether, in the fixation reflex, it is really attention that is
caught by something in the peripheral field, or whether attention moves the eye to
capture something there. I chose the title of this chapter because of just such an
ambiguity. However, the saccadic system is the only one which brings new objects to
bear upon the fovea, unless they themselves stray across its projection. This system
moves the eyes very rapidly about 200-700 degrees of visual angle per second, a rate
which, if maintained, would at fastest make our eyes spin round in a complete circle
about twice each second. Much has been written about the ballistic nature of these
movements, and complex engineering models made of them. For a time, their
resistance to modification once initiated led to a theory of quantal sampling which
implied that the visual sensorium might only take in information in little chunks, and

the timing of these coincided with the eye movements. We do know, however, that
the saccadic movements are controlled from the frontal cortex, and that the fibres then
descend to the para-median reticular formation, and the final common path of the
occulo-motor nuclei. There is a reaction time of about 200 msec, before a saccade is
made. These little quick movements are surely the ones that make eyes twinkle.
Quite separate from this and almost in competition with it is the Smooth Pursuit
system, which appears to be controlled from the posterior cortex. Appearing only 6
weeks after birth, this system is very sensitive to the effects of drugs such as alcohol.
It is much slower than the saccadic system, only managing to cover 45' of visual angle
in one second-perhaps just good enough for watching tennis-and it is not ballistic, that
is, the adjustments for the speed of the followed object are continuously graded.
Rashbass (1971, p. 445) has compared the interaction of the saccadic and following
systems as being like two drivers of the same car, one trying to keep the speedometer
pointing at thirty m.p.h., the other trying to keep alongside another car also going at
30 m.p.h. There are situations when the position man must override the other, but
ultimately they must work together.
The third system, the vergence system, is poorly understood, but similar to the
above. This is the one that brings the eyes out of alignment to converge on something
close. It is associated with the pupil aperture becoming smaller and the eye focusing
in the near triad. The simple reflex of focus is a common synonym for attention, but
to have something come into view in a way which makes detailed analysis possible
entrains all the eye movement systems, and so perhaps this is a misplaced semantic
identity.
The fourth system, the vestibul/ocular reflex, is just that, a reflex of great speed-about
four msec latency as opposed to the 200 msecs of a saccade, and its neuronal links are
confined to the brainstem. It is responsible for stabilizing the eye relative to inertial
space and by means of it visual fixation upon the stationary world is automatically
established during head rotation. There is another separate reflex system making eye
position compensate for longer term postural changes. In fact, this most primitive of
eye control systems is not at all irrelevant to students of interaction. They should note
that this system adjusts eye position much faster than conventional frame by frame
analysis can detect. When the head moves around in the fast and precise way that it
does as people talk, much of the expressive effect of the eyes is achieved by this
means. People with disorder of this system are often advised to wear specially

squashy shoes to stop the world jumping about as they walk. It should also be noted
that this system connects by a fiber tract in the brainstem and spinal cord-the medial
longitudinal fasciculus-to muscles of neck and trunk so that they too contribute to the
maintenance of visual stability.
The fifth systen, which produces oculo-kinetic nystagmus, is the one which makes
the eyes flick when someone is looking out of a train window at the outside
environment. Many here will testify to a most interesting corollary effect which
occurs when one is, in fact, stationary, and the environment, e.g., another train, is
slowly moving. The overwhelming effect is of personal movement. It is thought that
this system is mediated by the parietal association areas where there may be a kind of
continuous updating of the personal context going on.
Finally, one should mention that superimposed on all these five systems is a
continuous fine vibration of the eyes which keeps the otherwise stabilized image in
slight movement against the detail of the actual retinal receptors. Without this, there
is a fade out of the image, known as the Troxler phenomenon.
So we have five movement systems, one concerned with bringing objects onto the
fovea, and the rest with keeping it there. The system appears dedicated to achieving
stability of the visual image, and there are other mechanisms too which promote this.
There is the saccadic suppression which turns off perception while a fast eye
movement is in flight, and there is the supposed corrollary discharge which is an
entirely hypothetical feedback to the brain from the nerves governing eye movement,
to tell it that nothing really moved after all. It is necessary to believe in this because if
the eyes are moved artificially, the world does in fact seem to move. The source of it
has never been discovered. The world we see, unless malfunctioning, has a stability
independent of our own movements.
Before leaving eye movements, it is instructive to view some pictures of scan paths
made by the eyes when viewing a scene, for example a picture. Figure 7. 1 1 is taken
from Yarbus (1976), who was able to reproject the targeting of eye movements back
onto the scene surveyed using a cumbersome apparatus of suction caps applied to the
eye. The study of scan paths too has an enormous literature, and modem techniques
are a lot less invasive than that employed by Yarbus. Note that the dance of the eyes
more or less outlines the object. In fact, the eye movement strategy employed varied
according to the search task that the experimenter set the subject. In the saccadic
system, therefore, intention modifies the trajectory of the glances; but there is no

invariant pattern of scan which is followed for particular search tasks. There are
statistical probabilities which can to some extent be generalized across subjects, yet
each trajectory of glance is itself precisely programed. The dilemma is similar to that
in the analysis of speech. One can enunciate certain rules which speech is likely to
conform to, yet can never predict the precise form an utterance will take, or even if
there will be an utterance. The microanalysts of interaction does demonstrate,
however, that when speech and eye movement occur together, they are so precisely
coordinated as to make it likely that they share a timer.

FIG. 7.11. From Yarbus, 1976.
Scan Paths.

EVOKED POTENTIAL STUDIES

The precisely timed phenomena of eye movements tell us something of the timings of
cortical processes. There is another technique of investigation of attentional processes
which purports to estimate them directly. This is the Evoked Response, or Event-
Related Potential. There is an immense amount of work done on this (cf. Picton
1978, for a review). Effects studied included alterations in the Contingent Negative
Variation, a potential change which precedes an action, changes in the "N 100"
component, and changes in the "P 3(M" component. A word on Method is in order.
Electrical recordings are taken from the scalp. The "P" and "N" refer to positive or
negative respectively, and the numbers coming up after them refer to the latency in
msec of the response after the stimulus. The stimulus in all these experiments has to
be a very standard and simple one such as the detection of an unexpected
configuration in an auditory or visual series which can be repeated. The repetition is
necessary because of the technique itself which relies upon response averaging, the
accumulation of a large number of responses to cancel out noise. The noise is present
because of the remoteness of the signal from the recording electrode, and because so
much else is going on in the head besides the response to the stimulus under
investigation. Whereas before we could at least base our discussion on the neuro-
anatomic and physiological substrata, this connection is less secure with event-related
potentials, and so one is engaged in "black box" types of model building. In auditory
experiments, an "N 100" component has been identified which is thought to go with
the activation of short-term memory processes relevant to the analysis of attended
incoming information. The refractory period is about ten seconds. It has been
suggested that this long refractory period might be homologous with the duration of
the conscious present.
The most frequently studied of these waves, however, is the "P 300", a positivity
occurring at 300 msec after the stimulus. Such a wave is said to be related to the
subjective expectancy of the response, a kind of information content in the stimulus.
Another study found a large "P 300" with syntactic closure in utterances, a kind of
resolution of temporal uncertainty (Picton, 1978, p. 455). It would seem that the kind
of event that evokes large P 300s is an ah-ha experience, or as Picton et al. more
densely put it, "the late positive component, being in some way associated with
unequivocal task relevant information to its response in the context of the possible

responses to that information." It is tempting to equate this timing with the movement
of consciousness itself. Reaction time is also of this order (Donchin, Ritter,
McCallum, & Cheyne, 1978). Certainly, we know from the existence of reaction time
that quite complex situations can be understood and dealt with appropriately as fast as
this. And we have seen from the timing of the saccade that this also is how long eyes
take to twinkle. The evoked response studies fit very comfortably with timings of
cerebral processes from these other sources.

SOME TIMINGS

In an earlier paper (Mair, 198 1) 1 caricatured the brain in a simple three dimensional
line diagram which forms a simple conceptual summary of one interpretation of our
knowledge. In it, we see three sets of paired plates boxing in a hooked central
structure, and these represented the cortex and the core brain respectively. I imagined
a cycle of activity in this device, one part of the loop being in the core brain, and its
closure being action on the world, the results of which become incorporated and then
transformed in the next action, and so on (see Fig. 7.12).
The model is designed to generate text, which is seen as an ordered concatenation
of these cycles, each of which delivers a temporally stable state of play. This essay
has given more detail of what goes on during such a cycle for vision, and we continue
the speculative approach now by listing some of the timings which may be involved.

Function Time Taken

Nerve conduction, e.g., simple reflexes msec
such as the vestibulo-ocular

Fastest discdminatable visual impression 150 msec
Syllables, or phoneme clusters
Saccadic Eye Movements
Reaction time
Halliday's Tone Groups 200-300 msec
Late Positive Component (p300)

Short-term memory span 10 sec
Conscious present
Text Episode

I list the visual, auditory, and electrical phenomena in clusters together. Considered
as separate phenomena they do appear to cluster in this way, and microanalysts of
videotape of spontaneous interaction also demonstrates co-patterning (Kendon, 1973).
Kendon suggests perhaps a separate origin for the kinaesthetic/visual and auditory
modes, but at least at the level of output they appear to share a final common pathway.

FIG. 7.12. A Caricature of the Brain (a model of the text
generator). Highly simplified schematic view of one possible
interpretation of the cerebral realities.

CONCLUSION

In this final section, I would like to bring it all together as the brain itself must in so
far as the brain product is coherent text. As Paul Bouissac once suggested, it could be
that the workings of the mind are to some extent anarchic and do not make sense very
well. In struggling to achieve a comprehensible model of the process of generating
text we are inevitably engaged in the very process we seek to study and will have a
very definite bias towards order and simplicity. We might thus be imposing a value
judgment on our model, trying to make one which works well in producing something
which is only one aspect of the activity of the generator-viz. the production of well-
ordered, comprehensible text. Therein lies a joke. However, let us return to the
project outlined at the beginning of this chapter. Can this model address the specter
of the process of interaction revealed by the prolonged study of objective records of a
half-hour-long conversation between two people? In what sense can it be said to
account for it?
The digression into the physiology of vision shows something of the sort of
operations the brain is completing for sight, and the time they take. The existence of
synchronies in interaction, both in the sense of actually synchronously timed events
and interweavings of timings (as if two beings shared a common timer) become
comprehensible if the preconscious and conscious processes in those two beings are in
fact intermeshed. The paradigm for this is that of two people working together in a
visual scene, transforming it by their joint action. As they intersect with the timings
in this scene, for example in catching an animal on the move, they are inevitably
taking their timings from a common external source, and in so far as they are effective
in working together, will continue to work in synchrony. By outlining a model for the
text generator which is roughly in accord with its structure and functioning in the
temporal aspect, one can follow into the brain how successful coordinated activity
might be working.
For face-to-face interaction which involves speech, one might see a continuation of
this process into the human virtual world which is dependent on the sign system of
language. In this theory, the trajectories of vocal output are the manifestation of the
interweaving cerebral processes as well as part of the means, and the shapes of them
form the record of how the emerging outcomes were controlled (Mair, 1978, pp. 24,

34), in the same way that analysis of movement patterns in, say, physical combat,
shows them to be the means and the record of the outcome.
The model of cyclical activity in the brain which becomes coordinated when
working on a common scene which is either visual or virtual can follow the process,
and in this sense render the brain transparent. There are many opaque areas in the
model, but in principle it works, in terms of structure and timings, and is in accord
with some of the anatomical and physiological realities. But it does not account for
the actual form or content of any section of text, and I think that part of the reason for
this might be that the model is four-dimensional, whereas an adequate model might
need to employ five or more dimensions.
We saw that the three-dimensionality of the visual and somaesthetic world was
seen as a problem for the brain which has its primary sensory surfaces as one-
dimensional (hearing and smell) and two-dimensional (touch and vision). Nauta, and
Hubel and Wiesel suggested that the complexity of cortical processing might in part
be attributed to this difficulty of representing three dimensions on two-dimensional
surfaces. We know also that the direct perception of depth in vision requires the
subterfuge of disparate images from two eyes and it is at present unknown how three-
dimensional perceptions are achieved from this. A movement shape is however four-
dimensional, and it is these four-dimensional shapes which form the entities that the
brain must work on in interpreting actions and gestures. We appear to be up against a
limit of subjective comprehensibility in studying four-dimensional shapes. It appears
that we can deal in them, and must do, when engendering four-dimensional text, but
we cannot stand aside from them and manipulate them at the same time. We cannot
even represent them, but must put ourselves through a chunk of experience in order to
perceive them at all. This does not mean that five-or "n"-dimensional models cannot
be constructed, but it might mean that a four-dimensional model of the process of text
generation has no predictive value. Perhaps the Greeks were right, and brains do, in
fact, project out through the eye.

REFERENCES

Arden, G. B., Barnard, W. M., Mushin, A. S. (1974). Visual evoked responses in
Amblyopia. British Journal of Opthalmology, 58, 190.
Blake, R., & Lehnikuhle, S. W. (1976). On the site of strabismic suppression.
Investigative Ophthalmology, 15, 660.
Blakemore, C. (1977). Mechanics of the mind (p. 65). New York: Cambridge
University Press.
Campbell, F. W. & Robson, J. G. (1968). Application of Fourier analysis to the
visibility of gratings. Journal of Physiology, 197 551-566.
Condon, W. S., & Ogston, W. D. (1967). A segmentation of behaviour. Journal of
Psychiatric Research, 5, 221-235.
Critchley, M. (1979). Modes of reaction to central blindness. In The divine banquet
of the brain (p. 157). New York: Raven Press.
Davson, H. (1980). Physiology of the eye. London: Churchill Livingstone. Pp. 428.
Donchin, E. Ritter, W., McCallum, W. Cheyne (1978). Cognitive psychophysiology.
In Callaway et al. (Eds.), Event related brain potentials in man. New York:
Academic Press.
Fells, P. (1 979). Confusion, diplopia and suppression. Transactions of
Ophthalmological Society of the U.K., 99, 386-390.
Gibson, J. J. (1979). The ecological approach to vision. Boston: Houghton Mifflin.
Hubel, D. H., & Wiesel, T. N. (1 979, Sept.). Brain mechanisms of vision. Scientific
American, 241, No. 3.
Kendon, A. (1973). Studies in dyadic communication. In M. Van Cranach & I. Vine
(Eds.), Social communication and movement. European Monographs in
Psychology series.
Mair, M. W. (1 978). Steps towards principles of text regulation. Toronto Semiotic
Circle, Victoria
University, Toronto. 2, 24, 34.
Mair, M. W. (1981). A model of the text generator. In P. Perron (Ed.), The
neurological basis of signs in communication processes. Toronto Semiotic Circle.
Mountcastle, V. B. (1979). An organising principle for cerebral function. In B. M.
Edelman & V. B. Mountcastle (Eds.), The mindful brain, Cambridge: MIT Press.
Nauta, W. J. H., & Feirtag, M. (1979, Sept.). The organisation of the brain. Scientific
American, Vol. 241, No. 3.
Perratt, S. I., Smith, P., Milner, D., Jeeves, M. A., & Rogers, B. J. (1982). Visual
properties of temporal lobe neurons selectively receptive to the sight of faces.
Investigative Ophthalmology and Visual Science, 22, supplement.
Picton. (1 978). Neuropysiology of human attention. In J. Requin (Ed.), Attention
and performance VII. Lawrence Erlbaum Associates, Hillsdale, NJ.
Pratt-Johnson, J. A., & MacDonald, A. L. (1976). Binocular visualfield in strabismus.
Canadian Journal of Ophthalmology, 11, 37.
Pribram, K. (1975). Arousal, activation and effort in the control of attention. Psych
Review, 82, 116-149.
Rashbass, L. R. (1971). Second thoughts on smooth pursuit. In P. Bach y Rita et al.
(Eds.), The control of eye movements. New York: Academic Press.
Wurtz, R. H., Goldberg, M. E., & Robinson, D. L. (1982, June). Brain mechanisms
of visual attention. Scientific American Vol. 246, No. 6.
Yarbus, A. L. (1976). Eye movements and vision. New York: Plenum Press.

Zeki, S. M. (1977). Colour coding in the superior temporal sulcus of rhesus monkey
visual cortex. Proceedings of the Royal Society of London (Biol.), 197, 195.

The eye in the control of attention

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to The eye in the control of attention (20)

More from eyetech (6)

The eye in the control of attention