Temporal Binding and Segmentation in Visual Search:
A Computational Neuroscience Analysis

Eirini Mavritsaki1,2 and Glyn Humphreys2*

Abstrait

■ Human visual search operates not only over space but also
au fil du temps, as old items remain in the visual field and new items
appear. Preview search (where one set of distractors appears
before the onset of a second set) has been used as a para-
digm to study search over time and space [Watson, D. G., &
Humphreys, G. W. Visual marking: Prioritizing selection for
new objects by top–down attentional inhibition of old objects.
Psychological Review, 104, 90–122, 1997], with participants
showing efficient search when old distractors can be ignored
and new targets prioritized. The benefits of preview search
are lost, cependant, if a temporal gap is introduced between a
first presentation of the old items and the re-presentation of
all the items in the search display [Kunar, M.. UN., Humphreys,
G. W., & Forgeron, K. J.. History matters: The preview benefit in
search is not onset capture. Sciences psychologiques, 14, 181–
185, 2003un], consistent with the old items being bound by tem-
poral onset to the new stimuli. This effect of temporal binding

can be eliminated if the old items reappear briefly before the
new items, indicating also a role for the memory of the old
items. Here we simulate these effects of temporal coding in
search using the spiking search over time and space model
[Mavritsaki, E., Heinke, D., Allen, H., Deco, G., & Humphreys,
G. W. Bridging the gap between physiology and behavior: Evi-
dence from the sSoTS model of human visual attention. Psycho-
logical Review, 118, 3–41, 2011]. We show that a form of
temporal binding by new onsets has to be introduced to the
model to simulate the effects of a temporal gap, but that effects
of the memory of the old item can stem from continued neural
suppression across a temporal gap. We also show that the
model can capture the effects of brain lesion on preview search
under the different temporal conditions. The study provides a
proof-of-principle analysis that neural suppression and tempo-
ral binding can be sufficient to account for human search over
time and space. ■

INTRODUCTION

Search Over Space and Time

The ability to efficiently search and select a target for ac-
tion is crucial to human survival. Search across space has
been much studied, and the conditions supporting both
efficient and inefficient search have been established in
terms of the similarity and combinatorial relationships
that distinguish targets from distractors (Treisman,
1998; Wolfe, 1994; Duncan & Humphreys, 1989; Treisman
& Gelade, 1980). Search across time has been studied
largely using the preview procedure. Under preview con-
ditions, participants may be asked to carry out a conjunc-
tion search task (blue H target vs. blue O green H
distractors), but unlike standard conjunction search,
one set of distractors is presented before the second
set of distractors plus the target (when present; see Watson
& Humphreys, 1997). Although standard conjunction
search is typically difficult, with search slopes of the order
de 225 msec/item or more, preview search can be highly

1Birmingham City University, 2Oxford University
*Sadly, Prof. Humphreys passed away on January 14, 2016.

efficient—with slopes equated to when the displays con-
tain only the second set of distractors and the target. Tem-
poral differences between stimuli can be used to guide
attention efficiently to just the new set of items.

Although preview search is a relatively simple proce-
dure, the evidence suggests that several factors con-
tribute to performance. Par exemple, mere temporal
segmentation alone is insufficient to explain target selec-
tion because it requires relatively long time intervals be-
tween the preview and the search display to optimize
recherche (par exemple., of the order of 400 msec or more; Humphreys,
Olivier, & Braithwaite, 2006; Watson & Humphreys,
1997)—much longer than those required for temporal seg-
mentation ( Yantis & Gibson, 1994). On top of temporal
segmentation then, the data suggest that participants ac-
tively suppress the previewed items when they prioritize
search for the target. This means that probes are difficult
to detect when presented on old previewed stimuli com-
pared with when they fall on new items and even relative
to when they fall in the background of the search dis-
plays (Humphreys, Stalmann, & Olivier, 2004; Watson
& Humphreys, 2000). This suppression of old items
was not found when the primary task was probe detec-
tion and search for the new targets was not prioritized,

© 2016 Massachusetts Institute of Technology. Published under a
Creative Commons Attribution 3.0 Unported (CC PAR 3.0) Licence.

Journal des neurosciences cognitives 28:10, pp. 1553–1567
est ce que je:10.1162/jocn_a_00984

D
o
w
n
je
o
un
d
e
d

F
r
o
m

je

je

/

/

/

/
j

F
/

t
t

je
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
je
n
t
o
p
un
r
d
c
e
.
d
s
F
je
r
o
je
m
v
e
h
r
c
p
h
un
d
je
je
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
toi
c
n
o
/
c
un
n
r
un
t
r
je
t
je
c
c
je
e
e
–
p
–
d
p
d
2
F
8
/
1
2
0
8
/
1
1
5
0
5
/
3
1
1
5
9
5
5
3
1
/
5
1
6
0
7
8
o
5
c
3
n
4
_
9
un
/
_
j
0
o
0
c
9
n
8
4
_
un
p
_
d
0
0
b
9
oui
8
g
4
toi
.
e
p
s
t
d
o
F
n
b
0
oui
7
S
M.
e
je
p
T
e
m
L
je
b
b
e
r
r
un
2
r
0
2
je
3
e
s

/
j

.

/

t

F

toi
s
e
r

o
n

1
7

M.
un
oui

2
0
2
1

consistent with the effect stemming from a top–down set
to search for the new items and ignore the old. There is
also evidence that preview search can benefit from active
expectancies developed for the features of the upcoming
search targets (Braithwaite & Humphreys, 2003). For ex-
ample, if the target is in a small set of new items carrying
the same color as the preview, target selection can be
difficult but this effect is reduced if participants know
the color of the upcoming target. Another factor is atten-
tional capture by the onsets that define the new search
items. Donk and Theeuwes (2001) argued that the ben-
efit to preview search was eliminated when the new stim-
uli were not defined by luminance onsets—although
further work has shown that new onsets are not critical
providing there is a sufficiently long time interval be-
tween the preview and search display (Braithwaite,
Humphreys, Watson, & Hulleman, 2005).

Other work highlighting the role of visual onsets in
preview search comes from studies examining how the
temporal relations between the old and new items mod-
ulate preview search. Kunar, Humphreys, and Smith
(2003un) presented preview displays for around 450 msec
and followed them by a brief offset of the items before all
the search display appeared together (the initial items
and the new search stimuli, respectively at old and new
locations). Although the temporal interval between the
old and new items was sufficient to generate temporal
segmentation and to enable the old items to be sup-
pressed, the preview benefit was abolished. Kunar,
Humphreys, and Smith (2003b) argued that the benefit
was eliminated because all the display items grouped by
temporal onset when they reappeared together and that
this disrupted effects of temporal segmentation and/or
the inhibition of old distractors. Fait intéressant, though,
Kunar et al. (2003b) also showed that the reintroduction
of a brief preview before the new search items reinstated
the search benefit, though the brief preview itself was in-

sufficient to generate a gain in search efficiency relative to
a standard conjunction search. Ainsi, the second, brief
preview seemed to reinstate the latent presence of the
old preview, perhaps by breaking up the onset-based
grouping of the old and new items.

Modeling Search Over Space and Time
These studies of the preview benefit in search provide
important constraints for models of how visual attention
operates in time as well as space. One attempt to model
such effects using a biologically plausible framework was
put forward by Mavritsaki, Heinke, Allen, Deco, et
Humphreys (2011). The spiking search over time and
espace (sSoTS) model used biased competition to select
targets. The framework for the original model is set out
in Figure 1. sSoTS employed spiking level neural dynam-
ics, with time constants matching those found in real
neural systems. The model contained initial feature
maps, which we assume exist at intermediate stages of
vision, that are activated by the presence of particular vi-
sual features at particular locations. The maps were com-
posed of both excitatory units and inhibitory units, et
the inhibitory units acted to damp down activity when
there were multiple items containing the same features
(similar to lateral inhibition). These feature maps inter-
acted with a saliency map, which reflected the presence
of any feature at different locations, with the strength of
the activity based on the strength of the sensory signal. Dans
addition, expectancies for given target properties were
set by selectively preactivating feature maps or by selec-
tively activating a location in the saliency map. These
expectancies bias selection to favor either particular fea-
tures or a particular location. Target detection was based
on activity within the saliency map reaching a set thresh-
old when compared with the activity in the locations oc-
cupied by distractors. For this, we calculate an attention

Chiffre 1. The architecture of
sSoTS. Here binding was
implemented in terms of
excitatory inputs into the
saliency map given by
processing units (outside the
model) presumed to respond to
luminance onsets. This then fed
back to activate feature maps
(blue solid connections); le
feedback weight was increased
par 0.2.

1554

Journal des neurosciences cognitives

Volume 28, Nombre 10

D
o
w
n
je
o
un
d
e
d

F
r
o
m

je

je

/

/

/

/
j

F
/

t
t

je
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
je
n
t
o
p
un
r
d
c
e
.
d
s
F
je
r
o
je
m
v
e
h
r
c
p
h
un
d
je
je
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
toi
c
n
o
/
c
un
n
r
un
t
r
je
t
je
c
c
je
e
e
–
p
–
d
p
d
2
F
8
/
1
2
0
8
/
1
1
5
0
5
/
3
1
1
5
9
5
5
3
1
/
5
1
6
0
7
8
o
5
c
3
n
4
_
9
un
/
_
j
0
o
0
c
9
n
8
4
_
un
p
_
d
0
0
b
9
oui
8
g
4
toi
.
e
p
s
t
d
o
F
n
b
0
oui
7
S
M.
e
je
p
T
e
m
L
je
b
b
e
r
r
un
2
r
0
2
je
3
e
s

/
j

/

.

F

t

toi
s
e
r

o
n

1
7

M.
un
oui

2
0
2
1

index (IndA) based on Luce’s (1959, 1977) choice theorem
(for more details, please see Mavritsaki et al., 2011, p. 40).
Targets differing in their features from distractors can
be detected efficiently based on the selective activation
of their feature map and mutual inhibition within the fea-
ture maps for distractors. Selection of a target defined by
a conjunction of features is less efficient, cependant, être-
cause there is no differential inhibition of any of the fea-
ture maps which are mutually activated by targets and
distractors. Mavritsaki, Heinke, Humphreys, and Deco
(2006) showed that the time for a conjunction target to
be detected increased with the number of items compet-
ing for selection and the slope of the search function for
conjunction targets was around twice that for feature tar-
gets, matching data from human search (Treisman &
Gelade, 1980).

One interesting aspect of sSoTS is that the spiking neu-
rons show adaptation after firing, reflecting the build-up
of Ca2+ over time. This means that, within the saliency
map, neurons should reduce in activation after reaching
their threshold, effecting a drop in “attentional interest.”
The sluggish time course of this effect predicts the slow
time course of preview search, where old items lose their
attentional interest more slowly than predicted by tem-
poral segmentation alone ( Watson & Humphreys,
1997). En outre, to model the evidence for top–down
suppression of old items, Mavritsaki et al. (2006) intro-
duced an inhibitory parameter that could be applied to
the locations of items that were to be ignored in search.
Armed with the adaptation and suppression mechanisms,
Mavritsaki et al. (2006) showed that preview benefits
emerged in search for conjunction targets, with the slope
of the search function for preview targets matching that
obtained when only the new items were presented. Le
preview benefits followed a similar time course to that
found in human search. En outre, by convolving activ-
ity in the model with an assumed hemodynamic re-
sponse function, sSoTS was able to simulate fMRI data
on preview search (par exemple., Allen, Humphreys, & Matthieu,
2008), with activity in the posterior parietal cortex corre-
lating with the predicted activity within the saliency map
(Mavritsaki et al., 2011).

In the present article, we ask whether the sSoTS model
is able to capture the finer-grained aspects of the time
course of search, as studied by Kunar et al. (2003un).
Can the effects of introducing a temporal gap between
a preview and the presentation of all the search items dis-
rupt the preview benefit? Note that it is not clear that this
will necessarily be the case, given that the adaptation pro-
cess should be set in play even when items are removed
from the visual field, so a preview benefit may still be pre-
dicted. This was assessed in Part 1 of the article, and we
show that a new temporal binding process needs to be
built into the model to capture the effects of the tempo-
ral gap (the binding-spiking search over time and space
[b-sSoTS] model). In Part 2, we examined how well this
extended model was able to simulate an additional set of

data based on the effects of brain lesions on preview
recherche. Olivers and Humphreys (2004) evaluated the ef-
fects on search of lesions to the posterior parietal cortex.
They found that not only was conjunction search rela-
tively disrupted when compared with feature search
(see also Eglin, Robertson, & Knight, 1989; Riddoch &
Humphreys, 1987) but preview search was also impaired.
Fait intéressant, for targets falling on the contralesional side,
preview search was worse even than search when a tem-
poral gap was introduced between the preview and the
search display. This is the opposite of the result found
in normal participants. Olivers and Humphreys (2004) ar-
gued that the posterior parietal patients had reduced
sensitivity to the onsets of the new items and showed
weaker effects of temporal segmentation. Dans ce cas,
the temporal gap enhanced segmentation without per-
formance being disrupted by the old and new items on
setting together. In Part 2, we evaluated if the b-sSoTS
model could capture this opposite pattern of perfor-
mance after brain lesion. We compare the data patterns
generated by the model with those found in human par-
ticipants and reported in Kunar et al. (2003un) and Olivers
and Humphreys (2004)—focusing on changes in the ef-
fects of display size across the different conditions.

PART 1: SIMULATING NORMAL PERFORMANCE

Experiment 1: sSoTS

Methods

The sSoTS model. The sSoTS model is composed of ex-
citatory spiking neurons that represent pyramidal cells in
the brain and inhibitory spiking neurons that represent
interneurons. There are three layers in the network; dans
each layer, the number of excitatory (pyramidal cells)
and inhibitory (interneurons) neurons follows the ratio
that is usually found in human brain (80:20; Abeles,
1991). Two layers represent the feature maps for the
search asks we used, one representing the shape of the
items and the other the color. The third layer represents
the saliency map, where the units respond to outputs
coming from particular locations in both feature maps.
Each map has inhibitory and excitatory neurons separated
in different pools; the inhibitory neurons provide global in-
hibition to the neurons in the excitatory. Here we can think
that the shape feature map has layers for the H and A stim-
uli and the color map has layers for the colors blue and
vert, matching the items in the experiments (see Olivers
& Humphreys, 2004). For each dimension, there were six
pools that represented the six possible positions that an
item could be presented on the visual field. Each layer in
the maps also has one pool of neurons that add noise into
the system (the “nonspecific” pool). The saliency map re-
ceived forward connections from each feature map and
also had projections back to the feature maps. More details
on the organization of the model can be found in Figure 5
in Mavritsaki et al. (2011).

Mavritsaki and Humphreys

1555

D
o
w
n
je
o
un
d
e
d

F
r
o
m

je

je

/

/

/

/
j

t
t

F
/

je
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
je
n
t
o
p
un
r
d
c
e
.
d
s
F
je
r
o
je
m
v
e
h
r
c
p
h
un
d
je
je
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
toi
c
n
o
/
c
un
n
r
un
t
r
je
t
je
c
c
je
e
e
–
p
–
d
p
d
2
F
8
/
1
2
0
8
/
1
1
5
0
5
/
3
1
1
5
9
5
5
3
1
/
5
1
6
0
7
8
o
5
c
3
n
4
_
9
un
/
_
j
0
o
0
c
9
n
8
4
_
un
p
_
d
0
0
b
9
oui
8
g
4
toi
.
e
p
s
t
d
o
F
n
b
0
oui
7
S
M.
e
je
p
T
e
m
L
je
b
b
e
r
r
un
2
r
0
2
je
3
e
s

/
j

t

/

F

.

toi
s
e
r

o
n

1
7

M.
un
oui

2
0
2
1

The spiking neurons are integrate-and-fire neurons
with excitatory and inhibitory synaptic currents. The sub-
threshold membrane potential follows Equation 1,

(cid:1)

dV tð Þ
dt

¼ 1
Cm

−gm V tð Þ − VL

ð

Þ − Isyn tð Þ

(cid:3)

(1)

where Cm is the membrane capacitance, gm is the mem-
brane leak conductance, VL is the resting potential, et
Isyn is the synaptic currents. The synaptic currents we have
in sSoTS are composed of a fast excitatory AMPA current
(IAMPA,rec), a slow excitatory NMDA current (INMDA,rec),
an external AMPA current (IAMPA,ext), and an inhibitory
GABAergic current (IGABA). En outre, the model includes
a frequency adaptation current based on [Ca2+] sensitive
K+ current (IAHP). The synaptic currents that we have in
the model are given in Equation 2,

Isyn tð Þ ¼ IAMPA;ext tð Þ þ IAMPA;rec tð Þ þ INMDA;rec tð Þ

þ IGABA tð Þ þ IAHP tð Þ

(2)

The sSoTS model is composed of 5000 neurons distrib-
uted across the three layers. Because of limited compu-
tational power, it would have been impossible to be able
to identify the model’s parameter with so many equa-
tions to solve for each neuron. To solve this problem, un
“meanfield approach” was employed (Mavritsaki et al.,
2011; Deco & Rolls, 2005; Brunel & Wang, 2001). Dans ce
approche, the activation of groups of neurons is repre-
sented by a transfer function based on a number of ap-
proximations (these approximations can be found in
Figure 1B). The mean-field approach reduces signifi-
cantly the computational power needed to identify the
model’s parameters, because there is no longer a need
to solve so many equations for each neuron but just
one for a group of neurons.

The basic network parameters (par exemple., the weight from
the inhibitory pool to the feature dimension pools) étaient
identified using the mean-field method, and then the re-
maining parameters were set by hand in the spiking level.
These are set out in Table 1.

more details on the current used can be found in Figure 1A
in Mavritsaki et al. (2011).

Search conditions. The model was set to simulate three
search tasks: single feature search (blue H target vs. blue

Tableau 1. Parameters that Were Used in the Model

Parameter

gAMPA,rec excitatory

gAMPA,rec inhibitory

gNMDA excitatory
un

NE

NI

Suivant
w+

wi1

wi2

wi3

wi4
λin

λatt

maxAc

wbind

thrAc

Values

0.0208 nS

0.0162 nS

0.22 nS

0.18 μΜ

Description

AMPA recurrent synaptic conductance for excitatory neurons

AMPA recurrent synaptic conductance for inhibitory neurons

NMDA recurrent synaptic conductance for excitatory neurons

[Ca2+] influx when a spike occurs

1600 (800)

Number of excitatory neurons in each layer for the feature

maps (for the location map)

400 (200)

Number of inhibitory neurons in each layer for the feature

800

2.4

1.0

0.9

1.0

0.25

150 Hz

190 Hz

0.18

0.2

5 Hz

maps (for the location map)

Number of external neurons

Coupling for the pools in the feature maps

Inhibition for the two feature dimension maps

Inhibition for the location map

Connection weight from feature maps to location map

Connection weight from the location map to feature maps

The total input that each pool receives from the external neurons

to show that there is an item in the visual field.

The total top–down that the target pools receive to signify the

target’s characteristics.

The maximum Top Down Inhibition that can be applied to the

previewed distractors’ maps.

Binding Parameter, increase in the feed-backward weight from

LM to FMs due to grouping

Threshold for the pool in feature map being active

1556

Journal des neurosciences cognitives

Volume 28, Nombre 10

D
o
w
n
je
o
un
d
e
d

F
r
o
m

je

je

/

/

/

/
j

t
t

F
/

je
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
je
n
t
o
p
un
r
d
c
e
.
d
s
F
je
r
o
je
m
v
e
h
r
c
p
h
un
d
je
je
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
toi
c
n
o
/
c
un
n
r
un
t
r
je
t
je
c
c
je
e
e
–
p
–
d
p
d
2
F
8
/
1
2
0
8
/
1
1
5
0
5
/
3
1
1
5
9
5
5
3
1
/
5
1
6
0
7
8
o
5
c
3
n
4
_
9
un
/
_
j
0
o
0
c
9
n
8
4
_
un
p
_
d
0
0
b
9
oui
8
g
4
toi
.
e
p
s
t
d
o
F
n
b
0
oui
7
S
M.
e
je
p
T
e
m
L
je
b
b
e
r
r
un
2
r
0
2
je
3
e
s

/
j

t

/

.

F

toi
s
e
r

o
n

1
7

M.
un
oui

2
0
2
1

A distractors), conjunction search (blue H target vs.
green H and blue A distractors), and preview search (pre-
voir: green H distractors followed by blue A distractors
and blue H target). There were two display sizes (4 et
6 items), and the target was positioned equally often in
the six potential locations. There were also four preview
conditions: (1) the preview was presented for 450 msec
and remained in place when the new search items ap-
peared (standard preview); (2) the preview was presented
pour 450 msec and then removed for 450 msec before
being presented again at the same locations with the
new search items (preview gap); (3) the preview was pre-
sented for 300 msec and remained when the new items
appeared (short preview); (4) the preview was presented
pour 450 msec, removed for 450 msec, and then re-presented
pour 300 msec, and it remained in place when the search
items appeared (the “top-up” preview condition; see Kunar
et coll., 2003un). In our prior work, we showed that the pre-
view needed to be represented for at least 450 msec to gen-
erate a preview benefit in search (Mavritsaki et al., 2011).
The 300-msec preview should thus be too short to fully
establish a preview benefit. The question then is whether
the effects of this short preview may be enhanced from
the earlier memory of the preview, when presented for
450 msec. En outre, we assessed if the preview benefit
was lost if a temporal gap was inserted before the pre-
sentation of the new search items. Il y avait 100 runs
of each simulation (note that the results will vary as a
function of the noise in the model), and from these simula-
tion, we randomly created groups of 20 runs that were
grouped to form a single “participant” for the data analysis.

Results and Discussion

The results were analyzed using both the RTs and a re-
sponse efficiency index based on the mean RT/accuracy
(see Townsend & Ashby, 1983) to take the accuracy of
response into account alongside response latencies.
The basic pattern of the results did not change funda-
mentally as a function of the measure used.

The model was run in four conditions: single feature
(blue H target vs. blue A distractors), conjunction (blue
H target vs. blue A and green H distractors), standard
preview (450 msec green H followed by blue H target
and blue A distractors), and preview gap (450 msec green
H, 450 msec interval, then blue H target vs. blue A and
green H distractors). The results were analyzed by con-
trasting the RTs for the conditions of interest. In all but
the few cases noted below, the efficiency data followed
the RTs. Where the efficiency data did not (par exemple., due
to the error data contrasting with the RT effects), nous
report the results for efficiency alongside those for RTs.
The figures show the efficiency data to make data report-
ing compact (combining RTs and accuracy).

Single feature versus conjunction. RTs were faster for
the single feature condition (F(1, 9) = 269.04, p < .001), and there was a smaller effect of display size for the single feature versus the conjunction condition (F(1, 9) = 13.98, p < .005). For the RTs, the slopes of the functions were 17.96 and 57.03 msec/item, respectively. Standard preview versus single feature. RTs were over- all slower for the standard preview condition (F(1, 9) = 63.07, p < .001), but the conditions did not differ as a function of the display size (F(1, 9) = 2.02, p = .189, for the interaction of condition and display size). The slope of the search function for RTs was 21.56 msec/item for the standard preview condition. D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j f . / t u s e r o n 1 7 M a y 2 0 2 1 Standard preview versus conjunction. There was an advantage for the standard preview over the conjunction condition, both in terms of overall RTs and the slopes of the search functions (F(1, 9) = 356.17 and 14.01, p < .001 and p < .005, respectively). Preview gap versus single feature. There was an overall RT advantage for the single feature condition (F(1, 9) = 65.40, p < .001), but no difference in terms of the search slopes (F < 1.0). The search slope for RTs in the preview gap condition was 34.56 msec/item. Preview gap versus conjunction. There was a benefit for the preview gap condition both in terms of overall RTs and in terms of the slopes of the search functions (F(1, 9) = 423.0 and 18.29, p < .001 and .005, respectively). Preview gap versus standard preview. There was a small advantage in terms of overall RTs for the preview gap condition and a trend also for search to be more ef- ficient in the preview condition (F(1, 9) = 7.02, p < .05, and F(1, 9) = 4.62, p > .05).

The simulations show that the sSoTS model was able to
simulate human visual search for single feature, conjunc-
tion, and preview targets, replicating prior data (Mavritsaki
et coll., 2011). This is important because it demonstrates
comment, in a model using biologically plausible parameters
and spatially parallel processing, both efficient and ineffi-
cient search patterns can be captured, based on differen-
tial competition for visual selection. The conjunction
condition is more difficult than the single feature condi-
tion because the green H distractors compete for selection
along with the blue A distractors. In the preview condition,
the competition from the green H distractor is reduced
when the interval between the items is sufficiently long
for adaptation to occur and also for top–down inhibition
to be applied. The result is that the slope of the search
function is matched to the single feature baseline, même
though overall RTs are slower. This matches data from
preview search in humans (Watson & Humphreys, 1997).
The new result here is in the preview gap condition,
when a temporal interval is introduced between the pre-
view and the search display. In the model, performance
was slightly but not dramatically disrupted in the gap

Mavritsaki and Humphreys

1557

condition compared with the single feature and standard
preview conditions, and the slopes of the search func-
tions did not differ across these conditions. This occurred
because the processing units remained adapted and
because top–down inhibition continued to be applied so
that previewed distractors were effectively ignored. Given
that in the human studies the conditions are typically
blocked so that participants know that the previewed
items would remain in place and still be irrelevant when
the search display appeared, there seems no reason to
think that top–down inhibition would stop being applied.
There is also evidence that this inhibition is not reset
simply by the old items offsetting. Kunar et al. (2003b)
examined preview search when previewed items were
briefly occluded before appearing again with the new
search items. A preview benefit was found, despite the
previews offsetting and then onsetting again with the
new items. Offsetting the previews is not sufficient to
eliminate the benefit to preview search. The difficulty
ici, cependant, is that the result in the gap condition does
not match that found in human search, where the tempo-
ral gap was sufficient to disrupt preview search (et en-
deed push search in this condition back to the conjunction
baseline). sSoTS, as originally construed, fails to simulate
this aspect of human performance.

Experiment 2: b-sSoTS

In a second version of the sSoTS model (b-sSoTS), nous
incorporated a mechanism for binding together visual el-
ements with common onset time signals. The proposal
here is that early visual areas can register onsets, sending
excitatory signals forward to higher-level feature and sa-
liency maps. The parameter for this is given in Table 1.
The excitation from two simultaneously activated loca-
tions in different feature map further excites the saliency
map, making new onsets salient, by increasing their con-
nection through the increase in the feed-backward

weight. This is highlighted in Figure 1 by the increased
line width between the two activated pools in the blue
and A maps and the corresponding position in the saliency
map. Applied here it means that, under standard preview
conditions, there is enhanced activation from the onsets
of the new items, further biasing search against the old
(previewed) stimuli. Under the preview gap condition,
when the old items are re-presented, there are new onsets
for the old as well as the new stimuli. The extra excitatory
inputs from the onsets then acts against the adaptation
and top–down inhibition effects, so that the preview ad-
vantage should decrease.

Method

Unless otherwise mentioned, the conditions and param-
eters exactly matched those for the sSoTS model, avec
the single difference being that we introduced temporal
binding between the feature maps and the saliency map.
The binding parameter (wbind, please see Table 1) était
fine-tuned to simulate the behavioral results for single
feature, conjunction, standard preview, short preview,
preview gap, and “top-up” preview conditions.

Results and Discussion

The mean RT/accuracy data for the single feature, con-
junction, standard preview, and preview gap conditions
when binding is used in the model are presented in
Figures 2 et 3. The data for the short preview and
top-up preview conditions are shown in Figure 4, along
with the single feature and conjunction baselines.

We first present the contrasts between the conditions
illustrated in Figure 3 before presenting those relating to
the conditions in Figure 4.

Single feature versus conjunction. RTs were faster for
the single feature condition (F(1, 9) = 348.63, p < .001), D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j / t . f u s e r o n 1 7 M a y 2 0 2 1 Figure 2. The mean efficiency index (RT/accuracy) across the two display sizes for the single feature (SF), conjunction (CJ), standard preview (450 msec preview), and preview gap conditions (450 msec preview + 450 msec interval). 1558 Journal of Cognitive Neuroscience Volume 28, Number 10 dard preview conditions (F(1, 9) = 3.77, p > .05), mais
there was an interaction of conditions and display size
(F(1, 9) = 6.97, p = .03).

The data indicate that, when temporal binding was in-
troduced into sSoTS, the conditions more closely
matched those found in human search (Kunar et al.,
2003un). The single feature and standard preview search
conditions differed in overall RTs but not in terms of
search efficiency, and both were more efficient than
the conjunction baseline. This agrees with human search
data, where single feature and preview search conditions,
though they differ in RTs, are both more efficient than
the conjunction baseline (Watson & Humphreys, 1997).
The introduction of temporal binding did not disrupt the
advantages from reduced competition either when one
set of distractors was omitted (the single feature base-
line) or when distractors were suppressed (by adaptation
and top–down suppression, in the preview condition).
Cependant, there was a disruption to search in the preview
gap condition, when an interval was introduced between
the preview and the search display and all the items in
the search display onset together. Although there was
no difference in overall RTs, there was a big difference
in the slopes; the slope for standard preview was
13.23 msec/item, and the slope for the preview gap con-
dition was 60.93 msec/item. This result comes about
because re-presenting all the search items again, after
the interval, creates a set of common new onsets, coun-
teracting effects of adaptation and top–down inhibition,
which otherwise bias search against the old items. Le
preview benefit decreases. Chiffre 5 shows activation in
the original sSoTS and in b-sSoTS in the preview gap con-
dition. Note that the gain in activity for the target location

D
o
w
n
je
o
un
d
e
d

F
r
o
m

je

je

/

/

/

/
j

t
t

F
/

je
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
je
n
t
o
p
un
r
d
c
e
.
d
s
F
je
r
o
je
m
v
e
h
r
c
p
h
un
d
je
je
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
toi
c
n
o
/
c
un
n
r
un
t
r
je
t
je
c
c
je
e
e
–
p
–
d
p
d
2
F
8
/
1
2
0
8
/
1
1
5
0
5
/
3
1
1
5
9
5
5
3
1
/
5
1
6
0
7
8
o
5
c
3
n
4
_
9
un
/
_
j
0
o
0
c
9
n
8
4
_
un
p
_
d
0
0
b
9
oui
8
g
4
toi
.
e
p
s
t
d
o
F
n
b
0
oui
7
S
M.
e
je
p
T
e
m
L
je
b
b
e
r
r
un
2
r
0
2
je
3
e
s

/
j

/

.

t

F

toi
s
e
r

o
n

1
7

M.
un
oui

2
0
2
1

Chiffre 4. The mean efficiency index (RT/accuracy) for the single
feature (SF), conjunction (CJ), standard preview (PV 450 msec), short
preview (PV 300 msec), and the top-up preview conditions (PV 450 +
gap 450 + PV 300 msec).

Mavritsaki and Humphreys

1559

Chiffre 3. The mean efficiency index (RT/accuracy) for the single
feature (SF), conjunction (CJ), standard preview (PV), and preview gap
conditions (PV 450 + gap 450 msec).

and there was also an effect of display size for the single
feature versus the conjunction condition (F(1, 9) =
740.018, p < .001). The slopes of the RT functions were 17.87 and 65.29 msec/item, respectively. Standard preview versus single feature. RTs were over- all slower for the standard preview condition (F(1, 9) = 120.45, p < .001), but the conditions were not differen- tially affected by the display sizes (F(1, 9) = 4.45, p > .05,
for the interaction of condition and display size). Le
slope of the search function was 13.23 msec/item for
the standard preview condition.

Standard preview versus conjunction. There was an
advantage for the standard preview over the conjunction
condition, both in terms of overall RTs and the slopes of
the search functions (F(1, 9) = 390.57 et 389.66, les deux
p < .001, respectively). Preview gap versus single feature. There was an overall RT advantage for the single feature condition (F(1, 9) = 109.76, p < .001) but no difference in terms of the search slopes (F < 1.0). However, in this case there was a confound with search accuracy. For the efficiency measure (RT/ accuracy), there was a clear interaction of Condition × Display size (F(1, 9) = 37.90, p < .001). The search slope for the preview gap condition was 60.93 msec/item and greater than for the single feature baseline. Preview gap versus conjunction. There was a benefit for the preview gap condition both in terms of overall RTs and in terms of the slopes of the search functions (F(1, 9) = 454.68 and 416.44, p < .001, respectively). Preview gap versus standard preview. There was no overall difference between the preview gap and the stan- D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j t f / . u s e r o n 1 7 M a y 2 0 2 1 Figure 6. Activity in the saliency map for the three additional preview conditions, preview (A), short preview (B), and top-up preview (C); target is always red, and distractors are the other colors. The thin line shows activation without binding, and the thick line shows activation with binding. Binding fits better with the expected differences based on the behavioral data. Figure 5. Activity in the saliency map in the preview gap conditions for models without binding (thin line) and models with temporal binding (thick line); target is always red, and distractors are the other colors. Competition from the new distractors is increased when there is temporal binding, although for four items preview gap the target activation line for binding comes slightly earlier than the nonbinding and for 6 items preview gap the target activation line comes later. is slowed in b-sSoTS, reflecting the greater competition for selection when the old items are reactivated by their onsetting together. Figure 6 shows the activation for sSoTS and b-sSoTS for the preview, “short preview,” and “top-up preview” conditions. In a second set of contrasts, we examined the perfor- mance of b-sSoTS when the interval between the preview and the search display was reduced (short preview) and when the short preview was “topped up” by the earlier presentation of the preview gap conditions (top-up pre- view). Search in these conditions was compared with the single feature and conjunction baselines. In human search, the short preview leads to worse search performance rela- tive to when the preview is displayed for longer (Watson & Humphreys, 1997), and the top-up preview then reinstates the preview benefit (Kunar et al., 2003a). Short preview versus single feature. RTs were overall slower for the short preview condition (F(1, 9) = 1283.67, p < .001), and search was also less efficient (F(1, 9) = 102.54, p < .001, for the interaction of condition and display size). The slope of the search function was 41.74 msec/item for the short preview condition. Short preview versus conjunction. There was an advan- tage for the short preview over the conjunction condition, both in terms of overall RTs and the slopes of the search functions (F(1, 9) = 28.87 and 30.99, p < .01, respectively). Top-up preview versus single feature. There was an overall RT advantage for the single feature condition 1560 Journal of Cognitive Neuroscience Volume 28, Number 10 (F(1, 9) = 50.11, p < .001) and a small difference in terms of search slopes (F(1, 9) = 5.14, p = .05, p > .05). Ce
slope difference was eradicated when the efficiency
measure (RT/accuracy) was employed to correct for
speed–error trade-offs (F(1, 9) = 1.79, p = .213). Le
search slope for the top-up preview condition was
13.88 msec/item.

22 neurons from each pool from one part side of the sa-
liency map. These results do not directly mimic the per-
centage of lesioned regions found in studies of human
neuropsychological patients, but they provide a more
general reflection of the effects that can be expected as
lesion size varies. The data were compared qualitatively
with results from Olivers and Humphreys (2004).

Top-up preview versus conjunction. There was a bene-
fit for the preview gap condition both in terms of overall RTs
and in terms of the slopes of the search functions (F(1, 9) =
723.3 et 411.24, p < .001, respectively). Top-up preview versus short preview. There was an advantage in terms of overall RTs and search slopes for the top-up preview versus the short preview condition (F(1, 9) = 814.00 and 66.77, both p < .001). The results indicate that, although there was some advantage for the short preview condition over the con- junction baseline, there was then a cost relative to the single feature baseline in terms of search slopes; this is also demonstrated in human search data (Kunar et al., 2003a). In this case, the preview duration was insufficient for the full effects of adaptation and top–down inhibition to modulate search (see Watson & Humphreys, 1997, for data on human search). However, the effects of the short preview could be enhanced by the earlier presentation of stimuli equivalent to the preview gap conditions. Here the “trace”of the initial preview (generating adaptation and top–down suppression) could enhance the effects generated by the short preview under conditions in which the preview items did not onset with the new dis- tractors and target—that is, under conditions in which temporal binding across all the display items was elimi- nated. The results provide an existence proof that tempo- ral binding may place a significant part in visual search over time and matches data reported by Kunar et al. (2003a). PART 2: NEUROPSYCHOLOGICAL PERFORMANCE Lesioning b-sSoTS The lesioning applied in sSoTSb is an extended approach of the lesioning method that was applied in earlier stud- ies of the sSoTS model (Mavritsaki, Heinke, Deco, & Humphreys, 2009). Lesioning was implemented by re- moving at random a number of neurons from one side of the saliency map (to simulate a unilateral brain lesion; note that, with this procedure, the lesion could be un- equally distributed across the different pools). The mag- nitude of the lesion was varied, from 18.33% of the total number of neurons in each pool removed to 22.78% of the units removed, to simulate different magnitudes of brain lesion. The percentages used were calculated on the basis of the number of units removed from each side, for example, 18.33% means that we removed in average Method The simulations were the same as those presented in Part 1, except that different numbers of processing units were removed from one side of the saliency map (uni- lateral lesions). Lesioning was only applied here to the b-sSoTS model, given that this model better accounted for the time course of normal search than the original sSoTS model. The search conditions were (i) single feature, (ii) conjunction, (iii) standard preview (450 msec preview), and (iv) preview gap (450 msec preview and 450 temporal interval), matching the conditions reported in Olivers and Humphreys (2004). Results and Discussion The results are presented in Table 2, where we compare the data for each lesioned case against 10 batches where the model was unlesioned. The data reveal that, in all cases, there were costs to performance after lesioning, partic- ularly for targets falling on the contralesional side of a unilateral lesion (e.g., compared with when targets fell on the ipsilesional side) and particularly in the conjunction and standard preview conditions (please see Figure 7A for contralesional and Figure 7B for ipsilesional). Essentially unilateral lesions produce a spatial imbalance, which means that stimuli on the ipsilesional side dominate the competition for selection. In addition, the lesioning of the saliency map means that the model is less sensitive to the temporal differences between the old and new items on the contralesional side, showing weaker adap- tation and suppression of old stimuli. The net effect is that the preview condition is disrupted alongside the conjunc- tion condition. The patterns of activity generating these results are shown in Figure 8A and B. We also contrasted performance between the standard preview and preview gap conditions for the “control” runs of the model and for the lesioned versions. For this, we took the difference between the slopes of these con- ditions for the controls and compared it with the differ- ence in the slopes of the conditions for contralesional and then ipsilesional targets when the model was le- sioned using a modified t test to compare individual cases against a group (Crawford & Howell, 1998). This was done both for the smallest (18.33) and largest lesions (22.78) imposed on the model to test for generalization of the effects across the different lesion sizes. For con- tralesional targets, the contrast between the conditions for the lesioned model (where the slope for preview Mavritsaki and Humphreys 1561 D o w n l o a d e d f r o m l l / / / / j t t f / i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j f t . / u s e r o n 1 7 M a y 2 0 2 1 Table 2. Significance Test (Crawford & Howell, 1998) on Differences between Control Based on 10 Samples and Individual Test Based on One Lesion Level Table 2. (continued ) Statistical Results for SF Efficiency Slope, Contralesional and Ipsilesional Statistical Results for PV 450 msec Efficiency Slope, Contralesional and Ipsilesional Lesion Level Contralesional p t Ipsilesional p t 0.1833 0.1944 0.2056 0.2278 3.149 4.44 14.00 10.88 .012 0.11 .914 .0 .0 .0 −0.53 0.46 0 .60 .65 1 Statistical Results for CJ Efficiency Slope, Contralesional and Ipsilesional Lesion Level Contralesional p t Ipsilesional p t 0.1833 0.1944 0.2056 0.2278 16.45 17.56 17.59 25.52 .0 .0 .0 .0 −1.74 0.44 −2.43 −1.44 .11 .66 .03 .18 Statistical Results for PV 450 msec Followed by GAP 450 msec Efficiency Slope, Contralesional and Ipsilesional Lesion Level Contralesional p t 0.1833 0.1944 0.2056 0.2278 15.14 10.44 12.76 18.8 .0 .0 .0 .0 Ipsilesional p t −2.73 −2.56 −3.4 −0.903 .023 .03 .008 .39 Lesion Level 0.1833 0.1944 0.2056 0.2278 Contralesional p t −1.21 Ipsilesional p t −1.685 .25 .65 .086 .019 0.46 1.95 2.854 .12 .069 .38 .045 −2.06 −0.92 −2.32 On the basis of Bonferroni correction, the significance level is p < .0125. 1562 Journal of Cognitive Neuroscience Volume 28, Number 10 D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j . / t f u s e r o n 1 7 M a y 2 0 2 1 standard preview relative to the temporal gap condition when targets fell in the contralesional field. This cost re- flects the poor temporal segmentation of the preview and new search items in the contralesional field, under the standard preview condition. In this case, temporal seg- mentation can be aided by the time interval introduced between the preview and the new search items, giving a longer time for the weak adaptation effect to take place. In addition to this, stimuli at lesioned locations in the model show weaker temporal binding, due to the general reduction in activation on the lesioned side. The net re- sult is that there is less of a cost to performance for con- tralesional targets in the preview gap condition. A double dissociation emerges in which, for controls and for ipsile- sional targets in the lesioned model, the standard preview search is better than search under the preview gap condi- tion; in contrast, for contralesional targets in the lesioned model, the opposite occurs. This reversal of the standard pattern of performance matches that found in posterior parietal patients by Olivers and Humphreys (2004). GENERAL DISCUSSION Modeling Normal Search Data In this article, we examined whether a model using bio- logically plausible activation functions could capture the details of the time course of visual search under preview conditions ( Watson & Humphreys, 1997). We focused on the patterns of results reported when temporal intervals are introduced between preview and search displays. Kunar et al. (2003a) reported two results of primary in- terest. First, introducing a temporal gap after a preview display and re-presenting the old and new stimuli together (in a preview gap condition), severely disrupted search (compared with a standard preview condition in which there was no gap). Second, when a preview is too short to generate a full benefit to performance, a benefit can be reintroduced by “topping up” the short preview (by having the preview gap condition precede the short pre- view). Kunar et al. (2003a) account for the results by sug- gesting that search was affected by temporal binding between the preview and new items, when these items appear together (in the preview gap condition but not in the top-up preview). This binding brings the old items into competition for selection, disrupting search in the preview gap but not in the top-up condition. In Part 1 here, we contrasted simulations run with the sSoTS neural network model with a new variant of that model in which temporal binding was introduced (the b-sSoTS model). Binding took place when there was new input activity given to the feature units, which triggered excitatory activation between these units and units in the saliency map (Figure 1). The original sSoTS model made the incorrect prediction that search should remain efficient in the preview gap condition, because both adaptation following the firing of the saliency neurons and top–down Mavritsaki and Humphreys 1563 Figure 7. The difference in the efficiency index (RT/accuracy) across the two display sizes for the single feature (SF), conjunction (CJ), standard preview (PV 450 msec), and preview gap conditions (PV 450, gap 450 msec). (A) For contralesional targets and (B) for ipsilesional targets. The baseline = no lesion and the other conditions reflect the degree of lesioning. gap < standard preview) differed relative to that found for the control (nonlesioned) version (where standard preview slope < preview gap slope) (t(9) = 6.96 and 6.48, p < .001, for lesions 18.33 and 22.78, respectively). For ipsilesional targets, there were no differences relative to control versions of the model (t < 1.0 and t(9) = 1.01, p > .05 for lesions 18.33 et 22.78, respectivement). Ici
slopes for standard preview < preview gap. The results demonstrate contrasting patterns of performance in these two preview conditions before and after lesioning. The changes in preview search are of considerable in- terest. For ipsilesional targets, the pattern of performance matched that found with nonlesioned “control” simula- tions. There was a benefit to performance in the standard preview condition (relative to the temporal gap condi- tion) due to the suppression of the old, ipsilesional items and the binding of old to new items in the temporal gap case. In contrast, there was a cost to performance for the D o w n l o a d e d f r o m l l / / / / j t t f / i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j t f / . u s e r o n 1 7 M a y 2 0 2 1 Figure 8. Activity in the saliency map for the single feature (SF), conjunction (CJ), standard preview (PV), and preview gap (PV GAP) conditions. The left side shows the contralesional results, and the right side shows the ipsilesional results. The dotted thin line shows the preview for four items, and the dashed thick line shows the preview for six items. Only target position is shown here. D o w n l o a d e d f r o m l l / / / / j t t f / i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j f / t . u s e r o n 1 7 M a y 2 0 2 1 inhibition of the old stimuli should still operate across the temporal gap. In contrast, the b-sSoTS model did correctly predict that the preview gap condition should disrupt per- formance, because binding triggered by the common on- sets of the old and new items worked against the effects of adaptation and top–down suppression, once again introducing competition from distractors for selection. The b-sSoTS model was also successful in accounting for the relations between a short preview and top-up pre- view conditions (when the short preview was preceded by the preview gap conditions). When the preview is too short, search is not maximally efficient because insuf- ficient time has passed for the full effects of adaptation and top–down suppression to take effect in the model. However, the initial presentation of the preview gap con- dition means that there is lingering adaptation and top– down suppression in the model, provided that it is not offset by temporal binding between the old and new items. The top-up condition prevents this temporal bind- ing because the reintroduction of a short preview means that the old and new items do not bind. The model matches the data reported by Kunar et al. (2003a). 1564 Journal of Cognitive Neuroscience Volume 28, Number 10 Modeling Neuropsychological Data In Part 2, we applied the b-sSoTS model to data reported by Olivers and Humphreys (2004) on the effects of pos- terior parietal damage on preview search. Olivers and Humphreys (2004) found that unilateral parietal lesions disrupted preview as well as conjunction search, par- ticularly when targets fell on the side of space contra to the lesion. Moreover, although the patients showed the normal pattern of performance in which there was some- what better search for ipsilesional targets in the standard preview relative to the preview gap condition, the oppo- site occurred with contralesional targets. b-sSoTS was lesioned by the random removal of pro- cessing units on one side of the saliency map. This selec- tively disrupted conjunction compared with single feature search, a standard finding in the neuropsychological liter- ature (Eglin et al., 1989; Riddoch & Humphreys, 1987). In addition, there was disruption of search for contralesional targets under preview conditions, whereas performance benefitted from a temporal gap introduced between the preview and the search display (even though both the old and new items then onset together). The model pro- vides one explicit account of these results. Damage to units on one side of the saliency map in the model dis- rupts the dynamics of processing, weakening temporal adaptation, reducing top–down inhibition, and also tem- poral binding when items onset together. On the one hand, the disruption to adaptation and top–down inhibi- tion makes it more difficult to generate a benefit to search under standard preview conditions. On the other hand, the introduction of a temporal gap (and a longer time in- terval between the preview and the search display) en- ables even weak adaptation to have some effect, and the reduction in temporal binding means that the advantage from the initial preview is not overridden by the re-presented old items binding with the new search stimuli. The coun- terintuitive pattern of results reported for patients by Olivers and Humphreys (2004) were simulated. Links to Other Findings The temporal binding we have introduced to sSoTS can be thought similar to the notion of binding by temporal synchrony, which has received some neurophysiological and psychological support (Elliott & Muller, 2004; Usher & Donnelly, 1998; Singer & Gray, 1995). The idea here is that common temporal onsets trigger reciprocal activity between neurons, binding the activity that the neurons are coding. In this prior work, temporal binding has been based around neurons having learned associations— perhaps reflecting the statistics of the visual world (basic Gestalt grouping). Other researchers have additionally demonstrated that there is rapid and relatively effortless binding of color and shape when objects have diagnostic colors (e.g., yellow banana, orange carrot), so that targets defined by these colors “pop out” in visual search (but not when the targets carry unfamiliar colors; Wildegger, Riddoch, & Humphreys, 2015; Rappaport, Humphreys, & Riddoch, 2013). In accounting for such results, the authors argue that learning establishes temporal links between dif- ferent attributes of stimuli, so the common occurrence of the attributes generates stronger neural activity. On the other hand, the present results are built around the idea that there is temporal binding based on common onsets even between stimuli that are not grouped by common Gestalt properties or stored knowledge. Some neuropsy- chological evidence for grouping by common onsets was presented by Humphreys, Riddoch, Nys, and Heinke (2002). These authors study the phenomenon of “anti- extinction” whereby patients can be better at reporting two stimuli relative to their report of a single stimulus. Humphreys et al. proposed that antiextinction reflected grouping of two items by common onset, which then en- abled patients to select both items together—similar to when other forms of grouping have been shown to posi- tively modulate extinction (e.g., Humphreys, 1998; Ward, Goodrich, & Driver, 1994). Consistent with the argument for grouping by common onset, Humphreys et al. (2002) reported that there were no benefits for two-item trials when one item was defined by an onset and the other by an offset. In our simulations, the mere onset of two stimuli together was sufficient to enable them to be bound and to modulate subsequent processing. The account we present is in terms of preview search being dependent on a number of mechanisms. Other ac- counts stress the role of single mechanisms—for exam- ple, the role of onset capture by the new search items (Donk & Theeuwes, 2001). The current simulations are built on the idea that onsets do play a role in preview search, and grouping by common onset is a critical factor in capturing results on the timing of preview search (and why a temporal gap between the preview and search dis- plays disrupts performance). On the other hand, behavioral evidence for a role of top–down suppression (Humphreys et al., 2004) and evidence for a restoration of efficient search when a short preview is “topped up” by earlier its presenta- tion (Kunar et al., 2003a) provide clear support for the in- volvement of additional mechanisms, such as stimulus adaptation and active suppression; these additional mecha- nisms are incorporated here, enabling the model to simulate a wide body of data. The results on the effects of lesioning the saliency map also fit with neuropsychological studies. For example, unilateral neglect, a clinical deficit often linked to damage to posterior parietal cortex, is known to be linked not only to impaired spatial processing but also to slow tem- poral selection of targets (e.g., an increased attentional blink; Husain, Shapiro, Martin, & Kennard, 1997). Battelli and colleagues (Battellil, Walsh, Pascual-Leone, & Cavanagh, 2008; Battelli, Pascual-Leone, & Cavanagh, 2007) have reported that posterior parietal lesions dis- rupt the ability to segment between onsets and offsets, and Roberts, Lau, Chechlacz, and Humphreys (2012) Mavritsaki and Humphreys 1565 D o w n l o a d e d f r o m l l / / / / j t t f / i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j / f . t u s e r o n 1 7 M a y 2 0 2 1 used lesion symptom mapping to demonstrate a relation between posterior parietal damage and reduced ability to discriminate the temporal order of events. More recently, Howard, Chechlacz, and Humphreys (2015) documented that posterior parietal patients had an impaired ability to update visual stimuli as they changed over time. Our sim- ulations indicate that the effects of damage to the salien- cy map are to disrupt the dynamics of visual selection so that adaptation takes more time; there are reduced effects of top–down inhibition and also a reduced impact of new onsets (e.g., in the preview gap condition). According to the b-sSoTS model, visual selection is based on a dynamic saliency map where activity levels over time determine the efficiency of selection. Damage to the map affects tempo- ral as well as spatial aspects of selection. Although relevant neuropsychological data have yet to be reported, the current simulations indicate that the dis- ruption to search should increase as lesion size increases. This is perhaps not surprising but should be assessed. In addition, recent neuroanatomical studies have proposed that impaired attention after posterior parietal damage may reflect the disconnection of fiber tracts between parietal and frontal cortices, rather than the loss of pa- rietal gray matter per se (e.g., Lunven et al., 2015). Here we simulated the existing neuropsychological data (Olivers & Humphreys, 2004) by removing processing units— essentially an analog of gray matter lesioning. Whether sim- ilar patterns of data arise after disconnecting different parts of the model is a further question for future research. A final point concerns the exact relations between the present simulations and the human search data. Our stress here has been on qualitative similarities between the data sets—for example, whether there are changes in the slopes of the search functions across different conditions. One of the interesting aspects of b-sSoTS is that it attempts to mirror biological processing systems more directly than ap- proaches using more traditional, connectionist architec- tures, but there remain many approximations that mean that qualitative matches are more important than matching effects of the exact timing between stimuli or the exact slopes of the search functions. Here we show that the qual- itative patterns of human search over time can be captured, providing an existence proof that dynamic processes of binding, neural adaption, and top–down suppression are sufficient to account for human search. Acknowledgments This work was supported by a grant from the European Re- search Council (PePe 333833). Reprint requests should be sent to Eirini Mavritsaki, Depart- ment of Psychology, Birmingham City University, The Curzon Building, 4 Cardigan Street, Birmingham B4 7BD, UK, or via e-mail: eirini.mavritsaki@bcu.ac.uk or Glyn Humphreys, Department of Experimental Psychology, Oxford University, South Parks Road, Oxford OX1 3UD, UK, or via e-mail: glyn. humphreys@psy.ox.ac.uk. REFERENCES Abeles, M. (1991). Corticonics: Neural circuits of the cerebral cortex. New York: Cambridge University Press. Allen, H. A., Humphreys, G. W., & Matthews, P. M. (2008). A neural marker of content-specific active ignoring. Journal of Experimental Psychology: Human Perception and Performance, 34, 286–297. Battelli, L., Pascual-Leone, A., & Cavanagh, P. (2007). The “when” pathway of the right parietal lobe. Trends in Cognitive Sciences, 11, 204–210. Battellil, L., Walsh, V., Pascual-Leone, A., & Cavanagh, P. (2008). The ‘when’ parietal pathway explored by lesion studies. Current Opinion in Neurobiology, 18, 120–126. Braithwaite, J. J., & Humphreys, G. W. (2003). Inhibition and anticipation in visual search: Evidence from effects of color foreknowledge on preview search. Perception & Psychophysics, 65, 213–237. Braithwaite, J. J., Humphreys, G. W., Watson, D. G., & Hulleman, J. (2005). Revisiting preview search at isoluminance: New onsets are not necessary for the preview advantage. Perception & Psychophysics, 67, 1214–1228. Brunel, N., & Wang, X. J. (2001). Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition. Journal of Computational Neuroscience, 11, 63–85. Crawford, J. R., & Howell, D. C. (1998). Comparing an individual’s test score against norms derived from small samples. Clinical Neuropsychologist, 12, 482–486. Deco, G., & Rolls, E. T. (2005). Neurodynamics of biased competition and cooperation for attention: A model with spiking neurons. Journal of Neurophysiology, 94, 295–313. Donk, M., & Theeuwes, J. (2001). Visual marking beside the mark: Prioritizing selection by abrupt onsets. Perception & Psychophysics, 63, 891–900. Duncan, J., & Humphreys, G. W. (1989). Visual-search and stimulus similarity. Psychological Review, 96, 433–458. Eglin, M., Robertson, L. C., & Knight, R. T. (1989). Visual search performance in the neglect syndrome. Journal of Cognitive Neuroscience, 1, 372–385. Elliott, M. A., & Müller, H. J. (2004). Synchronization and stimulus timing: Implications for temporal models of visual information processing. In C. Kaernbach, E. Schröger, & H. Müller (Eds.), Psychophysics beyond Sensation (pp. 137–156). Mahwah, NJ: Lawrence Erlbaum and Associates. Howard, C., Chechlacz, M., & Humphreys, G. W. (2015). Neural mechanism of temporal resolution of attention. Cerebral Cortex. doi:10.1093/cercor/bhv101. Humphreys, G. W. (1998). Neural representation of objects in space: A dual coding account. Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences, 353, 1471–2970. Humphreys, G. W., Olivers, C. N. L., & Braithwaite, J. J. (2006). The time course of preview search with color-defmed, not luminance-defined, stimuli. Perception & Psychophysics, 68, 1351–1358. Humphreys, G. W., Riddoch, M. J., Nys, G., & Heinke, D. (2002). Transient binding by time: Neuropsychological evidence from anti-extinction. Cognitive Neuropsychology, 19, 361–380. Humphreys, G. W., Stalmann, B. J., & Olivers, C. (2004). An analysis of the time course of attention in preview search. Perception & Psychophysics, 66, 713–730. 1566 Journal of Cognitive Neuroscience Volume 28, Number 10 D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j . / t f u s e r o n 1 7 M a y 2 0 2 1 Husain, M., Shapiro, K., Martin, J., & Kennard, C. (1997). Abnormal temporal dynamics of visual attention in spatial neglect patients. Nature, 385, 154–156. Kunar, M. A., Humphreys, G. W., & Smith, K. J. (2003a). History matters: The preview benefit in search is not onset capture. Psychological Science, 14, 181–185. Kunar, M. A., Humphreys, G. W., & Smith, K. J. (2003b). Visual change with moving displays: More evidence for color feature map inhibition during preview search. Journal of Experimental Psychology: Human Perception and Performance, 29, 779–792. Luce, R. D. (1959). Individual choice behavior. New York: Wiley. Luce, R. D. (1977). The choice axiom after twenty years. Journal of Mathematical Psychology, 15, 215–233. Lunven, M., Thiebaut de Schotten, M., Bourlon, C., Duret, C., Migliaccio, R., Rode, G., et al. (2015). White matter lesional predictors of chronic visual neglect: A longitudinal study. Brain, 138, 746–760. Mavritsaki, E., Heinke, D., Allen, H., Deco, G., & Humphreys, G. W. (2011). Bridging the gap between physiology and behavior: Evidence from the sSoTS model of human visual attention. Psychological Review, 118, 3–41. Roberts, K. L., Lau, J. K. L., Chechlacz, M., & Humphreys, G. W. (2012). Spatial and temporal attention deficits following brain injury: A neuroanatomical decomposition of the temporal order judgement task. Cognitive Neuropsychology, 29, 300–324. Singer, W., & Gray, C. M. (1995). Visual feature integration and the temporal correlation hypothesis. In W. M. Cowan (Ed.), Annual review of neuroscience (Vol. 18, pp. 555–586). Palo Alto, CA: Annual Reviews, Inc. Townsend, J. T., & Ashby, F. G. (1983). The Stochastic Modeling of Elementary Psychological Processes. Cambridge: Cambridge University Press. Treisman, A. M. (1998). Feature binding, attention and object perception. Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences, 353, 1295–1306. Treisman, A. M., & Gelade, G. (1980). Feature-integration theory of attention. Cognitive Psychology, 12, 97–136. Usher, M., & Donnelly, N. (1998). Visual synchrony affects binding and segmentation in perception. Nature, 394, 179–182. Mavritsaki, E., Heinke, D., Deco, G., & Humphreys, G. W. Ward, R., Goodrich, S., & Driver, J. (1994). Grouping (2009). Simulating posterior parietal damage in a biologically plausible framework: Neuropsychological tests of the search over time and space model. Cognitive Neuropsychology, 26, 343–390. Mavritsaki, E., Heinke, D., Humphreys, G. W., & Deco, G. (2006). A computational model of visual marking using an interconnected network of spiking neurons: The spiking search over time & space model (sSoTS). Journal of Physiology-Paris, 100, 110–124. Olivers, C. N. L., & Humphreys, G. W. (2004). Spatiotemporal segregation in visual search: Evidence from parietal lesions. Journal of Experimental Psychology: Human Perception and Performance, 30, 667–688. Rappaport, S. J., Humphreys, G. W., & Riddoch, M. J. (2013). The attraction of yellow corn: Reduced attentional constraints on coding learned conjunctive relations. Journal of Experimental Psychology: Human Perception and Performance, 39, 1016–1031. Riddoch, M. J., & Humphreys, G. W. (1987). A case of integrative visual agnosia. Brain, 110, 1431–1462. reduces visual extinction: Neuropsychological evidence for weight-linkage in visual selection. Visual Cognition, 1, 101–129. Watson, D. G., & Humphreys, G. W. (1997). Visual marking: Prioritizing selection for new objects by top–down attentional inhibition of old objects. Psychological Review, 104, 90–122. Watson, D. G., & Humphreys, G. W. (2000). Visual marking: Evidence for inhibition using a probe-dot detection paradigm. Perception & Psychophysics, 62, 471–481. Wildegger, T., Riddoch, J., & Humphreys, G. W. (2015). Stored color-form knowledge modulates perceptual sensitivity in search. Attention Perception & Psychophysics, 77, 1223–1238. Wolfe, J. M. (1994). Guided Search 2.0—A revised model of visual-search. Psychonomic Bulletin & Review, 1, 202–238. Yantis, S., & Gibson, B. S. (1994). Object continuity in apparent motion and attention. Canadian Journal of Experimental Psychology-Revue Canadienne De Psychologie Experimentale, 48, 182–204. D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e o d u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 8 / 1 2 0 8 / 1 1 5 0 5 / 3 1 1 5 9 5 5 3 1 / 5 1 6 0 7 8 o 5 c 3 n 4 _ 9 a / _ j 0 o 0 c 9 n 8 4 _ a p _ d 0 0 b 9 y 8 g 4 u . e p s t d o f n b 0 y 7 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j . t / f u s e r o n 1 7 M a y 2 0 2 1 Mavritsaki and Humphreys 1567

Télécharger le PDF