Temporal Anticipation Based on Memory
André M. Cravo1, Gustavo Rohenkohl2, Karin Moreira Santos1,
and Anna C. Nobre2
Abstrakt
■ The fundamental role that our long-term memories play in
guiding perception is increasingly recognized, but the func-
tional and neural mechanisms are just beginning to be explored.
Although experimental approaches are being developed to in-
vestigate the influence of long-term memories on perception,
these remain mostly static and neglect their temporal and dy-
namic nature. Hier, we show that our long-term memories can
guide attention proactively and dynamically based on learned
temporal associations. Across two experiments, we found that
detection and discrimination of targets appearing within pre-
viously learned contexts are enhanced when the timing of target
appearance matches the learned temporal contingency. Neuronal
markers of temporal preparation revealed that the learned tem-
poral associations trigger specific temporal predictions. Our find-
ings emphasize the ecological role that memories play in
predicting and preparing perception of anticipated events, calling
for revision of the usual conceptualization of contextual asso-
ciative memory as a reflective and retroactive function. ■
EINFÜHRUNG
Perception is increasingly recognized to be a highly pro-
active process resulting in a selective (Re)construction of
the external milieu that emphasizes items and attributes
that may be adaptive in a given context. Goal-driven se-
lective attention has provided a successful paradigm for
investigating the sources and mechanisms of top–down
modulation of signal processing within perceptual streams.
Decades of research have yielded enormous progress in
revealing how the locations and feature-related attributes
of relevant events are prioritized and integrated along
the sensory hierarchies (Fries, 2015; Reynolds & Chelazzi,
2004; Kastner & Ungerleider, 2000; Desimone & Ducan,
1995). These top–down biases were subsequently shown
also to carry dynamic information about the estimated
timing of relevant events—a phenomenon called temporal
orienting of attention or, more generally, temporal expec-
Station (Nobre & Rohenkohl, 2014). Trying to understand
how temporal predictions of relevant events are extracted
and can guide top–down control has become an active
area of research, with promising inroads being made
(Calderone, Schlosser, Diener, & Castellanos, 2014; Cravo,
Rohenkohl, Wyart, & Nobre, 2013; Rohenkohl & Nobre,
2011; Schlosser, Karmos, Mehta, Ulbert, & Schroeder, 2008;
Doherty, Rao, Mesulam, & Nobre, 2005; Vangkilde, Coull,
& Bundesen, 2005).
As the attention field matures, scholars have returned to
older hypothesized sources of top–down control of per-
1Federal University of ABC (UFABC), Santo André, Brasilien, 2Oxford
Centre for Human Brain Activity, Universität Oxford
ception. In addition to current goals uploaded into short-
term stores, our long-term memories have been proposed
to guide perception from the earliest days of empirical
Psychologie (von Helmholtz, 1867). Contemporary re-
search using various types of tasks vindicates this classic
notion (Goldfarb et al., 2016; Kasper, Grafton, Eckstein,
& Giesbrecht, 2015; Giesbrecht, Sy, & Guerin, 2013; Zhao,
Al-Aidroos, & Turk-Browne, 2013; Hutchinson & Turk-
Browne, 2012; Stokes, Atherton, Patai, & Nobre, 2012;
Kunar, Flusberg, & Wolfe, 2008; Summerfield, Lepsien,
Gitelman, Mesulam, & Nobre, 2006; Chun, 2000). The tasks
gebraucht, Jedoch, tend to focus on static aspects of learned
contingencies, such as the location or identity of a target
within an array or scene. In the current study, we asked
whether our long-term memories can also carry temporal
information that can guide perceptual analysis proactively
and dynamically to enhance the processing of anticipated
target attributes at the right moment in time. The research
builds on recent discoveries of mechanisms for encoding
sequential and temporal information within memory sys-
Systeme (Davachi & DuBrow, 2015; Eradath, Mogami, Wang,
& Tanaka, 2015; Eichenbaum, 2013; MacDonald, Lepage,
Eden, & Eichenbaum, 2011; Dragoi & Buzsaki, 2006).
We designed a novel memory-based temporal orient-
ing-Aufgabe, based on previous work in the spatial domain
(Stokes et al., 2012; Summerfield, Rao, Garside, & Nobre,
2011; Summerfield et al., 2006), to test for performance
benefits conferred by learned temporal associations
between target items and complex contexts. In the cur-
rent study, participants learn that the target event occurs
after a specific temporal interval within a given context.
They subsequently perform a memory-based temporal
© 2017 Massachusetts Institute of Technology. Published under a
Creative Commons Attribution 3.0 Unportiert (CC BY 3.0) Lizenz.
Zeitschrift für kognitive Neurowissenschaften 29:12, S. 2081–2089
doi:10.1162/ jocn_a_01172
D
Ö
w
N
l
Ö
A
D
e
D
F
R
Ö
M
l
l
/
/
/
/
J
T
T
F
/
ich
T
.
:
/
/
H
T
T
P
:
/
D
/
Ö
M
w
ich
N
T
Ö
P
A
R
D
C
e
.
D
S
F
ich
R
Ö
l
M
v
e
H
R
C
P
H
A
D
ich
ich
R
R
e
.
C
C
T
.
Ö
M
M
/
J
e
Ö
D
u
C
N
Ö
/
C
A
N
R
A
T
R
ich
T
ich
C
C
l
e
e
–
P
–
D
P
D
2
F
9
/
1
2
2
9
/
2
1
0
2
8
/
1
2
1
0
9
8
5
1
3
/
2
1
5
6
7
8
Ö
6
C
7
N
6
_
6
A
/
_
J
0
Ö
1
C
1
N
7
2
_
A
P
_
D
0
1
B
1
j
7
G
2
u
.
e
P
S
T
D
Ö
F
N
B
0
j
7
S
M
e
ICH
P
T
e
M
L
ich
B
B
e
R
R
A
2
R
0
2
ich
3
e
S
/
J
F
.
/
T
u
S
e
R
Ö
N
1
7
M
A
j
2
0
2
1
orienting task in which they are asked to detect (Experi-
ment 1) or discriminate (Experiment 2) the target appear-
ance in the studied contexts.
METHODEN
Teilnehmer
Ten volunteers (three women, seven men; Durchschnittsalter =
19.4 Jahre) participated in Experiment 1 (detection), Und
18 (7 Frauen, 11 men; Durchschnittsalter = 20.17 Jahre) partici-
pated in Experiment 2 (discrimination). They all gave
informed consent. All had normal or corrected vision
and were free from psychological or neurological dis-
eases according to self-report. The number of partici-
pants was based on comparable sample sizes in the
Literatur (Stokes et al., 2012; Summerfield et al., 2006).
The experimental protocol was approved by the research
ethics committee of the Federal University of ABC and
the central university research ethics committee of the
Universität Oxford.
Apparatus
The stimuli were created on MATLAB v.7.10 (The Math-
Funktioniert, Inc., Natick, MA) and presented using the Psych-
toolbox v.3.0 package for MATLAB (Brainard & Vision,
1997). Images were displayed on a 21-in. CRT with a spa-
tial resolution of 1024 × 768 pixels and a vertical refresh
rate of 60 Hz, placed 100 cm in front of the participant.
Responses were collected via a response box (DirectIN
High SpeedButton/Empirisoft, New York, New York).
Stimuli and Task
We conducted two similar experiments, in which par-
ticipants learned new associations about the timing of a
target event occurring within a scene and then per-
formed an orienting task requiring detection (Experi-
ment 1) or discrimination (Experiment 2) of the target
event occurring within the learned context. In Experi-
ment 2, EEG activity was recorded during the perfor-
mance of the final, temporal orienting task requiring
target discrimination. Each experiment consisted of three
different tasks that took take place on the same day: A
learning task, a memory task, and a temporal orienting
Aufgabe. Participants performed a session of the learning
Aufgabe, followed by a memory task. They then performed
another session of the learning task and one more ses-
sion of the memory task. Endlich, they performed the
temporal orienting task.
learned the time for a target event to occur within each
scene. Scene stimuli were similar to those used by previ-
ous studies (Stokes et al., 2012; Summerfield et al., 2006,
2011), consisting of photographs of different indoor or
outdoor views. Scenes were prepared using MATLAB
and subtended 22° × 17° of visual angle at a viewing dis-
tance of 100 cm. Although we considered using dynamic
scenes, this would have conflated the timing of the target
event with a sequence of spatial and/or feature-related
changes that need not specifically rely on learning tem-
poral intervals.
Each scene was associated with a target event being
presented in a specific time and place that remained
fixed throughout the whole learning session. The target
event occurred between 5° and 7° of visual angle along
both the lateral and longitudinal axes and was preceded
by a placeholder presented at the exact same location.
Participants were instructed to learn when the target
event was presented within each scene. The interval
and location of the target within each scene were ran-
domized between participants. A briefly presented target
thus occurred at a precise moment within a static scene.
This arrangement was chosen over presenting a target
within an evolving animated context (film) because it
eliminates the possibility of learning relying only on
associations between the occurrence of the target and a
sequence of spatial or features within the dynamic con-
Text. By using the simpler approach, it was possible to iso-
late the effects of learning a purely temporal association.
Each trial started with the presentation of one of the
scenes and a fixation cue in the center of the screen. Nach
a period of 1.5 Sek, a placeholder black bomb (1° × 1°)
was presented in either the upper or lower quadrant of
the right or left side of the scene. After an interval of either
800 oder 2000 ms, the bomb changed its color to blue (go
target, 80% of the trials) or red (no-go target, 20% of the
Versuche). The type of target (go or no-go) was randomized
over scenes, and participants were instructed that the
same scene could have go or no-go targets in different
blocks. Half of the images (48 scenes) were associated
with each interval (short or long). Participants were in-
structed to respond as quickly as possible to go targets.
If participants responded correctly and under 600 ms,
a smoky cloud was presented, indicating that the re-
sponse was correct. If participants did not respond to
go targets within 600 msec or if they responded to no-go
targets, an explosion image was presented. The order of
scene presentation was randomized in each block. Partici-
pants performed three learning blocks in a row and then
performed a memory task. They then completed two
more learning blocks followed by another memory task.
Experiment 1: Detection
Learning Task
During the learning task, participants viewed 96 Komplex
scenes repeated in random order over five blocks and
Memory Task
During the memory task, participants viewed the same
96 naturalistic scenes repeated in random order. Der
scenes were presented on their own (no bombs appeared)
2082
Zeitschrift für kognitive Neurowissenschaften
Volumen 29, Nummer 12
D
Ö
w
N
l
Ö
A
D
e
D
F
R
Ö
M
l
l
/
/
/
/
J
F
/
T
T
ich
T
.
:
/
/
H
T
T
P
:
/
D
/
Ö
M
w
ich
N
T
Ö
P
A
R
D
C
e
.
D
S
F
ich
R
Ö
l
M
v
e
H
R
C
P
H
A
D
ich
ich
R
R
e
.
C
C
T
.
Ö
M
M
/
J
e
Ö
D
u
C
N
Ö
/
C
A
N
R
A
T
R
ich
T
ich
C
C
l
e
e
–
P
–
D
P
D
2
F
9
/
1
2
2
9
/
2
1
0
2
8
/
1
2
1
0
9
8
5
1
3
/
2
1
5
6
7
8
Ö
6
C
7
N
6
_
6
A
/
_
J
0
Ö
1
C
1
N
7
2
_
A
P
_
D
0
1
B
1
j
7
G
2
u
.
e
P
S
T
D
Ö
F
N
B
0
j
7
S
M
e
ICH
P
T
e
M
L
ich
B
B
e
R
R
A
2
R
0
2
ich
3
e
S
/
J
.
F
T
/
u
S
e
R
Ö
N
1
7
M
A
j
2
0
2
1
and remained on the screen until participants responded.
Their task was to indicate if the scene was associated with
a short (800 ms) or long (2000 ms) interval during
the learning task. Responses were made using index/
middle fingers of the right hand. Memory tasks were
performed after three blocks of the learning task and after
the final block of the learning task.
Temporal Orienting Task
After completing five blocks of the learning task and
two memory tasks, participants performed the temporal
orienting task. The task was similar in structure to the
learning task. Participants viewed the same 96 scenes, In
which a bomb changed color after a short or long interval.
In most of the trials (67%), the interval in the orienting
task was the same as the learned interval in the learning
Aufgabe. The scene therefore triggered a valid memory cue
for target timing. In the remaining trials (33%), the inter-
val was switched, and the scene provided an invalid tem-
poral memory cue. As before, participants were instructed
to respond as quickly as possible to go targets and to with-
hold responding to no-go targets. The temporal orienting
task consisted of three blocks, each with 96 scenes. In
each block, a different subset of the scenes was selected
to have an invalid memory cue. No feedback (smoky
cloud or explosion) was given during this task.
Experiment 2: Discrimination
The second experiment served as a replication and exten-
sion of Experiment 1, with EEG recordings made during
the orienting task. The experiment contained the same
three phases. The major differences were that, stattdessen
of using go/no-go targets, a change in bomb color (Blau
or green) required a discrimination response. Partici-
pants were instructed to press the right button when
the bomb turned blue and the left button when it turned
Grün (the mapping of color and response was counter-
balanced across participants). Blue and green bombs
were equiprobable and occurred arbitrarily for each
scene. Participants were instructed that each scene was
associated with the target event being presented in a
specific time and place but that there was no association
between the scene and the color of the bomb. Anstatt
performing five learning blocks as in Experiment 1, Par-
ticipants performed seven learning blocks. The memory
task was performed after four blocks of learning and then
after the final learning task block. The temporal orienting
task was performed last.
EEG Recording and Preprocessing
Continuous recording from 64 ActiCap electrodes (Gehirn
Products, München, Deutschland) bei 1000 Hz referenced to
FCz (AFz ground) provided the EEG signal. The elec-
trodes were positioned according to the International
10–10 system. Additional bipolar electrodes recorded
the EOG. EOG electrodes were placed to the side of each
eye (horizontal EOG) and above and below the right eye
(vertical EOG). EEG was recorded using a QuickAmp
amplifier and preprocessed using BrainVision Analyzer
(Brain Products). Data were downsampled to 250 Hz
and rereferenced to the averaged earlobes. To remove
eye blink artifacts, filtered data (0.05–30 Hz) were sub-
jected to independent component analysis. Eye-related
components were identified through comparison of
individual components with EOG channels and through
visual inspection. Vertical eye activity was removed using
independent component analysis.
For analyses of the contingent negative variation (CNV),
epochs were segmented from 250 msec before scene
onset until 800 msec after cue presentation. Epochs con-
taining excessive noise or drift (±100 μV at any electrode)
or eye artifacts (saccades) were rejected. Saccades were
identified as large deflections (±50 μV) in the horizontal
EOG electrodes. All data were subsequently checked by
visual inspection. Data from four participants were re-
moved because of excessive eye movements (two partici-
Hose) or an excessive number of rejected trials (zwei
Teilnehmer). A small proportion of trials of the remaining
participants were rejected (0.05 ± 0.01). We focused our
analyses on short and long valid cues, with an average of
around 90 clean epochs per condition.
ERGEBNISSE
Learning Task
During the learning task (Figure 1A) of both experiments,
participants viewed 96 scenes repeated in random order
over five (Experiment 1) or seven (Experiment 2) blocks
and learned the temporal interval at which the target
event occurred within each scene. To quantify the im-
provement in performance in the learning tasks, RTs from
the first and last blocks for short and long intervals were
submitted to a 2 × 2 repeated-measures ANOVA, with fac-
tors Interval (Short × Long) and Block (First × Last).
In Experiment 1, participants had better performance
at the end of the learning session for both short and
long intervals (two-way Interval × Block ANOVA: main
effect of Interval, F(1, 9) = 50.74, P < .001, η2
partial =
0.435; main effect of Block, F(1, 9) = 105.79, p < .001,
η2
partial = 0.852; interaction, F(1, 9) = 7.56, p = .02,
η2
partial = 0.063). However, learning was stronger for
scenes with short intervals (t(9) = 2.75, p = .02, d = 0.869).
For Experiment 2, benefits in performance depended
on the interval (two-way Interval × Block ANOVA: main
effect of Interval, F(1, 13) = 9.25, p = .009, η2
partial =
0.016; no main effect of Block, F(1, 13) = 3.04, p = .11,
η2
partial = 0.063; interaction, F(1, 13) = 5.09, p = .04,
η2
partial = 0.007). Specifically, RTs improved only for short
intervals (first vs. last blocks for short intervals, t(13) =
2.85, p = .014, d = 0.762, and long intervals, t(13) = 0.74,
Cravo et al.
2083
D
o
w
n
l
o
a
d
e
d
f
r
o
m
l
l
/
/
/
/
j
t
t
f
/
i
t
.
:
/
/
h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
-
p
-
d
p
d
2
f
9
/
1
2
2
9
/
2
1
0
2
8
/
1
2
1
0
9
8
5
1
3
/
2
1
5
6
7
8
o
6
c
7
n
6
_
6
a
/
_
j
0
o
1
c
1
n
7
2
_
a
p
_
d
0
1
b
1
y
7
g
2
u
.
e
p
s
t
d
o
f
n
b
0
y
7
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
2
i
3
e
s
/
j
.
t
f
/
u
s
e
r
o
n
1
7
M
a
y
2
0
2
1
Figure 1. Learning and
memory tasks. (A) During
the learning task, participants
viewed a complex scene and
learned the temporal interval
at which the target event
occurred within that scene.
After 1500 msec of the scene
presentation, a placeholder
(bomb) appeared. After an
800-msec (short) or 2000-msec
(long) interval, the placeholder
changed color. In Experiment 1,
the target changed to blue in
80% of trials (go-target) or red
in 20% of trials (no-go target).
In Experiment 2, the target
changed to blue or green in
an equal proportion of trials.
Participants had to detect the
target (Experiment 1) or
discriminate the color of the
target (Experiment 2). In both
tasks, participants’ RTs
decreased as a function of
Block, with a stronger effect
for short intervals. The dashed
lines represent when the
memory task was performed
in each experiment. All plots
show mean and SEM across
participants. (B) In the memory
task, participants viewed each
scene in isolation and indicated
whether it was associated with
a short or long interval. This
task was performed by the
participants halfway through
the experimental session (first
session) and at the end of
the learning task (second
session). Mean accuracies show
how participants improved
their performance over learning
blocks, forming reliable explicit
memories for the temporal
associations between scenes
and target presentation. All
plots show mean accuracy
across participants (darker
colors) and raw data from all
participants (lighter colors).
p = .47, d = 0.198). Thus, in both experiments, systematic
decreases in RTs suggested that participants learned the
temporal relationship between scenes and target intervals,
with more pronounced learning for the short interval,
as expected according to the hazard effect (Nobre &
Rohenkohl, 2014; Cravo, Rohenkohl, Wyart, & Nobre,
2011).
Memory Task
each scene (Figure 1B). The memory task was repeated
midway through the learning task (after Block 3 in Exper-
iment 1 and after Block 4 in Experiment 2) and after
completion of the learning task. During the memory task,
participants viewed each scene in isolation and indicated
whether it was associated with a short or long interval.
Mean accuracies for scenes with short and long intervals
for the two blocks of the memory task were submitted
to a repeated-measures ANOVA, with factors Interval
(Short × Long) and Block (First × Last).
The memory task assessed whether participants formed
an explicit memory for the temporal association within
In both Experiments, there was an increase in accuracy
as a function of learning (two-way Interval × Block ANOVA;
2084
Journal of Cognitive Neuroscience
Volume 29, Number 12
D
o
w
n
l
o
a
d
e
d
f
r
o
m
l
l
/
/
/
/
j
f
/
t
t
i
t
.
:
/
/
h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
-
p
-
d
p
d
2
f
9
/
1
2
2
9
/
2
1
0
2
8
/
1
2
1
0
9
8
5
1
3
/
2
1
5
6
7
8
o
6
c
7
n
6
_
6
a
/
_
j
0
o
1
c
1
n
7
2
_
a
p
_
d
0
1
b
1
y
7
g
2
u
.
e
p
s
t
d
o
f
n
b
0
y
7
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
2
i
3
e
s
/
j
f
.
t
/
u
s
e
r
o
n
1
7
M
a
y
2
0
2
1
Experiment 1: main effect of Block, F(1, 8) = 20.37, p = .002,
η2
partial = 0.730; no main effect of Interval, F(1, 8) = 0.04,
p = .84, η2
partial = 0.001; no interaction, F(1, 8) = 0.002, p =
.97, η2
partial = 0; Experiment 2: main effect of Block, F(1,
13) = 23.02, p < .001, η2
partial = 0.352; no main effect of
Interval, F(1, 13) = 3.74, p = .075, η2
partial = 0.065; no
interaction, F(1, 13) = 0.269, p = .613, η2
partial = 0.001).
The results showed that participants formed reliable ex-
plicit memories for the temporal associations between
scenes and target presentation (Figure 1B).
Orienting Task
The final orienting task probed whether the learned tem-
poral associations influenced behavioral performance to
expected targets. In most trials, the target occurred at
the remembered interval (valid cue), whereas in the re-
maining trials, target occurred at the other interval, and
the scene thus provided invalid temporal information
(invalid cue).
Mean RTs for correct responses were submitted to a
repeated-measures ANOVA with Interval (Short × Long)
and Cue ( Valid × Invalid) as factors. As shown in Figure 2,
performance was strongly influenced by long-term mem-
ory cues. In both experiments, RTs were shorter when
targets were presented at the learned temporal interval
(two-way Cue × Interval ANOVA; Experiment 1: main effect
of Cue, F(1, 9) = 30.47, p < .001, η2
partial = 0.290; main ef-
fect of Interval, F(1, 9) = 10.14, p = .01, η2
partial = 0.254; no
interaction, F(1, 9) = 2.3, p = .163, η2
partial = 0.020; Exper-
iment 2: main effect of Cue, F(1, 13) = 20.14, p = .001,
η2
partial = 0.029; no main effect of Interval, F(1, 13) =
0.42, p = .530, η2
partial = 0; no interaction, F(1, 13) =
0.023, p = .883, η2
partial = 0).
Figure 2. Temporal orienting
task. (A) In the temporal
orienting task, trial sequence
was similar to the learning task;
however, the interval when
the target appeared matched
that in the learning task in
most trials (67% valid cues),
whereas in the remaining
trials (33% invalid cues), the
target occurred at the other
interval. (B) Performance
was strongly influenced by
long-term memory cues,
and both RTs and perceptual
sensitivity were better for
valid ( V) than invalid (I)
scenes. All plots show mean
across participants (darker
colors) and raw data from all
participants (lighter colors).
D
o
w
n
l
o
a
d
e
d
f
r
o
m
l
l
/
/
/
/
j
t
t
f
/
i
t
.
:
/
/
h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
-
p
-
d
p
d
2
f
9
/
1
2
2
9
/
2
1
0
2
8
/
1
2
1
0
9
8
5
1
3
/
2
1
5
6
7
8
o
6
c
7
n
6
_
6
a
/
_
j
0
o
1
c
1
n
7
2
_
a
p
_
d
0
1
b
1
y
7
g
2
u
.
e
p
s
t
d
o
f
n
b
0
y
7
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
2
i
3
e
s
/
j
f
t
/
.
u
s
e
r
o
n
1
7
M
a
y
2
0
2
1
Cravo et al.
2085
0
We further calculated d
for each condition in the
temporal orienting task. In Experiment 1, hits were con-
sidered as a correct response for a go target, whereas
false alarms were considered when participants responded
0 were submitted to a repeated-
to a no-go target; d
measures ANOVA with Interval (Short × Long) and Cue
(Valid × Invalid) as factors. In Experiment 2, hits were cal-
culated as correct response for green targets; and false
0 were
alarms, as incorrect responses for blue targets; d
submitted to a repeated-measures ANOVA with Interval
(Short × Long) and Cue (Valid × Invalid) as factors.
As can be seen in Figure 2, long-term memory also im-
proved perceptual sensitivity for both detection (Experi-
ment 1, two-way Cue × Interval ANOVA: main effect of
Cue, F(1, 9) = 9.54, p = .013, η2
partial = 0.198; no main
effect of Interval, F(1, 9) = 0.54, p = .481, η2
partial =
0.017; interaction, F(1, 9) = 9.72, p = .012, η2
partial =
0.081) and discrimination (Experiment 2, two-way Cue ×
Interval ANOVA: main effect of Cue, F(1, 13) = 7.33,
p = .018, η2
partial = 0.066; no main effect of Interval, F(1,
13) = 0.05, p = .824, η2
partial = 0.001; no interaction, F(1,
13) = 0.70, p = .419, η2
partial = 0.010) tasks. For the detec-
tion task, perceptual sensitivity effects were restricted to
the short interval (paired t test between valid and invalid
cues for short intervals, t(9) = 4.64, p = .001, d = 1.467,
and long intervals, t(9) = 0.20, p = .845, d = 0.063).
CNV
In the orienting task of Experiment 2, analyses of the
CNV focused in central midline electrodes (F1/Fz/F2/
FC1/FC2) for scenes associated with short and long inter-
vals during the learning task. A cluster-based analysis
(Maris & Oostenveld, 2007) was applied to the whole pe-
riod (from −200 msec before scene onset until 800 msec
after the bomb was presented) to compare the CNV be-
tween conditions for the period between scene presen-
tation and the first possible moment of the target. The
nonparametric statistics were performed by calculating a
permutation test in which experimental conditions were
randomly intermixed within each participant and repeat-
ed 1000 times. The CNV for valid cues had higher (more
negative) amplitudes for the period from 90 to 340 msec
after cue presentation (cluster-stat = 202.05, cluster p =
.002) and for the period from 390 to 800 msec after cue
presentation (cluster-stat = 363.30, cluster p < .001).
To test whether the CNV reflected a stronger temporal
anticipation, we investigated if there was a relation be-
tween CNV at the single-trial level and RTs. This analysis
was performed in scenes associated with short intervals
in the learning task and that were presented at the
short interval in the temporal orienting task (short valid
cues). The CNV activity for the second cluster (from 390
to 800 msec after cue onset) was averaged for each trial,
z scored and separated into five bins (each with 20% of
the data). The associated RT for each bin was calculated,
and a nonparametric regression was calculated for each
participant. At the group level, the Fisher-transformed
estimated coefficients for the regression were compared
with zero using a t test. We found that the amplitude of
the CNV correlated significantly with RTs, indicating a
functional relation between neural preparation and be-
havioral performance (t(13) = 2.69, p = .018, d =
0.719; Figure 3C).
Memory Strength and Performance
An important property of learned temporal contextual
associations is that their strength can vary. To estimate
the strength of the temporal association memories, we
used the RTs during the memory task.
In a first step, we investigated whether these RTs were
correlated with response accuracy. For each participant,
RTs for all scenes during the second memory task (after
completion of the learning task) were separated into five
bins, each containing 20% of the data. RTs shorter or
longer than 2.5 SDs were removed before binning. For
each bin, the mean accuracy was calculated. A nonpara-
metric regression was performed separately for each
participant. At the group level, the Fisher-transformed
estimated coefficients were compared with zero using a
paired t test. Participants formed stronger temporal
memories for some scenes than for others as shown by
the association between RT and accuracy during the
memory test (t test on the estimated slopes, t(13) =
−3.53, p = .004, d = 0.943; Figure 3D).
Given the strong association between RT and accuracy,
we used these RTs as a memory strength index in two
following analyses. In a first analysis, we investigated
whether this index was associated with shorter RTs in
the subsequent temporal orienting task. If participants
had a stronger association between a given scene and
its learned interval, then they should benefit more
strongly from this association. We focused our analysis
on (1) the first block of the temporal orienting task,
(2) short valid trials, (3) trials in which participants gave
correct responses in the temporal orienting task, and
(4) scenes that participants judged correctly in the mem-
ory task. These restrictions were used to isolate as max-
imally as possible the effect of memory on performance.
For each trial in the temporal orienting task conform-
ing to the abovementioned restrictions, the RT for that
scene in the memory task was used as a predictor of
the RT in the temporal orienting task. The memory
strength index was calculated as the percentage of RTs
that were longer than each individual RT. For example,
for the shortest RT, all other RTs were longer, resulting
in a memory strength index of 100. A nonparametric
regression was performed with the RT in the temporal
orienting task as the dependent variable and with the
memory strength index as the predictor. At the group
level, the Fisher-transformed estimated coefficients were
compared with zero using a paired t test. As can be seen in
Figure 3, memory strength was predictive of behavioral
2086
Journal of Cognitive Neuroscience
Volume 29, Number 12
D
o
w
n
l
o
a
d
e
d
f
r
o
m
l
l
/
/
/
/
j
t
t
f
/
i
t
.
:
/
/
h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
-
p
-
d
p
d
2
f
9
/
1
2
2
9
/
2
1
0
2
8
/
1
2
1
0
9
8
5
1
3
/
2
1
5
6
7
8
o
6
c
7
n
6
_
6
a
/
_
j
0
o
1
c
1
n
7
2
_
a
p
_
d
0
1
b
1
y
7
g
2
u
.
e
p
s
t
d
o
f
n
b
0
y
7
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
2
i
3
e
s
/
j
/
.
t
f
u
s
e
r
o
n
1
7
M
a
y
2
0
2
1
Figure 3. Electrophysiological
results. (A) Topographies of
the grand-averaged CNV
and for the CNVs at the
short foreperiod for scenes
associated with short or long
intervals. (B) The CNV recorded
during the orienting task of
Experiment 2 was strongly
influenced by the temporal
association in memory (red
lines at the bottom represent
the two temporal clusters
where the CNV was larger
for short than long temporal
associations). (C) Larger CNV
amplitudes were associated
with shorter RTs (CNV values
were binned into five equally
sized bins for display purposes,
although analyses were
performed on raw data).
(D, left) During the memory
task, shorter RTs were
associated with higher accuracy.
Given this relation, RTs were
used to create a memory
strength index, which estimated
the quality of the temporal
association memory. Further
analyses showed that stronger
memories were associated
with shorter RTs during the
subsequent temporal orienting task (center) and with CNV amplitude (right). Memory strength was binned into five equally sized bins in the middle
and right for display purposes, although analyses were performed on raw data. All plots show mean and SEM across participants.
performance benefits (t test on the estimated slopes,
t(13) = −2.71, p = .018, d = 0.723).
stored temporal associations of specific intervals to pre-
pare neural activity for relevant upcoming events.
A similar analysis was performed to test whether this
index was also related to the CNV. The same restrictions
were used, and the memory strength index was calculated
in a similar way. The CNV was measured in the same
electrodes as previously mentioned and in the period of
the second significant cluster (390–800 msec). A nonpara-
metric regression was performed with the CNV as the
dependent variable and with the memory strength index
as the predictor. At the group level, the Fisher-transformed
estimated coefficients were compared with zero using a
paired t test. Similar to behavioral performance, memory
strength was also predictive of the CNV amplitude (t test
on the estimated slopes, t(13) = −2.33, p = .037, d =
0.620).
DISCUSSION
Across two experiments, we found that participants
were able to learn temporal associations between target
items and complex contexts. This learning was beneficial
in the orienting task, with participants responding faster
and more accurately for scenes tested at the learnt inter-
val. Our findings suggest that long-term memories can
guide our perception and behavior dynamically, utilizing
Our results contribute crucial insights to the understand-
ing of the influence of timing in contextual long-term mem-
ory. The relationship between timing and long-term memory
is attracting increasing interest. Most studies so far have con-
sidered how the temporal order of events is encoded (Ezzyat
& Davachi, 2014; Dragoi & Buzsaki, 2006) or how temporal
proximity and regularity can modulate retrieval (Schapiro,
Kustner, & Turk-Browne, 2012; Schwartz, Howard, Jing,
& Kahana, 2005). In our studies, it becomes clear that pre-
cise temporal intervals, and not only the order of events,
can be learned. Furthermore, these stored temporal asso-
ciations are projected dynamically to anticipate relevant
items at just the right moment to optimize performance.
Previous studies in long-term memory and attention,
using a similar task, have shown that learning spatial
locations of events can improve perceptual sensitivity
and RTs (Stokes et al., 2012; Summerfield et al., 2006,
2011). In these tasks, it has been suggested that the ef-
fects of allocating attention based on long-term memory
or on a symbolic cue might share similar anticipatory
brain states, as alpha desynchronization.
Similarly, in our results, we found that long-term mem-
ory modulated an electrophysiological marker consistently
found in temporal attention studies, the CNV. Targets that
Cravo et al.
2087
D
o
w
n
l
o
a
d
e
d
f
r
o
m
l
l
/
/
/
/
j
t
t
f
/
i
t
.
:
/
/
h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
-
p
-
d
p
d
2
f
9
/
1
2
2
9
/
2
1
0
2
8
/
1
2
1
0
9
8
5
1
3
/
2
1
5
6
7
8
o
6
c
7
n
6
_
6
a
/
_
j
0
o
1
c
1
n
7
2
_
a
p
_
d
0
1
b
1
y
7
g
2
u
.
e
p
s
t
d
o
f
n
b
0
y
7
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
2
i
3
e
s
/
j
t
/
.
f
u
s
e
r
o
n
1
7
M
a
y
2
0
2
1
appeared at the learnt moments presented CNVs with
higher amplitude and were judged faster and more accu-
rately. Importantly, how well a memory was stored
influenced not only the benefit in performance but also
CNV amplitude. The CNV has been traditionally linked
to temporal expectation (Cravo et al., 2011; Praamstra,
Kourtis, Kwok, & Oostenveld, 2006; Los & Heslenfeld,
2005; Pfeuty, Ragot, & Pouthas, 2005; Nobre, 2001).
Similar to studies that investigate the CNV in tasks with
voluntary and automatic deployment of temporal attention,
we found that its amplitude and time course were strongly
related to the moment of target presentation. Once again,
the effects of long-term memory on performance seem
to mimetize the neural correlates of the voluntary deploy-
ment of attention.
Combined with our previous findings, our results empha-
size the ecological role that memories play not only in storing
information but also in predicting and preparing perception.
They cast long-term memories in a new light. Rather than
emphasizing their reflective and retroactive role of recon-
stituting, or remembering past events, they highlight the
proactive role they play in predicting and preparing per-
ception dynamically by pre-membering anticipated events.
The findings open new lines of investigation into the
mechanisms through which mnemonic temporal asso-
ciations guide perception. A fuller understanding of
human perception will require understanding of dynamic
regulation by both top–down signals from long-term
memories and short-term biases related to current goals
and expectations.
Acknowledgments
The authors acknowledge support from a Wellcome Trust
Senior Investigator Award (A. C. N.) 104571/Z/14/Z, a James S.
McDonnell Foundation Understanding Human Cognition
Collaborative Award 220020448, a European Union FP7 Marie
Curie ITN Grant (no. 606901, INDIREA), and the NIHR Oxford
Health Biomedical Research Centre. The Wellcome Centre for
Integrative Neuroimaging is supported by core funding from
the Wellcome Trust (203139/Z/16/Z). A. M. C. was supported
by the Fundação de Amparo à Pesquisa do Estado de São Paulo
Research grant 13/24889-7. The authors also wish to thank Zita
Eva Patai, Freek van Ede, and Ryszard Auksztulewicz for useful
discussions and suggestions on earlier versions of this article.
Reprint requests should be sent to André M. Cravo, Center of
Mathematics, Computation and Cognition, Federal University of
ABC, Room 544-2, Rua Santa Adélia, 166 Santo Andre, SP, Brazil
09.210-170, or via e-mail: andre.cravo@ufabc.edu.br.
REFERENCES
Brainard, D. H., & Vision, S. (1997). The psychophysics toolbox.
Spatial Vision, 10, 433–436.
Calderone, D. J., Lakatos, P., Butler, P. D., & Castellanos, F. X.
(2014). Entrainment of neural oscillations as a modifiable
substrate of attention. Trends in Cognitive Sciences, 18,
300–309.
Chun, M. M. (2000). Contextual cueing of visual attention.
Trends in Cognitive Sciences, 4, 170–178.
Cravo, A. M., Rohenkohl, G., Wyart, V., & Nobre, A. C. (2011).
Endogenous modulation of low frequency oscillations by
temporal expectations. Journal of Neurophysiology, 106,
2964–2972.
Cravo, A. M., Rohenkohl, G., Wyart, V., & Nobre, A. C. (2013).
Temporal expectation enhances contrast sensitivity by phase
entrainment of low-frequency oscillations in visual cortex.
Journal of Neuroscience, 33, 4002–4010.
Davachi, L., & DuBrow, S. (2015). How the hippocampus
preserves order: The role of prediction and context. Trends
in Cognitive Sciences, 19, 92–99.
Desimone, R., & Ducan, J. (1995). Neural mechanisms of
selective visual attention. Annual Review of Neuroscience,
18, 193–222.
Doherty, J. R., Rao, A., Mesulam, M. M., & Nobre, A. C. (2005).
Synergistic effect of combined temporal and spatial
expectations on visual attention. Journal of Neuroscience,
25, 8259–8266.
Dragoi, G., & Buzsaki, G. (2006). Temporal encoding of place
sequences by hippocampal cell assemblies. Neuron, 50,
145–157.
Eichenbaum, H. (2013). Memory on time. Trends in Cognitive
Sciences, 17, 88.
Eradath, M. K., Mogami, T., Wang, G., & Tanaka, K. (2015).
Time context of cue–outcome associations represented by
neurons in perirhinal cortex. Journal of Neuroscience, 35,
4350–4365.
Ezzyat, Y., & Davachi, L. (2014). Similarity breeds proximity:
Pattern similarity within and across contexts is related to later
mnemonic judgments of temporal proximity. Neuron, 81,
1179–1189.
Fries, P. (2015). Rhythms for cognition: Communication through
coherence. Neuron, 88, 220–235.
Giesbrecht, B., Sy, J. L., & Guerin, S. A. (2013). Both memory
and attention systems contribute to visual search for targets
cued by implicitly learned context. Vision Research, 85,
80–89.
Goldfarb, E. V., Chun, M. M., Phelps, E. A., Goldfarb, E. V.,
Chun, M. M., & Phelps, E. A. (2016). Memory-guided
attention: Independent contributions of the hippocampus
and striatum. Neuron, 89, 1–8.
Hutchinson, J. B., & Turk-Browne, N. B. (2012). Memory-guided
attention: Control from multiple memory systems. Trends in
Cognitive Sciences, 16, 576–579.
Kasper, R. W., Grafton, S. T., Eckstein, M. P., & Giesbrecht, B.
(2015). Multimodal neuroimaging evidence linking
memory and attention systems during visual search cued
by context. Annals of the New York Academy of Sciences,
1339, 176–189.
Kastner, S., & Ungerleider, L. G. (2000). Mechanisms of
visual attention in the human cortex. Annual Review of
Neuroscience, 23, 315–341.
Kunar, M. A., Flusberg, S. J., & Wolfe, J. M. (2008). Time to guide:
Evidence for delayed attentional guidance in contextual
cueing. Visual Cognition, 16, 804–825.
Lakatos, P., Karmos, G., Mehta, A. D., Ulbert, I., & Schroeder,
C. E. (2008). Entrainment of neuronal oscillations as a
mechanism of attentional selection. Science, 320, 110–113.
Los, S. A., & Heslenfeld, D. J. (2005). Intentional and
unintentional contributions to nonspecific preparation:
Electrophysiological evidence. Journal of Experimental
Psychology: General, 134, 52–72.
MacDonald, C. J., Lepage, K. Q., Eden, U. T., & Eichenbaum, H.
(2011). Hippocampal “time cells” bridge the gap in memory
for discontiguous events. Neuron, 71, 737–749.
Maris, E., & Oostenveld, R. (2007). Nonparametric statistical
testing of EEG- and MEG-data. Journal of Neuroscience
Methods, 164, 177–190.
2088
Journal of Cognitive Neuroscience
Volume 29, Number 12
D
o
w
n
l
o
a
d
e
d
f
r
o
m
l
l
/
/
/
/
j
f
/
t
t
i
t
.
:
/
/
h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
-
p
-
d
p
d
2
f
9
/
1
2
2
9
/
2
1
0
2
8
/
1
2
1
0
9
8
5
1
3
/
2
1
5
6
7
8
o
6
c
7
n
6
_
6
a
/
_
j
0
o
1
c
1
n
7
2
_
a
p
_
d
0
1
b
1
y
7
g
2
u
.
e
p
s
t
d
o
f
n
b
0
y
7
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
2
i
3
e
s
/
j
f
t
.
/
u
s
e
r
o
n
1
7
M
a
y
2
0
2
1
Nobre, A. C. (2001). Orienting attention to instants in time.
Neuropsychologia, 39, 1317–1328.
Nobre, A. C., & Rohenkohl, G. (2014). Time for the fourth
dimension in attention. In A. C. Nobre & S. Kastner (Eds.),
The Oxford handbook of attention (pp. 676–721). Oxford:
Oxford University Press.
Pfeuty, M., Ragot, R., & Pouthas, V. (2005). Relationship between
CNV and timing of an upcoming event. Neuroscience Letters,
382, 106–111.
Praamstra, P., Kourtis, D., Kwok, H. F., & Oostenveld, R. (2006).
Neurophysiology of implicit timing in serial choice reaction-
time performance. Journal of Neuroscience, 26, 5448–5455.
Reynolds, J. H., & Chelazzi, L. (2004). Attentional modulation of
visual processing. Annual Review of Neuroscience, 27, 611–647.
Rohenkohl, G., & Nobre, A. C. (2011). Alpha oscillations related
to anticipatory attention follow temporal expectations.
Journal of Neuroscience, 31, 14076–14084.
Schapiro, A., Kustner, L., & Turk-Browne, N. (2012). Shaping
of object representations in the human medial temporal
lobe based on temporal regularities. Current Biology, 22,
1622–1627.
Schwartz, G., Howard, M. W., Jing, B., & Kahana, M. J. (2005).
Shadows of the past. Psychological Science, 16, 898–904.
Stokes, M. G., Atherton, K., Patai, E. Z., & Nobre, A. C. (2012).
Long-term memory prepares neural activity for perception.
Proceedings of the National Academy of Sciences, U.S.A.,
109, E360–E367.
Summerfield, J. J., Lepsien, J. J., Gitelman, D. R., Mesulam, M.,
& Nobre, A. C. (2006). Orienting attention based on long-term
memory experience. Neuron, 49, 905–916.
Summerfield, J. J., Rao, A., Garside, N., & Nobre, A. C. (2011).
Biasing perception by spatial long-term memory. Journal of
Neuroscience, 31, 14952–14960.
Vangkilde, S., Coull, J. T., & Bundesen, C. (2012). Great
expectations: temporal expectations modulates perceptual
processing speed. Journal of Experimental Psychology:
Human Perception and Performance, 38, 1183.
von Helmholtz, H. (1867). Treatise on physiological optics 3.
Leipzig, Germany: Voss.
Zhao, J., Al-Aidroos, N., & Turk-Browne, N. B. (2013). Attention
is spontaneously biased toward regularities. Psychological
Science, 24, 667–677.
D
o
w
n
l
o
a
d
e
d
f
r
o
m
l
l
/
/
/
/
j
f
/
t
t
i
t
.
:
/
/
h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
o
d
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
-
p
-
d
p
d
2
f
9
/
1
2
2
9
/
2
1
0
2
8
/
1
2
1
0
9
8
5
1
3
/
2
1
5
6
7
8
o
6
c
7
n
6
_
6
a
/
_
j
0
o
1
c
1
n
7
2
_
a
p
_
d
0
1
b
1
y
7
g
2
u
.
e
p
s
t
d
o
f
n
b
0
y
7
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
2
i
3
e
s
/
j
/
f
.
t
u
s
e
r
o
n
1
7
M
a
y
2
0
2
1
Cravo et al.
2089
