Neural Correlates of Subsequent Memory-Related - Specialized Research AI at MIT

Neural Correlates of Subsequent Memory-Related
Gaze Reinstatement

Jordana S. Wynn1*, Zhong-Xu Liu2*, and Jennifer D. Ryan3,4

Abstract

■ Mounting evidence linking gaze reinstatement—the recapit-
ulation of encoding-related gaze patterns during retrieval—to
behavioral measures of memory suggests that eye movements
play an important role in mnemonic processing. Yet, the nature
of the gaze scanpath, including its informational content and
neural correlates, has remained in question. In this study, we
examined eye movement and neural data from a recognition
memory task to further elucidate the behavioral and neural
bases of functional gaze reinstatement. Consistent with previ-
ous work, gaze reinstatement during retrieval of freely viewed
scene images was greater than chance and predictive of recog-
nition memory performance. Gaze reinstatement was also asso-
ciated with viewing of informationally salient image regions at
encoding, suggesting that scanpaths may encode and contain

high-level scene content. At the brain level, gaze reinstatement
was predicted by encoding-related activity in the occipital pole
and BG, neural regions associated with visual processing and
oculomotor control. Finally, cross-voxel brain pattern similarity
analysis revealed overlapping subsequent memory and subse-
quent gaze reinstatement modulation effects in the parahippo-
campal place area and hippocampus, in addition to the occipital
pole and BG. Together, these findings suggest that encoding-
related activity in brain regions associated with scene process-
ing, oculomotor control, and memory supports the formation,
and subsequent recapitulation, of functional scanpaths. More
broadly, these findings lend support to Scanpath Theory’s asser-
tion that eye movements both encode, and are themselves em-
bedded in, mnemonic representations. ■

INTRODUCTION

The human visual field is limited, requiring us to move our
eyes several times a second to explore the world around
us. This necessarily sequential process of selecting visual
features for fixation and further processing has important
implications for memory. Research using eye movement
monitoring indicates that, during visual exploration, fixa-
tions and saccades support the binding of salient visual
features and the relations among them into coherent
and lasting memory traces (e.g., Liu, Rosenbaum, &
Ryan, 2020; Liu, Shen, Olsen, & Ryan, 2017; for a review,
see Wynn, Shen, & Ryan, 2019). Moreover, such memory
traces may be stored and subsequently recapitulated as
patterns of eye movements or “scanpaths” at retrieval
(Noton & Stark, 1971a, 1971b; for a review, see Wynn
et al., 2019). Specifically, when presented with a previously
encoded stimulus or a cue to retrieve a previously en-
coded stimulus from memory, humans (and nonhuman
primates; see Sakon & Suzuki, 2019) spontaneously repro-
duce the scanpath enacted during encoding (i.e., gaze
reinstatement), and this reinstatement is predictive of
mnemonic performance across a variety of tasks (e.g.,
Wynn, Ryan, & Buchsbaum, 2020; Damiano & Walther,

1Harvard University, 2University of Michigan-Dearborn,
3Rotman Research Institute at Baycrest Health Sciences,
4University of Toronto
*Equal contribution.

2019; Wynn, Olsen, Binns, Buchsbaum, & Ryan, 2018;
Scholz, Mehlhorn, & Krems, 2016; Laeng, Bloem,
D’Ascenzo, & Tommasi, 2014; Olsen, Chiew, Buchsbaum,
& Ryan, 2014; Johansson & Johansson, 2013; Foulsham
et al., 2012; Laeng & Teodorescu, 2002; for a review, see
Wynn et al., 2019). Although there is now considerable
evidence supporting a link between gaze reinstatement
(i.e., reinstatement of encoding gaze patterns during re-
trieval) and memory retrieval, investigations regarding the
neural correlates of this effect are recent and few (see Bone
et al., 2019; Ryals, Wang, Polnaszek, & Voss, 2015), and no
study to date has investigated the patterns of neural activity
at encoding that predict subsequent gaze reinstatement.
Thus, to further elucidate the link between eye movements
and memory at the neural level, this study used concurrent
eye movement monitoring and fMRI to investigate the
neural mechanisms at encoding that predict functional gaze
reinstatement (i.e., gaze reinstatement that supports mne-
monic performance) at retrieval, in the vein of subsequent
memory studies (e.g., Brewer, Zhao, Desmond, Glover, &
Gabrieli, 1998; Wagner et al., 1998; for a review, see
Hannula & Duff, 2017).

Scanpaths have been proposed to at once contain, and
support the retrieval of, spatiotemporal contextual infor-
mation (Noton & Stark, 1971a, 1971b). According to
Noton and Stark’s (1971a, 1971b) seminal Scanpath
Theory, on which much of the current gaze reinstate-
ment literature is based (see Wynn et al., 2019),

Journal of Cognitive Neuroscience 34:9, pp. 1547–1562
https://doi.org/10.1162/jocn_a_01761

D
o
w
n
o
a
d
e
d

f
r
o
m
h

t
t

:
/
/

d
i
r
e
c
t
.

i
t
.

e
d
u

/
j

o
c
n
a
r
t
i
c
e
–
p
d

f
/

3
4
9
1
5
4
7
2
0
3
7
4
5
1

/
j

o
c
n
_
a
_
0
1
7
6
1
p
d

b
y
g
u
e
s
t

o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3

scanpaths consist of both image features and the fixations
made to them as “an alternating sequence of sensory and
motor memory traces.” Consistent with this proposal, re-
search using eye movement monitoring and neuroimag-
ing techniques has established an important role for eye
movements in visual memory encoding (for a review, see
Ryan, Shen, & Liu, 2020; Meister & Buffalo, 2016). For ex-
ample, at the behavioral level, recognition memory accu-
racy is significantly attenuated when eye movements
during encoding are restricted (e.g., to a central fixation
cross) as opposed to free (e.g., Liu et al., 2020; Damiano
& Walther, 2019; Henderson, Williams, & Falk, 2005). At
the neural level, restricting viewing to a fixed location
during encoding results in attenuated activity in brain
regions associated with memory and scene processing
including the hippocampus (HPC) and parahippocampal
place area (PPA), as well as reduced functional connectiv-
ity between these regions and other cortical regions (Liu
et al., 2020). When participants are free to explore, how-
ever, the number of fixations executed is positively pre-
dictive of subsequent memory performance (e.g.,
Fehlmann et al., 2020; Liu et al., 2017; Olsen et al., 2016;
Loftus, 1972) and of activity in the HPC (Liu et al., 2017,
2020; see also Olsen et al., 2016) and medial temporal lobe
(Fehlmann et al., 2020), suggesting that eye movements are
critically involved in the accumulation and encoding of
visual feature information into lasting memory traces.
That the relationships between gaze fixations and recogni-
tion memory performance (e.g., Wynn, Buchsbaum &
Ryan, 2021; see also Chan, Chan, Lee, & Hsiao, 2018) and
between gaze fixations and HPC activity (Liu, Shen, Olsen,
& Ryan, 2018) are reduced with age, despite an increase in
the number of fixations (e.g., Firestone, Turk-Browne, &
Ryan, 2007; Heisz & Ryan, 2011), further suggests that these
effects extend beyond the effects of mere attention or
interest.

Recent work suggests that eye movements not only play
an important role in memory encoding but also actively
support memory retrieval. Consistent with the Scanpath
Theory, several studies have provided evidence that gaze
patterns elicited during stimulus encoding are recapitu-
lated during subsequent retrieval and are predictive of
mnemonic performance (e.g., Wynn et al., 2018, 2020;
Damiano & Walther, 2019; Scholz et al., 2016; Laeng
et al., 2014; Olsen et al., 2014; Johansson & Johansson,
2013; Foulsham et al., 2012; Laeng & Teodorescu, 2002;
for a review, see Wynn et al., 2019). In addition to
advancing a functional role for eye movements in mem-
ory retrieval, this literature has raised intriguing questions
regarding the nature of the scanpath and its role in mem-
ory. For example, how are scanpaths created, and what
information do they contain? To answer these questions,
it is necessary not only to relate eye movement and be-
havioral patterns, as prior research has done, but also,
and perhaps more critically, to relate eye movement
and neural patterns. Yet, only two studies, to our knowl-
edge, have directly investigated the neural correlates of

gaze reinstatement, with both focusing on retrieval-related
activity patterns. In the first of these studies, Ryals et al.
(2015) demonstrated that trial-level variability in gaze sim-
ilarity (between previously viewed scenes and novel scenes
with similar feature configurations) was associated with
activity in the right HPC. Extending this work, Bone et al.
(2019) observed that gaze reinstatement (i.e., similarity be-
tween participant- and image-specific gaze patterns during
encoding and subsequent visualization) was positively cor-
related with whole-brain neural reinstatement (i.e., similar-
ity between image-specific patterns of brain activity evoked
during encoding and subsequent visualization) during a
visual imagery task. Considered together, these two studies
provide evidence that functional gaze reinstatement is re-
lated to neural activity patterns typically associated with
memory retrieval, suggesting a common mechanism.

Although there is now some evidence that mnemonic
retrieval processes support gaze reinstatement at the
neural level, the relationship between gaze reinstatement
and encoding-related neural activity has yet to be investi-
gated. Accordingly, this study used the data from Liu et al.
(2020) to elucidate the encoding mechanisms that
support the formation and subsequent recapitulation of
functional scanpaths. Participants encoded intact and
scrambled scenes under free or fixed (restricted) viewing
conditions (in the scanner) and subsequently completed a
recognition memory task with old (i.e., encoded) and new
(i.e., lure) images (outside the scanner). Previous analysis
of this data revealed that, when compared to free viewing,
restricting eye movements reduced activity in the HPC,
connectivity between the HPC and other visual and
memory regions, and, ultimately, subsequent memory
performance (Liu et al., 2020). These findings critically
suggest that eye movements and memory encoding are
linked at both the behavioral and neural levels. Here, we
extend this work further by investigating the extent to
which the patterns of eye movements, or scanpaths, that
are created at encoding are reinstated at retrieval to
support memory performance and also by investigating
the neural activity at encoding that predicts the subse-
quent reinstatement of scanpaths at retrieval.

To this end, we first computed the spatial similarity
between encoding and retrieval scanpaths (containing
information about fixation location and duration) and
used this measure to predict recognition memory accu-
racy. On the basis of prior evidence of functional gaze
reinstatement, we predicted that gaze reinstatement
would be both greater than chance and positively corre-
lated with recognition of old images. To further interro-
gate the nature of information represented in the
scanpath, we additionally correlated gaze reinstatement
with measures of visual (i.e., stimulus-driven; bottom–up)
and informational (i.e., participant-driven; bottom–up and
top–down) saliency. Given that prior work has revealed a
significant role for top–down features (e.g., meaning,
Henderson & Hayes, 2018; scene content, O’Connell &
Walther, 2015) in guiding eye movements, above and

1548

Journal of Cognitive Neuroscience

Volume 34, Number 9

D
o
w
n
o
a
d
e
d

f
r
o
m
h

t
t

:
/
/

d
i
r
e
c
t
.

i
t
.

e
d
u

/
j

o
c
n
a
r
t
i
c
e
–
p
d

f
/

3
4
9
1
5
4
7
2
0
3
7
4
5
1

/
j

o
c
n
_
a
_
0
1
7
6
1
p
d

b
y
g
u
e
s
t

o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3

beyond bottom–up image features (e.g., luminance, con-
trast; Itti & Koch, 2000), we hypothesized that gaze rein-
statement would be related particularly to the viewing of
informationally salient image regions. Finally, to uncover
the neural correlates of functional gaze reinstatement,
we analyzed neural activity patterns at encoding, both
across the whole brain and in memory-related ROIs (i.e.,
HPC, PPA; see Liu et al., 2020), to identify brain regions
that (1) predicted subsequent gaze reinstatement at
retrieval and (2) showed overlapping subsequent gaze
reinstatement and subsequent memory effects. Given that
previous work has linked gaze scanpaths, as a critical
component of mnemonic representations, to successful
encoding and retrieval, we hypothesized that functional
gaze reinstatement would be supported by encoding-
related neural activity in brain regions associated with
visual processing (i.e., ventral visual stream regions) and
memory (i.e., medial temporal lobe regions). By linking
the neural correlates and behavioral outcomes of gaze
reinstatement, this study provides novel evidence in
support of Noton and Stark’s assertion that scanpaths
both serve to encode and are themselves encoded into
memory, allowing them to facilitate retrieval via recapitu-
lation and reactivation of informationally salient image
features.

METHODS

Participants
Participants were 36 young adults (22 women) aged 18–
35 years (M = 23.58 years, SD = 4.17) with normal or

corrected-to-normal vision and no history of neurological
or psychiatric disorders. All participants were recruited
from the University of Toronto and surrounding
Toronto area community and were given monetary com-
pensation for their participation in the study. All partici-
pants provided written informed consent in accordance
with the Research Ethic Board at the Rotman Research
Institute at Baycrest Health Sciences.

Stimuli

Stimuli consisted of 864, 500 × 500-pixel, colored images,
made up of 24 images of each of 36 semantic scene cate-
gories (e.g., living room, arena, warehouse), varying along
the feature dimensions of size and clutter (six levels per
dimension = 36 unique feature level combinations, bal-
anced across conditions).1 Within each scene category,
eight images were assigned to the free-viewing encoding
condition and eight images were assigned to the fixed-
viewing encoding condition; images were randomly as-
signed to eight fMRI encoding runs (36 images per run
per viewing condition). The remaining eight images in
each scene category were used as novel lures at retrieval.
One hundred forty-four scene images from encoding (72
images per viewing condition from two randomly selected
encoding runs) and 72 scene images from retrieval (two
per scene category) were scrambled using six levels of tile
size (see Figure 1). Thus, in total, 432 intact scene images
and 144 scrambled color-tile images were viewed at
encoding, balanced across free- and fixed-viewing condi-
tions, and 648 intact scene images (432 old and 216 novel

Figure 1. (A) Visualization of the experimental procedure for the in-scan encoding task. Before each trial, a green or red fixation cross was presented
on the screen indicating whether participants would be required to freely view (green) or maintain fixation (red) during presentation of the
upcoming image. Note that although fixations are presented centrally here, during the experiment, they were presented in a random location within
a 100-pixel radius around the center of the screen. Participants completed six runs of scenes and two runs of scrambled color-tile images, consisting
of 72 images each. (B) Visualization of the gaze reinstatement analysis, with one example each of a high similarity score and a low similarity score.
Reinstatement (i.e., similarity) scores reflect the spatial overlap between patterns of fixations (defined by location and duration) corresponding to
the same image viewed by the same participant during encoding and retrieval, controlling for image-invariant (idiosyncratic) viewing biases (e.g.,
center bias).

Wynn, Liu, and Ryan

1549

D
o
w
n
o
a
d
e
d

f
r
o
m
h

t
t

:
/
/

d
i
r
e
c
t
.

i
t
.

e
d
u

/
j

o
c
n
a
r
t
i
c
e
–
p
d

f
/

3
4
9
1
5
4
7
2
0
3
7
4
5
1

/
j

o
c
n
_
a
_
0
1
7
6
1
p
d

b
y
g
u
e
s
t

o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3

lure) and 216 scrambled color-tile images (144 old and 72
novel lure) were viewed at retrieval. All images were bal-
anced for low-level image properties (e.g., luminance,
contrast)2 and counterbalanced across participants (for
assignment to experimental/stimulus conditions).

Procedure

In-scan Scene Encoding Task

Participants completed eight encoding runs in the scan-
ner, six containing scene images and two containing
scrambled images (run order was randomized within par-
ticipants3; see Figure 1A). Within each run, participants
viewed 72 images, half of which were studied under
free-viewing instructions and half of which were studied
under fixed-viewing instructions. Before the start of each
trial, participants were presented with a fixation cross for
1.72–4.16 sec (exponential distribution, M = 2.63 sec)
presented in a random location within a 100-pixel
(1.59° visual angle) radius around the center of the
screen. The color of the cross indicated the viewing in-
structions for the following image, with free viewing indi-
cated by a green cross and fixed viewing indicated by a
red cross. After presentation of the fixation cross, a scene
or scrambled image appeared for 4 sec, during which
time participants were instructed to encode as much in-
formation as possible. If the image was preceded by a red
cross, participants were to maintain fixation on the loca-
tion of the cross for the duration of image presentation.
The length of each run was 500 sec, with 10 and 12.4 sec
added to the beginning and end of the run, respectively.

Postscan Scene Recognition Task

After the encoding task, participants were given a 60-min
break before completing the retrieval task in a separate
testing room. For the retrieval task, participants viewed
all 576 images (432 scene images and 144 scrambled
color-tile images) from the encoding task along with 288
novel lure images (216 scene images and 72 scrambled
color-tile images), divided evenly into six blocks. Before
the start of each trial, participants were presented with a
fixation cross for 1.5 sec presented in a random location
within a 100-pixel radius around the center of the screen
(for old trials, the fixation cross was presented at the same
location in which it was presented during the encoding
task). After presentation of the fixation cross, a scene or
scrambled image (either old, i.e., previously viewed during
encoding, or novel lure) appeared for 4 sec. Participants
were given 3 sec to indicate whether the presented image
was “old” or “new” and rate their confidence in that
response, via keypress (z = high confidence “old,” x =
low confidence “old,” n = high confidence “new,” m =
low confidence “new”). To quantify recognition memory
for old images, points were assigned to each response as
follows: z = 2, x = 1, m = 0, and n = −1.

Eye-tracking Procedure

During the encoding task, monocular eye movements
were recorded inside the scanner using the Eyelink
1000 MRI-compatible remote eye tracker with a 1000-Hz
sampling rate (SR Research Ltd.). The eye tracker was
placed inside the scanner bore (behind the participant’s
head) and detected the pupil and corneal reflection via a
mirror mounted on the head coil. During the retrieval
task, monocular eye movements were recorded using
the Eyelink II head-mounted eye tracker with a 500-Hz
sampling rate (SR Research Ltd.). To ensure successful
tracking during both the encoding and retrieval tasks,
9-point calibration was performed before the start of the
task. Online manual drift correction to the location of the
upcoming fixation cross was performed between trials
when necessary. As head movements were restricted in
the scanner, drift correction was rarely performed.
Saccades greater than 0.5° of visual angle were identified
by Eyelink as eye movements having a velocity threshold
of 30°/sec, an acceleration threshold of 8000°/sec, and a
saccade onset threshold of 0.15°. Blinks were defined as
periods in which the saccade signal was missing for three
or more consecutive samples. All remaining samples (not
identified as a saccade or blink) were classified as
fixations.

MRI Protocol

As specified in Liu et al. (2020), a 3-T Siemens MRI scanner
with a standard 32-channel head coil was used to acquire
both structural and functional images. For structural
T1-weighted high-resolution MRI images, we used a stan-
dard 3-D magnetization prepared rapid gradient echo pulse
sequence with170 slices and using field of view = 256 ×
256 mm, 192 × 256 matrix, 1-mm isotropic resolution, echo
time/repetition time = 2.22/200 msec, flip angle = 9°, and
scan time = 280 sec. Functional images were obtained
using T2*-weighted EPI acquisition protocol with repetition
time = 2000 msec, echo time = 27 msec, flip angle = 70°,
and field of view = 192 × 192 with 64 × 64 matrix (3 mm ×
3 mm in-place resolution, slice thickness = 3.5 mm with
no gap). Two hundred fifty volumes were acquired for
each run. Both structural and functional images were
acquired in an oblique orientation 30° clockwise to the
AC–PC axis. Stimuli were presented with Experiment
Builder (SR Research Ltd.) back-projected to a screen
(projector resolution: 1024 × 768) and viewed with a
mirror mounted on the head coil.

Data Analysis

Gaze Reinstatement Analysis

To quantify the spatial overlap between the gaze patterns
elicited by the same participants viewing the same images
during encoding and retrieval, we computed gaze rein-
statement scores for each image for each participant.

1550

Journal of Cognitive Neuroscience

Volume 34, Number 9

D
o
w
n
o
a
d
e
d

f
r
o
m
h

t
t

:
/
/

d
i
r
e
c
t
.

i
t
.

e
d
u

/
j

o
c
n
a
r
t
i
c
e
–
p
d

f
/

3
4
9
1
5
4
7
2
0
3
7
4
5
1

/
j

o
c
n
_
a
_
0
1
7
6
1
p
d

b
y
g
u
e
s
t

o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3

Specifically, we computed the Fisher z-transformed
Pearson correlation between the duration-weighted fixa-
tion density (i.e., heat) map4 (σ = 80) for each image
for each participant during encoding and the correspond-
ing density map for the same image being viewed by
the same participant during retrieval (“match” similarity;
R eyesim package: https://github.com/ bbuchsbaum
/eyesim; see Figure 1B). Critically, although this measure
(“match” similarity) captures the overall similarity
between encoding and retrieval gaze patterns, it is possi-
ble that such similarity reflects participant-specific (e.g.,
tendency to view each image from left to right), image-
invariant (e.g., tendency to preferentially view the center
of the screen) viewing biases. Thus, to control for idiosyn-
cratic viewing tendencies (which were not of particular
interest for this study), we additionally computed the sim-
ilarity between participant- and image-specific retrieval
density maps and 50 other randomly selected encoding
density maps (within participant, stimulus type, and view-
ing condition). The resulting 50 scores were averaged to
yield a single “mismatch” similarity score for each partici-
pant for each image.

Match and mismatch similarity scores were contrasted
using an ANOVA with Similarity Value as the dependent
variable and Stimulus Type (scene, scrambled), Viewing
Condition (free, fixed), and Similarity Template (match,
mismatch) as the independent variables. For all subse-
quent analyses, gaze reinstatement was reported as the
difference between match and mismatch similarity
scores, thus reflecting the spatial similarity between en-
coding and retrieval scanpaths for the same participant
viewing the same image, controlling for idiosyncratic
viewing biases.

To investigate the effect of gaze reinstatement on mne-
monic performance, we ran a linear mixed effects model
(LMEM) on trial-level accuracy (coded for a linear effect:
high confidence miss = −1, low confidence miss = 0,
low confidence hit = 1, high confidence hit = 2) with fixed
effects including all interactions of gaze reinstatement
(match similarity − mismatch similarity; z scored), stimu-
lus type (scene*, scrambled), and viewing condition (free*,
fixed) as well as random effects including random inter-
cepts for participant and image. Backward model com-
parison (α = .05) was used to determine the most
parsimonious model ( p values approximated with the
lmerTest package; Kuznetsova, Brockhoff, & Christensen,
2017).

Saliency Analysis

To characterize gaze patterns at encoding, and specifically,
the type of information encoded into the scanpath, sa-
liency was computed for each image using two methods.
First, we used a leave-one-subject-out cross-validation
procedure to generate duration-weighted informational
saliency (participant data-driven) maps5 for each image
using the aggregated fixations of all participants

(excluding the participant in question) viewing that
image during encoding (mean number of fixations per
image = 204, aggregated from all included participants).
Second, we used the Saliency Toolbox ( Walther & Koch,
2006) to generate visual saliency maps by producing 204
pseudo-fixations for each image based on low-level image
properties including color, intensity, and orientation.
Critically, whereas the stimulus (Saliency Toolbox)-driven
saliency map takes into account primarily bottom–up
stimulus features (e.g., luminance, contrast), the partici-
pant data-driven saliency map takes into account any fea-
tures ( bottom–up or top–down) that might attract
viewing for any reason (e.g., semantic meaning, memory).
To quantify the extent to which individual gaze patterns
during encoding were guided by salient bottom–up and
top–down features, participant- and image-specific en-
coding gaze patterns were correlated with both the infor-
mational (participant data-driven) and visual (stimulus
[Saliency Toolbox]-driven) saliency maps in the same
manner as the gaze reinstatement analysis described
above. This analysis yielded two scores per participant
per image reflecting the extent to which fixations at
encoding were guided by high-level image features
(i.e., informational saliency; based on the data-driven
saliency map) and low-level image features (i.e., visual
saliency; based on the stimulus-driven saliency map).

To investigate the relationship between encoding gaze
patterns and gaze reinstatement, we ran an LMEM on
gaze reinstatement with visual and informational saliency
scores (z scored) as predictors. To compare the strength
of each saliency score in predicting gaze reinstatement,
saliency scores were dummy coded (visual saliency = 0,
informational saliency = 1). Random intercepts for partic-
ipant and image were also included in the model.

fMRI data preprocessing. The fMRI preprocessing pro-
cedure was previously reported in Liu et al. (2020); for
completeness, it is re-presented here. MRI images were
processed using SPM12 (Statistical Parametric Mapping,
Welcome Trust Center for Neuroimaging, University
College London; www.fil.ion.ucl.ac.uk/spm/software
/spm12/ Version: 7487) in the MATLAB environment
(The MathWorks, Inc.). Following the standard SPM12
preprocessing procedure, slice timing was first corrected
using sinc interpolation with the midpoint slice as the
reference slice. Then, all functional images were aligned
using a six-parameter linear transformation. Next, for
each participant, functional image movement parameters
obtained from the alignment procedure, as well as the
global signal intensity of these images, were checked
manually using the freely available toolbox ART (www
.nitrc.org/projects/artifact_detect/) to detect volumes
with excessive movement and abrupt signal changes.
Volumes indicated as outliers by ART default criteria were
excluded later from statistical analyses. Anatomical
images were coregistered to the aligned functional im-
ages and segmented into white matter, gray matter,

Wynn, Liu, and Ryan

1551

D
o
w
n
o
a
d
e
d

f
r
o
m
h

t
t

:
/
/

d
i
r
e
c
t
.

i
t
.

e
d
u

/
j

o
c
n
a
r
t
i
c
e
–
p
d

f
/

3
4
9
1
5
4
7
2
0
3
7
4
5
1

/
j

o
c
n
_
a
_
0
1
7
6
1
p
d

b
y
g
u
e
s
t

o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3

cerebrospinal fluid, skull/bones, and soft tissues using
SPM12 default six-tissue probability maps. These seg-
mented images were then used to calculate the transforma-
tion parameters mapping from the individuals’ native space
to the Montreal Neurological Institute (MNI) template
space. The resulting transformation parameters were used
to transform all functional and structural images to the MNI
template. For each participant, the quality of coregistration
and normalization was checked manually and confirmed by
two research assistants. The functional images were finally
resampled at a 2 × 2 × 2 mm resolution and smoothed
using a Gaussian kernel with an FWHM of 6 mm. The first
five fMRI volumes from each run were discarded to allow
the magnetization to stabilize to a steady state, resulting
in 245 volumes in each run.

fMRI Analysis

Parametric modulation analysis. To interrogate our
main research question, that is, which brain regions’ ac-
tivity during encoding was associated with subsequent
gaze reinstatement, we conducted a parametric modula-
tion analysis in SPM12. Specifically, we first added the
condition mean activation regressors for the free- and
fixed-viewing conditions, by convolving the onset of trials
of each condition with the canonical hemodynamic re-
sponse function in SPM12. We then added the trial-wise
gaze reinstatement measure as our interested linear mod-
ulator, which was also convolved with the hemodynamic
response function. We also added motion parameters, as
detailed in Liu et al. (2020), as regressors of no interest.
Default high-pass filters with a cutoff of 128 sec and a
first-order autoregressive model AR(1) were also applied.
Using this design matrix, we first estimated the modula-
tion effect of gaze reinstatement at the individual level.
These beta estimates, averaged across all scene runs, were
then carried to the group-level analyses in which within-
participant t tests were used to examine which brain
regions showed stronger activity when greater gaze rein-
statement was observed. For this analysis, we primarily fo-
cused on the free-viewing scene condition as this is the
condition in which the gaze reinstatement measure is
most meaningful (because participants were allowed to
freely move their eyes). In this analysis, the HPC and
PPA served as our a priori ROIs (see Supplementary
Figure S1 in Liu et al., 2020). As specified in Liu et al.
(2020), the HPC ROI for each participant was obtained
using Freesurfer recon-all function, Version 6.0 (surfer
.nmr.mgh.harvard.edu.myaccess.library.utoronto.ca;
Fischl, 2012). The PPA ROIs were obtained using the
“scene versus scrambled color tile” picture contrast. The
MNI coordinates for the peak activation of the PPA were
[32, −34, −18] for the right PPA and [−24, −46, −12]
for the left PPA. The left and right PPA ROIs contained
293 and 454 voxels, respectively.

To explore whether other brain regions showed gaze
reinstatement modulation effects, in addition to the

ROI analysis, we also obtained voxel-wise whole-brain re-
sults. As an exploratory analysis, we used a relatively le-
nient threshold of p = .005 with a 10-voxel extension
(no correction), which can also facilitate future meta-
analyses (Lieberman & Cunningham, 2009).

Brain activation pattern similarity between parametric
modulation of gaze reinstatement and subsequent
memory. To understand the extent to which there
was similar modulation of brain activity by gaze reinstate-
ment and by subsequent memory, we calculated cross-
voxel brain activation pattern similarity between the
two parametric modulation effects. This analysis allowed
us to test whether the brain activity associated with the
two behavioral variables (i.e., trial-wise gaze reinstate-
ment and subsequent memory) shares a similar pattern.
First, we obtained subsequent memory modulation ef-
fects as detailed in Liu et al. (2020). Specifically, in this
subsequent memory effect analysis, we coded subse-
quent recognition memory for each encoding trial based
on participants’ hit/miss response and confidence (cor-
rect recognition with high confidence = 2, correction
recognition with low confidence = 1, missed recognition
with low confidence = 0, missed recognition with high
confidence = −1). We then used this measure as a linear
parametric modulator to find brain regions that showed a
subsequent memory effect, that is, stronger activation
when trials were subsequently better remembered. We
averaged the subsequent memory effect estimates across
runs for each participant. We then extracted unthresh-
olded voxel-by-voxel subsequent memory effects and
gaze reinstatement effects (i.e., estimated betas) for the
HPC and PPA, separately. These beta values were then
vectorized, and Pearson correlations were calculated
between the two vectors of the two modulation effects
for each ROI. Finally, these Pearson correlations were
Fisher z transformed to reflect the cross-voxel pattern
similarity between the subsequent memory effect and
the gaze reinstatement modulation effect.

Although we mainly focused on the brain activation
pattern similarity between the two modulation effects
in the free-viewing scene condition, we also obtained
the same measure for the fixed-viewing scene condition
to provide a control condition. If the brain activation pat-
tern modulated by the gaze reinstatement measure is re-
lated to memory processing in the free-viewing scene
condition, it should show larger-than-zero pattern simi-
larity with the subsequent memory effects, which should
also be greater than those in the fixed-viewing scene con-
dition. Therefore, at the group level, we used one-sample
t tests to examine whether the similarity z scores in the
free-viewing scene condition were larger than zero and
used a paired t test to compare the similarity scores
against those in the fixed-viewing scene condition.

In addition to the ROI brain activation pattern similarity,
we also examined brain activation similarity between
subsequent memory and gaze reinstatement for the

1552

Journal of Cognitive Neuroscience

Volume 34, Number 9

D
o
w
n
o
a
d
e
d

f
r
o
m
h

t
t

:
/
/

d
i
r
e
c
t
.

i
t
.

e
d
u

/
j

o
c
n
a
r
t
i
c
e
–
p
d

f
/

3
4
9
1
5
4
7
2
0
3
7
4
5
1

/
j

o
c
n
_
a
_
0
1
7
6
1
p
d

b
y
g
u
e
s
t

o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3

whole brain in each voxel using a searchlight analysis (The
Decoding Toolbox v3.997; Hebart, Görgen, & Haynes,
2015). Specifically, for each voxel, we applied an 8-mm
spheric searchlight to calculate the across-voxel (voxels in-
cluded in this searchlight) brain activation pattern similar-
ity between the subsequent memory effect and the gaze
reinstatement modulation effect, using the same proce-
dure detailed above for the ROI analysis. We first generat-
ed the brain activation similarity z-score images for the
free- and fixed-viewing scene conditions separately for
each participant. At the group level, the individual partic-
ipants’ brain activation similarity z-score images were test-
ed against zero for the free-viewing scene condition and
compared to the similarity images in the fixed-viewing
scene condition using paired t tests. For this whole-brain
voxel-wise analysis, we used a threshold of p = .005 with a
10-voxel extension (uncorrected; see Lieberman &
Cunningham, 2009).

RESULTS

Behavioral Results

Results of the ANOVA on recognition memory perfor-
mance are reported in Liu et al., 2020. In short, a significant
interaction of Stimulus Type × Viewing Condition indi-
cated that recognition memory was significantly higher
in the free-viewing condition than in the fixed-viewing
condition, for scene images only, and for scenes relative
to scrambled images, for free-viewing only (see Figure 2E
in Liu et al., 2020).

Eye Movement Results

To determine whether gaze reinstatement was significantly
greater than chance, we ran an ANOVA with Similarity Value
as the dependent variable and Stimulus Type (scene, scram-
bled), Viewing Condition (free, fixed), and Similarity
Template (match, mismatch) as the independent variables.
If individual retrieval gaze patterns are indeed image

specific, they should be more similar to the gaze pattern
for the same image viewed at encoding (match) than for
other images within the same participant, image category,
and condition (mismatch). Results of the ANOVA revealed a
significant three-way interaction of Similarity Template,
Stimulus Type, and Viewing Condition, F(1, 34) = 7.09,
p = .012, ηp
2 = .17. Post hoc tests of the difference in mean
match and mismatch similarity scores indicated that match
similarity was significantly greater than mismatch similarity
in all conditions and categories [fixed scene: t(69.7) = 2.12,
p = .037; fixed scrambled: t(69.7) = 3.60, p = .001; free
scene: t(69.7) = 6.22, p < .001, see Figure 2A; free scram- bled: t(69.7) = 4.583, p < .001]. To explore the relationship between gaze reinstatement and mnemonic performance, we ran an LMEM on trial- level accuracy with interactions of gaze reinstatement (match similarity − mismatch similarity), stimulus type (scene*, scrambled), and viewing condition (free*, fixed) as fixed effects as well as participant and image as random effects. Results of the final best fit model indicated that accuracy was significantly greater for scenes relative to scrambled images (β = −0.24, SE = 0.03, t = −8.19, p < .001), and this effect was significantly attenuated for fixed viewing (Stimulus Type × Viewing Condition: β = 0.17, SE = 0.03, t = 5.10, p < .001). Accuracy was also sig- nificantly greater for free relative to fixed viewing (β = −0.17, SE = 0.16, t = −10.56, p < .001; see Figure 2A), and this effect was significantly attenuated for scrambled images (see Stimulus Type × Viewing Condition). Finally, the model revealed a significant positive effect of gaze reinstatement on accuracy (β = 0.06, SE = 0.01, t = 5.28, p < .001; see Figure 2B) for free-viewing scenes, and this effect was significantly attenuated for fixed viewing (Gaze Reinstatement × Viewing Condition: β = −0.04, SE = 0.14, t = −2.82, p = .005) and for scrambled images (Gaze Reinstatement × Stimulus Type: β = −0.06, SE = 0.16, t = −3.53, p < .001). The addition of number of gaze fixations to the model significantly improved the model fit (χ2 = 15.52, p < .001; see also Liu et al., 2020) but importantly did not abolish the effect of gaze Figure 2. Visualization of gaze reinstatement effect for free viewing of scenes. (A) Match similarity versus mismatch similarity scores. (B) Gaze reinstatement (match similarity − mismatch similarity) scores as a function of recognition memory accuracy. Sim = similarity; Hi Conf = high confidence; Lo Conf = low confidence. Wynn, Liu, and Ryan 1553 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 9 1 5 4 7 2 0 3 7 4 5 1 / / j o c n _ a _ 0 1 7 6 1 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 reinstatement. Furthermore, a correlation of mean gaze reinstatement scores and mean cumulative encoding gaze fixations was nonsignificant (r = .049, p = .79), sug- gesting that these effects were independent. To determine whether gaze reinstatement (i.e., the ex- tent to which encoding gaze patterns were recapitulated at retrieval) was related to gaze patterns (i.e., the types of information viewed) at encoding, we derived two mea- sures to capture the extent to which individual gaze pat- terns at encoding reflected “salient” image regions. Given that “saliency” can be defined by both bottom–up (e.g., bright) and top–down (e.g., meaningful) image features, with the latter generally outperforming the former in pre- dictive models (e.g., Henderson & Hayes, 2018; O’Connell & Walther, 2015), we computed two saliency maps for each image using the Saliency Toolbox (visual saliency map, reflecting bottom–up stimulus features) and aggregated participant data (informational saliency map, reflecting bottom–up and top–down features). Gaze patterns for each participant for each image were compared to both the visual and informational saliency maps, yielding two saliency scores. To probe the relation- ship between encoding gaze patterns and subsequent gaze reinstatement, we ran an LMEM on gaze reinstate- ment with saliency scores (visual*, informational) as fixed effects and participant and image as random effects. Results of the model revealed a significant effect of saliency on controlled gaze reinstatement (β = 0.10, SE = 0.01, t = 9.36, p < .001), indicating that similarity of individual encoding gaze patterns to the visual saliency map predicted subsequent gaze reinstatement at retrieval. Notably, the saliency effect was significantly increased when the informa- tional saliency map was used in place of the visual saliency map (β = 0.06, SE = 0.01, t = 5.01, p < .001), further indicating that gaze reinstatement is best predicted by encoding gaze patterns that prioritize “salient” image regions, being regions high in bottom–up and/or top–down informational content. fMRI Results To answer our main research question regarding the neu- ral activity patterns at encoding that predict subsequent gaze reinstatement (at retrieval), we first examined the brain regions in which activations during encoding were modulated by trial-wise subsequent gaze reinstatement scores (i.e., brain regions that showed stronger activation for trials with higher subsequent gaze reinstatement). Our ROI analyses did not yield significant effects for ei- ther the HPC or PPA (t = −0.31–1.13, p = .76–.26; Figure 3A). However, as evidenced by the whole-brain voxel-wise results (Figure 3B), the occipital poles bilater- ally showed a parametric modulation by subsequent gaze reinstatement at p = .005, with a 10-voxel extension (no correction). Two clusters in the BG also showed effects at this threshold. All regions that showed gaze reinstate- ment modulation effects at this threshold are presented in Table 1. As reported previously by Liu et al. (2020; see Figure 6A), both the PPA and HPC showed a parametric modulation by subsequent memory; that is, the PPA and HPC were activated more strongly for scenes that were later successfully recognized versus forgotten. Although PPA and HPC activation at the mean level were not modulated by subsequent gaze reinstatement, we investigated whether the variation across voxels within each ROI in supporting subsequent memory was similar to the varia- tion of these voxels in supporting subsequent gaze rein- statement. Critically, this cross-voxel brain modulation pattern similarity analysis can reveal whether the pattern of activation of voxels in an ROI contains shared informa- tion, or supports the overlap, between subsequent mem- ory and subsequent gaze reinstatement effects. Results of this analysis revealed significant pattern similarity be- tween the two modulation effects, subsequent memory and gaze reinstatement, in both the right PPA and right HPC, t = 2.37 and 3.31, and p = .024 and .002, respectively. Figure 3. Brain activation predicted by subsequent gaze reinstatement. (A) ROI analysis revealed no significant gaze reinstatement modulation effects for HPC and PPA (all ps > .05). (B) Voxel-wise whole-brain results for gaze reinstatement modulation (thresholded at p = .005, 10-voxel
extension, no corrections).

1554

Journal of Cognitive Neuroscience

Volume 34, Number 9

D
o
w
n
o
a
d
e
d

f
r
o
m
h

t
t

:
/
/

d
i
r
e
c
t
.

i
t
.

e
d
u

/
j

o
c
n
a
r
t
i
c
e
–
p
d

f
/

3
4
9
1
5
4
7
2
0
3
7
4
5
1

/
j

o
c
n
_
a
_
0
1
7
6
1
p
d

b
y
g
u
e
s
t

o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3

Table 1. Brain Regions That Positively Predicted Trial-wise Gaze Reinstatement

Anatomical Areas

Cluster Size

Precentral_L

Putamen_R

Occipital_Inf_R

Occipital_Mid_L

Putamen_L

Caudate_R

Supp_Motor_Area_L

Putamen_R

180

145

161

MNI Coordinates

t Value

3.983363

3.919645

3.858175

3.834414

3.815405

3.431526

3.112625

2.931331

p Value

.000164

.000197

.000235

.000251

.000265

.000778

.00184

.002952

−30

−28

−26

−10

−4

−94

−98

−4

All clusters survived the threshold of p < .005, with a 10-voxel extension, no correction. The names of the anatomical regions in the table, obtained using the automated anatomical labeling (AAL) toolbox for SPM12, follow the AAL template naming convention (Tzourio-Mazoyer et al., 2002). R/L = right/left hemisphere; Mid = middle; Inf = inferior. The left HPC showed a marginally significant effect, t = 1.88, p = .069, whereas the left PPA similarity effect was not significant, t = 1.41, p = .17 (Figure 4A and B). Since the occipital pole and BG regions showed stron- ger mean level activation for trials with greater subse- quent gaze reinstatement, we also examined the pattern similarity in the voxel clusters in these two regions. Specifically, we obtained the two voxel clusters in the BG and the occipital pole that survived the threshold of p = .005 (no correction) in the gaze reinstatement mod- ulation analysis (Figure 3B) and then computed the pat- tern similarity scores as we did for the PPA and HPC (see above). Similar to the PPA and HPC results, the right BG and right occipital pole ROIs showed significant pattern similarity between the subsequent memory and subse- quent gaze reinstatement modulation effects, t = 2.45 and 2.36, and p = .02 and .024, respectively. The left ROIs did not show any significant results, p > .05
(Figure 4C and D).

Directly comparing the brain activation pattern similar-
ity between the free- versus fixed-viewing condition
revealed greater brain pattern similarity in the free- ver-
sus fixed-viewing condition for the right PPA, HPC, and
occipital pole regions (t = 3.84, 3.55, and 2.24, and
p = .0005, .001, and .032, respectively). The left HPC
and a region in the left fusiform gyrus also showed

Figure 4. Pattern similarity
between gaze reinstatement
and subsequent memory
modulation effects. PPA =
parahippocampal place area;
HPC = hippocampus; Occip =
occipital; BG = basal ganglia;
+p < .09, *p < .05, **p < .005, ***p < .001. Wynn, Liu, and Ryan 1555 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 9 1 5 4 7 2 0 3 7 4 5 1 / / j o c n _ a _ 0 1 7 6 1 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 9 1 5 4 7 2 0 3 7 4 5 1 / / j o c n _ a _ 0 1 7 6 1 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 Figure 5. Whole-brain pattern similarity using a searchlight (8-mm sphere) between the subsequent memory and subsequent gaze reinstatement modulation effects (threshold p = .005, 10-voxel extension, no corrections). marginally significant effects (t = 2.02 and 1.91, and p = .051 and .065, respectively). occurred specifically in brain regions that are known to play key roles in visual memory encoding. As reported earlier, gaze reinstatement and memory performance were correlated at the behavioral level. Therefore, to ensure that the observed pattern similarity between the subsequent memory and gaze reinstatement modulation effects was specific to brain regions that were important for the scene encoding task, such as the ROIs tested above, and not general to all brain regions (i.e., re- flecting the shared variance between the two behavioral measures at the brain level), we employed a searchlight method in which a sphere with a radius of 8 mm was used to obtain the similarity value at each voxel of the brain. As shown in Figure 5A, not all brain regions showed the sim- ilarity effect. Instead, two large clusters in both the left and right HPC showed significant similarity between the subsequent memory and gaze reinstatement modulation effects (SPM small volume correction using HPC mask: cluster level pFWE-corr = .004 and .012, cluster size = 248 and 172 voxels). Other regions including regions in the ventral and dorsal visual stream also showed similar patterns. These results confirm that the pattern similarity effect (i.e., the brain manifestation of the shared variance between gaze reinstatement and memory performance) To further confirm the specificity of the pattern similarity effect, we conducted the same analysis for the fixed-viewing scene condition, which, consistent with our hypothesis, yielded no significant results in the HPC or in other ventral visual stream regions (Figure 5B). Directly contrasting the pattern similarity between the free- versus fixed-viewing conditions confirmed that the similarity between the subse- quent memory and subsequent gaze reinstatement mod- ulation effects was specific to brain regions typically implicated in scene encoding, such as the left and right HPC (SPM small volume correction using HPC mask: cluster level pFWE-corr = .023 and .022, cluster size = 126 and 130 voxels; Figure 5C), and specific to the free- viewing condition (Figure 5A and B). Notably, the occipital poles showed stronger activation bilaterally for subsequently remembered versus subse- quently forgotten trials (embedded brain image [right] in Figure 6) and for trials with stronger subsequent gaze reinstatement (embedded brain image [left] in Figure 6). This region also showed similar cross-voxel modulation patterns for the subsequent memory and gaze reinstate- ment effects. We thus hypothesized that the activation of Figure 6. Partial mediation effect of occipital pole activation on the predictive effect of gaze reinstatement on subsequent memory. The embedded brain activation image on the left shows the occipital clusters that were modulated by subsequent gaze reinstatement. The embedded brain activation image on the right shows the overlap between the occipital clusters that were modulated by subsequent gaze reinstatement (blue) and the clusters that showed subsequent memory effects (green). 1556 Journal of Cognitive Neuroscience Volume 34, Number 9 this region may mediate the relationship between gaze reinstatement and subsequent memory. To test this pre- diction, we conducted a mediation analysis in which we examined whether the effect of gaze reinstatement on subsequent memory could be significantly reduced when brain activity in the occipital pole, aggregated across the left and right, was entered as a mediator in the regression analysis. Specifically, for each participant, we first estimated brain activity for each scene image in each condition using the beta-series method (Rissman, Gazzaley, & D’Esposito, 2004). We then extracted the occipital pole activation corresponding to the left and right occipital pole ROIs. Next, at the individual level, we conducted a mediation analysis with the trial-wise gaze reinstatement measure as the predictor (x), occipital pole ROI activa- tion as the mediator (m), and the subsequent memory measure as the outcome variable ( y). The regression co- efficient a (see Figure 6) was obtained when x was used to predict m, b was obtained when m was used to predict 0 was obtained when x y (while controlling for x), and c was used to predict y (while controlling for m). Finally, 0 were averaged across runs the coefficients a, b, and c for each participant and then tested at the group level using t tests. In line with our prediction, occipital pole activation partially mediated the prediction of gaze rein- statement on subsequent memory (indirect path: t = 1.86, p = .035, one-tailed; Figure 6). DISCUSSION This study explored the neural correlates of functional gaze reinstatement—the recapitulation of encoding- related gaze patterns during retrieval that is significantly predictive of mnemonic performance. Consistent with the Scanpath Theory (Noton & Stark, 1971a, 1971b), re- search using eye movement monitoring has demon- strated that the spatial overlap between encoding and retrieval gaze patterns is correlated with behavioral per- formance across a number of memory tasks (e.g., Wynn et al., 2018, 2020; Damiano & Walther, 2019; Scholz et al., 2016; Laeng et al., 2014; Olsen et al., 2014; Johansson & Johansson, 2013; Foulsham et al., 2012; Laeng & Teodorescu, 2002; for a review, see Wynn et al., 2019). Indeed, guided or spontaneous gaze shifts to regions viewed during encoding (i.e., gaze reinstatement) have been proposed to support memory retrieval by reactivat- ing the spatiotemporal encoding context ( Wynn et al., 2019). In line with this proposal, recent work using con- current eye tracking and fMRI has indicated that gaze re- instatement elicits patterns of neural activity typically associated with successful memory retrieval, including HPC activity (Ryals et al., 2015) and whole-brain neural reactivation (Bone et al., 2019). Critically, however, these findings do not speak to the cognitive and neural pro- cesses at encoding that support the creation of functional scanpaths. This question is directly relevant to Scanpath Theory, which contends that eye movements not only facilitate memory retrieval but are themselves embedded in the memory trace (Noton & Stark, 1971a, 1971b). Accordingly, this study investigated the neural regions that support the formation and subsequent recapitula- tion of functional scanpaths. Extending earlier findings, and lending support to Scanpath Theory, here we show for the first time that functional gaze reinstatement is cor- related with encoding-related neural activity patterns in brain regions associated with sensory (visual) processing, motor (gaze) control, and memory. Importantly, these findings suggest that, like objects and the relations among them, scanpaths may be bound into memory rep- resentations, such that their recapitulation may cue, and facilitate the retrieval of, additional event elements (see Wynn et al., 2019). Consistent with previous work, this study found evi- dence of gaze reinstatement that was significantly greater than chance and significantly predictive of recognition memory accuracy when participants freely viewed repeated scenes. In addition, gaze reinstatement (measured during free viewing of scenes at retrieval) was positively associated with encoding-related neural activity in the BG and in the occipital pole. Previous work has linked the BG to voluntary saccade control, particularly when sac- cades are directed toward salient or rewarding stimuli (for a review, see Gottlieb, Hayhoe, Hikosaka, & Rangel, 2014), and to memory-guided attentional orient- ing (Goldfarb, Chun, & Phelps, 2016). Dense connections with brain regions involved in memory and oculomotor control including the HPC and FEFs (Shen, Bezgin, Selvam, McIntosh, & Ryan, 2016) make the BG ideally po- sitioned to guide visual attention to informative (i.e., high reward probability) image regions. The occipital pole has been similarly implicated in exogenous orienting (Fernández & Carrasco, 2020) and visual processing, in- cluding visual imagery (St-Laurent, Abdi, & Buchsbaum, 2015), partly because of the relationship between neural activity in the occipital pole and gaze measures including fixation duration (Choi & Henderson, 2015) and saccade length (Frey, Nau, & Doeller, 2020). Notably, the occipital pole region identified in the cur- rent study was spatially distinct from the occipital place area seen in other studies (e.g., Bonner & Epstein, 2017; Patai & Spiers, 2017; Dilks, Julian, Paunov, & Kanwisher, 2013), suggesting that it may differentially contribute to scene processing, possibly by guiding visual exploration. Moreover, the identified occipital pole region did not include area V1, suggesting that unlike (the number of ) gaze fixations, which modulate activity in early visual regions (Liu et al., 2017), gaze reinstate- ment does not directly reflect the amount of bottom–up visual input present at encoding. Rather, gaze reinstate- ment may be related more specifically to the selective sampling and processing of informative regions at encoding (see also Fehlmann et al., 2020). Indeed, during encoding, viewing of informationally salient regions, as defined by participant data-driven saliency maps Wynn, Liu, and Ryan 1557 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 9 1 5 4 7 2 0 3 7 4 5 1 / / j o c n _ a _ 0 1 7 6 1 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 capturing both low-level and high-level image features, was significantly more predictive of subsequent gaze reinstatement than viewing of visually salient regions, as defined by stimulus (Saliency Toolbox)-driven saliency maps capturing low-level image features. The occipital pole additionally partially mediated the effect of gaze re- instatement on subsequent memory, further suggesting that this region may contribute to mnemonic processes via the formation of gaze scanpaths reflecting informa- tionally salient image regions. Taken together with the neuroimaging results, these findings suggest that view- ing, and consequentially, encoding, regions high in infor- mational and/or rewarding content may facilitate the laying down of a scanpath that, when recapitulated, facil- itates recognition via comparison of presented visual in- put with stored features (see Wynn et al., 2019). To further interrogate the relationship between gaze reinstatement and memory at the neural level, we con- ducted a pattern similarity analysis to identify brain re- gions in which neural activity patterns corresponding to gaze reinstatement and those corresponding to subse- quent memory covaried. Results of this analysis revealed significant overlap between the subsequent memory and subsequent gaze reinstatement effects in the occipital pole and BG (regions that showed a parametric modula- tion by subsequent gaze reinstatement) and in the PPA and HPC (regions that showed a parametric modulation by subsequent memory; see Liu et al., 2020). These re- gions may therefore be important for scene encoding (see Liu et al., 2020), in part through their role in linking the scanpath to the resulting memory representation. Specifically, parametric modulation and pattern similarity effects in the occipital pole and BG suggest that, when in- formationally salient image features are selected for overt visual attention, those features are encoded into memory along with the fixations made to them, which are subse- quently recapitulated during retrieval. Consistent with Scanpath Theory’s notion of the scanpath as a sensory– motor memory trace (Noton & Stark, 1971a, 1971b), these findings suggest that eye movements themselves may be part and parcel of the memory representation. The absence of gaze reinstatement-related activity in object- or location-specific processing regions (e.g., PPA, lateral occipital cortex) or low-level visual regions (e.g., V1) further suggests that reinstated scanpaths (at least in the present task) cannot be solely attributed to overlap in bottom–up visual saliency or memory for particularly salient image features. Indeed, recent work from Wang, Baumgartner, Kaule, Hanke, and Pollmann (2019) indi- cates that simply following a face- or house-related gaze pattern (without seeing a face or house) is sufficient to elicit activity in the FFA or PPA, respectively, suggesting that visual identification is not based solely on visual fea- tures but rather can also be supported by efferent oculo- motor signals. The present findings further suggest that such signals, serving as a part of the memory representa- tion, may be referenced and used by the HPC, similar to other mnemonic features (e.g., spatial locations, temporal order; Yonelinas, 2013; Davachi, 2006), to cue retrieval of associated elements within memory. That is, although the HPC may not be directly involved in generating or storing the scanpath (which may instead rely on visual and oculo- motor regions), similar patterns of HPC activity that pre- dict subsequent gaze reinstatement and subsequent memory suggest that the HPC may index these oculomo- tor programs, along with other signals, in the service of mnemonic binding and retrieval functions (e.g., relative spatial position coding; see Connor & Knierim, 2017). Importantly, the finding that the HPC, in particular, similarly codes for subsequent memory and subsequent gaze reinstatement is consistent with its purported role in coordinating sensory and mnemonic representations (see Knapen, 2021). Indeed, early accounts positioned the HPC as the site at which already-parsed information from cortical processors are bound into lasting memory representations (Cohen & Eichenbaum, 1993). The no- tion that the oculomotor effector trace is included within the HPC representation is aligned with more recent work showcasing the inherent, and reciprocal, connections between the HPC and oculomotor systems. Research using computational modeling and network analyses, for example, indicates that the HPC and FEF are both anatomically and functionally connected (Ryan, Shen, Kacollja et al., 2020; Shen et al., 2016; for a review, see Ryan, Shen, & Liu, 2020). Indeed, whereas damage to the HPC leads to impairments on several eye-movement- based measures (e.g., Olsen et al., 2015, 2016; Hannula, Ryan, Tranel, & Cohen, 2007; Ryan, Althoff, Whitlow, & Cohen, 2000), disruption of the FEF (via TMS) leads to im- pairments in memory recall ( Wantz et al., 2016). Other work further suggests that visual and mnemonic processes share a similar reference frame, with connectivity between the HPC and V1 showing evidence of retinotopic orienta- tion during both visual stimulation and visual imagery (Knapen, 2021; see also Silson, Zeidman, Knapen, & Baker, 2021). That the HPC may serve as a potential “con- vergence zone” for binding disparate event elements, including eye movements, is further supported by evidence from intracranial recordings in humans and ani- mals suggesting that the coordination of eye movements with HPC theta rhythms supports memory encoding (Hoffman et al., 2013; Jutras, Fries, & Buffalo, 2013) and retrieval (Kragel et al., 2020) and by evidence of gaze- centric cells in the HPC (and entorhinal cortex; Meister & Buffalo, 2018; Killian, Jutras, & Buffalo, 2012) that re- spond to a particular gaze location (e.g., Chen & Naya, 2020; Rolls, Robertson, & Georges-François, 1997; for a re- view, see Nau, Julian, & Doeller, 2018). Extending this work, the present findings suggest that gaze reinstatement and subsequent memory share similar variance in the brain and may be supported by similar HPC mechanisms. Furthermore, these findings critically suggest that reinstat- ed gaze patterns may be recruited and used by the HPC in the service of memory retrieval. 1558 Journal of Cognitive Neuroscience Volume 34, Number 9 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 9 1 5 4 7 2 0 3 7 4 5 1 / / j o c n _ a _ 0 1 7 6 1 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 With this study, we provide novel evidence that encoding-related activity in the occipital pole and BG during free viewing of scenes is significantly predictive of subsequent gaze reinstatement, suggesting that scan- paths that are later recapitulated may contain important visuosensory and oculomotor information. Indeed, gaze reinstatement was correlated more strongly with encod- ing of informationally salient regions than visually salient regions, suggesting that the scanpath carries information related to high-level image content. Critically, visual, oc- ulomotor, and mnemonic ROIs (i.e., occipital pole, BG, PPA, HPC) showed similar patterns of activity corre- sponding to subsequent memory (see Liu et al., 2020) and subsequent gaze reinstatement, further supporting a common underlying neural mechanism. Lending sup- port to Scanpath Theory, the present results suggest that gaze scanpaths, beyond scaffolding memory retrieval, are themselves embedded in the memory representation (see Cohen & Eichenbaum, 1993), similar to other ele- ments, including spatial and temporal relations (see Yonelinas, 2013; Davachi, 2006), and may be utilized by the HPC to support memory retrieval. Given the nature of the present task, we focused here on the spatial over- lap (including fixation location and duration information) between gaze patterns during encoding and retrieval, but future work could also explore how temporal order infor- mation embedded in the scanpath may similarly or differ- entially contribute to memory retrieval. Thus, although further research will be needed to fully elucidate the neural mechanisms supporting functional gaze reinstate- ment, particularly across different tasks and populations, the current findings spotlight the unique interactions between overt visual attention and memory that extend beyond behavior to the level of the brain. Moreover, these findings speak to the importance of considering, and accounting for, effector systems, including the oculo- motor system, in models of memory and cognition more broadly. Acknowledgments This work was supported by a Vision: Science to Applications postdoctoral fellowship awarded to Z. X. L. Reprint requests should be sent to Jennifer D. Ryan, Rotman Research Institute, 3560 Bathurst St., Toronto, ON M6A 2E1, Canada, or via e-mail: jryan@research.baycrest.org. Author Contributions Jordana S. Wynn: Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Validation; Visualization; Writing—Original draft; Writing—Review & editing. Zhong-Xu Liu: Conceptualization; Data cura- tion; Formal analysis; Investigation; Methodology; Validation; Visualization; Writing—Original draft; Writing —Review & editing. Jennifer D. Ryan: Conceptualization; Funding acquisition; Project administration; Resources; Supervision; Writing—Review & editing. Funding Information Jennifer D. Ryan, Natural Sciences and Engineering Research Council of Canada (https://dx.doi.org/10.13039 /501100000038), grant number: RGPIN-2018-06399. Jennifer D. Ryan, Canadian Institutes of Health Research (https://dx.doi.org/10.13039/501100000026), grant number: MOP126003. Diversity in Citation Practices A retrospective analysis of the citations in every article published in this journal from 2010 to 2020 has revealed a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publish- ing in the Journal of Cognitive Neuroscience ( JoCN ) during this period were M(an)/M = .408, W(oman)/M = .335, M/ W = .108, and W/ W = .149, the comparable pro- portions for the articles that these authorship teams cited were M/M = .579, W/M = .243, M/ W = .102, and W/ W = .076 (Fulvio et al., JoCN, 33:1, pp. 3–7). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article’s gender ci- tation balance. The authors of this article report its pro- portions of citations by gender category to be as follows: M/M = .55, W/M = .175, M/ W = .075, and W/ W = .2. Notes 1. For further details regarding stimulus selection and feature equivalence, see Liu et al. (2020). 2. To achieve luminance and contrast balance, all color RGB im- ages were transferred to NTSC space using the built-in MATLAB function rgb2ntsc.m. Then, the luminance (i.e., the NTSC Y com- ponent) and contrast (i.e., the standard deviation of luminance) were obtained for each image, and the mean values were used to balance (i.e., equalize) the luminance and contrast for all images using SHINE toolbox ( Willenbockel et al., 2010). Finally, the im- ages were transferred back to their original RGB space using the MATLAB function ntsc2rgb.m. 3. For further details regarding the randomization procedure, see Liu et al. (2020). 4. For further details regarding the density map computation, see Wynn et al. (2020). * Reference variable. 5. Using the same density map computation as the gaze rein- statement analysis, see Wynn et al. (2020). REFERENCES Bone, M. B., St-Laurent, M., Dang, C., McQuiggan, D. A., Ryan, J. D., & Buchsbaum, B. R. (2019). Eye movement reinstatement and neural reactivation during mental imagery. Cerebral Cortex, 29, 1075–1089. https://doi.org/10.1093/cercor /bhy014, PubMed: 29415220 Bonner, M. F., & Epstein, R. A. (2017). Coding of navigational affordances in the human visual system. Proceedings of the National Academy of Sciences, U.S.A., 114, 4793–4798. https://doi.org/10.1073/pnas.1618228114, PubMed: 28416669 Wynn, Liu, and Ryan 1559 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 9 1 5 4 7 2 0 3 7 4 5 1 / / j o c n _ a _ 0 1 7 6 1 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 Brewer, J. B., Zhao, Z., Desmond, J. E., Glover, G. H., & Gabrieli, J. D. E. (1998). Making memories: Brain activity that predicts how well visual experience will be remembered. Science, 281, 1185–1187. https://doi.org/10.1126/science.281 .5380.1185, PubMed: 9712581 Chan, C. Y. H., Chan, A. B., Lee, T. M. C., & Hsiao, J. H. (2018). Eye-movement patterns in face recognition are associated with cognitive decline in older adults. Psychonomic Bulletin and Review, 25, 2200–2207. https://doi.org/10.3758/s13423 -017-1419-0, PubMed: 29313315 Chen, H., & Naya, Y. (2020). Automatic encoding of a view- centered background image in the macaque temporal lobe. Cerebral Cortex, 30, 6270–6283. https://doi.org/10.1093 /cercor/bhaa183, PubMed: 32637986 Choi, W., & Henderson, J. M. (2015). Neural correlates of active vision: An fMRI comparison of natural reading and scene viewing. Neuropsychologia, 75, 109–118. https://doi.org/10 .1016/j.neuropsychologia.2015.05.027, PubMed: 26026255 Cohen, N. J., & Eichenbaum, H. (1993). Memory, amnesia, and the hippocampal system. In Memory, amnesia, and the hippocampal system. Cambridge, MA: MIT Press. Connor, C. E., & Knierim, J. J. (2017). Integration of objects and space in perception and memory. Nature Neuroscience, 20, 1493–1503. https://doi.org/10.1038/nn.4657, PubMed: 29073645 Neuroscience, 34, 15497–15504. https://doi.org/10.1523 /JNEUROSCI.3270-14.2014, PubMed: 25392517 Hannula, D. E., & Duff, M. C. (Eds.). (2017). The hippocampus from cells to systems: Structure, connectivity, and functional contributions to memory and flexible cognition. https://doi.org/10.1007/978-3-319-50406-3 Hannula, D. E., Ryan, J. D., Tranel, D., & Cohen, N. J. (2007). Rapid onset relational memory effects are evident in eye movement behavior, but not in hippocampal amnesia. Journal of Cognitive Neuroscience, 19, 1690–1705. https:// doi.org/10.1162/jocn.2007.19.10.1690, PubMed: 17854282 Hebart, M. N., Görgen, K., & Haynes, J.-D. (2015). The decoding toolbox (TDT): A versatile software package for multivariate analyses of functional imaging data. Frontiers in Neuroinformatics, 8, 88. https://doi.org/10.3389/fninf.2014 .00088, PubMed: 25610393 Heisz, J. J., & Ryan, J. D. (2011). The effects of prior exposure on face processing in younger and older adults. Frontiers in Aging Neuroscience, 3, 15. https://doi.org/10.3389/fnagi.2011 .00015, PubMed: 22007169 Henderson, J. M., & Hayes, T. R. (2018). Meaning guides attention in real-world scene images: Evidence from eye movements and meaning maps. Journal of Vision, 18, 10. https://doi.org/10.1167/18.6.10, PubMed: 30029216 Damiano, C., & Walther, D. B. (2019). Distinct roles of eye Henderson, J. M., Williams, C. C., & Falk, R. J. (2005). Eye movements during memory encoding and retrieval. Cognition, 184, 119–129. https://doi.org/10.1016/j.cognition .2018.12.014, PubMed: 30594878 movements are functional during face learning. Memory & Cognition, 33, 98–106. https://doi.org/10.3758/BF03195300, PubMed: 15915796 Davachi, L. (2006). Item, context and relational episodic Hoffman, K. L., Dragan, M. C., Leonard, T. K., Micheli, C., encoding in humans. Current Opinion in Neurobiology, 16, 693–700. https://doi.org/10.1016/j.conb.2006.10.012, PubMed: 17097284 Dilks, D. D., Julian, J. B., Paunov, A. M., & Kanwisher, N. (2013). The occipital place area is causally and selectively involved in scene perception. Journal of Neuroscience, 33, 1331–1336. https://doi.org/10.1523/JNEUROSCI.4081-12.2013, PubMed: 23345209 Fehlmann, B., Coynel, D., Schicktanz, N., Milnik, A., Gschwind, L., Hofmann, P., et al. (2020). Visual exploration at higher fixation frequency increases subsequent memory recall. Cerebral Cortex Communications, 1, tgaa032. https://doi.org /10.1093/texcom/tgaa032, PubMed: 34296105 Fernández, A., & Carrasco, M. (2020). Extinguishing exogenous attention via transcranial magnetic stimulation. Current Biology, 30, 4078–4084. https://doi.org/10.1016/j.cub.2020.07 .068, PubMed: 32795447 Firestone, A., Turk-Browne, N. B., & Ryan, J. D. (2007). Age-related deficits in face recognition are related to underlying changes in scanning behavior. Neuropsychology, Development, and Cognition. Section B, Aging, Neuropsychology and Cognition, 14, 594–607. https://doi.org/10.1080/13825580600899717, PubMed: 18038358 Fischl, B. (2012). FreeSurfer. Neuroimage, 62, 774–781. https:// doi.org/10.1016/j.neuroimage.2012.01.021, PubMed: 22248573 Foulsham, T., Dewhurst, R., Nyström, M., Jarodzka, H., Johansson, R., Underwood, G., et al. (2012). Comparing scanpaths during scene encoding and recognition: A multi-dimensional approach. Journal of Eye Movement Research, 5, 1–14. https:// doi.org/10.16910/jemr.5.4.3 Frey, M., Nau, M., & Doeller, C. F. (2020). MR-based camera-less eye tracking using deep neural networks. BioRxiv. https://doi .org/10.1101/2020.11.30.401323 Goldfarb, E. V., Chun, M. M., & Phelps, E. A. (2016). Memory- guided attention: Independent contributions of the hippocampus and striatum. Neuron, 89, 317–324. https://doi .org/10.1016/j.neuron.2015.12.014, PubMed: 26777274 Gottlieb, J., Hayhoe, M., Hikosaka, O., & Rangel, A. (2014). Attention, reward, and information seeking. Journal of Montefusco-Siegmund, R., & Valiante, T. A. (2013). Saccades during visual exploration align hippocampal 3–8 Hz rhythms in human and non-human primates. Frontiers in Systems Neuroscience, 7, 43. https://doi.org/10.3389/fnsys.2013 .00043, PubMed: 24009562 Itti, L., & Koch, C. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 40, 1489–1506. https://doi.org/10.1016/S0042-6989 (99)00163-7, PubMed: 10788654 Johansson, R., & Johansson, M. (2013). Look here, eye movements play a functional role in memory retrieval. Psychological Science, 25, 236–242. https://doi.org/10.1177 /0956797613498260, PubMed: 24166856 Jutras, M. J., Fries, P., & Buffalo, E. A. (2013). Oscillatory activity in the monkey hippocampus during visual exploration and memory formation. Proceedings of the National Academy of Sciences, U.S.A., 110, 13144–13149. https://doi.org/10.1073 /pnas.1302351110, PubMed: 23878251 Killian, N., Jutras, M., & Buffalo, E. (2012). A map of visual space in the primate entorhinal cortex. Nature, 491, 761–764. https:// doi.org/10.1038/nature11587.A, PubMed: 23103863 Knapen, T. (2021). Topographic connectivity reveals task- dependent retinotopic processing throughout the human brain. Proceedings of the National Academy of Sciences, U.S.A., 118, e2017032118. https://doi.org/10.1073/pnas .2017032118, PubMed: 33372144 Kragel, J. E., VanHaerents, S., Templer, J. W., Schuele, S., Rosenow, J. M., Nilakantan, A. S., et al. (2020). Hippocampal theta coordinates memory processing during visual exploration. eLife, 9, e52108. https://doi.org/10.7554/eLife .52108, PubMed: 32167568 Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82, 1–26. https://doi.org/10 .18637/jss.v082.i13 Laeng, B., Bloem, I. M., D’Ascenzo, S., & Tommasi, L. (2014). Scrutinizing visual images: The role of gaze in mental imagery and memory. Cognition, 131, 263–283, https://doi.org/10.1016/j .cognition.2014.01.003, PubMed: 24561190 1560 Journal of Cognitive Neuroscience Volume 34, Number 9 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 9 1 5 4 7 2 0 3 7 4 5 1 / / j o c n _ a _ 0 1 7 6 1 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 Laeng, B., & Teodorescu, D.-S. (2002). Eye scanpaths during visual imagery reenact those of perception of the same visual scene. Cognitive Science, 26, 207–231. https://doi.org/10 .1016/S0364-0213(01)00065-9 Lieberman, M. D., & Cunningham, W. A. (2009). Type I and Type II error concerns in fMRI research: Re-balancing the scale. Social Cognitive and Affective Neuroscience, 4, 423–428. https://doi.org/10.1093/scan/nsp052, PubMed: 20035017 Liu, Z.-X., Rosenbaum, R. S., & Ryan, J. D. (2020). Restricting visual exploration directly impedes neural activity, functional connectivity, and memory. Cerebral Cortex Communications, 1, tgaa054. https://doi.org/10.1093 /texcom/tgaa054, PubMed: 33154992 Liu, Z.-X., Shen, K., Olsen, R. K., & Ryan, J. D. (2017). Visual sampling predicts hippocampal activity. Journal of Neuroscience, 37, 599–609. https://doi.org/10.1523/JNEUROSCI.2610-16.2017, PubMed: 28100742 Liu, Z.-X., Shen, K., Olsen, R. K., & Ryan, J. D. (2018). Age-related changes in the relationship between visual exploration and hippocampal activity. Neuropsychologia, 119, 81–91. https:// doi.org/10.1016/j.neuropsychologia.2018.07.032, PubMed: 30075215 Loftus, G. R. (1972). Eye fixations and recognition memory for pictures. Cognitive Psychology, 3, 525–551. https://doi.org/10 .1016/0010-0285(72)90021-7 Meister, M. L. R., & Buffalo, E. A. (2016). Getting directions from the hippocampus: The neural connection between looking and memory. Neurobiology of Learning and Memory, 134, 135–144. https://doi.org/10.1016/j.nlm.2015.12.004, PubMed: 26743043 Meister, M. L. R., & Buffalo, E. A. (2018). Neurons in primate entorhinal cortex represent gaze position in multiple spatial reference frames. Journal of Neuroscience, 38, 2430–2441. https://doi.org/10.1523/JNEUROSCI.2432-17.2018, PubMed: 29386260 Nau, M., Julian, J. B., & Doeller, C. F. (2018). How the brain’s navigation system shapes our visual experience. Trends in Cognitive Sciences, 22, 810–825. https://doi.org/10.1016/j.tics .2018.06.008, PubMed: 30031670 Noton, D., & Stark, L. (1971a). Scanpaths in eye movements during pattern perception. Science, 171, 308–311. https://doi .org/10.1126/science.171.3968.308, PubMed: 5538847 Noton, D., & Stark, L. (1971b). Scanpaths in saccadic eye movements while viewing and recognizing patterns. Vision Research, 11, 929–942. https://doi.org/10.1016/0042-6989(71) 90213-6, PubMed: 5133265 O’Connell, T. P., & Walther, D. B. (2015). Dissociation of salience-driven and content-driven spatial attention to scene category with predictive decoding of gaze patterns. Journal of Vision, 15, 20. https://doi.org/10.1167/15.5.20, PubMed: 26067538 Olsen, R. K., Chiew, M., Buchsbaum, B. R., & Ryan, J. D. (2014). The relationship between delay period eye movements and visuospatial memory. Journal of Vision, 14, 8. https://doi.org /10.1167/14.1.8, PubMed: 24403394 Olsen, R. K., Lee, Y., Kube, J., Rosenbaum, R. S., Grady, C. L., Moscovitch, M., et al. (2015). The role of relational binding in item memory: Evidence from face recognition in a case of developmental amnesia. Journal of Neuroscience, 35, 5342–5350. https://doi.org/10.1523/JNEUROSCI.3987-14 .2015, PubMed: 25834058 Olsen, R. K., Sebanayagam, V., Lee, Y., Moscovitch, M., Grady, C. L., Rosenbaum, R. S., et al. (2016). The relationship between eye movements and subsequent recognition: Evidence from individual differences and amnesia. Cortex, 85, 182–193. https:// doi.org/10.1016/j.cortex.2016.10.007, PubMed: 27842701 Patai, E. Z., & Spiers, H. J. (2017). Human navigation: Occipital place area detects potential paths in a scene. Current Biology, 27, R599–R600. https://doi.org/10.1016/j.cub.2017.05 .012, PubMed: 28633030 Rissman, J., Gazzaley, A., & D’Esposito, M. (2004). Measuring functional connectivity during distinct stages of a cognitive task. Neuroimage, 23, 752–763. https://doi.org/10.1016/j .neuroimage.2004.06.035, PubMed: 15488425 Rolls, E. T., Robertson, R. G., & Georges-François, P. (1997). Spatial view cells in the primate hippocampus. European Journal of Neuroscience, 9, 1789–1794. https://doi.org/10 .1111/j.1460-9568.1997.tb01538.x, PubMed: 9283835 Ryals, A. J., Wang, J. X., Polnaszek, K. L., & Voss, J. L. (2015). Hippocampal contribution to implicit configuration memory expressed via eye movements during scene exploration. Hippocampus, 25, 1028–1041. https://doi.org/10.1002/hipo .22425, PubMed: 25620526 Ryan, J. D., Althoff, R. R., Whitlow, S., & Cohen, N. J. (2000). Amnesia is a deficit in relational memory. Psychological Science, 11, 454–461. https://doi.org/10.1111/1467-9280 .00288, PubMed: 11202489 Ryan, J. D., Shen, K., Kacollja, A., Tian, H., Griffiths, J., Bezgin, G., et al. (2020). Modeling the influence of the hippocampal memory system on the oculomotor system. Network Neuroscience, 4, 217–233. https://doi.org/10.1162/netn_a _00120, PubMed: 32166209 Ryan, J. D., Shen, K., & Liu, Z.-X. (2020). The intersection between the oculomotor and hippocampal memory systems: Empirical developments and clinical implications. Annals of the New York Academy of Sciences, 1464, 115–141. https:// doi.org/10.1111/nyas.14256, PubMed: 31617589 Sakon, J. J., & Suzuki, W. A. (2019). A neural signature of pattern separation in the monkey hippocampus. Proceedings of the National Academy of Sciences, U.S.A., 116, 9634–9643. https://doi.org/10.1073/pnas.1900804116, PubMed: 31010929 Scholz, A., Mehlhorn, K., & Krems, J. F. (2016). Listen up, eye movements play a role in verbal memory retrieval. Psychological Research, 80, 149–158. https://doi.org/10.1007 /s00426-014-0639-4, PubMed: 25527078 Shen, K., Bezgin, G., Selvam, R., McIntosh, A. R., & Ryan, J. D. (2016). An anatomical interface between memory and oculomotor systems. Journal of Cognitive Neuroscience, 28, 1772–1783. https://doi.org/10.1162/jocn_a_01007, PubMed: 27378328 Silson, E. H., Zeidman, P., Knapen, T., & Baker, C. I. (2021). Representation of contralateral visual space in the human hippocampus. Journal of Neuroscience, 41, 2382–2392. https://doi.org/10.1523/JNEUROSCI.1990-20.2020, PubMed: 33500275 St-Laurent, M., Abdi, H., & Buchsbaum, B. R. (2015). Distributed patterns of reactivation predict vividness of recollection. Journal of Cognitive Neuroscience, 27, 2000–2018. https:// doi.org/10.1162/jocn_a_00839, PubMed: 26102224 Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., et al. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage, 15, 273–289. https://doi.org/10.1006/nimg.2001 .0978, PubMed: 11771995 Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W., Maril, A., Dale, A. M., et al. (1998). Building memories: Remembering and forgetting of verbal experiences as predicted by brain activity. Science, 281, 1188–1191. https://doi.org/10.1126 /science.281.5380.1188, PubMed: 9712582 Walther, D., & Koch, C. (2006). Modeling attention to salient proto-objects. Neural Networks, 19, 1395–1407. https://doi .org/10.1016/j.neunet.2006.10.001, PubMed: 17098563 Wang, L., Baumgartner, F., Kaule, F. R., Hanke, M., & Pollmann, S. (2019). Individual face- and house-related eye movement patterns distinctively activate FFA and PPA. Nature Wynn, Liu, and Ryan 1561 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 9 1 5 4 7 2 0 3 7 4 5 1 / / j o c n _ a _ 0 1 7 6 1 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 Communications, 10, 5532. https://doi.org/10.1016/j .cognition.2021.104746, PubMed: 31797874 Wantz, A. L., Martarelli, C. S., Cazzoli, D., Kalla, R., Müri, R., & Mast, F. W. (2016). Disrupting frontal eye-field activity impairs memory recall. NeuroReport, 27, 374–378. https://doi .org/10.1097/ WNR.0000000000000544, PubMed: 26901058 Willenbockel, V., Sadr, J., Fiset, D., Horne, G. O., Gosselin, F., & Tanaka, J. W. (2010). Controlling low-level image properties: The SHINE toolbox. Behavior Research Methods, 42, 671–684. https://doi.org/10.3758/BRM.42.3.671, PubMed: 20805589 Wynn, J. S., Buchsbaum, B. R., & Ryan, J. D. (2021). Encoding and retrieval eye movements mediate age differences in pattern completion. Cognition, 214, 104746. https://doi.org /10.1016/j.cognition.2021.104746, PubMed: 34034008 Wynn, J. S., Olsen, R. K., Binns, M. A., Buchsbaum, B. R., & Ryan, J. D. (2018). Fixation reinstatement supports visuospatial memory in older adults. Journal of Experimental Psychology: Human Perception and Performance, 44, 1119–1127. https:// doi.org/10.1037/xhp0000522, PubMed: 29469586 Wynn, J. S., Ryan, J. D., & Buchsbaum, B. R. (2020). Eye movements support behavioral pattern completion. Proceedings of the National Academy of Sciences, U.S.A., 53, 1689–1699. https://doi.org/10.1073/pnas.1917586117, PubMed: 32123109 Wynn, J. S., Shen, K., & Ryan, J. D. (2019). Eye movements actively reinstate spatiotemporal mnemonic content. Vision, 3, 21. https://doi.org/10.3390/vision3020021, PubMed: 31735822 Yonelinas, A. P. (2013). The hippocampus supports high- resolution binding in the service of perception, working memory and long-term memory. Behavioural Brain Research, 254, 34–44. https://doi.org/10.1016/j.bbr.2013.05 .030, PubMed: 23721964 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 9 1 5 4 7 2 0 3 7 4 5 1 / / j o c n _ a _ 0 1 7 6 1 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 1562 Journal of Cognitive Neuroscience Volume 34, Number 9 Neural Correlates of Subsequent Memory-Related image

Download pdf