What Happened When? Cerebral Processing of Modified - IA de Investigación especializada en el MIT

What Happened When? Cerebral Processing of Modified
Structure and Content in Episodic Cueing

Sophie Siestrup1,2 , Benjamin Jainta1, Nadiya El-Sourani1, Ima Trempler1,2,
Moritz F. Wurm3, Oliver T. Lobo 4, Sen Cheng4, and Ricarda I. Schubotz1,2

Abstracto

■ Episodic memories are not static but can change on the basis
of new experiences, potentially allowing us to make valid predic-
tions in the face of an ever-changing environment. Reciente
research has identified prediction errors during memory
retrieval as a possible trigger for such changes. en este estudio,
we used modified episodic cues to investigate whether different
types of mnemonic prediction errors modulate brain activity and
subsequent memory performance. Participants encoded epi-
sodes that consisted of short toy stories. During a subsequent
fMRI session, participants were presented videos showing the
original episodes, or slightly modified versions thereof. In mod-
ified videos, either the order of two subsequent action steps was
changed or an object was exchanged for another. Contenido

modifications recruited parietal, temporo-occipital, and parahip-
pocampal areas reflecting the processing of the new object infor-
formación. A diferencia de, structure modifications elicited activation in
right dorsal premotor, posterior temporal, and parietal areas,
reflecting the processing of new sequence information. en un
post-fMRI memory test, the participants’ tendency to accept
modified episodes as originally encoded increased significantly
when they had been presented modified versions already during
the fMRI session. After experiencing modifications, especially
those of the episodes’ structure, the recognition of originally
encoded episodes was impaired as well. Our study sheds light
onto the neural processing of different types of episodic predic-
tion errors and their influence on subsequent memory recall. ■

INTRODUCCIÓN

Episodic memories enable us to vividly relive events that
we experienced at some point in our personal life
(Tulving, 2002). Sin embargo, there is evidence that they are
not always veridical reconstructions of our past (Sotavento,
Nader, & Schiller, 2017; Scully, Napper, & Hupbach,
2017; Nader, 2015; Nader & Einarsson, 2010). Situations
we encounter in everyday life are usually not exactly the
same as those we experienced before. So, there is always
a certain discrepancy between our expectations, which we
derive from our memories, and the new events we expe-
rience. According to the predictive coding framework, este
discrepancy leads to a prediction error (Reichardt, Polner,
& Simor, 2020; Aprender, Mirolli, & Baltasar, 2013). Predic-
tion errors serve as bottom–up learning signals that allow
us to adapt our internal predictive models to an ever-
changing environment to maintain valid predictions in
the long run (Schubotz, 2015; Friston & Kiebel, 2009;
Friston, 2005). According to this view, it is adaptive that
memories are modified in favor of valid internal models
informed and updated by later experiences (Fernández,
Boccia, & Pedreira, 2016; Exton-McGuinness, Sotavento, &

1University of Münster, Alemania, 2Otto Creutzfeldt Center for
Cognitive and Behavioral Neuroscience, University of Münster,
Alemania, 3University of Trento, Italia, 4Ruhr University Bochum,
Alemania

Reichelt, 2015). Evidence accumulates that mnemonic
prediction errors are important drivers of memory
cambiar (Sinclair & Barense, 2019), and researchers
recently begun to address the question how mnemonic
prediction errors are processed by the brain (p.ej., Bein,
Duncan, & Davachi, 2020; kim, Lewis-Peacock, Norman,
& Turk-Browne, 2014).

The aim of this study was to characterize neural
responses to different types of mnemonic prediction
errors during episodic retrieval by targeting two basic
types of episodic memory information: either their con-
tent (“what”) or their structure (“when”; cf. Griffiths,
Dickinson, & Clayton, 1999). para hacerlo, we adapted a pre-
viously developed episodic cueing paradigm ( Jainta et al.,
2022; Schiffer, Ahlheim, Ulrichs, & Schubotz, 2013;
Schiffer, Ahlheim, Wurm, & Schubotz, 2012). After encod-
ing short episodes from videos and consolidating
memories in two further retrieval sessions, Participantes
went through a fMRI session and were either presented
original episode videos or slightly modified versions
thereof. To create the latter, a subset of videos was manip-
ulated with regard to the occurrence of an object (contenido
modification) or the order of two consecutive action steps
(structure modification) to elicit different types of mne-
monic prediction errors (ver figura 1 para un ejemplo).
In a post-fMRI memory test, participants’ memory for orig-
inal and modified episodes was probed.

© 2022 Instituto de Tecnología de Massachusetts. Published under
a Creative Commons Attribution 4.0 Internacional (CC POR 4.0) licencia.

Revista de neurociencia cognitiva 34:7, páginas. 1287–1305
https://doi.org/10.1162/jocn_a_01862

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Although today many agree that prediction errors drive
memory modification during episodic retrieval (barrón,
Auksztulewicz, & Friston, 2020; Fernández et al., 2016;
Kim y cols., 2014), there is no unifying model from which
neuroanatomical hypotheses can be derived. Based on
existing studies, we expected that some regions might
be engaged in episodic prediction errors in general,
whereas others would be engaged only in content or
structure prediction errors.

As to the former, the medial frontal cortex may serve
more general control over consolidation and retrieval of
long-term memories (Peters, David, marco, & Herrero,
2013; Euston, Gruber, & McNaughton, 2012). Más-
más, the hippocampus is regarded a core structure of

Cifra 1. Example of an original episode and its modified versions,
shown by the sequence of the main event steps. Twenty-four stories
existed in three different versions each: an original, a structure
modification, and a content modification. For the structure modifications,
two adjacent action steps were switched compared with the original.
In this example, the original shows the blonde woman join the scene
before the guinea pig is positioned on the sales counter; en el
structure modification, the guinea pig appears before the blonde
woman (red circles). For the content modifications, an object was
exchanged compared with the original (aquí: tortoise instead of guinea
pig on the sales counter in Step 5). Note that in the fMRI experiment,
each participant was only presented with one of the three versions of
a story. We do not reproduce photos of our stimulus material because
it is copyrighted material (PLAYMOBIL figures); en cambio, we provide
schematic images.

episodic memory (Horner & Doeller, 2017; Stachenfeld,
Botvinick, & Gershman, 2017; Maguire, Intraub, &
Mullally, 2016) and responds to mnemonic prediction
errores (Bein et al., 2020; Largo, Sotavento, & kühl, 2016). In addi-
tion to these common neural responses to episodic sur-
prise, structure and content episodic modifications were
expected to engage different brain regions. Estructura
modifications should elevate activity in premotor areas
because of their central role in sequential order processing
(Schubotz, 2004). More specifically, dorsal premotor and
adjacent prefrontal sites along the superior frontal sulcus
(SFS; dorsal premotor cortex [PMd]) were found for step-
wise ordinal linking of individual action or event steps, como
required in different predictive tasks (Pomp et al., 2021;
Hrkać, Wurm, & Schubotz, 2014; Schubotz, Cesta, Schiffer,
Stadler, & por cramon, 2012; Stadler et al., 2011; Tamber-
Rosenau, Esterman, Chiu, & Yantis, 2011; Kurby & zacks,
2008). Por el contrario, content modifications were expected
to engage areas related to object processing, including lat-
eral occipitotemporal cortex (OTC; Lingnau & Downing,
2015), anterior intraparietal sulcus (IPS; Schubotz, Wurm,
Wittmann, & por cramon, 2014; Creem-Regehr, 2009),
and fusiform gyrus (FG; Reber, Gitelman, Parrish, &
Mesulam, 2005).

If new content and/or structure information induced
updating of the original predictive model during fMRI, como
esperado, this should also reduce memory accuracy in a
post-fMRI memory test ( Jainta et al., 2022; Schiffer et al.,
2012, 2013). We thus expected a weakening of the original
episodic memory, eso es, false rejections of original videos
as new, and/or the creation of alternative episode repre-
sentaciones, eso es, false acceptances of modified videos
as originals.

MÉTODOS

Participantes

Forty-five women took part in the study. Participantes
had (corrected-to-) normal vision, were native German
speakers, and were right-handed as assessed by the Edin-
burgh Handedness Inventory (Oldfield, 1971). As in our
previous study ( Jainta et al., 2022), participants were all
female to achieve a good match between the hands in
the videos and the hands of the participants. Ellos
reported no history of neurological or psychiatric disor-
ders or substance abuse. Four participants started the
experiment but did not finish, either because of technical
problems during the second retrieval session (three par-
ticipants) or personal reasons (one participant). Data from
five additional participants were excluded from analyses
because of the incorrect presentation of video stimuli dur-
ing the fMRI session (one participant) and increased
movement during the fMRI session (four participants,
approximately 5-mm movement). Como consecuencia, 36 par-
ticipants were part of the final sample (m = 22 años,
DE = 2.78 años, range = 18–30 years). Similar sample

1288

Revista de neurociencia cognitiva

Volumen 34, Número 7

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

sizes have yielded stable results in our previous work (p.ej.,
Jainta et al., 2022; Pomp et al., 2021; El-Sourani, Trempler,
Wurm, Fink, & Schubotz, 2019). Participants received
course credits or money for their participation and gave
written informed consent to participate in this study.
The study was conducted in accordance with the Declara-
tion of Helsinki and approved by the local ethics commit-
tee of the University of Münster.

Estímulos

We used the same set of videos as previously reported
(Jainta et al., 2022; available upon request at https://www
.uni-muenster.de/IVV5PSY/AvicomSrv/). These were 78
short videos (duration = 8.80–17.88 sec, m = 12.71 segundo)
of stories that were played with PLAYMOBIL toys, demostración
only the toys and hands and underarms of an actress.
Stories comprised six to nine action steps (m = 7.4 steps)
and 4–14 separable objects (m = 6.93 objects), como
characters, animals, vehicles, and tools. The same object
appeared in only one of the stories.

Stories were filmed from above with a digital single-lens
reflex camera (Nikon D5300), which was centrally
mounted above the table and faced straight down. Matte
white paper served as a base. A frame of 47.5 cm ×
28 cm was taped on the paper, congruent with the section
captured by the camera (in the following referred to as
camera frame). Objects that were needed for a particular
story were positioned next to the camera frame and were
only moved into view in the moment at which they
appeared in the story. During filming, the actress wore a
black pullover and black rubber gloves. To facilitate future
imitation from demo videos, the back of the right hand
was marked with a yellow dot (Franz, Vado, & Werner,
2007). Video material was edited using Adobe Premiere
Pro CC (Adobe Systems Software, Versión 12.1.2).
All videos had a frame of size 1920 × 1080 pixels and a
frame rate of 25 frames per second. Videos started
with seven frames showing only background and ended
after seven frames showing the final toy constellation.
Throughout the experiment, videos were presented at
a visual angle of approximately 7.3° × 13° using Presenta-
tion software ( Versión 20.3 02.25.19, NeuroBehavioral
Sistemas).

On the basis of two pilot studies, we chose 24 out of
originally 30 stories for our stimulus set. Stories were
excluded when they were particularly difficult to imitate
or describe. Uno de los 30 stories was excluded because
of low memorability as indicated by low performance in
a signal detection task.

El 24 final stories existed in three different versions
cada: (1) an original version as encoded by the partici-
pants, (2) a version in which two adjacent action steps
were switched (structure modification), y (3) otro
variation of the original video in which one object was
exchanged (content modification). Story scripts were cre-
ated by five experimenters who all had to agree that the

original story and modifications thereof were semantically
válido (within a toy world) and that modifications did not
change the overall outcome of the story. For creating
videos with modifications, the respective stories were
played and filmed again exactly the way as for the original
video. The only aspect that differed between original and
modified versions was a single change of either the order
of two action steps (es decir., one transition out of 7.33 transi-
ciones, on average, for structure modifications) or one
object (es decir., one object out of 6.95 objects, on average,
for content modifications).

Modifications were never introduced in the first two
action steps so that the beginning of a video served as a
cue for prediction. Además, no modifications were
introduced in the last two action steps, either. The exact
time point of the modification in each video was deter-
mined by identifying the video frame that diverged from
the original version. For an example of an episode and
its modified versions, ver figura 1.

Six other stories were used in one version only. Four of
them were presented for the first time in the fMRI session,
we refer to them as novel episodes in the following. El
two remaining videos were only used for practice and did
not appear in the fMRI experiment and memory test.

Procedimiento

Encoding

Encoding sessions were conducted in a computer labora-
tory at the Department of Psychology at the University of
Münster and followed our previously reported protocol,
with some modifications ( Jainta et al., 2022). The encod-
ing consisted of two sessions that took place on two con-
secutive days and lasted about 2 y 1.5 hr, respectivamente.
During each of the two sessions, participants encoded half
of the episodes. We chose to split the training over 2 días
to avoid fatigue or a decrease in motivation because of the
relatively long duration of the task.

El 24 demo videos were organized in four subsets,
containing six videos each, balanced for the number of
action steps (A1, A2, B1, B2). On each day, Participantes
encoded one A and one B subset. This means that each
participant encoded each video either during Session 1
or during Session 2; the same video was not encoded on
both days. Which subsets were trained in which session
was balanced over participants. The order in which epi-
sodes were encoded was randomized for each participant.
The first session started with two practice videos to famil-
iarize participants with the task.

During encoding, participants sat at the same setup that
had been used for filming the stimulus material and like-
wise wore a black pullover and gloves with a yellow dot on
the right hand, so that they could be filmed while playing
the stories themselves. The experimenter sat opposite
of the participant, supervising the performance. For each
story, the toys were positioned next to the camera frame,

Siestrup et al.

1289

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

following the same arrangement as used while creating the
stimulus material (Cifra 2). Each episode video was pre-
enviado 3 times from the first-person perspective. Entonces,
participants had to imitate each story correctly 3 veces.
After imitation, participants had to deliver a detailed
description of the story to ensure that they understood
it correctly and had paid attention to all objects involved.
If participants made a mistake during an imitation or
description trial, they were immediately interrupted by
the experimenter to avoid encoding of incorrect scripts.
They would then start over with a new imitation/description
attempt. De término medio, participants only performed one
incorrect imitation attempt (m = 1.04, DE = 0.47).

Retrieval Sessions

To further consolidate episodic memories, Participantes
went through two additional sessions during which they
completed an active retrieval task of before encoded
episodes. Active retrieval is known to aid memory consol-
idation and improve retention (Rowland, 2014). The first
consolidation session took place on the day after the
second encoding session. The second session was con-
ducted approximately 1 week later (range = 4–8 days;
m = 6.36 días, DE = 0.93 días).

Participants always watched the first two steps of a demo
video. Entonces, the video stopped and a question was dis-
played below the still video frame, which either read
“Left?” or “Right?". The participants’ task was to visualize
the rest of the story from memory the way they had per-
formed it and then answer how many steps of the entire
story had been performed with the left or right hand,

Cifra 2. Encoding setup. During encoding, participants imitated toy
stories from demo videos, while sitting at the filming setup. Su
performance was monitored by the experimenter.

respectivamente. They answered by pressing a number key
(0–9) on their keyboard. Upon response delivery, el
video played until the end and participants were
instructed to carefully watch the video to self-check their
respuesta. Después, written feedback (“correct,” “incor-
rect”) was provided for 1.5 segundo. When feedback is included
in retrieval tasks, consolidation has been shown to occur
independent of initial retrieval success (Rowland, 2014;
Roediger & mayordomo, 2011). We chose this task because we
wanted to encourage active retrieval of the encoded epi-
sodes without laying a special focus on aspects that would
be modified during the fMRI session. En tono rimbombante, el
number of steps that were conducted with the left or right
hand did not change in videos containing either type of
modification. The task was self-paced and started with
two practice videos during which the experimenter care-
fully checked whether the participant had understood
the task correctly.

During the retrieval sessions, we established the two
experimental factors consolidation TIMES and SCHEDULE.
Para tal fin, half of all episodes were consolidated 2 veces
en total, the other half 8 veces (factor TIMES). Además,
half of the stories were consolidated during both separate
sessions, eso es, in a spaced manner, while the other half
of the stories were consolidated only in Session 2, eso es,
in a massed manner (factor SCHEDULE). Los participantes fueron
explicitly instructed to always visualize the story and not
just remember previous responses they gave. Each type
of question was presented equally often after each video
and per session. At the end of Session 2, participants went
through a short practice (four video trials, four question
ensayos, one null event) of the task they would conduct
during the fMRI scan.

The two described experimental factors consolidation
TIMES (2, 8) and SCHEDULE (spaced, massed ) are not further
addressed in this article, as they are central for a compan-
ion paper (Siestrup, Jainta, Trempler, cheng, & Schubotz,
in preparation) describing the influence of different con-
solidation strategies on brain activation during episodic
recordar. Both factors were balanced with respect to the fac-
tors reported here, so we can exclude any confounding
efectos (fully crossed design). Correct answer rates in the
retrieval task did not differ significantly between episodes,
which were later presented in the original (ori), estructura
modified (str), or content-modified (estafa) versión (Mori =
.854 ± .013; Mstr = .846 ± .012; Mcon = .852 ± .008; F(2,
70) = .314, pag = .732, ηp2 = .009), so that we can rule out
that any confound was introduced through the retrieval
sessions.

fMRI Session

The fMRI session took place approximately 1 week after
the second retrieval session (range = 6–13 days; m =
7.69 días, DE = 1.31 días) and was conducted as previ-
ously described ( Jainta et al., 2022). Participants were told

1290

Revista de neurociencia cognitiva

Volumen 34, Número 7

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

that videos of themselves playing the stories would be pre-
sented in the fMRI session. Although participants had actu-
ally been filmed during encoding, these videos were not
used during the fMRI experiment. This was only a cover
story to elevate personal identification with the videos to
benefit episode reactivation. We previously confirmed in a
pilot study that this cover story works as intended and
already applied it successfully in our previous fMRI study
( Jainta et al., 2022). Participants were fully debriefed
after completion of the study.

During the fMRI session, participants were presented
with original and modified videos reminiscent of the pre-
viously encoded episodes. En tono rimbombante, each video was
only shown in the original or one divergent version.
Following a previously used paradigm (Schiffer et al.,
2012, 2013), modified and original episodes were pre-
sented repeatedly to simulate the natural circumstances
that potentially foster memory modification, eso es, updat-
ing of internal models because of increasing evidence for
the validity of an alternative. De este modo, eight videos were
repeatedly presented in the original version; eight
included a structure modification; and eight, a content
modification. Which stories belonged to which conditions
varied between participants. Además, four novel stories
were included in the fMRI session.

The fMRI experiment consisted of six blocks, each con-
taining the 24 videos reminiscent of the previously
encoded episodes. Como consecuencia, each video was pre-
enviado 6 times over the course of the session. Within
bloques, videos were presented in pseudorandom order
so that transition probabilities between conditions were
balanced. Además, each block contained three null
events during which only a fixation cross was presented
(duración: 7–10 sec). Además, each novel video was
presented once per block. Por lo tanto, the whole experi-
ment contained 18 null events and 24 novel video trials.
Participants were not informed about the block structure
of the experiment.

Participants were instructed to attentively watch the
presented videos. They were told that after some videos,
a short description would be presented (p.ej., “Rescuing
princess”) that either matched or did not match the story
shown in the video (question trials). The task was to either
accept or reject the description by pressing one of two
buttons on a response box with the right index or middle
finger, respectivamente. This type of task has been used
successfully before to focus participants’ attention on com-
plex video stimuli ( Jainta et al., 2022; El-Sourani et al.,
2019). En tono rimbombante, neither type of modification influenced
the overall outcome of episodes so that all descriptions
used as questions were valid for all episode versions. Ques-
tions never highlighted any type of modification (Cifra 3).
Throughout the entire experiment, each story was once
followed by a matching description and once by a non-
matching description, resulting in a total number of 56
question trials in the experiment. Each block contained
9–10 question trials and, per block, aproximadamente 50% de

descriptions were to be accepted, y 50% were to be
rejected. The question was presented for a maximum of
3 sec or until participants responded. Upon response
delivery, participants received a 1-sec written feedback
whether they answered correctly, incorrectly, or too late,
in case no response was given. Participants were naive
with regard to this distribution of question trials.

Between trials, a fixation cross was presented for a
duration of 2 segundo (1 sec after question trials) to serve as
an interstimulus interval. Before each trial, a variable jitter
de 0, 0.5, 1, o 1.5 sec of fixation was added for enhance-
ment of the temporal resolution of the BOLD response
(Cifra 3). In total, the fMRI task had a duration of
aproximadamente 48 mín..

Post-fMRI Memory Test

Immediately after the fMRI session, participants com-
pleted an explicit memory test as described previously
( Jainta et al., 2022). En tono rimbombante, encoding occurred inci-
dentally, as participants were not informed beforehand
that their memory for episodes would be tested.

Participants were seated in a separate room in front of a
laptop and instructed to remember their encoding ses-
siones 2 weeks prior during which they had played the
stories themselves. They were presented all stories that
they had seen in the fMRI session in two different versions.
Más precisamente, when modified videos had been pre-
sented during the fMRI session, these modified videos
were presented again during the memory test and addi-
tionally each story was shown in the original version. Cuando
original episode videos had been presented during the
fMRI scan, these original videos were presented again in
the memory test and, además, each story was shown
in a modified version, either containing a structure modi-
fication in half of the cases or a content modification.

The participants’ task was to rate after each video
whether they knew this exact episode from the encoding
sessions, using a Likert scale including 1 ( Sí), 2 (bastante
Sí), 3 (rather no), y 4 (No), by pressing one out of four
marked keys on the laptop’s keyboard. Similar rating
schemes have previously been used in memory research
( Jainta et al., 2022; Kim y cols., 2014). Response time was
not restricted, but participants were instructed to respond
quickly and intuitively. Videos were presented in a pseu-
dorandomized order, so that half of the stories (of each
experimental condition) were first presented in their orig-
inal version followed by a modified version and vice versa.
Novel videos were shown twice in the same version, entonces
eso, en total, the memory test comprised 56 video trials.
The completion of the task took approximately 15 mín..

MRI Data Acquisition and Preprocessing

MRI scans were conducted with a 3-Tesla Siemens Magne-
tom Prisma MR tomograph using a 20-channel head coil.
Participants lay supine on the scanner bed with their right

Siestrup et al.

1291

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Cifra 3. Schematic depiction
of task during fMRI session.
Video trials consisted of a
variable jitter (0, 0.5, 1, o
1.5 sec of fixation), a video
showing a toy story (ca. 9–
18 segundo) and a 2-sec interstimulus
interval (fixation). Question
trials included a variable jitter,
a question regarding the story
shown in the preceding video
(maximally 3 sec long or
terminated by response), y
a 2-sec interstimulus interval.
The interstimulus interval after
question trials was divided into
a 1-sec feedback (“correct,"
“incorrect,” “too late”) y un
1-sec fixation. Aside from the
pregunta, it was depicted which
button should be pressed to
accept (izquierda, verde) or reject
(bien, rojo) the description.
For each modified video, nosotros
determined the exact video
frame during which the
modification occurred (time of
modification), which we used to
precisely model modification-
related brain activation. Para
original and novel videos,
comparable time points were
chosen.

index and middle finger positioned on the two appropri-
ate buttons on a response box. Head, arm, and hand move-
ments were minimized by tight fixation with form-fitting
cushions. Participants were provided with earplugs and
headphones to attenuate scanner noise. Stimuli were pro-
jected on a screen that the participants saw via an individu-
ally adjusted mirror, which was mounted on the head coil.
High resolution T1-weighted anatomical images were
obtained with a 3-D multiplanar rapidly acquired gradient
echo sequence before functional imaging. One hundred
ninety-two slices with a thickness of 1 mm were acquired,
using a repetition time of 2130 mseg, an echo time of
2.28 mseg, a flip angle of 8°, and a field of view of 256 ×
256 mm2. Functional images of the whole brain were
acquired in interleaved order along the anterior commissure–
posterior commissure plane using a gradient-echo EPI
sequence to measure BOLD contrast. Thirty-three axial
slices with a thickness of 3 mm (voxel size 3 mm3) eran
obtained, using a repetition time of 2000 mseg, an echo
time of 30 mseg, a field of view of 192 × 192 mm2, y un
flip angle of 90°.

Processing of imaging data was conducted with SPM12
( Wellcome Trust) implemented in MATLAB ( Versión
R2020b, The MathWorks Inc.). Data were preprocessed
by slice time correction to the middle slice, movimiento
correction and realignment to the mean image, coregistra-
tion of the functional data to individual structural scans,
normalization of functional and structural images into

the standard Montreal Neurological Institute (MNI) espacio
on the basis of segmentation parameters, y espacial
smoothing using a Gaussian kernel of FWHM of 8 mm.
Además, a 128-sec high-pass temporal filter was
aplicado.

Statistical Data Analysis

fMRI Design Specifications

Statistical analyses of the fMRI data were conducted with
SPM12. We used a general linear model (GLM) for serially
autocorrelated observations ( Worsley & Friston, 1995;
Friston et al., 1994) and convolved regressors with the
canonical hemodynamic response function. Regressors
were original videos (ori), videos containing a structure
modification (str), and videos containing a content modi-
fication (estafa), each comprising 48 ensayos. For str and con
ensayos, the onsets of events were time-locked to the point
in the video at which the modification occurred (time of
modification). For ori trials, we calculated a hypothetical
time of modification (mean of times that corresponded
to points of structure and content modification in the non-
modified video) to serve as a comparable onset. Estos
conditions were modeled as events as we were interested
in the phasic effect of the prediction violation at the pre-
cise moment it occurred. To each of those regressors, nosotros
added a parametric modulator to model the repeated

1292

Revista de neurociencia cognitiva

Volumen 34, Número 7

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

presentation of each video. El 24 novel videos were
modeled as events as well, with onsets timed to the middle
of the video. Two additional regressors modeled the
18 null events and the 56 question trials. The modeled
activation of null events and questions was time-locked
to their respective onsets. Null events were modeled
as epochs, containing their full presentation time (7–
10 segundo), whereas questions were modeled as events. El
six subject-specific rigid-body transformations obtained
from realignment were included as regressors of no inter-
est. Por lo tanto, the GLM comprised 15 regressors in total.
As a first step, we calculated first-level t-contrasts for
str > ori and con > ori as well as the direct contrasts str >
con and con > str to analyze brain activity in response to
the specific modification types. Además, we calculated
the first-level t-contrasts for each condition versus novel
(nov) videos (ori > nov, str > nov, con > nov). We used
this approach to demonstrate successful retrieval of
encoded episodes ( Jainta et al., 2022) and to validate that
brain responses to episodic modifications were qualita-
tively different from novelty responses. A conjunction of
str > ori and con > ori contrasts was calculated to detect
shared effects of both modifications (Nichols, Brett,
andersson, Apostar, & Poline, 2005). As an additional, más
liberal approach to detect shared activation, we aggre-
gated str and con modified (modificación) videos to calculate the
contrast mod > ori. Gray matter masking was applied on
the first level of the analysis. For masking, we used the
smoothed individual normalized gray matter image
(8-mm FWHM), which was thresholded at .2 usando
ImCalc in SPM12 to create a binary mask. Second-level
group analyses were performed by using one-sample
t tests across participants. We applied a threshold of p < .001 on the whole-brain level and then used false discovery rate (FDR) correction at p < .05 on the cluster level to cor- rect for multiple comparisons. Brain activation patterns were visualized with the software MRIcroGL ( Version 1.2.20200331, McCausland Center for Brain Imaging, University of South Carolina). To deepen our understanding of how prediction errors contribute to memory modification, we constructed a sec- ond GLM in which we split the str and con regressors into later false alarms and correct rejections in the post-fMRI memory test. The other regressors were the same as for the other GLM, but no parametric modulators were included. We contrasted false alarms with correct rejec- tions, separately for each modification type, to investigate whether we can identify brain activation that predicts later false memories. However, this analysis did not yield any significant results. Behavioral Data Analysis The behavioral data analysis was conducted using RStudio (R Core Team, 2020; Version 1.3.1073). memory accuracy in the memory test in general, we consid- ered the corrected hit rate (i.e., the discrimination index Pr, hit rate minus false alarm rate; Snodgrass & Corwin, 1988; ratings yes and rather yes were grouped as accep- tance, and no and rather no as rejection). To better under- stand how memory for original and modified episodes was influenced in detail, we also analyzed hit rates and false alarm rates separately. Furthermore, we examined RTs in the memory test, which can serve as an indicator of how long it takes to retrieve information (correctly) from mem- ory (Collins & Quillian, 1969). Longer RTs indicate increased difficulty of retrieval because of higher cognitive processing demands (Noppeney & Price, 2004; Larsen & Plunkett, 1987), which may also occur when competing versions of an episode are processed. For the analysis of corrected hit rates as well as hit rates, false alarm rates (unmodified videos = targets, modified videos = distractors), and RTs for modified videos in the memory test (modifiedMT), we applied a 2 × 2 within- subject factorial design with the factors MODIFICATIONFMRI (yes, no) and VERSIONMT (str, con). For analyzing RTs for original videos in the memory test (originalMT), we applied a within-subject design with the factor VERSIONFMRI (ori, str, con). RTs were averaged over all trials of the same factorial combination. Several participants did not give any correct answers (i.e., rejection) in response to modifiedMT videos for one or more factorial combinations. For this reason, the number of datapoints included in this specific analysis was reduced to 23 per factorial combination. We also conducted an explorative analysis on behavioral data from the fMRI session. We calculated the error rate and mean RT according to the within-subject factor VERSIONFMRI (ori, str, con, nov) per participant. No response was given in only 0.2% of all question trials, and these trials were not further considered in the analysis. For the choice of statistical tests, data were inspected for normal distribution using the Shapiro Wilk Test. Further- more, data were checked for extreme outliers as defined as values above quartile 3 + 3 × interquartile range or lower than quartile 1–3 × interquartile range. When data were normally distributed or could be transformed to fit normal distribution (RTs; logarithmic transformation) and showed no extreme outliers, we used conventional repeated- measures ANOVA (rmANOVA). When the prerequisites for parametric analysis were not met, we used a nonpara- metric rmANOVA based on aligned rank-transformed data (package ARTool; Wobbrock, Findlater, Gergle, & Higgins, 2011; corrected hit rates, hit rates, false alarm rates, error rates). Post hoc pair-wise comparisons were conducted with paired t tests or Wilcoxon signed-ranks tests (one- tailed when comparing ori and str and ori and con, two- tailed when comparing str and con; always two-tailed for explorative analysis of fMRI task). In addition, we used one-sample Wilcoxon signed-ranks tests to test whether corrected hit rates were significantly larger than zero. To test our hypothesis that repeated presentations of modified videos in the fMRI session lead to a decrease in As descriptive statistics, we report mean values and stan- dard errors of the mean. For all behavioral analyses, we Siestrup et al. 1293 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 7 1 2 8 7 2 0 2 8 4 5 7 / / j o c n _ a _ 0 1 8 6 2 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Table 1. Whole-Brain Activation for Shared Activation of Both Episodic Modifications at FDR p < .05 (Cluster Level) MNI Coordinates Localization pIPS extending into AG Superior parietal lobe/pIPS mIPS pIPS mIPS Precuneus IFS MFG OTC Cerebellum H R L L R R R R R R L Cluster Extent (str > ori)∩(con > ori)

147

mod > ori

152

l.m.

600

l.m.

207

l.m.

119

−27

−39

−6

−67

−61

−43

−67

−37

−67

−52

−82

−10

−37

t Value

5.03

5.59

4.03

6.21

4.58

4.49

5.39

3.89

5.87

5.11

H = Hemisphere; L = Left; R = Right; str = structure modification; con = content modification; ori = original; mod = modification (aggregated);
l.m. = local maximum.

applied a significance level of α = .05. p values were
adjusted according to the Bonferroni correction for mul-
tiple comparisons (Bonferroni, 1936). If the assumption
of sphericity was violated as assessed by Mauchly’s test
of sphericity, we report Greenhouse–Geisser-corrected
degrees of freedom and p values.

RESULTADOS

fMRI Results

Behavioral Performance during fMRI Session

We calculated a nonparametric rmANOVA on error rates for
the fMRI task with the factor VERSIONFMRI (ori, str, estafa, nov).
Descriptively, error rates were generally very low for all fac-
tor levels (Mori = .028 ± .007; Mstr = .036 ± .008; Mcon =
.028 ± .006; Mnov = .073 ± .014) and did not differ signifi-
cantly, F(3, 105) = 1.99, pag = .12, ηp2 = .05. There was a
significant effect of VERSIONFMRI on RTs, F(3, 105) = 14.32,
pag < .001, ηp 2 = .29. Post hoc tests revealed that participants’ took longer to respond after novel videos than after any other version (ori vs. nov: t(35) = −5.95, p < .001; str vs. nov: t(35) = −3.79, p = .003; con vs. nov: t(35) = −4.22, p < .001; Mori = 950.450 msec ± 26.813 msec; Mstr = 978.695 msec ± 30.695 msec; Mcon = 969.741 msec ± 26.250 msec; Mnov = 1034.822 msec ± 35.786 msec). Neural Responses to Modified Episodic Cueing with original episodes. To this end, we calculated the con- junction of the whole-brain contrasts str > ori and con >
ori, which revealed a significant activation cluster in right
posterior IPS (pIPS) extending into dorsal angular gyrus
(AG; Mesa 1). As a more liberal approach to detect com-
mon activation, we contrasted modified episodes, aggre-
gated over both modification types, with original episodes
(mod > ori). De nuevo, common activation was found in pIPS,
extending into middle IPS (mIPS) in both hemispheres.
Shared activation over both modification types was also
detected in right precuneus, inferior frontal sulcus (IFS),
middle frontal gyrus (MFG), OTC, and left cerebellum
(Mesa 1, Cifra 4).

To investigate which brain regions specifically respond
to structure modifications in episodes, we inspected the
contrast str > ori. Compared with episodes without mod-
ification, structurally modified episodes activated right
SFS/PMd, MFG, IFS (Brodmann’s area [BA] 44 y 45),
supramarginal gyrus (SMG), posterior superior temporal
sulcus (pSTS), IPS, and AG. Además, we found activa-
tion in left and right precuneus (Mesa 2, Figure 5A).

To characterize brain responses to content modification
during episodic cueing, we investigated the contrast con >
ori. Compared with episodes without modification, contenido
modifications bilaterally elicited higher activity in pIPS and
OTC, including FG and parahippocampal gyrus (PHG). En
the right hemisphere, there was a significant activation clus-
ter in IFS, including BA 44 y 45. Además, we found
activation in the left cerebellum (Mesa 2, Figure 5B).

Primero, we tested whether structure and content modifica-
tions elicit common brain activation patterns compared

To further verify the specificity of brain responses to
both modification types, we also investigated the direct

1294

Revista de neurociencia cognitiva

Volumen 34, Número 7

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

significant activation in SMG and pSTS, extending into
posterior middle temporal gyrus (pMTG) in the right
hemisferio (Mesa 3, Figure 6A). Content compared
with structure modifications triggered an elevated
brain response in pIPS and OTC, including FG and PHG
(Mesa 3, Figure 6B).

Neural Effects of Episodic Reactivation

In addition to our main research question, we investigated
which brain regions were activated during episodic
retrieval in general. Para tal fin, we contrasted each type
of episode (ori, str, estafa) with novel videos (nov). Para todos
three episode types, we found significant activation in
LG (only right for con > nov), cuneus and precuneus.
Original episodes and those with structure modifications
additionally activated posterior cingulate cortex (PCC)
and ACC. For structurally modified episodes, ACC activa-
tion extended into medial frontal gyrus and we found
another significant activation cluster in right AG (Mesa 4,
Cifra 7).

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

Cifra 4. Whole-brain activation for episodic modifications, aggregated
over both modification types. FDR-corrected ( pag < .05) t-map for mod > ori contrast. Ori = original; mod = modification (aggregated);
PCUN = precuneus.

contrasts between them (str > con, con > str). en contra-
trast to content modifications, structure modifications
elicited higher activation in right precuneus, MFG,
SFS/PMd, and lingual gyrus (LG). Bilaterally, we found

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Mesa 2. Whole-Brain Activation for Different Episodic Modifications at FDR p < .05 (Cluster Level) Localization H Cluster Extent SFS/PMd MFG, extending into IFS (BA 44/45) SMG Posterior superior temporal gyrus IPS AG Precuneus pIPS IFS (BA 44) BA 45 OTC Fusiform gyrus, extending into PHG OTC Fusiform gyrus, extending into PHG Cerebellum R R R R R R L R L R R R L L R R L str > ori

389

l.m.

501

l.m.

173

l.m.

con > ori

361

489

127

l.m.

282

l.m.

299

l.m.

MNI Coordinates

t Value

−40

−49

−46

−64

−61

−67

−58

−46

−52

−40

−82

−7

−16

−10

−19

−34

5.25

5.08

6.15

5.47

5.35

5.29

5.12

4.02

7.22

7.08

5.18

3.84

7.76

6.13

9.07

5.70

5.71

−3

−27

−42

−33

−6

H = Hemisphere; L = Left; R = Right; str = structure modification; con = content modification; ori = original; l.m. = local maximum.

Siestrup et al.

1295

Cifra 5. Whole-brain
activation for different episodic
modifications. (A) FDR-
corregido ( pag < .05) t map for str > ori contrast. (B) FDR-
corregido ( pag < .05) t-map for con > ori contrast. Ori =
original; str = structure
modification; con = content
modification; (pag)IPS =
(posterior) intraparietal sulcus;
PCUN = precuneus.

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Mesa 3. Whole-Brain Activation for Direct Contrast of Different Episodic Modifications at FDR p < .05 (Cluster Level) MNI Coordinates Localization H Cluster extent Precuneus SMG pSTS, extending into pMTG MFG SFS/PMd pSTS SMG LG pIPS OTC Fusiform gyrus PHG OTC Fusiform gyrus PHG R R R R R L L R R L R R R L L L str > estafa

602

l.m.

320

l.m.

295

l.m.

con > str

218

435

l.m.

349

l.m.

−57

−24

−42

−33

−49

−37

−34

−64

−40

−79

−55

−61

−43

−31

−61

−49

−34

−4

−10

−19

−16

−7

−16

t Value

5.45

6.21

4.80

5.65

5.28

5.01

4.08

5.09

5.00

5.97

10.92

7.14

3.87

8.26

5.87

5.09

H = Hemisphere; L = Left; R = Right; str = structure modification; con = content modification; ori = original; l.m. = local maximum.

1296

Revista de neurociencia cognitiva

Volumen 34, Número 7

Cifra 6. Whole-brain
activation for direct contrasts
between episodic
modifications. (A) FDR-
corregido ( pag < .05) t-map for str > con contrast. (B) FDR-
corregido ( pag < .05) t-map for con > str contrast. Str =
structure modification; con =
content modification; PCUN =
precuneus.

Post-fMRI Memory Test

Corrected Hit Rates

Primero, we investigated the general memory accuracy in the
memory test, using the corrected hit rate. We confirmed
that participants did not merely guess when rating videos
as corrected hit rates for each factorial combination were
significantly larger than zero (no-str: z = −4.43, pag < .001; yes-str: z = −4.17, p < .001; no-con: z = −5.44, p < .001; yes-con: z = −5.30, p < .001). A nonparametric rmANOVA with the factors MODIFICATIONFMRI (yes, no) and VERSIONMT (str, con) revealed a significant main effect of MODIFICATIONFMRI, F(1, 35) = 10.17, p = .003, ηp2 = .23, which was driven by higher corrected hit rates for no (M = .580 ± .041) compared with yes (M = .460 ± .035), indicating a better memory performance when no modifications had been presented during the fMRI session. There was also a significant main effect of VERSIONMT, F(1, 35) = 109.65, p < .001, ηp2 = .76, which was explained by higher corrected hit rates values for con (M = .757 ± .038) than for str (M = .283 ± .042). There was no significant interaction of MODIFICATIONFMRI and VERSIONMT, F(1, 35) = 0.11, p = .75, ηp2 = .00 (Figure 8A). Hit Rates Hit rates for originalMT episodes were close to ceiling for all factorial combinations. A nonparametric rmANOVA with the factors MODIFICATIONFMRI (yes, no) and VERSIONMT (str, con) revealed a significant main effect of MODIFICATIONFMRI, F(1, 35) = 12.43, p = .001, ηp2 = .26, which was driven by higher hit rates for no (M = .962 ± .016) than for yes (M = .929 ± .014). Thus, participants were more prone to erro- neously reject original episodes after the presentation of modified videos during the fMRI session. In addition, we found a significant main effect of VERSIONMT, F(1, 35) = 5.36, p = .027, ηp2 = .13, with higher hit rates for con (M = .955 ± .011) than for str (M = .936 ± .015). This indi- cates that participants were generally better at recognizing originally encoded episodes of which they also knew the content-modified version. The interaction of both factors was also significant, F(1, 35) = 8.18, p = .007, ηp2 = .19, and post hoc pairwise comparisons revealed that hit rates only decreased significantly after pre-experience with structure (z = −2.70, p = .015), but not content-modified episodes (z = −0.47, p = 1). Please note, however, that all differences in absolute values were quite small and thus should be interpreted with caution (Figure 8B). False Alarm Rates We computed a nonparametric rmANOVA with the factors MODIFICATIONFMRI (yes, no) and VERSIONMT (str, con) to ana- lyze false alarm rates for modifiedMT episodes. There was a significant main effect of MODIFICATIONFMRI, F(1, 35) = 4.93, p = .033, ηp2 = .12, which was driven by higher false alarm rates for yes (M = .469 ± .039) than for no (M = .382 ± .037). Thus, participants were more prone to accept mod- ified episode videos as originally encoded when a modi- fied version had already been presented during the fMRI session. In addition, there was a significant main effect of VERSIONMT, F(1, 35) = 113.51, p < .001, ηp2 = .76, as false alarm rates were higher for str (M = .653 ± .046) than for con (M = .198 ± .035). This shows that participants gen- erally accepted videos with modified structure much more readily than alternatives with modified content. We found a nonsignificant trend for an interaction of MODIFICA- TIONFMRI and VERSIONMT, F(1, 35) = 3.25, p = .080, ηp2 = .08. Descriptively, false alarm rates for structure modified videos were increased less by the previous experience of Siestrup et al. 1297 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 7 1 2 8 7 2 0 2 8 4 5 7 / / j o c n _ a _ 0 1 8 6 2 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Table 4. Whole-Brain Activation for Episodic recall at FDR p < .05 (Cluster Level) MNI Coordinates Localization H Cluster Extent x y z t Value PCC ACC LG Cuneus, extending into precuneus AG Medial frontal gyrus ACC PCC LG Cuneus, extending into precuneus LG Cuneus Precuneus R + L L R R R L R R + L L R R + L R R L R R L R L ori > nov

168

292

l.m.

2508

l.m.

str > nov

121

242

l.m.

144

2459

l.m.

con > nov

1382

l.m.

−6

−3

−9

−22

−91

−88

−85

−55

−22

−91

−85

−88

−94

−64

−70

−4

−7

7.73

5.21

5.18

13.11

11.13

9.25

5.74

5.63

4.99

4.42

9.02

13.30

8.99

7.91

11.66

5.93

5.37

5.84

5.55

H = Hemisphere; L = Left; R = Right; str = structure modification; con = content modification; ori = original; nov = novel; l.m. = local maximum.

Cifra 7. Whole-brain activation for videos showing original or slightly modified episodes, contrasted with novel videos. (A) FDR-corrected ( pag < .05) t map for ori > nov contrast. (B) FDR-corrected ( pag < .05) t-map for str > nov contrast. (C) FDR-corrected ( pag < .05) t-map for con > nov
contrast. Ori = original; str = structure modification; con = content modification; nov = novel ; CUN = cuneus; PCUN = precuneus.

1298

Revista de neurociencia cognitiva

Volumen 34, Número 7

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Cifra 8. Behavioral results
from post-fMRI memory test.
For modifiedMT and originalMT
videos, participants rated
whether they showed originally
encoded episodes or not.
OriginalMT videos were the
targets whereas modifiedMT
videos were distractors. (A)
Corrected hit rate. Estadísticas:
nonparametric rmANOVA with
the factors MODIFICATIONFMRI
(Sí, No) and VERSIONMT (str,
estafa), norte = 36. (B) Hit rates for
originalMT videos. Estadísticas:
nonparametric rmANOVA with
the factors MODIFICATIONFMRI
(Sí, No) and VERSIONMT (str,
estafa) and Wilcoxon signed-ranks
pruebas, norte = 36. (C) False alarm
rates for modifiedMT videos.
Estadísticas: nonparametric
rmANOVA with the factors
MODIFICATIONFMRI (Sí, No) y
VERSIONMT (str, estafa), norte = 36. (D)
RTs for modifiedMT videos. Estadísticas: rmANOVA with the factors MODIFICATIONFMRI (Sí, No) and VERSIONMT (str, estafa), norte = 23. (mi) RTs for originalMT
videos. Estadísticas: rmANOVA with the factor VERSIONFMRT (ori, str, estafa), norte = 36. Bar plots show means and standard errors. *pag < .05, **p < .01, ***p < .001. Ori = original; str = structure modification; con = content modification; a versus b indicates the main effect of MODIFICATIONFMRI. episodic modifications than those for content-modified videos (Figure 8C). To control for a general acceptance bias, we compared false alarm rates for novel videos and modifiedMT videos using the Wilcoxon signed-ranks test (one-tailed). False alarm rates for novel videos were at a floor level (M = .007 ± .005) and significantly lower than those for modifiedMT videos (z = −5.35, p < .001; M = .425 ± .034; Figure 8C). RTs A rmANOVA on RTs for modifiedMT videos with the factors MODIFICATIONFMRI (yes, no) and VERSIONMT (str, con) revealed a near significant effect of MODIFICATIONFMRI, F(1, 22) = 3.86, p = .062, ηp2 = .15, as participants tended to take longer to correctly reject modifiedMT episodes when the same had already been presented in the scanner (M = 782.520 msec ± 81.481 msec vs. M = 665.384 msec ± 79.401 msec). Descriptively, RTs were shorter for videos with content than with structure modification (M = 661.447 msec ± 76.010 msec vs. M = 786.458 msec ± 85.829 msec), but this difference did not reach signifi- cance either, F(1, 22) = 3.06, p = .094, ηp2 = .12. We found no significant interaction effect, F(1, 22) = 0.58, p = .45, ηp2 = .03 (Figure 8D). There was a near significant effect of VERSIONFMRI (ori, str, con) on RTs for originalMT videos, F(1.72, 60.09) = 3.21, p = .055, ηp2 = .08. Descrip- tively, RTs were longest for str (M = 923.092 msec ± 117.161 msec) compared with ori (M = 697.608 msec ± 37.210 msec) and con (M = 729.762 msec ± 62.904 msec; Figure 8E). DISCUSSION In this study, we investigated brain and behavioral responses to violation of episodic expectancy induced by cues with modified details in structure or content. As hypothesized, brain responses differed for these two types of episodic prediction errors, reflecting the processing of divergent object and structure information. Modified epi- sodes were mistaken for veridical originals more often in a post-fMRI memory test when already presented during the fMRI scan, whereas correct recognition of originally encoded episodes decreased. Together, findings provide evidence that different types of mnemonic prediction errors are processed differently by the brain and may contribute to memory changes. Neural Responses to Episodes and Episodic Modifications The presentation of original and slightly modified videos of encoded episodes recruited several brain regions asso- ciated with episodic retrieval compared with novel videos ( Jeong, Chung, & Kim, 2015; Rugg & Vilberg, 2013; Wiggs, Weisberg, & Martin, 1999). The activation patterns for epi- sodic retrieval closely resemble the one we detected in a previous study with a similar paradigm, comprising ACC, PCC, precuneus, cuneus, and LG ( Jainta et al., 2022). Thus, in parallel to replicating our previous findings, we vali- dated that participants had successfully encoded episodes and that the presentation of videos thereof cued episodic memories. Importantly, this was true for original and for modified episodes. This was to be expected because of Siestrup et al. 1299 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 7 1 2 8 7 2 0 2 8 4 5 7 / / j o c n _ a _ 0 1 8 6 2 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 the subtle changes in modified episodes, which, overall, were still highly familiar to the participants. New content and structure information of the episodic cue was expected to draw on distinct brain areas, but also to share some common activation in medial frontal cortex and the hippocampal formation. Although we could not confirm this hypothesis in this study, we found significant common activation in (right) pIPS, as revealed by the con- junction analysis and the aggregated modification con- trast. Therefore, we suggest that superior parietal regions might be involved in processing of prediction errors in the context of episodic memory, potentially by guiding updat- ing mechanisms. This interpretation fits the finding that dorsal parietal cortex plays an important role in the forma- tion of episodic memories (Uncapher & Wagner, 2009). In addition, this area is well known to be involved in the reori- entation of attention to salient and unexpected stimuli (Molenberghs, Mesulam, Peeters, & Vandenberghe, 2007; Corbetta & Shulman, 2002). It has been suggested that the superior parietal lobe, including pIPS, regulates top–down attention in memory. This is especially impor- tant when additional postretrieval processes are necessary to discriminate between what is true memorized content and what is not (Cabeza et al., 2011; Ciaramelli, Grady, & Moscovitch, 2008). Furthermore, the more liberal approach of aggregating episodes over different types of modification revealed common activation in right IFS. This reflects our previous finding that activity in ventrolateral pFC increases for inconsistent or highly informative detail in observed actions (El-Sourani et al., 2019; Hrkać, Wurm, Kühn, & Schubotz, 2015; Wurm & Schubotz, 2012). Structure and content modifications each recruited a set of brain regions unique to the modification type. Activa- tion patterns in contrast to original episodes closely resembled those for the direct contrasts between struc- ture and content modifications, indicating high specificity of brain responses for each modification type. On the one hand, we had expected that structure mod- ifications specifically lead to activation in brain regions involved in the temporal organization of episodes. We found that structure modifications co-activated right PMd/SFS, SMG, pSTS, and precuneus, suggesting this net- work contributes to the updating of predictive models because of unexpected new structure in episodes. This interpretation is consistent with previous reports about the functional characteristics of said areas. Accordingly, activity of a region comprising PMd/SFS is related to link- ing successive action steps (Pomp et al., 2021; Hrkać et al., 2014; Schubotz et al., 2012; Stadler et al., 2011) and could contribute to updating the current event or action model with respect to each next segment (Pomp et al., 2021; Schubotz et al., 2012; Tamber-Rosenau et al., 2011; Kurby & Zacks, 2008). SMG and precuneus have been demon- strated to be important for the sequential organization of memories (Foudil, Kwok, & Macaluso, 2020; Guidali, Pisoni, Bolognini, & Papagno, 2019) and involved in sequential learning (Burke, Bramley, Gonzalez, & McKeefry, 2013; Oishi et al., 2005). In line with this, we recently found that SMG is sensitive for perceived break points in actions (Pomp et al., 2021). Activity in (right) pSTS is char- acteristic for the processing of biological motion (Gilaie- Dotan, Kanai, Bahrami, Rees, & Saygin, 2013; Grossman, Battelli, & Pascual-Leone, 2005) and, in this context, action adaptation (Thurman, van Boxtel, Monti, Chiang, & Lu, 2016). pSTS activation for actions has been found to be goal-sensitive, responding more strongly when expected spatial transport targets are not met (Shultz, Lee, Pelphrey, & McCarthy, 2011). On the other hand, we found content modifications to specifically recruit pIPS and OTC, including FG, which were hypothesized on the basis of their role for processing of object properties in the context of actions (El-Sourani et al., 2019; Lingnau & Downing, 2015; Wiggett & Downing, 2011; Reber et al., 2005; Grill-Spector, Kourtzi, & Kanwisher, 2001). More specifically, pIPS encodes basic visual features of graspable objects (Mruczek, von Loga, & Kastner, 2013; Creem-Regehr, 2009), reflecting the inter- action with toy objects in our paradigm. In addition, con- tent modifications elicited activity in the hippocampal formation (PHG), which likely represents ongoing learning because of the detected mismatch. In general, hippocampus and PHG are important in learning contexts (Davachi & Wagner, 2002; Köhler, Crane, & Milner, 2002; O’Reilly & Rudy, 2000; Aguirre, Detre, Alsop, & D’Esposito, 1996) and there is evidence that the PHG is involved in processing of competing memories (Kuhl, Bainbridge, & Chun, 2012). Moreover, the hippocampal formation is believed to generate mismatch signals when predictions do not fit perceptual inputs (Long et al., 2016; Duncan, Curtis, & Davachi, 2009; Kumaran & Maguire, 2007). Because our post-fMRI memory test data imply that con- tent changes were more salient than structural changes, one could speculate that the discrepancy between what was predicted and what was perceived in case of content modifications was strong enough to be reflected in the activation of the hippocampal formation. Interestingly, the overall activation pattern we found for content modifications closely resembles the one Gläscher, Daw, Dayan, and O’Doherty (2010) identified for what they call state prediction errors, which is characterized by a mismatch of the expected and current state. Taken together, we found that structure and content modifications activated distinct networks, each specifically representing the processing of the type of unexpected new information. We therefore achieved the main aim of this study, that is, characterize brain responses to different types of prediction errors. Influence of Episodic Modifications on Post-fMRI Memory Performance As expected, those original episodes that had been pre- sented in a modified version during the fMRI were recalled 1300 Journal of Cognitive Neuroscience Volume 34, Number 7 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 7 1 2 8 7 2 0 2 8 4 5 7 / / j o c n _ a _ 0 1 8 6 2 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 less reliably, as reflected by a lower corrected hit rate. More specifically, previously modified videos were later more often mistaken for original ones, as evidenced by signifi- cantly increased false alarm rates. Others have reported that mnemonic prediction errors lead to an intrusion of new information into an established memory repertoire (Sinclair & Barense, 2018; Long et al., 2016). Also, in our study, the recognition of original videos was impaired after encoun- ters of alternative versions. Similarly, Kim et al. reported that prediction violations led to decreased recognition of original memory content (Kim et al., 2014). In particular, prior presentation of structurally modified videos led to a decrease in the hit rate in the post-fMRI memory test. This might be taken as a first hint that different types of episodic modifications could influence memory traces differently, which would fit the specific brain responses we detected for structure and content modifications. Interestingly, par- ticipants tended to take longer to correctly classify a video if they had already seen the same video in a modified version during fMRI. Longer response times in cued-memory para- digms are interpreted as indicative of increased difficulty of retrieval because of higher cognitive processing demands (Noppeney & Price, 2004; Larsen & Plunkett, 1987). Thus, it is likely that it became more difficult to differentiate between alternative competing versions of episodes when versions diverging from the original experience had been already encountered in the fMRI session. However, effects concerning RTs need to be interpreted with caution because they only approached significance. Taken together, the behavioral findings suggest that structure and content modifications during cueing of episodic retrieval influenced subsequent memory for these episodes. Our findings corroborate the observation that mnemonic prediction errors can trigger episodic memory modification (Sinclair & Barense, 2019). What remains unclear is how exactly memory traces were influenced by our intervention. For example, it has been discussed that memory modification can result from an interference of old and new memory traces (Sinclair & Barense, 2018, 2019; Klingmüller, Caplan, & Sommer, 2017; Sederberg, Gershman, Polyn, & Norman, 2011) or from source confusion (Hekkanen & McEvoy, 2002), which could both explain our findings. Then again, partic- ipants remembered correctly that novel videos had not been part of the original episode repertoire although novels had been repeatedly presented during the fMRI experiment as well. This speaks against source confusion in its simplest form as an explanation of the results of our post-fMRI memory test. Another interesting behavioral finding was that partici- pants generally had a strong tendency to accept structur- ally modified versions as originals in the memory test. Although both types of modifications resulted from a single change in the story, it is likely that structural modifications were generally less salient than content modifications. This would be matched by the fact that after the fMRI session, nearly all participants (86%) reported noticing at least one object swap, whereas only half of them had noticed a change in the sequence of action steps (53%). Recently, it was reported that memory perfor- mance following prediction errors differed depending on whether changes were detected (and remembered) by participants or not ( Wahlheim & Zacks, 2019). Whereas undetected changes led to reduced memory performance, detected changes had the opposite effect. Depending on contextual factors, prediction errors can even improve subsequent memory (Greve, Cooper, Kaula, Anderson, & Henson, 2017; Smith, Hasinski, & Sederberg, 2013). Although our behavioral results suggest that structural changes were rarely noticed, corresponding to a reduced memory performance, more frequent detection in the case of content modification did not lead to enhanced, but on the contrary, also to decreased memory perfor- mance. Thus, the final impact of a prediction violation on memory appears to be multifactorially determined. Limitations and Implications for Future Research One factor that may limit the generalizability of our find- ings is that, for practical reasons, only women participated in the study. However, because the processing of episodic memory in the brain seems to be broadly similar between women and men (Nyberg, Habib, & Herlitz, 2000), we are confident that our findings are applicable to a more gen- eral population. In the future, our paradigm could be adapted to circumvent such practical limitations, for exam- ple, by applying virtual reality techniques so that encoding could be detached from the true physical appearance of participants’ hands. Second, we used new content and structure informa- tion, which contrasted details of the encoded episodes to elicit prediction errors. Although these interventions can also be interpreted as contextual and associative novelty, respectively, the unexpected new input within the familiar context will give rise to mnemonic prediction errors according to the predictive coding framework (see the work of Reichardt et al., 2020, for a review). In addition, we could show that episodes with structure and content modifications, in contrast to completely novel videos, recruited different brain regions associated with episodic retrieval (Jeong et al., 2015; Rugg & Vilberg, 2013; Wiggs et al., 1999). Still, it would be interesting to find a way to keep the novelty constant and have participants make active predictions that are then either violated or not. Our findings from the post-fMRI memory test revealed that structure modifications were likely less salient and harder to detect than content modifications. Thus, we can- not exclude that the neural differences we detected were confounded by differences in prediction error strength. However, brain responses to structure modifications were highly specific and located in hypothesized areas. More- over, structure modifications elicited equally strong activa- tion as content modifications. It is thus highly unlikely that those differences simply arose because of quantitative Siestrup et al. 1301 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 7 1 2 8 7 2 0 2 8 4 5 7 / / j o c n _ a _ 0 1 8 6 2 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 difference between modification types. For these reasons, we are confident that our findings indeed represent differ- ential neural processing because of different types of epi- sodic information. As a caveat, we wish to point out that, in normal life, episodic memories are not trained or repeated in the strict sense as they were in our paradigm. Although exper- imental procedures must be rigorous to be able to test hypotheses, the rigor takes away from the applicability of the research. Last, our trial-wise analysis of later false alarms and correct rejections in the memory test did not reveal signif- icant brain activation predicting false memories. Our aim for future investigations is to optimize our paradigm to further analyze how later true and false memories are encoded by the brain. Conclusion When recalling episodes, our memory can change, for instance because of prediction errors in the reactivation process. Our results suggest that structural and content prediction errors in episode retrieval differ in their neural processing. The tendency to misclassify modified epi- sodes as originally experienced episodes increased after experiencing repeated structural and content prediction errors. Accordingly, different types of prediction errors can confuse episodic memory and possibly lead to the emergence of alternative versions of the same memory trace. Our results may provide a fruitful starting point for further research on the mutability of episodic memories. Acknowledgments The authors thank Monika Mertens, Lena Puder, Simon Wieczorek, Jamuna Halscheid, Leandra Feldhusen, and Anne Glombitza for their help during data collection. Furthermore, we thank Annika Garlichs, Helena Sydlik, and Yuyi Xu for their assistance during the creation of stimulus material and Christin Schwarzer for training new student assistants. Last, we thank Jennifer Pomp, Lena Schliephake, Falko Mecklenbrauck, and Nina Heins for advice regarding data analysis and the members of research unit FOR 2812 for valuable discussions. Reprint requests should be sent to Sophie Siestrup, University of Münster, Fliednerstraße 21, 48149 Münster, or via e-mail: s.siestrup@uni-muenster.de. Data Availability Statement All data reported here is publicly available at h t t p s : / / o s f . i o / m 7 d c u / ? v i e w _ o n l y =575d6ed3fbf544ada3bcb0519c86f94b. Author Contributions Sophie Siestrup: Formal analysis; Investigation; Methodol- ogy; Visualization; Writing—Original draft; Writing— Review & editing. Benjamin Jainta: Investigation; Methodology; Writing—Review & editing. Nadiya El-Sourani: Methodology; Writing—Review & editing. Ima Trempler: Formal analysis; Methodology; Writing—Review & editing. Moritz F. Wurm: Writing—Review & editing. Oliver T. Wolf: Writing—Review & editing. Sen Cheng: Conceptu- alization; Writing—Review & editing. Ricarda I. Schubotz: Conceptualization; Funding acquisition; Methodology; Resources; Supervision; Writing—Original draft; Writing— Review & editing. Funding Information This work was funded by the German Research Founda- tion (Deutsche Forschungsgemeinschaft) – project numbers 419037023, 419039274, 419037518. The funders had no role in study design, data collection, analysis and interpretation, decision to publish, or writing of the report. Diversity in Citation Practices Retrospective analysis of the citations in every article pub- lished in this journal from 2010 to 2021 reveals a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender iden- tification of first author/last author) publishing in the Jour- nal of Cognitive Neuroscience ( JoCN) during this period were M(an)/M = .407, W(oman)/M = .32, M/ W = .115, and W/ W = .159, the comparable proportions for the arti- cles that these authorship teams cited were M/M = .549, W/M = .257, M/ W = .109, and W/ W = .085 (Postle and Fulvio, JoCN, 34:1, pp. 1–3). Consequently, JoCN encour- ages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the oppor- tunity to report their article’s gender citation balance. REFERENCES Aguirre, G. K., Detre, J. A., Alsop, D. C., & D’Esposito, M. (1996). The parahippocampus subserves topographical learning in man. Cerebral Cortex, 6, 823–829. https://doi.org /10.1093/cercor/6.6.823, PubMed: 8922339 Barron, H. C., Auksztulewicz, R., & Friston, K. (2020). Prediction and memory: A predictive coding account. Progress in Neurobiology, 192, 101821. https://doi.org/10.1016/j .pneurobio.2020.101821, PubMed: 32446883 Barto, A., Mirolli, M., & Baldassarre, G. (2013). Novelty or surprise? Frontiers in Psychology, 4, 907. https://doi.org/10 .3389/fpsyg.2013.00907, PubMed: 24376428 Bein, O., Duncan, K., & Davachi, L. (2020). Mnemonic prediction errors bias hippocampal states. Nature Communications, 11, 3451. https://doi.org/10.1038/s41467 -020-17287-1, PubMed: 32651370 Bonferroni, C. E. (1936). Teoria statistica delle classi e calcolo delle probabilità. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze, 8, 3–62. Burke, M. R., Bramley, P., Gonzalez, C. C., & McKeefry, D. J. (2013). The contribution of the right supra-marginal gyrus to sequence learning in eye movements. Neuropsychologia, 51, 1302 Journal of Cognitive Neuroscience Volume 34, Number 7 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 7 1 2 8 7 2 0 2 8 4 5 7 / / j o c n _ a _ 0 1 8 6 2 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 3048–3056. https://doi.org/10.1016/j.neuropsychologia.2013 .10.007, PubMed: 24157539 Cabeza, R., Mazuz, Y. S., Stokes, J., Kragel, J. E., Woldorff, M. G., Ciaramelli, E., et al. (2011). Overlapping parietal activity in memory and perception: Evidence for the attention to memory model. Journal of Cognitive Neuroscience, 23, 3209–3217. https://doi.org/10.1162/jocn_a_00065, PubMed: 21568633 Ciaramelli, E., Grady, C. L., & Moscovitch, M. (2008). Top–down and bottom–up attention to memory: A hypothesis (AtoM) on the role of the posterior parietal cortex in memory retrieval. Neuropsychologia, 46, 1828–1851. https://doi.org /10.1016/j.neuropsychologia.2008.03.022, PubMed: 18471837 Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning and Verbal Behavior, 8, 240–247. https://doi.org/10.1016/S0022-5371(69) 80069-1 Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201–215. https://doi.org/10.1038/nrn755, PubMed: 11994752 Creem-Regehr, S. H. (2009). Sensory-motor and cognitive functions of the human posterior parietal cortex involved in manual actions. Neurobiology of Learning and Memory, 91, 166–171. https://doi.org/10.1016/j.nlm.2008.10.004, PubMed: 18996216 Davachi, L., & Wagner, A. D. (2002). Hippocampal contributions to episodic encoding: Insights from relational and item-based learning. Journal of Neurophysiology, 88, 982–990. https://doi .org/10.1152/jn.2002.88.2.982, PubMed: 12163547 Duncan, K., Curtis, C., & Davachi, L. (2009). Distinct memory signatures in the hippocampus: Intentional states distinguish match and mismatch enhancement signals. Journal of Neuroscience, 29, 131–139. https://doi.org/10.1523 /JNEUROSCI.2998-08.2009, PubMed: 19129391 El-Sourani, N., Trempler, I., Wurm, M. F., Fink, G. R., & Schubotz, R. I. (2019). Predictive impact of contextual objects during action observation: Evidence from functional magnetic resonance imaging. Journal of Cognitive Neuroscience, 32, 326–337. https://doi.org/10.1162/jocn_a _01480, PubMed: 31617822 Euston, D. R., Gruber, A. J., & McNaughton, B. L. (2012). The role of medial prefrontal cortex in memory and decision making. Neuron, 76, 1057–1070. https://doi.org/10.1016/j .neuron.2012.12.002, PubMed: 23259943 Exton-McGuinness, M. T. J., Lee, J. L. C., & Reichelt, A. C. (2015). Updating memories—The role of prediction errors in memory reconsolidation. Behavioural Brain Research, 278, 375–384. https://doi.org/10.1016/j.bbr.2014.10.011, PubMed: 25453746 Fernández, R. S., Boccia, M. M., & Pedreira, M. E. (2016). The fate of memory: Reconsolidation and the case of Prediction Error. Neuroscience and Biobehavioral Reviews, 68, 423–441. https://doi.org/10.1016/j.neubiorev.2016.06.004, PubMed: 27287939 Foudil, S. A., Kwok, S. C., & Macaluso, E. (2020). Context- dependent coding of temporal distance between cinematic events in the human precuneus. Journal of Neuroscience, 40, 2129–2138. https://doi.org/10.1523/JNEUROSCI.2296-19 .2020, PubMed: 31996453 Franz, E. A., Ford, S., & Werner, S. (2007). Brain and cognitive processes of imitation in bimanual situations: Making inferences about mirror neuron systems. Brain Research, 1145, 138–149. https://doi.org/10.1016/j.brainres.2007.01.136, PubMed: 17349983 Friston, K. (2005). A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360, 815–836. https://doi.org/10.1098/rstb.2005.1622, PubMed: 15937014 Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J.-P., Frith, C. D., & Frackowiak, R. S. J. (1994). Statistical parametric maps in functional imaging: A general linear approach. Human Brain Mapping, 2, 189–210. https://doi.org/10.1002 /hbm.460020402 Friston, K., & Kiebel, S. (2009). Predictive coding under the free-energy principle. Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences, 364, 1211–1221. https://doi.org/10.1098/rstb.2008.0300, PubMed: 19528002 Gilaie-Dotan, S., Kanai, R., Bahrami, B., Rees, G., & Saygin, A. P. (2013). Neuroanatomical correlates of biological motion detection. Neuropsychologia, 51, 457–463. https://doi.org/10 .1016/j.neuropsychologia.2012.11.027, PubMed: 23211992 Gläscher, J., Daw, N., Dayan, P., & O’Doherty, J. P. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66, 585–595. https://doi.org /10.1016/j.neuron.2010.04.016, PubMed: 20510862 Greve, A., Cooper, E., Kaula, A., Anderson, M. C., & Henson, R. (2017). Does prediction error drive one-shot declarative learning? Journal of Memory and Language, 94, 149–165. https://doi.org/10.1016/j.jml.2016.11.001, PubMed: 28579691 Griffiths, D., Dickinson, A., & Clayton, N. (1999). Episodic memory: What can animals remember about their past? Trends in Cognitive Sciences, 3, 74–80. https://doi.org/10 .1016/S1364-6613(98)01272-8, PubMed: 10234230 Grill-Spector, K., Kourtzi, Z., & Kanwisher, N. (2001). The lateral occipital complex and its role in object recognition. Vision Research, 41, 1409–1422. https://doi.org/10.1016/S0042-6989 (01)00073-6, PubMed: 11322983 Grossman, E. D., Battelli, L., & Pascual-Leone, A. (2005). Repetitive TMS over posterior STS disrupts perception of biological motion. Vision Research, 45, 2847–2853. https:// doi.org/10.1016/j.visres.2005.05.027, PubMed: 16039692 Guidali, G., Pisoni, A., Bolognini, N., & Papagno, C. (2019). Keeping order in the brain: The supramarginal gyrus and serial order in short-term memory. Cortex, 119, 89–99. https://doi.org/10.1016/j.cortex.2019.04.009, PubMed: 31091486 Hekkanen, S. T., & McEvoy, C. (2002). False memories and source-monitoring problems: Criterion differences. Applied Cognitive Psychology, 16, 73–85. https://doi.org/10.1002 /acp.753 Horner, A. J., & Doeller, C. F. (2017). Plasticity of hippocampal memories in humans. Current Opinion in Neurobiology, 43, 102–109. https://doi.org/10.1016/j.conb.2017.02.004, PubMed: 28260633 Hrkać, M., Wurm, M. F., Kühn, A. B., & Schubotz, R. I. (2015). Objects mediate goal integration in ventrolateral prefrontal cortex during action observation. PLoS One, 10, e0134316. https://doi.org/10.1371/journal.pone.0134316, PubMed: 26218102 Hrkać, M., Wurm, M. F., & Schubotz, R. I. (2014). Action observers implicitly expect actors to act goal-coherently, even if they do not: An fMRI study. Human Brain Mapping, 35, 2178–2190. https://doi.org/10.1002/hbm.22319, PubMed: 23983202 Jainta, B., Siestrup, S., El-Sourani, N., Trempler, I., Wurm, M. F., Werning, M., et al. (2022). Seeing what i did (not): Cerebral and behavioral effects of agency and perspective on episodic memory re-activation. Frontiers in Behavioral Neuroscience, 15, 793115. https://doi.org/10.3389/fnbeh .2021.793115, PubMed: 35069141 Jeong, W., Chung, C. K., & Kim, J. S. (2015). Episodic memory in aspects of large-scale brain networks. Frontiers in Human Neuroscience, 9, 454. https://doi.org/10.3389/fnhum.2015 .00454, PubMed: 26321939 Siestrup et al. 1303 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u / j / o c n a r t i c e - p d l f / / / 3 4 7 1 2 8 7 2 0 2 8 4 5 7 / / j o c n _ a _ 0 1 8 6 2 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Kim, G., Lewis-Peacock, J. A., Norman, K. A., & Turk-Browne, N. B. (2014). Pruning of memories by context-based prediction error. Proceedings of the National Academy of Sciences, U.S.A., 111, 8997–9002. https://doi.org/10.1073/pnas .1319438111, PubMed: 24889631 Klingmüller, A., Caplan, J. B., & Sommer, T. (2017). Intrusions in episodic memory: Reconsolidation or interference? Learning and Memory, 24, 216–224. https://doi.org/10.1101 /lm.045047.117, PubMed: 28416633 Köhler, S., Crane, J., & Milner, B. (2002). Differential contributions of the parahippocampal place area and the anterior hippocampus to human memory for scenes. Hippocampus, 12, 718–723. https://doi.org/10.1002/hipo .10077, PubMed: 12542224 Kuhl, B. A., Bainbridge, W. A., & Chun, M. M. (2012). Neural reactivation reveals mechanisms for updating memory. Journal of Neuroscience, 32, 3453–3461. https://doi.org/10 .1523/JNEUROSCI.5846-11.2012, PubMed: 22399768 Kumaran, D., & Maguire, E. A. (2007). Which computational mechanisms operate in the hippocampus during novelty detection? Hippocampus, 17, 735–748. https://doi.org/10 .1002/hipo.20326, PubMed: 17598148 Kurby, C. A., & Zacks, J. M. (2008). Segmentation in the perception and memory of events. Trends in Cognitive Sciences, 12, 72–79. https://doi.org/10.1016/j.tics.2007.11.004, PubMed: 18178125 Larsen, S. F., & Plunkett, K. (1987). Remembering experienced and reported events. Applied Cognitive Psychology, 1, 15–26. https://doi.org/10.1002/acp.2350010104 Lee, J. L. C., Nader, K., & Schiller, D. (2017). An update on memory reconsolidation updating. Trends in Cognitive Sciences, 21, 531–545. https://doi.org/10.1016/j.tics.2017.04 .006, PubMed: 28495311 Lingnau, A., & Downing, P. E. (2015). The lateral occipitotemporal cortex in action. Trends in Cognitive Sciences, 19, 268–277. https://doi.org/10.1016/j.tics.2015 .03.006, PubMed: 25843544 Long, N. M., Lee, H., & Kuhl, B. A. (2016). Hippocampal mismatch signals are modulated by the strength of neural predictions and their similarity to outcomes. Journal of Neuroscience, 36, 12677–12687. https://doi.org/10.1523 /JNEUROSCI.1850-16.2016, PubMed: 27821577 Maguire, E. A., Intraub, H., & Mullally, S. L. (2016). Scenes, spaces, and memory traces: What does the hippocampus do? Neuroscientist, 22, 432–439. https://doi.org/10.1177 /1073858415600389, PubMed: 26276163 Molenberghs, P., Mesulam, M. M., Peeters, R., & Vandenberghe, R. R. C. (2007). Remapping attentional priorities: Differential contribution of superior parietal lobule and intraparietal sulcus. Cerebral Cortex, 17, 2703–2712. https://doi.org/10 .1093/cercor/bhl179, PubMed: 17264251 Noppeney, U., & Price, C. J. (2004). Retrieval of abstract semantics. Neuroimage, 22, 164–170. https://doi.org/10.1016 /j.neuroimage.2003.12.010, PubMed: 15110006 Nyberg, L., Habib, R., & Herlitz, A. (2000). Brain activation during episodic memory retrieval: Sex differences. Acta Psychologica, 105, 181–194. https://doi.org/10.1016/s0001 -6918(00)00060-3, PubMed: 11194411 Oishi, K., Toma, K., Bagarinao, E. T., Matsuo, K., Nakai, T., Chihara, K., et al. (2005). Activation of the precuneus is related to reduced reaction time in serial reaction time tasks. Neuroscience Research, 52, 37–45. https://doi.org/10.1016/j .neures.2005.01.008, PubMed: 15811551 Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9, 97–113. https://doi.org/10.1016/0028-3932(71)90067-4, PubMed: 5146491 O’Reilly, R. C., & Rudy, J. W. (2000). Computational principles of learning in the neocortex and hippocampus. Hippocampus, 10, 389–397. https://doi.org/10.1002/1098 -1063(2000)10:4<389::AID-HIPO5>3.0.CO;2-PAG

Peters, GRAMO. J., David, C. NORTE., marco, METRO. D., & Herrero, D. METRO. (2013).
The medial prefrontal cortex is critical for memory retrieval
and resolving interference. Learning and Memory, 20,
201–209. https://doi.org/10.1101/lm.029249.112, PubMed:
23512936

Pomp, J., Heins, NORTE., Trempler, I., Kulvicius, T., Tamosiunaite,

METRO., Mecklenbrauck, F., et al. (2021). Touching events
predict human action segmentation in brain and behavior.
Neuroimagen, 243, 118534. https://doi.org/10.1016/j
.neuroimage.2021.118534, PubMed: 34469813

R Core Team. (2020). R: A language and environment for
statistical computing. Viena, Austria: R Foundation for
Statistical Computing. https://www.R-project.org/.

Reber, PAG. J., Gitelman, D. r., Parrish, t. B., & Mesulam, METRO. METRO.
(2005). Priming effects in the fusiform gyrus: Changes in
neural activity beyond the second presentation. Cerebral
Corteza, 15, 787–795. https://doi.org/10.1093/cercor/bhh179,
PubMed: 15371295

Reichardt, r., Polner, B., & Simor, PAG. (2020). Novelty

manipulations, memory performance, and predictive coding:
The role of unexpectedness. Fronteras en lo humano
Neurociencia, 14, 152. https://doi.org/10.3389/fnhum.2020
.00152, PubMed: 32410975

Roediger, h. l., & mayordomo, A. C. (2011). The critical role of

retrieval practice in long-term retention. Tendencias en Cognitivo
Ciencias, 15, 20–27. https://doi.org/10.1016/j.tics.2010.09.003,
PubMed: 20951630

Rowland, C. A. (2014). The effect of testing versus restudy on

retention: A meta-analytic review of the testing effect.
Boletín Psicológico, 140, 1432–1463. https://doi.org/10
.1037/a0037559, PubMed: 25150680

Mruczek, R. mi. B., von Loga, I. S., & Kastner, S. (2013). El

Rugg, METRO. D., & Vilberg, k. l. (2013). Brain networks underlying

representation of tool and non-tool object information in the
human intraparietal sulcus. Revista de neurofisiología, 109,
2883–2896. https://doi.org/10.1152/jn.00658.2012, PubMed:
23536716

Nader, k. (2015). Reconsolidation and the dynamic nature of
memory. Cold Spring Harbor Perspectives in Biology, 7,
a021782. https://doi.org/10.1101/cshperspect.a021782,
PubMed: 26354895

Nader, K., & Einarsson, mi. Ö. (2010). Memory reconsolidation:
An update. Annals of the New York Academy of Sciences,
1191, 27–41. https://doi.org/10.1111/j.1749-6632.2010
.05443.X, PubMed: 20392274

Nichols, T., Brett, METRO., andersson, J., Apostar, T., & Poline, j. B.
(2005). Valid conjunction inference with the minimum
statistic. Neuroimagen, 25, 653–660. https://doi.org/10.1016/j
.neuroimage.2004.12.005, PubMed: 15808966

episodic memory retrieval. Opinión actual en
Neurobiología, 23, 255–260. https://doi.org/10.1016/j.conb
.2012.11.005, PubMed: 23206590

Schiffer, A. METRO., Ahlheim, C., Ulrichs, K., & Schubotz, R. I. (2013).
Neural changes when actions change: Adaptation of strong and
weak expectations. Mapeo del cerebro humano, 34, 1713–1727.
https://doi.org/10.1002/hbm.22023, PubMed: 22422724
Schiffer, A. METRO., Ahlheim, C., Wurm, METRO. F., & Schubotz, R. I.

(2012). Surprised at all the entropy: Hippocampal, caudate
and midbrain contributions to learning from prediction
errores. Más uno, 7, e36445. https://doi.org/10.1371/journal
.pone.0036445, PubMed: 22570715

Schubotz, R. I. (2004). Human premotor cortex: Beyond motor
actuación. MPI Series in Human Cognitive and Brain
Ciencias (volumen. 50). Leipzig: Max Planck Institute for Human
Cognitive and Brain Sciences.

1304

Revista de neurociencia cognitiva

Volumen 34, Número 7

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu

/
j

oh
C
norte
a
r
t
i
C
mi
–
pag
d

F
/

3
4
7
1
2
8
7
2
0
2
8
4
5
7

/
j

oh
C
norte
_
a
_
0
1
8
6
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Schubotz, R. I. (2015). Prediction and expectation. In A. W..
Toga (Ed.), Brain mapping: An encyclopedic reference
(volumen. 3, páginas. 295–302). Prensa académica, Elsevier. https://doi
.org/10.1016/B978-0-12-397025-1.00205-0

Schubotz, R. I., Cesta, F. METRO., Schiffer, A. METRO., Stadler, w., & von
Cramón, D. Y. (2012). The fraction of an action is more than
a movement: Neural signatures of event segmentation in
resonancia magnética funcional. Neuroimagen, 61, 1195–1205. https://doi.org/10.1016/j
.neuroimage.2012.04.008, PubMed: 22521252

Schubotz, R. I., Wurm, METRO. F., Wittmann, METRO. K., & por cramon,
D. Y. (2014). Objects tell us what action we can expect:
Dissociating brain areas for retrieval and exploitation of
action knowledge during action observation in fMRI.
Fronteras en psicología, 5, 636. https://doi.org/10.3389/fpsyg
.2014.00636, PubMed: 25009519

Scully, I. D., Napper, l. MI., & Hupbach, A. (2017). Hace

reactivation trigger episodic memory change? A meta-analysis.
Neurobiology of Learning and Memory, 142, 99–107.
https://doi.org/10.1016/j.nlm.2016.12.012, PubMed: 28025069
Montaña de cedro, PAG. B., Gershman, S. J., Polyn, S. METRO., & Norman, k. A.
(2011). Human memory reconsolidation can be explained
using the temporal context model. Boletín psiconómico
and Review, 18, 455–468. https://doi.org/10.3758/s13423-011
-0086-9, PubMed: 21512839

Shultz, S., Sotavento, S. METRO., Pelphrey, K., & McCarthy, GRAMO. (2011). El

posterior superior temporal sulcus is sensitive to the
outcome of human and non-human goal-directed actions.
Social Cognitive and Affective Neuroscience, 6, 602–611.
https://doi.org/10.1093/scan/nsq087, PubMed: 21097958
Siestrup, S., Jainta, B., Trempler, I., cheng, S., & Schubotz, R.
(in preparation). Solidity meets surprise: How memory
consolidation affects cerebral and behavioral processing of
episodic prediction errors. Department of Psychology,
University of Münster.

Sinclair, A. h., & Barense, METRO. D. (2018). Surprise and

destabilize: Prediction error influences episodic memory
reconsolidation. Learning and Memory, 25, 369–381. https://
doi.org/10.1101/lm.046912.117, PubMed: 30012882

Sinclair, A. h., & Barense, METRO. D. (2019). Prediction error and
memory reactivation: How incomplete reminders drive
reconsolidation. Trends in Neurosciences, 42, 727–739.
https://doi.org/10.1016/j.tins.2019.08.007, PubMed: 31506189

Herrero, t. A., Hasinski, A. MI., & Montaña de cedro, PAG. B. (2013). El

context repetition effect: Predicted events are remembered
mejor, even when they don’t happen. Diario de
Psicología experimental: General, 142, 1298–1308. https://
doi.org/10.1037/a0034067, PubMed: 23957285

Snodgrass, j. GRAMO., & Corwin, j. (1988). Pragmatics of measuring
recognition memory: Applications to dementia and amnesia.
Revista de Psicología Experimental: General, 117, 34–50.
https://doi.org/10.1037/0096-3445.117.1.34, PubMed:
2966230

Stachenfeld, k. l., Botvinick, METRO. METRO., & Gershman, S. j. (2017).
The hippocampus as a predictive map. Neurociencia de la naturaleza,
20, 1643–1653. https://doi.org/10.1038/nn.4650, PubMed:
28967910

Stadler, w., Schubotz, R. I., por cramon, D. y., Saltador, A.,
Graf, METRO., & Príncipe, W.. (2011). Predicting and memorizing
observed action: Differential premotor cortex involvement.
Mapeo del cerebro humano, 32, 677–687. https://doi.org/10.1002
/hbm.20949, PubMed: 20225220

Tamber-Rosenau, B. J., Esterman, METRO., Chiu, Y. C., & Yantis, S.
(2011). Cortical mechanisms of cognitive control for shifting
attention in vision and working memory. Diario de
Neurociencia Cognitiva, 23, 2905–2919. https://doi.org/10
.1162/jocn.2011.21608, PubMed: 21291314

Thurman, S. METRO., van Boxtel, j. j. A., Monti, METRO. METRO., Chiang, j. NORTE.,
& Lu, h. (2016). Neural adaptation in pSTS correlates with
perceptual aftereffects to biological motion and with autistic
traits. Neuroimagen, 136, 149–161. https://doi.org/10.1016/j
.neuroimage.2016.05.015, PubMed: 27164327

Tulving, mi. (2002). Episodic memory: From mind to brain.

Annual Review of Psychology, 53, 1–25. https://doi.org/10
.1146/annurev.psych.53.100901.135114, PubMed: 11752477

Uncapher, METRO. r., & Wagner, A. D. (2009). Posterior parietal

cortex and episodic encoding: Insights from fMRI subsequent
memory effects and dual-attention theory. Neurobiology of
Learning and Memory, 91, 139–154. https://doi.org/10.1016/j
.nlm.2008.10.011, PubMed: 19028591

Wahlheim, C. NORTE., & zacks, j. METRO. (2019). Memory guides the
processing of event changes for older and younger adults.
Revista de Psicología Experimental: General, 148, 30–50.
https://doi.org/10.1037/xge0000458, PubMed: 29985021

Wiggett, A. J., & Downing, PAG. mi. (2011). Representation of action

in occipito-temporal cortex. Revista de Cognitivo
Neurociencia, 23, 1765–1780. https://doi.org/10.1162/jocn
.2010.21552, PubMed: 20807060

Wiggs, C. l., Weisberg, J., & Martín, A. (1999). Neural correlates

of semantic and episodic memory retrieval.
Neuropsicología, 37, 103–118. https://doi.org/10.1016
/s0028-3932(98)00044-X, PubMed: 9920476

Wobbrock, j. o., Findlater, l., Gergle, D., & Higgins, j. j.
(2011). The aligned rank transform for nonparametric
factorial analyses using only ANOVA procedures. In CHI ’11:
CHI Conference on Human Factors in Computing
Sistemas (páginas. 143–146). https://doi.org/10.1145/1978942
.1978963

Worsley, k. J., & Friston, k. j. (1995). Analysis of fMRI

time-series revisited—Again. Neuroimagen, 2, 173–181.
https://doi.org/10.1006/nimg.1995.1023, PubMed: 9343600
Wurm, METRO. F., & Schubotz, R. I. (2012). Squeezing lemons in
the bathroom: Contextual information modulates action
recognition. Neuroimagen, 59, 1551–1559. https://doi.org/10
.1016/j.neuroimage.2011.08.038, PubMed: 21878395