The Feedback-related Negativity Reflects the Combination - Ricerca sull'intelligenza artificiale specializzata al MIT

The Feedback-related Negativity Reflects the Combination
of Instantaneous and Long-term Values of
Decision Outcomes

Roman Osinsky1, Natalie Ulrich1, Patrick Mussel2, Lena Feser2,
Aruni Gunawardena2, and Johannes Hewig2

D
o
w
N
l
o
UN
D
e
D

F
R
o
M

Astratto

■ Hundreds of ERP studies have reported a midfrontal negative-
going amplitude shift following negative compared with positive
action outcomes. This feedback-related negativity (FRN) effect is
typically thought to reflect an early and binary mechanism of
action evaluation in the posterior midcingulate cortex. Tuttavia,
in prior research on the FRN effect, the instantaneous value and
the long-term value of action outcomes have been perfectly
confounded. Questo è, instantaneously positive outcomes were
generally consistent with task goals, whereas instantaneously
negative outcomes were inconsistent with task goals. In
this study, we disentangled these two outcome aspects in two

esperimenti. Our results reveal an interaction of instantaneous
and long-term outcome values. More precisely, our findings
strongly suggest that the FRN effect is mainly driven by a reward
positivity, which is evoked only by outcomes that possess an
instantaneously positive value and also help the organism to
reach its long-term goals. These findings add to a recent literature
according to which the posterior midcingulate cortex acts as a
hierarchical reinforcement learning system and suggest that this
system integrates instant and long-term action–outcome values.
Questo, in turn, might be crucial for learning optimal behavioral
strategies in a given setting. ■

INTRODUCTION

For a considerable time, cognitive psychologists and
neuroscientists have been interested in the neuronal
foundations of performance monitoring, questo è, IL
brain’s ability to quickly and efficiently evaluate action
and decision outcomes. Although research on this topic
has already yielded remarkable insights (Ullsperger,
Danielmeier, & Jocham, 2014), many open questions
remain. For instance, hundreds of studies have shown that,
about 200–350 msec after the onset of a negative com-
pared with positive action feedback, a negative-going
amplitude shift occurs at frontomedial sites in the scalp-
recorded EEG. This difference between positive and
negative action feedback is commonly referred to as
feedback-related negativity (FRN) effect (for recent over-
views, see Sambrook & Goslin, 2015; Walsh & Anderson,
2012). Tuttavia, what exactly defines whether a particular
action outcome is positive or negative? Imagine, for
esempio, you are working on an important grant proposal
the deadline of which will end in a few hours. Although
this work is still far from being ready for submission,
you decide to take a break and watch some funny Internet
videos. Although the outcome of this decision may be
positive on the short run, it may be rather negative with

1University of Osnabrück, 2University of Würzburg

regard to your long-term goals. To date, it is unclear
whether the FRN effect reflects the instantaneous value
of action and decision outcomes or its long-term value.
As we will outline below, an answer to this question would
be highly relevant to further our understanding of the
functional significance of the FRN effect and its neural
generator. In the present work, we therefore addressed
this issue in two experiments.

The FRN effect has first been described by Miltner,
Braun, and Coles (1997), and since then an enormous
amount of research has been conducted to uncover the
neurocognitive functions underlying it. Today, there is
more or less consensus that the FRN effect reflects an
early binary distinction between positive and negative
outcomes (per esempio., Hajcak, Moser, Holroyd, & Simons,
2006; Holroyd, Hajcak, & Larsen, 2006; Yeung & Sanfey,
2004; Gehring & Willoughby, 2002) and that it is gener-
ated in the posterior midcingulate cortex (pMCC; per esempio.,
Hauser et al., 2014; Gruendler, Ullsperger, & Huster, 2011;
Bellebaum & Daum, 2008; Hewig et al., 2007; Gehring &
Willoughby, 2002; Miltner et al., 1997; also see, Warren,
Hyman, Seamans, & Holroyd, 2015; Emeric et al., 2008).
According to initial accounts of the FRN effect, it mainly
arises from neural responses to negative action out-
comes (cioè., a punishment or non-reward negativity; per esempio.,
Gehring & Willoughby, 2002; Holroyd & Coles, 2002;
Miltner et al., 1997). Tuttavia, more recent research has

Journal of Cognitive Neuroscience 29:3, pag. 424–434
doi:10.1162/jocn_a_01055

/
j

T
T

F
/

io
T
.

:
/
/

H
T
T
P
:
/
D
/
o
M
w
io
N
T
o
P
UN
R
D
C
e
.
D
S
F
io
R
o
l
M
v
e
H
R
C
P
H
UN
D
io
io
R
R
e
.
C
C
T
.
o
M
M
/
j
e
D
o
tu
C
N
o
/
C
UN
N
R
UN
T
R
io
T
io
C
C
l
e
e
–
P
–
D
P
D
2
F
9
/
3
2
9
4
/
2
3
4
/
1
4
9
2
5
4
2
/
2
1
9
7
7
8
o
5
C
9
N
4
_
9
UN
/
_
j
0
o
1
C
0
N
5
5
_
UN
P
_
D
0
1
B
0
sì
5
G
5
tu
.
e
P
S
T
D
o
F
N
B
0
sì
8
S
M
e
IO
P
T
e
M
l
io
B
B
e
R
R
UN
2
R
0
io
2
3
e
S

/
j

tu
S
e
R

o
N

1
7

M
UN
sì

2
0
2
1

indicated that the FRN effect may rather be driven by a
reward positivity (Rew-P), which attenuates a default
frontomedial N2 component and which is present for
positive but not for negative outcomes (Holroyd, Pakzad-
Vaezi, & Krigolson, 2008; also see, Frömer, Stürmer, &
Sommer, 2016; Gibbons, Schnuerch, & Stahl, 2016;
Sambrook & Goslin, 2016; Proudfit, 2015; Varona-Moya,
Moris, & Luque, 2015; Becker, Nitsch, Miltner, & Straube,
2014; Kujawa, Smith, Luhmann, & Hajcak, 2013; Kreussel
et al., 2012; Luque, López, Marco-Pallares, Càmara, &
Rodriguez-Fornells, 2012; Warren & Holroyd, 2012; Foti,
Weinberg, Dien, & Hajcak, 2011; Holroyd, Krigolson, & Lee,
2011; Hewig et al., 2007, 2010; Eppinger, Kray, Mock, &
Mecklinger, 2008; Holroyd et al., 2008; Cohen, Elger, &
Ranganath, 2007; Potts, Martin, Burton, & Montague, 2006).
With regard to the detailed functional meaning of the
FRN effect, it has been assumed that it reflects the sig-
naling of reward prediction errors in the pMCC (cioè., an
outcome is worse than expected vs. better than ex-
pected), which may use these signals to optimize task
behavior (per esempio., Osinsky, Seeger, Mussel, & Hewig, 2016;
Sambrook & Goslin, 2014; Nieuwenhuis, Holroyd, Mol, &
Coles, 2004; Holroyd & Coles, 2002). If this function of
the pMCC is located on a low hierarchical level of behav-
ioral organization and control, the FRN effect should be
sensitive only to the isolated instantaneous value of a
single simple action (cioè., whether the instant action out-
come is positive vs. negative, irrespective of its long-term
value). Such view would be consistent, for instance, con
findings showing that the FRN effect is not sensitive to
task correctness of an outcome in terms of counterfactual
comparisons (cioè., an obtained small monetary gain might
have a positive instantaneous value but would be in-
correct with regard to the long-term task if the unchosen
option would have resulted in a larger gain; per esempio., Osinsky,
Walter, & Hewig, 2014; Kujawa et al., 2013; Yeung &
Sanfey, 2004; Gehring & Willoughby, 2002; but also see
Nieuwenhuis, Yeung, Holroyd, Schurger, & Cohen, 2004).
Tuttavia, recent empirical and theoretical work suggests
that the pMCC may act as a system of hierarchical re-
inforcement learning and therefore rather plays an impor-
tant role in the selection and maintenance of hierarchically
more complex behavior (per esempio., a sequence of several simple
actions to complete a complex task; Holroyd & McClure,
2015; Holroyd & Yeung, 2012; Ribas-Fernandes et al.,
2011). For instance, Holroyd and Yeung (2012) presented
a model according to which the pMCC selects between
extended sequences of behavioral acts to reach a particu-
lar goal (per esempio., preparing a meal by your own vs. ordering it
via telephone) and continuously receives information
about task progress from a critic module consisting of
the OFC and the ventral striatum. In case of an event that
threatens task completion, the pMCC initiates adaptations
in control to maintain the system on-task. If, Tuttavia, an
event brings the individual closer to reaching its long-term
goal, the pMCC uses this information to learn about opti-
mal behavioral strategies in a given context. Briefly, IL

pMCC “is more concerned with the selection and main-
tenance of the task itself than with the minutiae of task
execution” (Holroyd & Yeung, 2012, P. 123). Given that
the FRN effect reflects the incoming and/or utilization
of such task-related outcome information in the pMCC
(cf. Ribas-Fernandes et al., 2011) it should be more reflec-
tive of the long-term task value of an action outcome
rather than the instantaneous, task-unrelated value.

To clarify whether the FRN effect is more reflective of
instantaneous or long-term values, it would be necessary
to disentangle these two outcome aspects in a single task
progetto. In prior research on the FRN effect, researchers
have frequently used simple gambling or guessing tasks,
during each trial of which participants can decide be-
tween two or more alternatives, leading either to a mon-
etary favorable or unfavorable outcome (per esempio., wins and
losses; per esempio., Mushtaq, Wilkie, Mon-Williams, & Schaefer,
2016; Mussel, Reiter, Osinsky, & Hewig, 2015; Liu, Nelson,
Bernat, & Gehring, 2014; Cui, Chen, Wang, Shum, &
Chan, 2013; Kreussel et al., 2012; Osinsky, Mussel, &
Hewig, 2012; Carlson, Foti, Mujica-Parodi, Harmon-Jones,
& Hajcak, 2011; Foti et al., 2011; Walsh & Anderson, 2011;
Moser & Simons, 2009; Hajcak et al., 2006; Hajcak,
Holroyd, Moser, & Simons, 2005; Holroyd, Larsen, &
Cohen, 2004). With no other explicit instruction, the in-
stantaneous value (loss = negative; win = positive) of an
outcome in such tasks directly corresponds with its long-
term value as the individual will usually intend to make
as much money as possible across the task. Così, in most
prior research on the FRN effect, the instantaneous value
and the long-term value of an outcome have been perfectly
confounded. In the two experiments presented here, we
aimed to distinguish between these two outcome aspects
by using special task instructions. In both experiments,
participants performed a simple guessing task, consisting
of two crucial task blocks which differed by instruction.
In Experiment 1, the individual played a game in which
she/he navigated through a magic maze (Guarda la figura 1).
In each trial, she/he was asked to choose one of three
doors, directly leading to a passageway with a turnstile or
an impasse containing a treasure chest. Importantly, IL
passageway was associated with a monetary loss and,
Perciò, a negative instantaneous value, whereas the
impasse was associate with a monetary win and, hence, UN
positive instantaneous value. As more extended task con-
testo, in one block participants were instructed to collect
as much money as possible (cioè., the standard block),
whereas they should escape the maze in the other block
(cioè., the reversed block). Therefore, instantaneous and
long-term values converged in the standard block (cioè.,
positive instant value/task-supportive long-term value;
negative instant value/task-unsupportive long-term value)
and diverged in the reversed block (cioè., positive instant
value/task-unsupportive long-term value; negative instant
value/task-supportive long-term value). By comparing the
four possible combinations of long-term and instantaneous
value, we were able to test whether the FRN effect reflects

Osinsky et al.

425

D
o
w
N
l
o
UN
D
e
D

F
R
o
M

/
j

F
/

T
T

io
T
.

:
/
/

/
j

tu
S
e
R

o
N

1
7

M
UN
sì

2
0
2
1

Figura 1. Schematic depiction
of trial events in the behavioral
tasks used in Experiment 1.
In Experiment 2, pictures of
spiders and bunnies were
applied as trial outcomes.

D
o
w
N
l
o
UN
D
e
D

F
R
o
M

/
j

F
/

T
T

io
T
.

:
/
/

/
j

tu
S
e
R

o
N

1
7

M
UN
sì

2
0
2
1

only a single outcome aspect (cioè., instantaneous outcome
value or long-term value), the additive combination of
both, or more complex interactions. Whereas we used
abstract visual stimuli to signal wins and losses in Experi-
ment 1, we applied natural pictures of bunnies and spiders
in Experiment 2. Such pictures probably possess a more
inherent instantaneous value in terms of emotional valence
than abstract symbols. Inoltre, in Experiment 2, we
added a third block type, in which no explicit long-term
goal was formulated (we call this the task neutral block).
By comparing the results of the two experiments, we are
able to draw some conclusions with regard to the general-
izability of any potential effects of instantaneous and long-
term outcome value on the FRN.

EXPERIMENT 1

Methods

Participants

Thirty-five individuals from the population of psychology
students in Würzburg, Germany, initially responded to an
announcement of the study. As six of these individuals
did not show up at the experimental session, the final
sample consisted of 29 participants (5 men, 24 women,
mean age = 21.3 years, age range = 18–29 years). Tutto
had normal or corrected-to-normal vision and reported
to be free of any mental or neurological disorder. For
participation, they received course credit as well as a
fixed monetary compensation of A6.10 (see below).
Participants gave written informed consent. The study
was approved by the local ethics committee and was in
accordance with the declaration of Helsinki. After task
completion, participants received a full debriefing with
regard to the pseudorandom trial outcomes (see below).

Guessing Task and Outcome Ratings

Participants completed a guessing task in which they
should imagine that they navigate through a magic virtual

maze (Guarda la figura 1). They were told that this maze con-
sists of numerous rooms, in each of which there are
three doors. They could choose one of the doors by
button press. Each of the doors could lead either to a
passageway to the next room or to an impasse. Inoltre,
participants were informed that in each passageway
there is a turnstile where they have to pay 5 centesimi da
their game account to pass through. In contrasto, in each
impasse there was a treasure chest containing 10 cents,
which would be added to their account.1 Hence, pas-
sageways led to an instantaneous monetary loss whereas
impasses led to an instantaneous monetary win. Partici-
pants were also told that, after entering an impasse, Essi
would return to the last room, but because the maze is
magic, the doors would have shuffled. Inoltre, Essi
were informed that they only have a limited number of
moves before the game ends. Finally and most impor-
tantly, in the standard block participants were instructed
to find as many treasure chests as possible. In contrasto, In
the reversed block they should try to find as many pas-
sageways as possible to escape the maze before the
game ends. Così, the long-term value of the outcomes
differed between the two blocks, whereas the instanta-
neous value was always identical.

Each block consisted of 120 critical trials with a pseudo-
random outcome order and an equal number of im-
passes (60) and passageways (60). Inoltre, at the
end of each block a final trial was presented with an out-
come consistent with the overall task-block goal (cioè., an
impasse in the standard block and an exit sign in the re-
versed block). Therefore, each participant gained a total
monetary outcome of 6.10 Euros during the whole task.
Block order was counterbalanced across participants.
Each trial started with a blank screen being presented
for 250 msec. Afterwards, the three doors were shown
until the participant chose one of them by pressing a re-
spective button. A central fixation cross was then pre-
sented for 1000 msec before the outcome stimulus (cioè., an
impasse with a treasure chest or a passageway with a turn-
stile) was shown for 1.500 msec. The task was presented
on a 21-in. monitor and was controlled by Presentation

426

Journal of Cognitive Neuroscience

Volume 29, Numero 3

experimental software (Neurobehavioral Systems, Inc.,
Albany, CA). Responses were given on customary key-
board, using the hlefti, hupi, and hrighti buttons. All stimuli
were presented on a black background.

Following each block, participants rated the two out-
comes on the 5-point valence scale of the Self-Assessment
Manikin (Bradley & Lang, 1994; 1 = positive, 3 = neutral,
5 = negative). Inoltre, they should indicate on a
5-point scale what outcome they generally intended to find
(1 = passageways, 5 = impasses).

EEG Recordings and Analyses

EEG was recorded with a sampling rate of 250 Hz at
31 scalp positions (Fp1, Fp2, F9, F7, F3, Fz, F4, F8,
F10, FC5, FC1, FCz, FC2, FC6, T7, C3, C4, T8, TP9,
CP1, CP2, TP10, P7, P3, Pz, P4, P8, PO9, O1, O2, E
PO10), using a BrainAmp System (Brain Products GmbH,
Gilching, Germany), Fast’nEasy electrode caps (Easy Cap
GmbH, Herrsching, Germany), and Brainvision Recorder
software (Brain Products GmbH). Recording sites were
prepared so that impedances were below 10 kΩ. IL
online reference was mounted at position Cz, and the
ground electrode was placed between positions Fpz
and Fz. During recording, the signal was band-passed fil-
tered (0.016–80 Hz).

Offline data processing was conducted using Brain-
vision Analyzer software. Primo, data were re-referenced
to the mean of electrodes TP9 and TP10 (mastoids),
and former reference at Cz was reinstated as a new data
channel. Afterwards, data were further filtered using a 30-Hz
(3 dB point) low-pass filter (Butterworth Zero Phase Filter)
with a 48-dB/octave roll-off function. An independent
component analysis (extended infomax algorithm)-based
correction method was then applied to correct for ocular
artifacts. Resulting data were segmented around the
outcome onset (−200 to 800 msec). Segments with
remaining artifacts were rejected when containing voltage
steps of 20 μV/msec or more and if the max–min difference
within the segment was equal to or larger than 150 μV.
Afterwards, segments were averaged per condition and
baseline-corrected, using the −200 to 0 msec time
finestra. At least 20 segments were available per person
and condition for averaging.

Quantification of Variables and Statistical Analysis

Postblock picture ratings were clearly not normally dis-
tributed and therefore analyzed using the nonparametric
Wilcoxon test. For the valence ratings, we isolated the
main effect of instantaneous value by aggregating valence
ratings for each outcome (impasses/wins, passageways/
losses) across both blocks. Conversely, to isolate the
main effect of long-term value, we separately aggregated
valence ratings for task-supportive and task-unsupportive
outcomes across impasses and passageways. Finalmente, A
analyze the interaction between instantaneous and

long-term value, we calculated the difference between
task-unsupportive and task-supportive outcomes sepa-
rately for passageways and impasses and also compared
these difference scores.

To quantify outcome-driven adjustments in behavior,
we calculated rates of postoutcome switches in choice
from trial n to trial n + 1. These rates could vary between
0 (no switching at all ) E 1 (switching in choice
behavior following each trial ) and were analyzed using
UN 2 × 2 repeated-measures ANOVA with the factors
Instantaneous Value (impasse/win, passageway/loss) E
Long-term Value (task-supportive, task-unsupportive).

The FRN was quantified as mean amplitude between
220 E 300 msec at Fz and FCz. Mean FRN amplitudes
were analyzed using a 2 × 2 × 2 repeated-measures
ANOVA with the factors Electrode (Fz, FCz), Instantaneous
Value (impasse/win, passageway/loss), and Long-term
Value (task-supportive, task-unsupportive). Critical alpha
level was set at .05. For significant effects, partial eta
squared (ηp

2) values are reported.

Results

Explicit Outcome Ratings

As mentioned above, explicit ratings were analyzed using
nonparametric Wilcoxon tests, and therefore, z statistics
are reported in this section. With regard to the valence
ratings, passageways/losses were generally rated more
negative than impasses/wins (z = −3.04, p = .002; Vedere
Figura 2). Even more clearly, task-supportive outcomes
were rated more positive than task-unsupportive out-
comes (z = −4.65, P < .001). Moreover, we detected an interaction effect when comparing passageways and impasses with regard to the difference between task- unsupportive and task-supportive outcomes (z = −2.08, p = .037). Thus, the effect of long-term value was slightly more pronounced for passageways/losses (mean differ- ence between negative and positive long-term value = 2.57) compared with impasses/wins (mean difference between negative and positive long-term value = 2.09). In addition, participants reported that they intended to find the impasses in the standard block and the passage- ways in the reversed block (z = 4.79, p < .001; not shown in Figure 2; standard block: M = 4.76, SE = 0.08; reversed block: M = 1.34, SE = 0.16). Switching Rates Mean postoutcome switching rates are shown in Figure 2. The main effect of Long-term Value was significant (F(1, 28) = 26.78, p < .001, ηp 2 = .49), reflecting higher switch- ing rates after task-unsupportive compared with support- ive outcomes. The main effect of Instantaneous Value (F(1, 28) = 1.45, p = .24) and the interaction of both factors (F(1, 28) = 0.43, p = .84) were not significant. Osinsky et al. 427 D o w n l o a d e d f r o m l l / / / / j t t f / i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 9 / 3 2 9 4 / 2 3 4 / 1 4 9 2 5 4 2 / 2 1 9 7 7 8 o 5 c 9 n 4 _ 9 a / _ j 0 o 1 c 0 n 5 5 _ a p _ d 0 1 b 0 y 5 g 5 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 i 2 3 e s / j f / . t u s e r o n 1 7 M a y 2 0 2 1 FRN Amplitude Mean FRN values across electrode Fz and FCz are shown in Figure 2. Outcome-locked ERP waveforms at Fz are presented in Figure 3. The main effect of Electrode was significant, reflecting generally more negative FRN ampli- tudes at Fz compared with FCz (F(1, 28) = 18.86, p < .001, ηp 2 = .40). The main effect of Instantaneous Out- come was not significant (F(1, 28) = 2.12, p = .16). How- ever, we observed a significant main effect of Long-term Value (F(1, 28) = 5.74, p = .023, ηp 2 = .17), which was qualified by a two-way interaction of Long-term Value × Instantaneous Value (F(1, 28) = 4.73, p = .038, ηp 2 = .15). Analyses of simple effects revealed that there was a sig- nificant effect of Long-term Value for the impasses/wins (F(1, 28) = 8.57, p = .007, ηp 2 = .23), reflecting more positive FRN amplitudes when the impasses/wins sup- ported the task goal than when being opposed to the task goal. For the passageways/ losses, the effect of Long-term Value was not significant (F(1, 28) = 0.76, p = .39). Finally, a significant three-way interaction of Electrode × Long-term Value × Instantaneous Value indi- cated that this pattern was more pronounced at electrode Fz (F(1, 28) = 9.16, p = .005, ηp 2 = .25). To analyze whether the FRN is linked to behavioral ad- justments, we calculated simple correlations for each out- come condition. All these correlations were small (range: −.20 to .29) and did not significantly deviate from zero (all ps > .12). The same applied when calculating cor-
relations between FRN and switching difference scores
(task-unsupportive minus task-supportive) for impasses/
wins (r = −.20, p = .30) and passageways/losses (r =
.17, p = .39).

Discussion Experiment 1

The explicit ratings of the outcomes following each block
indicate that participants generally complied with the
block instructions. Our electrophysiological analyses
clearly show that amplitude in the FRN time range does
neither reflect the isolated instantaneous value of an ac-
tion outcome nor its isolated long-term value alone. Piuttosto,
we observed an interaction of the two outcome aspects,
which was driven by a modulatory effect of long-term value
on the FRN response to instantaneously positive but not
negative outcomes. In particular, amplitude in the FRN
time range appears to be more positive when an instan-
taneously positive outcome fits the long-term task goals.
Critically, it could be argued that the abstract visual out-
comes used in Experiment 1 do not possess an inherent
instantaneous value. Così, it might be possible that partic-
ipants volitionally reformulated the instantaneous values
of the outcomes in a block-specific fashion. This would
also be consistent with the strong main effect of long-term
value on explicit valence ratings. We therefore designed a
second experiment, using visual stimuli that are more likely
to possess an inherent value in terms of emotional valence,
questo è, pictures of spiders versus pictures of bunnies.

Figura 2. Means and SEM (error bars) for the valence ratings,
postoutcome switching rates, and FRN amplitudes in Experiments 1 E
2. Note that valence ratings (top) refer to a 5-point scale, ranging from
1 (very positive) A 5 (very negative). Switching rates (middle) can
vary between 0 (no switching) E 1 (choice switching after every
choice that led to the respective outcome).

EXPERIMENT 2

Methods

Participants

The sample comprised 35 women (mean age = 23.8 years,
age range = 19–32 years), some of whom were Psychology

428

Journal of Cognitive Neuroscience

Volume 29, Numero 3

D
o
w
N
l
o
UN
D
e
D

F
R
o
M

/
j

T
T

F
/

io
T
.

:
/
/

/
j

tu
S
e
R

o
N

1
7

M
UN
sì

2
0
2
1

D
o
w
N
l
o
UN
D
e
D

F
R
o
M

/
j

F
/

T
T

io
T
.

:
/
/

/
j

tu
S
e
R

o
N

1
7

M
UN
sì

2
0
2
1

Figura 3. The left and middle displays show grand-averaged waveforms locked to outcome onset at electrode Fz in Experiment 1. The rectangle
marks the time window used for FRN analyses. The right display shows the voltage distribution of the difference between task-supportive wins
and task-unsupportive wins in the FRN time range.

students. All were recruited from the Würzburg com-
munity population, and none had participated in Exper-
iment 1. All had a normal or corrected-to-normal vision and
reported to be free of any mental or neurological disorder.
Participants received either course credit for Psychology
students or a monetary compensation of 15 Euros. Tutto
gave written informed consent. The study was approved
by the local ethics committee and was in accordance with
the declaration of Helsinki. After task completion, partici-
pants received a full debriefing with regard to the pseudo-
random trial outcomes (see below).

Guessing Task and Outcome Ratings

The general task design in Experiment 2 was very similar
to the one used in Experiment 1. Participants were asked
to imagine that they are walking through a new kind of
zoo (cioè., the maze zoo) in which only spiders and bunnies
live. Inoltre, they were told that this zoo consists of
numerous rooms, each of which houses a single animal
(cioè., either a spider or a bunny). In each trial, they could
choose between two doors by button press. Afterwards,
they would enter the chosen room and see the animal
that is living there (cioè., the choice outcome). Al
end of the room they will find two other doors leading
to the next rooms and so on. They were also informed
that after a certain number of rooms they would reach
the exit of the zoo, Quale, Tuttavia, does not mean that
they have seen all of the rooms/animals.

On the basis of prior research, we assumed that spiders
compared with bunnies should possess a more negative
inherent emotional valence (Öhman & Mineka, 2001).
Accordingly, the instantaneous outcome value should be
more positive for bunnies compared with spiders. A
manipulate the long-term value, we used different instruc-
tions in the three task blocks. In the standard block, the par-
ticipant should imagine that she is working in the zoo and

that the zoo director has asked her to find as many bunnies
as possible in one walk-through. In contrasto, participants
were asked to find as many spiders as possible in the re-
versed block. Finalmente, in the neutral block they had no spe-
cial instruction to find a specific animal but should imagine
that they are just going through the zoo for a private walk.
The block order was counterbalanced across participants.
Each block consisted of 80 trials with a pseudorandom
outcome order and an equal number of bunnies (40)
and spiders (40). Ten spider pictures and 10 bunny pic-
tures were used, with each picture being presented four
times per block. Each trial started with a blank screen be-
ing presented for 500 msec. Afterwards, the two doors
were shown until the participant chose one of them by
pressing a respective button. A central fixation cross was
then displayed for 750 msec before the animal picture was
shown for 1500 msec. The same hardware and software
components were used as in Experiment 1. All stimuli were
presented on a gray background.

Following each block, we asked participants to indicate
emotional valence for the two animal categories on the
respective scales of the Self-Assessment Manikin (Bradley
& Lang, 1994; 1 = positive/not all arousing, 3 = neutral/
somewhat arousing, 5 = negative/very arousing). More-
Sopra, participants should indicate whether they intended
to find the spiders (5) or the bunnies (1) on a 5-point scale.

EEG Recordings and Analyses

The same setups, procedures, and approaches for FRN
quantification were used as described for Experiment 1.

Quantification of Variables and Statistical Analysis

Again, explicit ratings were analyzed using nonparametric
tests. For valence ratings, we isolated the main effect of Instan-
taneous Value by aggregating across the three task blocks

Osinsky et al.

429

separately for spiders and bunnies. The main effect of Long-
term Value was analyzed by entering the aggregated valence
ratings for task-supportive outcomes, task-unsupportive out-
comes, and task-neutral outcomes into a Friedman test.
Any potential interaction between both factors was ana-
lyzed by pairwise comparisons using Wilcoxon tests.

Rates for postoutcome switching in choice behavior
were calculated and entered in a 2 × 3 repeated-measures
ANOVA with the with-subject factors Instantaneous Value
(negative/spiders, positive/bunnies) and Long-term Value
(in support of task goal, opposed to task goal, neutro).

FRN amplitudes were entered into a 2 × 2 × 3 repeated-
measures ANOVA with the with-subject factors Electrode
(Fz, FCz), Instantaneous Value (negative/spiders, positive/
bunnies), and Long-term Value (in support of task goal,
opposed to task goal, neutro). p values were adjusted
using the Greenhouse–Geisser correction when the Mauchly
test indicated a violation of sphericity assumption. In such
cases, uncorrected degrees of freedom and epsilon values
(ε) are reported. All other statistical procedures and tests
were the same as described in Experiment 1.

Results

Explicit Outcome Ratings

Bunnies compared with spiders were generally rated more
positive (z = −4.74, P < .001; see Figure 2). Moreover, the main effect of Long-term Value was significant (χ2 = 8.22, p = .016). Pairwise comparisons by means of Wilcoxon tests showed that task-unsupportive outcomes were rated slightly more negative than task-neutral outcomes (z = −2.01, p = .045), whereas there was no significant differ- ence between task-supportive and task-unsupportive out- comes (z = −1.62, p = .11) and also no difference between task-supportive and task-neutral outcomes (z = −0.03, p = .98). When analyzing the effect of long-term value separately for spiders and bunnies, we did not ob- serve a significant effect for bunnies (χ2 = 0.89, p = .64) but for spiders (χ2 = 9.86, p = .007). Pairwise com- parisons revealed that spiders were rated more negative when being task-unsupportive compared with task- supportive (z = −2.26, p = .024). The other comparisons did not reach statistical significance (all zs > −1.75, Tutto
ps > .08).

Finalmente, the three task blocks significantly differed with
regard to what kind of animal participants intended to
find, as indicated by a Friedman test (χ2 = 52.07, P < .001). Pairwise comparisons by means of Wilcoxon tests revealed significant differences between all three blocks (all zs < −3.58, all ps < .001). Thus, participants re- ported the intention to find the bunnies in the standard block, the spiders in the reversed block, and a tendency toward the bunnies in the neutral block (not shown in Figure 2; standard block: M = 4.83, SE = 0.51; reversed block: M = 1.91, SE = 1.42; neutral block: M = 1.94, SE = 1.00). Switching Rates Mean postoutcome switching rates are shown in Figure 2. Although participants tended to more often switch in choice behavior following spiders compared with bunnies, this difference failed to reach statistical significance (F(1, 34) = 3.76, p = .061). The main effect of Long-term Value was significant (F(2, 68) = 14.62, p < .001, ε = .98, ηp 2 = .30). Pairwise comparisons revealed that switching rates were much lower following task-supportive outcomes compared with task-unsupportive (t(34) = 4.68, p < .001) and task-neutral outcomes (t(34) = 4.33, p < .001). The latter two conditions did not differ significantly (t(34) = 0.52, p = .61). Moreover, there was a significant two-way interaction of Instantaneous Value and Long-term Value (F(2, 68) = 4.21, p = .027, ε = .82, ηp 2 = .11). Analyses of simple effects revealed that only for task-neutral out- comes switching rates were significantly higher following spiders (mean = 0.52) compared with bunnies (mean = 0.40, t(34) = 2.52, p = .017). The comparison between spiders and bunnies was not significant for task-supportive (t(34) = 0.10, p = .92) and task-unsupportive outcomes (t(34) = 0.88, p = .39). FRN Amplitude Mean FRN values across electrode Fz and FCz are shown in Figure 2. Outcome-locked ERP waveforms at electrode Fz are presented in Figure 4. FRN amplitude was generally more negative at Fz compared with FCz (F(1, 34) = 4.89, p < .034, ηp 2 = .13). The main effect of Instantaneous Value was not significant (F(1, 34) = 1.65, p = .21). As in Experiment 1, we observed a significant main effect of Long-term Value (F(2, 68) = 8.16, p = .001, ηp 2 = .19), which was further qualified by a significant interaction of Long-term Value × Instantaneous Value (F(2, 68) = 3.69, p = .04, ηp 2 = .10). Analyses of simple effects revealed that for instantaneous positive outcomes (i.e., bunnies) there was a significant effect of Long-term Value (F(2, 68) = 14.78, p < .001, ε = .96, ηp 2 = .30), whereas this effect was not significant for instantaneous negative outcomes (i.e., spiders; F(2, 68) = 0.51, p = .58, ε = .87). Pairwise comparisons showed that FRN amplitude was substantially reduced when bunnies were task-supportive compared with task-unsupportive (t(34) = 4.42, p < .001) and task- neutral (t(34) = 4.58, p < .001). The latter two conditions did not differ significantly (t(34) = 0.14, p = .89). As in Experiment 1, we calculated correlations between FRN and switching rates (absolute scores and difference scores), using data from the standard block and the reversed block. Again, all correlations were small (−.24 to .27) and statistically not significant (all ps > .11).

Discussion Experiment 2

In Experiment 2, we replicated the results of Experiment 1,
showing that amplitude in the FRN time range does neither

430

Journal of Cognitive Neuroscience

Volume 29, Numero 3

D
o
w
N
l
o
UN
D
e
D

F
R
o
M

/
j

T
T

F
/

io
T
.

:
/
/

/
j

tu
S
e
R

o
N

1
7

M
UN
sì

2
0
2
1

D
o
w
N
l
o
UN
D
e
D

F
R
o
M

/
j

T
T

F
/

io
T
.

:
/
/

/
j

tu
S
e
R

o
N

1
7

M
UN
sì

2
0
2
1

Figura 4. The left and middle displays show grand-averaged waveforms locked to outcome onset at electrode Fz for Experiment 2. The rectangles
mark the time windows used for FRN analyses. The right display shows the voltage distribution of the difference between task-supportive
bunnies and task-unsupportive bunnies in the FRN time range.

reflect the isolated instantaneous outcome value nor the
mere long-term outcome value. Piuttosto, the FRN ampli-
tudes were subject to a more complex interaction of both
outcome aspects. As in Experiment 1, we only observed
a modulatory influence of long-term task relevance on
the FRN response to instantaneously positive outcomes
(cioè., bunnies). For the FRN response to negative outcomes,
Tuttavia, long-term goals appear to be more or less irrele-
vant, resulting in virtually identical deflections in the FRN
time range. Importantly and in contrast to Experiment 1,
the results for the explicit ratings did not reveal strong in-
fluences of task instructions on emotional valence of the
outcome stimuli. This generally supports our notion that
the animal pictures used in Experiment 2 possess a more
inherent emotional value than the more abstract symbols
used in Experiment 1. As the FRN results in both experi-
ments were very similar, it is rather unlikely that the results
in the first experiment were driven by a volitional re-
formulation of instantaneous outcome values (per esempio., IL
reformulation of a win as something bad in the reversed
block).

GENERAL DISCUSSION

The obvious similarity of the electrophysiological results
in both experiments indicates that the observed inter-
action effect of instantaneous and long-term values is
not restricted to a particular kind of outcomes. Invece,
in the light of the vastly different outcome stimuli, IL
strong resemblance between the FRN patterns in Exper-
iments 1 E 2 clearly point to a generic principle of FRN
generation and its function in action evaluation. Che cosa,
Poi, is this principle? Overall, the observed pattern is
consistent with the idea that the FRN effect (cioè., the dif-
ference between negative and positive outcomes) risultati
from a Rew-P in response to favorable action outcomes

rather than a negativity in response to unfavorable out-
comes (Holroyd et al., 2008; also see, Frömer et al.,
2016; Gibbons et al., 2016; Sambrook & Goslin, 2016;
Proudfit, 2015; Varona-Moya, Moris, & Luque, 2015; Becker
et al., 2014; Kujawa et al., 2013; Kreussel et al., 2012; Luque
et al., 2012; Foti et al., 2011; Holroyd et al., 2011; Hewig
et al., 2007, 2010; Eppinger et al., 2008; Cohen et al.,
2007; Potts et al., 2006). According to this assumption,
the negative-going component in the FRN time range is
a default N2 response of the brain, which is suppressed
by a Rew-P when the outcome of an action is positive. Im-
portantly, our findings indicate that, for this Rew-P to occur,
an instantaneous positive value of an action outcome alone
is not sufficient. Apparently, such instantaneously positive
outcome must also possess a positive long-term value, Quello
È, it must help the organism to reach more extended task
goals. Conversely, a positive long-term value of an outcome
per se is obviously also not sufficient to evoke the Rew-P
because, for instantaneously negative outcomes, ampli-
tudes in the FRN time window were insensitive to long-
term values in both experiments. Così, the Rew-P seems
to be evoked only by action outcomes that are positive
both in an instant and in a long-term, task-related fashion.
As we have outlined in the Introduction, it has recently
been assumed that the pMCC selects extended behavioral
sequences and maintains the system on-task. Especially
for the latter, it may continuously integrate information
about task progress, which it receives from a critique
module (Holroyd & Yeung, 2012). Inoltre, there is
already first evidence that amplitude differences in the
FRN time range reflect the signaling of such evaluative
information in the pMCC (Ribas-Fernandes et al., 2011).
Our findings add to this literature by specifying the
manner of outcome distinction in this process. In particu-
lar, it seems as if outcomes that possess both an instan-
taneous positive value and a long-term positive (cioè.,

Osinsky et al.

431

task-supportive) value are distinguished from all other
types of action outcomes in a given environment. Given
that the pMCC is also crucially involved in hierarchical
reinforcement learning (cf. Holroyd & McClure, 2015;
Holroyd & Yeung, 2012), such distinction would be highly
functional. In particular, it could foster the learning of an
optimal behavioral strategy, questo è, a sequence of actions
each of which previously had a direct instant positive con-
sequence and also brought the organism closer to accom-
plishing the task as a whole. Inoltre, several prior
studies have suggested that negative-going amplitude
proportions in the FRN time window are related to sub-
sequent adjustments in behavior (Sallet, Camille, & Procyk,
2013; Van der Helden, Boksem, & Blom, 2010; Cohen &
Ranganath, 2007; Hewig et al., 2007; Holroyd & Krigolson,
2007; Yeung & Sanfey, 2004; also see Yasuda, Sato, Miyawaki,
Kumano, & Kuboki, 2004). As the negative-going compo-
nent of the FRN (cioè., the feedback-locked N2) appears to
be the default response to an action outcome (cf. Holroyd
et al., 2008), this could implicate that the pMCC initiates
adjustments in task-oriented behavior in a default manner
until an outcome occurs that has an instantaneous as well
as a long-term positive value. In other words, the pMCC
may maintain the system in an exploration state as long
as behavioral outcomes in a given task setting are sub-
optimal and switches the system to an exploitation state
when an optimal behavioral strategy has been found. At first
glance, the absence of any substantive relation between
amplitudes in the FRN time range and behavioral adap-
tations (cioè., postoutcome switching) in this study appears
to be inconsistent with such functioning of the pMCC. How-
ever, it should also be noted that, in our tasks, all outcomes
were presented in a pseudorandom order and therefore no
learning of an adaptive, optimal strategy was possible. Questo
could have obscured the interrelation between the FRN
effect/Rew-P and postoutcome adjustments in behavior.
Future studies may therefore further investigate the inter-
relation between the FRN effect/Rew-P and the exploration–
exploitation dimension of behavior by considering both
instantaneous and long-term values of action outcomes in
a task where adaptive behavioral strategies can be learned.
As a potential limitation of our study, the manipulation
of long-term compared with instantaneous outcome value
might be considered as rather weak, especially in Experi-
ment 1. In particular, the instantaneous value of an out-
come in our first experiment might have had a higher
personal relevance (cioè., losing or wining some money)
than the more abstract prospect of losing or winning
the game, which had no monetary consequences. Questo
could have reduced the influence of long-term values
especially for the instantaneous negative outcomes. How-
ever, it should also be noted that such differences in per-
sonal relevance between outcome aspects are rather
unlikely for Experiment 2. Inoltre, the explicit out-
come ratings in Experiment 1 clearly indicate that partic-
ipants generally followed the instruction to achieve the
long-term task goal, and this was also the case when

task-supportive outcomes had an instantaneous negative
value. Così, it seems rather unlikely that our findings are
mainly driven by differences in personal relevance be-
tween instantaneous and long-term values. Nevertheless,
in future studies on this issue, efforts should be made to
control for potential effects of personal relevance, for in-
stance by including more rewarding and/or punishing
long-term outcomes (per esempio., winning or losing extra money
when completing or failing the task).

In summary, our findings support the notion that ampli-
tude variance in the FRN time range is mainly driven by a
Rew-P. To the best of our knowledge, this is the first study
to show that this Rew-P is only evoked by outcomes that
possess an instantaneous positive value and also fit with
the individual’s long-term goals. Hence, our results add
to recent literature according to which the pMCC, as a
likely generator of the Rew-P, acts as a hierarchical re-
inforcement learning system, which selects and maintains
extended behavioral sequences to achieve certain goals
(per esempio., Holroyd & McClure, 2015; Holroyd & Yeung, 2012).
In particular, our study suggests that this system may inte-
grate instantaneous and long-term action–outcome values.
Questo, in turn, might be a crucial process in learning the
optimal behavioral strategy in a given environmental set-
ting, questo è, one that leads to instant as well as long-term
ricompense. Future research may further investigate this topic
by considering instantaneous and long-term outcome
values in an orthogonal design that also allows the learning
of optimal and suboptimal behavioral strategies.

Reprint requests should be sent to Roman Osinsky, Institute of
Psychology, University Osnabrück, Seminarstr. 20, 49074
Osnabrück, Germany, or via e-mail: roman.osinsky@uni-
osnabrueck.de.

Note

1. Higher objective gains than losses were used as the subjec-
tive total value of losses is typically larger than the subjective
value of gains (Tversky & Kahneman, 1992).

REFERENCES

Becker, M. P. I., Nitsch, UN. M., Miltner, W. H. R., & Straube, T.
(2014). A single-trial estimation of the feedback-related
negativity and its relation to BOLD responses in a time-
estimation task. Journal of Neuroscience, 34, 3005–3012.

Bellebaum, C., & Daum, IO. (2008). Learning-related

changes in reward expectancy are reflected in the
feedback-related negativity. European Journal of Neuroscience,
27, 1823–1835.

Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: IL
self-assessment manikin and the semantic differential.
Journal of Behavior Therapy and Experimental Psychiatry,
25, 49–59.

Carlson, J. M., Foti, D., Mujica-Parodi, l. R., Harmon-Jones, E.,
& Hajcak, G. (2011). Ventral striatal and medial prefrontal
BOLD activation is correlated with reward-related electrocortical
activity: A combined ERP and fMRI study. Neuroimage, 57,
1608–1616.

432

Journal of Cognitive Neuroscience

Volume 29, Numero 3

D
o
w
N
l
o
UN
D
e
D

F
R
o
M

/
j

T
T

F
/

io
T
.

:
/
/

/
j

tu
S
e
R

o
N

1
7

M
UN
sì

2
0
2
1

Cohen, M. X., Elger, C. E., & Ranganath, C. (2007). Reward

expectation modulates feedback-related negativity and EEG
spectra. Neuroimage, 35, 968–978.

Cohen, M. X., & Ranganath, C. (2007). Reinforcement learning
signals predict future decisions. Journal of Neuroscience,
27, 371–378.

Cui, J., Chen, Y., Wang, Y., Shum, D. H. K., & Chan, R. C. K.

(2013). Neural correlates of uncertain decision making: ERP
evidence from the Iowa Gambling Task. Frontiers in Human
Neuroscience, 7, 776.

Emeric, E. E., Brown, J. W., Leslie, M., Pouget, P., Stuphorn, V.,
& Schall, J. D. (2008). Performance monitoring local field
potentials in the medial frontal cortex of primates: Anterior
cingulate cortex. Journal of Neurophysiology, 99, 759–772.
Eppinger, B., Kray, J., Mock, B., & Mecklinger, UN. (2008). Better

or worse than expected? Aging, apprendimento, and the ERN.
Neuropsychologia, 46, 521–539.

Foti, D., Weinberg, A., Dien, J., & Hajcak, G. (2011). Event-related
potential activity in the basal ganglia differentiates rewards
from nonrewards: Temporospatial principal components
analysis and source localization of the feedback negativity.
Human Brain Mapping, 32, 2207–2216.

Frömer, R., Stürmer, B., & Sommer, W. (2016). The better, IL
bigger: The effect of graded positive performance feedback
on the reward positivity. Biological Psychology, 114, 61–68.
Gehring, W. J., & Willoughby, UN. R. (2002). The medial frontal
cortex and the rapid processing of monetary gains and losses.
Scienza, 295, 2279–2282.

Gibbons, H., Schnuerch, R., & Stahl, J. (2016). From positivity
to negativity bias: Ambiguity affects the neurophysiological
signatures of feedback processing. Journal of Cognitive
Neuroscience, 28, 542–557.

Gruendler, T. O. J., Ullsperger, M., & Huster, R. J. (2011).

Event-related potential correlates of performance-monitoring
in a lateralized time-estimation task. PLoS One, 6, 1.

Hajcak, G., Holroyd, C. B., Moser, J. S., & Simons, R. F. (2005).
Brain potentials associated with expected and unexpected
good and bad outcomes. Psychophysiology, 42, 161–170.
Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2006).
The feedback-related negativity reflects the binary evaluation
of good versus bad outcomes. Biological Psychology,
71, 148–154.

Hauser, T. U., Iannaccone, R., Stämpfli, P., Drechsler, R., Brandeis,
D., Walitza, S., et al. (2014). The feedback-related negativity
(FRN) revisited: New insights into the localization, Senso
and network organization. Neuroimage, 84, 159–168.

Hewig, J., Kretschmer, N., Trippe, R. H., Hecht, H., Coles, M. G.,
Holroyd, C. B., et al. (2010). Hypersensitivity to reward in
problem gamblers. Biological Psychiatry, 67, 781–783.

Hewig, J., Trippe, R., Hecht, H., Coles, M. G. H., Holroyd, C. B.,
& Miltner, W. H. R. (2007). Decision-making in Blackjack: An
electrophysiological analysis. Cerebral Cortex, 17, 865–877.

Holroyd, C. B., & Coles, M. G. (2002). The neural basis of

human error processing reinforncement learning, dopamine,
and the error-related negativity. Psychological Review,
109, 679–709.

Holroyd, C. B., Hajcak, G., & Larsen, J. T. (2006). The good,
the bad and the neutral: Electrophysiological responses
to feedback stimuli. Brain Research, 11, 93–101.

Holroyd, C. B., & Krigolson, O. E. (2007). Reward prediction

error signals associated with a modified time estimation task.
Psychophysiology, 44, 913–917.

Holroyd, C. B., Krigolson, O. E., & Lee, S. (2011). Reward positivity

elicited by predictive cues. NeuroReport, 22, 249–252.

Holroyd, C. B., Larsen, J. T., & Cohen, J. D. (2004). Context

dependence of the event-related brain potential
associated with reward and punishment. Psychophysiology,
41, 245–253.

Holroyd, C. B., & McClure, S. M. (2015). Hierarchical control
over effortful behavior by rodent medial frontal cortex: UN
computational model. Psychological Review, 122, 54.

Holroyd, C. B., Pakzad-Vaezi, K. L., & Krigolson, O. E. (2008).
The feedback correct-related positivity: Sensitivity of the
event-related brain potential to unexpected positive feedback.
Psychophysiology, 45, 688–697.

Holroyd, C. B. & Yeung, N. (2012). Motivation of extended

behavior by anterior cingulate cortex. Trends in Cognitive
Scienze, 16, 122–128.

Kreussel, L., Hewig, J., Kretschmer, N., Hecht, H., Coles, M. G.,
& Miltner, W. H. (2012). The influence of the magnitude,
probability, and valence of potential wins and losses on
the amplitude of the feedback negativity. Psychophysiology,
49, 207–219.

Kujawa, A., Smith, E., Luhmann, C., & Hajcak, G. (2013).
The feedback negativity reflects favorable compared to
nonfavorable outcomes based on global, not local,
alternatives. Psychophysiology, 50, 134–138.

Liu, Y., Nelson, l. D., Bernat, E. M., & Gehring, W. J. (2014).
Perceptual properties of feedback stimuli influence the
feedback-related negativity in the flanker gambling task.
Psychophysiology, 51, 782–788.

Luque, D., López, F. J., Marco-Pallares, J., Càmara, E., &
Rodriguez-Fornells, UN. (2012). Feedback-related brain
potential activity complies with basic assumptions of
associative learning theory. Journal of Cognitive Neuroscience,
24, 794–808.

Miltner, W. H. R., Braun, C. H., & Coles, M. G. H. (1997). Event-
related brain potentials following incorrect feedback in a time-
estimation task: Evidence for a “generic” neural system for error
detection. Journal of Cognitive Neuroscience, 9, 788–798.
Moser, J. S., & Simons, R. F. (2009). The neural consequences
of flip-flopping: The feedback-related negativity and salience
of reward prediction. Psychophysiology, 46, 313–320.

Mushtaq, F., Wilkie, R. M., Mon-Williams, M. A., & Schaefer, UN.
(2016). Randomised prior feedback modulates neural signals
of outcome monitoring. Neuroimage, 125, 868–879.
Mussel, P., Reiter, UN. M. F., Osinsky, R., & Hewig, J. (2015).
State- and trait-greed, its impact on risky decision-making
and underlying neural mechanisms. Social Neuroscience,
10, 126–134.

Nieuwenhuis, S., Holroyd, C. B., Mol, N., & Coles, M. G. (2004).
Reinforcement-related brain potentials from medial frontal
cortex: Origins and functional significance. Neuroscience
and Biobehavioral Reviews, 28, 441–448.

Nieuwenhuis, S., Yeung, N., Holroyd, C. B., Schurger, A., &

Cohen, J. D. (2004). Sensitivity of electrophysiological activity
from medial frontal cortex to utilitarian and performance
feedback. Cerebral Cortex, 14, 741–747.

Öhman, A., & Mineka, S. (2001). Fears, phobia, E

preparedness: Toward an evolved module of fear and fear
apprendimento. Psychological Reviews, 108, 483–522.

Osinsky, R., Mussel, P., & Hewig, J. (2012). Feedback-related
potentials are sensitive to sequential order of decision
outcomes in a gambling task. Psychophysiology, 49, 1579–1589.

Osinsky, R., Seeger, J., Mussel, P., & Hewig, J. (2016).

Face-induced expectancies influence neural mechanisms
of performance monitoring. Cognitive Affective and
Behavioral Neuroscience, 16, 261–275.

Osinsky, R., Walter, H., & Hewig, J. (2014). What is and

what could have been: An ERP study on counterfactual
comparisons. Psychophysiology, 51, 773–781.

Potts, G. F., Martin, l. E., Burton, P., & Montague, P. R.

(2006). When things are better or worse than expected:
The medial frontal cortex and the allocation of processing
resources. Journal of Cognitive Neuroscience, 18,
1112–1119.

Osinsky et al.

433

D
o
w
N
l
o
UN
D
e
D

F
R
o
M

/
j

T
T

F
/

io
T
.

:
/
/

/
j

tu
S
e
R

o
N

1
7

M
UN
sì

2
0
2
1

Proudfit, G. H. (2015). The reward positivity: From basic
research on reward to a biomarker for depression.
Psychophysiology, 52, 449–459.

Ribas-Fernandes, J. J. F., Solway, A., Diuk, C., McGuire, J. T.,
Barto, UN. G., Niv,, Y, & Botvinick, M. M. (2011). Neural
Signature of hierarchical reinforcement learning. Neuron, 71,
370–379.

Sallet, J., Camille, N., & Procyk, E. (2013). Modulation of

feedback-related negativity during trial-and-error exploration
and encoding of behavioral shifts. Frontiers in Neuroscience,
7, 209.

Sambrook, T. D. & Goslin, J. (2014). Mediofrontal event-related
potentials in response to positive, negative and unsigned
predictions errors. Neuropsychologia, 61, 1–10.

Sambrook, T. D., & Goslin, J. (2015). A neural reward prediction
error revealed by a meta-analysis of ERPs using great grand
medie. Psychological Bulletin, 141, 213–235.

Sambrook, T. D., & Goslin, J. (2016). Principal components
analysis of reward prediction errors in a reinforcement
learning task. Neuroimage, 124, 276–286.

Tversky, A., & Kahneman, D. (1992). Advances in prospect

theory: Cumulative representation of uncertainty. Journal of
Risk and Uncertainty, 5, 297–323.

Ullsperger, M., Danielmeier, C., & Jocham, G. (2014).

Neurophysiology of performance monitoring and adaptive
behavior. Physiological Reviews, 94, 35–79.

Van der Helden, J., Boksem, M. A., & Blom, J. H. (2010).

The importance of failure: Feedback-related negativity

predicts motor learning efficiency. Cerebral Cortex, 20,
1596–1603.

Varona-Moya, S., Moris, J., & Luque, D. (2015). Reward

positivity is elicited by monetary reward in the absence of
response choice. NeuroReport, 26, 152–156.

Walsh, M. M., & Anderson, J. R. (2011). Modulation of the

feedback-related negativity by instruction and experience.
Proceedings of the National Academy of Sciences, U.S.A.,
108, 19048–19053.

Walsh, M. M., & Anderson, J. R. (2012). Learning from

experience: Event-related potential correlates of reward
processing, neural adaptation, and behavioral choice.
Neuroscience and Biobehavioral Reviews, 36, 1870–1884.

Warren, C. M., & Holroyd, C. B. (2012). The impact of

deliberative strategy dissociates ERP components related to
conflict processing vs. reinforcement learning. Frontiers in
Neuroscience, 6, 43.

Warren, C. M., Hyman, J. M., Seamans, J. K., & Holroyd, C. B.
(2015). Feedback-related negativity observed in rodent
anterior cingulate cortex. Journal of Physiology-Paris, 109,
87–94.