On the Utility of Positive and Negative Feedback in a - 麻省理工学院人工智能研究专业

On the Utility of Positive and Negative Feedback in a
Paired-associate Learning Task

Yael Arbel1,2, Anthony Murphy2, and Emanuel Donchin2

抽象的

■ This study offers a neurophysiological examination of the
relationship between feedback processing and learning. A
two-choice paired-associate learning task borrowed and modi-
fied from Tricomi and Fiez [Tricomi, E., & Fiez, J. A. Feedback
signals in the caudate reflect goal achievement on a declarative
memory task. Neuroimage, 41, 1154–1167, 2008] was em-
ployed to examine the mediofrontal electrophysiological brain
activity associated with the processing of performance feedback
in a learning task and to elucidate the extent to which the pro-
cessing of the initial informative feedback is related to learning
结果. Twenty participants were tasked with learning to
correctly pair 60 novel objects with their names by choosing
on a trial-by-trial basis between two possible names and receiv-
ing feedback about the accuracy of their selection. The novel
objects were presented in three blocks of trials (rounds), each

of which presented the same set of 60 objects once. The rounds
allowed the separation of the initial informative feedback in
圆形的 1 from the other feedback stimuli in Rounds 2 和 3.
The results indicated differences in the processing of initial in-
formative and proceeding feedback stimuli. 进一步来说,
the difference appeared to be driven by the change in the pro-
cessing of positive feedback. 而且, very first positive feed-
back provided in association with a particular new object was
found associated with learning outcomes. The results imply that
signs of successful and unsuccessful learning may be detected
as early as the initial positive feedback provided in a learning
任务. The results suggest that the process giving rise to the feed-
back-related negativity is sensitive to the utility of the feedback
and that the processing of the first informative positive feed-
back is associated with learning outcomes. ■

介绍

Instructive feedback is an integral part of learning. 这是
provided in the school environment, at home, 还有
as in the work place. Feedback can serve different func-
系统蒸发散. It may inform the learner of the correct or expected
response when there is no other way of determining what
that response should be. It may resolve uncertainty and
facilitate learning when knowledge of the correct response
is still not fully stored. It also informs the learner about
the extent to which learning is successful. Broadly speak-
英, feedback in a learning task changes its role from
being informative (guiding learning) to being increasingly
evaluative (assessing learning). The error-related negativ-
性 (ERN) is an ERP associated with error commission. 这是
elicited when participants make an erroneous response
in speeded RT tasks (例如, Gehring, 戈斯, Coles, 迈耶, &
Donchin, 1993; Falkenstein, Hohnsbein, Hoormann, &
Blanke, 1990) and when the fact that a response was erro-
neous is communicated by a feedback event (例如, Miltner,
布劳恩, & Coles, 1997). The ERN elicited by feedback
(namely feedback-related negativity[FRN] or fERN) is elic-
ited in time estimation tasks (例如, Ferdinand, Mecklinger,
Kray, & Gehring, 2012; Gruendler, Ullsperger, & Huster,
2011; Oliveira, 麦当劳, & 古德曼, 2007; Miltner

1Massachusetts General Hospital Institute of Health Professions,
2University of South Florida

等人。, 1997), in gambling tasks (例如, Goyer, Woldorff, &
Huettel, 2008; Hajcak, 莫塞尔, Holroyd, & Simons, 2007;
Gehring & Willoughby, 2002), as well as in learning tasks
(例如, Arbel, Goforth, & Donchin, 2013; Sailer, Fischmeister,
& Bauer, 2010; van der Helden, Boksem, & Blom, 2010;
Eppinger, Mock, & Kray, 2009; Krigolson, Pierce, Holroyd,
& Tanaka, 2009; Pietschmann, 西蒙, Endrass, & Kathmann,
2008; Holroyd & Coles, 2002). The reinforcement learning
theory of the ERN (Holroyd & Coles, 2002) suggests that this
component is generated in ACC as a consequence of a phasic
decrease in the activity of the mesencephalic dopamine
system occurring when the monitoring system evaluates
events as worse than expected. It is also hypothesized that
this FRN signal is used by ACC for the adaptive modifica-
tion of behavior. It is still to be elucidated the extent to
which the FRN signal represents the process of evaluat-
ing performance (signals worse or better than expected
结果), the process of extracting information from
the feedback to facilitate behavior adaptation (IE。, signals
the utility of the feedback), 或两者. Learning tasks appear
optimal for the examination of this question. In gambling
任务, in which responses are not learnable, feedback
serves as a deliverer or denier of rewards. Its role can be
considered stable across the task. In learning tasks, 然而,
the role of the feedback is fluid as feedback serves differ-
ent functions at different stages of learning. 所以,
learning tasks that capture the changing role of feedback

认知神经科学杂志 26:7, PP. 1445–1453
土井:10.1162/jocn_a_00617

D
哦
w
n
我
哦
A
d
e
d

F
r
哦
米

我

/
j

t
t

F
/

我
t
.

:
/
/

H
t
t
p
:
/
D
/
哦
米
w
我
n
t
哦
p
A
r
d
C
e
.
d
s
F
我
r
哦
我
米
v
e
H
r
C
p
H
A
d
我
我
r
r
e
.
C
C
t
.
哦
米
米
/
j
e
d
哦
你
C
n
哦
/
C
A
n
r
A
t
r
我
t
我
C
C
我
e
e
–
p
–
d
p
d
2
F
6
/
7
2
6
1
/
4
7
4
/
5
1
1
4
9
4
4
5
7
/
4
1
0
7
1
8
哦
1
C
4
n
5
_
1
A
/
_
j
0
哦
0
C
6
n
1
7
_
A
p
_
d
0
0
乙
6
y
1
G
7
你
.
e
p
s
t
d
哦
F
n
乙
0
y
8
S
中号
e
我
p
时间
e
米
L
我
乙
乙
e
r
r
A
2
r
0
2
我
3
e
s

/
j

你
s
e
r

哦
n

1
7

中号
A
y

2
0
2
1

can shed light of the functional significance of the FRN and
elucidate the extent to which the process(英语) 产生
to feedback related ERPs is(是) related to the utilization
of feedback for task performance (IE。, the extent to which
feedback is informative). Such learning task was designed
by Tricomi and Fiez (2008), who divided a paired-associate
learning task into three rounds, each containing 60 对
presented once within each round. This design allowed
the separation of informative feedback (feedback pro-
vided during the first round of trials) from evaluative
feedback (feedback provided on the following rounds).
Their fMRI investigation displayed differences between
the processing of informative and evaluative feedback.
进一步来说, they found an increased caudate ac-
tivation and decreased left dorsolateral prefrontal cortex
(DLPFC) activation as feedback changed its role from being
informative to becoming more evaluative (IE。, an “earned”
feedback), with minimal caudate and DLPFC activation
differences between positive and negative when feedback
is purely informative (feedback provided on Round 1).
Their data indicated that it was the processing of positive
feedback that was associated with the changes in DLPFC
activation (Tricomi & Fiez, 2008) and with the relation-
ship between caudate activation and learning outcomes
(Tricomi & Fiez, 2012). In a previous report, we demon-
strated a relationship between the processing of positive
feedback as measured by the FRN and learning outcomes
(Arbel et al., 2013). We found that the activity elicited by
positive rather than negative feedback was association
with learning outcomes. 然而, in this previous report,
a four-choice paired-associate learning task was employed
in which positive feedback was more informative than
negative feedback. In a four-choice task, a positive feedback
confirms the correct selection, whereas a negative feed-
back implies that there are three other possible correct
responses. In light of the differences in the information
carried by the positive and negative feedback, the rela-
tionship found between positive feedback and learning out-
comes could be attributed to the valence of the feedback,
the information carried by the feedback, 或两者. 一
of the goals of the study reported here is to resolve this un-
certainty by employing a two-choice paired-associate learn-
ing task in which positive and negative feedback are equally
informative. In such a task, whereas positive feedback
informs the learner that the current choice is accurate,
negative feedback informs the learner that the alternative
choice is correct. 此外, this design permits the eval-
uation of the extent to which the processing of the very
first informative feedback provided in association with an
item is indicative of whether or not the item will be learned.

方法

参加者

Twenty undergraduate students (5 男人, 15 女性) aged
19–36 years (mean age = 22.85 年) from the Depart-

ment of Psychology at the University of South Florida
participated in this experiment. Participants reported to
be right-handed with no history of developmental dis-
orders or any other neurological deficits. They received
course credit for their participation in the study.

Task and Procedure

A two-choice paired-associate learning task was employed
in which participants were instructed to try to learn the
名字 (nonword) 的 60 novel objects. Participants were
presented with four blocks, which will be referred to as
“rounds” in this paper. ERPs were recorded during the first
three rounds. During the first round, participantsʼ choices
were followed by positive and negative feedback of equal
probability (.5 积极的, .5 negative). 所以, associa-
tions between objects and names were determined during
this round (and were different for each participant) 和
were kept throughout the remainder of the experiment.
和 60 trials in each of the three rounds, 的数量
positive and negative feedback presentations was identical
during the first round (30 trials during which positive feed-
back was presented, 和 30 trials with negative feedback).
The number of trials associated with positive and negative
feedback during the following two rounds varied based on
individual learning speed. The fourth round served to eval-
uate learning and to allow participants to complete a con-
fidence rating scale for each of the 60 对. During each
审判, participants were presented with a novel object ac-
companied by two possible names. Participants were asked
to choose one of the two names by pressing one of two
buttons on a response box. Each response was followed
by a performance feedback indicating whether the par-
ticipant made the correct choice. Positive feedback was
presented as “√√√,” and negative feedback was presented
as “xxx.” Each of the 60 objects was presented once within
each round.

To evaluate the relationship between the processing of
the initial informative feedback and learning outcomes, 我们
segmented the data of Round 1 based on the valence of
the feedback (positive and negative) and based on learn-
ing outcomes. Learning was determined based on the per-
formance on Round 4. Items that were correctly paired
with their name on Round 4 and for which participant
chose the confidence rating of 3 或者 4 (3 = maybe correct,
4 = sure correct) were categorized as “learned”; items that
were not correctly paired on Round 4 and for which the
confidence rating was 1 或者 2 (1 = sure incorrect, 2 =
maybe incorrect) were categorized as “not learned.”
Confidence rating data were used to ascertain that correct
guesses and incorrect learning are excluded from the
分析. This segmentation yielded four categories:

1. Positive feedback learned: Positive feedback pro-
vided during Round 1 to items that were subsequently
learned based on performance on Round 4 (received posi-
tive feedback on Round 4 with high confidence rating).

1446

认知神经科学杂志

体积 26, 数字 7

D
哦
w
n
我
哦
A
d
e
d

F
r
哦
米

我

/
j

F
/

t
t

我
t
.

:
/
/

/
j

你
s
e
r

哦
n

1
7

中号
A
y

2
0
2
1

2. Positive feedback not learned: Positive feedback
provided during Round 1 to items that were not sub-
sequently learned.
3. Negative feedback learned: Negative feedback
provided during Round 1 to items that were sub-
sequently learned.
4. Negative feedback not learned: Negative feedback
provided during Round 1 to items that were not sub-
sequently learned.

To evaluate the potential differences in the process-
ing of positive and negative feedback across the three
rounds, ERPs were segmented based on the valence
of the feedback (positive and negative) and based
on Rounds (圆形的 1, 圆形的 2, and Round 3). 这
segmentation yielded six categories: Round 1-Positive,
Round 1-Negative, Round 2-Positive, Round 2-Negative,
Round 3-Positive, and Round 3-Negative.

Stimuli

The novel objects were borrowed from Kroll and Potter
(1984). Nonwords were produced from the ARC Nonword
Database (Rastle, Harrington, & Coltheart, 2002). The non-
words were in three-letter consonant–vowel–consonant
(中央VC) 格式 (例如, joz) and were phonologically legal
in English. The number of orthographic neighbors (IE。,
real words in English that are written similarly) was be-
补间 0 和 10 (米= 5.9); the number of phonological
neighbors (IE。, real words in English whose pronunciation
is similar) was between 5 和 20 (米= 14).

EEG Recording Parameters

The EGI System 200 was used to acquire and analyze
dense array EEG data. The EEG was recorded using
129-channel HydroCel Geodesic Sensor Nets from EGI.
The EEG was continuously recorded at a 250-Hz sampling
rate with a band pass of 0.1–100 Hz. The electrode im-
pedances were kept below 50 kΩ. The continuous EEG
data were filtered using an offline 40-Hz low-pass filter.
The filtered data were then segmented into 800-msec long
纪元, each starting 200 msec before the presentation
of the feedback stimulus and ending 600 msec after the
feedback presentation. Baseline correction was performed
在 100 msec preceding the onset of the feedback.
An algorithm developed by Gratton, Coles, and Donchin
(1983) for offline removal of ocular artifacts was used to
correct for eye movements and blinks. 平均而言 0.018
的 160 ERP epochs was excluded from the analysis
due to excessive artifacts. Averages of the artifact free
baseline corrected epochs were calculated for each type
of feedback (Positive and Negative) and were separated
by rounds (圆形的 1, 圆形的 2, and Round 3) and by learn-
ing outcomes based on performance on Round 4 (学到了,
not learned). No ERP data were collected during Round 4.

The averaged EEG epochs were re-referenced to linked
mastoid.

数据分析

Analysis was done on 600-msec-long epochs, starting at
the onset of the feedback stimuli and ending 600 毫秒
following the feedback. To reduce the dimensionality of
the large data set and disentangle overlapping ERP com-
ponents, a spatiotemporal PCA as described by Spencer,
Dien, and Donchin (2001) was utilized. The spatial PCA
was performed by computing the covariance among
electrode sites across the time points of each of the feed-
back stimuli and participants, yielding a set of spatial
因素. In the next step of the analysis, the factor scores
for each of the participants, feedback stimuli, and elec-
trodes were computed for all time points. The plot of
these factor scores across the time axis created “virtual
ERPs” (Spencer et al., 2001), which were submitted to
a temporal PCA, analyzing the covariance among time
points for each of the spatial factors, feedback stimuli,
and participants. The resulting temporal factor scores
for each spatial factor were used to measure the activity
in the ERP with the morphology and scalp distributions
of interest. For both spatial and temporal PCAs, 这
factors that were required to account for 95% 的
variance in the input data set were retained for Varimax
rotation. The factor scores of the temporal and spatial
factors of interest were used for statistical analysis.

结果

平均而言, participants committed errors on 47.5% 的
the trials on Round 2, 45.3% on Round 3, 和 40.8% 在
圆形的 4. This relatively high error rate was expected as
each novel object was presented only four times (一次
during each of the four rounds) throughout the experi-
蒙特. Although error rate was relatively high, it is im-
portant to note that on average 50% (标准差= 12, range =
40–80%) of the correct responses on Round 1 were also
correct on Round 2, 60% of correct response on Round 2
were also correct on Round 3 (标准差= 11, range = 45–77%),
和 70% (标准差= 13, range = 56–95%) of the correct
responses on Round 3 were also correct on Round 4.
These findings suggest that, although participants did not
learn all of the associations by Round 4, they became
more consistent in their correct responses, indicating that
their responses after Round 1 were not merely guesses.
Repeated-measure analysis of error rate revealed a round
影响, F(3, 57) = 5.72, p = .01. Post hoc paired com-
parison indicated that the reduction in error rate occurred
between Round 1 and Round 2, F(1, 19) = 4.6, p = .047,
and between Round 3 and Round 4, F(1, 19) = 9.93, p =
.006, whereas no significant differences were found
between Rounds 2 和 3, F(1, 19) = .63, p = .43. 我们
first defined learning as a correct response on Round 4

Arbel, 墨菲, and Donchin

1447

D
哦
w
n
我
哦
A
d
e
d

F
r
哦
米

我

/
j

F
/

t
t

我
t
.

:
/
/

/
j

你
s
e
r

哦
n

1
7

中号
A
y

2
0
2
1

accompanied by a high-confidence rating (a rating of 3 或者
4). An examination of learning as a function of feedback
valence on Round 1 indicated that more learned items
were associated with positive feedback on Round 1 比
with negative feedback, F(1, 19) = 6.4, p = .02. An exam-
ination of the confidence rating on Round 4 as a function
of feedback valence on Round 1 indicated no significant
差异, F(1, 19) = 2, p = 1.7, suggesting that con-
fidence rating on Round 4 was not affected by valence on
圆形的 1. When learning was defined as correct responses
on Round 4 regardless of confidence rating, the effect of
feedback valence remained, F(1, 19) = 8.6, p = .009. 这些
results suggest that, although positive and negative feed-
back on Round 1 were equally informative and equally
probable, positive feedback resulted in better learning
outcomes than negative feedback.

ERP Data

To examine whether the processing of the initial infor-
mative feedback (feedback on Round 1) is different from
the processing of the proceeding feedback stimuli which
become increasingly evaluative (feedback on Rounds 2
和 3), separate averages were computed for each of the
rounds. 数字 1 presents the grand-averaged ERP data
from electrode FCz in which the FRN is typically exam-
内德. As can be seen in the figure, the ERPs elicited during
圆形的 1 appear different from those elicited during
Rounds 2 和 3. The difference can be described as a
smaller difference between the activities associated with
positive and negative feedback on Round 1 in comparison
with the other two.

A more detailed analysis is provided by the STPCA. 这
spatial factor that captures the FRN activity is SF1 (fronto-
central; 见图 2). The virtual ERPs presented in

数字 2 seem to mirror the activity shown in the grand
average of FCz, with a small difference between positive
and negative feedback in Round 1. Temporal factor 4 曾是
selected for further analysis as it represents the epoch
during which the FRN was elicited (negativity with a
latency of about 250 毫秒).

A 2 × 3 repeated-measure analysis with two levels
of Feedback Valence (positive and negative feedback)
and three levels of Rounds (圆形的 1, 圆形的 2, 和
圆形的 3) was conducted on the scores of SF1-TF4. A
main effect of Feedback Valence was found, F(1, 19) =
35.46, p < .0001. A main effect of Rounds was not found, F(2, 38) = 1.82, p = .17. However, an interaction between Valence and Rounds was found, F(2, 38) = 3.4, p = .05. A post hoc paired comparison revealed that this effect was driven by the difference between the activity elicited by positive feedback on Round 1 and the activity elicited by positive feedback on the next two rounds, such that the activity associated with positive feedback was the smallest during Round 1 (positive feedback: Round 1 vs. Round 2, t(1, 19) = −3.24, p = .005; Round 1 vs. Round 3, t(1, 19) = −2.56, p = .02). No differences were found between the negative feedback on Round 1 and the other two rounds (negative feedback: Round 1 vs. Round 2, t(1, 19) = −0.48, p = .96; Round 1 vs. Round 3, t(1, 19) = −0.69, p = .5). These results support the visual evaluation of a smaller difference between positive and negative feedback during the first round. Initial Feedback and Learning Outcomes A separate analysis has been conducted (including PCA) to evaluate the potential relationship between the process- ing of the initial informative feedback (feedback process- ing on Round 1) and learning outcomes. To examine this Figure 1. Grand-averaged ERP data from electrode FCz for positive (dashed line) and negative (solid line) feedback for Rounds 1 (left), 2 (middle), and 3 (right). 1448 Journal of Cognitive Neuroscience Volume 26, Number 7 D o w n l o a d e d f r o m l l / / / / j t t f / i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 6 / 7 2 6 1 / 4 7 4 / 5 1 1 4 9 4 4 5 7 / 4 1 0 7 1 8 o 1 c 4 n 5 _ 1 a / _ j 0 o 0 c 6 n 1 7 _ a p _ d 0 0 b 6 y 1 g 7 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j f . / t u s e r o n 1 7 M a y 2 0 2 1 D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 6 / 7 2 6 1 / 4 7 4 / 5 1 1 4 9 4 4 5 7 / 4 1 0 7 1 8 o 1 c 4 n 5 _ 1 a / _ j 0 o 0 c 6 n 1 7 _ a p _ d 0 0 b 6 y 1 g 7 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j f / . t u s e r o n 1 7 M a y 2 0 2 1 Figure 2. Virtual map of Spatial Factor 1 (frontocentral), virtual ERPs of Spatial Factor 1 for positive (dashed line) and negative (solid line) feedback, for each of the three rounds (Round 1 on the left, Round 2 in the middle, and Round 3 on the right). relationship, we examined the ERPs obtained on Round 1 as they were related to performance on Round 4 which served as the learning outcome measure. A 2 × 2 (2 levels of feedback valence and 2 levels of learning) repeated- measure analysis was conducted on the factor scores of the FRN component (SF1-TF3, fronto-central with a la- tency of about 250 msec; see virtual ERPs in Figure 3). A Valence effect was found, F(1, 19) = 11.69, p = .003. No Learning effect, F(1, 19) = .44, p = .51, was found. However, an interaction between Feedback Valence and Learning was found, F(1, 19) = 4.2, p = .05. Post hoc paired comparison suggests that differences related to learning were only associated with positive feedback, such that positive feedback provided to items that were subsequently learned was different from positive feedback provided to items that were not learned, t(1, 19) = −3.6, p = .006. No such differences were found for the nega- tive feedback, t(1, 19) = 0.78, p = .45. These results suggest that the processing of the very first positive feedback is associated with learning outcomes and there- fore may be indicative of successful and unsuccessful learning. These findings of the association between the process- ing of the initial feedback and learning outcomes called for an additional analysis of the possible relationship between the feedback provided on Rounds 2 and 3 and learning outcomes. No associations were found between positive and negative feedback provided on Rounds 2 and 3 and learning outcomes (i.e., no Learning effect, F(1, 19) = 3.05 p = .58, or an interaction between Valence and Learning, F(1, 19) = 1.07, p = .31). DISCUSSION The study examined two questions related to the process- ing of feedback in a learning task. The first question was concerned with the extent to which the neurophysiological markers of feedback processing are sensitive to the role of the feedback in the learning process. The second ques- tion was whether the processing of the initial informative feedback is related to learning outcomes. Our data point to processing differences between the initial informative feedback and the proceeding performance feedback stimuli. More specifically, we found that the ERP activity elicited in association with positive feedback changed as the feedbackʼs role developed from being purely informative to being both informative and evaluative, with a reduction in FRN amplitude from the first round to the Arbel, Murphy, and Donchin 1449 other two. Our findings that changes in the FRN amplitude were associated with positive rather than negative feed- back are in line with previous reports of greater modula- tion of the FRN to positive feedback (e.g., Arbel et al., 2013; Kreussel et al., 2012; Baker & Holroyd, 2011; Foti, Weinberg, Dien, & Hajcak, 2011; San Martin, Manes, Hurtado, Isla, & Ibanez, 2010; Eppinger, Mock, & Kray, 2009; Eppinger, Kray, Mock, & Mecklinger, 2008; Cohen, Elger, & Ranganath, 2007). Interestingly, on Round 1, when positive and negative feedback were equally informative and probable, an FRN was elicited by both positive and negative feedback, with greater magnitude of FRN to negative feedback. The examination of the FRN on Round 1 also revealed that the activation associated with positive feedback was related to the learning outcomes. Positive feedback provided to items that were subsequently learned elicited a larger FRN when compared with posi- tive feedback provided to items that were not learned. These results are in line with our previous findings of a relationship between FRN elicited by positive feedback in a learning task and learning outcomes (Arbel et al., 2013). The results of the current experiment suggest that, even when the positive and negative feedback are equally in- formative and probable, it is the positive feedback which is associated with learning outcomes. In the next section, we will examine our findings within the framework of existing theories and within our pro- posed utility account. The Reward Prediction Error Account Growing evidence of greater modulation of the FRN to positive feedback has led to some revisions to the re- inforcement learning theory of the ERN. In the original report (Holroyd & Coles, 2002), the FRN was presented as a negative signal of worse than expected outcomes. It was later suggested that the FRN is an N200 elicited by events that fail to indicate that a task goal has been achieved (Holroyd, Pakzad-Vaezi, & Krigolson, 2008) or by any unexpected outcomes regardless of valence (Holroyd, Krigolson, & Lee, 2011) and that the amplitude of this component is reduced by reward, which elicits the feedback-related correct response positivity (fCRP) in the same time window. It is important to note that a reward prediction component, namely the P2a, was pre- viously described by Potts, Martin, Burton, and Montague (2006). Although the N200 fCRP account is appealing, it is not yet clear whether the N200 is elicited by all un- expected outcomes and is being reduced by the fCRP or whether it is elicited only by negative outcomes. It is also not clear whether the fCRP is sensitive to reward expectancy or to the assessment of task achievement. In a learning task, the initial positive feedback is assumed to be the most unpredicted positive outcome. Therefore, if the fCRP or P2a reflects the processing of unexpected reward, in a learning task it should be the largest during the first phase of the learning process and to be reduced as positive feedback becomes more predictable. On the other hand, if the fCRP represents the assessment of task achievement, one would expect the amplitude of this positivity to become larger as learning progresses. It appears that these two possible interpretations create opposite predictions in a learning task. Our data do not support the reward expectancy account, as the fCRP became more positive during Rounds 2 and 3 when positive feedback was less surprising. If we adopt the alternative account that the fCRP represents the assess- ment of task achievement, it can be argued that as the learning process progressed, the increase in fCRP ampli- tude reflected a clearer assessment of the positive feed- back as an indication of task achievement. Our finding that positive feedback that was associated with successful learning elicited a larger negativity does not sit well with either the expectancy account of the fCRP or with the task achievement one. If the fCRP reflects better than expected outcomes or an achievement of a task goal, one would expect that larger positivity will be associated with successful learning. Figure 3. Virtual ERPs of Spatial Factor 1 (frontocentral) of feedback provided on Round 1 as a function of learning outcomes as indicated by performance on Round 4. Positive feedback provided to items that were subsequently learned (PosFd-Learn, black solid line); positive feedback provided to items that were not learned (PosFd-NotLearn, black dashed line); negative feedback provided to items that were subsequently learned (NegFd-Learn, gray solid line); and negative feedback provided to items that were not learned (NegFd-NotLearn, gray dashed line). The Expectancy Account Several FRN theories discuss the FRN as a product of the assessment of outcomes in relation to expectancy. However, there are conflicting views about the relation- ship between valence and the processing of expectancy. Whereas the reinforcement learning account of the FRN 1450 Journal of Cognitive Neuroscience Volume 26, Number 7 D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 6 / 7 2 6 1 / 4 7 4 / 5 1 1 4 9 4 4 5 7 / 4 1 0 7 1 8 o 1 c 4 n 5 _ 1 a / _ j 0 o 0 c 6 n 1 7 _ a p _ d 0 0 b 6 y 1 g 7 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j t . / f u s e r o n 1 7 M a y 2 0 2 1 distinguishes between the processing of worse than ex- pected from that of better than expected (or unexpected reward), others suggest that the FRN is sensitive to violation of expectancy regardless of feedback valence. Ferdinand et al. (2012) reported that in an experiment whose design allowed the separation of valence and expectancy, FRN was elicited by both positive and nega- tive feedback when expectancy was violated. Our find- ings that FRN was elicited by both positive and negative feedback during the first round can be explained within the framework of Ferdinand et al.ʼs expectancy account. One may suggest that during the first round of our ex- periment participants did not form specific expectancies and therefore any feedback was in a sense unexpected. This explanation is difficult to test, and the validity of the notion that participants were equally surprised by positive and negative feedback during this phase is in our opinion questionable. A more likely scenario is that learners had some expectations that varied based on previous learning experiences. Generally speaking, it would have been expected for learners to be more surprised by positive feedback than by negative feed- back at this initial stage of learning. In that case, the expectancy account will not fit well with our findings as the larger FRN amplitude was associated with negative feedback. The Proposed Utility Account We propose that the activity associated with the pro- cessing of feedback as depicted by the FRN is indicative of the utility of the feedback. More specifically, we sug- gest that at the stage at which feedback is “purely” infor- mative, the utility rather than the valence of the feedback is given a processing “priority.” Because both negative and positive feedback stimuli are equally informative at this stage, they both elicit the FRN. The larger FRN magni- tude associated with negative feedback during this round may suggest that, although the utility of the feedback played a primary role, feedback valence also affected the processing of this informative feedback. The reduction in the amplitude of the FRN associated with positive feed- back is consistent with this view. In our data set, the amplitude of the FRN elicited in association with positive feedback decreased from the first round to the other two rounds. Within our proposed utility account, this change may reflect the process of decreased relevance of the positive feedback to the learning process after the correct associations were learned (on Rounds 2 and 3). Whereas positive feedback loses its relevance after the correct asso- ciation has been learned, negative feedback remains rele- vant for task performance across the three rounds as the learners continue to use the feedback to extract informa- tion. Within this framework, negative feedback was asso- ciated with a stable activation of this process as it served to inform the learner of the correct association throughout the task, whereas positive feedback became redundant or less informative after the correct associations have been learned. Given that in this task negative feedback was more likely to follow prelearning errors rather than post- learning slips, we hypothesize that a reduction in the FRN amplitude would have been observed after negative feedback had lost its relevance for task performance if more trials were presented. Although this hypothesis cannot be tested with our data, it is in line with previous reports of reduced FRN amplitude in probabilistic learn- ing tasks after the correct associations have been learned (e.g., Holroyd & Coles, 2002). Our data suggest that the activity associated with posi- tive feedback during the initial round was related to learning outcomes. One explanation for the results could be the initial positive feedback is more important for the learning process than the initial negative feedback. This suggestion is supported by our behavioral data that indicate that items that received positive feedback on Round 1 were more likely to be subsequently learned when compared with items that received negative feedback on Round 1. An alternative explanation is that, although the assumption was that in a two-choice paired-associate learning task positive and negative feedback are equally informative, it is possible that extracting information from the positive feedback is less demanding than extracting information from the negative feedback. Whereas positive feedback reinforces the accuracy of the current choice, negative feedback requires the learner to reject the cur- rent hypothesis and to adopt the alternative, resulting in the need to switch the item to be held in working memory. The utility account can be applied to explain the amplitude differences between positive feedback that resulted in successful learning and positive feedback that resulted in unsuccessful learning. Within the frame- work of this account, larger negativity was associated with greater utilization of the feedback. Within our proposed utility account, we suggest that the utility is defined by the role of the feedback in a particular task. For example, FRN known to be elicited in association with losses and its amplitude is reported to be sensitive to violations of reward expectancies (e.g., Bellebaum, Polezzi, & Daum, 2010; Holroyd & Krigolson, 2007; Holroyd, Nieuwenhuis, Yeung, & Cohen, 2003). In these cases, the feedback is not informative for task performance in the same manner feedback in a learning task is. However, in a task in which the participantʼs goal is to gain as much money as possible, feedback can be viewed as providing relevant information that may be used by the participant to adjust future choices or make predictions. We suggest that the process giving rise to the FRN is concerned with extracting relevant information from the feedback. It is not merely a “good” versus “bad” process or one that weighs outcomes in comparison with expected outcomes, but rather a more complex process that may extract dif- ferent types of information from the feedback depending on the needs of the processor of information. The lack Arbel, Murphy, and Donchin 1451 D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 6 / 7 2 6 1 / 4 7 4 / 5 1 1 4 9 4 4 5 7 / 4 1 0 7 1 8 o 1 c 4 n 5 _ 1 a / _ j 0 o 0 c 6 n 1 7 _ a p _ d 0 0 b 6 y 1 g 7 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j t . f / u s e r o n 1 7 M a y 2 0 2 1 of FRN in association with gains in gambling tasks may point to a separate activation that is concerned with the process of reward delivery. It is possible that the feedback-related ERPs reflect a combination of processes associated with both the utility of the feedback and its valence (or reward value). It is important to emphasize that, in the current report, differences in the processing of positive feedback for items that were subsequently learned and those that were not were detected following the very first feedback provided in association with a particular new object. The results imply that signs of successful and unsuccessful learning may be detected as early as the initial positive feedback provided in a learning task. Future studies should examine the extent to which predictions can be made about learning outcomes based on the initial pro- cessing of positive feedback. Reprint requests should be sent to Yael Arbel, MGH-Institute of Health Professions, 36 1st Ave., Boston, MA 02129, or via e-mail: yarbel@mghihp.edu, yarbel@mail.usf.edu. REFERENCES Arbel, Y., Goforth, K., & Donchin, E. (2013). The good, the bad, or the useful? The examination of the relationship between the feedback related negativity (FRN) and long-term learning outcomes. Journal of Cognitive Neuroscience, 25, 1249–1260. Baker, T. E., & Holroyd, C. B. (2011). Dissociated roles of the anterior cingulate cortex in reward and conflict processing as revealed by feedback error-related negativity and N200. Biological Psychology, 87, 25–34. Bellebaum, C., Polezzi, D., & Daum, I. (2010). It is less than you expected: The feedback-related negativity reflects violations of reward magnitude expectations. Neuropsychologia, 48, 3343–3350. Cohen, M. X., Elger, C. E., & Ranganath, C. (2007). Reward expectation modulates feedback-related negativity and EEG spectra. Neuroimage, 35, 968–978. Eppinger, B., Kray, J., Mock, B., & Mecklinger, A. (2008). Better or worse than expected? Aging, Learning, and the ERN. Neuropsychologia, 46, 521–539. Eppinger, B., Mock, B., & Kray, J. (2009). Developmental differences in learning and error processing: Evidence from ERPs. Psychophysiology, 46, 1043–1053. Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., & Donchin, E. (1993). A neural system for error detection and compensation. Psychological Science, 4, 385–390. Gehring, W. J., & Willoughby, A. R. (2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science, 295, 2279–2282. Goyer, J. P., Woldorff, M. G., & Huettel, S. A. (2008). Rapid electrophysiological brain responses are influenced by both valence and magnitude of monetary rewards. Journal of Cognitive Neuroscience, 20, 2058–2069. Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A new method for off-line removal of ocular artifact. Electroencephalography and Clinical Neurophysiology, 55, 468–484. Gruendler, T. O. J., Ullsperger, M., & Huster, R. J. (2011). Event related potential correlates of performance monitoring in a lateralized time estimation task. PLoS One, 6, e25591. Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2007). Itʼs worse than you thought: The feedback negativity and violations of reward prediction in gambling tasks. Psychophysiology, 44, 905–912. Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679–709. Holroyd, C. B., & Krigolson, O. E. (2007). Reward prediction error signals associated with a modified time estimation task. Psychophysiology, 44, 913–917. Holroyd, C. B., Krigolson, O. E., & Lee, S. (2011). Reward positivity elicited by predictive cues. NeuroReport, 22, 249–252. Holroyd, C. B., Nieuwenhuis, C. A. S., Yeung, N., & Cohen, J. D. (2003). Errors in reward prediction are reflected in the event-related brain potential. NeuroReport, 14, 2481–2484. Holroyd, C. B., Pakzad-Vaezi, K. L., & Krigolson, O. E. (2008). The feedback correct-related positivity: Sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology, 45, 688–697. Kreussel, L., Hewig, J., Kretschmer, N., Hecht, H., Coles, M. G. H., & Miltner, W. H. R. (2012). The influence of the magnitude, probability, and valence of potential wins and losses on the amplitude of the feedback negativity. Psychophysiology, 49, 207–219. Krigolson, O. E., Pierce, L. J., Holroyd, C. B., & Tanaka, J. W. (2009). Learning to become an expert: Reinforcement learning and the acquisition of perceptual expertise. Journal of Cognitive Neuroscience, 21, 1834–1841. Kroll, J. F., & Potter, M. C. (1984). Recognizing words, pictures, and concepts: A comparison of lexical, object, and reality decisions. Journal of Verbal Learning and Verbal Behavior, 23, 39–66. Falkenstein, M., Hohnsbein, J., Hoormann, J., & Blanke, L. Miltner, W. H. R., Braun, C. H., & Coles, M. G. H. (1997). (1990). Effects of errors in choice reaction tasks on the ERP under focused and divided attention. In C. H. M. Brunia, A. W. K. Gaillard, & A. Kok (Eds.), Psychophysiological brain research (pp. 192–195). Tilburg, The Netherlands: Tilburg University Press. Ferdinand, N. K., Mecklinger, A., Kray, J., & Gehring, W. J. (2012). The processing of unexpected positive response outcome in the mediofrontal cortex. The Journal of Neuroscience, 32, 12087–12092. Foti, D., Weinberg, A., Dien, J., & Hajcak, G. (2011). Event-related potential activity in the basal ganglia differentiates reward from non-rewards: Temporal spatial principal components analysis and source localization of the feedback negativity. Human Brain Mapping, 32, 2207–2216. Event-related brain potentials following incorrect feedback in a time-estimation task: Evidence for a “generic” neural system for error detection. Journal of Cognitive Neuroscience, 9, 788–798. Oliveira, F. T., McDonald, J. J., & Goodman, D. (2007). Performance monitoring in the anterior cingulate is not all error related: Expectancy deviation and the representation of action-outcome associations. Journal of Cognitive Neuroscience, 19, 1994–2004. Pietschmann, M., Simon, K., Endrass, T., & Kathmann, N. (2008). Changes of performance monitoring with learning in older and younger adults. Psychophysiology, 45, 559–568. Potts, G. F., Martin, L. E., Burton, P., & Montague, P. R. (2006). When things are better or worse than expected: The 1452 Journal of Cognitive Neuroscience Volume 26, Number 7 D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 6 / 7 2 6 1 / 4 7 4 / 5 1 1 4 9 4 4 5 7 / 4 1 0 7 1 8 o 1 c 4 n 5 _ 1 a / _ j 0 o 0 c 6 n 1 7 _ a p _ d 0 0 b 6 y 1 g 7 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j / f t . u s e r o n 1 7 M a y 2 0 2 1 medial frontal cortex and the allocation of processing resources. Journal of Cognitive Neuroscience, 18, 1112–1119. Spencer, K., Dien, J., & Donchin, E. (2001). Spatiotemporal analysis of the late ERP responses to deviant stimuli. Psychophysiology, 38, 343–358. Rastle, K., Harrington, J., & Coltheart, M. (2002). 358,534 Tricomi, E., & Fiez, J. A. (2008). Feedback signals in the caudate nonwords: The ARC nonword database. Quarterly Journal of Experimental Psychology, 55A, 1339–1362. reflect goal achievement on a declarative memory task. Neuroimage, 41, 1154–1167. San Martin, R., Manes, F., Hurtado, E., Isla, P., & Ibanez, A. (2010). Size and probability of rewards modulate the feedback error-related negativity associated with wins but not losses in a monetarily rewarded gambling task. Neuroimage, 51, 1194–1204. Sailer, U., Fischmeister, F. P., & Bauer, H. (2010). Effects of learning on feedback-related brain potentials in a decision-making task. Brain Research, 1342, 85–93. Tricomi, E., & Fiez, J. A. (2012). Information content and reward processing in the human striatum during performance of a declarative memory task. Cognitive Affective Behavioral Neuroscience, 12, 361–372. van der Helden, J., Boksem, M. A. S., & Blom, J. H. G. (2010). The importance of failure: Feedback-related negativity predicts motor learning efficiency. Cerebral Cortex, 20, 1596–1603. D o w n l o a d e d f r o m l l / / / / j t t f / i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 6 / 7 2 6 1 / 4 7 4 / 5 1 1 4 9 4 4 5 7 / 4 1 0 7 1 8 o 1 c 4 n 5 _ 1 a / _ j 0 o 0 c 6 n 1 7 _ a p _ d 0 0 b 6 y 1 g 7 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 2 i 3 e s / j / f . t u s e r o n 1 7 M a y 2 0 2 1 Arbel, Murphy, and Donchin 1453 On the Utility of Positive and Negative Feedback in a image

下载pdf