Neural Representation of Reward Probability:
Evidence from the Illusion of Control

Wouter Kool, Sarah J. Getz, and Matthew M. Botvinick

Abstract

■ To support reward-based decision-making, the brain must
encode potential outcomes both in terms of their incentive value
and their probability of occurrence. Recent research has made it
clear that the brain bears multiple representations of reward mag-
nitude, meaning that a single choice option may be represented
differently—and even inconsistently—in different brain areas.
There are some hints that the same may be true for reward prob-
ability. Preliminary evidence hints that, even as systematic distor-
tions of probability are expressed in behavior, these may not
always be uniformly reflected at the neural level: Some neural
representations of probability may be immune from such distor-

tions. This study provides new evidence consistent with this pos-
sibility. Participants in a behavioral experiment displayed a classic
“illusion of control,” providing higher estimates of reward prob-
ability for gambles they had chosen than for identical gambles
that were imposed on them. However, an fMRI study of the same
task revealed that neural prediction error signals, arising when
gamble outcomes were revealed, were unaffected by the illusion
of control. The resulting behavioral–neural dissociation reinforces
the case for multiple, inconsistent internal representations of
reward probability, while also prompting a reinterpretation of
the illusion of control effect itself. ■

INTRODUCTION

To make principled choices between available lines of
action, a decision-maker must evaluate potential outcomes
in terms of both value and probability. These two attri-
butes stand as the key ingredients in classical theories
of economic choice (Bernoulli, 1954; von Neumann &
Morgenstern, 1944), providing the necessary and suf-
ficient materials for computing expected utility. More
recent behavioral economic theories have introduced
the important idea that subjective representations of
outcome value and probability may both be subject to
distortions, that is, nonlinear transformations. Even here,
however, value and probability maintain their role as the
two key pillars supporting decision-making (Hertwig,
Barron, Weber, & Erev, 2004; Starmer, 2000; Tversky &
Kahneman, 1992; Kahneman, Knetsch, & Thaler, 1990;
Kahneman & Tversky, 1979; Allais, 1953).

There has been a major effort over recent years to
characterize representations of reward magnitude and
probability in the brain, using neurophysiological, neuro-
psychological, and neuroimaging techniques (Kobayashi,
Lauwereyns, Koizumi, Sakagami, & Hikosaka, 2002; Platt
& Glimcher, 1999). One important result of such work
has been to corroborate the nonlinear transformations in
value and probability posited by behavioral economic the-
ories (FitzGerald, Seymour, Bach, & Dolan, 2010; Berns,
Capra, Chappelow, Moore, & Noussair, 2008; Paulus &

Princeton University

© 2013 Massachusetts Institute of Technology

Frank, 2006; Trepel, Fox, & Poldrack, 2005). Another
insight, not fully anticipated by economics, is that the brain
encodes multiple representations of reward magnitude
and probability, which differ in format and functional role.
The case for such representational variety is quite strong in
the case of reward magnitude, where numerous studies
have documented encodings that arise concurrently but
differ according to pertinent outcome (Rangel & Hare,
2010; Padoa-Schioppa & Assad, 2006), frame of reference
(De Martino, Kumaran, Holt, & Dolan, 2009), current plan
of action (Roesch, Singh, Brown, Mullins, & Schoenbaum,
2009), or flexibility in the face of change (Simon & Daw,
2011). Particularly striking are cases where concurrent
reward representations are inconsistent with one another
(Hutcherson, Plassmann, Gross, & Rangel, 2012; McClure,
Ericson, Laibson, Loewenstein, & Cohen, 2007), suggesting
that choice options are being encoded in parallel by dis-
sociable decision-making or learning mechanisms (Daw,
Niv, & Dayan, 2005).

There is scattered evidence for the possibility that re-
ward probability may also map onto multiple, potentially
divergent internal representations. An fMRI study by Tobler,
Christopoulos, OʼDoherty, Dolan, and Schultz (2008)
observed that encodings of reward probability within two
regions of pFC were subject to different nonlinear trans-
formations. Intriguingly, the same study found that prob-
ability representation in the striatum was essentially
linear, leading the authors to propose “a neuronal disso-
ciation between veridical and distorted probability pro-
cessing in the striatum and prefrontal cortex, respectively”

Journal of Cognitive Neuroscience 25:6, pp. 852–861
doi:10.1162/jocn_a_00369

D
o
w
n
l
o
a
d
e
d

f
r
o
m

l

l

/

/

/

/
j

f
/

t
t

i
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
d
o
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
–
p
–
d
p
d
2
f
5
/
6
2
5
8
/
5
6
2
/
1
8
9
5
4
2
5
/
3
1
5
7
7
7
o
9
c
2
n
5
_
4
a
/
_
j
0
o
0
c
3
n
6
9
_
a
p
_
d
0
0
b
3
y
6
g
9
u
.
e
p
s
t
d
o
f
n
b
0
y
8
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
i
2
3
e
s

/
j

/

t

.

f

u
s
e
r

o
n

1
7

M
a
y

2
0
2
1

(p. 11704). Convergent evidence for such a cortical–
subcortical dissociation can be gleaned from other studies.
In particular, whereas numerous studies have reported
nonlinear representations of outcome probability in corti-
cal areas (FitzGerald et al., 2010; Berns et al., 2008; Paulus &
Frank, 2006), Abler and colleagues (Abler, Walter, Erk,
Kammerer, & Spitzer, 2006) found that reward prediction
error signals in ventral striatum scaled linearly with outcome
probability (see also Hsu, Krajbich, Zhao, & Camerer, 2009).
To shed further light on neural encodings of outcome
probability, we turned to a behavioral effect known as the
“illusion of control” (IOC). First reported by Langer
(1975), the IOC is a tendency to overestimate the prob-
ability of favorable outcomes in chance situations, when
those situations are chosen rather than imposed.1 In
Langerʼs classic study, participants who chose one from
several lottery tickets were later less willing to trade their
tickets for others, compared with participants who had
simply been given a ticket without choice, although the
probability of winning the lottery was the same for both
groups. Subsequent research has documented that this
behavioral bias reflects, at least in part, a distortion of
outcome probability estimates, with chosen gambles
being accorded a higher probability of yielding a favor-
able outcome (for reviews, see Thompson, Armstrong,
& Thomas, 1998; Presson & Benassi, 1996).

To our knowledge, the neural correlates of the IOC
have not been investigated. However, the “illusion,” like
other probability distortion effects, offers an opportunity
to probe the neural processes that underlie subjective
judgments of reward probability. To pursue this, we
measured brain activity with fMRI as participants per-
formed an experiment that elicited the IOC.

Our primary focus in analyzing the resulting data was
on the moment that reward outcomes were revealed. A
large body of research has shown that reward outcomes
trigger a reward prediction error (RPE) signal, originating
in midbrain dopaminergic nuclei, but detectable in an
array of cortical and subcortical structures, most notably
the ventral striatum (Schultz, Dayan, & Montague, 1997;
Montague, Dayan, & Sejnowski, 1996). The RPE encodes
the difference between observed and expected reward
(Sutton & Barto, 1998; Rescorla & Wagner, 1972) and
thus varies with outcome probability: The less likely a
positive outcome is judged to be, the larger the RPE
when it occurs (Fiorillo, Tobler, & Schultz, 2003).

Because neural RPE signals reflect prospective estimates
of reward probability, it seems plausible that they should
be impacted by the IOC. In particular, because the IOC
inflates estimates of reward probability, it might be ex-
pected to reduce RPE magnitude in the case of positive
outcomes, because it effectively makes them less surpris-
ing. By the same token, the IOC might be expected to
increase the (negative) amplitude of the RPE triggered by
nonreward outcomes by making them more surprising.

The behavioral study was aimed at confirming that our
chosen behavioral task yields the classic IOC effect, trig-
gering inflated estimates of reward probability. In the
neuroimaging experiment, participants performed the
same task while undergoing fMRI. Our prediction was that
neural RPE signals, for example, within the ventral stria-
tum, would reflect the IOC, being reduced for outcomes
in gambles that had been chosen compared with gambles
that were imposed.

As detailed in what follows, the data flatly contradicted
this prediction. At the level of behavior, participants
showed a clear IOC, evincing higher estimates of reward
probability following choice. However, striatal RPE sig-
nals appeared to be entirely immune to this effect. As
we shall argue, this surprising finding adds to the emerg-
ing evidence that subcortical probability representations
can be impervious to behaviorally expressed distortions,
while also inviting a reinterpretation of the IOC effect
itself.

EXPERIMENT 1

In this initial experiment, we sought to replicate the behav-
ioral IOC effect in a novel, multitrial task paradigm.

Methods

Participants and Procedures

Fourteen participants from the Princeton University com-
munity (aged 18–23 years, 5 women) took part in Experi-
ment 1. All participants received a nominal payment and
provided informed consent, following procedures approved
by the Princeton University Institutional Review Board.

The task was computer based and programmed using
the Psychophysics Toolbox for Matlab (Brainard, 1997;
Pelli, 1997). On each of the 96 trials, three spinner dials
were presented, depicted as wheels of fortune or spin-
ners: white dials containing a green “win sector” and a
black arrow (Figure 1A). They were positioned at the cor-
ners of a (hidden) randomly rotated equilateral triangle.
The size of the win sector varied from trial to trial but was
identical across the three spinners shown together on each
individual trial. They were drawn from one of four uniform
ranges of width 0.10 around four means (0.2, 0.4, 0.6,
and 0.8).

Participants were specifically and truthfully informed
that the three spinners in each trial had an equal probabil-
ity of a win outcome. Despite this, participants were re-
quired to select one of the three available options using
a mouse. To avoid any bias to a given spinner based on
proximity, the mouse cursor was set in the center of the
screen at the start of the trial. A blue circle appeared
around the selected spinner, indicating the participantʼs
preference.

To test this prediction, we conducted two experiments,
one behavioral and one using functional neuroimaging.

In the second part of the trial, the approval phase, on
an arbitrary one half of trials the participantʼs choice was

Kool, Getz, and Botvinick

853

D
o
w
n
l
o
a
d
e
d

f
r
o
m

l

l

/

/

/

/
j

f
/

t
t

i
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
d
o
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
–
p
–
d
p
d
2
f
5
/
6
2
5
8
/
5
6
2
/
1
8
9
5
4
2
5
/
3
1
5
7
7
7
o
9
c
2
n
5
_
4
a
/
_
j
0
o
0
c
3
n
6
9
_
a
p
_
d
0
0
b
3
y
6
g
9
u
.
e
p
s
t
d
o
f
n
b
0
y
8
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
i
2
3
e
s

/
j

t

.

f

/

u
s
e
r

o
n

1
7

M
a
y

2
0
2
1

D
o
w
n
l
o
a
d
e
d

f
r
o
m

l

l

/

/

/

/
j

f
/

t
t

i
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
d
o
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
–
p
–
d
p
d
2
f
5
/
6
2
5
8
/
5
6
2
/
1
8
9
5
4
2
5
/
3
1
5
7
7
7
o
9
c
2
n
5
_
4
a
/
_
j
0
o
0
c
3
n
6
9
_
a
p
_
d
0
0
b
3
y
6
g
9
u
.
e
p
s
t
d
o
f
n
b
0
y
8
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
i
2
3
e
s

/
j

t

.

f

/

u
s
e
r

o
n

1
7

M
a
y

2
0
2
1

Figure 1. (A) Sequence of events in the experiments. In Experiment 2 (panel with dotted border), each trial started with participants centering the
joystick. Participantsʼ selected one of three “spinners” with the mouse (Experiment 1) or the joystick (Experiment 2), and this original decision
was marked with a blue circle. When the initial choice was vetoed (as depicted), a yellow circle appeared around another spinner, otherwise a
yellow circle would replace the blue circle around the original choice. In the behavioral experiment, participants then reported their confidence using
a visual analogue scale (dashed border). Next, the ultimately selected spinner would start revolving with decelerating speed, resulting either in a
$1 prize (as depicted) or a 90¢ loss. (B) The reported confidence ratings in a win outcome (n = 14) are plotted against the objective win sector size, separately for trials on which original spinner choice was vetoed and accepted. Participants were more confident in a win outcome when their original spinner choice was accepted. Error bars on the green and red lines indicate SEM. Error bars on the columns indicate within-subject SEM. *p < .05. “vetoed”; the blue circle was then replaced by a yellow circle around one of the other two spinners. On the other half of the trials, the initial choice was accepted and re- mained highlighted; the blue circle around the selected spinner then turned yellow. Participants then indicated their level of confidence that the trial would result in a win outcome. For this purpose, a horizontal rectangle and the question, “What is your gut feeling?” appeared on screen. The rectangle functioned as a visual analogue scale to indicate confidence in the like- lihood of a win outcome, ranging from “sure loss” to “sure win.” Participants were instructed to click the horizontal point in the rectangle that best reflected their gut feeling. Finally, all spinners were again presented on screen and the arrow of the selected spinner revolved with de- celerating speed. If it ended on the win sector, a green “$” appeared on screen and the running total would in- crease by $1. Otherwise, a red “X” appeared on screen and the running total decreased by 90¢. A new set of spinners was then presented, beginning a new trial. At the end of the experiment, participants were paid their accumulated final money reward. Unannounced to participants, the trials comprised 24 instances of each possible combinations of spinner outcome and approval condition (i.e., win/vetoed, lose/ vetoed, win/accepted, and lose/accepted). To collect enough data for low probability outcomes, the proportion of wins and losses reflected a compressed probability space of those depicted by the size of the win sectors (see Table 1). The trial sequence randomly intermixed these trial types.2 Participants were offered a 1-min break after every 16 trials. Data Analysis We subjected the participantsʼ confidence ratings to a mixed-effects two-way repeated-measures ANOVA with within-subject factors for choice condition and win sector Table 1. Frequency of Different Trial Types in Both Experiments Outcome Approval Condition Win Sector Size Frequency Win Vetoed Accepted Lose Vetoed Accepted 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 3 5 7 9 3 5 7 9 9 7 5 3 9 7 5 3 854 Journal of Cognitive Neuroscience Volume 25, Number 6 size and the participants as the random factor. Statistical significance was evaluated at α = .05. One participantʼs confidence ratings were heavily nonmonotonic over win sector sizes and were excluded from the analyses. All effects remained significant with this participant included. Results and Discussion In this behavioral study, we aimed to confirm that our task gives rise to the standard IOC effect. Following the literature, we predicted that participants would provide higher confidence ratings on trials where the initial spin- ner choice was accepted than on trials where it was vetoed. Not surprisingly, we found that the confidence ratings were sensitive to win sector size, F(3, 39) = 163.08, p < .001, η2 = 0.93; as shown in Figure 1B, the ratings in- creased monotonically with win sector size (Bonferonni- corrected pairwise comparisons, all ps < .001). More importantly, there was a significant main effect of Approval Phase Outcome. Confidence ratings were significantly higher when participantsʼ original choice was accepted than when it was vetoed, F(1, 13) = 7.32, p < .05, η2 = 0.36, Cohenʼs d = 0.72. To quantify the magnitude of the IOC effect in subjec- tive probability, we proceeded as follows. For each partic- ipant, we ran a linear regression analysis to estimate the slope of the linear relationship between the size of the win sector and the reported confidence in a win out- come for trials in which the playerʼs choice was accepted. Next, we multiplied the multiplicative inverse of the aver- age slope, that is, the increase in win sector size for a one-unit increase in confidence, with the average differ- ence in reported confidence between the choice condi- tions. The size of this effect was 4.6% (SE = 1.7 ), equivalent to a 16.6° increase in win sector size.3 EXPERIMENT 2 Having confirmed that our spinner task yields the IOC, we used fMRI to measure regional brain activity during performance of the task. Methods Participants and Procedures Twenty-nine new participants from the Princeton Uni- versity community (18–32, 16 women) completed Experiment 2. The task used in Experiment 2 followed the sequence used in Experiment 1, except that the confidence rating step was omitted and that participants used a nonferro- magnetic joystick to indicate initial choice. At the begin- ning of each trial, participants were required to bring the joystick in its center position to avoid any bias to a given spinner based on proximity. The timing of task events was changed from Experi- ment 1 to facilitate estimation of the BOLD signal. The blue circle indicating the initial choice remained on screen 2.5 sec. The yellow circle indicating final choice was pres- ented for 2–4 sec. The arrowhead spun for 2–4 sec; the final outcome remained on screen 2 sec; and there was a 8-sec blank screen between trials. We used the same number of trials as in Experiment 1, randomly ordered over six scanning runs with equal num- bers of trials, which yielded an average of 1139 functional volumes per participant. Before scanning, participants completed a practice set of 16 trials. At the end of the scanning session, par- ticipants completed a questionnaire in which they were asked whether they believed the spinners were equiva- lent in terms of their likelihood of a win outcome. Two participants indicated that they came to believe there was a difference between the spinners in each display and were excluded from further analysis. All effects reported in the main text remained statistically signifi- cant when analyses were repeated with these partici- pants included. fMRI Acquisition and Preprocessing Scanning was conducted with a 3T Siemens Allegra scan- ner at Princeton University. The data were analyzed using AFNI (Cox, 1996) and Matlab. Each session began with a MPRAGE anatomical scan, consisting of 160 1-mm sagittal slices (repetition time = 2.5, echo time = 4.38 msec, flip angle = 8°, field of view = 256 mm). During each func- tional imaging block, an EPI sequence was used to obtain 34 contiguous 3-mm axial slices aligned to the AC–PC line with repetition time = 2 sec, echo time = 30 msec, flip angle = 90°, matrix = 64 × 64 voxels and field of view = 192 mm, yielding 3-mm isotropic voxels. Slice acquisition time correction was performed using Fourier interpolation, and images were motion corrected using a six-parameter rigid body transformation to co- register functional scans. Volumes that exhibited a large change in motion parameters or a spike in spatially aver- aged global signal were excluded from further analysis. The data were spatially smoothed until a 3-D 6-mm FWHM Gaussian kernel approximated the estimated spa- tial autocorrelation. Finally, we normalized the signal in each voxel to reflect percent change. Data Analysis For each participant separately, we analyzed each voxelʼs time course using a general linear model (GLM), with baseline regressors for zero- through third-order polyno- mial trends, motion parameters, and the averaged whole- brain signal time course. We used six covariates to model the following task events: (i) fixation point onset, (ii) time until centering of joystick, (iii) spinner presentation, Kool, Getz, and Botvinick 855 D o w n l o a d e d f r o m l l / / / / j t t f / i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 5 / 6 2 5 8 / 5 6 2 / 1 8 9 5 4 2 5 / 3 1 5 7 7 7 o 9 c 2 n 5 _ 4 a / _ j 0 o 0 c 3 n 6 9 _ a p _ d 0 0 b 3 y 6 g 9 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 i 2 3 e s / j / t . f u s e r o n 1 7 M a y 2 0 2 1 (iv) approval phase onset, (v) spinner rotation, and (vi) the moment of spin outcome. The mean-centered size of the win sector was entered into the GLM as parametric regressor at events (iii) and (vi). We modeled the choice condition of each trial with a mean-centered categorical regressor at events (iv) and (vi). Finally, we entered the specific outcome of the trial (win vs. lose) as a mean- centered categorical regressor at event (vi). Each of these covariates was convolved with a canonical hemodynamic response function. For each participant, the analysis yielded maps of param- eter estimates (beta values) for each aforementioned regressor. We spatially normalized these maps by warp- ing the participantʼs anatomical image to match a tem- plate in Talairach space (Talairach & Tournoux, 1988), using a 12-parameter affine and cosine transformation. Next, we applied this transformation to each participantʼs statistical maps. After spatial normalization, we tested the maps of these parameter estimates in group level t tests. The AFNI tool AlphaSim was used to determine a combi- nation of cluster size and p threshold that controlled whole-brain α to .05. In one additional GLM, we modeled signed and unsigned RPEs at the moment of spinner outcome, re- placing the original win vs. lose and win sector size re- gressors. The results of this GLM were used to test whether the areas responsive to the approval manipu- lation were reflective of unsigned RPEs. In a further GLM, we included four regressors to separately model the brainʼs response in each of the four conditions: win/accepted, win/vetoed, lose/accepted, and lose/ vetoed. Note that the experiment and analyses were designed to avoid collinearity between regressors. All regressors of interest were orthogonal at the outcome event and the results of our ROI analyses should therefore not be affected by artifacts of “double dipping” (Vul & Kanwisher, 2010). Results and Discussion The focus of analysis was on neural activity linked to spin outcomes. As explained in the Introduction, we pre- dicted that areas including the ventral striatum would re- flect an RPE signal at trial outcome and, critically, that this signal would be modulated by IOC. More specifically, we predicted that areas carrying an RPE signal would also show a main effect of Approval Phase Outcome, display- ing lower activity at spin outcome following choice acceptance than following veto events. To evaluate events at spin outcome, we first contrasted regional brain activity (BOLD signal) for win outcomes against activity for lose outcomes. In keeping with pre- vious studies (Schultz et al., 1997; Montague et al., 1996), we found strong bilateral activation in the BG, with peak activation in ventral striatum (Figure 2A; peaks xleft = 19, yleft = −5, zleft = −1 and xright = −17, yright = −5, zright = −1). For each participant individually, we used this win–lose contrast to construct an ROI in this area (thresholds at t = 2.5). Within the resulting ROI, activ- ity at spin outcome varied inversely with win sector size, t(26) = −6.03, p < .001, as would be expected of an RPE signal (Figure 2B). The ROI also showed phasic activity positively correlating with win sector size at the moment each set of spinners was first presented, t(26) = 2.07, p < .05, again consistent with an interpretation in terms of RPE signaling (Schultz et al., 1997). To our surprise, although ventral striatum encoded RPEs at two junctures in our task, signal in the same area at spin outcome displayed no sensitivity to the approval phase outcome, t(26) = −0.06, right-sided p = .52; VS activity at spin outcome was no greater on trials where D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 5 / 6 2 5 8 / 5 6 2 / 1 8 9 5 4 2 5 / 3 1 5 7 7 7 o 9 c 2 n 5 _ 4 a / _ j 0 o 0 c 3 n 6 9 _ a p _ d 0 0 b 3 y 6 g 9 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 i 2 3 e s / j / t . f u s e r o n 1 7 M a y 2 0 2 1 Figure 2. (A) Group analysis results of the Experiment 2. Activity in the BG, with peaks in ventral striatum, showed increased activity in response to a win outcome when compared with a lose outcome. We constructed our subject-wise ventral striatum ROIs based on this contrast. (B) Average parameter of the regressor coding for the effect of the win sector size, the approval condition, and unsigned PE for this ROI in ventral striatum. Our results indicate that striatum only encoded the size of the win sector at spinner outcome. 856 Journal of Cognitive Neuroscience Volume 25, Number 6 D o w n l o a d e d f r o m l l / / / / j f / t t i t . : / / h t t p : / D / o m w i n t o p a r d c e . d s f i r o l m v e h r c p h a d i i r r e . c c t . o m m / j e d o u c n o / c a n r a t r i t i c c l e e - p - d p d 2 f 5 / 6 2 5 8 / 5 6 2 / 1 8 9 5 4 2 5 / 3 1 5 7 7 7 o 9 c 2 n 5 _ 4 a / _ j 0 o 0 c 3 n 6 9 _ a p _ d 0 0 b 3 y 6 g 9 u . e p s t d o f n b 0 y 8 S M e I p T e m L i b b e r r a 2 r 0 i 2 3 e s / j f . / t u s e r o n 1 7 M a y 2 0 2 1 Figure 3. (A) Group analysis results of Experiment 2. Regions responsive to the difference between the outcomes of accepted spinners compared with vetoed spinners included ACC and PCC/precuneus. There were only regions that showed increased activation on accepted trials compared with vetoed trials. Whole-brain threshold at p < .005. (B) Average parameter of the regressor coding for unsigned RPEs for ROIs in ACC and PCC/precuneus. Our results indicate that ACC and PCC/precuneus encoded unsigned RPEs or salience signals. (C) Average parameter of the regressor coding for the approval condition (vetoed vs. accepted), separately for trials on which the spinner resulted in a win and a lose outcome for ROIs in ACC and PCC/precuneus. If these regions would have coded an unsigned version of the IOC, this regressor should have differed in sign, that is, in the amount of surprise, between trials involving win and lose outcomes (as depicted). We found, however, that the approval effect was positive for both types of trials and was not significantly different between them ( ps > .40) in both regions. Error bars indicate SEM.
*p < .05, **p < .01, ***p < .001. participantsʼ spinner choice had been vetoed than on trials where it had been accepted. Indeed, there was not even a numerical trend in this direction (average β = −0.0004, 95% confidence interval = −0.0173 − 0.0164; see Figure 2B). To interpret this null result, it is important to consider whether the experimental design carried sufficient power to detect an approval outcome effect had such an effect been present. As it turns out, the same data provided the opportunity to conduct just such a power analysis. Recall that our behavioral study had indicated that the mean impact of the IOC on outcome prediction was compa- rable to increasing win sector size by 4.6%. The fact that the ventral striatum ROI displayed a robust effect of win sector size allowed us to estimate the effect of such a win sector expansion on striatal activity. To do so, we ran new GLMs (100 iterations), which modeled the likelihood of a win outcome separately for two equally sized groups of trials. The trials in these groups were randomly selected with the only constraint that the difference between their average win sector sizes approximated 4.6%, the subjective addition to the win sector size in the case of an accepted spinner as cal- culated in Experiment 1. For each of these GLMs, we entered an additional regressor, contrasting the two sets, to calculate the effect size of a difference in the size of RPEs induced by a 4.6% difference in win sector size in the striatum ROIs. A post hoc power analysis based on the effect sizes in this set of 100 GLMs indicated a mean Cohenʼs d = 0.26. The aforementioned p value asso- ciated with the approval effect revealed that we had con- siderable power, 1 − β = 0.93,4 making it unlikely that the absence of an approval phase outcome effect re- flected a type II error. An exploratory whole-brain group analysis, again focused on spin outcome, corroborated this result, revealing no effect of approval phase outcome anywhere in the BG (Fig- ure 3A). The contrast did reveal an effect in several other areas, specifically in left and right posterior cingulate cortex/ precuneus (PCC/precuneus; 49 and 29 voxels, peaks xleft = 16, yleft = 73, zleft = 20 and xright = −14, yright = 61, zright = 14), ACC (25 voxels, peak x = −2, y = −11, z = 38), and middle temporal gyrus (34 voxels, peak x = 40, y = 76, z = 8). In all of these areas, activity at spin outcome was greater on trials where the participantʼs spinner choice had been accepted than when it had been vetoed. Importantly, the same whole-brain analysis and follow-up ROI analyses indicated that none of these Kool, Getz, and Botvinick 857 regions responded monotonically to win sector size, as would be expected from a region carrying an RPE signal ( ps > .25; Figure 3B). Instead, activity in both ACC and
PCC/precuneus (though not temporal gyrus) resembled
an “unsigned” RPE (Hayden, Heilbronner, Pearson, & Platt,
2011) or saliency signal (Litt, Plassmann, Shiv, & Rangel,
2011): Activity correlated positively with win sector size
on lose outcomes but correlated negatively on win out-
comes (t(26) = 3.08, p < .01 for ACC; t(26) = 2.58, p < .05 for PCC/precuneus; Figure 3B). Note that this latter result makes it difficult to interpret the approval phase out- come effect in these regions as reflecting a distortion of reward probability. Such an interpretation would require the approval effect, like the win sector effect, to differ in sign between trials involving win and lose outcomes (Fig- ure 3C): Under the IOC, lose outcomes should be more salient than win outcomes. However, the approval effect in each region was positive for both win and lose trials and did not differ significantly between the two ( ps >
.40; see Figure 3C).

Interestingly, although ventral striatum showed no ap-
proval effect at spin outcome, it did show such an effect
at the moment of choice approval itself. As shown in
Figure 4, VS along with ventromedial pFC (vmPFC),
thalamus, left caudate, PCC, and middle temporal gyrus
displayed greater activity during the approval phase
following choice acceptance than after veto (vmPFC:
415 voxels, peak x = 1, y = −38, z = −1; left ventral
striatum: 83 voxels, peak x = 22, y = 1, z = −4; bilateral
thalamus: 42 voxels, peak x = 1, y = 13, z = 8; left caudate:
32 voxels, x = 28, y = −2, z = 20; PCC: 26 voxels, x = 10,
y = 37, z = 35; middle temporal gyrus: 65 voxels, peaks
xleft = 49, yleft = 73, zleft = 29 and xright = −41, yright =
64, zright = 29). In addition, we found a broad cluster
of activation in occipital cortex and FEFs that displayed
lower activity following choice acceptance than after veto
(occipital cortex: 4279 voxels, peak x = −26, y = 82,

Figure 4. Group analysis results of Experiment 2. The figure depicts
the regions that showed differential response between vetoed and
accepted trials during the approval phase. We found clusters in
occipital cortex, parietal cortex, and FEFs that were significantly
increased in response to vetoed spinners. Clusters in vmPFC,
right VS, bilateral thalamus, and left caudate showed increased
activation to the acceptance of spinner choice compared with a
veto event. Whole-brain threshold at p < .005. z = 23; left FEF: 70 voxels, peak x = 37, y = 13, z = 47; right FEF: 108 voxels, peak x = −23, y = 7, z = 47). A final set of analyses tested whether the IOC manip- ulation might have affected activity in regions previously implicated in risk representation, including anterior insula (Preuschoff, Quartz, & Bossaerts, 2008) and posterior parietal cortex (Huettel, Stowe, Gordon, Warner, & Platt, 2006). Drawing ROIs from relevant studies, we tested for an effect of the choice manipulation both during the approval phase and at spin outcome. No clear effect was observed. Regions in anterior insula described by Preuschoff et al. (2008) were not responsive to the IOC manipulation at either juncture, ps > .24. A region in pos-
terior parietal cortex reported by Huettel et al. (2006)
showed a marginal IOC effect during the approval phase
( p = .06), but not during the spin outcome ( p = .26).

GENERAL DISCUSSION

The question of how the brain represents reward out-
come probabilities is central to research on decision-
making. Previous studies have revealed reward probability
representations in several brain regions, often reflecting
nonlinearities or distortions that have been inferred from
behavioral studies. A surprising prospect, arising from
such work, is that the brain may carry multiple represen-
tations of outcome probability that differ in form; prob-
ability representations in different brain structures may
reflect different distortions or may be immune to the dis-
tortions reflected in choice behavior (De Martino et al.,
2009).

In this study, we investigated the neural representation
of outcome probability in the setting of the IOC (Langer,
1975), a probability distortion effect whose neural cor-
relates have not been studied. In a behavioral experiment,
we demonstrated that the classic IOC effect emerged in a
novel gambling task: Participants assigned greater prob-
ability to positive outcomes for gambles that were chosen
than for equivalent gambles that were imposed. Taking
a version of the same task to fMRI, we tested a straight-
forward prediction. Under the IOC, positive outcomes
should be less surprising (and negative outcomes more
surprising) in chosen gambles. Neural RPEs, as routinely
observed for example in the ventral striatum, should there-
fore presumably be affected by IOC. In our experiment,
the specific prediction was for a main effect of choice
(approval phase outcome) on RPE magnitude when gam-
ble outcomes were presented.

The neuroimaging results unambiguously contradicted
this prediction. Although a strong RPE signal was present
in the ventral striatum, it was entirely unaffected by the
choice manipulation. Additional analyses indicated that
our experiment carried considerable power for detecting
an IOC effect, making it unlikely that the absence of an
effect reflected a type II error. In short, the results indi-
cate that RPE signals in the ventral striatum are immune
to the IOC.

858

Journal of Cognitive Neuroscience

Volume 25, Number 6

D
o
w
n
l
o
a
d
e
d

f
r
o
m

l

l

/

/

/

/
j

t
t

f
/

i
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
d
o
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
–
p
–
d
p
d
2
f
5
/
6
2
5
8
/
5
6
2
/
1
8
9
5
4
2
5
/
3
1
5
7
7
7
o
9
c
2
n
5
_
4
a
/
_
j
0
o
0
c
3
n
6
9
_
a
p
_
d
0
0
b
3
y
6
g
9
u
.
e
p
s
t
d
o
f
n
b
0
y
8
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
i
2
3
e
s

/
j

/

.

f

t

u
s
e
r

o
n

1
7

M
a
y

2
0
2
1

Our findings add to previous evidence suggesting that
veridical neural representations of reward probability can
exist alongside distorted representations manifesting
either at the neural level or in behavior. As reviewed
earlier, Tobler et al. (2008) reported probability repre-
sentations in ventral striatum that failed to reflect non-
linearities expressed both in choice behavior and in
neural representations in other anatomical regions. Simi-
larly, Jessup and OʼDoherty (2011) showed that striatal
RPEs were not affected by the so-called “gamblerʼs
fallacy,” another behaviorally expressed distortion of
objective probabilities. And Clark, Lawrence, Astley-Jones,
and Gray (2009), using a choice manipulation related to
our own, found that it magnified the effect of near-misses
both on subsequent gambling behavior and on outcome
responses in medial frontal cortex, but that it had no such
effect on striatal outcome responses.

As in the Jessup and OʼDoherty (2011) study, the central
finding in the present work involved a behavioral–neural
dissociation: We observed behavior directly expressive
of outcome probability distortion, in conjunction with
neural responses displaying no such distortion. Somewhat
surprisingly, especially given previous reports of neural–
neural dissociations (Clark et al., 2009; Tobler et al.,
2008), our fMRI data revealed no area with activity directly
parallelling the distorted probability judgments our partici-
pants offered in their behavioral responses. However, the
fMRI results did reveal two other effects arising from our
choice manipulation, each of which points to a novel
explanation for how the IOC may give rise to inflated prob-
ability judgments.

The first of these findings involved an effect of the IOC
on regional activation during the choice acceptance pe-
riod: Greater activation was seen in ventral striatum
and vmPFC in response to choice acceptance than veto
events. This is consistent with recent data on the “value
of choice.” A number of studies have indicated that the
freedom to choose among response alternatives is asso-
ciated with intrinsic value, and consistent with this, the
opportunity to choose has been shown to trigger ventral
striatal activation (Leotti & Delgado, 2011; Leotti, Iyengar,
& Ochsner, 2010; Bown, Read, & Summers, 2003). A re-
lated set of studies has suggested, additionally, that posi-
tive affect resulting from free choice (as well as from
other sources) can engender optimistic predictions
about future events (Isen & Geva, 1987; Isen & Patrick,
1983; Langer & Rodin, 1976). Putting these findings
together, the present results are consistent with an inter-
pretation of the IOC according to which the value of
choice—reflected in striatal and vmPFC activity—triggers
affective changes, which in turn translate into inflated
estimates of reward probability.

The second IOC-related effect observed in our data
pertained to the spin outcome period, where we ob-
served greater activation in ACC and PCC/precuneus fol-
lowing choice approval than following veto. RPE signals
were also detected in both of these regions, making it

tempting to interpret the choice approval effect as a
modulation of the RPE along the lines we had originally
predicted. However, the RPE signals in both ACC and
PCC/precuneus (unlike ventral striatum) took an “un-
signed” form. That is, signal varied inversely with outcome
probability for both gains and losses. This pattern, which
has been reported in several other studies of ACC func-
tioning (Browning & Harmer, 2012; Hayden et al., 2011;
Litt et al., 2011), makes it difficult to interpret the choice
approval effect as reflecting a distortion of reward prob-
ability concordant with the IOC, because the latter would
predict differential effects on gains and losses. An inter-
pretation that fits better with the data would be in terms
of outcome saliency (see also Litt et al., 2011): The effect
of the IOC could be interpreted as an enhancement of
outcome saliency in the case of chosen gambles. Given
that outcomes for chosen gambles may plausibly have
been considered more self-relevant, it is also interesting
to note that paired ACC–PCC activation has been reported
in a number of studies involving self-referential process-
ing ( Johnson et al., 2002, 2006; Ochsner et al., 2005;
Fossati et al., 2003).

In summary, our findings point to the possibility that
the IOC, although it impacts probability judgments,
may take root outside the probability domain, arising
instead from either affective or self-referential process-
ing. Further research will be necessary to evaluate these
possibilities and to flesh out the neural mechanisms by
which each might translate into distorted probability
judgments. What the present results do show is that,
whatever factors underlie the IOC, they do not uniformly
affect all neural representations of outcome probability.
Consistent with the picture emerging from other recent
studies, the IOC appears to give rise to a situation in
which distorted probability estimates coexist with sepa-
rate representations that are resistant to distortion.

Acknowledgments
This work was supported by Collaborative Activity Award from
the James S. McDonnell Foundation to M. M. B. We thank
Daniel M. Oppenheimer and Andrew R. Conway for useful dis-
cussions about the experimental design and statistical analyses.

Reprint requests should be sent to Wouter Kool, Department of
Psychology, Green Hall, Princeton University, Princeton, NJ
08540, or via e-mail: wkool@princeton.edu.

Notes

1. The IOC is understood more broadly to cover situations
where chance situations carry superficial
features that are
associated with control over outcomes. Such features include
choice, perceived competition, familiarity, the need for control,
and mood. Our focus in this study was exclusively on effects of
choice.
2. Recent behavioral work on decision-making has highlighted
the potentially important distinction between stated and ex-
perienced probabilities (Hertwig & Erev, 2009). Given this, it is
worth noting that, within our task, outcome probabilities were

Kool, Getz, and Botvinick

859

D
o
w
n
l
o
a
d
e
d

f
r
o
m

l

l

/

/

/

/
j

t
t

f
/

i
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
d
o
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
–
p
–
d
p
d
2
f
5
/
6
2
5
8
/
5
6
2
/
1
8
9
5
4
2
5
/
3
1
5
7
7
7
o
9
c
2
n
5
_
4
a
/
_
j
0
o
0
c
3
n
6
9
_
a
p
_
d
0
0
b
3
y
6
g
9
u
.
e
p
s
t
d
o
f
n
b
0
y
8
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
i
2
3
e
s

/
j

/

f

t

.

u
s
e
r

o
n

1
7

M
a
y

2
0
2
1

both explicitly communicated (via the win sector size) and
directly experienced. The task was thus not designed to dis-
tinguish between these two potentially very different sources
of probability information.
3. Alternatively, one can compute the subjective magnitude of
the IOC effect for each participant separately and then average
over these scores. This leads to an IOC effect equivalent to 5.8%
(SE = 2.2) or a 21° increase in win sector size. To err on the
conservative side, our main analysis focused on the smaller
value reported in the main text.
4. A second post hoc power analysis, based on the alternative
increase in win sector size, described in Footnote 2, yielded a
mean Cohenʼs d = 0.32 and a power of 1 − β = 0.96 to detect a
difference in striatal response to the IOC.

REFERENCES

Abler, B., Walter, H., Erk, S., Kammerer, H., & Spitzer, M.
(2006). Prediction error as a linear function of reward
probability is coded in human nucleus accumbens.
Neuroimage, 31, 790–795.

Allais, P. M. (1953). Le comportement de lʼhomme rationnel

devant le risque: Critique des postulats et axiomes de lʼécole
Américaine. Econometrica: Journal of the Econometric
Society, 21, 503–546.

Bernoulli, D. (1954). Exposition of a new theory on the

measurement of risk. Econometrica, 22, 23–36.

Berns, G. S., Capra, C. M., Chappelow, J., Moore, S., & Noussair, C.
(2008). Nonlinear neurobiological probability weighting
functions for aversive outcomes. Neuroimage, 39,
2047–2057.

Bown, N. J., Read, D., & Summers, B. (2003). The lure of
choice. Journal of Behavioral Decision Making, 16,
297–308.

Brainard, D. H. (1997). The psychophysics toolbox. Spatial

Vision, 10, 433–436.

Browning, M., & Harmer, C. J. (2012). Expectancy and surprise

predict neural and behavioral measures of attention to
threatening stimuli. Neuroimage, 59, 1942–1948.

Clark, L., Lawrence, A. J., Astley-Jones, F., & Gray, N. (2009).
Gambling near-misses enhance motivation to gamble and
recruit win-related brain circuitry. Neuron, 61, 481–490.
Cox, R. (1996). AFNI: Software for analysis and visualization of
functional magnetic resonance neuroimages. Computers and
Biomedical Research, 29, 162–173.

Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based
competition between prefrontal and dorsolateral striatal
systems for behavioral control. Nature Neuroscience, 8,
1704–1711.

De Martino, B., Kumaran, D., Holt, B., & Dolan, R. J. (2009). The
neurobiology of reference-dependent value computation.
The Journal of Neuroscience, 29, 3833–3842.

Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete

coding of reward probability and uncertainty by dopamine
neurons. Science, 299, 1898–1902.

FitzGerald, T. H. B., Seymour, B., Bach, D. R., & Dolan, R. J.
(2010). Differentiable neural substrates for learned and
described value and risk. Current Biology, 20, 1823–1829.
Fossati, P., Hevenor, S. J., Graham, S. J., Grady, C., Keightley, M. L.,

Craik, F., et al. (2003). In search of the emotional self: An
fMRI study using positive and negative emotional words.
American Journal of Psychiatry, 160, 1938–1945.

Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L.
(2011). Surprise signals in anterior cingulate cortex: Neuronal
encoding of unsigned reward prediction errors driving
adjustment in behavior. Journal of Neuroscience, 31,
4178–4187.

Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004).

Decisions from experience and the effect of rare events
in risky choice. Psychological Science, 15, 534–539.

Hertwig, R., & Erev, I. (2009). The description-experience gap
in risky choice. Trends in Cognitive Sciences, 13, 517–523.
Hsu, M., Krajbich, I., Zhao, C., & Camerer, C. F. (2009). Neural
response to reward anticipation under risk is nonlinear in
probabilities. Journal of Neuroscience, 29, 2231–2237.
Huettel, S., Stowe, C., Gordon, E., Warner, B., & Platt, M.

(2006). Neural signatures of economic preferences for risk
and ambiguity. Neuron, 49, 765–775.

Hutcherson, C. A., Plassmann, H., Gross, J. J., & Rangel, A.

(2012). Cognitive regulation during decision-making shifts
behavioral control between ventromedial and dorsolateral
prefrontal value systems. The Journal of Neuroscience,
32, 13543–13554.

Isen, A. M., & Geva, N. (1987). The influence of positive affect
on acceptable level of risk: The person with a large canoe has
a large worry. Organizational Behavior and Human
Decision Processes, 39, 145–154.

Isen, A. M., & Patrick, R. (1983). The effect of positive feelings
on risk taking: When the chips are down. Organizational
Behavior and Human Performance, 31, 194–202.

Jessup, R. K., & OʼDoherty, J. P. (2011). Human dorsal striatal
activity during choice discriminates reinforcement learning
behavior from the gamblerʼs fallacy. Journal of Neuroscience,
31, 6296–6304.

Johnson, M. K., Raye, C. L., Mitchell, K. J., Touryan, S. R., Green,
E. J., & Nolen-Hoeksema, S. (2006). Dissociating medial
frontal and posterior cingulate activity during self-reflection.
Social Cognitive and Affective Neuroscience, 1, 56–64.
Johnson, S. C., Baxter, L. C., Wilder, L. S., Pipe, J. G., Heiserman,

J. E., & Prigatano, G. P. (2002). Neural correlates of
self-reflection. Brain, 125, 1808–1814.

Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1990).

Experimental tests of the endowment effect and the
Coase theorem. The Journal of Political Economy, 98,
1325–1348.

Kahneman, D., & Tversky, A. (1979). Prospect theory: An

analysis of decision under risk. Econometrica: Journal of
the Econometric Society, 47, 263–292.

Kobayashi, S., Lauwereyns, J., Koizumi, M., Sakagami, M., &
Hikosaka, O. (2002). Influence of reward expectation on
visuospatial processing in macaque lateral prefrontal cortex.
Journal of Neurophysiology, 87, 1488–1498.

Langer, E. J. (1975). The illusion of control. Journal of
Personality and Social Psychology, 32, 311–328.

Langer, E. J., & Rodin, J. (1976). The effects of choice and
enhanced personal responsibility for the aged: A field
experiment in an institutional setting. Journal of Personality
and Social Psychology, 34, 191–198.

Leotti, L. A., & Delgado, M. (2011). The inherent reward of

choice. Psychological Science, 22, 1310–1318.

Leotti, L. A., Iyengar, S. S., & Ochsner, K. N. (2010). Born to
choose: The origins and value of the need for control.
Trends in Cognitive Sciences, 14, 457–463.

Litt, A., Plassmann, H., Shiv, B., & Rangel, A. (2011).
Dissociating valuation and saliency signals during
decision-making. Cerebral Cortex, 21, 95–102.

McClure, S. M., Ericson, K. M., Laibson, D. I., Loewenstein, G., &
Cohen, J. D. (2007). Time discounting for primary rewards.
Journal of Neuroscience, 27, 5796.

Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A

framework for mesencephalic dopamine systems based on
predictive Hebbian learning. The Journal of Neuroscience,
16, 1936–1947.

Ochsner, K. N., Beer, J. S., Robertson, E. R., Cooper, J. C.,

Gabrieli, J. D. E., Kihlstrom, J. F., et al. (2005). The neural

D
o
w
n
l
o
a
d
e
d

f
r
o
m

l

l

/

/

/

/
j

f
/

t
t

i
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
d
o
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
–
p
–
d
p
d
2
f
5
/
6
2
5
8
/
5
6
2
/
1
8
9
5
4
2
5
/
3
1
5
7
7
7
o
9
c
2
n
5
_
4
a
/
_
j
0
o
0
c
3
n
6
9
_
a
p
_
d
0
0
b
3
y
6
g
9
u
.
e
p
s
t
d
o
f
n
b
0
y
8
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
i
2
3
e
s

/
j

f

.

t

/

u
s
e
r

o
n

1
7

M
a
y

2
0
2
1

860

Journal of Cognitive Neuroscience

Volume 25, Number 6

correlates of direct and reflected self-knowledge.
Neuroimage, 28, 797–814.

Padoa-Schioppa, C., & Assad, J. A. (2006). Neurons in the

orbitofrontal cortex encode economic value. Nature, 441,
223–226.

Paulus, M. P., & Frank, L. R. (2006). Anterior cingulate activity
modulates nonlinear decision weight function of uncertain
prospects. Neuroimage, 30, 668–677.

substrate of prediction and reward. Science, 275,
1593–1599.

Simon, D. A., & Daw, N. D. (2011). Neural correlates of forward
planning in a spatial decision task in humans. Journal of
Neuroscience, 31, 5526–5539.

Starmer, C. (2000). Developments in non-expected utility

theory: The hunt for a descriptive theory of choice under
risk. Journal of Economic Literature, 38, 332–382.

Pelli, D. G. (1997). The VideoToolbox software for visual

Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An

psychophysics: Transforming numbers into movies. Spatial
Vision, 10, 437–442.

Platt, M. L., & Glimcher, P. W. (1999). Neural correlates of

decision variables in parietal cortex. Nature, 400, 233–238.
Presson, P. K., & Benassi, V. A. (1996). Illusion of control: A
meta-analytic review. Journal of Social Behavior and
Personality, 11, 493–510.

Preuschoff, K., Quartz, S., & Bossaerts, P. (2008). Human insula
activation reflects risk prediction errors as well as risk. The
Journal of Neuroscience, 28, 2745–2752.

Rangel, A., & Hare, T. (2010). Neural computations associated
with goal-directed choice. Current Opinion in Neurobiology,
20, 262–270.

Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian
conditioning: Variations in the effectiveness of reinforcement
and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.),
Classical conditioning: II. Current research and theory
(pp. 64–99). New York: Appleton-Century-Crofts.
Roesch, M. R., Singh, T., Brown, P. L., Mullins, S. E., &

Schoenbaum, G. (2009). Ventral striatal neurons encode
the value of the chosen action in rats deciding between
differently delayed or sized rewards. Journal of Neuroscience,
29, 13365–13376.

Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural

introduction. Cambridge, MA: MIT Press.

Talairach, J., & Tournoux, P. (1988). Co-planar stereotaxic
atlas of the human brain. New York: Theme Medical
Publishers.

Thompson, S. Z., Armstrong, W., & Thomas, C. (1998).

Illusions of control, underestimations, and accuracy: A
control heuristic explanation. Psychological Bulletin,
123, 143–161.

Tobler, P. N., Christopoulos, G. I., OʼDoherty, J. P., Dolan, R. J.,

& Schultz, W. (2008). Neuronal distortions of reward
probability without choice. Journal of Neuroscience, 28,
11703–11711.

Trepel, C., Fox, C. R., & Poldrack, R. A. (2005). Prospect theory
on the brain? Toward a cognitive neuroscience of decision
under risk. Cognitive Brain Research, 23, 34–50.

Tversky, A., & Kahneman, D. (1992). Advances in prospect

theory: Cumulative representation of uncertainty. Journal of
Risk and Uncertainty, 5, 297–323.

von Neumann, J., & Morgenstern, O. (1944). Theory of games and
economic behavior. Princeton, NJ: Princeton University Press.

Vul, E., & Kanwisher, N. (2010). Begging the question: The

non-independence error in fMRI data analysis. In S. J. Hanson
& M. Bunzl (Eds.), Foundational Issues in Human Brain
Mapping (pp. 71–91). Cambridge, MA: MIT Press.

D
o
w
n
l
o
a
d
e
d

f
r
o
m

l

l

/

/

/

/
j

f
/

t
t

i
t
.

:
/
/

h
t
t
p
:
/
D
/
o
m
w
i
n
t
o
p
a
r
d
c
e
.
d
s
f
i
r
o
l
m
v
e
h
r
c
p
h
a
d
i
i
r
r
e
.
c
c
t
.
o
m
m
/
j
e
d
o
u
c
n
o
/
c
a
n
r
a
t
r
i
t
i
c
c
l
e
e
–
p
–
d
p
d
2
f
5
/
6
2
5
8
/
5
6
2
/
1
8
9
5
4
2
5
/
3
1
5
7
7
7
o
9
c
2
n
5
_
4
a
/
_
j
0
o
0
c
3
n
6
9
_
a
p
_
d
0
0
b
3
y
6
g
9
u
.
e
p
s
t
d
o
f
n
b
0
y
8
S
M
e
I
p
T
e
m
L
i
b
b
e
r
r
a
2
r
0
i
2
3
e
s

/
j

t

.

/

f

u
s
e
r

o
n

1
7

M
a
y

2
0
2
1

Kool, Getz, and Botvinick

861

Download pdf