The Case of Pretense: Observing Actions - Specialized Research AI at MIT

The Case of Pretense: Observing Actions
and Inferring Goals

Ricarda I. Schubotz1 and D. Yves von Cramon2

Abstract

& When we observe an action, we know almost immediately
what goal is pursued by the actor. Strikingly, this applies also to
pretend action (pantomime), which provides relevant informa-
tion about the manipulation itself but not about the manipulated
objects. The present fMRI study addressed the issue of goal
inference from pretend action as compared with real action. We
found differences as well as commonalities for the brain corre-
lates of inferring goals from both types of action. They differed
with regard to the weights of the underlying action observation
network, indicating the exploitation of object information in the

case of real actions and manipulation information in the case of
pretense. However, goal inferences from manipulation informa-
tion resulted in a common network for both real and pretend
action. Interestingly, this latter network also comprised areas that
are not identified by action observation and that might be due
to the processing of scene gist and to the evaluation of fit of
putative action goals. These findings suggest that observation of
pretense emphasizes the requirement to internally simulate the
observed act but rule out fundamental differences of how ob-
servers cope with real and pretend action. &

INTRODUCTION

When we witness a pretend action, that is, pantomime,
we have an immediate idea about what goal is pursued
by the actor. How is this achieved? Although several im-
aging studies have investigated the performance of pan-
tomime (Hermsdo¨rfer, Terlinden, Mu¨hlau, Goldenberg,
& Wohlschla¨ger, 2007; Imazu, Sugio, Tanaka, & Inui,
2007; Kro´liczak, Cavina-Pratesi, Goodman, & Culham,
2007; Buxbaum, Kyle, & Menon, 2005; Ohgami, Matsuo,
Uchida, & Nakai, 2004; Moll et al., 2000) and the obser-
vation of real action (e.g., Newman-Norlund, van Schie,
van Zuijlen, & Bekkering, 2007; Calvo-Merino, Gre`zes,
Glaser, Passingham, & Haggard, 2006; Costantini et al.,
2005; Schubotz & von Cramon, 2004; Johnson-Frey et al.,
2003; Manthey, Schubotz, & von Cramon, 2003; Buccino
et al., 2001; for a topical review, cf. Vogt & Thomaschke,
2007), we are still ignorant about the neural correlates
of understanding goals in observed pantomime. The
only study implementing observation of pretend action
(German, Niehaus, Roarty, Giesbrecht, & Miller, 2004)
used a covert instruction, that is, participants were
not directed to attend to the actors’ intention or goal
but rather to the occurrence of a screen interrupting
the action.

In the present study, we tested the hypothesis that
the brain correlates of inferring goals from pretend and

1Max Planck Institute for Neurological Research, Cologne, Ger-
many, 2Max Planck Institute for Human Cognitive and Brain
Sciences, Leipzig, Germany

real action are partly comparable, as specified in more
detail in the following. Apparently there are good rea-
sons to assume differences as well as commonalities for
the brain correlates of inferring goals from pretend and
real action, respectively.

On the one hand, although we are able to tell apart
real from pretend action, it is not plausible to assume
that neural processes subserving the interpretation of
pretend actions should be fundamentally different from
those subserving the interpretation of real actions. That
is not only because both entail a multitude of percep-
tual, mnemonic, and cognitive processes that are trig-
gered by a complex and socially relevant stimulus, but
more specifically because both crucially entail, under
natural conditions, the intense analysis of the hands’
posture and movements, that is, manipulation infor-
mation. Models from motor control theory have been
recently used to describe how we analyze observed
actions to infer our conspecifics’ goals (Grush, 2004;
Miall, 2003; Wolpert & Flanagan, 2001). These models,
designed to describe how we continuously adapt our
movements to changing environmental conditions and
on-line error correction, state that multiple forward
models are set up to predict upcoming events from an
unfolding action, no matter whether performed by our-
selves or merely observed. Accordingly, the processes
underlying goal
inference computationally amount to
the running of a simulation of several action scripts in
parallel until the best fitting script wins. For instance,
observing an actor grasping a cup, we predict him to
either bring it to his mouth, or clean it, or move it onto

D 2008 Massachusetts Institute of Technology

Journal of Cognitive Neuroscience 21:4, pp. 642–653

D
o
w
n
l
o
a
d
e
d

/
j

f
/

t
t

i
t
.

:
/
/

f
r
o
m
D
h
o
t
w
t
n
p
o
:
a
/
d
/
e
m
d
i
f
t
r
o
p
m
r
c
h
.
s
p
i
l
d
v
i
e
r
e
r
c
c
t
.
h
m
a
i
r
e
.
d
u
c
o
o
m
c
/
n
j
a
o
r
c
t
i
n
c
/
e
a
–
p
r
d
t
i
2
c
1
l
4
e
–
6
p
4
d
2
f
1
/
9
2
3
1
7
/
5
4
0
/
2
6
o
4
c
2
n
/
1
2
0
7
0
6
9
0
0
2
0
1
7
0
4
/
9
j
o
p
c
d
n
.
b
2
y
0
g
0
u
9
e
.
s
t
2
o
1
n
0
4
0
9
8
.
S
p
e
d
p
f
e
m
b
y
b
e
g
r
u
2
e
0
2
s
3
t

/
j

o
n

1
8

M
a
y

2
0
2
1

a shelf, or pass it to someone else, and so on. Testing
hypotheses about currently valid goal options, no matter
whether based on real or pretend actions, thus calls
for sensorimotor transformation (for internal simulation
of action scripts), working memory (for the selection of
currently valid scripts), and internal reward evaluation
(for the motivational driving of the ongoing estimation
of script fit and incremental reduction of currently
tested goal options).

On the other hand, although the analysis of observed
action no matter whether real or pretend entails testing
hypotheses about currently valid goals, these hypothe-
ses are derived from at least partially different sources
for real and pretend action. As object information can-
not be exploited to infer the goal from pretend action,
one would expect that components of the action obser-
vation network that are engaged in the analysis of hand
postures and motions, that is, manipulation, should be
particularly enhanced in pretend action. Conversely,
those engaged in the processing of object information
should be more active for real actions. Even when iso-
lated, both sources of information, manipulations and
objects, are known to provide excellent hints for action
goals, as demonstrated by the early emergence of pre-
tend or symbolic play in child development on the one
hand (Fein, 1981) and experimental
investigations of
object affordance on the other hand (Helbig, Graf, &
Kiefer, 2006).

The present fMRI study addressed the issue of goal
inference from pretend action as compared with real
action. To this end, we presented short video clips that
showed either pretend or real actions (two-level factor,
Type). We expected both pretend and real action to en-
gage the action observation network reported in the
literature, comprising, among others, the ventral pre-
motor cortex (PMv), the anterior intraparietal sulcus
(aIPS), and the posterior superior temporal sulcus (pSTS)
(Rizzolatti & Craighero, 2004). However, due to the
different significance of manipulation and object informa-
tion in the analysis of pretend and real action, respec-
tively, we expected higher signals in the extrastriate body
area (EBA; Taylor, Wiggett, & Downing, 2007), the hu-
man motion-selective area (hMT; Greenlee, 2000; cf. also
Peuskens, Vanrie, Verfaillie, & Orban, 2005), and the pSTS
(Puce & Perrett, 2003) for pretend as compared with real
action. Conversely, the lateral occipital complex (LOC;
Grill-Spector, Kourtzi, & Kanwisher, 2001) was expected
to be elevated for real as compared with pretend action.
Although this action observation network was expected
to be weighted differently for pretend and real actions,
the network reflects diverse perceptual, mnemonic, and
cognitive processes not all of which necessarily contribute
to goal inference. For instance, the attentive analysis of
the observed action is expected to continue even after
the goal has been successfully recognized. Therefore, in
an attempt to exclusively tap goal inference processes, we
implemented a switching protocol. The rationale of this

protocol was related to the so-called repetition attenua-
tion or suppression effect. Repetition suppression refers
to the fact that the repetition of a stimulus leads to a de-
creased BOLD signal in areas that encode that stimulus
(Hamilton & Grafton, 2006; Grill-Spector & Malach, 2001;
Naccache & Dehaene, 2001; Thompson-Schill, D’Esposito,
& Kan, 1999). Here we contrasted trials with new infor-
mation (switch trials, hereafter) with those containing no
new information (repetition trials, hereafter), relative to
the preceding trial. Areas that are engaged in processing
a particular type of information should be more engaged
in switch trials than in repetition trials.

We used three types of trials (the three-level factor,
Switch): (a) ‘‘goal switch’’ trials (G) in that both the
manipulations and the objects of the presented action
in the current trial n differed from those in the preced-
ing trial n (cid:1) 1; (b) ‘‘object switch’’ trials (O) in that only
the object of the presented action in trial n differed from
the object used in the preceding trial n (cid:1) 1 (while the
manipulations were repeated in either case); and (c)
‘‘manipulation switch’’ trials (M) in that only the manip-
ulations of the presented action in trial n differed from
the manipulation in the preceding trial n (cid:1) 1 (while the
objects in use were repeated in either case). By con-
trasting goal switch trials with object switch trials (G >
O), we aimed to identify brain areas that contribute
inference on the basis of manipulation infor-
to goal
mation. Note that because G trials provided both new
manipulation and new object information, the contrasts
G > O (and G > M, see below) did not identify a relative
difference between two different types of information
but rather the relative difference between new and old
(=repeated) information of the same type.

Note that we consider this contrast to highlight goal
inference processes according to the experimental op-
erationalization of goal inference that we choose in the
present study. Particularly, we took the approach that
goal inference amounts to a set of different cognitive
subprocesses, not to a moment of unitary aha experi-
ence. Thus, for inferring goals on the basis of manipu-
lation information, no matter whether from real or from
pretend actions, we expected PMv, aIPS, and adjacent
supramarginal gyrus (SMG) as areas relevant for sen-
sorimotor transformation (Rizzolatti & Luppino, 2001),
lateral prefrontal cortex (lPFC) subserving working mem-
ory in adaptive goal-directed behavior (Watanabe, 2007;
Petrides, 2005), and OFC as an area known to be engaged
in reward evaluation (Wallis, 2007).

Manipulation switch (M) trials were employed to
balance the probability of novel information being pro-
vided either by objects or by manipulations. Object in-
formation, in contrast to manipulation information, was
expected to be exploited only in real actions and largely
ignored in pretend actions. Therefore, we expected
the contrast G > M that reflected goal inference on
the basis of object information to yield no common ac-
tivations for real and pretend actions.

Schubotz and von Cramon

643

D
o
w
n
l
o
a
d
e
d

/
j

t
t

f
/

i
t
.

:
/
/

/
j

o
n

1
8

M
a
y

2
0
2
1

METHODS

Participants

Eighteen right-handed, healthy volunteers (eight women;
age range = 21–32 years; mean age = 26.4 years) par-
ticipated in the study. After being informed about poten-
tial risks and screened by a physician of the institution,
subjects gave informed consent before participating.
The experimental standards were approved by the local
ethics committee of the University of Leipzig. Data were
handled anonymously.

Stimuli and Tasks

Subjects were presented with movies showing actions
and with short verbal action descriptions referring to
these actions. Each trial (6 sec) started with a movie
(2 sec) followed by a fixation phase. The length of the
fixation phase (2.5–4 sec) depended on the variable jit-
ter times (0, 500, 1000, or 1500 msec) that were inserted
before the movie to enhance the temporal resolution of
the BOLD response. Actions were either performed on
appropriate objects (e.g., pouring water from a bottle
into a glass) or on inappropriate objects (e.g., making
the same movements with a bin and a key). These two
classes of actions will hereafter be referred to as ‘‘real

actions’’ and ‘‘pretend actions,’’ respectively. Note that
to generate rich informational content from both ma-
nipulations and objects, each movie clip we presented
showed a chain of specific manipulations (e.g., grasping,
turning, and opening) and combinations of two objects
(e.g., a cup and a spoon).

Subjects were instructed to attend to the presented
movies. They were informed that some of the movies
were followed by a trial that started with an action de-
scription that either matched or did not match the con-
tent of the preceding movie. It was emphasized that it
did not play any role whether actions to that the action
description referred to were real or pretend actions.
In case a trial containing an action description was pre-
sented, subjects immediately delivered their responses
on a two-button response box using their index finger
for affirmative responses and their middle finger for re-
jections. Fifty percent of the action descriptions were to
be affirmed and 50% to be rejected.

In addition to the two-level stimulus factor Type [real
action (R) and pretend action (P)], a three-level factor
Switch [goal (G), objects (O), and manipulations (M)]
was implemented. The trial succession was implemented
such that trials were either switch trials or repetition trials
with respect to the manipulations, to the physical objects
in use, or both (see Figure 1). All combinations of these

D
o
w
n
l
o
a
d
e
d

/
j

t
t

f
/

i
t
.

:
/
/

/
j

o
n

1
8

M
a
y

2
0
2
1

Figure 1. Experimental design. Examples for goals (e.g., ‘‘writing with pen’’) are given in colored boxes, in which photos indicate the physical
object actually presented in the corresponding movie clip. Levels of the experimental factor Switch (object, manipulation, and goal) correspond
to columns, with columns 1–3 (solid frames) and 4–6 (dashed frames) corresponding to the two levels of the experimental factor Type (real and
pretend). The first row of boxes represents the goal in a trial n (cid:1) 1, whereas the residual rows represent examples of actions, in trial n, either
repeating (second row) or switching (third row) the manipulated objects (first and fourth column), manipulations (second and fifth column), or goals
(third and sixth column). The fields for goal repetition are empty because this trial type was not part of the experimental design.

644

Journal of Cognitive Neuroscience

Volume 21, Number 4

two factors were possible except an identical repetition
of trial n (cid:1) 1. Moreover, the transition frequencies of
real and pretend action were counterbalanced. Twenty-
five percent of the movies (i.e., 21 of 84 real actions and
21 of 84 pretend actions) were followed by an action de-
scription that had the length of a regular trial (2 sec de-
scription, including response phase, plus 4 sec fixation
phase), resulting in 42 additional trials. Each action de-
scription was followed by a dummy trial that was a regular
movie of either a real or a pretend action but neither a
regular switch nor a repetition trial. Accordingly, these
dummy trials (n = 42) entered the analysis contrast-
ing real and pretend actions (adding up to 84 + 21 =
105 trials for real actions and 84 + 21 = 105 trials for
pretend actions) but not the analyses on switch or repeti-
tion effects of manipulations and objects. Finally, 20 empty
trials (resting state) were presented intermixed with the
experimental trials.

Altogether, 272 trials were presented: 84 real actions
plus 21 real action dummies, 84 pretend actions plus
21 pretend action dummies, 21 action descriptions fol-
lowing real actions, 21 action descriptions following pre-
tend actions, and 20 empty trials.

MRI Data Acquisition

Imaging was carried out on a 3-T Bruker (Ettlingen,
Germany) Medspec 30/100 system equipped with the
standard birdcage head coil. Participants were placed
on the scanner bed in a supine position with their right
index and middle fingers positioned on the appropriate
response buttons of a response box. Form-fitting cush-
ions were utilized to prevent head, arm, and hand
movements. Participants were provided earplugs so that
scanner noise would be attenuated. Twenty-two axial
slices (192 mm field of view; 64 (cid:2) 64 pixel matrix; 4 mm
thickness; 1 mm spacing; in-plane resolution of 3 (cid:2)
3 mm) parallel to bicommissural line (AC–PC) covering
the whole brain were acquired using a single-shot gradi-
ent EPI sequence (2000 msec repetition time; 30 msec
echo time; 908 flip angle; 100 kHz acquisition bandwidth)
sensitive to BOLD contrast. Prior to the functional imag-
ing, 22 anatomical T1-weighted MDEFT images (Norris,
2000; Ugurbil et al., 1993) and 22 T1-weighted EPI
images with the same spatial orientation as the func-
tional data were acquired. In a separate session, high-
resolution whole-brain images were acquired from each
subject to improve the localization of activation foci
using a T1-weighted 3-D-segmented MDEFT sequence
covering the whole brain.

MRI Data Analysis

Data were processed using the software package LIPSIA
(Lohmann et al., 2001). Functional data were first
motion-corrected using a matching metric based on lin-
ear correlation. To correct for the temporal offset be-

tween the slices acquired in one image, a cubic-spline
interpolation was employed. Low-frequency signal
changes and baseline drifts were removed using a tem-
poral high-pass filter with a cutoff frequency of 1/85 Hz.
Spatial smoothing was performed with a Gaussian filter of
5.65 mm FWHM. To align the functional data slices with
a 3-D stereotactic coordinate reference system, a rigid lin-
ear registration with six degrees of freedom (three rota-
tional, three translational) was performed. The rotational
and the translational parameters were acquired on the
basis of the MDEFT and the EPI-T1 slices to achieve an
optimal match between these slices and the individual
3-D reference dataset. The MDEFT volume dataset with
160 slices and 1-mm slice thickness was standardized to
the Talairach stereotactic space (Talairach & Tournoux,
1988). The rotational and the translational parameters
were subsequently transformed by linear scaling to a stan-
dard size. The resulting parameters were then used to
transform the functional slices using trilinear interpolation,
so that the resulting functional slices were aligned with
the stereotactic coordinate system, thus generating output
data with a spatial resolution of 3 (cid:2) 3 (cid:2) 3 mm (27 mm3).
The statistical evaluation was based on a least-squares
estimation using the general
linear model for serially
autocorrelated observations (Friston et al., 1995; Worsley
& Friston, 1995). The design matrix was generated with
a box-car function, convolved with the hemodynamic
response function and its first derivative. Brain activations
were analyzed time-locked to onset of the movies, and
the analyzed epoch comprised the full duration (2 sec)
of the presented movies. The model equation, includ-
ing the observation data, the design matrix, and the error
term, was convolved with a Gaussian kernel of dispersion
of 4 sec FWHM to account for the temporal autocorrela-
tion (Worsley & Friston, 1995). In the following, contrast
images, that is, beta value estimates of the raw-score dif-
ferences between specified conditions, were generated
for each participant. As all individual functional datasets
were aligned to the same stereotactic reference space,
the single-subject contrast images were entered into a
second-level random effects analysis for each of the con-
trasts. One-sample t tests were employed for the group
analyses across the contrast images of all subjects that
indicated whether observed differences between condi-
tions were significantly distinct from zero. The t values
were subsequently transformed into Z scores. To correct
for false-positive results, in a first step, an initial voxelwise
z-threshold was set to Z = 2.33 ( p = .01, uncorrected). In
a second step, the results were corrected for multiple
comparisons using cluster-size and cluster-value thresh-
olds obtained by Monte Carlo simulations at a significance
level of p = .005, that is, the reported activations are
significantly activated at p < .005, corrected for multiple comparisons at the cluster level. To investigate more thoroughly the comparability of brain responses in the areas identified by contrasts, per- centage signal change analyses of the BOLD response Schubotz and von Cramon 645 D o w n l o a d e d l l / / / / j t t f / i t . : / / f r o m D h o t w t n p o : a / d / e m d i f t r o p m r c h . s p i l d v i e r e r c c t . h m a i r e . d u c o o m c / n j a o r c t i n c / e a - p r d t i 2 c 1 l 4 e - 6 p 4 d 2 f 1 / 9 2 3 1 7 / 5 4 0 / 2 6 o 4 c 2 n / 1 2 0 7 0 6 9 0 0 2 0 1 7 0 4 / 9 j o p c d n . b 2 y 0 g 0 u 9 e . s t 2 o 1 n 0 4 0 9 8 . S p e d p f e m b y b e g r u 2 e 0 2 s 3 t / j t / . . . f o n 1 8 M a y 2 0 2 1 were carried out where the mean signal change over a 6-sec epoch, starting 4 sec after movie onset, were ex- tracted from selected voxels within significantly acti- vated brain areas for the experimental and the resting baseline conditions. The mean signal change of a voxel for each condition was calculated in relation to the mean signal intensity of that voxel across all time steps. mentary motor area (pre-SMA), and the left superior intraparietal sulcus. Extensive activation was also found in the fusiform gyrus (FG) and in the occipital gyri in- cluding probably the LOC as well as an area we will hereafter refer to as EBA/hMT, as hMT overlaps closely with EBA (Downing, Wiggett, & Peelen, 2007; Figure 2, Table 1). RESULTS Behavioral Results Performance was assessed by error rates and reaction times. Repeated measures ANOVAs were performed for each of these measures with the two-level factor Type (pretend and real) and the three-level factor Switch (goal, objects, and manipulations). Regarding reaction times, a main effect for the factor Type [F(1,17) = 13.791, p < .005] and an interaction Type (cid:2) Switch [F(2,34) = 4.344, p < .05] was found. These effects re- flected that responses to the action descriptions were slower for pretend (mean ± standard error, 421 ± 32 msec) as compared with real actions (393 ± 26 msec). The t tests showed that in case of pretend actions, responses to trials in that only objects switched were faster (386 ± 26 msec) than when only manipulations were switched (446 ± 32 msec; t18 = 5.059, p < .001) and were marginally faster than when both manipula- tions and objects were switched (431 ± 39 msec; t18 = 1.858, p = .081). For error rates, a significant main effect was found for the factor Switch [F(2,34) = 5.023, p < .01] and for the factor Type [F(1,17) = 9.305, p < .05]. In par- ticular, subjects made more errors when matching action descriptions with pretend actions (7.4 ± 3.3%) than with real actions (1.1 ± 0.8%). The t tests showed that action descriptions for trials in that only objects switched were easier (1.4 ± 1.1% errors) than for trials in that only manipulations (5.3 ± 2.2%, t18 = 3.487, p < .005) or both manipulations and objects (6.0 ± 2.8%, t18 = 2.955, p < .01) switched. Overall, behavioral performance implicated that inferring goals was slightly more demanding from pretend than from real actions, and that the inspection of manipulation information was more demanding than the inspection of object information. However, as all effects remained below differences of 60 msec and about 6% errors, we did not expect them to account for activation differences in our BOLD contrasts. fMRI Results Observing Real and Pretend Action The network commonly activated by either the obser- vation of real action and that of pretend action [conjunc- tion (real > rest) \ (pretend > rest)] comprised the
bilateral PMv [Broca’s area (BA) 6/44], the left inferior
frontal sulcus (IFS; BA 9/46), the aIPS, the left SMG, the
left dorsal premotor cortex (PMd), the left presupple-

Observing Pretend versus Real Action, and Vice Versa

Among these areas, the observation of pretend action
yielded significantly more activation than real action
((pretend > real) \ ((real > rest) \ (pretend > rest)))
in the left PMv, the left aIPS extending into SMG, the left
IFS (BA 9/46), the left pSTS, and the right EBA/hMT
(Figure 3A, Table 2). In contrast, the observation of real
action yielded significantly more activation than pretend
action ((real > pretend) \ ((real > rest) \ (pretend >
rest))) in the FG and/or LOC bilaterally (hereafter LOC),
comprising anterior and posterior compartments in the
right superior parietal lobule (BA 7) and in the right
postcentral gyrus (Figure 3B, Table 2).

Inferring Goals

Goals can be inferred from observed action on the basis
of manipulation and object information. To identify
brain areas involved in goal inference on the basis of
manipulations, we analyzed the effect of providing sub-
jects with new manipulation information (manipulation
switch effect), and subsequently to identify brain areas
involved in goal inference on the basis of object infor-
mation, we analyzed the effect of providing subjects with
new object information (object switch effect). Note that
in a direct comparison between the M and the O trials,
the effects of manipulation switches and object repeti-
tion would have been inextricably confounded. Ac-
cordingly, manipulation switch effects were tested
while controlling for object switches by contrasting G
with O trials and object switch effects by contrasting G
with M trials.

Figure 2. The network conjointly activated by the observation of
real and pretend actions (as compared with rest) comprised the
areas that are typically seen for action observation, including PMv
extending from BA 6 into BA 44, anterior parietal regions (aIPS and
SMG) as well as the pSTS. For further abbreviations, see Results section.

646

Journal of Cognitive Neuroscience

Volume 21, Number 4

D
o
w
n
l
o
a
d
e
d

/
j

t
t

f
/

i
t
.

:
/
/

/
j

o
n

1
8

M
a
y

2
0
2
1

Table 1. Action Observation Network Common to Pretend
and Real Action: Conjunction of Observation of Real Action
as Compared with Rest and Observation of Pretend Action
as Compared with Rest

Area

Conjunction of Real Action versus Rest and Pretend Action

versus Rest

PMv

PMd

pre-SMA

IFS (BA 9/46)

Anterior IPS

Superior IPS

SMG (BA 40)

EBA/hMT/pSTS

LOC

(cid:1)47

(cid:1)26

(cid:1)5

(cid:1)38

(cid:1)35

(cid:1)29

(cid:1)59

(cid:1)41

(cid:1)35

(cid:1)8

(cid:1)35

(cid:1)71

(cid:1)23

(cid:1)54

(cid:1)63

(cid:1)45

(cid:1)84

(cid:1)81

(cid:1)3

(cid:1)6

5.25

5.9

5.16

4.14

5.05

5.862

5.178

5.984

6.127

6.789

6.93

6.536

5.98

6.622

Anatomical specification, Talairach coordinates, maximum Z value
(volume is not given as all activations were local maxima of a common
activation).

Abbreviations: PMv = ventral premotor cortex; PMd = dorsal premotor
cortex; pre-SMA = presupplementary motor area; IFS = inferior frontal
sulcus; IPS = intraparietal sulcus; SMG = supramarginal gyrus; EBA/
hMT = extrastriate body area/human motion-selective area; pSTS =
posterior superior temporal sulcus; FG = fusiform gyrus; LOC = lateral
occipital complex.

Inferring Goals by Analyzing New Manipulations

For the observation of actions in that both manipula-
tions and objects switched as compared with those in
that only objects switched (G > O, manipulation switch
effect), activations were located in the left central OFC

Table 2. Different Weights of the Action Observation
Network: Direct Contrasts between Observation of Real
and Pretend Action ((Pretend > Real) \ ((Real > Rest) \
(Pretend > Rest))) and ((Real > Pretend) \ ((Real >
Rest) \ (Pretend > Rest)))

Area

mm3

Observation of Pretend versus Real Action

PMv

IFS (BA 9/46)

aIPS

SMG

(cid:1)45

(cid:1)42

33 4.35

3672

15 3.50

1296

(cid:1)42 (cid:1)39

54 2.99

2025

(cid:1)53 (cid:1)23

36 2.95

l.m.

EBA/hMT/pSTS

(cid:1)54 (cid:1)54

12 3.32

3078

EBA/hMT

48 (cid:1)57

3 3.80

1728

Observation of Real versus Pretend Action

LOC

(cid:1)27 (cid:1)66 (cid:1)9 4.12

9504

(cid:1)29 (cid:1)92

3 4.05

l.m.

21 (cid:1)69 (cid:1)6 4.82 18738

22 (cid:1)83 (cid:1)3 4.63

l.m.

Superior parietal lobule (SPL)

21 (cid:1)48

66 3.77

2322

Postcentral gyrus (SII)

63 (cid:1)12

27 4.32

3294

l.m. = local maximum.

(BA 11/10), left anterior IFS (BA 9/46), left PMv (BA 6),
slightly extending into the opercular part of the inferior
frontal gyrus (BA 44), and in a right inferior temporal re-
gion around the collateral sulcus, probably reflecting the
parahippocampal place area (PPA; Epstein & Kanwisher,
1998) (Figure 4, Table 3). A signal change analysis in the
identified areas corroborated that manipulation switch
effects were not statistically different for real and pre-
tend actions. There was a main effect for the factor Type
(real action and pretend action) due to higher signals in
pretend as compared with real actions in PMv [F(2,34) =
29.332, p < .000], BA 44 [F(2,34) = 6.113, p < .05], and D o w n l o a d e d l l / / / / j t t f / i t . : / / f r o m D h o t w t n p o : a / d / e m d i f t r o p m r c h . s p i l d v i e r e r c c t . h m a i r e . d u c o o m c / n j a o r c t i n c / e a - p r d t i 2 c 1 l 4 e - 6 p 4 d 2 f 1 / 9 2 3 1 7 / 5 4 0 / 2 6 o 4 c 2 n / 1 2 0 7 0 6 9 0 0 2 0 1 7 0 4 / 9 j o p c d n . b 2 y 0 g 0 u 9 e . s t 2 o 1 n 0 4 0 9 8 . S p e d p f e m b y b e g r u 2 e 0 2 s 3 t / j / . f . t . Figure 3. Direct contrasts between observation of real and pretend actions revealed different weights of the action observation network. (A) Areas elevated by the observation of pretend as compared with real action included left IFS (BA 9/46), left PMv, left aIPS, and EBA/hMT extending into pSTS in the left hemisphere. (B) Conversely, areas more engaged in processing real as compared with pretend actions were primarily found in LOC bilaterally. o n 1 8 M a y 2 0 2 1 Schubotz and von Cramon 647 D o w n l o a d e d l l / / / / j f / t t i t . : / / f r o m D h o t w t n p o : a / d / e m d i f t r o p m r c h . s p i l d v i e r e r c c t . h m a i r e . d u c o o m c / n j a o r c t i n c / e a - p r d t i 2 c 1 l 4 e - 6 p 4 d 2 f 1 / 9 2 3 1 7 / 5 4 0 / 2 6 o 4 c 2 n / 1 2 0 7 0 6 9 0 0 2 0 1 7 0 4 / 9 j o p c d n . b 2 y 0 g 0 u 9 e . s t 2 o 1 n 0 4 0 9 8 . S p e d p f e m b y b e g r u 2 e 0 2 s 3 t / j . / . t f . o n 1 8 M a y 2 0 2 1 Figure 4. Manipulation switch. Contrast between trials showing new versus repeated manipulations (G > O) and corresponding signal changes
in (1) OFC, (2) PPA, (3) anterior, (4) opercular inferior frontal sulcus (BA 44), and (5) PMv. These areas were elevated when goals were to
be inferred on the basis of changed manipulation information. Bar charts indicate percentages signal changes due to new versus old objects
(blue), manipulations (yellow), or both (i.e., goals; red). White bars show the signal during rest.

IFS [F(2,34) = 10,016, p < .01]. However, there were no significant interactions of Type (cid:2) Switch, underlining that the considered brain areas showed a comparable ma- nipulation switch effect in both real as well as pretend Table 3. Inferring Goals: Contrasts between New and Repeated/Old Information about Manipulations (G > O)
and Objects (G > M)

Area

mm3

New versus Repeated Manipulations (Contrast G > O)

OFC (BA 11/10)

PMv

(cid:1)24

(cid:1)42

IFS pars opercularis (BA 44) (cid:1)50

45 (cid:1)3

4.80

2619

3.71

3132

3.36

l.m.

IFG/IFS

PPA

(cid:1)41

3.83

36 (cid:1)33 (cid:1)6

5.12

864

756

New versus Repeated Objects (Contrast G > M)

LOC

(cid:1)33 (cid:1)66 (cid:1)3

3.31

1188

33 (cid:1)81 (cid:1)3

3.72

3375

(cid:1)33 (cid:1)48 (cid:1)9

4.33

7101

33 (cid:1)48 (cid:1)9

3.58

2781

Sensorimotor cortex

33 (cid:1)18

3.83

6939

actions. We found a main effect for the factor Switch
(object, manipulation, or both) in OFC [F(2,34) = 10.325,
p < .005] due to significant differences between G > O
(t18 = 6.304, p < .001) and G > M (t18 = 2.485, p < .05); the same was true for BA 44 [Switch, F(2,34) = 10.708, p < .001; G > O, t18 = 3.977, p < .001; G > M, t18 =
3.169, p < .01], for IFS [Switch, F(2,34) = 11.414, p < .001; G > O, t18 = 5.203, p < .001; G > M, t18 = 3.708,
p < .005], and for PMv [Switch, F(2,34) = 6.598, p < .01; G > O, t18 = 4.22, p < .001; G > M, t18 = 2.876 p < .01]; only for PPA, G and M did not differ though showing the same trend [Switch, F(2,34) = 5.15, p < .05; G > O, t18 =
4.018 p < .001]. We finally also calculated the interaction contrast G >
O (cid:2) Pretend > Real. In line with the signal change
analyses, this contrast did not yield any significant activa-
tions, thereby corroborating that pretend and real ac-
tions were associated with the same activation pattern
with respect to switching.

Inferring Goals by Analyzing New Objects

As stated in the beginning, we did not expect common
activations for real and pretend actions for trials present-
ing new objects versus repeated objects, as object in-
formation was expected to be largely ignored in pretend
actions. Unexpectedly, however, for object switch versus

648

Journal of Cognitive Neuroscience

Volume 21, Number 4

object repetition trials (G > M), activation was found
bilaterally in four subregions of the LOC (Figure 5,
Table 3). To further explore this result, a signal change
analysis was calculated, revealing a pattern that was
comparable for all four analyzed areas: the signal change
was comparably high for all conditions except ma-
nipulation switch trials in pretend action that induced
a lower signal. For left and right anterior LOC (aLOC)
and right posterior LOC (pLOC), there was a main ef-
fect Switch [left aLOC, F(2,34) = 7.272, p < .01; right aLOC, F(2,34) = 12,177, p < .001; right pLOC: F(2,34) = 6.802, p < .005] but also an interaction Switch (cid:2) Type [left aLOC, F(2,34) = 4.008, p < .05; right aLOC, F(2,34) = 10.597, p < .001; tendency in right pLOC, F(2,34) = 3.187, p = .054] that was due to a higher signal in all pretend actions that contained an object switch [left aLOC: OP > MP (t18 = 4.215, p < .001), GP >
MP (t18 = 3.933, p < .001); right aLOC: OP > MP (t18 =
6.815, p < .001), GP > MP (t18 = 4.149, p < .001); right pLOC: OP > MP (t18 = 5.403, p < .001), GP > MP (t18 =
4.106, p < .001)]. Left pLOC showed also a main effect Switch [F(2,34) = 7.453, p < .005] that was due to a higher signal in all actions that contained an object switch [O > M (t18 = 3.558, p < .005), G > M (t18 = 3.104, p < .01)] and a main effect for Type due to a higher signal in real as compared with pretend action [F(2,34) = 18.327, p < .001] but no interaction Switch (cid:2) Type. DISCUSSION The present fMRI study investigated goal inference from pretend action as compared with real action. The in- spection of real and pretend actions was considered to differ with regard to the weighting of specific compo- nents of the action observation network, pointing to- ward an attentional focus on manipulation information for pretend relative to real actions and an attentional fo- cus on object information for real relative to pretend ac- tions. In contrast, goal inference as an internal simulation of the currently observed manipulations was expected to yield comparable activations for real and pretend actions in regions related to sensorimotor integration, working memory, and internal reward evaluation. Observing Pretend versus Real Actions: Relying on Manipulation Information Relative to the resting baseline, observation of real and pretend actions yielded highly similar brain responses in the typical action observation network, emphasizing commonalities rather than fundamental differences in the way we cope with real and pretend action. How- ever, among the areas that were activated for obser- vation of real and pretend actions, four areas showed relatively enhanced response to pretend as compared with real actions: left PMv, left lPFC, left aIPS (extend- ing into SMG), and right and left EBA/hMT, the latter extending into left pSTS as well (for anatomical con- nections, cf. Schmahmann et al., 2007). As the two fron- tal areas, left PMv and left lPFC, were also found to respond to goal switches, they will be considered sepa- rately below. Enhanced activation in EBA/hMT and pSTS had been hypothesized for pretend versus real action observation due to their function in body, motion, and biological mo- tion processing, respectively (Downing et al., 2007; Taylor et al., 2007; Peelen, Wiggett, & Downing, 2006; Puce & Perrett, 2003; Downing, Jiang, Shuman, & Kanwisher, 2001). Because the presence of biological motion as well as body parts was balanced between pretend and real D o w n l o a d e d l l / / / / j t t f / i t . : / / f r o m D h o t w t n p o : a / d / e m d i f t r o p m r c h . s p i l d v i e r e r c c t . h m a i r e . d u c o o m c / n j a o r c t i n c / e a - p r d t i 2 c 1 l 4 e - 6 p 4 d 2 f 1 / 9 2 3 1 7 / 5 4 0 / 2 6 o 4 c 2 n / 1 2 0 7 0 6 9 0 0 2 0 1 7 0 4 / 9 j o p c d n . b 2 y 0 g 0 u 9 e . s t 2 o 1 n 0 4 0 9 8 . S p e d p f e m b y b e g r u 2 e 0 2 s 3 t / j / f t . . . Figure 5. Object switch. Contrast between trials showing new versus repeated objects (G > M) yielded
extended activation in the
LOC (aLOC = anterior, pLOC
posterior). Signal changes in
the four local maxima show
that repetition of objects
caused attenuation only in
pretend actions, whereas
object information was
processed in real actions no
matter whether repeated
(yellow bars) or new (red
and blue bars).

o
n

1
8

M
a
y

2
0
2
1

Schubotz and von Cramon

649

actions, we suggest that the inspection of motion and
body information was intensified during pretend actions,
that is, when goal inference had to rely solely on manip-
ulation information, whereas in real actions, object infor-
mation could also be exploited for the same purpose.

However, the fact that the left aIPS/SMG was more
active for pretend as compared with real action obser-
vation adds a very interesting facet as this region was
reported to be relevant for the performance of panto-
mime ( Johnson-Frey, 2004; Ohgami et al., 2004; Moll
et al., 2000) and was suggested for the explicit retrieval
of tool-related hand movements for different behavioral
purposes (Imazu et al., 2007). Moreover, Hamilton and
Grafton (2006) reported activation in IPS to systemati-
cally attenuate by the repetition of reached objects (but
not reaching trajectories) during action observation.
Our findings particularly corroborate results from pa-
tients with left inferior parietal lesions demonstrating
a strong relationship between the recognition and the
imitation (performance) of object-related pantomime
(Buxbaum, Kyle, et al., 2005) that can be considered to
reflect expressions of deficits in internal models for
planning object-related actions (Buxbaum, Johnson-
Frey, & Bartlett-Williams, 2005). Along these lines, eleva-
tion of inferior parietal activation for observing pretend
as compared with real action can be interpreted as mani-
festations of higher demands on this internal modeling
due to missing (external) object information.

Notably,

inferior parietal activation comprised both
SMG and aIPS in the present study. Macaque research
implicates two fairly different processes in the puta-
tive homologues of SMG (macaque area PF) and aIPS
(macaque area AIP; cf. Committeri et al., 2007; McGeoch,
Brang, & Ramachandran, 2007). The former contains
parietal mirror neurons and mediates between PMv and
pSTS in a network for both action observation and action
execution (Keysers & Perrett, 2004); the latter is sug-
gested to provide the PMv with a pragmatic description
of objects (Fagg & Arbib, 1998). Functionally, it is more
plausible to interpret our findings along the lines of pa-
rietal mirror neurons and the analysis of observed action.
However, additional
involvement of aIPS could be ex-
plained in two ways. As object information pointing to-
ward the currently valid goal was not available in pretend
action, aIPS may either reflect the imagery of tested
classes of objects matching the currently observed ma-
nipulations, driven by top–down modulation from ventral
imagery is known to lead to
premotor areas. Indeed,
higher BOLD responses than perception in many cases
(e.g., Imazu et al., 2007). Alternatively, aIPS may reflect
the suppression of currently invalid pragmatic object in-
formation stemming from the wildcard objects presented
in the pretend action condition. Note in this context that
one could suggest that the presence of inappropriate
objects in the pretend condition might lead to activation
of brain regions involved in dealing with incongruity.
However, the two areas mostly suggested in enhanced

cognitive control during resolution of incongruity, that is,
the ACC and the DLPF cortex (Carter & van Veen, 2007),
were not part of the pretend versus real action contrast.
Therefore, it appears that incongruity effects and result-
ing increase of cognitive control did not play a significant
role for the pretend action condition.

Observing Real versus Pretend Actions: Relying
on Object Information

The contrast between real and pretend action observa-
tion was expected to stress the exploitation of object
information. It revealed extended LOC activity, an area
known to play an important role in human object rec-
ognition. In the context of the present study, LOC’s in-
volvement is highly plausible as an area that represents
the shapes of objects independent of low-level visual cues
such as color, motion, or texture (Grill-Spector et al.,
2001). As an object’s shape rather than its color or texture
determines/rules hand posture and motion during action,
recognition of object shape was of primary task relevance.

Inferring Goals: Inspecting New Manipulations

Using a switching protocol, we set out to more specifically
investigate goal inference on the basis of manipulation
information. Although this information was considered to
be especially relevant for the understanding of pretend
actions, it was expected to be exploited for goal inference
in both real and pretend actions, and it was taken to be
particularly elevated in trials in that this type of informa-
tion was altered relative to the preceding trial. The G >
O contrast revealed enhanced activation in four areas two
of that were part of the action observation network and
also enhanced for pretend versus real action, namely, the
left ventral premotor cortex (PMv), here extending into
the pars opercularis of the inferior frontal gyrus (BA 44)
and the left lPFC. In contrast, the left OFC (BA 11/10, OFC
hereafter) and the right PPA were not primarily identified
in the action observation network but exclusively found
by contrasting G > O.

As to the functions attributed to the OFC, a recent
review suggests that OFC holds information about the
value of reward outcomes in working memory when
we formulate action plans and predict and monitor ex-
pected outcomes ( Wallis, 2007). To fully appreciate this
interpretation with regard to the present findings, it is
important to consider that, firstly, we found OFC (as
well as PPA) only for manipulation switches but not for
object switches (see below), and secondly, this activity
was observed when contrasting trials in that both ma-
nipulation and objects were switched (goal switch) with
trials in that only objects switched, thereby ruling out
unspecific switching effects. In close keeping with Wallis
(2007), we suggest that the OFC subserves the assess-
ment of trade-offs when a scenario allows for alterna-
tive action goals and determines how well the actually

650

Journal of Cognitive Neuroscience

Volume 21, Number 4

D
o
w
n
l
o
a
d
e
d

/
j

f
/

t
t

i
t
.

:
/
/

/
j

o
n

1
8

M
a
y

2
0
2
1

observed outcome satisfies currently tested forward
models. Concurrently with OFC, manipulation switches
enhanced activity in the right PPA and in two areas of the
action observation network: left lPFC (BA 9/46) and left
PMv. These areas are interconnected: OFC has some
connections to PM (Morecraft, Geula, & Mesulam, 1992)
and intense connections with the lPFC (see Wallis, 2007)
that in turn has connections with the PMv (Lu, Preston,
& Strick, 1994; Barbas & Pandya, 1987; Matelli, Camarda,
Glickstein, & Rizzolatti, 1986); moreover, PPA projects
to both OFC (Barbas, 1988) and lPFC (Goldman-Rakic,
Selemon, & Schwartz, 1984). How do these areas func-
tionally interact in the context of goal switches?

It is largely agreed upon that activation of our motor
system during action observation is due to an internal
simulation of the observed action; that is, the system is
activated similarly as if we would perform the observed
action ourselves ( Jeannerod, 2001). Computationally,
the notion of multiple forward models running in par-
allel has been used to explain the efficiency with which
we engage in goal
inference when observing actions
(Miall, 2003; Wolpert & Flanagan, 2001). Putting these
perspectives together, the picture emerging from the
present findings is that OFC activation reflects the cal-
culation of the value of a reward outcome of the cur-
rently tested forward models of the observed action.

Wallis (2007) elaborates that the lPFC uses the reward
signal from the OFC to plan behavior toward obtaining
the goal. In the context of the presently used action
observation task, the lPFC hence may select currently
potentially relevant goals to be subjected to a reward
analysis by OFC. This includes also holding these alter-
native plans in working memory. Potentially relevant ac-
tion goals are bottom–up provided by input from PMv
that runs sensorimotor simulations in connection with
its parietal projection sites. On the basis of the reward
signals from OFC, lPFC may in turn alter the selection of
action goals currently tested (simulated) in PMv. Note-
worthy, the parietal projection site of the PMv, the aIPS,
was missing in the G > O contrast. Thus, activity in aIPS
was not significantly enhanced by the observation of
new as compared with repeated manipulations, possibly
because novelty of object information was controlled for
in this contrast.

With respect to the functional contribution of the PPA
in this context, an exciting explanation refers to this
area’s role in processing scene gist (Epstein, 2005). Just
as for OFC, PPA was only seen for goal switch effects but
not for action observation per se, indicating that OFC
and PPA may provide a transient input, possibly being
top–down in the case of OFC and bottom–up in that of
PPA, to lPFC, thereby modulating its influence on PMv.

Inferring Goals: Inspecting New Objects

In contrast to the inspection of new versus repeated
manipulations, which resulted in the same pattern of ac-

tivations for real and pretend actions, the inspection of
new versus repeated objects was found to modulate LOC
exclusively for pretend actions (see Figure 5). The pattern
of activations implicates that although the processing of
object shape was suppressed or canceled early in case of
object repetition in pretend actions, new manipulations
triggered a reconsideration of object information, even
when repeated, in the case of real actions.

‘‘Theory of Mind’’ versus ‘‘Mirror Neuron System’’

Contrasting the observation of pretend with real action,
we found activation in areas that have been discussed to
belong to the so-called ‘‘mirror neuron system’’ (MNS)
network (Rizzolatti & Craighero, 2004). In contrast, the
only other study comparing the observation of pretend
and real action (German et al., 2004) reported activa-
tions that relate to mental state attribution, which is also
referred to as ‘‘theory of mind’’ (ToM) (cf. Gallagher &
Frith, 2003). There is an ongoing debate on the rela-
tionship between MNS and ToM and their respective
roles in understanding observed actions (cf. Saxe, 2005).
Although we are not in the position to bridge the puz-
zling gap between the neural correlates of goal inference
and mind reading in general, considerable differences
in the experimental design of the German et al. (2004)
study and ours have to be considered, which may ac-
count for discrepant results. Firstly, the sight of whole
persons may much more provoke ToM processes than
the sight of hands on objects. Secondly, because objects
were entirely missing in the pretend actions employed
by German et al., subjects may have been more engaged
in active considerations and inferences about the class
of objects that was pretended to be manipulated. Fi-
nally, short movies may have biased a direct matching
of the observed manipulations onto one’s own action
repertoire, whereas in case of longer observation times,
additional cognitive processes may evolve, even auto-
matically, that relate to the actor’s intentions and men-
tal states.

Inferring Goals from Pretense: General
Concluding Remarks

The picture emerging from the present findings sheds
new light on the mechanisms driving the inference of
goals from observed pretend as well as observed real
action. We found clear evidence for both commonalities
and differences between the neural correlates of under-
standing pretend and real actions. Strikingly, our find-
ings show that not all of the components of the action
observation network are engaged in the inference of
new goals in observed actions and that, conversely, not
all areas that are engaged in the inference of a new ac-
tion goal are integral part of the action observation
network. This dissociation may be due to the fact that
inference processes are more subtle and phasic
goal

Schubotz and von Cramon

651

D
o
w
n
l
o
a
d
e
d

/
j

t
t

f
/

i
t
.

:
/
/

/
j

o
n

1
8

M
a
y

2
0
2
1

than the massive and tonic activity triggered by the ob-
servation of an ongoing action with all its facets of per-
ceptual analysis and mnemonic demands.

The present results implicate that the requirement to
infer a new goal draws inter alia on a selection of those
areas that subserve the analysis of manipulation rather
than object information. The bias toward manipulation
is probably linked to the fact that we operationalized
goals as chains of manipulations of objects and not, as
would have been certainly possible, as object targets
(e.g., a disk, a cookie), spatial targets (e.g., left, right),
or combinations of both (cf. Hamilton & Grafton, 2006).
Our findings implicate that goals, if operationalized this
way, are more closely linked to how somebody is mov-
ing toward objects than to the objects themselves, pos-
sibly because a chain of hand postures and movements
is less ambiguous with respect to the intended goal,
whereas in contrast objects provide diverse options for
goal-directed manipulations.

Acknowledgments

We cordially thank Anna Abraham, Uta Wolfensteller, and
Kirsten Volz for their very helpful comments on the manu-
script, Gaby Lohmann and Karsten Mueller for support in MRI
statistics, Andrea Gast-Sandmann and Kerstin Flake for support
in graphic layout and stimulus materials, and Marcel Muecke
for experimental assistance.

Reprint requests should be sent to Ricarda I. Schubotz, Motor
Cognition Group, Max Planck Institute for Neurological Re-
search, Gleueler Str. 50, 50931 Ko¨ln, Germany, or via e-mail:
schubotz@nf.mpg.de.

REFERENCES

Barbas, H. (1988). Anatomic organization of basoventral and

mediodorsal visual recipient prefrontal regions in the rhesus
monkey. Journal of Comparative Neurology, 276, 313–342.

Barbas, H., & Pandya, D. N. (1987). Architecture and frontal
cortical connections of the premotor cortex (area 6) in
the rhesus monkey. Journal of Comparative Neurology,
256, 211–228.

Buccino, G., Binkofski, F., Fink, G. R., Fadiga, L., Fogassi, L.,

Gallese, V., et al. (2001). Action observation activates
premotor and parietal areas in a somatotopic manner:
An fMRI study. European Journal of Neuroscience, 13,
400–404.

Buxbaum, L. J., Johnson-Frey, S. H., & Bartlett-Williams, M.

(2005). Deficient internal models for planning hand–object
interactions in apraxia. Neuropsychologia, 43, 917–929.
Buxbaum, L. J., Kyle, K. M., & Menon, R. (2005). On beyond

mirror neurons: Internal representations subserving
imitation and ecognition of skilled object-related actions
in humans. Brain Research. Cognitive Brain Research,
25, 226–239.

Calvo-Merino, B., Gre`zes, J., Glaser, D. E., Passingham, R. E.,
& Haggard, P. (2006). Seeing or doing? Influence of visual
and motor familiarity in action observation. Current Biology,
16, 1905–1910.

Carter, C. S., & van Veen, V. (2007). Anterior cingulate cortex and
conflict detection: An update of theory and data. Cognitive,
Affective & Behavioral Neuroscience, 7, 367–379.

Committeri, G., Pitzalis, S., Galati, G., Patria, F., Pelle, G.,
Sabatini, U., et al. (2007). Neural bases of personal and
extrapersonal neglect in humans. Brain, 130, 431–441.
Costantini, M., Galati, G., Ferretti, A., Caulo, M., Tartaro, A.,
Romani, G. L., et al. (2005). Neural systems underlying
observation of humanly impossible movements: An fMRI
study. Cerebral Cortex, 15, 1761–1767.

Downing, P. E., Jiang, Y., Shuman, M., & Kanwisher, N.

(2001). A cortical area selective for visual processing of
the human body. Science, 293, 2470–2473.

Downing, P. E., Wiggett, A., & Peelen, M. (2007). fMRI
investigation of overlapping lateral occipitotemporal
activations using multi-voxel pattern analysis. Journal
of Neuroscience, 27, 226–233.

Epstein, R. A. (2005). The cortical basis of visual scene

processing. Visual Cognition, 12, 954–978.
Epstein, R. A., & Kanwisher, N. (1998). A cortical

representation of the local visual environment. Nature,
392, 598–601.

Fagg, A., & Arbib, M. (1998). Modeling parietal–premotor
interactions in primate control of grasping. Neural
Networks, 11, 1277–1303.

Fein, G. G. (1981). Pretend play in childhood: An integrative

review. Child Development, 52, 1095–1118.

Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. P., Frith,
C. D., & Frackowiak, R. S. J. (1995). Statistical parametric
maps in functional imaging: A general linear approach.
Human Brain Mapping, 2, 189–210.

Gallagher, H. L., & Frith, C. D. (2003). Functional imaging
of ‘‘theory of mind’’. Trends in Cognitive Sciences, 7,
77–83.

German, T. P., Niehaus, J. L., Roarty, M. P., Giesbrecht, B.,
& Miller, M. B. (2004). Neural correlates of detecting
pretense: Automatic engagement of the intentional
stance under covert conditions. Journal of Cognitive
Neuroscience, 16, 1805–1817.

Goldman-Rakic, P. S., Selemon, L. D., Schwartz, M. L. (1984).
Dual pathways connecting the dorsolateral prefrontal cortex
with the hippocampal formation and parahippocampal
cortex in the rhesus monkey. Neuroscience, 12, 719–743.

Greenlee, M. W. (2000). Human cortical areas underlying
the perception of optic flow: Brain imaging studies.
International Review of Neurobiology, 44, 269–292.
Grill-Spector, K., Kourtzi, Z., & Kanwisher, N. (2001). The

lateral occipital complex and its role in object recognition.
Vision Research, 41, 1409–1422.

Grill-Spector, K., & Malach, R. (2001). FMR-adaptation: A tool
for studying the functional properties of human cortical
neurons. Acta Psychologica, 107, 293–321.

Grush, R. (2004). The emulation theory of representation:
Motor control, imagery, and perception. Behavioral and
Brain Sciences, 27, 377–442.

Hamilton, A. F., & Grafton, S. T. (2006). Goal representation

in human anterior intraparietal sulcus. Journal of
Neuroscience, 25, 1133–1137.

Helbig, H. B., Graf, M., & Kiefer, M. (2006). The role of

action representation in visual object recognition. Brain
Research. Experimental Brain Research, 174, 221–228.
Hermsdo¨rfer, J., Terlinden, G., Mu¨hlau, M., Goldenberg, G.,
& Wohlschla¨ger, A. M. (2007). Neural representations
of pantomimed and actual tool use: Evidence from an
event-related fMRI study. Neuroimage, 36, T109–T118.
Imazu, S., Sugio, T., Tanaka, S., & Inui, T. (2007). Differences

between actual and imagined usage of chopsticks: An
fMRI study. Cortex, 43, 301–307.

Jeannerod, M. (2001). Neural simulation of action: A

unifying mechanism for motor cognition. Neuroimage, 14,
S103–S109.

652

Journal of Cognitive Neuroscience

Volume 21, Number 4

D
o
w
n
l
o
a
d
e
d

/
j

t
t

f
/

i
t
.

:
/
/

/
j

o
n

1
8

M
a
y

2
0
2
1

Johnson-Frey, S. H. (2004). The neural bases of complex tool
use in humans. Trends in Cognitive Sciences, 8, 71–78.
Johnson-Frey, S. H., Maloof, F. R., Newman-Norlund, R.,
Farrer, C., Inati, S., & Grafton, S. T. (2003). Actions or
hand–object interactions? Human inferior frontal cortex
and action observation. Neuron, 39, 1053–1058.
Keysers, C., & Perrett, D. I. (2004). Demystifying social

cognition: A Hebbian perspective. Trends in Cognitive
Sciences, 8, 501–507.

Kro´liczak, G., Cavina-Pratesi, C., Goodman, D. A., & Culham,
J. C. (2007). What does the brain do when you fake it?
An fMRI study of pantomimed and real grasping. Journal
of Neurophysiology, 97, 2410–2422.

Lohmann, G., Muller, K., Bosch, V., Mentzel, H., Hessler, S.,
Chen, L., et al. (2001). LIPSIA—A new software system for
the evaluation of functional magnetic resonance images of
the human brain. Computerized Medical Imaging and
Graphics, 25, 449–457.

Lu, M., Preston, J., & Strick, P. (1994). Interconnections

between the prefrontal cortex and the premotor areas in
the frontal lobe. Journal of Comparative Neurology, 341,
375–392.

Manthey, S., Schubotz, R. I., & von Cramon, D. Y. (2003).

Premotor cortex in observing erroneous action: An fMRI
study. Brain Research. Cognitive Brain Research, 15,
296–307.

Matelli, M., Camarda, R., Glickstein, M., & Rizzolatti, G.

(1986). Afferent and efferet projections of the inferior
area 6 in the macaque monkey. Journal of Comparative
Neurology, 251, 281–298.

McGeoch, P. D., Brang, D., & Ramachandran, V. S. (2007).

Apraxia, metaphor and mirror neurons. Medical Hypotheses,
69, 1165–1168.

Miall, R. C. (2003). Connecting mirror neurons and forward

models. NeuroReport, 14, 2135–2137.

Moll, J., de Oliveira-Souza, R., Passman, L. J., Cunha, F. C.,
Souza-Lima, F., & Andreiuolo, P. A. (2000). Functional
MRI correlates of real and imagined tool-use pantomimes.
Neurology, 54, 1331–1336.

Morecraft, R. J., Geula, C., & Mesulam, M. M. (1992).

Cytoarchitecture and neural afferents of orbitofrontal cortex
in the brain of the monkey. Journal of Comparative
Neurology, 323, 341–358.

Naccache, L., & Dehaene, S. (2001). The priming method:

Imaging unconscious repetition priming reveals an abstract
representation of number in the parietal lobes. Cerebral
Cortex, 11, 966–974.

Newman-Norlund, R. D., van Schie, H. T., van Zuijlen, A. M.,

& Bekkering, H. (2007). The mirror neuron system is more
active during complementary compared with imitative
action. Nature Neuroscience, 10, 817–818.

Norris, D. G. (2000). Reduced power multiscale MDEFT
imaging. Journal of Magnetic Resonance Imaging, 11,
445–451.

Ohgami, Y., Matsuo, K., Uchida, N., & Nakai, T. (2004). An
fMRI study of tool-use gestures: Body part as object and
pantomime. NeuroReport, 15, 1903–1906.

Peelen, M. V., Wiggett, A. J., & Downing, P. E. (2006). Patterns

of fMRI activity dissociate overlapping functional brain
areas that respond to biological motion. Neuron, 49,
815–822.

Petrides, M. (2005). Lateral prefrontal cortex: Architectonic
and functional organization. Philosophical Transactions
of the Royal Society of London, Series B, Biological
Sciences, 360, 781–795.

Peuskens, H., Vanrie, J., Verfaillie, K., & Orban, G. A.

(2005). Specificity of regions processing biological motion.
European Journal of Neuroscience, 21, 2864–2875.
Puce, A., & Perrett, D. (2003). Electrophysiology and brain

imaging of biological motion. Philosophical Transactions
of the Royal Society of London, Series B, Biological
Sciences, 358, 435–445.

Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron
system. Annual Review of Neuroscience, 27, 169–192.
Rizzolatti, G., & Luppino, G. (2001). The cortical motor

system. Neuron, 31, 889–901.

Saxe, R. (2005). Against simulation: The argument from

error. Trends in Cognitive Sciences, 9, 174–179.

Schmahmann, J. D., Pandya, D. N., Wang, R., Dai, G., D’Arceuil,
H. E., de Crespigny, A. J., et al. (2007). Association fibre
pathways of the brain: Parallel observations from diffusion
spectrum imaging and autoradiography. Brain, 130,
630–653.

Schubotz, R. I., & von Cramon, D. Y. (2004). Sequences of
abstract nonbiological stimuli share ventral premotor
cortex with action observation and imagery. Journal of
Neuroscience, 24, 5467–5474.

Talairach, J., & Tournoux, P. (1988). Co-planar stereotaxic

atlas of the human brain. New York: Thieme.

Taylor, J. C., Wiggett, A. J., & Downing, P. E. (2007). fMRI
analysis of body and body part representations in the
extrastriate and fusiform body areas. Journal of
Neurophysiology, 98, 1626–1633.

Thompson-Schill, S. L., D’Esposito, M., & Kan, I. P. (1999).
Effects of repetition and competition on activity in left
prefrontal cortex during word generation. Neuron, 23,
513–522.

Ugurbil, K., Garwood, M., Ellermann, J., Hendrich, K., Hinke,
R., Hu, X., et al. (1993). Imaging at high magnetic fields:
Initial experiences at 4 T. Magnetic Resonance Quarterly,
9, 259–277.

Vogt, S., & Thomaschke, R. (2007). From visuo-motor

interactions to imitation learning: Behavioural and brain
imaging studies. Journal of Sports Sciences, 25, 497–517.
Wallis, J. D. (2007). Orbitofrontal cortex and its contribution
to decision-making. Annual Review of Neuroscience, 30,
31–56.

Watanabe, M. (2007). Role of anticipated reward in cognitive
behavioral control. Current Opinion in Neurobiology,
17, 213–219.

Wolpert, D. M., & Flanagan, J. R. (2001). Motor prediction.

Current Biology, 11, R729–R732.

Worsley, K. J., & Friston, K. J. (1995). Analysis of fMRI

time-series revisited—Again. Neuroimage, 2, 173–181.

Schubotz and von Cramon

653

D
o
w
n
l
o
a
d
e
d