Planning the Potential Future during
Multi-item Visual Working Memory
Rose Nasrawi1
and Freek van Ede1,2
抽象的
■ Working memory allows us to retain visual information to
guide upcoming future behavior. In line with this future-oriented
purpose of working memory, recent studies have shown that
action planning occurs during encoding and retention of a single
visual item, for which the upcoming action is certain. We asked
whether and how this extends to multi-item visual working
记忆, when visual representations serve the potential future.
Human participants performed a visual working-memory task
with a memory-load manipulation (one/two/four items) and a
delayed orientation-reproduction report (of one item). 我们
measured EEG to track 15- to 25-Hz beta activity in electrodes
contralateral to the required response hand—a canonical marker
of action planning. We show an attenuation of beta activity, 不是
only in Load 1 (with one certain future action) but also in Load 2
(with two potential future actions), compared with Load 4 (和
low prospective-action certainty). 而且, in Load 2, 潜在的
action planning occurs regardless whether both visual items
afford similar or dissimilar manual responses, and it predicts
the speed of ensuing memory-guided behavior. This shows that
potential action planning occurs during multi-item visual work-
ing memory and brings the perspective that working memory
helps us prepare for the potential future. ■
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
j
/
哦
C
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
哦
C
n
_
A
_
0
1
8
7
5
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
介绍
Working memory allows us to hold onto visual information
to prepare for and guide potential future action (来自 埃德,
2020; 高贵 & Stokes, 2019; Rainer, 饶, & 磨坊主, 1999;
Baddeley, 1992; Fuster & 亚历山大, 1971). 例如,
when a football player breaks through the defense line, 这
player may look to see where the left- and right-wing
attackers are located. This information is retained in mem-
ory as the player sprints toward the goal and prepares to
potentially pass the ball to either attacker, 根据
the development of the attack. 在这个例子中, visual
information is retained in working memory in anticipation
of multiple potential future actions. 这样, visual
working memory allows for flexible behavior—being pre-
pared for multiple potential future actions offers a way to
deal with uncertainty in a dynamically unfolding environ-
蒙特 (Cisek & 卡拉斯卡, 2010).
A vast body of research into visual working memory has
provided a detailed understanding of the mechanisms of
encoding and retention (例如, Schneegans & Bays, 2017;
Serences, 2016; D’Esposito & Postle, 2015; Luck & 沃格尔,
2013; Harrison & Tong, 2009). 最终, working mem-
ory serves as a bridge between perception and upcoming
1Institute for Brain and Behavior Amsterdam, Vrije Universiteit
阿姆斯特丹, 荷兰人, 2Oxford Centre for Human Brain
Activity, 牛津大学, 英国
行动. 所以, it is also important to consider the role
of potential action planning alongside encoding and reten-
tion in visual working memory (豪雅, Ohl, & Rolfs, 2020;
Olivers & 罗尔夫塞玛, 2020; 来自 埃德, 2020; 迈尔斯, Stokes,
& 高贵, 2017). Two recent EEG studies provide evidence
that action planning and visual retention can co-occur dur-
ing working memory (Boettcher, Gresch, 高贵, & van
Ede, 2021; 施耐德, Barth, & Wascher, 2017). At least,
they show that this occurs during encoding and retention
of a single visual item, for which the upcoming action can
be fully predetermined in advance.
From the literature of motor planning research, 这
concept of parallel action planning proposes that we often
plan multiple potential actions in parallel, even before
selecting the relevant action for implementation (Gallivan,
巴顿, Chapman, Wolpert, & 弗拉纳根, 2015; 2016;
Grent-’t-Jong, Oostenveld, 詹森, 梅登多普, & Praamstra,
2013; Cisek, 2007; Cisek & 卡拉斯卡, 2005). A recent study
tentatively suggests that multiple potential actions may
also be planned alongside visual working memory. 什么时候
two visual items are retained in visual working memory,
and one of the two items is probed for action, visual and
motor representations are selected concurrently after the
memory delay (来自 埃德, Chekroud, Stokes, & 高贵,
2019). 然而, it is yet to be demonstrated how the
planning of multiple potential actions unfolds during
the memory delay, alongside the encoding and retention
of more than one visual item in working memory.
© 2022 麻省理工学院. Published under a
Creative Commons Attribution 4.0 国际的 (抄送 4.0) 执照
认知神经科学杂志 34:8, PP. 1534–1546
https://doi.org/10.1162/jocn_a_01875
In this study, we used EEG to address whether and how
multiple potential actions are planned alongside the
encoding and retention of multiple visual items in working
记忆. We envisioned two hypothetical scenarios. 在
the one-or-none scenario (Figure 1C), action planning
may occur alongside visual working memory only when
we retain one visual item for which the required action
is known. 在这种情况下, one would expect a relative attenu-
ation of EEG-beta activity (a canonical marker of action
规划; van Wijk, Daffertshofer, Roach, & Praamstra,
2009; Neuper, Wörtz, & Pfurtscheller, 2006; Mcfarland,
Miner, Vaughan, & Wolpaw, 2000; Salmelin & Hari,
1994) during the memory delay only when we retain
one visual
in the graded scenario
(Figure 1D), action planning may occur alongside visual
工作记忆, even when we retain multiple visual
items in anticipation of multiple potential actions. 在这个
案件, one would expect to observe an attenuation of
beta activity that depends on the number of action pos-
sibilities, as has previously been reported in studies con-
sidering action planning without simultaneous item
retention in visual working memory (Tzagarakis, 西方,
& Pellizzer, 2015, 2021; Tzagarakis, Ince, Leuthold, &
Pellizzer, 2010).
item. 或者,
To preview our results, we show that: (我) action planning
of multiple potential actions co-occurs with visual
retention of multiple visual items; (二) this effect occurs
regardless of whether potential actions require a similar
or dissimilar manual response; (三、) the degree to which
actions are planned during the memory delay is predictive
of the speed of memory-guided action afterward.
方法
参加者
Twenty-five healthy human adults (mean age 25.32 年,
标准差= 4.27 年, sex of the participants is unknown, 四
left-handed) participated in the experiment. All partici-
pants had normal or corrected-to-normal vision. None of
the participants were excluded from the analyses. 这
experiment was approved for by the Central University
Research Ethics Committee of the University of Oxford.
Participants provided written informed consent before
participating in the study. They received a monetary com-
pensation of £10 per hour after participation.
Experimental Design and Procedure
Participants performed a visual working-memory task with
a delayed orientation-reproduction report (Figure 1A). A
blocked memory-load manipulation was implemented in
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
j
/
哦
C
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
哦
C
n
_
A
_
0
1
8
7
5
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
数字 1. Experimental design and hypothetical action-planning scenarios. (A) Visual working-memory task. Participants performed a delayed
orientation-reproduction task. Participants viewed four colored oriented bars at encoding and memorized their orientations during the delay;
after which one of the bar colors was probed for action, indicating which orientation should be reproduced by moving a computer mouse. (乙)
Memory-load manipulation. We implemented a block-wise memory-load manipulation by preceding each block with a precue, indicating which bar
colors were relevant for the upcoming block. In Load 1, participants were asked to memorize the orientation of one bar (例如, the orientation of
the orange bar); in Load 2, participants were asked to memorize the orientation of two bars (例如, the orientation of the green and the blue bar);
and in Load 4, participants were asked to memorize the orientations of all four bars. (C, D) Two different scenarios for main results. In the first
scenario, one-or-none (C) action planning occurs during the memory delay only when we retain one visual item, but not when we retain more than
one visual item. 或者, in the graded scenario (D), the degree of action planning depends on the number of action possibilities, with more
action planning in Load 2 than in Load 4, despite the fact that it remains unknown throughout the memory delay what action will become relevant
in both conditions.
Nasrawi and van Ede
1535
the task. To achieve this, each block of trials was preceded
with an instruction display (block-wise precue) that indi-
cated the relevant color(s) of that block (Figure 1B). 帕-
ticipants were asked to encode and retain only the bars of
the instructed color(s). In Load 1, participants were required
to retain the orientation of the one bar whose color matched
that of the instruction cue; in Load 2, participants were
required to retain the orientations of the two bars whose
colors matched the instruction cue; in Load 4, 参与者
were required to retain the orientations of all four colored
bars. In all cases, four bars were presented on the screen at
encoding, as such controlling for bottom–up stimulus dif-
ferences between the load conditions.
The encoding display was followed by a memory delay
(1500 毫秒) in which only the fixation cross remained on
屏幕. After the delay, a response dial was presented
on the screen (as in the work of van Ede, Niklaus, & 高贵,
2017). This dial consisted of a gray circle with two smaller
circles (or handles), positioned opposite each other on
the larger circle, that together represented an orientation.
The color of the two handles indicated which bar orienta-
tion should be reproduced. The color was always chosen
randomly from the relevant colors in that block.
The position of the two handles could be adjusted by
moving the computer mouse in the direction of the orien-
tation of the cued item. By moving the computer mouse in
a certain direction, the response dial handles automatically
turned to match this direction, regardless of how they
were positioned before movement onset—thus linking
memorized orientation to specific manual actions, 看待-
less of the dial’s starting position. Participants confirm
their response by making a left-side mouse click. For con-
sistency, the mouse was always controlled with the right
手 (even in the few participants who preferred their left
手). The initial orientation of the response dial was ran-
domly varied on a trial-by-trial basis.
Participants had an unlimited amount of time to initiate
their response after the response dial had been presented
on the screen. Once their response was initiated, partici-
pants had a maximum of 2500 msec to confirm their
orientation-reproduction report with a left-side mouse
click. A visualization of an hourglass was presented under
the response dial to indicate the amount of time that had
通过了. After response confirmation, participants received
feedback on their orientation-reproduction precision. 如果
the absolute difference in orientation between the
response and the target was smaller than 15°, the fixation
cross turned green; if the absolute difference was larger
than 15°, or if the response deadline had passed, the fixa-
tion cross turned red. The intertrial interval was randomly
varied between 500 和 800 毫秒.
Participants were seated at a viewing distance of 90 厘米
from the computer screen. The bars had a diameter of 5.7°
visual angle and were centered at a 5.7° visual angle dis-
tance from fixation. Every encoding display contained four
colored oriented bars (红色的, 绿色的, 蓝色的 [RGB] 颜色
价值观) (绿色的, RGB: 0, 210, 63; 蓝色的, RGB: 0, 128, 255;
orange, RGB: 255, 127, 39; purple, RGB: 238, 0, 238) 那
were presented on a gray background (RGB: 25, 25, 25) 为了
500 毫秒. The relevant colors indicated in the block-wise
precue were randomly chosen. Color locations and bar ori-
entations were both randomly chosen on a trial-by-trial basis.
Preceding the main task, participants performed one
practice block of 20 trials for each memory-load condition
(IE。, 60 trials in total). During the main task, 参与者
performed two consecutive sessions with a 10- to 15-min
break in between. Each session contained 10 blocks of 20
trials for each memory-load condition (IE。, 2 × 10 × 20 ×
3 = 1200 trials in total).
Load conditions were always grouped into three con-
secutive subblocks of 20 trials each, with load conditions
1, 2, 和 4 occurring in random order. After every 60 试验,
participants were prompted to have a self-paced break.
参加者 1 和 2 执行的 12 blocks during each ses-
锡安 (IE。, 1440 trials in total). After realizing that number of
trials took a considerable amount of time, 的数量
blocks per session was reduced from 12 到 10 from Partic-
ipant 3 向前.
Behavioral Data Analyses
All behavioral analyses were performed in R (R核心团队,
2020). The variables of interest for the behavioral analyses
were absolute error (in degrees) and decision time (在
毫秒). Absolute error was defined as the absolute differ-
ence between the reported and the target orientation.
Decision time was defined as the time between the onset
of the response dial and the initiation of the mouse
response (as in the work of van Ede et al., 2017, 2019).
Before turning to the main analyses, behavioral data
were cleaned by removing outlier decision times. 第一的, tri-
als with decision times smaller than 200 msec or larger
比 5000 msec were excluded from further analyses.
下一个, for each participant, trials were excluded with deci-
sion times larger than the mean plus 2.5 times the stan-
dard deviation. Means and standard errors of the variables
of interest were calculated for each participant and
memory load using the Rmisc package (Hope, 2013),
and visualized using the ggplot2 package ( Wickham,
2016). Two one-way repeated-measures ANOVAs were
performed to statistically evaluate the effect of memory
load on the variables of interest. For each memory-load
比较, and each variable of interest, post hoc com-
parisons were done using the Tukey honest significant
difference test.
EEG Acquisition and Analyses
EEG was measured using Synamps amplifiers and Neuro-
scan acquisition software (Compumedics Neuroscan),
using the standard 10–10 System 64 electrode setup. 这
left mastoid was used as an active reference. The ground
electrode was placed on the left upper arm. During acqui-
位置, the data were low-pass filtered with a 250-Hz cutoff
and sampled at 1000 赫兹.
1536
认知神经科学杂志
体积 34, 数字 8
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
j
/
哦
C
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
哦
C
n
_
A
_
0
1
8
7
5
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
预处理
All EEG analyses were performed in MATLAB (2020乙;
The MathWorks, 2020) using the FieldTrip toolbox
(Oostenveld, 薯条, 马里斯, & 锄, 2011; https://
fieldtriptoolbox.org). After acquisition, data were rerefer-
enced to an average of the left and right mastoids. 然后,
50-Hz noise was filtered using a dft filter, and the data were
down-sampled to 200 赫兹. The data were epoched from
−1000 to 3000 毫秒, relative to memory encoding onset.
Independent component analysis (ICA) was used to
correct for eye-movement artifacts. The appropriate ICA
components used for artifact rejection were identified by
correlating the time-courses of the ICA components with
those of the measured horizontal and vertical EOG. 后
blink correction, the FieldTrip function ft_rejectvisual was
used to visually assess which trials had exceptionally high
方差, which were marked for rejection. Trials that had
been marked as too fast or too slow (as described in the
Behavioral Data Analyses section) were also rejected
from further analyses. A surface Laplacian transform was
applied to increase the spatial resolution of the central
15- to 25-Hz beta signal of interest (as in the work of van
Ede et al., 2019).
Channel and Frequency-Band Selection
For all reported analyses, channel and frequency-band
selections were predetermined. To investigate motor
activation contralateral to the hand used for reporting
(IE。, the right hand), we focused on EEG activity in chan-
nel C3 (IE。, a canonical EEG channel over the left motor
cortex). To zoom in on the beta-band, we additionally
extracted 15- to 25-Hz beta activity for all time course-
based visualizations (although note that we always also
statistically evaluated our data in the full time–frequency
飞机). These selections are in line with previous research
(例如, Boettcher et al., 2021; van Wijk et al., 2009; Neuper
等人。, 2006; Mcfarland et al., 2000; Salmelin & Hari, 1994)
and were set a priori.
Time–Frequency Analysis
Time–frequency responses for the frequency-range from 3
到 40 赫兹 (in steps of 1 赫兹) were obtained using the short-
time Fourier transform. Data were Hanning-tapered with a
sliding time window of 300 毫秒, progressing in steps of
50 毫秒. Time–frequency responses were contrasted for
each memory-load comparison (Load 1 与. 4; Load 1 与.
2; Load 2 与. 4), using a normalized subtraction to express
load effects as a percentage change: ((a − b) / (A + 乙)) ×
100. Time–frequency responses were averaged across
participants in Channel C3. To focus on the delay period
of interest, we considered all data in the time-window of
−100 to 2500 毫秒 (relative to memory encoding onset).
For topographies, time–frequency responses were aver-
aged for the predetermined beta frequency-band from
15 到 25 赫兹, and visualized in three consecutive time-
windows covering the full delay period: 500–1000 msec
(IE。, the first 500 msec after encoding offset), 1000–
1500 毫秒, 1500–2000 msec. To obtain beta time-courses,
time–frequency responses were averaged over the 15- 到
25-Hz band.
Dependence on Orientation-Similarity
To assess whether the observed differences between
Loads 2 和 4 depended on the item similarity in Load 2
试验, we also separately examined Load 2 trials in which
the items were similar versus dissimilar. 为此, 这
absolute difference in orientation was calculated between
the two relevant items in Load 2. Trials were marked as
similar if this absolute difference was smaller than 45°;
trials were marked as dissimilar if this absolute difference
was larger than 45°. The previously described calculation
of time–frequency responses, topographies, and time-
courses were repeated for the following contrasts: Load
2-similar versus 4; Load 2-different versus 4; Load
2-similar versus 2-dissimilar.
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
j
/
哦
C
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
哦
C
n
_
A
_
0
1
8
7
5
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Dependence on Decision Time
We also aimed to assess whether action planning—as
reflected in EEG beta activity in C3—during the memory
delay may have impacted decision times after the delay.
为此, trials in each memory load and participant
were marked as fast or slow using a median split for
decision times. We did this separately for each load
condition and ran the previously described calculation of
time–frequency responses, topographies, and time-
courses for the following contrasts: Load 1 fast versus
slow; Load 2 fast versus slow; Load 4 fast versus slow.
Statistical Evaluation
Cluster-based permutations (马里斯 & Oostenveld, 2007)
were performed for the statistical evaluation of the
above-described EEG contrasts for load, orientation-
相似, and behavior. This nonparametric approach
(or Monte Carlo method) offers a solution for the
multiple-comparisons problem in the statistical evaluation
of EEG data, 哪个, in our case, included a sizeable num-
ber of time–frequency comparisons. It does so by reduc-
ing the data to a single metric (例如, the largest cluster of
neighboring data points that exceed a certain threshold)
and evaluating this (in the full data space under consider-
化) against a single randomly permuted empirical null
分配. Cluster-based permutations were performed
on the time–frequency responses (considering clusters
in time and frequency) and time-courses (considering
clusters in time) of each above-described contrast, 使用
10.000 permutations, and an alpha level of 0.025.
Nasrawi and van Ede
1537
结果
Working Memory Performance Improves as a
Function of Item Certainty
Before turning to the main EEG results, we characterized
the effect of memory load on task performance (IE。, abso-
lute error and decision time). With an increase in memory
load, the absolute difference between the target orienta-
tion and the reported orientation (IE。, absolute error)
significantly increases (图2A; F(2, 72) = 52.02, p <
.001). Post hoc comparisons revealed a significantly lower
absolute error in Load 1 compared with Load 2 and
Load 4, and in Load 2 compared with Load 4 (all p <
.001). Similarly, the time it takes to initiate the mouse
response (i.e., decision time) also significantly increases
with an increase in memory load (Figure 2B; F(2, 72) =
20.77, p < .001). Post hoc comparisons revealed signifi-
cantly faster decision times in Load 1 compared with
Load 2 and Load 4 (both p < .001), and in Load 2 com-
pared with Load 4 ( p = .011). These effects of memory
load on absolute errors and decision times can further
be appreciated by a visualization of their respective den-
sity plots (Figure 2A, B, right).
These results confirm the effectiveness of the memory-
load manipulation: Although in each memory-load
condition four bars were always presented on the screen
at encoding, one, two, or four bars were selectively
retained in visual working memory, as instructed by the
block-wise precue. Moreover, these results show that,
with lower memory loads, participants’ orientation-
reproduction reports are initiated faster and are more
precise. With lower memory loads, there is a higher cer-
tainty about which item will be probed and, therefore, a
higher certainty about the required action. Hence, faster
response initiation with lower memory loads might at least
partly be accompanied by an increase in action planning
during the working-memory delay. Next, we will present
neural evidence for this idea. Critically, we will show that
this holds not only when comparing Load 1 to Loads 2 and
4, but also when comparing Loads 2–4, although, in both
conditions, the prospective action is unknown during the
memory delay.
Planning Multiple Potential Actions alongside
Visual Working Memory
We now turn to our central question: whether and how
multiple potential actions are planned alongside multi-
item visual encoding and retention in working memory.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
e
d
u
/
j
/
o
c
n
a
r
t
i
c
e
-
p
d
l
f
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
o
c
n
_
a
_
0
1
8
7
5
p
d
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Figure 2. Memory performance improves as a function of item certainty. (A) Effect of memory load on absolute orientation-reproduction error (°).
Bar graphs show average absolute error; error bars represent within-participant standard error (SE ); gray lines represent averages for each individual
participant; density plots (right) show density of target-deviation for each memory-load condition, averaged across participants. (B) Effect of memory
load on decision time (msec); the time from memory-probe onset to response onset. Bar graphs show average decision times; error bars represent
within-participant standard error (SE ); gray lines represent averages for each individual participant; density plots show density of decision times for
each memory-load condition, averaged across participants.
1538
Journal of Cognitive Neuroscience
Volume 34, Number 8
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
e
d
u
/
j
/
o
c
n
a
r
t
i
c
e
-
p
d
l
f
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
o
c
n
_
a
_
0
1
8
7
5
p
d
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Figure 3. Planning multiple potential actions alongside visual working memory. Comparison of neural activity in C3 for (A) Load 1 versus 4, (B) Load 1
versus 2, and (C) Load 2 versus 4. For each memory-load comparison—(i) difference in time–frequency response in electrode C3, aligned to
memory encoding. Colors indicate percentage change between the load conditions; black cluster outline indicates significant difference from a
cluster-based permutation analysis. (ii) Topographies of beta (15–25 Hz) percentage change for three different time windows during the memory
delay. (iii) Time-courses of beta (15–25 Hz) percentage change in electrode C3. Light-gray shadings around time-course indicate standard error;
black horizontal lines indicate significant clusters; black vertical lines indicate memory encoding (at 0 sec) and memory probe (at 2 sec).
To investigate this, we used EEG to track 15- to 25-Hz beta
attenuation in electrodes contralateral to the response
hand (i.e., C3)—a canonical neural marker of action
planning (e.g., Boettcher et al., 2021; van Wijk et al.,
2009; Neuper et al., 2006; Mcfarland et al., 2000; Salmelin
& Hari, 1994). To disentangle the one-or-none scenario
(Figure 1C; i.e., action planning only occurs alongside
visual working memory during retention of single visual
item for which the required computer mouse movement
direction can be fully planned in advance) from the
graded scenario (Figure 1D; i.e., action planning occurs
alongside visual working memory, even during retention
of multiple visual items in anticipation of multiple poten-
tial actions), we compared beta activity during the
memory delay across each possible memory-load com-
parison: Load 1 versus Load 4; Load 1 versus Load 2;
and Load 2 versus Load 4.
We observed a significant relative attenuation of
beta activity in C3 for Load 1, compared with Load 4
(Figure 3A-i; time–frequency map cluster p < .0001). This
relative beta attenuation in Load 1 showed a left-central
topography (i.e., contralateral to the response hand;
Figure 3A-ii). Moreover, in line with Boettcher et al.
(2021), it showed a bimodal temporal profile (Figure 3A-iii;
time-course cluster p < .0001). Similarly, we observed a
significant relative attenuation of beta activity in C3 for
Load 1 compared with Load 2 (Figure 3B-i; time–
frequency map cluster p < .0001), with similar topo-
graphical (Figure 3B-ii) and temporal (Figure 3B-iii;
time-course early cluster p < .0001, late cluster p =
.0004) characteristics as in the comparison between
Load 1 and Load 4. These data are consistent with the
notion that, in Load 1, participants know with certainty
at encoding which visual item they will need to report
at the end of the working-memory delay. Accordingly,
participants can plan the required action ahead of time,
leading to a stronger action-planning signal in the EEG in
Load 1 compared with Loads 2 and 4 (i.e., when more
than one item can become relevant later).
The most critical finding emerged when we compared
action planning in Load 2 versus Load 4. In both condi-
tions, participants remained oblivious about which item
Nasrawi and van Ede
1539
would be probed for report at the end of the memory
delay. Nevertheless, when directly comparing these condi-
tions, we also observed a significant relative attenuation
of beta activity in C3 for Load 2 compared with Load 4
(Figure 3C-i; time–frequency map; early cluster p = .0013,
late cluster p = .0021). As before, this effect was character-
ized by a similar C3-centered topography (Figure 3C-ii) and
bimodal temporal profile (Figure 3C-iii; time-course; early
cluster p = .0011, late cluster p = .0014).
In accordance with previous research (Boettcher et al.,
2021; Schneider et al., 2017), we confirm that action plan-
ning occurs alongside the retention of a single visual item
in working memory when the required action is certain.
The key novelty here is the emergence of this action-
planning signature during the retention of more than a
single visual item in working memory, when the to-be-
implemented action remains uncertain throughout the
memory delay. These results are in line with the graded
scenario (Figure 1D) we previously envisioned: We
observed an attenuation of beta activity that depends on
the number of action possibilities (largest in Load 1,
intermediate in Load 2, smallest in Load 4).
Similar and Dissimilar Potential Actions Are
Planned alongside Visual Working Memory
During the retention of two oriented bars (i.e., in Load 2),
the difference in orientation between those two bars var-
ies between trials: The difference can be smaller (i.e.,
when both orientations are similar) or larger (i.e., when
both orientations are dissimilar). Consequently, the
potentially required actions can also be similar (i.e., when
they both require the mouse to be moved a similar direc-
tion) or dissimilar (i.e., when they each require the mouse
to be moved in dissimilar directions).
Next, we aimed to rule out the possibility that the
observed relative attenuation of beta activity for Load 2
(compared with Load 4) was driven primarily by trials with
similar memorized orientations (as suggested in the work
of Grent-’t-Jong, Oostenveld, Jensen, Medendorp, &
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
e
d
u
/
j
/
o
c
n
a
r
t
i
c
e
-
p
d
l
f
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
o
c
n
_
a
_
0
1
8
7
5
p
d
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Figure 4. Similar and dissimilar potential actions are planned alongside visual working memory. Comparison of neural activity in C3 for (A) Load 2-
similar versus 4, (B) Load 2-dissimilar versus 2, and (C) Load 2-similar versus -dissimilar. Using the absolute difference in orientation between two
items in Load 2, trials were marked as similar if this difference was smaller than 45°, or as dissimilar if this difference was larger than 45°. For each
comparison—(i) difference in time–frequency response in electrode C3, aligned to memory encoding. Colors indicate percentage change between
the conditions; black cluster outline indicates significant difference from a cluster-based permutation analysis. (ii) Topographies of beta (15–25 Hz)
percentage change for three different time windows during the memory delay. (iii) Time-courses of beta (15–25 Hz) percentage change in electrode
C3. Light-gray shadings around time-course indicate standard error; black horizontal lines indicate significant clusters; black vertical lines indicate
memory encoding (at 0 sec) and memory probe (at 2 sec).
1540
Journal of Cognitive Neuroscience
Volume 34, Number 8
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
e
d
u
/
j
/
o
c
n
a
r
t
i
c
e
-
p
d
l
f
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
o
c
n
_
a
_
0
1
8
7
5
p
d
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Figure 5. Potential action planning alongside visual working memory allows for faster memory-guided behavior. Comparison of neural activity in C3
for (A) Load 1: fast versus slow; (B) Load 2: fast versus slow; and (C) Load 4: fast versus slow. Trials in each memory load and participant were
marked as fast or slow using a median split for decision times. For each comparison—(i) difference in time–frequency response in electrode C3,
aligned to memory encoding. Colors indicate percentage change between the conditions; black cluster outline indicates significant difference from a
cluster-based permutation analysis. (ii) Topographies of beta (15–25 Hz) percentage change for three different time windows during the memory
delay. (iii) Time-courses of beta (15–25 Hz) percentage change in electrode C3. Light-gray shadings around time-course indicate standard error; black
horizontal lines indicate significant clusters; black vertical lines indicate memory encoding (at 0 sec) and memory probe (at 2 sec).
Praamstra, 2014), which maybe have been associated with,
or reduced to, a single action plan. To this end, we sepa-
rated trials in Load 2 as follows: Trials were marked as
similar when the absolute difference in orientation
between two bars was smaller than 45°; trials were marked
as dissimilar when this difference was larger than 45°.
Next, we compared beta attenuation in C3 during the
memory delay for Load 2 compared with Load 4, while
this time distinguishing between Load 2-similar and
Load 2-dissimilar trials. For completeness, Load 2-
similar and -dissimilar were also compared directly.
In line with the data presented in Figure 3C, we
observed a significant relative attenuation of beta activity
in C3 when comparing Load 2-similar to Load 4. This
attenuation started soon after encoding (Figure 4A-i;
time–frequency map early cluster p = .0021, late cluster
p = .0021), had a left-central motor topography (Figure 4A-
ii), and showed a bimodal temporal profile (Figure 4A-iii;
time-course early cluster p = .0009, late cluster p =
.012). Crucially, when we exclusively included trials from
Load 2 that were marked as dissimilar (and thus, required
distinct potential actions) in our comparison, we still
observed a significant relative attenuation of beta activity
in C3 for Load 2-dissimilar compared with Load 4
(Figure 4B-i; time–frequency map early cluster p = .011,
late cluster p = .0005), with the same topographical
(Figure 4B-ii) and temporal (Figure 4B-iii; time-course
early cluster p = .0026, late cluster p = .0005) character-
istics as previously described. Moreover, when directly
comparing Load 2-similar and Load 2-dissimilar trials,
we did not observe a significant attenuation of beta
activity in C3 (Figure 4C).
These results show that the observed relative attenua-
tion of beta activity in Load 2 compared with Load 4 was
not merely driven by trials in Load 2 where both memo-
rized item orientations were similar. Accordingly, these
results indicate that multiple potential actions can be
planned during visual working memory, even when two
visual representations in working memory require distinct
actions for reproduction.
Nasrawi and van Ede
1541
Potential Action Planning alongside Visual Working
Memory Allows for Faster Memory-guided Behavior
Finally, we investigated whether potential action planning
during the memory delay might be beneficial for perfor-
mance, specifically for the speed of action implementation
after the working-memory delay. To examine this, we
marked trials as fast or slow, based on the onset time of
the orientation-reproduction report after the onset of
the memory probe. To this end, we performed a median
split separately for each memory-load condition and each
participant. We reasoned that, if preparedness for poten-
tial future actions is beneficial for the speed at which one
of these actions is later implemented, the degree of beta
attenuation in C3 after encoding and during retention
should be stronger in trials with faster decision times than
in those with slower decision time. Moreover, this should
only be the case in situations where action planning occurs
alongside visual working memory.
For trials in Load 1, we observed a significant relative
attenuation of beta activity in C3 for fast compared with
slow trials, starting soon after encoding (Figure 5A-i;
time–frequency map cluster p = .00029). This effect again
showed a left-central topography (Figure 5A-ii) and a
bimodal temporal profile (Figure 5A-iii; time-course early
cluster p = .011, late cluster p = .0004). Critically, when
performing the same median split analysis for Load 2,
we again observed a significant relative attenuation of
beta activity in C3 for fast compared with slow trials. This
effect also emerged soon after encoding (Figure 5B-i;
time–frequency map early cluster p < .0001, late cluster
p < .0001), had a left-central topography (Figure 5B-ii),
and a bimodal temporal profile (Figure 5B-iii; time-course
early cluster p = .0003, late cluster p = .0002). In contrast,
when comparing fast to slow trials in Load 4, we did not
observe such a significant attenuation of beta activity
during the memory delay (Figure 5C). Nonetheless, after
the memory delay, beta activity still became significantly
predictive of response-onset times (Figure 5C-i, iii; time–
frequency map cluster p = .0025; time-course cluster
p = .0031).
These results indicate that preparedness for multiple
potential future actions is beneficial for the speed at which
one of these actions is later implemented when we retain
one or two visual items, but not (or at least to a nonsig-
nificant extent) when we retain four visual items. This is
consistent with the finding (Figure 3C) that there is more
action planning during the memory delay in Load 2
(i.e., when action certainty is intermediate) than in Load 4
(i.e., when action certainty is low).
DISCUSSION
Although visual working memory allows us to retain infor-
mation from the past, it inherently serves the future. It
forms the bridge between perception and action, allowing
us to use detailed visual representations from memory to
guide potential future action (van Ede, 2020; Nobre &
Stokes, 2019; Rainer et al., 1999; Baddeley, 1992; Fuster
& Alexander, 1971). Critically, working memory often con-
tains not one but multiple pieces of information that may
become relevant for upcoming behavior (Ma, Husain, &
Bays, 2014; Cowan, 2001; Luck & Vogel, 1997), and it
may therefore serve not just the future, but the potential
future. Accordingly, we asked whether and how multiple
potential actions are planned during visual working mem-
ory, alongside the encoding and retention of multiple
visual items. We show an attenuation of beta activity in
central electrodes contralateral to the required response
hand that depends on the number of action possibilities
(strongest in Load 1, intermediate in Load 2, weakest in
Load 4). In Load 2, this effect occurred regardless of
whether both potential actions required a similar or dis-
similar manual response. Moreover, the degree of beta atten-
uation during the memory delay (in Load 1 and Load 2) was
predictive of the speed of the ensuing memory-guided
action after. These results are in line with the previously
envisioned graded scenario, whereby action planning
occurs alongside visual working memory, even when we
retain more than one visual item in anticipation of multiple
potential actions. This brings the concept of parallel action
planning (Cisek, 2007; Cisek & Kalaska, 2010) to the
domain of multi-item retention in visual working memory.
Previous research focusing on working memory of a sin-
gle visual item already showed that action planning of its
associated certain action occurs during the working-
memory delay (Boettcher et al., 2021; Schneider et al.,
2017). We directly build upon these findings, showing
that, alongside the encoding and retention of two visual
items, their associated potential actions are also planned
during the memory delay. This occurs despite the uncer-
tainty throughout the delay which of the two items will be
probed for action later. Moreover, the action planning sig-
nature we observed followed a similar bimodal activation
pattern as recently reported for single-item action plan-
ning during visual working memory (Boettcher et al.,
2021). Another recent study that focused on working
memory of multiple visual items previously showed that,
when either of two visual items in visual working memory
is probed for report, visual and motor information are
selected concurrently (van Ede et al., 2019). This
study—which focused on neural activity after the memory
delay—provided tentative evidence for the idea that paral-
lel action planning may co-occur with multi-item visual
retention. Here, we provide direct, complementary evi-
dence for this interpretation by focusing on EEG activity
in the delay-period itself.
The concept of parallel action planning has been around
for more than a decade. An early study showed that when
primates decide between two reaching actions toward dif-
ferent target locations, both actions are planned in parallel
at first, and one of these actions is selected for implemen-
tation later (Cisek & Kalaska, 2005). This has led to the
proposition of the affordance competition hypothesis
1542
Journal of Cognitive Neuroscience
Volume 34, Number 8
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
e
d
u
/
j
/
o
c
n
a
r
t
i
c
e
-
p
d
l
f
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
o
c
n
_
a
_
0
1
8
7
5
p
d
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
(Cisek, 2007), suggesting that behavior is a competition
between parallel representations of potential action affor-
dances. Recent work has argued that potential action
planning is also prevalent in humans when they plan and
perform reaching actions toward multiple potential loca-
tions ( Wong & Haith, 2017; Gallivan et al., 2015, 2016;
Grent-’t-Jong et al., 2013; Stewart, Baugh, Gallivan, &
Flanagan, 2013). Yet, so far, it has been considered pre-
dominantly in tasks without concurrent item retention in
working memory, whereby visual objects guide our actions.
We now provide evidence for the notion that multiple
potential actions, which are guided by detailed visual item
representations, are also planned alongside encoding and
retention during visual working memory. At the same
time, our data show that action planning is more profound
when the relevant action is fully known in advance (i.e., in
Load 1) as compared with when there are multiple poten-
tial courses of action (i.e., in Loads 2 and 4). This is in line
with other earlier research, which showed that beta atten-
uation is inversely related to the number of action possibil-
ities, being larger with higher action certainty, and vice
versa (Tzagarakis et al., 2010, 2015, 2021).
We interpret our data in the Load 2 condition as reflect-
ing the planning of multiple potential actions. However,
one might alternatively argue that participants plan one
single action selectively, even when they anticipate multi-
ple potential actions. Indeed, concluding the occurrence
of parallel action planning from trial-average data is
notoriously difficult (Dekleva, Kording, & Miller, 2018).
However, three aspects of our data argue against this
alternative interpretation. First, if this were true, it logically
follows that the degree of action planning in Loads 1 and 2
should be comparable (i.e., in both cases one action is
planned). In other words, we should observe no differ-
ence in beta activity during the memory delay in Load 1
compared with Load 2, contrasting our observations.
Second, in half of the trials in Load 2, the action that is
selectively planned should be the “incorrect” action that
is associated with the visual item that is not probed later.
This should be detrimental to decision times, as this would
require a switch of plans in this half of the trials. Yet, we
observed that larger beta attenuation in Load 2 during
the memory delay predicts faster decision times later,
suggesting that beta attenuation generally facilitated per-
formance. A third possibility is that multiple potential
actions in Load 2 are “merged into one” whenever two
visual items require a similar manual response. However,
we observed an attenuation of beta activity in Load 2 (com-
pared with Load 4) regardless of whether the two potential
actions required a similar or dissimilar manual response.
These data support out parallel-planning interpretation
by countering the possibility that one potential action is
selectively planned during the memory delay, even when
there are multiple potential courses of action.
We revealed the existence of planning multiple potential
actions during multi-item visual working memory—linking
the study of multi-item visual working memory to the vast
literature on motor planning. Although our work focused
on demonstrating action planning during multi-item visual
working memory, it leaves open the exact computational
mechanisms of retention and subsequent selection of
motor plans alongside multi-item visual working memory.
From the literature of motor planning research, the exis-
tence of parallel feedback loops for task and body states in
premotor and posterior parietal cortices has been previ-
ously demonstrated, and was argued to allow for the inte-
gration of the body within the task (e.g., Haar & Donchin,
2020; Shadmehr & Krakauer, 2008). Such computational
theories of motor control provide a relevant avenue for
future research on action planning alongside visual work-
ing memory. For example, do multiple parallel feedback
loops operate for each of the multiple potential action
plans that may be held available alongside multi-item
visual working memory? And how may such loops inte-
grate visual and motor information that are held in work-
ing memory concurrently?
Several lines of previous research have focused on bidi-
rectional influences between visual working memory and
action (for recent reviews, see the works of Heuer et al.,
2020; Olivers & Roelfsema, 2020; van Ede, 2020; Myers
et al., 2017). It has been shown, for example, that action
planning can benefit visual working-memory perfor-
mance: Memory performance is higher when the memo-
rized locations of visual memoranda and (planned) actions
are congruent, both for eye movements (Ohl & Rolfs,
2017, 2018, 2020; Hanning & Deubel, 2018; Hanning,
Jonikaitis, Deubel, & Szinte, 2016) and for manual actions
(Hanning & Deubel, 2018; Heuer & Schubö, 2017, 2018).
We focused on the reverse direction and considered how
retention during visual working memory may naturally
recruit action planning. Moreover, we show that the
degree of action planning during the working-memory
delay is beneficial for the speed of memory-guided action
afterward. At the same time, we found no relation between
action planning and the precision of the memory-guided
orientation-reproduction report. This suggests that action
planning in our task did not necessarily influence the qual-
ity of visual working-memory representations. Instead,
planning multiple potential actions may have occurred
alongside visual working-memory retention, allowing
both action plans and visual representations to be readily
available for fast response-implementation after the
memory delay.
Although visual working memory allows us to retain
information that is no longer physically available to us,
visual working memory is not merely a temporary storage
mechanism. Instead, we often rely on representations in
visual working memory to guide and plan potential future
actions, even—or perhaps especially—under varying
degrees of action certainty. This is useful in our everyday
lives where we are often faced with multiple sources of
visual information that we need to retain and that each
afford distinct potential actions. Being prepared for more
than one action scenario allows us to cope with action
Nasrawi and van Ede
1543
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
e
d
u
/
j
/
o
c
n
a
r
t
i
c
e
-
p
d
l
f
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
o
c
n
_
a
_
0
1
8
7
5
p
d
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
uncertainty in a dynamically unfolding world (Cisek &
Kalaska, 2010) and, ultimately, allows us to act rapidly
when working-memory contents become relevant for
behavior. The findings presented here provide evidence
that multiple potential actions can be planned alongside
visual working memory. They also reinforce the idea that
visual working memory is ultimately future-oriented.
Acknowledgments
This research was supported by a Newton International Fellow-
ship from the Royal Society and the British Academy
(NF140330), a Marie Skłodowska-Curie Fellowship from the
European Commission (ACCESS2WM), and an ERC Starting
Grant from the European Research Council (MEMTICIPATION,
850636); awarded to F. v. E. Data were collected while F. v. E.
was a fellow in the Brain & Cognition lab of Anna C. (Kia)
Nobre. The authors wish to thank Kia Nobre for her valuable
exchanges throughout the lifetime of the project, Sammi
Chekroud for his assistance during the collection of the data,
and Baiwei Liu and Merlijn Breunesse for their valuable com-
ments on the manuscript.
Reprint requests should be sent to Rose Nasrawi, Institute for
Brain and Behavior Amsterdam, Department of Experimental
and Applied Psychology, Vrije Universiteit Amsterdam,
1081BT, Amsterdam, The Netherlands, or via e-mail: r.nasrawi
@vu.nl or Freek van Ede, Institute for Brain and Behavior
Amsterdam, Department of Experimental and Applied Psychol-
ogy, Vrije Universiteit Amsterdam, 1081BT, Amsterdam, The
Netherlands, or via e-mail: freek.van.ede@vu.nl.
Code and Data Availability
All the code for the EEG and behavioral analysis are
a v a i l a b l e a t h t t p s : / / g i t h u b . c o m / r o s e n a s r a w i
/ MultiplePotentialActionsVWM. The data can be made
available upon reasonable request.
Author Contributions
Rose Nasrawi: Conceptualization; Formal analysis; Investi-
gation; Visualization; Writing—Original draft; Writing—
Review & editing. Freek van Ede: Conceptualization; Data
curation; Funding acquisition; Investigation; Supervision;
Writing—Original draft; Writing—Review & editing.
Funding Information
Freek van Ede, British Academy (https://dx.doi.org/10
.13039/501100000286), grant number: NF140330. Freek
van Ede, European Commission (https://dx.doi.org/10
.13039/501100000780), grant number: ACCESS2WM.
Freek van Ede, European Research Council, grant number:
MEMTICIPATION, 850636.
Diversity in Citation Practices
Retrospective analysis of the citations in every article pub-
lished in this journal from 2010 to 2021 reveals a persistent
pattern of gender imbalance: Although the proportions
of authorship teams (categorized by estimated gender
identification of first author/ last author) publishing in
the Journal of Cognitive Neuroscience ( JoCN ) during
this period were M(an)/ M = .407, W(oman)/ M = .32,
M/ W = .115, and W/ W = .159, the comparable propor-
tions for the articles that these authorship teams cited
were M/ M = .549, W/ M = .257, M/ W = .109, and
W/ W = .085 (Postle and Fulvio, JoCN, 34:1, pp. 1–3).
Consequently, JoCN encourages all authors to consider
gender balance explicitly when selecting which articles
to cite and gives them the opportunity to report their
article’s gender citation balance.
REFERENCES
Baddeley, A. (1992). Working memory. Science, 255, 556–559.
https://doi.org/10.1126/science.1736359, PubMed: 1736359
Boettcher, S. E. P., Gresch, D., Nobre, A. C., & van Ede, F.
(2021). Output planning at the input stage in visual working
memory. Science Advances, 7, 8212–8236. https://doi.org/10
.1126/sciadv.abe8212, PubMed: 33762341
Cisek, P. (2007). Cortical mechanisms of action selection:
The affordance competition hypothesis. Philosophical
Transactions of the Royal Society of London, Series B:
Biological Sciences, 362, 1585–1599. https://doi.org/10.1098
/rstb.2007.2054, PubMed: 17428779
Cisek, P., & Kalaska, J. F. (2005). Neural correlates of reaching
decisions in dorsal premotor cortex: Specification of multiple
direction choices and final selection of action. Neuron,
45, 801–814. https://doi.org/10.1016/j.neuron.2005.01.027,
PubMed: 15748854
Cisek, P., & Kalaska, J. F. (2010). Neural mechanisms for
interacting with a world full of action choices. Annual Review
of Neuroscience, 33, 269–298. https://doi.org/10.1146
/annurev.neuro.051508.135409, PubMed: 20345247
Cowan, N. (2001). The magical number 4 in short-term
memory: A reconsideration of mental storage capacity.
Behavioral and Brain Sciences, 24, 87–114. https://doi.org
/10.1017/S0140525X01003922, PubMed: 11515286
Dekleva, B. M., Kording, K. P., & Miller, L. E. (2018). Single
reach plans in dorsal premotor cortex during a two-target
task. Nature Communications, 9, 3556. https://doi.org/10
.1038/s41467-018-05959-y, PubMed: 30177686
D’Esposito, M., & Postle, B. R. (2015). The cognitive
neuroscience of working memory. Annual Review of
Psychology, 66, 115–142. https://doi.org/10.1146/annurev
-psych-010814-015031, PubMed: 25251486
Fuster, J. M., & Alexander, G. E. (1971). Neuron activity related
to short-term semory. Science, 173, 652–654. https://doi.org
/10.1126/science.173.3997.652, PubMed: 4998337
Gallivan, J. P., Barton, K. S., Chapman, C. S., Wolpert, D. M., &
Flanagan, J. R. (2015). Action plan co-optimization reveals
the parallel encoding of competing reach movements.
Nature Communications, 6, 7428. https://doi.org/10.1038
/ncomms8428, PubMed: 26130029
Gallivan, J. P., Bowman, N. A. R., Chapman, C. S., Wolpert, D. M.,
& Flanagan, J. R. (2016). The sequential encoding of
competing action goals involves dynamic restructuring of
motor plans in working memory. Journal of Neurophysiology,
115, 3113–3122. https://doi.org/10.1152/jn.00951.2015,
PubMed: 27030738
Grent-’t-Jong, T., Oostenveld, R., Jensen, O., Medendorp, W. P.,
& Praamstra, P. (2013). Oscillatory dynamics of response
competition in human sensorimotor cortex. Neuroimage, 83,
1544
Journal of Cognitive Neuroscience
Volume 34, Number 8
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
e
d
u
/
j
/
o
c
n
a
r
t
i
c
e
-
p
d
l
f
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
o
c
n
_
a
_
0
1
8
7
5
p
d
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
27–34. https://doi.org/10.1016/j.neuroimage.2013.06.051,
PubMed: 23796548
Grent-’t-Jong, T., Oostenveld, R., Jensen, O., Medendorp, W. P.,
& Praamstra, P. (2014). Competitive interactions in
sensorimotor cortex: Oscillations express separation between
alternative movement targets. Journal of Neurophysiology,
112, 224–232. https://doi.org/10.1152/jn.00127.2014,
PubMed: 24760786
Haar, S., & Donchin, O. (2020). A revised computational
neuroanatomy for motor control. Journal of Cognitive
Neuroscience, 32, 1823–1836. https://doi.org/10.1162/jocn_a
_01602, PubMed: 32644882
Hanning, N. M., & Deubel, H. (2018). Independent effects of
eye and hand movements on visual working memory.
Frontiers in Systems Neuroscience, 12, 37. https://doi.org/10
.3389/fnsys.2018.00037, PubMed: 30174593
Hanning, N. M., Jonikaitis, D., Deubel, H., & Szinte, M. (2016).
Oculomotor selection underlies feature retention in visual
working memory. Journal of Neurophysiology, 115,
1071–1076. https://doi.org/10.1152/jn.00927.2015, PubMed:
26581875
Harrison, S. A., & Tong, F. (2009). Decoding reveals the
contents of visual working memory in early visual areas.
Nature, 458, 632–635. https://doi.org/10.1038/nature07832,
PubMed: 19225460
Heuer, A., Ohl, S., & Rolfs, M. (2020). Memory for action: A
functional view of selection in visual working memory. Visual
Cognition, 28, 388–400. https://doi.org/10.1080/13506285
.2020.1764156
Heuer, A., & Schubö, A. (2017). Selective weighting of
action-related feature dimensions in visual working memory.
Psychonomic Bulletin and Review, 24, 1129–1134. https://
doi.org/10.3758/s13423-016-1209-0, PubMed: 27896631
Heuer, A., & Schubö, A. (2018). Separate and combined effects
of action relevance and motivational value on visual working
memory. Journal of Vision, 18, 14. https://doi.org/10.1167/18
.5.14, PubMed: 29904789
Hope, R. M. (2013). Rmisc: Ryan Miscellaneous. R package
version 1.5. https://CRAN.R-project.org/package=Rmisc
Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working
memory for features and conjunctions. Nature, 390,
279–281. https://doi.org/10.1038/36846, PubMed: 9384378
Luck, S. J., & Vogel, E. K. (2013). Visual working memory
capacity: From psychophysics and neurobiology to individual
differences. Trends in Cognitive Sciences, 17, 391–400.
https://doi.org/10.1016/j.tics.2013.06.006, PubMed: 23850263
Ma, W. J., Husain, M., & Bays, P. M. (2014). Changing concepts
of working memory. Nature Neuroscience, 17, 347–356.
https://doi.org/10.1038/nn.3655, PubMed: 24569831
Maris, E., & Oostenveld, R. (2007). Nonparametric statistical
testing of EEG- and MEG-data. Journal of Neuroscience
Methods, 164, 177–190. https://doi.org/10.1016/j.jneumeth
.2007.03.024, PubMed: 17517438
Mcfarland, D. J., Miner, L. A., Vaughan, T. M., & Wolpaw, J. R.
(2000). Mu and beta rhythm topographies during motor
imagery and actual movements. Brain Topography, 12,
177–186. https://doi.org/10.1023/A:1023437823106, PubMed:
10791681
Myers, N. E., Stokes, M. G., & Nobre, A. C. (2017). Prioritizing
information during working memory: Beyond sustained
internal attention. Trends in Cognitive Sciences, 21, 449–461.
https://doi.org/10.1016/j.tics.2017.03.010, PubMed: 28454719
Neuper, C., Wörtz, M., & Pfurtscheller, G. (2006). ERD/ERS
patterns reflecting sensorimotor activation and deactivation.
Progress in Brain Research, 159, 211–222. https://doi.org/10
.1016/S0079-6123(06)59014-4, PubMed: 17071233
Nobre, A. C., & Stokes, M. G. (2019). Premembering
experience: A hierarchy of time-scales for proactive attention.
Neuron, 104, 132–146. https://doi.org/10.1016/j.neuron.2019
.08.030, PubMed: 31600510
Ohl, S., & Rolfs, M. (2017). Saccadic eye movements impose a
natural bottleneck on visual short-term memory. Journal
of Experimental Psychology: Learning, Memory, and
Cognition, 43, 736–748. https://doi.org/10.1037/xlm0000338,
PubMed: 27762577
Ohl, S., & Rolfs, M. (2018). Saccadic selection of stabilized items
in visuospatial working memory. Consciousness and
Cognition, 64, 32–44. https://doi.org/10.1016/j.concog.2018
.06.016, PubMed: 30231988
Ohl, S., & Rolfs, M. (2020). Bold moves: Inevitable saccadic
selection in visual short-term memory. Journal of
Vision, 20, 11. https://doi.org/10.1167/jov.20.2.11, PubMed:
32106297
Olivers, C. N. L., & Roelfsema, P. R. (2020). Attention for action
in visual working memory. Cortex, 131, 179–194. https://doi
.org/10.1016/j.cortex.2020.07.011, PubMed: 32892152
Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J. M. (2011).
FieldTrip: Open source software for advanced analysis
of MEG, EEG, and invasive electrophysiological data.
Computational Intelligence and Neuroscience, 2011,
156869. https://doi.org/10.1155/2011/156869, PubMed:
21253357
R Core Team. (2020). R: A language and environment for
statistical computing. R Foundation for Statistical
Computing, Vienna, Austria. https://www.R-project.org/
Rainer, G., Rao, S. C., & Miller, E. K. (1999). Prospective
coding for objects in primate prefrontal cortex. Journal
of Neuroscience, 19, 5493–5505. https://doi.org/10.1523
/JNEUROSCI.19-13-05493.1999, PubMed: 10377358
Salmelin, R., & Hari, R. (1994). Spatiotemporal characteristics
of sensorimotor neuromagnetic rhythms related to
thumb movement. Neuroscience, 60, 537–550.
https://doi.org/10.1016/0306-4522(94)90263-1, PubMed:
8072694
Schneegans, S., & Bays, P. M. (2017). Neural architecture
for feature binding in visual working memory. Journal
of Neuroscience, 37, 3913–3925. https://doi.org/10.1523
/JNEUROSCI.3493-16.2017, PubMed: 28270569
Schneider, D., Barth, A., & Wascher, E. (2017). On the
contribution of motor planning to the retroactive cuing
benefit in working memory: Evidence by mu and beta
oscillatory activity in the EEG. Neuroimage, 162, 73–85.
https://doi.org/10.1016/j.neuroimage.2017.08.057, PubMed:
28847491
Serences, J. T. (2016). Neural mechanisms of information
storage in visual short-term memory. Vision Research, 128,
53–67. https://doi.org/10.1016/j.visres.2016.09.010, PubMed:
27668990
Shadmehr, R., & Krakauer, J. W. (2008). A computational
neuroanatomy for motor control. Experimental Brain
Research, 185, 359–381. https://doi.org/10.1007/s00221-008
-1280-5, PubMed: 18251019
Stewart, B. M., Baugh, L. A., Gallivan, J. P., & Flanagan, J. R.
(2013). Simultaneous encoding of the direction and
orientation of potential targets during reach planning:
Evidence of multiple competing reach plans. Journal of
Neurophysiology, 110, 807–816. https://doi.org/10.1152/jn
.00131.2013, PubMed: 23699052
Tzagarakis, C., Ince, N. F., Leuthold, A. C., & Pellizzer, G. (2010).
Beta-band activity during motor planning reflects
response uncertainty. Journal of Neuroscience, 30,
11270–11277. https://doi.org/10.1523/JNEUROSCI.6026-09
.2010, PubMed: 20739547
Tzagarakis, C., West, S., & Pellizzer, G. (2015). Brain oscillatory
activity during motor preparation: Effect of directional
uncertainty on beta, but not alpha, frequency band. Frontiers
Nasrawi and van Ede
1545
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
e
d
u
/
j
/
o
c
n
a
r
t
i
c
e
-
p
d
l
f
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
o
c
n
_
a
_
0
1
8
7
5
p
d
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
in Neuroscience, 9, 246. https://doi.org/10.3389/fnins.2015
.00246, PubMed: 26257597
Tzagarakis, C., West, S., & Pellizzer, G. (2021). Neural encoding
of the reliability of directional information during the
preparation of targeted movements. BioRxiv. https://doi.org
/10.1101/2020.09.17.302000
van Ede, F. (2020). Visual working memory and action:
Functional links and bi-directional influences. Visual
Cognition, 28, 401–413. https://doi.org/10.1080/13506285
.2020.1759744, PubMed: 33223921
van Ede, F., Chekroud, S. R., Stokes, M. G., & Nobre, A. C.
van Ede, F., Niklaus, M., & Nobre, A. C. (2017). Temporal
expectations guide dynamic prioritization in visual working
memory through attenuated α oscillations. Journal of
Neuroscience, 37, 437–445. https://doi.org/10.1523
/JNEUROSCI.2272-16.2016, PubMed: 28077721
van Wijk, B. C. M., Daffertshofer, A., Roach, N., & Praamstra, P.
(2009). A role of beta oscillatory synchrony in biasing
response competition? Cerebral Cortex, 19, 1294–1302.
https://doi.org/10.1093/cercor/bhn174, PubMed: 18836098
Wickham, H. (2016). ggplot2: Elegant Graphics for Data
Analysis. New York: Springer-Verlag.
(2019). Concurrent visual and motor selection during visual
working memory guided action. Nature Neuroscience, 22,
477–483. https://doi.org/10.1038/s41593-018-0335-6, PubMed:
30718904
Wong, A. L., & Haith, A. M. (2017). Motor planning flexibly
optimizes performance under uncertainty about task goals.
Nature Communications, 8, 14624. https://doi.org/10.1038
/ncomms14624, PubMed: 28256513
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
e
d
u
/
j
/
o
c
n
a
r
t
i
c
e
-
p
d
l
f
/
/
/
3
4
8
1
5
3
4
2
0
3
3
1
8
7
/
/
j
o
c
n
_
a
_
0
1
8
7
5
p
d
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
1546
Journal of Cognitive Neuroscience
Volume 34, Number 8
