Rapid and Accumulated Modulation of - IA de Investigación especializada en el MIT

Modulación rápida y acumulada de
Action-Effects on Action

Liyu Cao, Wilfried Kunde, and Barbara Haendel

Abstracto

■ Auditory feedback to a keypress is used in many devices to
facilitate the motor output. The timing of auditory feedback is
known to have an impact on the motor output, yet it is not known
if a keypress action can be modulated on-line by an auditory feed-
back or how quick an auditory feedback can influence an ongoing
keypress. Además, it is not clear if the prediction of auditory
feedback already changes the early phase of a keypress action in-
dependent of sensory feedback, which would suggest that such
prediction changes the motor plan. En el estudio actual, participar-
pants pressed a touch-sensitive device with auditory feedback in a
self-paced manner. The auditory feedback was given either after a
corto (60 mseg) or long (160 mseg) delay, and the delay was either

predictable or not. Our results showed that the keypress peak
force was modulated by the amount of auditory feedback delay
even when the delay was unpredictable, thus demonstrating an
on-line modulation effect. The latency of the on-line modulation
was suggested to be as low as 70 mseg, indicating a very fast sen-
sory to motor mapping circuit in the brain. When the auditory
feedback delay was predictable, a change in the very early phase
of keypress motor output was found, suggesting that the predic-
tion of sensory feedback is crucial to motor control. Por lo tanto,
even a simple keypress action contains rich motor dynamics,
which depend on expected as well as on-line perceived sensory
comentario. ■

INTRODUCCIÓN

Actions are accompanied by sensory feedback. Para examen-
por ejemplo, we hear our own voices when we speak. The impor-
tance of auditory feedback in regulating motor control
has been recognized since the 1950s using the delayed
auditory feedback paradigm (van Vugt & Tillmann, 2015;
Sasisekaran, 2012; Pfordresher & Dalla Bella, 2011;
Chase, harvey, Standfast, Rapin, & suton, 1959; Kalmus,
Denes, & Fry, 1955; Negro, 1950; Sotavento, 1950). In most natural
situations, the auditory feedback from actions is usually in-
stant (a counterexample is when we speak in a large room;
see Black, 1950). In the delayed auditory feedback para-
digm, the latency of auditory feedback is experimentally
manipulated so that the auditory feedback follows actions
with a small amount of delay. A typical finding is that
delayed auditory feedback has adverse effects on the per-
formance of actions in various systems. Por ejemplo, cuando
people hear their own voice with a 200-msec delay, ellos
speak louder and slower.

In a manual tapping task, delayed auditory feedback leads
to increased tapping peak force (p.ej., Chase et al., 1959). En
some recent demonstrations, participants were asked to
make keypresses, and they pressed more lightly when a
sound followed the press without any delay as compared
to when the sound was delayed or when no sound was pres-
ent (Cao, Steinborn, Podría, & Haendel, 2020; Neszmélyi &
Horváth, 2018; see also Chase, Rapin, Gilden, suton, &

Julius-Maximilians-Universität Würzburg, Alemania

Guilfoyle, 1961). Although it is clear that finger tapping
behavior can be modulated by auditory feedback, the un-
derlying mechanism is not clear. A motor response starts
with a motor program and can be modified by on-line feed-
atrás (Shadmehr, Herrero, & Krakauer, 2010; todorov, 2004;
Wolpert, Miall, & Loco, 1998). Two questions are relevant
aquí. The first question is how quick the auditory feedback
can modulate an ongoing keypress, or stated as whether an
on-line modulation is possible for a keypress. A keypress is a
very quick movement, which lasts about 300 mseg. De este modo, él
may be programmed to be rather ballistic, and an on-line
modulation is not possible. The answer to the question can-
not be derived from existing studies as the feedback delay in
a series of keypresses is constant. When a series of key-
presses is followed by auditory feedback of the same delay,
eso es, the feedback delay is predictable, the prediction of
the auditory feedback may change the motor output
(Shadmehr et al., 2010; Flanagan, Vetter, Johansson, &
Wolpert, 2003). Por lo tanto, it is not known if a keypress
peak force increase because of a delayed auditory feedback
results from the auditory feedback of the current keypress
or learning from previous trials. Sin embargo, if an unpredict-
able auditory feedback delay also leads to a modulation of
keypress peak force (p.ej., the force is higher for delay vs. No
delay or for long delay vs. short delay), an on-line modula-
tion process can be assumed to be at work.

The second question is closely related to the first ques-
ción, eso es, whether a motor-prediction-related signal can
be detected in the keypress behavior when the auditory
feedback is constantly delayed. Motor prediction refers to

Revista de neurociencia cognitiva 32:12, páginas. 2333–2341
https://doi.org/10.1162/jocn_a_01633

D
oh
w
norte
yo
oh
a
d
mi
d

/
j

t
t

F
/

i
t
.

:
/
/

F
r
oh
metro
D
oh
h
w
t
t
norte
pag
oh
:
a
/
d
/
mi
metro
d
i
F
r
t
oh
pag
metro
r
C
h
.
s
pag
i
yo
d
v
i
r
mi
mi
r
C
t
C
.
metro
h
a
i
mi
r
d
.
tu
C
oh
oh
metro
C
norte
/
j
a
oh
r
t
C
i
C
norte
mi
/
–
a
pag
r
d
t
i
3
C
2
yo
1
mi
2
–
2
pag
3
d
3
F
3
/
2
3
0
2
1
/
4
1
1
2
2
/
0
2
3
oh
3
C
3
norte
_
/
a
1
_
8
0
6
1
2
6
2
3
6
3
6
pag
/
d
j
oh
b
C
y
norte
gramo
_
tu
a
mi
_
s
0
t
1
oh
6
norte
3
0
3
8
.
S
pag
d
mi
F
pag
mi
b
metro
y
b
mi
gramo
r
tu
2
mi
0
s
2
t
3

/
j

oh
norte

0
5

METRO
a
y

2
0
2
1

the process of predicting the sensory feedback from own
comportamiento (reafferent inputs), and it is deemed as a part of
motor program ( Wolpert & Flanagan, 2001). In the time
course of a keypress, the dependent variable of keypress
peak force used in existing studies occurs too late to be a
motor-prediction-related signal if it receives on-line mod-
ulación. For motor-prediction-related signals, one needs to
focus on the very early phase of motor output in which an
influence of sensory feedback is physiologically impossible
(because of sensory feedback delay). Por lo tanto, a success-
ful detection of a modulation in the early phase of keypress
force trajectory because of feedback predictability reflects
a change in motor program and the related motor predic-
ción. This may open up a possibility to study motor control
processes within keypressing behavior, a very common
response in cognitive studies, in a very fine-grained fashion.
To answer the above two questions, a keypress experi-
ment was performed with a manipulation of the predictability
of auditory feedback delay. The keypress force was recorded
as the dependent variable. We show that the keypress peak
force is modulated by the auditory feedback delay when the
delay is predictable, and most importantly, also when the
delay is unpredictable (evidence for on-line modulation).
We also show that the early phase of keypress force trajectory
is modulated by the predictability of the auditory feedback
delay (evidence for a change in motor program).

MÉTODOS

Participantes

Forty-nine healthy participants (28 women; edad media =
28.7; DE = 8.8) volunteered to participate in the study. De
all the participants, 23 were colleagues and friends, y el
rest were recruited from a local participant pool. Only exter-
nal participants received monetary compensation for partic-
ipation. Forty-one participants were included in the peak
force analysis and 39 in the force trajectory analysis (ver
Data Exclusion section). Informed consent was obtained
from all participants. The study was approved by the local
ethics committee (Institute of Psychology, Faculty for
Human Sciences, University of Würzburg; Project No. GZ
2018-27) and conducted in accordance with the
Declaration of Helsinki and the General Data Protection
Regulation in Europe.

Tarea, Estímulos, and Apparatus

Participants sat in front of a laptop and completed a self-
paced keypress task. The keypress, in our case, was made
against a flat surface similarly to making a keypress on a touch
screen without any movement of the key itself (Figura 1A).
The auditory feedback after a keypress was manipulated
(Figura 1B). There were two predictable auditory feedback

D
oh
w
norte
yo
oh
a
d
mi
d

/
j

t
t

F
/

i
t
.

:
/
/

/
j

oh
norte

0
5

METRO
a
y

2
0
2
1

Cifra 1. Experiment details. (A) The key used in the study (izquierda), an example of force trajectory of a keypress (middle), and the empirically measured
relationship between the sensor measured raw force value and force value in Newton (bien). The three dots in the example force trajectory (middle) representar
the force start point (es decir., the last point with a force value of 0 before the keypress force escalates; Punto 1), the keypress start point (raw force value at 200; Punto 2),
and the peak force point (Punto 3). Each circle in the right plot shows a measurement point in assessing the mapping between sensor-measured raw force and
force in Newton. A polynomial fitting curve is overlaid. (B) Experimental conditions. Each participant completed the experiment in one of the four orders, con el
included number of participants in the final analysis shown on the right. Each box represents a testing block (50 ensayos). NS = No sound; Mix = Mix60 and Mix160.

2334

Revista de neurociencia cognitiva

Volumen 32, Número 12

delay conditions, Fix60 and Fix160. In the Fix60 condition
(100 keypresses), a keypress was always followed by a
60-msec delayed sound. In the Fix160 condition (100 key-
presses), a keypress was always followed by a 160-msec
delayed sound. Fix60 and Fix160 conditions were presented
in separate testing blocks so that the same delay was pre-
sented in a testing block. There were two unpredictable
auditory feedback delay conditions, Mix60 and Mix160. En
the Mix60 condition (100 keypresses), a keypress was always
followed by a 60-msec delayed sound. In the Mix160 condi-
ción (100 keypresses), a keypress was always followed by a
160-msec delayed sound. The Mix60 and Mix160 conditions
were presented in the same testing block (‘Mix’ En figura
1B). To make sure that the feedback delay was unpredict-
capaz, the feedback delays in this testing block were ran-
domized using the randperm function in MATLAB (El
MathWorks). The presentation order of predictable and un-
predictable auditory feedback conditions was counterba-
lanced among participants. Además, a NoSound
condición (100 keypresses) was included in separate testing
bloques, in which no auditory feedback was provided after a
keypress. The NoSound condition served as a baseline con-
dition in which the general effect of auditory feedback on
keypress can be compared (Neszmélyi & Horváth, 2018).
Para cada participante, 500 keypresses (or trials) were col-
lected in blocks of 50 keypresses (10 blocks in total), y un
self-paced break was given between blocks. During each
testing block, participants were asked to fixate the screen
center, where the block number was constantly presented.
Participants were told to make a keypress after about every
2 segundo. Del 41 participants included in the final analysis
(see Data Exclusion section), the average interkeypress
interval was 2.54 (DE = 0.98) sec in NoSound condition,
2.34 (DE = 0.99) in mix condition, 2.36 (DE = 1.07) en
Fix60 condition, y 2.52 (DE = 1.03) in Fix160 condition.
No significant difference was found between conditions in
the interkeypress interval with a within-subject one-way
ANOVA (F(3, 120) = 1.47, pag = .24, ηp

2 = 0.04).

The auditory feedback was a 1000-Hz tone (50 msec in
duración; 5-msec rise/fall ramp), which was presented via
headphones (Vic Firth SIH1) at a comfortable volume level
(same volume for all participants). Participants wore the
headphones throughout the experiment (es decir., incluido
the NoSound condition). The force information of a key-
press was collected by a force sensing resistor (FSR;
Modelo 402, Interlink Electronics Inc.). The FSR, cual es
also the key being pressed, has a circular active area with
a diameter of ∼1.3 cm. A 10-kΩ resistor, from which the
voltage information was collected, was connected in series
with the FSR with an input voltage of 5V. With increased
force applied on the active area of the FSR, the FSR would
reduce its resistance, which would result in increased cur-
rent flow in the circuit and increased voltage on the 10-kΩ
resistor. Analog to digital conversion was achieved using a
microcontroller ATMega16 (10-bit precision; Microchip
Technology Inc.). Each digit in the recorded data repre-
sents ∼5 mV (5V/1023). The force was sampled at

500 Hz. The relationship between applied force and
recorded voltage (es decir., the raw FSR force value) is mono-
tonic but not linear (Figura 1A, bien). During the testing,
participants put the index finger of their dominant hand
directly on the FSR (softly in contact). The index finger
was always in contact with the FSR, resting very softly on
the sensor before and after each keypress so that no force
was detected by the FSR, even during the break between
bloques. This contact with the FSR was most often below
the FSR force detection threshold (alrededor 0.3 norte). Él
brought two advantages with the finger always being in
contact with the device: (a) No auditory feedback from
the keypress itself (es decir., physical vibrations) could be heard
by participants, y (b) the relative position between the
finger and the FSR did not change throughout the exper-
mento, which made the FSR output comparable among all
keypresses. In conditions with auditory feedback, the tim-
ing of auditory feedback was controlled by the FSR output.
A sound was played after the FSR output reached a thresh-
old value of 200 (∼0.87 N), which was referred to as the
keypress start point. A keypress was made by briskly de-
pressing the index finger. The measured delay between
the keypress start and physical sound wave output was
60.6 mseg (DE = 6.0; 100 sound measurements) y
161.3 mseg (DE = 6.8; 100 sound measurements) para el
intended 60- and 160-msec delay, respectivamente.

Participants were given a few practice keypresses
(between four and eight keypresses) to get familiarized
with the key (FSR) before data collection. During the prac-
tice, participants were instructed on the correct way to
make a keypress. Participants always had to keep the finger
in contact with the key, briskly depress the key, y luego
wait for about 2 sec before the next keypress. Each success-
ful keypress was followed by a letter “k” on the screen im-
mediately after the keypress start point (no visual feedback
was given in the formal testing). At the end of the formal
pruebas, participants were asked to make three additional
strong keypresses. These three keypresses gave larger
peak FSR output values than keypresses from any testing
condición, indicating that the keypress force collected dur-
ing the testing was in the working range of FSR. After the
data collection, participants were asked if they realized
anything wrong with respect to the sound. No one men-
tioned the delay of the sound.

Data Exclusion

For each keypress, the associated force trajectory and the
peak force were extracted (see Figure 1A for an example).
In our experimental setup, the data stream of FSR output
was not available for controlling the auditory feedback until
1 sec after the start of each trial. Por lo tanto, in conditions
with auditory feedback, if a keypress was made immedi-
ately after the preceding keypress (dentro 1 segundo), el en-
tended timing manipulation of auditory feedback based
on the FSR output cannot be achieved. These trials were
excluded. Trials that had double peaks in the keypress

Cao, Podría, and Haendel

2335

D
oh
w
norte
yo
oh
a
d
mi
d

/
j

t
t

F
/

i
t
.

:
/
/

/
j

oh
norte

0
5

METRO
a
y

2
0
2
1

force trajectory with at least 40 msec of raw force value
lower than 200 between the two peaks were also excluded
as there was a pause during the keypress, and the manip-
ulation of auditory feedback timing may also not be accu-
tasa. Para cada participante, the number of excluded trials
was calculated for each testing block. A testing block
was excluded if more than 20% of trials (es decir., 10 ensayos)
were excluded. A participant was excluded if no blocks
remained after the block exclusion procedure for one or
more auditory feedback conditions (Mix60, Mix160, Fix160,
Fix60). In total, eight participants were excluded after this
step, eso es, 41 participants were included in the peak
force analysis. Among the remaining participants, 13
bloques (3.2% of all blocks) were excluded from seven
Participantes. De término medio, 96.3 (DE = 12.0) trials were left
in the NoSound condition, 93.0 (DE = 18.1) trials in Mix60,
91.8 (DE = 16.6) trials in Mix160, 93.8 (DE = 15.4) trials in
Fix60, y 97.3 (DE = 3.3) trials in Fix160. No significant
difference in the number of trials was found between con-
ditions (one-way within-subject ANOVA: F(4, 160) = 1.93,
pag = .14, ηp

2 = 0.05).

In the force trajectory analysis, the force trajectory of
each trial was aligned to the force start point and averaged
entre ensayos. For each keypress included in the peak force
análisis, the force start point was defined as the last time
point with a force value of 0 within the 400-msec time win-
dow ending at the peak force point. If no points with a force
value of 0 were found, the trial was excluded. Two more
participants were excluded from further analysis as no more
than 15 trials were left in at least one condition. el promedio
number of trials included in the force trajectory analysis was
86.5 (DE = 22.9) in the NoSound condition, 88.1 (DE = 17.7)
in Mix60, 84.8 (DE = 17.9) in Mix160, 85.7 (DE = 20.6) en
Fix60, y 81.8 (DE = 23.8) in Fix160. There was no signifi-
cant difference in the number of trials between conditions
in the force trajectory analysis (one-way within-subject
ANOVA: F(4, 152) = 0.83, pag = .47, ηp

2 = 0.02).

Análisis de los datos

All the results were reported with force values converted to
Newton in the main text. Sin embargo, qualitatively similar
results were obtained from the data analysis with raw FSR
force values. Because the keypress peak force was not nor-
mally distributed, robust statistical measurements based
on randomization were used, and individual data were
always presented for clarity. The data analysis was per-
formed with MATLAB (The MathWorks Inc.). The p value
de .05 was taken as the statistical significance cutoff, y
corrections for multiple comparisons were made and
mentioned where relevant.

To answer Question 1, the critical test was to compare
the keypress peak force between Mix60 and Mix160. If a
difference in the auditory feedback delay can lead to a
difference in the peak force when the delay was not pre-
dictable, the difference in the peak force could only be be-
cause of an on-line modulation. We performed a two-level

nonparametric test. Primero, a randomization test was made to
test whether Mix160 had a higher peak force than Mix60
for each participant so that each participant had a p value
(referred to as “prand” later). One-tailed test was used here
because it has been shown that a long auditory feedback
delay led to a higher keypress force than a short keypress
delay (Neszmélyi & Horváth, 2018; Ruhm & Cooper,
1962). The rationale of the randomization test is that, si
there are no differences between two conditions, values from
the two conditions should be exchangeable. Por lo tanto, un
individual prand value was obtained in the following four
steps: 1. Compare Mix160 and Mix60 using an unpaired
t test, and keep the t value as the original t. 2. Take all
the force values from Mix160 and Mix60, and randomize
the order of those force values before assigning them back
to the two conditions. Compare the two conditions after
randomization, and get a t value as a randomized t. 3.
Repeat Step 2 para 1,000 times so that there are 1,000
randomized t values. 4. Check the proportion of random-
ized t values that are bigger than the original t, cual es
then the individual prand value. We report all prand values
as prand < 0.001 when prand value is 0 after 1,000 random- izations. Note that, in this case, a small prand value indicates that Mix160 is higher than Mix60, and a large prand value indicates the opposite. Second, the group level prand value for Mix160 having higher peak force than Mix60 was obtained by comparing the individual prand values to 0.5 (chance level) with the randomization test. The procedure was similar to the procedure of getting individual prand values except that the randomization only took place within participants, that is, the condition relabeling was performed for each participant separately so that each participant always had the same pair of data points. The same random- ization test was also used to make comparisons between, for example, Fix160 and Fix60 (see the Results section). To answer Question 2, the average keypress force trajec- tory was analyzed. This analysis allows us to examine the early phase of motor output that is free from the influence of sensory feedback. The force trajectory of each keypress was selected from 200 msec before the force start point to 800 msec after. Within-subject two-tailed t tests were made on the average force trajectory at each time point (500 time points in total) of the force trajectory between Mix60 and Mix160, between Fix60 and Fix160, between the average of Mix60 and Mix160 (“Mix” in Figure 3A legend) and the average of Fix60 and Fix160 (“Fix” in Figure 3A legend), and between the NoSound condition and the average of the four conditions with auditory feedback (“WithSound” in Figure 3A legend). Multiple comparisons over time points were corrected for each between-condition com- parison using a cluster based permutation method with a cutoff p value at .05 (Maris & Oostenveld, 2007). Data and Code Availability The original data and MATLAB analysis code are freely avail- able on-line at: doi.org/10.6084/m9.figshare.12146271.v1. 2336 Journal of Cognitive Neuroscience Volume 32, Number 12 D o w n l o a d e d l l / / / / j f / t t i t . : / / f r o m D o h w t t n p o : a / d / e m d i f r t o p m r c h . s p i l d v i r e e r c t c . m h a i e r d . u c o o m c n / j a o r t c i c n e / - a p r d t i 3 c 2 l 1 e 2 - 2 p 3 d 3 f 3 / 2 3 0 2 1 / 4 1 1 2 2 / 0 2 3 o 3 c 3 n _ / a 1 _ 8 0 6 1 2 6 2 3 6 3 6 p / d j o b c y n g _ u a e _ s 0 t 1 o 6 n 3 0 3 8 . S p d e f p e b m y b e g r u 2 e 0 s 2 t 3 / j t / . f o n 0 5 M a y 2 0 2 1 RESULTS The Classic Delayed Auditory Feedback Effect The manipulation of the auditory feedback from keypress had a significant influence on the keypress (Figure 2, left). Consistent with previous studies on the effect of delayed auditory feedback, all the four conditions in which an auditory feedback followed the keypress had significantly weaker peak force than the condition in which no auditory feedback was provided (all the four between-condition comparisons had prand < 0.001 with one-tailed randomiza- tion test; NoSound: mean = 5.56 N, SD = 6.21 N; Mix60: mean = 2.07 N, SD = 3.05 N; Mix160: mean = 2.19 N, SD = 3.34 N; Fix60: mean = 1.49 N, SD = 1.17 N; Fix160: mean = 1.79 N, SD = 1.05 N). A long fixed delay (Fix160) led to higher keypress peak force than a short fixed delay (Fix60; prand < 0.001, one-tailed randomization test; Figure 2, middle). On-Line Modulation Revealed by Peak Force Analysis To test the effect of an on-line modulation of auditory feed- back on the keypress action, a comparison of peak force was made between the 60-msec delayed sound and the 160-msec delayed sound when both sounds were pre- sented in a random order (i.e., unpredictable). Indeed, a 160-msec delayed sound still led to a stronger keypress force than a 60-msec delayed sound in this case (Mix160 >
Mix60, prand < 0.001, one-tailed randomization test; Figure 2, right), which showed that auditory feedback could influence the triggering action on the fly. Excluding the two participants showing a large difference between Mix160 and Mix60 (Figure 2, right) had no influence on the statistical results. Similar results were obtained with the data analysis on raw force values. However, the on-line modulation from auditory feedback can only occur if the auditory feedback arrived at a time point earlier than the peak force. Therefore, a sanity check was made. The mean duration between the keypress start and the peak force point (Figure 1A) was 95.53 msec (Mix60) and 108.29 msec (Mix160; 176.45 msec in NoSound, 83.21 msec in Fix60, and 121.25 msec in Fix160). The 95.53-msec delay of peak force in the Mix60 condition may seem a bit short to be in- fluenced by a 60-msec delayed sound (35.53-msec reaction latency from sound onset to motor response), but it was impossible for a 160-msec delayed sound to have a direct influence on the peak force at 108.29 msec in the Mix160 condition. This peak force delay was an average of all the 41 participants included in the data analysis, and not every- one showed an on-line modulation effect. If the participants showing a strong on-line adjustment effect were considered ( prand value smaller than 0.05; n = 7), the average peak force delay was 133.02 msec in the Mix60 and 183.01 msec in the Mix160. If participants were grouped based on the prand value, half of the participants with small prand values ( prand < 0.3; n = 20) had an average peak force delay of 112.84 msec in the Mix60 and 133.28 msec in the Mix160; the other half with large prand values ( prand > 0.3; norte = 21)
had an average peak force delay of 79.03 msec in the Mix60
y 84.49 msec in the Mix160. Por lo tanto, there is enough
time for a 60-msec delayed sound to exert its on-line mod-
ulation effect on motor output. The individual difference in
the on-line modulation effect strongly depends on the
idiosyncratic keypress behavior.

Accumulated Modulation Revealed by Force
Trajectory Analysis

The modulation effect of delayed auditory feedback on
peak force was stronger when the delay was predictable
(Fix160-Fix60; mean = 0.30 norte; DE = 0.46 norte) than when
the delay was not predictable (Mix160-Mix60; mean =
0.13 norte; DE = 0.38 norte, prand = 0.04, two-tailed randomization
prueba). This suggests that there are other sources of modula-
tion in the Fix conditions besides on-line modulation. Nosotros
tested the idea that the extra modulation effect might even
be detectable in the early phase of the keypress force trajec-
tory beyond the influence of any sensory feedback. Hacer
este, single trial keypress force trajectories were aligned

D
oh
w
norte
yo
oh
a
d
mi
d

/
j

F
/

t
t

i
t
.

:
/
/

/
j

oh
norte

0
5

METRO
a
y

2
0
2
1

Cifra 2. Peak force results. Mean peak force in each condition (izquierda), and individual peak force comparison between Fix60 and Fix160 (middle) y
between Mix60 and Mix160 (bien). Each line represents the data of an individual participant (note that the mean force value from the contrasted
conditions was subtracted for clear visualization, es decir., the demeaned force as shown on the y axis). Black lines show a difference in the same direction
as the group average, and red lines show the opposite. Vertical lines indicate ± 1 standard error. NoSound: keypress without auditory feedback; Mix:
160- and 60-msec delayed auditory feedback were mixed; Fix60 and Fix160: solo 60- or 160-msec delayed sound was presented in one testing block.

Cao, Podría, and Haendel

2337

D
oh
w
norte
yo
oh
a
d
mi
d

/
j

t
t

F
/

i
t
.

:
/
/

/
j

Cifra 3. The results of force trajectory analysis. (A) On the left, the average keypress force trajectory from each condition between −200 and 800 mseg,
with time 0 being the force start point. The horizontal black bars show the time windows of a significant difference in the comparison indicated on the
left after applying the cluster correction for multiple comparisons. Numbers above the horizontal bar indicate the time window covered by the bar.
Notablemente, sound predictability led to a very early modulation of keypress force trajectory (Mix vs Fix, Fix60 vs. Fix160). The empty triangle shows the
average onset time of the 60-msec delayed sound calculated from the force start point, and the filled triangle shows the onset time of the 160-msec
delayed sound. On the right, a clearer plot of the first 100 msec from the force start point. (B) A difference plot is shown for better visualization of the
comparisons made in (A). Por ejemplo, the blue line shows the result of subtracting the average force trajectory in Mix60 from Mix160. (C–F) Individual
results from four significant time windows. Each line represents an individual participant and shows the average force value in the selected time window
(the mean force value from the contrasted conditions was subtracted for clear visualization, es decir., the demeaned force as shown on the y axis). Black lines
show a difference in the same direction as the group average, and red lines show the opposite. NoSound: keypress without auditory feedback;
WithSound: the average of the keypress with auditory feedback conditions; Mix60 and Mix160: 60- and 160-msec delayed sounds were presented in
aleatorio; Fix60 and Fix160: solo 60- or 160-msec delayed sound was presented in one testing block; Mix: the average of Mix160 and Mix60; Fix: el
average of Fix160 and Fix60.

oh
norte

0
5

METRO
a
y

2
0
2
1

2338

Revista de neurociencia cognitiva

Volumen 32, Número 12

to the force start point (Figura 1A, middle), averaged like
an event-related-potential study in electroencephalo-
gram, and compared between conditions (Figura 3A,
B). Curiosamente, significant differences in the average
force trajectory were found between the Fix60 and the
Fix160 conditions in two clusters. In the first cluster
(starting at 20 msec after the force start point), el
Fix60 condition had a higher force than the Fix160 con-
condición (see Figure 3B for a difference plot). Sin embargo, en-
dividual results suggested that this pattern was not stable
as more than 45% of participants showed the effect in the
other direction (Figura 3C). Además, there was no
such cluster when the data analysis was performed with
the raw force values. The second cluster (starting at
72 mseg) showed a reliable pattern that the Fix160 con-
dition had a higher force than the Fix60 condition (38 de 39
participants showed results in this direction; Figura 3D).
Because the starting point of a cluster cannot be taken
as the point where statistical difference emerges (ver
Sassenhagen & Draschkow, 2019), the Fix60 and Fix160
conditions were compared with a within-subject t test at
each time point in the entire time window. This was
followed by a false discovery rate adjustment. The first sig-
nificant point from this analysis was at 80 mseg, which was
much earlier than the delivery of the 60-msec delayed au-
ditory feedback. The average delivery time of the 60-msec
delayed sound calculated from the force start point was at
acerca de 117 mseg (the average sound delivery time was
marked with triangles in Figure 3A, B). Por lo tanto, la diferencia-
ference in force trajectory between the Fix60 and Fix160
conditions started much before the sound delivery and
cannot be explained by auditory feedback. The early differ-
ence can only be because of different motor commands.
The same comparison between the Mix60 and Mix160
conditions led to a significant cluster in the time window
entre 182 y 520 mseg. The earliest difference point
as identified through a whole time window comparison
followed by a false discovery rate adjustment was at
192 mseg, eso es, after the delivery of the 60-msec delayed
sound. This difference can be interpreted as a feedback
modulation, with a short delayed sound leading to a
weaker force as compared to a long delayed sound. Más
strikingly, when the average force trajectory of the Mix con-
ditions (Mix60, Mix160) was compared to the average force
trajectory of the Fix conditions (Fix60, Fix160), a significant
cluster was found between 4 y 114 mseg, eso es, en el
very early stage of the keypress action. As can be seen on
the right of Figure 3A, force trajectories from the condi-
tions with auditory feedback formed two bundles in the
early phase. The first bundle, with a shallow slope, era
formed by the keypresses with predictable feedback timing
(Fix60, Fix160). The second bundle, with a steep slope, era
formed by the keypresses with unpredictable feedback
timing (Mix60, Mix160). Similar results were obtained
with the data analysis on raw force values. Results from
the force speed (slope of force value over time) y fuerza
aceleración (slope of force speed over time) analysis also

showed a similar pattern, that the Fix conditions had lower
speed and acceleration than the Mix conditions in the very
beginning from the keypress force start (results not
mostrado). When there was no auditory feedback (NoSound
condición), the keypress force was higher than when the
auditory feedback was provided, across the entire analysis
time window.

DISCUSIÓN

The human motor system generates actions, cual, dentro
given task constraints, are optimized with respect to
certain variables such as effort, comfort, or variability
(todorov, 2004). When interacting with force sensitive
devices, where only the exerted force matters, it seems
natural that the motor system aims to reduce unnecessary
forces and, de este modo, minimize metabolic effort. It has been
shown that the latency and the availability of auditory feed-
back plays an important role in regulating the motor output
during a keypress (Cao et al., 2020; Neszmélyi & Horváth,
2018; van Vugt & Tillmann, 2015; Pfordresher & Dalla Bella,
2011; Chase et al., 1959, 1961; Kalmus et al., 1955). En el
estudio actual, we showed that auditory feedback can exert
an on-line modulation on keypress behavior before the
keypress is complete. We also showed that predictable
delays in auditory feedback can lead to a change in motor
programa.

On-line control suggests that force output is increased
until extra sensory feedback, which signals that enough
force has been exerted, comes in. This process was clearly
confirmed by demonstrating that the keypress peak force
was reduced with early (60 mseg) compared to late
(160 mseg) auditory feedback when both were equally un-
predictable. In motor control studies, there is an ongoing
debate on how quick sensory feedback can influence the
motor output. Por ejemplo, human-goal-directed move-
mentos (p.ej., pointing to an object with the finger: eliot,
Helsen, & Chua, 2001; Woodworth, 1899) probably require
a latency of at least 100 msec before a visual signal can in-
fluence the motor output (Elliott et al., 2001). In the cur-
rent study, the average latency from the onset of auditory
feedback to the time point of keypress peak force, where a
significant modulation from auditory feedback was found,
era 73 mseg (133–60) in the Mix60 condition for the seven
participants showing a strong on-line modulation effect.
This suggests that the latency for an on-line modulation
of motor output from sensory feedback may be overesti-
acoplado, at least in the auditory domain. Curiosamente, nuestro
results are consistent with reports of fast orienting re-
sponse to visual stimulus. Neck muscles in monkeys
(Corneil & Munoz, 2014; Corneil, Olivier, & Munoz,
2004) and arm muscles in humans (Pruszynski et al.,
2010) show a very short latency response to visual stimuli
(55–95 msec in monkeys). A very recent study in monkeys
also showed that at least for the initiation of smooth pursuit
eye movements, a latency as short as 50 msec was enough

Cao, Podría, and Haendel

2339

D
oh
w
norte
yo
oh
a
d
mi
d

/
j

F
/

t
t

i
t
.

:
/
/

/
j

oh
norte

0
5

METRO
a
y

2
0
2
1

for a visual signal to show an impact on the oculomotor
producción (Buonocore, Skinner, & Hafed, 2019). The current
estudiar, a lo mejor de nuestro conocimiento, is the first to demon-
strate that a self-generated auditory feedback can have an
almost comparably fast influence on motor output (finger
movimiento).

The motor program change was observed in the early
phase of keypress force trajectory. A long predictable audi-
tory feedback delay (160 mseg) led to a steeper increase of
keypress force than a short predictable auditory feedback
delay (60 mseg) in a time window far earlier than the arrival
of auditory feedback. Por lo tanto, the early force difference
cannot be explained by sensory feedback and can only be
explained by a change in the motor program. The motor
program change is related to the predictability of the audi-
tory feedback delay. Supporting this, the early force differ-
ence was not observed when the auditory feedback delay
was not predictable (Mix60 vs. Mix160). There may be at
least two explanations for the motor program change.
One explanation is that the effect from on-line modulation
accumulates, and the motor program change is a passive
adaptation to the auditory feedback delay. According to
this explanation, the order of the steepness of early force
trajectory should be (from high to low): Fix160, Mix160,
Mix60, Fix60. This is because a long feedback delay leads
to a higher force than a short feedback delay, and a higher
force should be associated with a steeper slope (see the
force trajectory of NoSound condition; Figura 3A). Fix160
had a 160-msec feedback delay in all trials and should have
the steepest slope. Mix160 also received a 160-msec feed-
back delay in all trials but was interrupted by some 60-msec
feedback trials in between, so it should have the second
steepest slope. Following the same rationale, it may not
be difficult to figure out that Mix60 and Fix60 should have
the third and the fourth steepest slope, respectivamente.
Sin embargo, the results clearly showed that the force trajecto-
ries of both the Mix160 and Mix60 conditions were above
the Fix160 and Fix60 conditions (Figura 3A, bien).
Además, the average of the Mix160 and Mix60 condi-
tions had a statistically significant steeper slope than the
average of the Fix160 and Fix60 conditions in the time
window between 4 y 114 msec from the keypress force
comenzar. Por lo tanto, an explanation of passive motor adapta-
tion is highly unlikely for the early keypress force slope
cambiar (note that we do not exclude the possibility that
a passive motor adaptation may exist). The other explana-
tion is that the motor program change is a result of an
active motor adaptation, which involves the prediction of
the auditory feedback. This explanation is consistent with
neurophysiological findings that the brain can learn the
auditory feedback delay (Elijah, Le Pelley, & Whitford,
2018; Cao, Veniero, Thut, & Bruto, 2017) and that the
prediction of sensory feedback is crucial to motor control
(Podría, Koch, & Hoffmann, 2004; Flanagan et al., 2003;
Wolpert & Flanagan, 2001). Por lo tanto, the change in the
early phase of keypress force trajectory may well be a sign
of an updated motor prediction.

The approach of averaging force trajectories over trials is
not without caveats. Por ejemplo, the difference between
the Mix60 and Mix160 conditions in the force trajectory
only showed up from 182 mseg, which was after the peak
of the average force trajectory. It was already shown that
the peak force was different between the two conditions
from previous analyses. De este modo, trial averaging smears the
timing information of between-condition differences
because of an intertrial variation in the force trajectory.
Sin embargo, the early component of the force trajectory can
be considered as a realization of the motor command
behind, and trial-averaging should work to reveal the signal
in a way similar to the ERP technique in human neuroim-
aging studies. There are surely other approaches to the
analysis of force profiles (p.ej., looking at the skewness
and kurtosis of each keypress, see Ulrich, Rinkenauer, &
Molinero, 1998). The trial-averaging approach, as we dis-
cussed, is theoretically sound for the purpose of the cur-
rent study. We conclude that the force profile difference
between the Fix60 and Fix160 conditions, between the
average of the Mix conditions and the average of the Fix
condiciones, in the very early time window clearly indicates
an open-loop component of motor control, mientras que el
force profile difference between the Mix60 and Mix160
conditions should be interpreted as a sign of a closed-loop
control (cf. Wing, 1977).

En general, our findings suggest that the prediction of the
auditory feedback changes the motor program, which indi-
cates that the prediction of sensory feedback is crucial to
motor control. Al mismo tiempo, unpredictable auditory
feedback can lead to an on-line modulation of motor out-
put at a very low latency (∼70 msec).

Expresiones de gratitud

We thank Normann Mangold for technical support, Lisa von
Boros for assistance with data collection, Michael Steinborn for
helpful comments on an earlier draft, and Victoria Nicholls
for English editing. l. C. y B. h. were supported by the
European Research Council (grant 677819, awarded to B. h.).

Reprint requests should be sent to Liyu Cao, Department of
Psicología (III), Julius-Maximilians-Universität Würzburg,
Röntgenring 11, 97070, Würzburg, Alemania, o por correo electrónico:
liyu.cao@uni-wuerzburg.de.

REFERENCIAS

Negro, j. W.. (1950). The effect of room characteristics upon vocal
intensity and rate. Journal of the Acoustical Society of America,
22, 174–176. DOI: https://doi.org/10.1121/1.1906585

Buonocore, A., Skinner, J., & Hafed, z. METRO. (2019). Eye position
error influence over “open-loop” smooth pursuit initiation.
Revista de neurociencia, 39, 2709–2721. DOI: https://doi.
org/10.1523/JNEUROSCI.2178-18.2019, PMID: 30709895,
PMCID: PMC6445996

Cao, l., Steinborn, METRO., Podría, w., & Haendel, B. (2020). Acción
force modulates action binding: Evidence for a multisensory
information integration explanation. Cerebro experimental
Investigación, 238, 2019–2029. DOI: https://doi.org/10.1007/
s00221-020-05861-4, PMID: 32617882, PMCID: PMC7438375

2340

Revista de neurociencia cognitiva

Volumen 32, Número 12

D
oh
w
norte
yo
oh
a
d
mi
d

/
j

F
/

t
t

i
t
.

:
/
/

/
j

oh
norte

0
5

METRO
a
y

2
0
2
1

Cao, l., Veniero, D., Thut, GRAMO., & Bruto, j. (2017). Role of the
cerebellum in adaptation to delayed action effects. Actual
Biología, 27, 2442–2451. DOI: https://doi.org/10.1016/j.
cub.2017.06.074, PMID: 28781049, PMCID: PMC5571438
Chase, R. A., harvey, S., Standfast, S., Rapin, I., & suton, S. (1959).
Comparison of the effects of delayed auditory feedback on
speech and key tapping. Ciencia, 129, 903–904. DOI: https://
doi.org/10.1126/science.129.3353.903, PMID: 13635035
Chase, R. A., Rapin, I., Gilden, l., suton, S., & Guilfoyle, GRAMO.

(1961). II Sensory feedback influences on keytapping motor
tareas. Revista trimestral de psicología experimental, 13,
153–167. DOI: https://doi.org/10.1080/17470216108416488
Corneil, B. D., & Munoz, D. PAG. (2014). Overt responses during
covert orienting. Neurona, 82, 1230–1243. DOI: https://doi.
org/10.1016/j.neuron.2014.05.040, PMID: 24945769

Corneil, B. D., Olivier, MI., & Munoz, D. PAG. (2004). Visual responses
on neck muscles reveal selective gating that prevents express
saccades. Neurona, 42, 831–841. DOI: https://doi.org/10.1016/
S0896-6273(04)00267-3, PMID: 15182721

Elijah, R. B., Le Pelley, METRO. MI., & Whitford, t. j. (2018). Act now, play
más tarde: Temporal expectations regarding the onset of self-initiated
sensations can be modified with behavioral training. Diario de
Neurociencia Cognitiva, 30, 1145–1156. DOI: https://doi.org/
10.1162/jocn_a_01269, PMID: 29668396

eliot, D., Helsen, W.. F., & Chua, R. (2001). A century later:

Woodworth’s (1899) two-component model of goal-directed
aiming. Boletín Psicológico, 127, 342–357. DOI: https://
doi.org/10.1037/0033-2909.127.3.342, PMID: 11393300
Flanagan, j. r., Vetter, PAG., Johansson, R. S., & Wolpert, D. METRO.

(2003). Prediction precedes control in motor learning. Actual
Biología, 13, 146–150. DOI: https://doi.org/10.1016/S0960-9822
(03)00007-1, PMID: 12546789

Kalmus, h., Denes, PAG., & Fry, D. B. (1955). Effect of delayed acoustic
feed-back on some non-vocal activities. Naturaleza, 175, 1078.
DOI: https://doi.org/10.1038/1751078a0, PMID: 14394110

Podría, w., Koch, I., & Hoffmann, j. (2004). Anticipated action
effects affect the selection, initiation, and execution of actions.
Revista trimestral de psicología experimental, Section A:
Human Experimental Psychology, 57, 87–106. DOI: https://
doi.org/10.1080/02724980343000143, PMID: 14681005

Sotavento, B. S. (1950). Some effects of side-tone delay. Journal of the
Acoustical Society of America, 22, 639–640. DOI: https://doi.
org/10.1121/1.1906665

Maris, MI., & Oostenveld, R. (2007). Nonparametric statistical testing
of EEG- and MEG-data. Journal of Neuroscience Methods, 164,
177–190. DOI: https://doi.org/10.1016/j.jneumeth.2007.03.024,
PMID: 17517438

Neszmélyi, B., & Horváth, j. (2018). Temporal constraints in
the use of auditory action effects for motor optimization.
Revista de Psicología Experimental: Percepcion humana
y rendimiento, 44, 1815–1829. DOI: https://doi.org/
10.1037/xhp0000571, PMID: 30091635

Pfordresher, PAG. P., & Dalla Bella, S. (2011). Delayed auditory

feedback and movement. Revista de Psicología Experimental:

Percepción y desempeño humanos, 37, 566–579. DOI:
https://doi.org/10.1037/a0021487, PMID: 21463087

Pruszynski, j. A., Rey, GRAMO. l., Boisse, l., Scott, S. h., Flanagan, j. r.,
& Munoz, D. PAG. (2010). Stimulus-locked responses on human
arm muscles reveal a rapid neural pathway linking visual input
to arm motor output. European Journal of Neuroscience, 32,
1049–1057. DOI: https://doi.org/10.1111/j.1460-9568.2010.07380.x,
PMID: 20726884

Ruhm, h. B., & Cooper, W.. A., Jr. (1962). Low sensation level
effects of pure-tone delayed auditory feedback. Diario de
Speech and Hearing Research, 5, 185–193. DOI: https://doi.
org/10.1044/jshr.0502.185, PMID: 14495207

Sasisekaran, j. (2012). Effects of delayed auditory feedback on
speech kinematics in fluent speakers. Perceptual and Motor
Habilidades, 115, 845–864. DOI: https://doi.org/10.2466/15.22.
PMS.115.6.845-864, PMID: 23409597, PMCID: PMC3718456

Sassenhagen, J., & Draschkow, D. (2019). Cluster-based
permutation tests of MEG/EEG data do not establish
significance of effect latency or location. Psychophysiology,
56, e13335. DOI: https://doi.org/10.1111/psyp.13335,
PMID: 30657176

Shadmehr, r., Herrero, METRO. A., & Krakauer, j. W.. (2010). Error
correction, sensory prediction, and adaptation in motor
control. Revisión anual de neurociencia, 33, 89–108. DOI:
https://doi.org/10.1146/annurev-neuro-060909-153135,
PMID: 20367317

todorov, mi. (2004). Optimality principles in sensorimotor control.

Neurociencia de la naturaleza, 7, 907–915. DOI: https://doi.org/
10.1038/nn1309, PMID: 15332089, PMCID: PMC1488877
Ulrich, r., Rinkenauer, GRAMO., & Molinero, j. (1998). Effects of stimulus
duration and intensity on simple reaction time and response
fuerza. Revista de Psicología Experimental: Humano
Perception and Performance, 24, 915–928. DOI: https://doi.
org/10.1037/0096-1523.24.3.915, PMID: 9627425

van Vugt, F. T., & Tillmann, B. (2015). Auditory feedback in

error-based learning of motor regularity. Brain Research, 1606,
54–67. DOI: https://doi.org/10.1016/j.brainres.2015.02.026,
PMID: 25721795

Wing, A. METRO. (1977). Perturbations of auditory feedback delay
and the timing of movement. Revista de experimentación
Psicología: Percepción y desempeño humanos, 3,
175–186. DOI: https://doi.org/10.1037/0096-1523.3.2.175,
PMID: 864391

Wolpert, D. METRO., & Flanagan, j. R. (2001). Motor prediction.
Biología actual, 11, R729–R732. DOI: https://doi.org/
10.1016/S0960-9822(01)00432-8, PMID: 11566114

Wolpert, D. METRO., Miall, R. C., & Loco, METRO. (1998). Internal models
in the cerebellum. Tendencias en Ciencias Cognitivas, 2, 338–347.
DOI: https://doi.org/10.1016/S1364-6613(98)01221-2, PMID:
21227230

Woodworth, R. S. (1899). Accuracy of voluntary movement.
Revisión psicológica: Monograph Supplements, 3, 1–119.
DOI: https://doi.org/10.1097/00005053-189912000-00005,
https://doi.org/10.1037/h0092992

Cao, Podría, and Haendel

2341

D
oh
w
norte
yo
oh
a
d
mi
d

/
j

F
/

t
t

i
t
.

:
/
/

/
j

oh
norte

0
5

METRO
a
y

2
0
2
1 Rapid and Accumulated Modulation of image

Descargar PDF