LETTER
Communicated by Stefano Panzeri
Bayesian Integration in a Spiking Neural System
for Sensorimotor Control
Massimo Grillo
massimo.grillo97@gmail.com
Nearlab, Department of Electronics, Information and Bioengineering,
Politecnico di Milano, 20133, Milan, Italy
Alice Geminiani
alice.geminiani@unipv.it
Department of Brain and Behavioral Sciences, University of Pavia 27100, Italy
Cristiano Alessandro
cri.alessandro@gmail.com
Department of Brain and Behavioral Sciences, University of Pavia 27100, Italy, and
School of Medicine and Surgery/Sport and Exercise Science, University of
Milano-Bicocca, 20126 Milan, Italy
Egidio D’Angelo
egidiougo.dangelo@unipv.it
Department of Brain and Behavioral Sciences, University of Pavia 27100, Italy, and
Brain Connectivity Center, IRCCS Mondino Foundation, Pavia 27100, Italy
Alessandra Pedrocchi
alessandra.pedrocchi@polimi.it
Nearlab, Department of Electronics, Information and Bioengineering,
Politecnico di Milano, 20133, Milan, Italy
Claudia Casellato
claudia.casellato@unipv.it
Department of Brain and Behavioral Sciences, University of Pavia 27100, Italy
The brain continuously estimates the state of body and environment,
with specific regions that are thought to act as Bayesian estimator, op-
timally integrating noisy and delayed sensory feedback with sensory
predictions generated by the cerebellum. In control theory, Bayesian
estimators are usually implemented using high-level representations. In
this work, we designed a new spike-based computational model of a
Bayesian estimator. The state estimator receives spiking activity from two
Alessandra Pedrocchi and Claudia Casellato are co–last authors.
Neural Computation 34, 1893–1914 (2022) © 2022 Massachusetts Institute of Technology.
https://doi.org/10.1162/neco_a_01525
Published under a Creative Commons
Attribution 4.0 International (CC BY 4.0) license.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1894
M. Grillo et al.
neural populations encoding the sensory feedback and the cerebellar pre-
diction, and it continuously computes the spike variability within each
population as a reliability index of the signal these populations encode.
The state estimator output encodes the current state estimate. We simu-
lated a reaching task at different stages of cerebellar learning. The activity
of the sensory feedback neurons encoded a noisy version of the trajectory
after actual movement, with an almost constant intrapopulation spiking
variability. Conversely, the activity of the cerebellar output neurons de-
pended on the phase of the learning process. Before learning, they fired
at their baseline not encoding any relevant information, and the variabil-
ity was set to be higher than that of the sensory feedback (more reliable,
albeit delayed). When learning was complete, their activity encoded the
trajectory before the actual execution, providing an accurate sensory pre-
diction; in this case, the variability was set to be lower than that of the sen-
sory feedback. The state estimator model optimally integrated the neural
activities of the afferent populations, so that the output state estimate was
primarily driven by sensory feedback in prelearning and by the cerebel-
lar prediction in postlearning. It was able to deal even with more com-
plex scenarios, for example, by shifting the dominant source during the
movement execution if information availability suddenly changed. The
proposed tool will be a critical block within integrated spiking, brain-
inspired control systems for simulations of sensorimotor tasks.
1 Introduction
Humans can perform complex movements that require the coordination of
many muscles and joints, automatically and unthinkingly (Thach, 1998).
Even in noisy conditions (e.g., in foggy or dark environments) our brain can
integrate multiple available sensory information with previous knowledge
in order to estimate the current state of the body and the environment and
to use this estimate to generate appropriate motor commands (Alessandro
et al., 2016; Kawato, 1999; Shadmehr & Krakauer, 2008a; Wolpert, Good-
body, & Husain, 1998). This process of state estimation is essential to deal
with the inherent delays in our sensory systems. As an example, during a
tennis match, the information about the position of the ball extracted from
the visual input becomes available to the central nervous system (CNS)
with a delay of about 100 ms (Wolpert & Ghahramani, 2000). If not appro-
priately compensated, such noisy and delayed information would lead to
inappropriate motor commands, eventually resulting in unsatisfactory
movement execution. Instead, the CNS can filter the noisy sensory feed-
back and combine this information with fast sensory predictions that com-
pensate for delays due to movement execution and feedback reafferences
(Wolpert et al., 1998). In the previous example, the tennis player can suc-
cessfully estimate the spin of the ball using reliable predictions generated
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Spiking State Estimator
1895
by internal models of the ball and of the body dynamics, ultimately gener-
ating appropriate motor commands, not just compensating for delays but
also in case of uncertain sensory feedbacks (e.g., foggy day) (Wolpert &
Ghahramani, 2000).
Several brain areas are involved in this process. There is large evi-
dence that the CNS computes sensory predictions by means of the cerebel-
lum (Kawato, 1999; Kawato & Gomi, 1992; Shadmehr & Krakauer, 2008a;
Wolpert et al., 1998). The cerebellum, acting as a forward model, learns
to predict sensory consequences of actions using the planned motor com-
mands received from the motor cortex through the efference copy (Miall &
Wolpert, 1996; Popa & Ebner, 2019; Stein, 2009). Therefore, it facilitates fast
and smooth coordination and aids accurate and well-timed sensorimotor
execution and adaptation (Miall & Wolpert, 1996; Thach, 1998). Cerebellar
predictions are integrated with the actual sensory information in a process
that computes reliable estimates of the state of the body interacting with the
environment. This process is thought to be carried out by the parietal cor-
tex (Shadmehr & Krakauer, 2008a), which receives, through the thalamus,
projections from both the deep cerebellar nuclei (output of the cerebellum;
Palesi et al., 2014) and peripheral sensory structures (Dum, Levinthal, &
Strick, 2009). Accordingly, it has been suggested that damages to the pari-
etal cortex cause performance errors compatible with the inability to com-
pute state estimates (Wolpert et al., 1998; Wolpert & Ghahramani, 2000).
Over the years, several theories have been proposed to explain how the
CNS combines sensory information originating from the periphery and the
cerebellum. It has been suggested that the CNS integrates these informa-
tion sources through a process of Kalman filtering or Bayesian integration
(Körding & Wolpert, 2004; Shadmehr & Krakauer, 2008a). These ideas have
been investigated using computational models (de Xivry, Coppe, Blohm, &
Lefèvre, 2013; Körding & Wolpert, 2004). However, given their abstract and
high-level computational nature, these models provide little insight into the
biological features of the underlying neural mechanisms. A more biologi-
cally plausible model of the neuromotor control system has been developed
by Deneve et al. (2007), who implemented an optimal sensorimotor integra-
tion model for state estimation using recurrent neural networks of cortical
circuits. However, how these processes may be implemented in spike-based
systems remains an unresolved issue.
Here, we designed a spike-based state estimator model that operates
with the naturalistic time coding representations typical of neuronal activ-
ity. The model receives spiking signals from two afferent neuronal pop-
ulations, emulating the cerebellar output and the sensory feedback, and
computes the spike synchronicity within each of these populations. These
measures are used to define relative reliability of the incoming signals, al-
lowing the computation of an optimal Bayesian estimation of the body
state (Barak, 2017). The state estimator model was tested at different stages
of the cerebellar learning process, emulated by modulating the spiking
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1896
M. Grillo et al.
variability of the cerebellar output. Indeed, recent studies suggest that
a strong increase of within-population synchronicity (i.e., a decrease of
within-population variability) is a neural correlate of learning (Chervyakov
et al., 2016; Sedaghat-Nejad, Pi, Hage, Fakharian, & Shadmehr, 2022). Ac-
cordingly, the synchronicity among cerebellar neurons is maximal at the
end of the adaptation (Wagner et al., 2019).
Furthermore, we tested our model in more complex scenarios: (1) with
a sudden interruption of the sensory feedback during movement execution
in a middle-learning condition and (2) with an unexpected perturbation in
postlearning. In all of these contexts, our model generated appropriate state
estimates. In the future, we will integrate the state estimator proposed here
with whole-brain models, including a realistic spiking network of the plas-
tic cerebellar circuit (de Schepper et al., 2021; Geminiani et al., 2018; Gemini-
ani, Pedrocchi, D’Angelo, & Casellato, 2019), paving the way of bio-inspired
multiarea brain simulations in closed-loop controllers.
2 Materials and Methods
2.1 System Design. According to a well-established sensorimotor con-
trol model (Shadmehr & Krakauer, 2008a; Wolpert & Ghahramani, 2004),
the state estimator block computes reliable estimates about the current state
of the system. To this end, it integrates the sensory feedback originating
from the receptors in the body with the sensory predictions computed by
the cerebellum. While the former is affected by the inherent delays, the cere-
bellum provides, as a forward model, sensory predictions by exploiting an
efference copy of the motor commands originating in the primary motor
cortex (Miall & Wolpert, 1996). We therefore designed our state estimator
block to receive inputs from two afferent neuronal populations (see Figure
1A). The neural activity of these afferent populations encoded the expected
sensory feedback and cerebellar predictions during a reaching task, simu-
lated by the movement of a point mass from a starting position to a desired
target, in bidimensional space (mass of 1 kg). In order to simulate the delay
of the sensory feedback signals, the trajectory of the point mass was de-
layed by 100 ms before translating it into the neural activity of the sensory
population (“executed trajectory”).
2.2 Neural Populations and Signal Encoding. The three neural
populations—cerebellar output neurons, sensory feedback neurons, and
state estimator neurons—were implemented as spiking neural networks
(SNN) (Ghosh-Dastidar & Adeli, 2009) in NEST (Eppler et al., 2009; Jor-
dan et al., 2019). The sensory feedback and cerebellar output neurons were
modeled as Poisson single-point neurons whose spike trains were gener-
ated by applying a frequency coding strategy based on a Poisson distri-
bution of spikes (Brette, Roland, Panzeri, & Graham, 2015). The controlled
body was a point mass that could move over a bidimensional space:
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Spiking State Estimator
1897
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
Figure 1: State estimator design. (A) Sensory reafferences from periphery and
cerebellum project to the state estimator block, which in turn generates the op-
timal state signal. (B) Spiking architecture of the system. Each state estimator
neuron (blue) receives spikes from sensory feedback and cerebellar output neu-
rons. It stores the spikes into two separate buffers, counts the number of input
spikes from each presynaptic neuron, and computes the “instantaneous” vari-
ability of each afferent population.
position at time t was identified by two variables x(t), y(t). Consistent with
the foundational concept on neuronal population coding movement direc-
tions and the neural representation of variables with vectorial attributes
(Georgopoulos, Schwartz, & Kettner, 1986), each population was subdi-
vided into two subpopulations (x and y) and further divided into two
groups (positive and negative), encoding the signal of opposite signs. The
firing rates were linearly proportional to the position of the point mass
along the specific axis and direction, plus a baseline firing rate of 50 Hz.
This basal discharge corresponded to the initial neutral configuration and
was considered the physiologically minimum neural activity (i.e., neurons
functionally silent). Therefore, the position of the point mass on each axis
could be decoded by computing the net firing rate (i.e., the difference be-
tween the firing rates of the positive and the negative groups) and then
dividing this net rate by the gain factor. Each of these groups consisted of
100 neurons. In summary, there were 200 neurons for each axis, for a total
of 400 sensory feedback neurons. The cerebellar output and state estimator
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1898
M. Grillo et al.
populations were analogously organized. Therefore, the overall system
consisted of 1200 neurons. This system is a proof-of-concept for the spiking
implementation of the state estimator; in principle, it can be easily scaled in
terms of multijoint tasks by increasing the number of neurons, subpopula-
tions, and groups, accordingly.
2.3 State Estimation. The state estimator neurons exploited the theory
of Bayesian inference to optimally integrate sensory feedback and cerebellar
prediction. Bayes’ theorem asserts that a certain prior knowledge p(XPred)
can be optimally integrated with new additional information p(XFbk) by
considering its likelihood (Körding & Wolpert, 2004; Ma, Beck, Latham, &
Pouget, 2006):
p(XPred
| XFbk) = p(XFbk
| XPred ) · p(XPred )
p(XFbk)
.
(2.1)
Here, p(XPred) represents the prior probability of the cerebellar prediction,
p(XFbk) is the probability of the sensory feedback, and p(XFbk|XPred) repre-
sents the likelihood that the system state is equal to XFbk when the cerebellar
prediction is XPred. The posterior distribution, p(XPred|XFbk) in equation 2.1
represents the optimal integration of the two sources of information. As-
suming both p(XPred) and p(XFbk) follow a gaussian distribution with stan-
dard deviations σ Pred and σ Fbk, respectively, the optimal state estimate can
be computed by maximizing the posterior probability, obtaining:
Xstate =
σ 2
Pred
+ σ 2
Fbk
σ 2
Pred
· XFbk
+
σ 2
σ 2
Pred
Fbk
+ σ 2
Fbk
· μ
Pred
,
(2.2)
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
+ σ 2
Pred/(σ 2
where σ 2
Fbk) represents the reliability of the current sensory
Pred
Fbk/(σ 2
feedback XFbk, σ 2
Fbk) represents the reliability of the cerebellar
Pred
prediction, and μ
Pred is the average cerebellar prediction (i.e., mean of the
prior). In other words, the optimal state estimate is the weighted sum of
XFbk and μPred based on their relative variabilities.
+ σ 2
To implement the spiking sensory feedback, we considered that accord-
ing to physiological data (Wagner et al., 2019, 2021), the reliability of a spike-
based signal strongly depends on the level of correlation and synchronicity
across the neurons within the neural population coding that signal. There-
fore, in order to estimate the reliability of the input signals, each state
estimator neuron received input from all sensory feedback and cerebellar
output neurons, collecting these spikes into two buffers (one for each af-
ferent population; see Figure 1B). The position of the incoming spikes into
the buffers depended on the unique ID of the presynaptic neuron, allowing
us to keep track of the number of spikes received from each afferent neu-
ron. For each buffer, the variance in the number of incoming spikes across
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Spiking State Estimator
1899
neurons was computed within a 25 ms time window t, obtaining a time-
varying signal. The two values of variance were then scaled by the average
number of spikes recorded in each buffer, so getting values independent
of the “instantaneous” population activity. The following equation was ap-
plied for the cerebellar output neurons:
σ
Pred(t) =
(cid:2)
N
i=1 (XPredi (t)−μ
Pred (t))2
N−1
Pred(t)
μ
,
(2.3)
where N is the number of neurons, XPredi is the sum of the input spikes re-
ceived by the ith neuron from the cerebellar population, and μ
Pred is the
average number of spikes received from the cerebellar output neurons. An
analogous formula was applied to the buffer of the sensory feedback neu-
rons, obtaining σ
Fbk from XFbki (the sum of the input spikes received by the
ith neuron from the sensory feedback population) and μ
Fbk (the average
number of spikes received from the sensory feedback neurons).
Finally, these variabilities were used to compute the reliabilities of the
afferent populations and the weighted sum described in equation 2.3. The
obtained signal was then converted into spike patterns by the state estima-
tor Poisson single-point neurons. The estimate can be read out as the net
difference in firing rates between the positive and negative groups of the
state estimator population.
2.4 Task Design and Tests. The state estimator model was evaluated in
the context of a reaching movement of a point mass along the x-direction,
from (0, 0) to (1, 0) m in 500 ms. The trajectory of the point mass was defined
as a minimum-jerk fifth-order polynomial with bell-shaped velocity profile,
the typical trajectory observed in humans during reaching tasks (Flash &
Hogan, 1985). We decided to simulate these 1D movements for simplicity
without loss of generality. However, our model allows the simulation of 2D
movements, as illustrated in the supplementary material.
The variability among sensory feedback neurons was defined by the
level of noise and the amount of the delay (Wolpert & Ghahramani, 2000),
both of which affect the reliability of the sensory feedback (Wagner et al.,
2021). In particular, each neuron was corrupted by a gaussian noise with
zero mean and standard deviation dependent on the amplitude of the
transmitted signal (i.e., signal-dependent noise; Clamann, 1969; Harris &
Wolpert, 1998), such that their firing activity was slightly desynchronized
as it occurs in real biological networks (Wagner et al., 2019). The amount
of variability of the sensory feedback neurons was not altered depending
on the stage of cerebellar learning. In contrast, it has been observed that
the synchronicity among cerebellar neurons varies with learning (Casellato
et al., 2014), showing its maximum at the end of the adaptation (Wagner
et al., 2019). Therefore, we considered two scenarios: high variability to
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1900
M. Grillo et al.
simulate inaccurate cerebellar predictions (prelearning), and low variabil-
ity to simulate accurate cerebellar predictions (postlearning). Indeed, in
the prelearning condition, the cerebellar neurons did not encode any task-
relevant information, while in the postlearning condition, they encoded
the planned trajectory, the way that the sensory feedback neurons do but
without delay. Due to this lack of delay, postlearning cerebellar predictions
should be more accurate than the sensory feedback (Shadmehr & Krakauer,
2008b); hence, we set the variability of the sensory feedback neurons to be
higher than that of the cerebellar neurons. On the contrary, in the prelearn-
ing condition, we set the variability of the sensory feedback neurons to be
lower than that of the cerebellar neurons.
Two further tests were carried out in order to challenge the state estima-
tor model in the face of unexpected perturbations. First, a sudden switch-off
of the sensory feedback during the movement was simulated. A reaching
trajectory from (0, 0) to (1, 0) m was simulated over a period of 2 seconds.
In this simulation, a scenario of intermediate cerebellar adaptation was de-
fined by setting the cerebellar output neuron variability to be slightly higher
than the feedback neuron variability. At half movement (t = 1 s), the sensory
feedback was switched off (simulated with null input trajectory and vari-
ability set to infinite), leaving the cerebellar prediction, still not completely
reliable as in postlearning, as the only source of information. This simula-
tion allowed us to test the robustness of the state estimator output when the
reliability of the afferent populations changed during the movement.
Second, a mismatch between the planned and the executed trajectory
was simulated in postlearning (e.g., visuo-motor rotation after cerebellar
adaptation). Since the postlearning condition simulates a scenario in which
the cerebellum was already adapted, the cerebellar output neurons accu-
rately (i.e., low interneuron variability) encoded the planned trajectory,
from (0, 0) to (1, 0) m in 0.5 s. At the same time, however, the sensory feed-
back neurons encoded the actual movement under the perturbation: from
(0, 0) to (−1, 0) m in 0.5 s. Therefore, the state estimator received inconsistent
information from the afferent neural populations, both with a good level of
reliability.
3 Results
3.1 Neural Coding of the Afferent Populations to the State Estimator.
The two afferent populations to the state estimator emulated the sensory
feedback and the cerebellar output, and provided the state estimator with
spike-based signals according to a rate-based coding during movement (see
Figure 2). All results relate to one subpopulation for each neural popula-
tion, as the movement occurred along the x-axis. Since the reaching move-
ment was not perturbed in this simulation, the spike patterns of the sensory
feedback neurons at pre- and postlearning conditions were very similar to
each other (except for minor differences due to the Poisson characteristics
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Spiking State Estimator
1901
of the neuron activity). Since the movement evolved in the positive direc-
tion along x-axis, the neural activity of the positive group increased along
movement while that of the negative group remained constant to its back-
ground level (approximately 50 Hz). The net firing rate of the two groups
encoded the executed trajectory, with the inherent delay of the movement
actuation and sensory feedback loop (see Figure 2A).
It is known that the spike patterns of the cerebellar output neurons
(namely, deep cerebellar nuclei cells) are modulated according to the learn-
ing stage in terms of timing, amplitude, and synchronization (Antonietti,
Martina, Casellato, D’Angelo, & Pedrocchi, 2019; Casellato et al., 2014). At
prelearning, when the cerebellum had not yet encoded any task-relevant in-
formation, all neurons fired at the constant background rate (see Figure 2B).
On the contrary, at postlearning, the cerebellar output neurons encoded the
predicted movement, with the positive group gradually increasing its activ-
ity during movement trial and the negative group maintaining a constant
background activity (see Figure 2C; Casellato et al., 2014; Naveros et al.,
2019). Similar results were obtained on both subpopulations (x- and y-axes)
during bidimensional movement (see Figure S1).
If there are no perturbations, the net firing rate between the positive and
the negative cerebellar output neurons is therefore predictive of the exe-
cuted trajectory without delay.
3.2 Reliability of the Afferent Signals to the State Estimator. The state
estimator neurons computed the spike-rate variability within each of the
afferent populations (see equation 2.3). As expected (see section 2), in the
prelearning condition, the variability of the sensory prediction was higher
than that of the sensory feedback at almost all time samples of the move-
ment trial (see Figure 3A). On average, σ Fbk was 61 ± 8 Hz and 62 ± 7 Hz
for the positive and negative groups, respectively, while σ Pred was 76 ± 9 Hz
and 76 ± 14 Hz for the positive and negative groups, respectively (see Fig-
ure 3A, left). Conversely, in the postlearning condition, the variability of the
sensory prediction was lower than that of the sensory feedback in almost
all time samples of the movement trial (see Figure 3B, left). On average,
σ Fbk was 64 ± 8 Hz and 69 ± 9 Hz for the positive and negative groups, re-
spectively, while σ Pred was lower than at prelearning, 49 ± 7.0 Hz and 50 ±
8 Hz for the positive and negative groups, respectively (see Figure 3C, left).
The reliability values (i.e., the relative weights in equation 2.2) were then
computed at each time sample (see Figure 3, right column). The reliability
of the cerebellar prediction was lower than that of the sensory feedback at
prelearning (0.45 versus 0.55 for the cerebellum and the sensory feedback
positive groups, respectively) but not at postlearning (e.g., 0.57 versus 0.43).
In other words, the state estimator considered the information from the sen-
sory feedback (affected by the delay) more trustworthy than the cerebellar
prediction before learning and the cerebellar prediction more trustworthy
than the sensory feedback after learning.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1902
M. Grillo et al.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Figure 2: Spiking activity of the afferent neural populations. The first col-
umn reports the raster plots and the second column the corresponding pop-
ulation rate-based signals (computed with time bins of 25 ms). (A) Sensory
feedback subpopulation neurons (encoding the movement along the x-axis)
(prelearning (cid:2) postlearning). While the negative group shows a constant back-
ground firing rate (approximately 50 Hz), the positive group generates a num-
ber of spikes that depend on the progressive moving away of the point mass
along the x-axis from the starting point toward the desired target. The difference
in firing rate between positive and negative groups encodes the actual position
profile, which ranges from 0 to 1 m (dashed black line). (B) Cerebellar output
subpopulation neurons (encoding the movement along x-axis) in prelearning.
Both groups show a constant background firing rate (∼50 Hz); therefore, the
Spiking State Estimator
1903
3.3 Output of the State Estimator in Stable and Perturbed Environ-
ments. The reliability of the afferent populations was used by the state es-
timator to generate the output neural activity that encoded the estimated
state of the body (see Figure 4). In both pre- and postlearning conditions, the
neural activity of the positive group of the state estimator neurons was mod-
ulated during the movement trial, while the neural activity of the negative
group was stable around a background firing rate. This behavior was con-
sistent with the movement, which evolved in the positive direction. Further-
more, it suggests that the model generated state estimates independently on
the stage of cerebellar learning.
A closer look at the net firing rate of the state estimator output allows
us to better describe the behavior of the model in the two learning condi-
tions. In the prelearning condition, when the cerebellum did not provide
task-relevant information, the output of the state estimator preferentially
followed the sensory feedback (see Figure 4A, right), which was more re-
liable than the cerebellar prediction despite its inherent delay. As a result,
the generated estimate was delayed with respect to the planned trajectory.
In the postlearning condition, the state estimator mainly relied on the cere-
bellar prediction. As a result, the firing rate of the state estimator neurons
increased earlier than in the prelearning condition (see Figure 4B left), and
the generated estimate matched the planned trajectory with no delay (see
Figure 4B right).
While in the experiments above, the reliability of the afferent popula-
tions was kept constant throughout the movement trial, the proposed state
estimator model computes the reliability continuously. This feature allows
the system to continuously modulate the relative weights of the two affer-
ent populations and therefore generate a robust state estimate despite pos-
sible changes of environmental or internal contexts during the movement
trial. When the sensory feedback became suddenly unavailable from mid-
trial, the state estimator continued to robustly provide state estimates solely
based on the cerebellar prediction, which became the most reliable source
of information, even if the cerebellar learning was still incomplete (see
Figure 5).
net activity does not encode any relevant information. (C) Cerebellar output
subpopulation neurons (encoding the movement along the x-axis) in postlearn-
ing. While the negative group shows a constant background firing rate (about
50 Hz), the positive group generates a number of spikes that depend on the pro-
gressive moving away of the point mass along the x-axis, from the starting point
toward the desired target. The net activity encodes the predicted position pro-
file, which ranges from 0 to 1 m (dashed black line). Note that this predicted po-
sition (“planned trajectory”) (given as the analog input to the cerebellar output
neurons in postlearning) anticipates by 100 ms the actual position (“executed
trajectory”), given as analog input to the sensory feedback neurons).
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1904
M. Grillo et al.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Figure 3: Variability and reliability of the afferent neural populations. Time-
varying spike variability of sensory feedback and cerebellar output neurons
(positive and negative groups for each) in the prelearning (A) and postlearn-
ing (B) conditions. The corresponding reliability time profiles are reported on
the right. These reliability values are then used as weights for the “continuous”
Bayesian integration. Finally, the mean and standard deviation of variability
and reliability are computed (C) across a lengthy task duration.
Furthermore, we tested a scenario in which sensory feedback and cere-
bellar predictions had comparable reliability but encoded inconsistent in-
formation. To this end, we introduced an unpredictable perturbation that
Spiking State Estimator
1905
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
Figure 4: Spiking activity of the state estimator population. The raster plot and
the corresponding population rate signals (computed with time bins of 25 ms) of
the state estimator neurons are reported in the prelearning (A) and postlearning
(B) conditions. While the negative group shows a constant background firing
rate (∼50 Hz), the positive group generates a number of spikes that depend on
the progressive moving away of the point mass along x-axis. The corresponding
net activity (blue rate profile) is reported on the right and compared with the
net activity of the sensory feedback and cerebellar output populations.
created a mismatch between predicted and executed trajectory in postlearn-
ing, when both afferent populations to the state estimator were comparably
reliable (see Figure 6). Since these afferent signals were equal in magnitude
with opposite signs, there was activity in both positive and negative state
estimator neurons (see Figure 6A). As a result, the decoded state estimate
(i.e., net firing rate of the neurons) was almost the average between cerebel-
lar prediction and sensory feedback (see Figure 6B). This inaccurate state
estimation was expected and would be adjusted during cerebellar retrain-
ing as detailed in section 4.
4 Discussion
In this work, we implemented a spiking neural network model of a state
estimator based on Bayesian integration theory. Our model is a proof-of-
concept and was tested here as an isolated block, receiving spike-encoded
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1906
M. Grillo et al.
Figure 5: State estimation in changing environment: a sudden unavailability
of the sensory feedback. (A) The net population rate signals of sensory feed-
back, cerebellar output, and state estimator populations (computed with time
bins of 25 ms) during a reaching trial (lasting 2 s) in an intermediate cerebellar
learning condition. (B) The sensory feedback is suddenly interrupted at half trial
duration.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Figure 6: State estimation in changing environment: an unexpected perturba-
tion. Raster plot of state estimator neurons (left) and corresponding net activity
(blue rate profile) compared with the net activity of the sensory feedback and
cerebellar output populations (right), in postlearning conditions with the action
of a sudden force field causing a movement in the opposite direction. Both cere-
bellar prediction and sensory feedback convey reliable information in this case.
As a result, the state estimator provides an intermediate signal with both groups
(positive and negative) firing since sensory feedback and cerebellar prediction
have opposite signs; therefore, the resulting net activity of the state estimator
neurons is an almost flat trajectory.
Spiking State Estimator
1907
input signals. When no reliable cerebellar predictions were available (pre-
learning), the model estimated body states mainly relying on (delayed) sen-
sory feedback. In this scenario, the cerebellar network, acting as a forward
internal model, is still untrained and its output is unreliable and noisy. In
more advanced stages of the learning process, the cerebellum acquires an
accurate internal model of the body and environment, providing a reliable
prediction of the planned movement with negligible noise and delay (Free-
man, 2014; Ito, 2000). Accordingly, in the postlearning condition, our model
estimated the body state using cerebellar prediction as the dominant source
of information. This work provides a useful tool that can be used within
spiking bio-inspired sensorimotor controllers for the simulation of motor
tasks and could be easily integrated into existing brain models.
Here, the functionalities of the proposed state estimator model were
tested in isolation from the other brain areas (e.g., the cerebellum and the
sensory feedback). To do so, we had to make assumptions on the statistics
of these areas’ neural activity based on literature: in the prelearning condi-
tion, the reliability of the (delayed) sensory feedback was higher than that of
the cerebellar prediction (because before learning, the cerebellum provides
erroneous predictions; Shadmehr & Krakauer, 2008b); in the postlearning
condition, on the contrary, the reliability of the sensory feedback was set
to be lower than that of the cerebellum, which provides accurate predic-
tions with no delay, showing a strong interneuron variability (Wagner et al.,
2019). These assumptions allowed us to illustrate that the proposed model
indeed worked in accordance with the Bayesian integration theory, consid-
ering the dominant source of information as that with the highest reliability.
In biological sensorimotor systems, sensory feedback is typically noisy
and delayed due to latencies in information processing (Wolpert & Ghahra-
mani, 2000) or even absent due to the lack of the appropriate sensory re-
ceptors and excessive movement speed (e.g., in ballistic movements like
saccades; Abrams, Meyer, & Kornblum, 1989). Compensating for the delay
or the lack of sensory feedback requires a process of state estimation that
uses internal prediction about the time-varying body state. Based on the
hypothesis that an optimal state estimation should exploit the variability
of the incoming spike trains (Scott, 2002), the state estimator model imple-
mented here extracted such variability metrics from sensory feedback and
cerebellar output neurons to evaluate the reliability of the signals encoded
by these two neural populations. This implies that along the learning pro-
cess, the cerebellar output signal should acquire an advantageous signal-to-
noise ratio (Wagner et al., 2021). This coherent spike activity of the cerebellar
output neurons occurs when the cerebellum becomes able, throughout the
repetition of several movement trials, to translate the motor efference copy
it received into accurate and well-timed predictions of the corresponding
sensory consequences (Shadmehr, 2017; Tseng et al., 2007).
By integrating the two source signals by means of their relative reliabil-
ities, the spiking state estimator model was able to properly move from an
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1908
M. Grillo et al.
initial state estimation relying only on the noisy and delayed sensory feed-
back in the prelearning condition (when the cerebellar spike trains were
strongly desynchronized), to an accurate predictive estimate of the upcom-
ing state of the moving body in the postlearning condition. It is worth not-
ing that even after learning, the overall 100 ms delay was not completely
compensated since a nonzero weight was set for the sensory feedback sig-
nal to face any possible further unpredictable event. Importantly, the contin-
uous computation of the reliability measures during movement execution
allows the system to deal with unexpected changes of environmental or in-
ternal contexts, such as a sudden interruption of the sensory feedback or an
unpredictable perturbation (Haith & Krakauer, 2013).
It could be claimed that activity correlations may be associated with a
reduction of the encoded stimulus information (e.g., impaired perceptual
discrimination). However, as discussed in Valente et al. (2021), correlations
are higher when correct choices and movements are made, thus showing
that the effects of correlations in enhancing decoding of behavioral choices
from sensory information overcome their detrimental information-limiting
effects.
Accordingly, the basic assumption of our work is that correlation is asso-
ciated with consistency of information across neurons and time, and there-
fore it is maximal when proper motor responses are learned (Valente et al.,
2021). Synchronization of spikes among a group of neurons is indeed a spe-
cial form of temporal coding (Sedaghat-Nejad et al., 2022).
4.1 Limitations. In our model, signals amplitude was encoded in the
firing rates of direction-dependent neural subpopulations (Georgopoulos
et al., 1986). Each subpopulation was in turn divided into positive and neg-
ative groups, encoding signals of opposite signs (e.g., agonist and antago-
nist signals) The neuron baseline value represents the background activity
of a functionally silent neuron (ten Brinke et al., 2017).
In our simulations, the baseline firing rate was set for all neurons to 50 Hz
to ensure a sustained neural activity in all neurons, hence guaranteeing a
good resolution in signal encoding even with a relatively low number of
neurons. However, in future implementations with large-scale brain models
and more proper population numerosity, this baseline rate could be flexibly
set to match in vivo recordings, differentiating each neuronal population.
4.2 Future Work. The results obtained here provide a solid basis for
future investigations on how spiking neural mechanisms and interaction
of different brain areas generate accurate and timely motor commands.
Indeed, the state estimator block has been implemented as a spiking pro-
cessing unit, complementing current models based on high-level represen-
tations of cortical computations or artificial neural networks (Lanillos & van
Gerven, 2021; Parrell, Ramanarayanan, Nagarajan, & Houde, 2019; Xu, Hu,
Han, & Zhang, 2021). The spiking approach increases the biological realism
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Spiking State Estimator
1909
of the model and paves the way to future studies on how the brain performs
the integration of sensory feedback and sensory prediction, with a level of
neuronal firing variability and synchronicity strongly informative, which
depends on learning stage and environmental context.
In future work, a full spiking model of the cerebellar microcircuit en-
dowed with plasticity rules, which was previously tuned and validated
on experimental data (Casali, Marenzi, Medini, Casellato, & D’Angelo,
2019; de Schepper et al., 2021; Geminiani et al., 2019), could be connected
to the spiking state estimator. In this construct, the cerebellar prediction
will emerge from adaptive circuit processing throughout task repetition
(D’Angelo et al., 2016). This will allow us to let the spiking variability of
cerebellar neuronal populations to evolve along with the acquisition of an
accurate internal dynamic model. Indeed, during the formation of internal
predictions, the cerebellar circuit undergoes an adaptation process based on
the error between the predicted body movement and the actual movement
revealed by sensory afferences. This “sensory prediction error” would be
conveyed to the cerebellum through the inferior olive circuit.
We showed that in the face of unexpected perturbations in the postlearn-
ing condition, our model does not generate an estimate that corresponds to
the actual executed movement. If the model was embedded in a complete
sensorimotor control loop (Shadmehr & Krakauer, 2008b) in such a situa-
tion, the difference between the sensory feedback and cerebellar prediction
(i.e., sensory prediction error; Tseng, Diedrichsen, Krakauer, Shadmehr,
& Bastian, 2007) would generate activity in the inferior olive, signaling
inaccurate predictions and triggering plastic processes that would even-
tually allow compensating for the perturbation. This error would increase
the variability of the cerebellar output neurons, potentially due to the
olivo-DCN collaterals (Lu, Yang, & Jaeger, 2016). As a result, the state es-
timator would start to preferentially follow the sensory feedback until the
cerebellum provided new, accurate predictions (i.e., reincreased reliability).
Experimental recordings from parietal cortex and cerebellum neurons in
behaving mice could be fundamental to validating this process.
The spiking state estimator could also be introduced in a control system
embedding spiking models of different brain areas wired using connec-
tome data (Oh et al., 2014). This will interestingly generate sensorimotor
behaviors and allow monitoring the underlying dynamics of all involved
neuronal populations. This modular system could embed blocks at differ-
ent scales and levels of neuronal detail and could be used to control more
complex dynamic bodies, with several degrees of freedom interacting with
the environment in realistic scenarios. Using this general system to control
detailed models of the musculoskeletal apparatus will be instrumental
to investigating open issues in the intricate relationship between neural
control and musculoskeletal biomechanics (e.g., the emergence of muscle
synergies; Alessandro, Carbajal, & d’Avell, 2012), and the sensory inte-
gration for the regulation of internal joint loading (Barroso, Alessandro,
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1910
M. Grillo et al.
& Tresch, 2019). Finally, the implemented system could be used to control
neurorobots, exploiting the already available interfaces with the software
MUSIC to synchronize the brain controller and the actuation of a robotic
body (Weidel, Djurfeldt, Duarte, & Morrison, 2016) and can be embedded
into neurorobotic environments like the Neurorobotics Platform (Falotico
et al., 2017) or real robots (Antonietti et al., 2019; Casellato et al., 2014).
5 Release of the Code
The spiking state estimator code implemented here is available as open
source at the following repository: https://github.com/dbbs-lab/state
-estimator. Starting from the proof-of-concept applications tested here, it
could be generalized to more complex scenarios (with additional encoded
variables and increased number of neurons) and/or embedded in closed-
loop control systems to simulate the full control of actions resulting from
the coordinated activity of multiple brain areas.
Acknowledgments
This research has received funding from the European Union’s Hori-
zon 2020 Framework Programme for Research and Innovation under
the specific grant agreement 945539 (Human Brain Project SGA3) and
was supported by the EBRAINS platform and the ICEI-FENIX research
infrastructure.
References
Abrams, R. A., Meyer, D. E., & Kornblum, S. (1989). Speed and accuracy of saccadic
eye movements: Characteristics of impulse variability in the oculomotor system.
Journal of Experimental Psychology: Human Perception and Performance, 15(3), 529–
543. 10.1037/0096-1523.15.3.529, PubMed: 2527960
Alessandro, C., Beckers, N., Goebel, P., Resquin, F., González, J., & Osu, R. (2016). Mo-
tor control and learning theories. Biosystems and Biorobotics, 10, 225–250. 10.1007/
978-3-319-24901-8_9
Alessandro C., Carbajal J. P., & d’Avella A. (2012). Synthesis and adaptation of effec-
tive motor synergies for the solution of reaching tasks. Lecture Notes in Artificial
Intelligence. Berlin: Springer.
Antonietti, A., Martina, D., Casellato, C., D’Angelo, E., & Pedrocchi, A. (2019).
Control of a humanoid NAO robot by an adaptive bioinspired cerebellar mod-
ule in 3D motion tasks. Computational Intelligence and Neuroscience, 2019, 1–15.
10.1155/2019/4862157
Barak, O. (2017). Recurrent neural networks as versatile tools of neuroscience
research. Current Opinion in Neurobiology, 46, 1–6. 10.1016/j.conb.2017.06.003,
PubMed: 28668365
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Spiking State Estimator
1911
Barroso, F. O., Alessandro, C., & Tresch, M. C. (2019). Adaptation of muscle activa-
tion after patellar loading demonstrates neural control of joint variables. Scientific
Reports, 9(1), 1–12. 10.1038/s41598-019-56888-9, PubMed: 30626917
Brette, R., Roland, P. E., Panzeri, S., & Graham, L. J. (2015). Philosophy of the spike:
Rate-based versus spike-based theories of the brain. 10.3389/fnsys.2015.00151
Casali, S., Marenzi, E., Medini, C., Casellato, C., & D’Angelo, E. (2019). Reconstruc-
tion and simulation of a scaffold model of the cerebellar network. Frontiers in
Neuroinformatics, 13, 37. 10.3389/fninf.2019.00037, PubMed: 31156416
Casellato, C., Antonietti, A., Garrido, J. A., Carrillo, R. R., Luque, N. R., Ros, E., . . .
D’Angelo, E. (2014). Adaptive robotic control driven by a versatile spiking cere-
bellar network. PLOS One, 9(11), 1–17. 10.1371/journal.pone.0112265
Chervyakov, A. V., Sinitsyn, D. O., Piradov, M. A., Lebedev, M., Merchant, H., &
Sakurai, Y. (2016). Variability of neuronal responses: Types and functional signif-
icance in neuroplasticity and neural Darwinism. Frontiers in Human Neuroscience,
10(603). 10.3389/fnhum.2016.00603, PubMed: 27932969
Clamann, H. P. (1969). Statistical analysis of motor unit firing patterns in a human
skeletal muscle. Biophysical Journal, 9(10). 10.1016/S0006-3495(69)86448-9
D’Angelo, E., Antonietti, A., Casali, S., Casellato, C., Garrido, J. A., Luque, N. R.,
. . . Ros, E. (2016). Modeling the cerebellar microcircuit: New strategies for a
long-standing issue. Frontiers in Cellular Neuroscience, 10(July), 1–29. 10.3389/
fncel.2016.00176
de Schepper, R., Geminiani, A., Masoli, S., Rizza, M. F., Antonietti, A., Casellato, C.,
& D’Angelo, E. (2021). Scaffold modelling captures the structure-function-dynamics
relationship in brain microcircuits. bioRxiv. 10.1101/2021.07.30.454314
de Xivry, J.-J. O., Coppe, S., Blohm, G., & Lefèvre, P. (2013). Kalman filtering nat-
urally accounts for visually guided and predictive smooth pursuit dynamics.
Journal of Neuroscience, 33(44), 17301–17313. 10.1523/JNEUROSCI.2321-13.2013,
PubMed: 24174663
Denève, S., Duhamel, J.-R., & Pouget, A. (2007). Optimal sensorimotor integration in
recurrent cortical networks: A neural implementation of Kalman filters. Journal of
Neuroscience, 27(21), 5744–5756. 10.1523/JNEUROSCI.3985-06.2007
Dum, R. P., Levinthal, D. J., & Strick, P. L. (2009). The spinothalamic system targets
motor and sensory areas in the cerebral cortex of monkeys. Journal of Neuroscience,
29(45), 14223–14235. 10.1523/JNEUROSCI.3398-09.2009, PubMed: 19906970
Eppler, J. M., Helias, M., Muller, E., Diesmann, M., Gewaltig, M., & Stewart, T. C.
(2009). PyNEST: A convenient interface to the NEST simulator, 2(January), 1–12.
10.3389/neuro.11.012.2008
Falotico, E., Vannucci, L., Ambrosano, A., Albanese, U., Ulbrich, S., Vasquez Tieck,
J. C., . . . Gewaltig, M.-O. (2017). Connecting artificial brains to robots in a com-
prehensive simulation framework: The neurorobotics platform. Frontiers in Neu-
rorobotics, 11(2). 10.3389/fnbot.2017.00002, PubMed: 28179882
Flash, T., & Hogan, N. (1985). The coordination of arm movements: An experi-
mentally confirmed mathematical model. Journal of Neuroscience, 5(7), 1688–1703.
10.1523/JNEUROSCI.05-07-01688.1985, PubMed: 4020415
Freeman, J. H. (2014). Cerebellar learning mechanisms. Brain Research, 1621, 1–10.
10.1016/j.brainres.2014.09.062, PubMed: 25557403
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1912
M. Grillo et al.
Geminiani, A., Casellato, C., Locatelli, F., Prestori, F., Pedrocchi, A., & D’Angelo,
E. (2018). Complex dynamics in simplified neuronal models: Reproducing Golgi
cell electroresponsiveness. Frontiers in Neuroinformatics, 12(88), 1–19. 10.3389/
fninf.2018.00088, PubMed: 29456498
Geminiani, A., Pedrocchi, A., D’Angelo, E., & Casellato, C. (2019). Response dynam-
ics in an olivocerebellar spiking neural network with nonlinear neuron proper-
ties. Frontiers in Computational Neuroscience, 13(68). 10.3389/fncom.2019.00068
Georgopoulos, A. P., Schwartz, A. B., & Kettner, R. E. (1986). Neuronal popu-
lation coding of movement direction. Science, 233(4771), 1416–1419. 10.1126/
science.3749885, PubMed: 3749885
Ghosh-Dastidar, S., & Adeli, H. (2009). Spiking neural networks. International Journal
of Neural Systems, 19(4), 295–308. 10.1142/S0129065709002002, PubMed: 19731402
Haith, A. M., & Krakauer, J. W. (2013). Model-based and model-free mechanisms of
human motor learning. Advances in Experimental Medicine and Biology, 782, 1–21.
10.1007/978-1-4614-5465-6_1, PubMed: 23296478
Harris, C. M., & Wolpert, D. M. (1998). Signal-dependent noise determines motor
planning. Nature, 394(6695), 780–784. 10.1038/29528, PubMed: 9723616
Ito, M. (2000). Mechanisms of motor learning in the cerebellum. Brain Research, 886,
237–245. 10.1016/S0006-8993(00)03142-5, PubMed: 11119699
Jordan, J., Mørk, H., Vennemo, S. B., Terhorst, D., Peyser, A., Ippen, T., Deepu, R.,
. . . Plesser, H. E. (2019). NEST 2.18.0. 10.5281/ZENODO.2605422
Kawato, M. (1999). Internal models for motor control and trajectory planning.
Current Opinion in Neurobiology, 9(6), 718–727. 10.1016/S0959-4388(99)00028-8,
PubMed: 10607637
Kawato, M., & Gomi, H. (1992). The cerebellum and VOR/OKR learning mod-
els. Trends in Neurosciences, 15(11), 445–453. 10.1016/0166-2236(92)90008-V,
PubMed: 1281352
Körding, K. P., & Wolpert, D. M. (2004). Bayesian integration in sensorimotor learn-
ing. Nature, 427(6971), 244–247. 10.1038/nature02169
Lanillos, P., & van Gerven, M. (2021). Neuroscience-inspired perception-action in
robotics: Applying active inference for state estimation, control and self-perception.
arXiv:2105.04261.
Lu, H., Yang, B., & Jaeger, D. (2016). Cerebellar nuclei neurons show only small ex-
citatory responses to optogenetic olivary stimulation in transgenic mice: In vivo
and in vitro studies. Frontiers in Neural Circuits, 10, 21. 10.3389/fncir.2016.00021,
PubMed: 27047344
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006). Bayesian inference with
probabilistic population codes. Nature Neuroscience, 9(11). 10.1038/nn1790
Miall, R. C., & Wolpert, D. M. (1996). Forward models for physiological mo-
tor control. Neural Networks, 9(8), 1265–1279. 10.1016/S0893-6080(96)00035-4,
PubMed: 12662535
Naveros, F., Luque, N. R., Ros, E., & Arleo, A. (2019). VOR adaptation on a humanoid
iCub robot using a spiking cerebellar model. IEEE Transactions on Cybernetics, 1–
14. 10.1109/TCYB.2019.2899246
Oh, S. W., Harris, J. A., Ng, L., Winslow, B., Cain, N., Mihalas, S., . . . Zeng, H. (2014).
A mesoscale connectome of the mouse brain. Nature, 508(7495), 207–214. 10.1038/
nature13186, PubMed: 24695228
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Spiking State Estimator
1913
Palesi, F., Tournier, J.-D., Calamante, F., Muhlert, N., Castellazzi, G., Chard, D.,
. . . Wheeler-Kingshott, C. A. M. (2014). Contralateral cerebello-thalamo-cortical
pathways with prominent involvement of associative areas in humans in
vivo. Brain Structure and Function, 220(6), 3369–3384. 10.1007/s00429-014-0861-2,
PubMed: 25134682
Parrell, B., Ramanarayanan, V., Nagarajan, S., & Houde, J. (2019). The FACTS model
of speech motor control: Fusing state estimation and task-based control. PLOS
Computational Biology, 15(9). 10.1371/journal.pcbi.1007321, PubMed: 31479444
Popa, L. S., & Ebner, T. J. (2019). Cerebellum, predictions and errors. Frontiers in
Cellular Neuroscience, 12, 524. 10.3389/fncel.2018.00524, PubMed: 30697149
Scott, S. H. (2002). Optimal strategies for movement: Success with variability. Nature
Neuroscience, 5(11). 10.1038/nn1102-1110
Sedaghat-Nejad, E., Pi, J. S., Hage, P., Fakharian, M. A., & Shadmehr, R. (2022). Syn-
chronous spiking of cerebellar Purkinje cells during control of movements. In
Proceedings of the National Academy of Sciences, 119(14). 10.1073/PNAS.2118954119
Shadmehr, R. (2017). Learning to predict and control the physics of our movements.
Journal of Neuroscience, 37(7), 1663–1671. 10.1523/JNEUROSCI.1675-16.2016,
PubMed: 28202784
Shadmehr, R., & Krakauer, J. W. (2008a). A computational neuroanatomy for motor
control. Experimental Brain Research, 185(3), 359–381. 10.1007/s00221-008-1280-5
Shadmehr, R., & Krakauer, J. W. (2008b). A computational neuroanatomy for motor
control. Experimental Brain Research, 185(3), 359–381. 10.1007/s00221-008-1280-5
Stein, J. (2009). Cerebellar forward models to control movement. Journal of Physiology,
587. 10.1113/jphysiol.2008.167627, PubMed: 19147672
ten Brinke, M. M., Heiney, S. A., Wang, X., Proietti-Onori, M., Boele, H. J., Bakermans,
J., . . . de Zeeuw, C. I. (2017). Dynamic modulation of activity in cerebellar nuclei
neurons during Pavlovian eyeblink conditioning in mice. eLife, 6, 1–27. 10.7554/
eLife.28132
Thach, W. T. (1998). A role for the cerebellum in learning movement coordination.
Neurobiology of Learning and Memory, 70(1–2), 177–188. 10.1006/nlme.1998.3846,
PubMed: 9753595
Tseng, Y., Diedrichsen, J., Krakauer, J. W., Shadmehr, R., & Bastian, A. J. (2007). Sen-
sory prediction errors drive cerebellum-dependent adaptation of reaching. Jour-
nal of Neurophysiology, 98(1), 54–62. 10.1152/jn.00266.2007, PubMed: 17507504
Valente, M., Pica, G., Bondanelli, G., Moroni, M., Runyan, C. A., Morcos, A. S.,
. . . Panzeri, S. (2021). Correlations enhance the behavioral readout of neural
population activity in association cortex. Nature Neuroscience, 24(7), 975–986.
10.1038/s41593-021-00845-1, PubMed: 33986549
Wagner, M. J., Kim, T. H., Kadmon, J., Nguyen, N. D., Ganguli, S., Schnitzer, M. J., &
Luo, L. (2019). Shared cortex-cerebellum dynamics in the execution and learning
of a motor task. Cell, 177(3), 669–682. 10.1016/J.CELL.2019.02.019
Wagner, M. J., Savall, J., Hernandez, O., Mel, G., Inan, H., Rumyantsev, . . . Schnitzer,
M. J. (2021). A neural circuit state change underlying skilled movements. Cell,
184(14), 3731–3747. 10.1016/J.CELL.2021.06.001
Weidel, P., Djurfeldt, M., Duarte, R. C., & Morrison, A. (2016). Closed loop interac-
tions between spiking neural network and robotic simulators based on MUSIC
and ROS. Frontiers in Neuroinformatics, 10(31). 10.3389/FNINF.2016.00031
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
1914
M. Grillo et al.
Wolpert, D., Goodbody, S., & Husain, M. (1998). Maintaining internal representa-
tions: The role of the human superior parietal lobe. Nature Neuroscience, 1(6), 529–
533. 10.1038/2245, PubMed: 10196553
Wolpert, D. M., & Ghahramani, Z. (2000). Computational principles of movement
neuroscience. Nature Neuroscience 3(11), 1212–1217. 10.1038/81497, PubMed:
11127840
Wolpert, D. M., & Ghahramani, Z. (2004). Computational motor control. Science, 269,
1880–1882. 10.1126/science.7569931
Xu, D., Hu, A., Han, X., & Zhang, L. (2021). A nonlinear system state estimation
method based on adaptive fusion of multiple kernel functions. Complexity, 2021.
10.1155/2021/5124841
Received December 5, 2021; accepted April 30, 2022.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
n
e
c
o
a
r
t
i
c
e
–
p
d
/
l
f
/
/
/
/
3
4
9
1
8
9
3
2
0
5
1
5
9
4
n
e
c
o
_
a
_
0
1
5
2
5
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3
Download pdf