焦点功能:
Connectivity, 认识, and Consciousness
Thalamocortical contribution to flexible
learning in neural systems
Mien Brabeeba Wang1,2 and Michael M. Halassa1
1Department of Brain and Cognitive Science, 麻省理工学院, 剑桥, 嘛, 美国
2Computer Science and Artificial Intelligence Laboratory, 麻省理工学院, 剑桥, 嘛, 美国
关键词: Meta-learning, Credit assignment, Continual learning, Thalamocortical interactions,
Basal ganglia, Thalamus
开放访问
杂志
抽象的
Animal brains evolved to optimize behavior in dynamic environments, flexibly selecting
actions that maximize future rewards in different contexts. A large body of experimental work
indicates that such optimization changes the wiring of neural circuits, appropriately mapping
environmental input onto behavioral outputs. A major unsolved scientific question is how
optimal wiring adjustments, which must target the connections responsible for rewards, 可
accomplished when the relation between sensory inputs, action taken, and environmental
context with rewards is ambiguous. The credit assignment problem can be categorized into
context-independent structural credit assignment and context-dependent continual learning. 在
this perspective, we survey prior approaches to these two problems and advance the notion
that the brain’s specialized neural architectures provide efficient solutions. Within this
框架, the thalamus with its cortical and basal ganglia interactions serves as a systems-
level solution to credit assignment. 具体来说, we propose that thalamocortical interaction is
the locus of meta-learning where the thalamus provides cortical control functions that
parametrize the cortical activity association space. By selecting among these control functions,
the basal ganglia hierarchically guide thalamocortical plasticity across two timescales to
enable meta-learning. The faster timescale establishes contextual associations to enable
behavioral flexibility, while the slower one enables generalization to new contexts.
作者总结
Deep learning has shown great promise over the last decades, allowing artificial neural
networks to solve difficult tasks. The key to success is the optimization process by which task
errors are translated to connectivity patterns. A major unsolved question is how the brain
optimally adjusts the wiring of neural circuits to minimize task error analogously. In our
看法, we advance the notion that the brain’s specialized architecture is part of the
solution and spell out a path towards its theoretical, computational, and experimental testing.
具体来说, we propose that the interaction between the cortex, thalamus, and basal ganglia
induces plasticity in two timescales to enable flexible behaviors. The faster timescale
establishes contextual associations to enable behavioral flexibility, while the slower one
enables generalization to new contexts.
引文: 王, 中号. B., & Halassa, 中号. 中号.
(2022). Thalamocortical contribution to
flexible learning in neural systems.
网络神经科学, 6(4), 980–997.
https://doi.org/10.1162/netn_a_00235
DOI:
https://doi.org/10.1162/netn_a_00235
已收到: 26 九月 2021
公认: 19 一月 2022
利益争夺: 作者有
声明不存在竞争利益
存在.
通讯作者:
Michael M. Halassa
mhalassa@mit.edu
处理编辑器:
Randy McIntosh
版权: © 2022
麻省理工学院
在知识共享下发布
归因 4.0 国际的
(抄送 4.0) 执照
麻省理工学院出版社
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
t
/
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
Reward prediction error:
A quantity represented by the
difference between the expected
reward and actual reward.
Credit assignment:
A computational problem to
determine which stimulus, 行动,
internal states, and context lead to
outcome.
Continual learning:
A computational problem to learn
tasks sequentially to both learn new
tasks faster and not forget old tasks.
介绍
Learning to flexibly choose appropriate actions in uncertain environments is a hallmark of
智力 (磨坊主 & 科恩, 2001; 尼夫, 2009; Thorndike, 2017). When animals explore unfa-
miliar environments, they tend to reinforce actions that lead to unexpected rewards. A com-
mon notion in contemporary neuroscience is that such behavioral reinforcement emerges from
changes in synaptic connectivity, where synapses that contribute to the unexpected reward are
加强 (Abbott & 纳尔逊, 2000; Bliss & Lomo, 1973; Dayan & Abbott, 2005; Hebb,
2002; Whittington & Bogacz, 2019). A prominent model for connecting synaptic to behav-
ioral reinforcement is dopaminergic innervation of basal ganglia (BG), where dopamine
(DA) carries the reward prediction error (RPE) signals to guide synaptic learning (Bamford,
Wightman, & Sulzer, 2018; Bayer & Glimcher, 2005; Montague, Dayan, & Sejnowski,
1996; Schultz, Dayan, & Montague, 1997). This circuit motif is thought to implement a basic
form of the reinforcement learning algorithm (Houk, 戴维斯, & Beiser, 1994; 莫里斯, Nevet,
Arkadir, Vaadia, & 伯格曼, 2006; Roesch, Calu, & Schoenbaum, 2007; 苏里 & Schultz,
1999; 右. Sutton & Barto, 2018; 右. S. Sutton & Barto, 1990; Wickens & Kotter, 1994), 哪个
has had much success in explaining simple Pavlovian and instrumental conditioning (Ikemoto
& Panksepp, 1999; 尼夫, 2009; 右. Sutton & Barto, 2018; 右. S. Sutton & Barto, 1990). 然而,
it is unclear how this circuit can reinforce the appropriate connections in complex natural
environments where animals need to dynamically map sensory inputs to different action in
a context-dependent way. If one naively credits all synapses with the RPE signals, 学习
will be highly inefficient since different cues, 上下文, and actions contribute to the RPE sig-
nals differently. To properly credit the cues, 语境, and actions that lead to unexpected
reward is a challenging problem, known as the credit assignment problem (Lillicrap, Santoro,
Marris, Akerman, & 欣顿, 2020; Minsky, 1961; Rumelhart, 欣顿, & 威廉姆斯, 1986;
Whittington & Bogacz, 2019).
One can roughly categorize the credit assignment into context-independent structural
credit assignment and context-dependent continual learning. In structural credit assignment,
animals may make decisions in a multi-cue environment and should be able to credit those
cues that contribute to the rewarding outcome. 相似地, if actions are being chosen based
on internal decision variables, then the underlying activity states must also be reinforced. 在
such cases, neurons that are selective to external cues or internal latent variables need to
adjust their downstream connectivity based on its contribution of their downstream targets to
the RPE. This is a challenging computation to implement because, for upstream neurons, 这
RPE will be dependent on downstream neurons that are several connections away. 为了
例子, a sensory neuron needs to know the action chosen in the motor cortex to selec-
tively credit the sensory synapses that contribute to the action. In continual learning, 动物
not only need to appropriately credit the sensory cues and actions that lead to the reward
but also need to credit the sensorimotor combination in the right context to retain the
behaviors learned from different contexts and even to generalize to novel contexts. 那里-
fore, animals can continually learn and generalize across different contexts while retaining
behaviors in familiar contexts. 例如, when one is in the United States, one learns to
first look left before crossing the street, whereas in the United Kingdom, one learns to look
right instead. 然而, after spending time in the United Kingdom, someone from the
United States should not unlearn the behavior of looking left first when they return home
because their brain ought to properly assign the credit to a different context. 此外,
once one learns how to cross the street in the United States, it is much easier to learn how
to cross the street in the United Kingdom because the brain flexibly generalize behaviors
across contexts.
网络神经科学
981
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
/
t
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
t
.
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
Backpropagation:
An algorithm to compute the error
gradient of an artificial neural
network through chain rules.
In this perspective, we will first go over common approaches from machine learning to
tackle these two credit assignment problems. 在这样做, we highlight the challenge in their
efficient implementation within biological neural circuits. We also highlight some recent pro-
posals that advance the notion of specialized neural hardware that approximate more general
solutions for credit assignment (Fiete & Seung, 2006; Ketz, Morkonda, & O’Reilly, 2013;
Kornfeld et al., 2020; Kusmierz, Isomura, & Toyoizumi, 2017; Lillicrap, Cownden, Tweed,
& Akerman, 2016; 刘, 史密斯, Mihalas, Shea-Brown, & Sümbül, 2020; O’Reilly, 1996;
O’Reilly, Russin, Zolfaghar, & Rohrlich, 2021; Richards & Lillicrap, 2019; 罗尔夫塞玛 &
Holtmaat, 2018; 罗尔夫塞玛 & van Ooyen, 2005; 萨克拉门托, Ponte Costa, 本吉奥, & Senn,
2018; Schiess, Urbanczik, & Senn, 2016; Zenke & 甘古利, 2018). Along these lines, we pro-
pose an efficient systems-level solution involving the thalamus and its interaction with the
cortex and BG for these two credit assignment problems.
COMMON MACHINE LEARNING APPROACHES TO CREDIT ASSIGNMENT
One solution to structural credit assignment in machine learning is backpropagation (Rumelhart
等人。, 1986). Backpropagation recursively computes the vector-valued error signal for synapses
based on their contribution to the error signal. There is much empirical success of backpropa-
gation in surpassing human performance in supervised learning such as image recognition
(他, 张, Ren, & Sun, 2016; 克里热夫斯基, 吸勺, & 欣顿, 2012) and reinforcement
learning such as playing the game of Go and Atari (Mnih et al., 2015; Schrittwieser et al.,
2020; Silver et al., 2016; Silver et al., 2017). 此外, comparing artificial networks trained
with backpropagation with neural responses from the ventral visual stream of nonhuman pri-
mates shows comparable internal representations (Cadieu et al., 2014; Yamins et al., 2014).
Despite its empirical success in superhuman-level performance and matching the internal
representation of actual brains, backpropagation may not be straightforward to implement
in biological neural circuits, as we explain below.
In its most basic form, backpropagation requires symmetric connections between neurons
(forward and backward connections). 从数学上来说, we can write down the backpropaga-
tion in Equation 1:
在哪里
δWi ∝
∂E
∂Wi
ð
¼ eif ai−1
Þ⊤;
ei ¼ W T
iþ1eiþ1 ∘ f 0 aið
Þ;
(1)
E is the total error, ei is the vector error at layer i, Wi is the synaptic weight connecting layer i − 1
to layer i, and f is the nonlinearity. 直观地, this is saying that the change of synaptic weight Wi
is computed by a Hebbian learning rule between backpropagation error ei and activity from last
layer f(ai−1), while the backpropagation error is computed by backpropagating the error in the
next layer through symmetric feedback weights W ⊤
iþ1. 重要的, in this algorithm, error sig-
nals do not alter the activity of neurons in the preceding layers and instead operate indepen-
dently from the feedforward activity. 然而, such arrangement is not observed in the brain;
symmetric connections across neurons are not a universal feature of circuit organization, 和
biological neurons may encode both feedforward inputs and errors through changes in spike
输出 (changes in activity; Crick, 1989; Richards & Lillicrap, 2019). 所以, it is hard to
imagine how the basic form of backpropagation (symmetry and error/activity separation) 是
physically implemented in the brain.
网络神经科学
982
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
/
t
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
Catastrophic forgetting:
A phenomenon in which the network
forgets about the previous tasks upon
learning new tasks.
此外, while an animal can continually learn to behave across different contexts,
artificial neural networks trained by backpropagation struggle to learn and remember different
tasks in different contexts: a problem known as catastrophic forgetting (法语, 1999; Kemker,
McClure, Abitino, 海耶斯, & Kanan, 2018; Kumaran, Hassabis, & 麦克莱兰, 2016; McCloskey
& 科恩, 1989; 帕里西, Kemker, 部分, Kanan, & Wermter, 2019). 具体来说, 这个问题
occurs when the tasks are trained sequentially because the weights optimized for former tasks
will be modified to fit the later tasks. One of the common solutions is to interleave the tasks
from different contexts to jointly optimize performance across contexts by using an episodic
memory system and replay mechanism (Kumaran et al., 2016; 麦克莱兰, McNaughton, &
O’Reilly, 1995). This approach has received empirical success in artificial neural networks,
including learning to play many Atari games (Mnih et al., 2015; Schrittwieser et al., 2020).
然而, since one needs to store past training data in memory to replay during learning, 这
approach demands a high computational overhead and can be is inefficient as the number of
the contexts increases. 另一方面, humans and animals acquire diverse sensorimotor
skills in different contexts throughout their life span: a feat that cannot be solely explained by
memory replay (中号. 中号. 穆雷, Lewkowicz, Amedi, & 华莱士, 2016; Parisi et al., 2019;
力量 & 施拉加尔, 2017; Zenke, Gerstner, & 甘古利, 2017). 所以, biological neural
circuits are likely to employ other solutions to continual learning in addition to memory replay.
所以, to solve these two credit assignment problems in the brain, one needs to seek
different solutions. One of the pitfalls of backpropagation is that it is a general algorithm that
works on any architecture. 然而, actual brains are collections of specialized hardware put
together in a specialized way. It can be conceived that through clever coordination between
different cell types and different circuits, the brains can solve the credit assignment problem by
leveraging its specialized architectures. Along this line of ideas, many investigators have pro-
posed cellular (Fiete & Seung, 2006; Kornfeld et al., 2020; Kusmierz et al., 2017; 刘等人。,
2020; Richards & Lillicrap, 2019; Sacramento et al., 2018; Schiess et al., 2016) and circuit-level
mechanisms (Lillicrap et al., 2016; O’Reilly, 1996; 罗尔夫塞玛 & Holtmaat, 2018; 罗尔夫塞玛 &
van Ooyen, 2005) to assign credit appropriately. In this perspective, we would like to advance
the notion that the specialized hardware arrangement also happens at the system level and pro-
pose that the thalamus and its interaction with basal ganglia and the cortex serve as a system-
level solution for these three types of credit assignment.
A PROPOSAL: THALAMOCORTICAL–BASAL GANGLIA INTERACTIONS ENABLE
META-LEARNING TO SOLVE CREDIT ASSIGNMENT
To motivate the notion of thalamocortical–basal ganglia interactions being a potential solution
for credit assignment, we will start with a brief introduction. 皮质, thalamus, and basal
ganglia are the three major components of the mammalian forebrain—the part of the brain to
which high-level cognitive capacities are attributed to (亚历山大, DeLong, & 针织, 1986;
Badre, Kayser, & D’Esposito, 2010; 考克斯 & Witten, 2019; Makino, Hwang, Hedrick, &
Komiyama, 2016; 磨坊主, 2000; 磨坊主 & 科恩, 2001; 尼夫, 2009; Seo, 李, & Averbeck,
2012; Wolff & Vann, 2019). Each of these components has its specialized internal architec-
特雷斯; the cortex is dominated by excitatory neurons with extensive lateral connectivity profiles
(Fuster, 1997; Rakic, 2009; 歌手, Sejnowski, & Rakic, 2019), the thalamus is grossly divided
into different nuclei harboring mostly excitatory neurons devoid of lateral connections (哈里斯
等人。, 2019; 琼斯, 1985; Sherman & Guillery, 2005), and the basal ganglia are a series of
inhibitory structures driven by excitatory inputs from the cortex and thalamus (Gerfen &
Bolam, 2010; Lanciego, Luquin, & Obeso, 2012; Nambu, 2011) (数字 1). A popular view
within system neuroscience stipulates that BG and the cortex underwent different learning
网络神经科学
983
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
/
t
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
t
.
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
数字 1. Distinct architectures of cortex, thalamus, and basal ganglia. Cortex is largely composed
of excitatory neurons with extensive recurrent connectivity. Thalamus consists of mostly excitatory
neurons without lateral connections. Basal ganglia consist of mostly inhibitory neurons driven by
cortical and thalamic inputs, and the corticostriatal plasticity is modulated by dopamine.
paradigms, where BG is involved in reinforcement learning while the cortex is involved in
unsupervised learning (Doya, 1999, 2000). 具体来说, the input structure of the basal ganglia
known as the striatum is thought to be where reward gated plasticity takes place to implement
reinforcement learning (Bamford et al., 2018; 考克斯 & Witten, 2019; Hikosaka, Kim, Yasuda, &
Yamamoto, 2014; Kornfeld et al., 2020; 尼夫, 2009; Perrin & Venance, 2019). One such evi-
dence is the high temporal precision of DA activity in the striatum. To accurately attribute the
action that leads to positive RPE, DA is released into the relevant corticostriatal synapses.
然而, DA needs to disappear quickly to prevent the next stimulus-response combination
from being reinforced. In the striatum, this elimination process is carried out by dopamine
active transporter (DAT) to maintain a high temporal resolution of DA activity on a timescale
of around 100 ms–1 s to support reinforcement learning (Cass & Gerhardt, 1995; Ciliax et al.,
1995; Garris & Wightman, 1994). 相比之下, although the cortex also has dopaminergic
innervation, cortical DAT expression is low and therefore DA levels may change at a timescale
that is too slow to support reinforcement learning (Cass & Gerhardt, 1995; Garris & Wightman,
1994; Lapish, Kroener, Durstewitz, Lavin, & Seamans, 2007; Seamans & Robbins, 2010) 但
instead supports other processes related to learning (Badre et al., 2010; 磨坊主 & 科恩, 2001).
实际上, ample evidence indicates that cortical structures undergo Hebbian-like long-term
potentiation (LTP) and long-term depression (LTD; 库克 & Bear, 2010; 费尔德曼, 2009;
Kirkwood, Rioult, & Bear, 1996). 然而, despite the unsupervised nature of these processes,
cortical representations are task-relevant and include appropriate sensorimotor mappings that
lead to rewards (Allen et al., 2017; 多纳休 & 李, 2015; Enel, Wallis, & Rich, 2020; Jacobs &
Moghaddam, 2020; 彼得森, 2019; Tsutsui, Hosokawa, Yamada, & Iijima, 2016). How could
this arise from an unsupervised process? One possible explanation is that basal ganglia acti-
vate the appropriate cortical neurons during behaviors and the cortical network collectively
consolidates high-reward sensorimotor mappings via Hebbian-like learning (Andalman & Fee,
2009; Ashby, Ennis, & Spiering, 2007; Hélie, Ell, & Ashby, 2015; Tesileanu, Olveczky, &
Balasubramanian, 2017; Warren, Tumer, Charlesworth, & Brainard, 2011). Previous computa-
tional accounts of this process have emphasized a consolidation function for the cortex in this
过程, which naively would beg the question of why duplicate a process that seems to func-
tion well in the basal ganglia and perhaps include a lot of details of the associated experience?
网络神经科学
984
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
t
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
t
.
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
Meta-learning:
A learning paradigm in which a
network learns how to learn more
efficiently.
数字 2. Two views of learning in the cortex. (A) One possible view is that the Hebbian cortical
plasticity consolidates the sensorimotor mapping from BG to learn a stimulus-action mapping at =
F(st). (乙) We propose that thalamocortical systems perform meta-learning by consolidating the
teaching signals from BG to learn a context-dependent mapping at = fc(st), where the context c is
computed by past stimulus history and represented by different thalamic activities.
The answer to this question is the core of our proposal. We propose that the learning pro-
cess is not a duplication, but instead that the reinforcement process in the basal ganglia selects
thalamic control functions that subsequently activate cortical associations to allow flexible
mappings across different contexts (数字 2).
To understand this proposition, we need to take a closer look at the involvement of these
distinct network elements in task learning. Learning in basal ganglia happens in corticostriatal
synapses where the basic form of reinforcement learning is implemented. 具体来说, the coac-
tivation of sensory and motor cortical inputs generates eligibility traces in corticostriatal synap-
ses that get captured by the presence or absence of DA (Fee & Goldberg, 2011; Fiete, Fee, &
Seung, 2007; Kornfeld et al., 2020). This reinforcement learning algorithm is fast at acquiring
simple associations but slow at generalization to other behaviors. 另一方面, the cortical
plasticity operates in a much slower timescale but seems to allow flexible behaviors and fast
generalization (Kim, 约翰逊, Cilles, & 金子, 2011; Mante, Sussillo, 谢诺伊, & Newsome,
2013; 磨坊主, 2000; 磨坊主 & 科恩, 2001). How does the cortex exhibit slow synaptic plasticity
and flexible behaviors at the same time? An explanatory framework is meta-learning (博特维尼克
等人。, 2019; 王等人。, 2018), where the flexibility arises from network dynamics and the
generalization emerges from slow synaptic plasticity across different contexts. 换句话说,
synaptic plasticity stores a higher order association between contexts and sensorimotor associ-
ations while the network dynamics switches between different sensorimotor associations based
on this higher order association. 然而, properly arbitrating between synaptic plasticity and
network dynamics to store such higher order association is a nontrivial task (Sohn, Meirhaeghe,
Rajalingham, & Jazayeri, 2021). We propose that the thalamocortical system learns these
dynamics, where the thalamus provides control nodes that parametrize the cortical activity asso-
ciation space. Basal ganglia inputs to the thalamus learn to select between these different control
节点, directly implementing the interface between weight adjustment and dynamical controls.
Our proposal rests on the following three specific points.
第一的, building on a line of the literature that shows diverse thalamocortical interaction in
感官的, cognitive, and motor cortex, we propose that thalamic output may be described as
control functions over cortical computations. These control functions can be purely in the
sensory domain like attentional filtering, in the cognitive domain like manipulating working
记忆, or in the motor domain like preparation for movement (Bolkan et al., 2017; 瓦. Guo,
Clause, Barth-Maron, & Polley, 2017; Z. V. Guo et al., 2017; Mukherjee et al., 2020; Rikhye,
Gilra, & Halassa, 2018; Saalmann & Kastner, 2015; Schmitt et al., 2017; Tanaka, 2007;
Wimmer et al., 2015; 周, Schafer, & 德西莫内, 2016). These functions directly relate
网络神经科学
985
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
t
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
t
.
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
thalamic activity patterns to different cortical dynamical regimes and thus offer a way to estab-
lish higher order association between context and sensorimotor mapping within the thalamo-
cortical pathways. 第二, based on previous studies on direct and indirect BG pathways that
influence most cortical regions (Hunnicutt et al., 2016; Jiang & Kim, 2018; Nakajima, 施密特,
& Halassa, 2019; Peters, 法布尔, Steinmetz, 哈里斯, & Carandini, 2021), we propose that BG
hierarchically selects these thalamic control functions to influence activities of the cortex
toward rewarding behavioral outcomes. 最后, we propose that thalamocortical structure con-
solidates the selection of BG through a two-timescale Hebbian learning process to enable
元学习. 具体来说, the faster corticothalamic plasticity learns the higher order associ-
ation that enables flexible contextual switching with different thalamic patterns (Marton,
Seifikar, Luongo, 李, & Sohal, 2018; Rikhye et al., 2018), while the slower cortical plasticity
learns the shared representations that allow generalization to new behaviors. 以下, 我们将
go over the supporting literature that leads us to this proposal.
MORE GENERAL ROLES OF THALAMOCORTICAL INTERACTION AND
BASAL GANGLIA
Classical literature has emphasized the role of the thalamus in transmitting sensory inputs to
the cortex. This is because some of the better studied thalamic pathways are those connected
to sensors on one end and primary cortical areas on another (Hubel & Wiesel, 1961; Lien &
Scanziani, 2018; Reinagel, Godwin, Sherman, & 科赫, 1999; Sherman & Spear, 1982; Usrey,
Alonso, & 里德, 2000). From that perspective, thalamic neurons being devoid of lateral
connection transmit their inputs (例如, from the retina in the case of the lateral geniculate
nucleus, LGN) to the primary sensory cortex ( V1 in this same example case), and the input
转型 (center-surround to oriented edges) occurs within the cortex (Hoffmann, Stone,
& Sherman, 1972; Hubel & Wiesel, 1962; Lien & Scanziani, 2018; Usrey et al., 2000). In many
案例, these formulations of thalamic “relay” have generalized to how motor and cognitive
thalamocortical interactions may be operating. 然而, in contrast to the classical relay view
of the thalamus, more recent studies have shown diverse thalamic functions in sensory, cog-
nitive, and motor processing (Bolkan et al., 2017; 瓦. Guo et al., 2017; Z. V. Guo et al., 2017;
Rikhye et al., 2018; Saalmann & Kastner, 2015; Schmitt et al., 2017; Tanaka, 2007; Wimmer
等人。, 2015; Zhou et al., 2016). For example in mice, sensory thalamocortical transmission can
be adjusted based on prefrontal cortex (全氟碳化物)-dependent, top-down biasing signals transmitted
through nonclassical basal ganglia pathways involving the thalamic reticular nucleus (TRN;
Nakajima et al., 2019; Phillips, Kambi, & Saalmann, 2016; Wimmer et al., 2015). 有趣的是,
these task-relevant PFC signals themselves require long-range interactions with the associative
mediodorsal (医学博士) thalamus to be initiated, 保持, and flexibly switched (Rikhye et al.,
2018; Schmitt et al., 2017; Wimmer et al., 2015). One can also observe nontrivial control
functions in the motor thalamus. Motor preparatory activities in the anterior motor cortex
(ALM) show persistent activities that predicted future actions. 有趣的是, the motor thalamus
also shows similar preparatory activities that predict future actions and by optogenetically
manipulating the motor thalamus activities, the persistent activities in ALM quickly diminished
(Z. V. Guo et al., 2017). 最近, Mukherjee, Lam, Wimmer, and Halassa (2021) discovered
two cell types within MD thalamus differentially modulate the cortical evidence accumulation
dynamics depending on whether the evidence is conflicting or sparse to boost the signal-to-
noise ratio in decision-making. Based on the above studies, we propose that the thalamus
provides a set of control functions to the cortex. 具体来说, cortical computations may be
flexibly switched to different dynamical modes by activating a particular thalamic output that
corresponds to that mode.
网络神经科学
986
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
t
/
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
另一方面, the selective role of BG in motor and cognitive control also has dom-
inated the literature because thalamocortical–basal ganglia interaction is the most well studied
in frontal systems (考克斯 & Witten, 2019; Makino et al., 2016; McNab & Klingberg, 2008;
Monchi, Petrides, Strafella, Worsley, & Doyon, 2006; Seo et al., 2012). 然而, classical
and contemporary studies have recognized that all cortical areas, including primary sensory
地区, project to the striatum (Hunnicutt et al., 2016; Jiang & Kim, 2018; Peters et al., 2021).
相似地, the basal ganglia can project to the more sensory parts of the thalamus through lesser
studied pathways to influence the sensory cortex (Hunnicutt et al., 2016; Nakajima et al.,
2019; Peters et al., 2021). 具体来说, a nonclassical BG pathway projects to TRN, 哪个
in turn modulates the activities of LGN to influence sensory thalamocortical transmission
(Nakajima et al., 2019). 另一方面, it has also been argued that BG is involved in
gating working memory (McNab & Klingberg, 2008; Voytek & 骑士, 2010). This shows that
BG has a much more general role than classical action and action strategy selection. 那里-
fore, combining with our proposals on thalamic control functions, we propose that BG hier-
archically selects different thalamic control functions to influence all cortical areas in different
contexts through reinforcement learning.
此外, there are series of the work that indicates the role of BG to guide plasticity in
thalamocortical structures (Andalman & Fee, 2009; Fiete et al., 2007; Hélie et al., 2015;
Mehaffey & Doupe, 2015; Tesileanu et al., 2017). 尤其, there is evidence that BG is
critical for the initial learning and less involved in the automatic behaviors once the behaviors
are learned across different species. In zebra finches, the lesion of BG in adult zebra finch has
little effect on song production, but the lesion of BG in juvenile zebra finch prevents the bird
from learning the song (Fee & Goldberg, 2011; Scharff & Nottebohm, 1991; Sohrabji,
Nordeen, & Nordeen, 1990). Similar patterns can be observed in people with Parkinson’s dis-
舒适. Parkinson’s patients who have a reduction of DA and striatal defects have troubles in
solving procedural learning tasks but can produce automatic behaviors normally (Asmus,
Huber, Gasser, & Schöls, 2008; Soliveri, 棕色的, Jahanshahi, Caraceni, & Marsden, 1997;
Thomas-Ollivier et al., 1999). This behavioral evidence suggests that thalamocortical struc-
tures consolidate the learning from BG as the behaviors become more automatic. 此外,
on the synaptic level, a songbird learning circuit also demonstrates this cortical consolidation
motif (Mehaffey & Doupe, 2015; Tesileanu et al., 2017). In a zebra finch, the premotor nucleus
HVC (a proper name) projects to the motor nucleus robust nucleus of the arcopallium (RA) 到
produce the song. 另一方面, RA also receives BG nucleus Area X mediated inputs
from the lateral nucleus of the medial nidopallium (LMAN). The latter pathway is believed to
be a locus of reinforcement learning in the songbird circuit. By burst stimulating both input
pathways in different time lags, one can discover that HVC-RA and LMAN-RA underwent
opposite plasticity (Mehaffey & Doupe, 2015). This suggests that the learning is gradually
transferred from LMAN-RA to HVC-RA pathway (Fee & Goldberg, 2011; Mehaffey & Doupe,
2015; Tesileanu et al., 2017). This indicates a general role of BG as the trainer for cortical
plasticity.
THE THALAMOCORTICAL STRUCTURE CONSOLIDATES THE BG SELECTIONS
ON THALAMIC CONTROL FUNCTIONS IN DIFFERENT TIMESCALES TO
ENABLE META-LEARNING
在这个部分, in addition to BG’s role as the trainer for cortical plasticity, we further propose
that BG is the trainer in two different timescales for thalamocortical structures to enable meta-
学习. The faster timescale trainer trains the corticothalamic connections to select the
网络神经科学
987
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
t
/
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
数字 3. Two-timescale learning in thalamocortical structures. We propose that one can learn the
thalamocortical structure to enable meta-learning by applying the general network motif in two dif-
ferent timescales. 第一的, one can learn the corticothalamic connections by applying the motif on the
blue loop with a faster timescale. This allows the network to consolidate flexible switching behav-
iors. 第二, one can learn the cortical connections by applying the motif on the orange loop in a
slower timescale. This allows cortical neurons to develop a task-relevant shared representation that
can generalize across contexts.
appropriate thalamic control functions in different contexts, while the slower timescale trainer
trains the cortical connections to form a task-relevant and generalizable representation.
From the songbird example, we see how thalamocortical structures can consolidate sim-
ple associations learned through the basal ganglia. To enable meta-learning, we propose that
this general network consolidation motif operates over two different timescales within
thalamocortical–basal ganglia interactions (数字 3). 第一的, combining the idea of thalamic
outputs as control functions over cortical network activity patterns and the basal ganglia
selecting such functions, we frame learning in basal ganglia as a process that connects con-
textual associations (higher order) with the appropriate dynamical control that maximizes
reward at the sensorimotor level (lower order). Under this framing, corticothalamic plasticity
consolidates the higher order association within a fast timescale. This allows flexible switch-
ing between different thalamic control functions in different contexts. 另一方面, 这
cortical plasticity consolidates the sensorimotor association over a slow timescale to allow
shared representation that can generalize across different contexts. As the thalamocortical
structures learn the higher order association, the behaviors become less BG-dependent
and the network is able to switch between different thalamic control functions to induce
different sensorimotor mappings in different contexts. By having two learning timescales, 年-
mals can conceivably both adapt quickly in changing environments with fast learning of
corticothalamic connections and maintain the important information across the environment
in the cortical connections. One should note that this separation of timescales is indepen-
dent from different timescales across cortex (高, van den Brink, Pfeffer, & Voytek, 2020;
J. D. Murray et al., 2014). While different timescales across cortex allows animals to process
information differentially, the separation of corticothalmic and cortical plasticity allows the
thalamocortical system to learn the higher contextual association to modulate cortical
dynamics flexibly.
Some anatomical observations support this idea. The thalamostriatal neurons have a more
modulatory role to the cortical dynamics in a diffusive projection, while thalamocortical neu-
rons have a more driver role to the cortical dynamic in a topographically restricted dense pro-
jection (Sherman & Guillery, 2005). This indicates that thalamostriatal neurons might serve as
the role of control functions in the faster consolidation loop with the feedback to striatum to
conduct credit assignment. 另一方面, thalamocortical neurons might be more
involved in the slower consolidation loop with the feedback to striatum coming from the cor-
tex to train the common cortical representation across contexts.
网络神经科学
988
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
t
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
总之, this two-timescale network consolidation scheme provides a general way for
BG to guide plasticity in the thalamocortical architecture to enable meta-learning and thus
solves structural credit assignment as a special case. Along these lines, experimental evidence
supports the notion that when faced with multisensory inputs, the BG can selectively disinhibit
a modality-specific subnetwork of the thalamic reticular nucleus (TRN) to filter out the sensory
inputs that are not relevant to the behavior outcomes and thus solve the structural credit
assignment problem.
In the discussion above, we discuss our proposal under a general formulation of thalamic
control functions. In the next section, we will specify other thalamic control functions sug-
gested by recent studies and observe how they can solve continual learning under this
framework as well.
THE THALAMUS SELECTIVELY AMPLIFIES FUNCTIONAL CORTICAL CONNECTIVITY
AS A SOLUTION TO CONTINUAL LEARNING AND CATASTROPHIC FORGETTING
One of the pitfalls of the artificial neural network is catastrophic forgetting. If one trains an
artificial neural network on a sequence of tasks, the performance on the older task will quickly
deteriorate as the network learns the new task (法语, 1999; Kemker et al., 2018; Kumaran
等人。, 2016; McCloskey & 科恩, 1989; Parisi et al., 2019). 另一方面, the brain can
achieve continual learning, the ability to learn different tasks in different contexts without cat-
astrophic forgetting and even generalize the performance to novel context (Lewkowicz, 2014;
中号. 中号. Murray et al., 2016; 力量 & 施拉加尔, 2017; Zenke, Gerstner, & 甘古利, 2017).
There are three main approaches in machine learning to deal with catastrophic forgetting.
第一的, one can use the regularization method to mostly update the weights that are less impor-
tant to the prior tasks (Fernando et al., 2017; 荣格, Ju, 荣格, & Kim, 2018; Kirkpatrick et al.,
2017; 李 & Hoiem, 2018; Maltoni & Lomonaco, 2019; Zenke, Poole, & 甘古利, 2017). 这
idea is inspired by experimental and theoretical studies on how synaptic information is selec-
tively protected in the brain (Benna & Fusi, 2016; Cichon & Gan, 2015; Fusi, Drew, & Abbott,
2005; Hayashi-Takagi et al., 2015; 哪个, Pan, & Gan, 2009). 然而, it is unclear how to
biologically compute the importance of each synapse to prior tasks nor how to do global reg-
ularization locally. 第二, one can also use a dynamic architecture in which the network
expands the architecture by allocating a subnetwork to train with the new information while
preserving old information (科尔特斯, Gonzalvo, 库兹涅佐夫, Mohri, & 哪个, 2017; Draelos et al.,
2017; Rusu et al., 2016; Xiao, 张, 哪个, 彭, & 张, 2014). 然而, this type of
method is not scalable since the number of neurons needs to scale linearly with the number
of tasks. 最后, one can use a memory buffer to replay past tasks to avoid catastrophic forget-
ting by interleaving the experience of the past tasks with the experience of the present task
(Kemker & Kanan, 2018; Kumaran et al., 2016; 麦克莱兰等人。, 1995; Shin, 李, Kim, &
Kim, 2017). 然而, this type of method cannot be the sole solution, as the memory buffer
needs to scale linearly with the number of tasks and potentially the number of trials.
We propose that the thalamus provides another way to solve continual learning and cata-
strophic forgetting via selectively amplifying parts of the cortical connections in different con-
文本 (数字 4). 具体来说, we propose that a population of thalamic neurons topographically
amplify the connectivity of cortical subnetworks as their control functions. During a behavioral
任务, BG selects subsets of the thalamus that selectively amplify the connectivity of cortical
subnetworks. Because of the reinforcement learning in BG, the subnetwork that is the most
relevant to the current task will be more preferentially activated and updated. By selecting only
the relevant subnetwork to activate in one context, the thalamus protects other subnetworks
网络神经科学
989
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
t
/
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
数字 4. A thalamocortical architecture with interaction with BG for continual learning. 期间
task execution, BG selects thalamic neurons that amplify the relevant cortical subnetwork. This pro-
tects other parts of the network that are important for another context from being overwritten. 什么时候
the other task comes, BG selects other thalamic neurons and since the synapses are protected from
the last task, animals can freely switch from different tasks without forgetting the previous tasks.
此外, as the corticothalamic synapses learn how to select the right thalamic neurons in a
different context (blue dashed line), task execution can become less BG dependent.
that can have useful information in another context from being overwritten. The corticothala-
mic structures can then consolidate these BG-guided flexible switching behaviors via our
proposed network motif, and the switching becomes less BG-dependent. 此外, 我们的
proposed solution has implications on generalization as well. Different tasks can have princi-
ples in common that can be transferred. 例如, although the rules of chess and Go are
very different, players in both games all need to predict what the other players are going to do
and counterattack based on the prediction. Since BG selects the subnetwork at each hierarchy
that is most relevant to the current tasks, in addition to selecting different subnetworks to pre-
vent catastrophic forgetting, BG can also select subnetworks that are beneficial to both tasks as
well to achieve generalization. 所以, the cortex can develop a modular hierarchical rep-
resentation of the world that can be easily generalized.
The idea of protecting relevant information from the past tasks to be overwritten has been
applied before computationally and has decent success in combating catastrophic forgetting in
deep learning (Kirkpatrick et al., 2017). Experimentally, we also have found that thalamic neu-
rons selectively amplify the cortical connectivity to solve the continual learning problem. 在一个
task where the mice need to switch between different sets of task cues that guided the attention
to the visual or auditory target, the performance of the mice does not deteriorate much after
switching to the original context, which is an indication of continual learning (Rikhye et al.,
2018). Through electrophysiological recording of PFC and mediodorsal thalamic nucleus
(医学博士) 神经元, we discovered that PFC neurons preferentially code for the rule of the attention,
while MD neurons preferentially code for the contexts of different sets of the cues. Thalamic
neurons that encode the task-relevant context translate this neural representation into the
amplification of cortical activity patterns associated with that context (despite the fact that cor-
tical neurons themselves only encode the context implicitly). These experimental observations
are consistent with our proposed solution: By incorporating the thalamic population that can
selectively amplify connectivity of cortical subnetworks, the thalamus and its interaction with
cortex and BG solve the continual learning problem and prevent catastrophic forgetting.
网络神经科学
990
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
t
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
结论
总之, in contrast to the traditional relay view of the thalamus, we propose that thala-
mocortical interaction is the locus of meta-learning where the thalamus provides cortical con-
trol functions, such as sensory filtering, working memory gating, or motor preparation, 那
parametrize the cortical activity association space. 此外, we propose a two-timescale
learning consolidation framework in which BG hierarchically selects these thalamic control
functions to enable meta-learning, solving the credit assignment problem. The faster plasticity
learns contextual associations to enable rapid behavioral flexibility, while the slower plasticity
establishes cortical representation that generalizes. By considering the recent observation of
the thalamus selectively amplifying functional cortical connectivity, the thalamocortical–basal
ganglia network is able to flexibly learn context-dependent associations without catastrophic
forgetting while generalizing to the new contexts. This modular account of the thalamocortical
interaction may seem to be in contrast with the recent proposed dynamical perspectives
(Barack & Krakauer, 2021) on thalamocortical interaction in which the thalamus shapes and
constrains the cortical attractor landscapes (Shine, 2021). We would like to argue that both
the modular and the dynamical perspectives are compatible with our proposal. The crux of
the perspectives is that the thalamus provides control functions that parametrize cortical
dynamics, and these control functions can be of modular nature or of dynamical nature
depending on their specific input-output connectivity. Flexible behaviors can be induced by
selecting either the control functions that amplify the appropriate cortical subnetworks or those
that adjust the cortical dynamics to the appropriate regimes.
作者贡献
Mien Wang: 概念化; 调查; 方法; Writing – original draft; Writing –
review & 编辑. Michael M. Halassa: 概念化; 资金获取; 方法;
监督; 写作——复习 & 编辑.
资金信息
Michael M. Halassa, National Institute of Mental Health (https://dx.doi.org/10.13039
/100000025), 奖项ID: 5R01MH120118-02.
参考
Abbott, L. F。, & 纳尔逊, S. 乙. (2000). Synaptic plasticity: Taming the
beast. 自然神经科学, 3, 1178–1183. https://doi.org/10
.1038/81453, 考研: 11127835
亚历山大, G. E., DeLong, 中号. R。, & 针织, 磷. L. (1986). Parallel
organization of functionally segregated circuits linking basal
ganglia and cortex. Annual Review of Neuroscience, 9, 357–381.
https://doi.org/10.1146/annurev.ne.09.030186.002041, 考研:
3085570
艾伦, 瓦. E., Kauvar, 我. 五、, 陈, 中号. Z。, Richman, 乙. B., 哪个, S. J。,
Chan, K., … Deisseroth, K. (2017). Global representations of
goal-directed behavior in distinct cell types of mouse neocortex.
神经元, 94(4), 891–907. https://doi.org/10.1016/j.neuron.2017
.04.017, 考研: 28521139
Andalman, A. S。, & Fee, 中号. S. (2009). A basal ganglia-forebrain
circuit in the songbird biases motor output to avoid vocal errors.
美国国家科学院院刊, 106(30),
12518–12523. https://doi.org/10.1073/pnas.0903214106,
考研: 19597157
Ashby, F. G。, Ennis, J. M。, & Spiering, 乙. J. (2007). A neurobiological
theory of automaticity in perceptual categorization. Psychologi-
cal Review, 114(3), 632–656. https://doi.org/10.1037/0033-295X
.114.3.632, 考研: 17638499
Asmus, F。, Huber, H。, Gasser, T。, & Schöls, L. (2008). Kick and rush:
Paradoxical kinesia in Parkinson disease. Neurology, 71(9), 695.
https://doi.org/10.1212/01.wnl.0000324618.88710.30, 考研:
18725599
Badre, D ., Kayser, A. S。, & D’Esposito, 中号. (2010). Frontal cortex
and the discovery of abstract action rules. 神经元, 66(2),
315–326. https://doi.org/10.1016/j.neuron.2010.03.025,
考研: 20435006
Bamford, 氮. S。, Wightman, 右. M。, & Sulzer, D. (2018). Dopamine’s
effects on corticostriatal synapses during reward-based
网络神经科学
991
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
t
/
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
t
.
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
behaviors. 神经元, 97(3), 494–510. https://doi.org/10.1016/j
.neuron.2018.01.006, 考研: 29420932
Barack, D. L。, & Krakauer, J. 瓦. (2021). Two views on the cognitive
脑. 自然评论神经科学, 22(6), 359–371. https://土井
.org/10.1038/s41583-021-00448-6, 考研: 33859408
Bayer, H. M。, & Glimcher, 磷. 瓦. (2005). Midbrain dopamine neurons
encode a quantitative reward prediction error signal. 神经元,
47(1), 129–141. https://doi.org/10.1016/j.neuron.2005.05.020,
考研: 15996553
Benna, 中号. K., & Fusi, S. (2016). Computational principles of synaptic
memory consolidation. 自然神经科学, 19(12), 1697–1706.
https://doi.org/10.1038/nn.4401, 考研: 27694992
Bliss, 时间. 五、, & Lomo, 时间. (1973). Long-lasting potentiation of synaptic
transmission in the dentate area of the anaesthetized rabbit
following stimulation of the perforant path. 生理学杂志,
232(2), 331–356. https://doi.org/10.1113/jphysiol.1973
.sp010273, 考研: 4727084
Bolkan, S. S。, Stujenske, J. M。, Parnaudeau, S。, Spellman, 时间. J。,
Rauffenbart, C。, Abbas, A. 我。, … Kellendonk, C. (2017). Thalamic
projections sustain prefrontal activity during working memory
维护. 自然神经科学, 20(7), 987–996. https://土井
.org/10.1038/nn.4568, 考研: 28481349
博特维尼克, M。, 里特尔, S。, 王, J. X。, Kurth-Nelson, Z。, Blundell, C。,
& Hassabis, D. (2019). Reinforcement learning, fast and slow.
认知科学的趋势, 23(5), 408–422. https://doi.org/10
.1016/j.tics.2019.02.006, 考研: 31003893
Cadieu, C. F。, 洪, H。, Yamins, D. L. K., Pinto, N。, Ardila, D .,
Solomon, 乙. A。, … DiCarlo, J. J. (2014). Deep neural networks
rival the representation of primate IT cortex for core visual object
认出. 公共科学图书馆计算生物学, 10(12), 1–18. https://
doi.org/10.1371/journal.pcbi.1003963, 考研: 25521294
Cass, 瓦. A。, & Gerhardt, G. A. (1995). In vivo assessment of dopa-
mine uptake in rat medial prefrontal cortex: Comparison with
dorsal striatum and nucleus accumbens. Journal of Neurochem-
istry, 65(1), 201–207. https://doi.org/10.1046/j.1471-4159.1995
.65010201.X, 考研: 7790861
Cichon, J。, & Gan, 瓦. 乙. (2015). Branch-specific dendritic Ca(2+)
spikes cause persistent synaptic plasticity. 自然, 520(7546),
180–185. https://doi.org/10.1038/nature14251, 考研:
25822789
Ciliax, 乙. J。, Heilman, C。, Demchyshyn, L. L。, Pristupa, Z. B., Ince, E.,
Hersch, S. M。, … Levey, A. 我. (1995). The dopamine transporter:
Immunochemical characterization and localization in brain.
神经科学杂志, 15(3 Pt. 1), 1714–1723. https://doi.org
/10.1523/JNEUROSCI.15-03-01714.1995, 考研: 7534339
库克, S. F。, & Bear, 中号. F. (2010). Visual experience induces
long-term potentiation in the primary visual cortex. 杂志
神经科学, 30(48), 16304–16313. https://doi.org/10.1523
/JNEUROSCI.4333-10.2010, 考研: 21123576
科尔特斯, C。, Gonzalvo, X。, 库兹涅佐夫, 五、, Mohri, M。, & 哪个, S.
(2017). AdaNet: Adaptive structural learning of artificial neural
网络. In Proceedings of the 34th international conference
on machine learning (卷. 70, PP. 874–883). Retrieved from
https://proceedings.mlr.press/v70/cortes17a.html
考克斯, J。, & Witten, 我. 乙. (2019). Striatal circuits for reward learning
and decision-making. 自然评论神经科学, 20(8),
482–494. https://doi.org/10.1038/s41583-019-0189-2, 考研:
31171839
Crick, F. (1989). The recent excitement about neural networks.
自然, 337(6203), 129–132. https://doi.org/10.1038/337129a0,
考研: 2911347
Dayan, P。, & Abbott, L. F. (2005). Theoretical neuroscience:
Computational and mathematical modeling of neural systems.
与新闻界.
多纳休, C. H。, & 李, D. (2015). Dynamic routing of task-
relevant signals for decision making in dorsolateral prefrontal
cortex. 自然神经科学, 18(2), 295–301. https://doi.org/10
.1038/nn.3918, 考研: 25581364
Doya, K. (1999). What are the computations of the cerebellum, 这
basal ganglia and the cerebral cortex? Neural Networks, 12(7–8),
961–974. https://doi.org/10.1016/S0893-6080(99)00046-5,
考研: 12662639
Doya, K. (2000). Complementary roles of basal ganglia and cere-
bellum in learning and motor control. Current Opinion in Neu-
robiology, 10(6), 732–739. https://doi.org/10.1016/S0959-4388
(00)00153-7, 考研: 11240282
Draelos, 时间. J。, Miner, 氮. E., Lamb, C. C。, 考克斯, J. A。, Vineyard, C. M。,
卡尔森, K. D ., … Aimone, J. 乙. (2017). Neurogenesis deep learn-
英: Extending deep networks to accommodate new classes. 在
2017 international joint conference on neural networks (IJCNN)
(PP. 526–533). https://doi.org/10.1109/IJCNN.2017.7965898
Enel, P。, Wallis, J. D ., & Rich, 乙. L. (2020). Stable and dynamic
representations of value in the prefrontal cortex. 电子生活, 9, e54313.
https://doi.org/10.7554/eLife.54313, 考研: 32628108
Fee, 中号. S。, & Goldberg, J. H. (2011). A hypothesis for basal ganglia–
dependent reinforcement learning in the songbird. 神经科学,
198, 152–170. https://doi.org/10.1016/j.neuroscience.2011.09
.069, 考研: 22015923
费尔德曼, D. 乙. (2009). Synaptic mechanisms for plasticity in neocortex.
Annual Review of Neuroscience, 32, 33–55. https://doi.org/10.1146
/annurev.neuro.051508.135516, 考研: 19400721
Fernando, C。, Banarse, D ., Blundell, C。, Zwols, Y。, Ha, D ., Rusu,
A. A。, … Wierstra, D. (2017). Pathnet: Evolution channels gradi-
ent descent in super neural networks. CoRR, abs/1701.08734.
Retrieved from https://arxiv.org/abs/1701.08734. https://doi.org
/10.48550/arXiv.1701.08734
Fiete, 我. R。, Fee, 中号. S。, & Seung, H. S. (2007). Model of birdsong
learning based on gradient estimation by dynamic perturbation
of neural conductances. 神经生理学杂志, 98(4),
2038–2057. https://doi.org/10.1152/jn.01311.2006, 考研:
17652414
Fiete, 我. R。, & Seung, H. S. (2006). Gradient learning in spiking neu-
ral networks by dynamic perturbation of conductances. Physical
Review Letters, 97, 048104. https://doi.org/10.1103/PhysRevLett
.97.048104, 考研: 16907616
法语, 右. 中号. (1999). Catastrophic forgetting in connectionist net-
作品. 认知科学的趋势, 3(4), 128–135. https://土井
.org/10.1016/S1364-6613(99)01294-2, 考研: 10322466
Fusi, S。, Drew, 磷. J。, & Abbott, L. F. (2005). Cascade models of syn-
aptically stored memories. 神经元, 45(4), 599–611. https://土井
.org/10.1016/j.neuron.2005.02.001, 考研: 15721245
Fuster, J. (1997). The prefrontal cortex: Anatomy, physiology, 和
neuropsychology of the frontal lobe. Lippincott-Raven. Retrieved
来自 https://books.google.com/books?id=YupqAAAAMAAJ
高, R。, van den Brink, 右. L。, Pfeffer, T。, & Voytek, 乙. (2020). 新-
ronal timescales are functionally dynamic and shaped by cortical
网络神经科学
992
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
/
t
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
t
.
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
microarchitecture. 电子生活, 9, e61277. https://doi.org/10.7554/eLife
.61277, 考研: 33226336
Garris, 磷. A。, & Wightman, 右. 中号. (1994). Different kinetics govern
dopaminergic transmission in the amygdala, 前额皮质,
and striatum: An in vivo voltammetric study. Journal of Neurosci-
恩斯, 14(1), 442–450. https://doi.org/10.1523/ JNEUROSCI.14
-01-00442.1994, 考研: 8283249
Gerfen, C。, & Bolam, J. (2010). The neuroanatomical organization
of the basal ganglia. Handbook of Behavioral Neuroscience, 20,
3–28. https://doi.org/10.1016/B978-0-12-374767-9.00001-9
Guo, W., Clause, A. R。, Barth-Maron, A。, & Polley, D. 乙. (2017). A
corticothalamic circuit for dynamic switching between feature
detection and discrimination. 神经元, 95(1), 180–194. https://
doi.org/10.1016/j.neuron.2017.05.019, 考研: 28625486
Guo, Z. 五、, Inagaki, H. K., Daie, K., Druckmann, S。, Gerfen, C. R。, &
Svoboda, K. (2017). Maintenance of persistent activity in a fron-
tal thalamocortical loop. 自然, 545(7653), 181–186. https://土井
.org/10.1038/nature22324, 考研: 28467817
哈里斯, J. A。, Mihalas, S。, Hirokawa, K. E., Whitesell, J. D ., Choi, H。,
Bernard, A。, … Zeng, H. (2019). Hierarchical organization of cor-
tical and thalamic connectivity. 自然, 575(7781), 195–202.
https://doi.org/10.1038/s41586-019-1716-z, 考研: 31666704
Hayashi-Takagi, A。, Yagishita, S。, Nakamura, M。, Shirai, F。, 吴,
是. 我。, Loshbaugh, A. L。, … Kasai, H. (2015). Labelling and optical
erasure of synaptic memory traces in the motor cortex. 自然,
525(7569), 333–338. https://doi.org/10.1038/nature15257,
考研: 26352471
他, K., 张, X。, Ren, S。, & Sun, J. (2016). Deep residual learning
for image recognition. 在 2016 IEEE conference on computer
vision and pattern recognition (CVPR) (PP. 770–778). https://土井
.org/10.1109/CVPR.2016.90
Hebb, D. (2002). The organization of behavior: A neuropsycholog-
ical theory. 泰勒 & Francis. Retrieved from https://books.google
.com/books?id=gUtwMochAI8C
Hikosaka, 奥。, Kim, H. F。, Yasuda, M。, & Yamamoto, S. (2014). Basal
ganglia circuits for reward value-guided behavior. Annual
Review of Neuroscience, 37, 289–306. https://doi.org/10.1146
/annurev-neuro-071013-013924, 考研: 25032497
Hoffmann, K. P。, Stone, J。, & Sherman, S. 中号. (1972). Relay of
receptive-field properties in dorsal lateral geniculate nucleus of
the cat. 神经生理学杂志, 35(4), 518–531. https://土井
.org/10.1152/jn.1972.35.4.518, 考研: 4338566
Houk, J. C。, 戴维斯, J. L。, & Beiser, D. G. (1994). Adaptive critics and
the basal ganglia. In Models of information processing in the
基底神经节 (PP. 215–232). 与新闻界. https://doi.org/10.7551
/mitpress/4708.003.0018
Hubel, D. H。, & Wiesel, 时间. 氮. (1961). Integrative action in the cat’s
lateral geniculate body. 生理学杂志, 155, 385–398.
https://doi.org/10.1113/jphysiol.1961.sp006635, 考研:
13716436
Hubel, D. H。, & Wiesel, 时间. 氮. (1962). Receptive fields, binocular
interaction and functional architecture in the cat’s visual cortex.
生理学杂志, 160, 106–154. https://doi.org/10.1113
/jphysiol.1962.sp006837, 考研: 14449617
Hunnicutt, 乙. J。, Jongbloets, 乙. C。, Birdsong, 瓦. T。, Gertz, K. J。,
Zhong, H。, & Mao, 时间. (2016). A comprehensive excitatory input
map of the striatum reveals novel functional organization. 电子生活, 5,
e19103. https://doi.org/10.7554/eLife.19103, 考研: 27892854
Hélie, S。, Ell, S. W., & Ashby, F. G. (2015). Learning robust
cortico-cortical associations with the basal ganglia: An integra-
tive review. Cortex, 64, 123–135. https://doi.org/10.1016/j
.cortex.2014.10.011, 考研: 25461713
Ikemoto, S。, & Panksepp, J. (1999). The role of nucleus accumbens
dopamine in motivated behavior: A unifying interpretation with
special reference to reward-seeking. Brain Research Reviews, 31(1),
6–41. https://doi.org/10.1016/S0165-0173(99)00023-5, 考研:
10611493
Jacobs, D. S。, & Moghaddam, 乙. (2020). Prefrontal cortex represen-
tation of learning of punishment probability during reward-
motivated actions. 神经科学杂志, 40(26), 5063–5077.
https://doi.org/10.1523/ JNEUROSCI.0310-20.2020, 考研:
32409619
Jiang, H。, & Kim, H. F. (2018). Anatomical inputs from the sensory
and value structures to the tail of the rat striatum. Frontiers in
Neuroanatomy, 12, 30. https://doi.org/10.3389/fnana.2018
.00030, 考研: 29773980
琼斯, 乙. G. (埃德。). (1985). The thalamus. Springer US. https://doi.org
/10.1007/978-1-4615-1749-8
荣格, H。, Ju, J。, 荣格, M。, & Kim, J. (2018). Less-forgetful learning for
domain expansion in deep neural networks. In AAAI conference
on artificial intelligence. Retrieved from https://www.aaai.org/ocs
/index.php/AAAI/AAAI18/paper/view/17073
Kemker, R。, & Kanan, C. (2018). FearNet: Brain-inspired model for
incremental learning. In International conference on learning rep-
resentations. Retrieved from https://openreview.net/forum?id
=SJ1Xmf-Rb
Kemker, R。, McClure, M。, Abitino, A。, 海耶斯, T。, & Kanan, C.
(2018). Measuring catastrophic forgetting in neural networks. 在
AAAI conference on artificial intelligence. Retrieved from https://
aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16410
Ketz, N。, Morkonda, S. G。, & O’Reilly, 右. C. (2013). Theta coordi-
nated error-driven learning in the hippocampus. PLoS Computa-
tional Biology, 9(6), 1–9. https://doi.org/10.1371/journal.pcbi
.1003067, 考研: 23762019
Kim, C。, 约翰逊, 氮. F。, Cilles, S. E., & 金子, 乙. 时间. (2011). Common
and distinct mechanisms of cognitive flexibility in prefrontal cor-
tex. 神经科学杂志, 31(13), 4771–4779. https://doi.org
/10.1523/JNEUROSCI.5923-10.2011, 考研: 21451015
柯克帕特里克, J。, Pascanu, R。, Rabinowitz, N。, Veness, J。, Desjardins,
G。, Rusu, A. A。, … Hadsell, 右. (2017). Overcoming catastrophic
forgetting in neural networks. Proceedings of the National Acad-
emy of Sciences, 114(13), 3521–3526. https://doi.org/10.1073
/pnas.1611835114, 考研: 28292907
Kirkwood, A。, Rioult, 中号. C。, & Bear, 中号. F. (1996). 经验-
dependent modification of synaptic plasticity in visual cortex.
自然, 381(6582), 526–528. https://doi.org/10.1038/381526a0,
考研: 8632826
Kornfeld, J。, Januszewski, M。, Schubert, P。, Jain, 五、, Denk, W., &
Fee, 中号. (2020). An anatomical substrate of credit assignment in
reinforcement learning. bioRxiv. https://doi.org/10.1101/2020.02
.18.954354
克里热夫斯基, A。, 吸勺, 我。, & 欣顿, G. 乙. (2012). ImageNet classi-
fication with deep convolutional neural networks. In Advances in
神经信息处理系统 (卷. 25). 柯伦联合公司,
Inc. Retrieved from https://proceedings.neurips.cc/paper/2012
/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
网络神经科学
993
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
/
t
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
Kumaran, D ., Hassabis, D ., & 麦克莱兰, J. L. (2016). 什么
learning systems do intelligent agents need? Complementary
learning systems theory updated. 认知科学的趋势,
20(7), 512–534. https://doi.org/10.1016/j.tics.2016.05.004,
考研: 27315762
Kusmierz, L。, Isomura, T。, & Toyoizumi, 时间. (2017). Learning with
three factors: modulating Hebbian plasticity with errors. 当前的
Opinion in Neurobiology, 46, 170–177. https://doi.org/10.1016/j
.conb.2017.08.020, 考研: 28918313
Lanciego, J. L。, Luquin, N。, & Obeso, J. A. (2012). Functional neu-
roanatomy of the basal ganglia. Cold Spring Harbor Perspectives
in Medicine, 2(12), a009621. https://doi.org/10.1101
/cshperspect.a009621, 考研: 23071379
Lapish, C. C。, Kroener, S。, Durstewitz, D ., Lavin, A。, & Seamans,
J. K. (2007). The ability of the mesocortical dopamine system
to operate in distinct temporal modes. Psychopharmacology,
191(3), 609–625. https://doi.org/10.1007/s00213-006-0527-8,
考研: 17086392
Lewkowicz, D. J. (2014). Early experience and multisensory
perceptual narrowing. Developmental Psychobiology, 56(2),
292–315. https://doi.org/10.1002/dev.21197, 考研:
24435505
李, Z。, & Hoiem, D. (2018). Learning without forgetting. IEEE Trans-
actions on Pattern Analysis and Machine Intelligence, 40(12),
2935–2947. https://doi.org/10.1109/ TPAMI.2017.2773081,
考研: 29990101
Lien, A. D ., & Scanziani, 中号. (2018). Cortical direction selectivity
emerges at convergence of thalamic synapses. 自然, 558(7708),
80–86. https://doi.org/10.1038/s41586-018-0148-5, 考研:
29795349
Lillicrap, 时间. P。, Cownden, D ., Tweed, D. B., & Akerman, C. J.
(2016). Random synaptic feedback weights support error back-
propagation for deep learning. Nature Communications, 7,
13276. https://doi.org/10.1038/ncomms13276, 考研:
27824044
Lillicrap, 时间. P。, Santoro, A。, Marris, L。, Akerman, C. J。, & 欣顿, G.
(2020). Backpropagation and the brain. Nature Reviews Neuro-
科学, 21(6), 335–346. https://doi.org/10.1038/s41583-020
-0277-3, 考研: 32303713
刘, 是. H。, 史密斯, S。, Mihalas, S。, Shea-Brown, E., & Sümbül, U.
(2020). A solution to temporal credit assignment using
cell-type-specific modulatory signals. bioRxiv. https://doi.org/10
.1101/2020.11.22.393504
Makino, H。, Hwang, 乙. J。, Hedrick, 氮. G。, & Komiyama, 时间. (2016).
Circuit mechanisms of sensorimotor learning. 神经元, 92(4),
705–721. https://doi.org/10.1016/j.neuron.2016.10.029,
考研: 27883902
Maltoni, D ., & Lomonaco, V. (2019). Continuous learning in
single-incremental-task scenarios. Neural Networks, 116,
56–73. https://doi.org/10.1016/j.neunet.2019.03.010, 考研:
31005851
Mante, 五、, Sussillo, D ., 谢诺伊, K. 五、, & Newsome, 瓦. 时间. (2013).
Context-dependent computation by recurrent dynamics in pre-
frontal cortex. 自然, 503(7474), 78–84. https://doi.org/10
.1038/nature12742, 考研: 24201281
Marton, 时间. F。, Seifikar, H。, Luongo, F. J。, 李, A. T。, & Sohal, V. S.
(2018). Roles of prefrontal cortex and mediodorsal thalamus
in task engagement and behavioral flexibility. 杂志
神经科学, 38(10), 2569–2578. https://doi.org/10.1523
/JNEUROSCI.1728-17.2018, 考研: 29437889
麦克莱兰, J. L。, McNaughton, 乙. L。, & O’Reilly, 右. C. (1995).
Why there are complementary learning systems in the hippo-
campus and neocortex: Insights from the successes and failures
of connectionist models of learning and memory. Psychological
审查, 102(3), 419–457. https://doi.org/10.1037/0033-295X
.102.3.419, 考研: 7624455
McCloskey, M。, & 科恩, 氮. J. (1989). Catastrophic interference
in connectionist networks: The sequential learning problem.
In G. H. Bower (埃德。), Psychology of learning and motivation
(卷. 24, PP. 109–165). 学术出版社. https://doi.org/10.1016
/S0079-7421(08)60536-8
McNab, F。, & Klingberg, 时间. (2008). Prefrontal cortex and basal
ganglia control access to working memory. 自然神经科学,
11(1), 103–107. https://doi.org/10.1038/nn2024, 考研:
18066057
Mehaffey, 瓦. H。, & Doupe, A. J. (2015). Naturalistic stimulation
drives opposing heterosynaptic plasticity at two inputs to song-
bird cortex. 自然神经科学, 18(9), 1272–1280. https://土井
.org/10.1038/nn.4078, 考研: 26237364
磨坊主, 乙. K. (2000). The prefontral cortex and cognitive control.
自然评论神经科学, 1(1), 59–65. https://doi.org/10
.1038/35036228, 考研: 11252769
磨坊主, 乙. K., & 科恩, J. D. (2001). An integrative theory of pre-
frontal cortex function. Annual Review of Neuroscience, 24,
167–202. https://doi.org/10.1146/annurev.neuro.24.1.167,
考研: 11283309
Minsky, 中号. (1961). Steps toward artificial intelligence. 会议记录
of the IRE, 49(1), 8–30. https://doi.org/10.1109/ JRPROC.1961
.287775
Mnih, 五、, Kavukcuoglu, K., Silver, D ., Rusu, A. A。, Veness, J。,
Bellemare, 中号. G。, … Hassabis, D. (2015). Human-level control
through deep reinforcement learning. 自然, 518(7540),
529–533. https://doi.org/10.1038/nature14236, 考研:
25719670
Monchi, 奥。, Petrides, M。, Strafella, A. P。, Worsley, K. J。, & Doyon, J.
(2006). Functional role of the basal ganglia in the planning and
execution of actions. Annals of Neurology, 59(2), 257–264.
https://doi.org/10.1002/ana.20742, 考研: 16437582
Montague, 磷. R。, Dayan, P。, & Sejnowski, 时间. J. (1996). A framework
for mesencephalic dopamine systems based on predictive
Hebbian learning. 神经科学杂志, 16(5), 1936–1947.
https://doi.org/10.1523/ JNEUROSCI.16-05-01936.1996,
考研: 8774460
莫里斯, G。, Nevet, A。, Arkadir, D ., Vaadia, E., & 伯格曼, H.
(2006). Midbrain dopamine neurons encode decisions for future
行动. 自然神经科学, 9(8), 1057–1063. https://doi.org/10
.1038/nn1743, 考研: 16862149
Mukherjee, A。, Bajwa, N。, Lam, 氮. H。, Porrero, C。, Clasca, F。, &
Halassa, 中号. 中号. (2020). Variation of connectivity across exemplar
sensory and associative thalamocortical loops in the mouse.
电子生活, 9, e62554. https://doi.org/10.7554/eLife.62554, 考研:
33103997
Mukherjee, A。, Lam, 氮. H。, Wimmer, 右. D ., & Halassa, 中号. 中号.
(2021). Thalamic circuits for independent control of prefrontal
signal and noise. 自然, 600(7887), 100–104. https://doi.org
/10.1038/s41586-021-04056-3, 考研: 34614503
网络神经科学
994
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
t
/
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
t
.
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
穆雷, J. D ., Bernacchia, A。, 弗里德曼, D. J。, 罗莫, R。, Wallis,
J. D ., Cai, X。, … Wang, X. J. (2014). A hierarchy of intrinsic
timescales across primate cortex. 自然神经科学, 17(12),
1661–1663. https://doi.org/10.1038/nn.3862, 考研:
25383900
穆雷, 中号. M。, Lewkowicz, D. J。, Amedi, A。, & 华莱士, 中号. 时间.
(2016). Multisensory processes: A balancing act across the
lifespan. Trends in Neurosciences, 39(8), 567–579. https://土井
.org/10.1016/j.tins.2016.05.003, 考研: 27282408
Nakajima, M。, 施密特, L. 我。, & Halassa, 中号. 中号. (2019). Prefrontal
cortex regulates sensory filtering through a basal ganglia-to-
thalamus pathway. 神经元, 103(3), 445–458. https://doi.org/10
.1016/j.neuron.2019.05.026, 考研: 31202541
Nambu, A. (2011). Somatotopic organization of the primate basal
ganglia. Frontiers in Neuroanatomy, 5, 26. https://doi.org/10
.3389/fnana.2011.00026, 考研: 21541304
尼夫, 是. (2009). Reinforcement learning in the brain. 杂志
Mathematical Psychology, 53(3), 139–154. https://doi.org/10
.1016/j.jmp.2008.12.005
O’Reilly, 右. C. (1996). Biologically plausible error-driven learning
using local activation differences: The generalized recirculation
algorithm. 神经计算, 8(5), 895–938. https://doi.org/10
.1162/neco.1996.8.5.895
O’Reilly, 右. C。, Russin, J. L。, Zolfaghar, M。, & Rohrlich, J. (2021).
Deep predictive learning in neocortex and pulvinar. 杂志
Cognitive Neuroscience, 33(6), 1158–1196. https://doi.org/10
.1162/jocn_a_01708, 考研: 34428793
帕里西, G. 我。, Kemker, R。, 部分, J. L。, Kanan, C。, & Wermter, S. (2019).
Continual lifelong learning with neural networks: A review.
Neural Networks, 113, 54–71. https://doi.org/10.1016/j.neunet
.2019.01.012, 考研: 30780045
Perrin, E., & Venance, L. (2019). Bridging the gap between striatal
plasticity and learning. 神经生物学的当前观点, 54,
104–112. https://doi.org/10.1016/j.conb.2018.09.007, 考研:
30321866
Peters, A. J。, 法布尔, J. 中号. J。, Steinmetz, 氮. A。, 哈里斯, K. D ., &
Carandini, 中号. (2021). Striatal activity topographically reflects
cortical activity. 自然, 591, 420–425. https://doi.org/10.1038
/s41586-020-03166-8, 考研: 33473213
彼得森, C. C. H. (2019). Sensorimotor processing in the rodent
barrel cortex. 自然评论神经科学, 20(9), 533–546.
https://doi.org/10.1038/s41583-019-0200-y, 考研: 31367018
Phillips, J. M。, Kambi, 氮. A。, & Saalmann, 是. 乙. (2016). A subcorti-
cal pathway for rapid, goal-driven, attentional filtering. 趋势
神经科学, 39(2), 49–51. https://doi.org/10.1016/j.tins.2015
.12.003, 考研: 26743499
力量, J. D ., & 施拉加尔, 乙. L. (2017). Neural plasticity across the
lifespan. Wiley Interdisciplinary Reviews: Developmental Biology,
6(1), e216. https://doi.org/10.1002/wdev.216, 考研: 27911497
Rakic, 磷. (2009). Evolution of the neocortex: A perspective from
developmental biology. 自然评论神经科学, 10(10),
724–735. https://doi.org/10.1038/nrn2719, 考研: 19763105
Reinagel, P。, Godwin, D ., Sherman, S. M。, & 科赫, C. (1999).
Encoding of visual information by LGN bursts. Journal of Neuro-
physiology, 81(5), 2558–2569. https://doi.org/10.1152/jn.1999
.81.5.2558, 考研: 10322089
Richards, 乙. A。, & Lillicrap, 时间. 磷. (2019). Dendritic solutions to the
credit assignment problem. 神经生物学的当前观点,
54, 28–36. https://doi.org/10.1016/j.conb.2018.08.003,
考研: 30205266
Rikhye, 右. 五、, Gilra, A。, & Halassa, 中号. 中号. (2018). Thalamic
regulation of switching between cortical representations enables
cognitive flexibility. 自然神经科学, 21(12), 1753–1763.
https://doi.org/10.1038/s41593-018-0269-z, 考研:
30455456
罗尔夫塞玛, 磷. R。, & Holtmaat, A. (2018). Control of synaptic plas-
ticity in deep cortical networks. 自然评论神经科学,
19(3), 166–180. https://doi.org/10.1038/nrn.2018.6, 考研:
29449713
罗尔夫塞玛, 磷. R。, & van Ooyen, A. (2005). Attention-gated rein-
forcement learning of internal representations for classification.
神经计算, 17(10), 2176–2214. https://doi.org/10
.1162/0899766054615699, 考研: 16105222
Roesch, 中号. R。, Calu, D. J。, & Schoenbaum, G. (2007). Dopamine
neurons encode the better option in rats deciding between
differently delayed or sized rewards. 自然神经科学,
10(12), 1615–1624. https://doi.org/10.1038/nn2013, 考研:
18026098
Rumelhart, D. E., 欣顿, G. E., & 威廉姆斯, 右. J. (1986). 学习
representations by back-propagating errors. 自然, 323(6088),
533–536. https://doi.org/10.1038/323533a0
Rusu, A. A。, Rabinowitz, 氮. C。, Desjardins, G。, Soyer, H。, 柯克帕特里克,
J。, Kavukcuoglu, K., … Hadsell, 右. (2016). Progressive neural
网络. CoRR, abs/1606.04671. Retrieved from https://arxiv
.org/abs/1606.04671. https://doi.org/10.48550/arXiv.1606.04671
Saalmann, 是. B., & Kastner, S. (2015). The cognitive thalamus. Fron-
tiers in Systems Neuroscience, 9, 39. https://doi.org/10.3389
/fnsys.2015.00039, 考研: 25852498
萨克拉门托, J。, Ponte Costa, R。, 本吉奥, Y。, & Senn, 瓦. (2018).
Dendritic cortical microcircuits approximate the backpropaga-
tion algorithm. In Advances in neural information processing sys-
特姆斯 (卷. 31, PP. 8735–8746). 柯伦联合公司, Inc. Retrieved
f r o m h t t p s : / / p r o c e e d i n g s . n e u r i p s . c c / p a p e r / 2 0 1 8 / f i l e
/1dc3a89d0d440ba31729b0ba74b93a33-Paper.pdf
Scharff, C。, & Nottebohm, F. (1991). A comparative study of the
behavioral deficits following lesions of various parts of the zebra
finch song system: Implications for vocal learning. 杂志
神经科学, 11(9), 2896–2913. https://doi.org/10.1523
/JNEUROSCI.11-09-02896.1991, 考研: 1880555
Schiess, M。, Urbanczik, R。, & Senn, 瓦. (2016). Somato-dendritic
synaptic plasticity and error-backpropagation in active dendrites.
公共科学图书馆计算生物学, 12(2), 1–18. https://doi.org/10
.1371/journal.pcbi.1004638, 考研: 26841235
施密特, L. 我。, Wimmer, 右. D ., Nakajima, M。, Happ, M。, Mofakham,
S。, & Halassa, 中号. 中号. (2017). Thalamic amplification of cortical
connectivity sustains attentional control. 自然, 545(7653),
219–223. https://doi.org/10.1038/nature22073, 考研:
28467827
Schrittwieser, J。, Antonoglou, 我。, Hubert, T。, Simonyan, K., Sifre, L。,
施密特, S。, … Silver, D. (2020). Mastering Atari, Go, chess and
shogi by planning with a learned model. 自然, 588(7839),
604–609. https://doi.org/10.1038/s41586-020-03051-4,
考研: 33361790
Schultz, W., Dayan, P。, & Montague, 磷. 右. (1997). A neural substrate
of prediction and reward. 科学, 275(5306), 1593–1599. https://
doi.org/10.1126/science.275.5306.1593, 考研: 9054347
网络神经科学
995
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
t
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
Seamans, J. K., & Robbins, 时间. 瓦. (2010). Dopamine modulation of
the prefrontal cortex and cognitive function. In The dopamine
receptors (PP. 373–398). Totowa, 新泽西州: Humana Press. https://土井
.org/10.1007/978-1-60327-333-6_14
Seo, M。, 李, E., & Averbeck, 乙. 乙. (2012). Action selection and
action value in frontal-striatal circuits. 神经元, 74(5), 947–960.
https://doi.org/10.1016/j.neuron.2012.03.037, 考研:
22681697
Sherman, S. M。, & Guillery, 右. 瓦. (2005). Exploring the thalamus
and its role in cortical function (2nd 版。). 与新闻界.
Sherman, S. M。, & Spear, 磷. D. (1982). Organization of visual path-
ways in normal and visually deprived cats. Physiological
评论, 62(2), 738–855. https://doi.org/10.1152/physrev.1982
.62.2.738, 考研: 6280221
Shin, H。, 李, J. K., Kim, J。, & Kim, J. (2017). Continual learning with
deep generative replay. In Advances in neural information
processing systems (卷. 30). 柯伦联合公司, Inc. Retrieved
f r o m h t t p s : / / p r o c e e d i n g s . n e u r i p s . c c / p a p e r / 2 0 1 7 / f i l e
/0efbe98067c6c73dba1250d2beaa81f9-Paper.pdf
Shine, J. 中号. (2021). The thalamus integrates the macrosystems of
the brain to facilitate complex, adaptive brain network dynamics.
Progress in Neurobiology, 199, 101951. https://doi.org/10.1016/j
.pneurobio.2020.101951, 考研: 33189781
Silver, D ., 黄, A。, Maddison, C. J。, Guez, A。, Sifre, L。, van den
Driessche, G。, … Hassabis, D. (2016). Mastering the game of Go
with deep neural networks and tree search. 自然, 529(7587),
484–489. https://doi.org/10.1038/nature16961, 考研:
26819042
Silver, D ., Schrittwieser, J。, Simonyan, K., Antonoglou, 我。, 黄,
A。, Guez, A。, … Hassabis, D. (2017). Mastering the game of Go
without human knowledge. 自然, 550(7676), 354–359. https://
doi.org/10.1038/nature24270, 考研: 29052630
歌手, W., Sejnowski, T。, & Rakic, 磷. (2019). The neocortex. 和
按. Retrieved from https:// books.google.com/ books?id
=aL60DwAAQBAJ. https://doi.org/10.7551/mitpress/12593.001
.0001
Sohn, H。, Meirhaeghe, N。, Rajalingham, R。, & Jazayeri, 中号. (2021).
A network perspective on sensorimotor learning. Trends in Neu-
rosciences, 44(3), 170–181. https://doi.org/10.1016/j.tins.2020
.11.007, 考研: 33349476
Sohrabji, F。, Nordeen, 乙. J。, & Nordeen, K. 瓦. (1990). Selective
impairment of song learning following lesions of a forebrain
nucleus in the juvenile zebra finch. Behavioral and Neural Biology,
53(1), 51–63. https://doi.org/10.1016/0163-1047(90)90797-A,
考研: 2302141
Soliveri, P。, 棕色的, 右. G。, Jahanshahi, M。, Caraceni, T。, & Marsden,
C. D. (1997). Learning manual pursuit tracking skills in patients
with Parkinson’s disease. Brain, 120(Pt. 8), 1325–1337. https://
doi.org/10.1093/brain/120.8.1325, 考研: 9278626
苏里, 右. E., & Schultz, 瓦. (1999). A neural network model with
dopamine-like reinforcement signal that learns a spatial delayed
response task. 神经科学, 91(3), 871–890. https://doi.org/10
.1016/S0306-4522(98)00697-6, 考研: 10391468
Sutton, R。, & Barto, A. (2018). Reinforcement learning: An introduc-
的. 与新闻界. Retrieved from https://books.google.com/books
?id=sWV0DwAAQBAJ
Sutton, 右. S。, & Barto, A. G. (1990). Time-derivative models of
Pavlovian reinforcement. In Learning and computational
神经科学: Foundations of adaptive networks (PP. 497–537).
与新闻界.
Tanaka, 中号. (2007). Cognitive signals in the primate motor thala-
mus predict saccade timing. 神经科学杂志, 27(44),
12109–12118. https://doi.org/10.1523/ JNEUROSCI.1873-07
.2007, 考研: 17978052
Tesileanu, T。, Olveczky, B., & Balasubramanian, V. (2017). 规则
and mechanisms for efficient two-stage learning in neural
circuits. 电子生活, 6, e20944. https://doi.org/10.7554/eLife.20944,
考研: 28374674
Thomas-Ollivier, 五、, Reymann, J. M。, Le Moal, S。, Schück, S。,
Lieury, A。, & Allain, H. (1999). Procedural memory in recent-
onset Parkinson’s disease. Dementia and Geriatric Cognitive
Disorders, 10(2), 172–180. https://doi.org/10.1159/000017100,
考研: 10026393
Thorndike, 乙. (2017). Animal intelligence: Experimental studies.
泰勒 & Francis. Retrieved from https:// books.google.com
/ b o o k s ? i d = 1 _ h A D w A A Q B A J . h t t p s : / / d o i . o r g / 1 0 . 4 3 2 4
/9781351321044
Tsutsui, K., Hosokawa, T。, Yamada, M。, & Iijima, 时间. (2016). Repre-
sentation of functional category in the monkey prefrontal cortex
and its rule-dependent use for behavioral selection. 杂志
神经科学, 36(10), 3038–3048. https://doi.org/10.1523
/JNEUROSCI.2063-15.2016, 考研: 26961957
Usrey, 瓦. M。, Alonso, J. M。, & 里德, 右. C. (2000). Synaptic interac-
tions between thalamic inputs to simple cells in cat visual cortex.
神经科学杂志, 20(14), 5461–5467. https://doi.org/10
.1523/JNEUROSCI.20-14-05461.2000, 考研: 10884329
Voytek, B., & 骑士, 右. 时间. (2010). Prefrontal cortex and basal gan-
glia contributions to visual working memory. 诉讼程序
美国国家科学院, 107(42), 18167–18172. https://
doi.org/10.1073/pnas.1007277107, 考研: 20921401
王, J. X。, Kurth-Nelson, Z。, Kumaran, D ., Tirumala, D ., Soyer,
H。, Leibo, J. Z。, … Botvinick, 中号. (2018). Prefrontal cortex as a
meta-reinforcement learning system. 自然神经科学,
21(6), 860–868. https://doi.org/10.1038/s41593-018-0147-8,
考研: 29760527
Warren, 时间. L。, Tumer, 乙. C。, Charlesworth, J. D ., & Brainard, 中号. S.
(2011). Mechanisms and time course of vocal learning and
consolidation in the adult songbird. 神经生理学杂志,
106(4), 1806–1821. https://doi.org/10.1152/jn.00311.2011,
考研: 21734110
Whittington, J. C. R。, & Bogacz, 右. (2019). Theories of error
back-propagation in the brain. 认知科学的趋势,
23(3), 235–250. https://doi.org/10.1016/j.tics.2018.12.005,
考研: 30704969
Wickens, J. R。, & Kotter, 右. (1994). Cellular models of reinforce-
蒙特. In Models of information processing in the basal ganglia.
与新闻界. https://doi.org/10.7551/mitpress/4708.003.0017
Wimmer, 右. D ., 施密特, L. 我。, 戴维森, 时间. J。, Nakajima, M。,
Deisseroth, K., & Halassa, 中号. 中号. (2015). Thalamic control of
sensory selection in divided attention. 自然, 526(7575),
705–709. https://doi.org/10.1038/nature15398, 考研:
26503050
Wolff, M。, & Vann, S. D. (2019). The cognitive thalamus as a gate-
way to mental representations. 神经科学杂志, 39(1),
3–14. https://doi.org/10.1523/ JNEUROSCI.0479-18.2018,
考研: 30389839
网络神经科学
996
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
t
/
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
t
.
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
Thalamocortical contribution to flexible learning in neural systems
Xiao, T。, 张, J。, 哪个, K., 彭, Y。, & 张, Z. (2014). Error-
driven incremental learning in deep convolutional neural net-
work for large-scale image classification. In ACM multimedia.
https://doi.org/10.1145/2647868.2654926
Yamins, D. L。, 洪, H。, Cadieu, C. F。, Solomon, 乙. A。, Seibert, D .,
& DiCarlo, J. J. (2014). Performance-optimized hierarchical
models predict neural responses in higher visual cortex. Proceed-
ings of the National Academy of Sciences, 111(23), 8619–8624.
https://doi.org/10.1073/pnas.1403112111, 考研: 24812127
哪个, G。, Pan, F。, & Gan, 瓦. 乙. (2009). Stably maintained dendritic
spines are associated with lifelong memories. 自然, 462(7275),
920–924. https://doi.org/10.1038/nature08577, 考研: 19946265
Zenke, F。, & 甘古利, S. (2018). SuperSpike: Supervised learning in
multilayer spiking neural networks. 神经计算, 30(6),
1514–1541. https://doi.org/10.1162/neco_a_01086, 考研:
29652587
Zenke, F。, Gerstner, W., & 甘古利, S. (2017). The temporal para-
dox of Hebbian learning and homeostatic plasticity. 当前的
Opinion in Neurobiology, 43, 166–176. https://doi.org/10.1016
/j.conb.2017.03.015, 考研: 28431369
Zenke, F。, Poole, B., & 甘古利, S. (2017). Continual learning
through synaptic intelligence. In Proceedings of the 34th interna-
tional conference on machine learning (卷. 70, PP. 3987–3995).
Retrieved from https://proceedings.mlr.press/v70/zenke17a.html
周, H。, Schafer, 右. J。, & 德西莫内, 右. (2016). Pulvinar-cortex
interactions in vision and attention. 神经元, 89(1), 209–220.
https://doi.org/10.1016/j.neuron.2015.11.034, 考研:
26748092
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
t
/
e
d
你
n
e
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
/
6
4
9
8
0
2
0
5
6
2
6
0
n
e
n
_
A
_
0
0
2
3
5
p
d
.
t
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
网络神经科学
997
