Toward an Integration of Deep Learning and Neuroscience.
Toward an Integration of Deep Learning and Neuroscience.
复制标题
DOI：
10.3389/fncom.2016.00094
复制
发表时间：
2016
影响因子：
3.2
通讯作者：
Kording KP
中科院分区：
文献类型：
--
作者：
Marblestone AH;Wayne G;Kording KP
神经科学专注于计算的详细实现，研究神经代码，动力学和电路。然而，在机器学习中，人工神经网络倾向于避开精确设计的代码、动力学或电路，而倾向于对成本函数进行蛮力优化，通常使用简单且相对统一的初始架构。机器学习领域最近出现了两个发展，为连接这些看似不同的观点创造了机会。首先，使用结构化架构，包括用于注意力、递归和各种形式的短期和长期记忆存储的专用系统。第二，成本函数和培训程序变得更加复杂，而且在不同层次和不同时期都有所不同。这里我们从这些观点来思考大脑。我们假设（1）大脑优化成本函数，（2）成本函数是多样的，并且在大脑位置和发育过程中有所不同，（3）优化在与行为所带来的计算问题相匹配的预结构化架构中运行。为了支持这些假设，我们认为，通过多层神经元的信用分配的一系列实现与我们目前对神经电路的了解是兼容的，并且大脑的专门系统可以被解释为能够有效地优化特定的问题类。这种异质优化的系统，由一系列相互作用的成本函数实现，有助于使学习数据高效，并精确地针对生物体的需求。我们提出了神经科学可以寻求改进和测试这些假设的方向。
Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures. Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion and various forms of short- and long-term memory storage. Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas. We hypothesize that (1) the brain optimizes cost functions, (2) the cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior. In support of these hypotheses, we argue that a range of implementations of credit assignment through multiple layers of neurons are compatible with our current knowledge of neural circuitry, and that the brain's specialized systems can be interpreted as enabling efficient optimization for specific problem classes. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses.
登录
查看更多内容
影响因子：
3
作者：
Gonzalez Andino SL;Grave de Peralta Menendez R
通讯作者：
Grave de Peralta Menendez R
影响因子：
8.2
作者：
Allen, Kachina;Ibara, Steven;Botvinick, Matthew
通讯作者：
Botvinick, Matthew
DOI：
10.1073/pnas.84.17.6297
发表时间：
1987-09-01
影响因子：
11.1
作者：
ANDERSON, CH;VANESSEN, DC
通讯作者：
VANESSEN, DC
影响因子：
2.5
作者：
Acuna, Daniel E.;Wymbs, Nicholas F.;Kording, Konrad P.
通讯作者：
Kording, Konrad P.
影响因子：
3.7
作者：
Abbott, LF;Blum, KI
通讯作者：
Blum, KI