RI: Small: Generation and modulation of variability in trial and error learning

强化学习是一种强大的学习策略，它对输出进行随机扰动，并对导致更成功输出的变化进行强化。动物通过反复试验来学习执行高技能的任务，但生物神经网络如何在一次又一次的试验中产生可变性尚不清楚。在鸟类的歌唱学习中，可以找到一个理想的模型来检验变异性和学习之间的关系。鸟类通过反复试验来学习它们的歌声，并能够在成年后调整它们的歌声。此外，学习电路的基本神经结构已经绘制出来；行为背景对变异性的一些影响也是已知的。因此，变异性是如何产生的，以及它是如何被控制的，因此可以从机械的角度来研究变异性的产生和控制。本项目将研究变异性的产生和控制的两种候选机制：皮质样区的混沌动力学和基底节回路中同步的动态调制。虽然通用神经网络是混乱的，但这个项目将探索如何在生物物理上看似合理的神经元的皮质网络中修改这种动力学，该网络具有基于学习电路的已知拓扑结构的结构化连接。此外，鸟鸣皮质区域接受由基底节产生的输入。这种投入会如何影响产出的可变性？有人认为，基底节活动影响皮层的能力可能依赖于基底节输出的同步性。通过与两个实验室的合作，这些研究将受到来自这些候选领域的实验数据的限制。这项工作将有助于对生物学习中的可变性的总体理解，并可能提出在运动学习期间构建噪声输入的策略，从而为机器人应用带来有效的结果。此外，费尔霍尔博士正在为广泛的推广项目编写材料，其中包括一个大规模的在线公开课，向非常不同的受众介绍计算神经科学的概念，以及为对应用于生命科学的数学感兴趣的中学女孩举办研讨会。

Reinforcement learning is a powerful learning strategy in which random perturbations are made to an output and changes that lead to more successful outputs are reinforced. Animals learn to perform highly skilled tasks by trial and error, but how biological neural networks generate variability from trial to trial is not known. An ideal model for examining this relationship between variability and learning can be found in song learning by birds. Birds learn their songs by trial and error, and are able to adjust their songs into adulthood. In addition, the basic neural architecture of the learning circuit has been mapped out; some effects of behavioral context on variability are also known. How variability is generated, and how it is controlled, can thus be studied mechanistically.This project will study two candidate mechanisms for the generation and control of variability: chaotic dynamics in a cortex-like area, and dynamical modulation of synchrony in a basal ganglia circuit. While generic neural networks are chaotic, this project will explore how such dynamics are modified in a cortical network of biophysically plausible neurons with structured connectivity based on the known topography of the learning circuit. Furthermore, the birdsong cortical area receives inputs generated by basal ganglia. How might this input influence the variability of the output? It is proposed that the ability of activity in basal ganglia to affect cortex may depend on the synchrony of basal ganglia outputs. The studies will be constrained by experimental data from these candidate areas through collaborations with two laboratories. The work will contribute to a general understanding of variability in biological learning, and may suggest strategies for structuring noisy input during motor learning that can lead to efficient outcomes for robotics applications. Further, Dr Fairhall is developing materials for broad outreach programs that include a Massive Online Open Course to introduce concepts in computational neuroscience to a very diverse audience, and a workshop for middle school girls interested in mathematics applied to the life sciences.