Network Dynamics and Computational Mechanisms of Rule Learning II

动物学习通常被认为是一个渐进的过程，经过多次试验，涉及刺激、反应和结果之间的关联的逐步加强，这一观点深深植根于行为主义理论，并固有于大多数神经网络学习模型。然而，近年来有越来越多的证据表明，动物学习，即使在表面上简单的条件反射任务中，也被更好地理解为一种积极的推理和决策过程。这一观点来自对个体、一次又一次学习过程的仔细统计检查，以及对伴随学习过程的前额叶皮质(PFC)不同行为规则编码的神经集成状态之间的突然转换的观察。为了解决关于这些神经系综转换的计算意义以及多巴胺能系统在其中所起作用的相互矛盾的假设，我们在一个新设计的概率规则转换任务中从大鼠的PFC开始了多个单单位记录，以及在这个任务中苯丙胺或局部光遗传刺激对多巴胺释放的影响。初步结果表明，突然的神经转换可能反映了选择标准的变化，而不是其他行为过程(如不确定性)，而且多巴胺也可能主要影响选择过程，而不是价值更新。在这些结果和其他观察的基础上，我们的目标是进一步验证和扩展我们目前的理解，沿着三个主要方向，使用多个四极管记录，光遗传操作，以及基于学习过程的高级时间序列和计算模型的分析：1)我们将更详细地剖析在哪些任务期间多巴胺输入是最关键的，多巴胺神经元活动如何随着任务的进展与PFC活动相协调，以及它如何影响规则学习的各个子组件，如动作选择和价值更新；2)我们将更详细地讨论关于大鼠PFC不同分区中主动推理过程潜在的神经动力学机制的具体假设；3)通过基本行为任务设计的变化，我们将探索当前观察支持的更高层次的概念，如作为结构推理的学习和主动信息寻找。因此，我们将继续朝着在神经生理、神经动力学和神经计算水平上全面理解规则学习作为主动推理的目标。

Animal learning is often conceived as a gradual process that develops over many trials and involves the incremental strengthening of associations among stimuli, responses, and outcomes, a view deeply rooted in behaviorist theory and inherent in most neural network learning models. However, in recent years there is mounting evidence that animal learning, even in apparently simple conditioning tasks, is better understood as an active inference and decision making process. This view comes from careful statistical examination of the individual, trial-by-trial learning progress, and from the observation of sudden transitions among neural ensemble states coding for different behavioral rules in prefrontal cortex (PFC) which accompany the learning process. In order to address conflicting hypotheses regarding the computational meaning of these neural ensemble transitions, and of the role of the dopaminergic system within them, we had started multiple single-unit recordings from the rat PFC during a newly designed probabilistic rule-shift task, and the effects of amphetamine or local optogenetic stimulation of dopamine release during this task. Preliminary results suggest that sudden neural transitions might reflect a change in choice criterion rather than other behavioral processes (like uncertainty), and that dopamine may also primarily affect the choice process rather than value updating. Building on these results and other observations, here we aim to further validate and extend our current understanding along three major directions, using a combination of multi-tetrode recordings, optogenetic manipulations, and advanced time series and computational model based analysis of the learning process:1) We will dissect in more detail the task periods during which dopamine input is most crucial, how dopamine neuron activity is coordinated with PFC activity as the task progresses, and how it impacts on various subcomponents of rule learning like action selection and value updating;2) we will address in more detail specific hypotheses regarding the neuro-dynamical mechanisms underlying the active inference process in various subdivisions of the rat PFC;3) through variations of the basic behavioral task design we will explore higher-level concepts supported by current observations like learning as structural inference and active information seeking.Thus, we will continue toward our goal of a comprehensive understanding of rule learning as active inference at the neurophysiological, neuro-dynamical, and neuro-computational levels.