A one-shot learning signal in monkey prefrontal cortex-猫眼课题宝

猴子前额皮质的一次性学习信号

DOI：

10.1101/2020.11.27.401422

发表时间：

2020

期刊：

影响因子：

通讯作者：

中科院分区：

文献类型：

作者：

Achterberg J

许多动物的学习是缓慢的，行为的累积变化是由奖励预测错误驱动的。然而，当一个问题的抽象结构已知时，动物和正式的学习模型都可以快速地将新的项目附加到它们在这个结构中的角色上，有时只需要一次试验。额叶皮层可能在这一过程中发挥关键作用。为了在已知的问题结构中检查信息寻求和使用，我们在一个新的探索/利用任务中训练猴子，要求动物首先测试对象与奖励的关联，然后，一旦发现奖励对象，在进一步的试验中重新选择它们以获得进一步的奖励。额叶皮层中的许多细胞表现出探索/利用偏好，在信号试验中改变活动，以与猴子行为中的一次性学习保持一致。与这种二进制开关相反，这些细胞几乎没有表现出与预期或预测错误相关的连续变化的证据。探索/利用偏好是独立的两个阶段的审判，对象选择和接收反馈。在既定的任务结构中，额叶活动可能控制探索和利用的单独操作，在一次试验中在两者之间切换。显著性声明许多动物的学习是缓慢的，行为的累积变化由奖励预测错误驱动。然而，当问题的抽象结构已知时，动物和正式的学习模型都可以快速地将新的项目附加到它们在这个结构中的角色上。为了解决一次性学习过程中神经活动的转变，我们使用熟悉的物体和高度熟悉的任务结构训练猴子进行探索/利用任务。与反映预期或预测错误的连续变化相反，额叶神经元在探索和利用之间表现出二元的一次性切换。在一个既定的任务结构中，额叶活动可以控制探索替代对象以确定其当前角色的单独操作，然后利用这些知识获得进一步的奖励。

Much animal learning is slow, with cumulative changes in behavior driven by reward prediction errors. When the abstract structure of a problem is known, however, both animals and formal learning models can rapidly attach new items to their roles within this structure, sometimes in a single trial. Frontal cortex is likely to play a key role in this process. To examine information seeking and use in a known problem structure, we trained monkeys in a novel explore/exploit task, requiring the animal first to test objects for their association with reward, then, once rewarded objects were found, to re-select them on further trials for further rewards. Many cells in the frontal cortex showed an explore/exploit preference, changing activity in a signal trial to align with one-shot learning in the monkeys’ behaviour. In contrast to this binary switch, these cells showed little evidence of continuous changes linked to expectancy or prediction error. Explore/exploit preferences were independent for two stages of the trial, object selection and receipt of feedback. Within an established task structure, frontal activity may control the separate operations of explore and exploit, switching in one trial between the two.Significance statementMuch animal learning is slow, with cumulative changes in behavior driven by reward prediction errors. When the abstract structure a problem is known, however, both animals and formal learning models can rapidly attach new items to their roles within this structure. To address transitions in neural activity during one-shot learning, we trained monkeys in an explore/exploit task using familiar objects and a highly familiar task structure. In contrast to continuous changes reflecting expectancy or prediction error, frontal neurons showed a binary, one-shot switch between explore and exploit. Within an established task structure, frontal activity may control the separate operations of exploring alternative objects to establish their current role, then exploiting this knowledge for further reward.