Learning the payoffs and costs of actions
Learning the payoffs and costs of actions
复制标题
DOI：
10.1371/journal.pcbi.1006285
复制
发表时间：
2019-02-01
影响因子：
4.3
通讯作者：
Bogacz, Rafal
中科院分区：
文献类型：
--
作者：
Moller, Moritz;Bogacz, Rafal
一组被称为基底神经节的皮层下核对学习动作的价值至关重要。基底神经节包括两条通路，分别与接近行为和避免行为有关，并受中脑多巴胺投射的差异调节。受有影响力的对手行动者学习模型的启发，我们证明，在某些情况下，这些路径可能代表对个人行为的积极和消极后果（回报和成本）的学习估计。在该模型中，多巴胺的活动水平编码了动机状态，并控制了回报和成本在多大程度上进入了对行为的总体评估。我们证明了一组先前提出的可塑性规则适用于从预测误差信号中提取收益和成本，如果它们发生在不同的时间点。对于这些可塑性规则，成功的学习需要两条路径上的积极和消极结果预测误差的不同影响，以及在试验过程中突触权重的微弱衰减。我们还通过模拟证实，该模型再现了药物诱导的工作意愿变化，正如在d2拮抗剂氟哌啶醇的经典实验中观察到的那样。
A set of sub-cortical nuclei called basal ganglia is critical for learning the values of actions. The basal ganglia include two pathways, which have been associated with approach and avoid behavior respectively and are differentially modulated by dopamine projections from the midbrain. Inspired by the influential opponent actor learning model, we demonstrate that, under certain circumstances, these pathways may represent learned estimates of the positive and negative consequences (payoffs and costs) of individual actions. In the model, the level of dopamine activity encodes the motivational state and controls to what extent payoffs and costs enter the overall evaluation of actions. We show that a set of previously proposed plasticity rules is suitable to extract payoffs and costs from a prediction error signal if they occur at different moments in time. For those plasticity rules, successful learning requires differential effects of positive and negative outcome prediction errors on the two pathways and a weak decay of synaptic weights over trials. We also confirm through simulations that the model reproduces drug-induced changes of willingness to work, as observed in classical experiments with the D2-antagonist haloperidol.