Dynamic computation of hierarchical prediction errors during sequence learning

序列学习的神经影像学研究在很大程度上独立于计算强化学习的研究，因为重点主要是学习刺激序列的统计依赖性，而不是通过学习优化性能。在这里，我们想研究无模型强化学习的机制是否以及在多大程度上也可以用于顺序学习。使用一个实验范例，我们精确地控制时间依赖性的第一和第二阶转移概率，我们提出了一个组合的fMRI计算建模研究，以测试以下假设：（1）人类大脑实现常见的计算重复学习机制，以估计高阶序列依赖性，以类似的方式，因为它估计平均奖励。(2)可区分的高阶和低阶预测误差被编码在不同的时间顺序和可分离的大脑区域。(3)编码不同顺序条件概率的大脑区域之间的连通性受到平均奖励的调制。

Neuroimaging research on sequence learning has proceeded largely independently of the research on computational reinforcement learning, because the focus is primarily on learning statistical dependencies of stimulus sequences rather than on optimizing performance through learning. Here we want to investigate whether and to what extend mechanisms of model-free reinforcement learning are also be employed in sequential learning. Using an experimental paradigm where we precisely control temporal dependencies by first and second-order transition probabilities, we propose a combined fMRI-computational modeling study in order to test the following hypotheses: (1) The human brain implements common computational reinforcement-learning mechanism for estimating higher-order sequential dependencies in a similar way as it estimates average rewards. (2) Distinguishable higher- and lower-order prediction errors are encoded in different temporal orders and in dissociable brain regions. (3) The connectivity among brain regions that encode different orders of conditional probabilities is modulated by average rewards.