Confidence-based learning: establishing a novel form of learning without feedback

基于信心的学习：建立一种无需反馈的新颖学习形式

• 批准号: 403630675
• 负责人: Dr. Marcus Rothkirch, since 2/2022
• 金额: --
• 依托单位: Klinik für Psychiatrie und Psychotherapie (CCM)
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2018
• 资助国家: 德国
• 起止时间: 2017-12-31 至 2022-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/403630675?language=en
• 关键词: Confidence; based; learning; establishing; novel

Theories of reinforcement learning explain fundamental characteristics of human learning and behaviour, but also how aberrancies in learning can lead to the manifestation of dysfunctional behavioural patterns and psychiatric diseases. However, a critical challenge for reinforcement learning accounts is that many forms of learning occur in the absence of external feedback and important aspects of psychiatric diseases are best characterized by self-reinforcing processes that do not depend on external feedback. Recently, theoretical progress and empirical findings in the field of perceptual learning have made the case for an intriguing extension of reinforcement learning, proposing that internal confidence in one’s own actions serves as a teaching signal when no external feedback is available. Moreover, it has been found that the computational principles and the neural underpinnings of such learning are highly similar to those of normative reinforcement learning based on external feedback. On this basis, the overarching goal of the present proposal is to 1) establish the generality of this novel form of learning at the behavioural and neurobiological level, and 2) to investigate interindividual differences in such learning in the paradigmatic case of proneness to anhedonia.In a first work package, we will investigate in two functional magnetic resonance imaging experiments whether the observed parallel between normative and confidence-based learning generalizes to the two most generic forms of reinforcement learning − classical and instrumental conditioning. The first experiment will examine whether the repeated pairing of a stimulus with induced levels of confidence in a distractor task leads to ‘confidence-based classical conditioning’, and thus to the behavioural, physiological and neurobiological effects known from normative classical conditioning. The second experiment will investigate confidence-based conditioning in an instrumental learning task, in which participants receive or do not receive external feedback. Using a novel confidence-based reinforcement learning model, we will test the hypothesis that confidence affects value representations by means of neurocomputational mechanisms known from normative instrumental conditioning.In a second work package we will investigate interindividual differences in confidence-based learning in a large general population sample recruited from the online marketplace Amazon Mechanical Turk. Adhering to a continuum view of psychiatric disease and using model-based mechanistic markers of confidence-based learning, we will test the hypothesis of a link between reduced confidence-based learning and proneness to anhedonia.Overall, the proposed project is expected to provide major insights into the fundamental behavioural and neurobiological nature of confidence-based learning and its significance for a key psychiatric symptom dimension, anhedonia.

强化学习理论解释了人类学习和行为的基本特征，但也解释了学习中的异常如何导致功能失调的行为模式和精神疾病的表现。然而，强化学习理论面临的一个关键挑战是，许多形式的学习是在没有外部反馈的情况下发生的，精神疾病的重要方面最好的特征是不依赖于外部反馈的自我强化过程。最近，感知学习领域的理论进展和实证发现为强化学习提供了一个有趣的扩展，提出当没有外部反馈可用时，对自己行为的内部信心可以作为教学信号。此外，研究发现，这种学习的计算原理和神经基础与基于外部反馈的规范强化学习高度相似。在此基础上，本研究的首要目标是：1)在行为和神经生物学水平上建立这种新型学习形式的普遍性；2)在快感缺乏倾向的典型案例中研究这种学习的个体间差异。在第一个工作包中，我们将在两个功能性磁共振成像实验中研究观察到的规范学习和基于信心的学习之间的相似性是否可以推广到两种最常见的强化学习形式——经典条件反射和工具条件反射。第一个实验将检验在分心任务中，刺激与诱导的信心水平的重复配对是否会导致“基于信心的经典条件反射”，从而导致从规范经典条件反射中已知的行为、生理和神经生物学效应。第二个实验将调查基于信心的条件作用在工具性学习任务中，参与者接受或不接受外部反馈。使用一种新的基于信心的强化学习模型，我们将通过规范工具条件反射中已知的神经计算机制来检验信心影响价值表征的假设。在第二个工作包中，我们将调查从在线市场亚马逊土耳其机械招募的大型一般人群样本中基于信心的学习的个体间差异。坚持精神疾病的连续性观点，并使用基于模型的基于信心的学习机制标记，我们将检验基于信心的学习减少与快感缺乏症倾向之间联系的假设。总的来说，拟议的项目有望为基于自信的学习的基本行为和神经生物学性质及其对关键精神症状维度快感缺乏症的重要性提供重要见解。