Recurrent neural network models for exploration in dynamic environments.

用于动态环境中探索的循环神经网络模型。

• 批准号: 496990750
• 负责人: Professor Dr. Jan Peters
• 金额: --
• 依托单位: Department Psychologie
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/496990750?language=en
• 关键词: Recurrent; neural; network; models; exploration

Recurrent neural networks models have received increasing interest in cognitive and systems neuroscience. These models have recently been successfully trained on tasks from the human and animal neuroscience literature, and might yield insights into potential computational mechanisms underlying higher cognitive functions. Here we will use recurrent neural network models to shed light on a fundamental problem in reinforcement learning – the exploration/exploitation trade-off. Agents are regularly faced with the problem of deciding whether to select well-known options for reward maximization, or whether to explore novel options for information gain. Human exploration is supported by at least two strategies – choice randomization (random exploration) and directed exploration of uncertain options for information gain. For the first objective, we will use computational modeling of recurrent neural network behavior to test the prediction that these network models exhibit computational strategies similar to those observed in humans to solve the exploration/exploitation trade-off. For the second objective, the computations and representations underlying network performance will be examined via a detailed analysis of the dynamics embedded in their hidden unit time courses. Together, the project will shed light on computational mechanisms that might support learning and decision-making in dynamic environments.

递归神经网络模型在认知和系统神经科学领域受到越来越多的关注。这些模型最近已经成功地在人类和动物神经科学文献中的任务上进行了训练，并可能对潜在的高级认知功能的计算机制产生见解。在这里，我们将使用递归神经网络模型来阐明强化学习中的一个基本问题——探索/利用权衡。智能体经常面临这样的问题：是选择已知的选项来实现奖励最大化，还是探索新的选项来获得信息。人类探索至少有两种策略支持——选择随机化（随机探索）和为获取信息而对不确定选项进行定向探索。对于第一个目标，我们将使用循环神经网络行为的计算建模来测试这些网络模型表现出类似于在人类中观察到的计算策略来解决探索/开发权衡的预测。对于第二个目标，将通过对其隐藏单位时间过程中嵌入的动态的详细分析来检查网络性能的计算和表示。总之，该项目将阐明在动态环境中支持学习和决策的计算机制。