Recurrent neural network models for exploration in dynamic environments.

递归神经网络模型在认知和系统神经科学领域受到越来越多的关注。这些模型最近在人类和动物神经科学文献中的任务上得到了成功的训练，可能会对潜在的高级认知功能的计算机制产生洞察力。在这里，我们将使用递归神经网络模型来阐明强化学习中的一个基本问题-探索/开发之间的权衡。代理人经常面临这样的问题：是选择众所周知的选择来实现报酬最大化，还是探索新的选择来获得信息。人类的探索至少得到两种战略的支持--选择随机化(随机探索)和定向探索不确定的选择以获得信息。对于第一个目标，我们将使用递归神经网络行为的计算建模来测试预测，即这些网络模型展示了与人类观察到的类似的计算策略，以解决勘探/开采之间的权衡。对于第二个目标，将通过详细分析隐藏的单位时间进程中嵌入的动态来检查潜在网络性能的计算和表示。总而言之，该项目将阐明可能支持动态环境中的学习和决策的计算机制。

Recurrent neural networks models have received increasing interest in cognitive and systems neuroscience. These models have recently been successfully trained on tasks from the human and animal neuroscience literature, and might yield insights into potential computational mechanisms underlying higher cognitive functions. Here we will use recurrent neural network models to shed light on a fundamental problem in reinforcement learning – the exploration/exploitation trade-off. Agents are regularly faced with the problem of deciding whether to select well-known options for reward maximization, or whether to explore novel options for information gain. Human exploration is supported by at least two strategies – choice randomization (random exploration) and directed exploration of uncertain options for information gain. For the first objective, we will use computational modeling of recurrent neural network behavior to test the prediction that these network models exhibit computational strategies similar to those observed in humans to solve the exploration/exploitation trade-off. For the second objective, the computations and representations underlying network performance will be examined via a detailed analysis of the dynamics embedded in their hidden unit time courses. Together, the project will shed light on computational mechanisms that might support learning and decision-making in dynamic environments.