Exploring the limits of behavioral complexity in rats: a novel experimental approach via reinforcement learning and information theory

神经学家最终感兴趣的是使动物和人类能够在野外进行复杂行为的神经过程。然而，实验室中的动物通常是在行为自由受到严格限制的实验环境中进行研究的。例如，在许多老鼠实验中，老鼠要么可以推或不推杠杆，要么可以在两个可能的杠杆之间做出选择--除此之外什么都不能做。动物实验中严重限制行为的一个主要原因是缺乏设计、建模和分析不受限制行为的实验的方法。本项目的目标是采用强化学习理论中的方法来建模和分析不受限制的大鼠的行为，并将这些新方法应用到一类新的真实大鼠的行为实验中。在尝试真实地模拟不受限制的行为时，应该考虑到动物的神经资源有限。例如，一只大鼠可能无法准确地记住通往遥远食物来源的最短可能路径(在无限神经资源的情况下，这是最佳行为)，但可能能够在更长的路径上到达食物来源(在有限神经资源情况下，最佳行为)。我将把强化学习的数学理论与信息论中的信息约束概念结合起来，在规定的资源限制下对动物的行为进行建模。这里介绍的新的行为范式鼓励大鼠随着时间的推移增加其行为的复杂性：当大鼠表现良好时，这些适应性任务的难度自动增加，当大鼠表现不佳时，难度自动降低。我提出的建模方法将允许在具有(几乎)不受限制的行为自由的任务中，用指定数量的神经资源来预测大鼠的行为。用真实的大鼠运行建议的自适应任务将使我能够调查大鼠可以处理的复杂极限及其首选的复杂程度。最后，我将记录行为过程中的听觉神经元，以寻找行为复杂性和任务难度的神经关联。

Neuroscientists are ultimately interested in the neural processes that enable animals and humans to carry out complex behaviors in the wild. However, animals in laboratories are usually studied in experimental environments with heavily restricted behavioral freedom. In many rat experiments, for instance, rats can either push a lever or not, or choose between two possible levers – and nothing else. A main reason for heavily restricting behavior in animal experiments has been a lack of methods for the design, modeling, and analysis of experiments with unrestricted behavior.The goal of this project is to adapt methods from the theory of reinforcement learning to model and analyze the behavior of unrestricted rats and to apply these new methods to a novel class of behavioral experiments in real rats.An attempt to realistically model unrestricted behavior should take into account that animals have limited neural resources. For example, a rat might be unable to precisely remember the shortest possible path to a remote food source (the optimal behavior given unlimited neural resources), but might be able to reach the food source on some longer path (optimal behavior given limited neural resources). I will combine the mathematical theory of reinforcement learning with the notion of information constraints from information theory to model the behavior of animals with prescribed limits on resources. The novel behavioral paradigms introduced here encourage rats to increase the complexity of their behavior over time: the difficulty of these adaptive tasks automatically increases whenever the rat performs well, and decreases whenever the rat performs poorly.The modeling methods I propose will allow to predict rat behavior with prescribed amounts of neural resources in tasks with (almost) unrestricted behavioral freedom. Running the proposed adaptive tasks with real rats will enable me to investigate the limits of complexity that rats can handle and their preferred complexity levels. Lastly, I will record from auditory neurons during behavior in order to search for neural correlates of behavioral complexity and task difficulty.