Exploring the limits of behavioral complexity in rats: a novel experimental approach via reinforcement learning and information theory

神经科学家最终感兴趣的是使动物和人类能够在野外进行复杂行为的神经过程。然而，实验室中的动物通常在行为自由受到严重限制的实验环境中进行研究。例如，在许多老鼠实验中，老鼠要么推动杠杆，要么不推动杠杆，要么在两个可能的杠杆之间做出选择--除此之外别无选择。在动物实验中严重限制行为的一个主要原因是缺乏设计、建模，本项目的目标是采用强化学习理论的方法来建模和分析无限制大鼠的行为，并将这些新方法应用于一类新的真实的大鼠行为实验。行为应该考虑到动物的神经资源有限。例如，一只老鼠可能无法精确记住到达遥远食物源的最短路径（在神经资源无限的情况下的最佳行为），但可能能够通过一些更长的路径到达食物源（在神经资源有限的情况下的最佳行为）。我将联合收割机的强化学习的数学理论与信息理论的信息约束的概念相结合，以模拟动物的行为与规定的资源限制。这里介绍的新的行为范例鼓励大鼠增加他们的行为随着时间的推移的复杂性：这些适应性任务的难度自动增加时，大鼠表现良好，并降低时，大鼠表现不佳，我提出的建模方法将允许预测大鼠的行为与规定数量的神经资源的任务（几乎）不受限制的行为自由。用真实的老鼠来执行所提出的适应性任务，将使我能够研究老鼠能够处理的复杂性的极限以及它们偏好的复杂性水平。最后，我将记录行为过程中的听觉神经元，以寻找行为复杂性和任务难度的神经相关性。

Neuroscientists are ultimately interested in the neural processes that enable animals and humans to carry out complex behaviors in the wild. However, animals in laboratories are usually studied in experimental environments with heavily restricted behavioral freedom. In many rat experiments, for instance, rats can either push a lever or not, or choose between two possible levers – and nothing else. A main reason for heavily restricting behavior in animal experiments has been a lack of methods for the design, modeling, and analysis of experiments with unrestricted behavior.The goal of this project is to adapt methods from the theory of reinforcement learning to model and analyze the behavior of unrestricted rats and to apply these new methods to a novel class of behavioral experiments in real rats.An attempt to realistically model unrestricted behavior should take into account that animals have limited neural resources. For example, a rat might be unable to precisely remember the shortest possible path to a remote food source (the optimal behavior given unlimited neural resources), but might be able to reach the food source on some longer path (optimal behavior given limited neural resources). I will combine the mathematical theory of reinforcement learning with the notion of information constraints from information theory to model the behavior of animals with prescribed limits on resources. The novel behavioral paradigms introduced here encourage rats to increase the complexity of their behavior over time: the difficulty of these adaptive tasks automatically increases whenever the rat performs well, and decreases whenever the rat performs poorly.The modeling methods I propose will allow to predict rat behavior with prescribed amounts of neural resources in tasks with (almost) unrestricted behavioral freedom. Running the proposed adaptive tasks with real rats will enable me to investigate the limits of complexity that rats can handle and their preferred complexity levels. Lastly, I will record from auditory neurons during behavior in order to search for neural correlates of behavioral complexity and task difficulty.