NCS-FO: The biology and technology of online planning

计划对日常生活至关重要。规划是一项具有挑战性的研究，因为它涉及到对未来行动可能性的内部搜索，在没有任何迹象表明这正在从外部发生的情况下。该项目将研究实时规划的两个协同方面：其生物学和技术。目前的人工智能方法需要大量的力量，大量的训练，以及对数百万种可能的未来进行检查，以解决简单的问题，例如回合制棋盘游戏中的下一步棋。相比之下，哺乳动物只需要很少的力量，很少或根本不需要训练，也不需要对未来的可能性进行研究，就可以解决复杂的问题，比如去哪里躲避跟踪的捕食者。该项目将开发一种新的在线规划代理机器人“捕食者”，它将与经过训练的动物互动，以在复杂的栖息地内躲避它。机器人将与实验室动物互动，这些动物的大脑活动被记录下来，同时它们通过特殊设计的复杂栖息地受到挑战，以采用策略行为来避开机器人。这将测试和推进神经机制的理论，这些神经机制是以一种节能的方式在真实的时间内进行日常计划的能力的基础。计划行动的能力比被动的、反射性的或习惯性的行为能产生更大的回报。虽然人类在规划和执行日常活动方面表现出很高的熟练度，但对长期威胁的反应却很差。多步规划的研究还处于起步阶段，部分受到生态有效性低的行为任务的限制。由于人工智能的快速发展，理论已经进步，但大多数形式化需要如此多的计算能力，以至于实时规划是不可能的。动物似乎也会以其他方式形成实时计划。在之前的工作中，PI表明，视觉引导规划的选择性好处可能促进了3.8亿年前向陆地的过渡，因为动物可以在空气中看到比通过水更远的目标。在提供长视线的栖息地中，规划捕食者与猎物交战的好处最大化，同时也提供了可以隐藏对手的障碍。在这些条件下，如原始人最早出现的大草原般的栖息地，规划着它的巅峰优势。在该项目的目标1中，这个想法被建模以确定最大规划回报的位置（通过网络连接性度量），并用于预测动物的神经计算。这种初始算法在实现相同的模拟猎物存活率方面比机器学习领域的领先竞争对手快10，000倍。这使得能够创建行为测定，其中活动物受到具有与它们自己相似的规划能力的对手的挑战。随着这一原则转化为硬件，目标2将产生双向效益。首先，神经活动-使用Neuropixels探针在自由行为的小鼠中-将与该团队的理论预测进行实时比较;他们预测海马体中的边界检测细胞和内嗅皮层中的延迟间隔细胞对于修剪计划的神经计算负担很重要。第二，在记录过程中，动物将与一个真实的计划的机器人互动。该项目由理解神经和认知系统（NCS）的综合策略资助，这是一个由生物学（BIO），计算机和信息科学与工程（CISE），教育和人力资源（EHR），工程（ENG）和社会，行为，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Planning is centrally important for everyday life. Planning is challenging to study, as it involves an internal search through future possibilities for action, in the absence of any sign this is occurring from the outside. The project will investigate two synergistic aspects of real-time planning: its biology and technology. Current artificial intelligence methods require vast amounts of power, a large amount of training, and examination of millions of possible futures to tackle simple problems such as the next move in a turn-based board game. In contrast, mammals require very little power, little to no training, and examination of few possible futures to tackle complex problems such as where to go to hide from a stalking predator. This project will develop a new online planning agent—a robotic “predator”—that will interact with animals trained to evade it inside a complex habitat. The robot will interact with laboratory animals whose brain activity is being recorded while they are challenged—via specially-designed complex habitats—to employ strategic behaviors in avoiding the robot. This will test and advance theory of neural mechanisms underlying the everyday ability to plan in real time in an energy-efficient manner. The ability to plan actions can produce much larger rewards than reactive, reflexive, or habitual behaviors. Whereas humans exhibit great proficiency in planning and executing daily movements, poor response to long-term threats shows its limits. Research on multi-step planning is in its infancy, constrained in part by behavioral tasks with low ecological validity. Theory has advanced due to rapid progress in artificial intelligence, but most formalizations require so much computing power that real-time planning is impossible. Animals seem likely to form real-time plans in some other way. In prior work, the PIs showed that a selective benefit of visually guided planning may have facilitated the transition onto land 380 million years ago because animals can see targets much farther in air than through water. The benefit of planning in predator-prey engagements is maximized in habitats that afford long sightlines while also providing obstacles that can hide adversaries. In these conditions, such as savanna-like habitats where hominins first emerged, planning its peak advantage. In Aim 1 of the project, this idea is modeled to identify locations of maximal planning payoff (via a network connectivity measure) and used to predict neural computation in animals. This initial algorithm is 10,000 times faster in achieving the same survival rate of simulated prey than a leading competitor in machine learning. This enables creation of a behavioral assay in which live animals are challenged by an adversary with similar planning abilities to their own. With the principle translated into hardware, a bidirectional benefit will emerge for Aim 2. First, neural activity—using Neuropixels probes in freely behaving mice—will be compared to the team’s theory predictions in real-time; they predict that boundary detection cells in the hippocampus and delay interval cells in entorhinal cortex are important for trimming the neurocomputational burden of plans. Second, during recordings, animals will engage with a robot that plans in real time. This project is funded by Integrative Strategies for Understanding Neural and Cognitive Systems (NCS), a multidisciplinary program jointly supported by the Directorates for Biology (BIO), Computer and Information Science and Engineering (CISE), Education and Human Resources (EHR), Engineering (ENG), and Social, Behavioral, and Economic Sciences (SBE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.