KDI: Sequential Decision Making in Animals and Machines

9873531亨德森在现实环境中，尽管对世界的状态和行动的影响知之甚少，可移动的生物仍能以惊人的速度和灵活性做出准确的行为决定。如果要在类似的环境中灵活操作，像移动机器人这样的人工代理必须具备这种能力。该研究的主要目标是对动物和机器人的顺序决策进行详细的跨学科研究，重点是实时学习和控制信息收集和导航行为。该项目将采用比较的方法，结合来自人眼运动控制研究的心理物理和认知研究技术，来自昆虫导航研究的行为研究技术，以及来自移动机器人研究的计算方法。所有这些系统都为研究部分可观察环境下的实时决策提供了实验可处理的测试平台。该研究以一类称为马尔可夫决策过程（MDP）的顺序决策模型为指导。这些模型很有吸引力，因为它们为在不确定环境中计算最优行为提供了一个正式的框架。然而，这些模型并没有完全捕捉到生物决策的复杂性。我们将利用从生物行为研究和人工代理算法中获得的见解来探索MDP框架的扩展。这种协同作用将导致对生物有机体中顺序决策的更好的理论理解，并为人工代理开发有效的算法。该项目的一个主要成果将是展示人工生物（机器人）的设计如何能够被动物的顺序行为研究所指导，并作为指导。了解机器人设计师面临的挑战，以及他们为应对这些挑战而开发的正式框架，会导致有关生物行为的新问题。同样，从生物体中获得的见解将有助于提出改进构建智能人工代理的算法的方法

9873531HendersonMobile organisms make accurate behavioral decisions with extraordinary speed and flexibility in real-world environments despite incomplete knowledge about the state of the world and the effects of their actions. This ability must be shared by artificial agents such as mobile robots if they are to operate flexibly in similar environments. The main goal of the research is to undertake a detailed interdisciplinary study of sequential decision making across animals and robots, with a focus on real time learning and control of information gathering and navigational behaviors.The project will take a comparative approach, combining psychophysical and cognitive research techniques from the study of human eye movement control, behavioral research techniques from the study of insect navigation, and computational methods from the study of mobile robots. All of these systems provide experimentally tractable test-beds for studying real-time decision making in partially observable environments.The research is guided by a class of sequential decision making models called Markov decision processes (MDP). These models are attractive because they provide a formal framework for computing optimal behavior in uncertain environments. However, these models do not fully capture the complexity of decision making in organisms. We will explore extensions of the MDP framework using insights gained from the study of behavior in organisms and algorithms in artificial agents. This synergy will lead both to a better theoretical understanding of sequential decision making in biological organisms, and to the development of efficient algorithms for artificial agents.A major outcome of the project will be to show how the design of artificial creatures (robots) can be guided by, and serve as a guide for, the study of sequential behavior in animals. Understanding the challenges that robot designers face, and the formal framework that they have developed to tackle these challenges, leads to novel questions about organisms behavior. Similarly, insights gained from organisms will help suggest ways for improving algorithms for building intelligent artificial agents.***