Sensing and perception for autonomous agents

移动性是智能的一个关键方面。移动性使代理能够探索其环境，与其环境交互，并提供了一个现实的定义，在这里与那里。但移动性也引入了与感知和感知相关的复杂问题。例如，移动性引入了一些基本问题，例如：代理在环境中的位置？它的姿态是什么？以及与姿势变化相关的问题：是否有特工移动？如果有，距离有多远，方向是什么？ 对于代理来说，回答这些问题之所以困难，是因为它很少能够访问一些可以直接回答这些问题的外部过程。相反，系统（通常）必须依赖于与代理本身相关联的传感器和感测算法，并且必须将该信息与意图沿着集成，以便获得对这些问题的答案的最佳估计。此外，对这些问题的错误答案可能会对代理本身产生灾难性的后果。例如，潜水员、宇航员或飞行员的定向障碍是一个常见的问题，会导致危及生命的后果和使命失败。这些基本问题在一系列学科中都有发现，并且在不同的研究文化中以及使用不同的传感器和传感器集合时具有不同的名称。例如，估计姿态变化的任务存在于自运动估计、运动矢量、运动结构和视觉里程计中。同样，自定位和自运动估计的鲁棒解决方案为各种应用提供了信息，从人类在具有挑战性的环境中的表现，到虚拟现实，再到自动驾驶汽车的设计和开发。该建议的主要研究目标是自主代理的运动和姿态估计的鲁棒性和有效的算法的发展。虽然生物和机器系统可以使用各种潜在的传感器和感测模式来解决这些问题，但这里特别感兴趣的是理解视觉，特别是双目视觉如何与惯性感测耦合以估计姿态和运动。 在我的研究计划中要遵循的基本方法是基于一种在智能机器的其他方面被证明特别有效的策略。生物系统如何解决类似任务的模型将用于激励机器的解决方案。然后，这些解决方案将在实验室和复杂的非结构化环境中进行评估，以测试最终系统的性能，并探索基础生物模型的功效。

Mobility is a key aspect of intelligence. Mobility enables an agent to explore its environment, to interact with its environment, and provides a realistic definition of here versus there. But mobility also introduces complex problems associated with sensing and perception. For example, mobility introduces fundamental questions such as; where is an agent in the environment? what is its pose? and questions related to change in pose: has an agent moved? and if so, how far and in what direction? What makes answering these problems difficult for an agent is that it rarely has access to some external process that can answer these questions directly. Rather the system (typically) must rely on sensors and sensing algorithms associated with the agent itself, and must integrate this information along with intent in order to obtain a best estimate for answers to these problems. Furthermore, getting answers to these questions wrong can have catastrophic consequences for the agent itself. For example, disorientation in divers, astronauts or pilots is a common problem leading to life-threatening consequences and mission failure. These fundamental problems are found across a range of disciplines, and not surprisingly have different names in different research cultures and when different sensors and sensor collections are used. For example, the task of estimating pose changes is found in egomotion estimation, motion from vection, structure from motion, and visual odometry. Similarly, robust solutions to the estimation of self-orientation and self-motion informs a wide variety of applications, from human performance in challenging environments, to virtual reality, to the design and development of autonomous vehicles. The primary research objective of this proposal is the development of robust and effective algorithms for motion and pose estimation for autonomous agents. Although biological and machine systems have access to a wide range of potential sensors and sensing modalities to address these questions, of particular interest here is in understanding how vision, and in particular binocular vision can be coupled with inertial sensing to estimate pose and motion. The basic approach to be followed in my research program is based upon a strategy that has proven particularly effective in other aspects of intelligent machines. Models of how biological systems solve similar tasks will be used to motivate solutions for machines. These solutions will then be evaluated in the lab and then in complex unstructured environments to test both the performance of the resulting system as well as to explore the efficacy of the underlying biological model.