Interactive autonomous machines

尽管我们在设计和部署自主系统的能力方面取得了重大进展，但对该领域未来发展至关重要的关键问题仍未得到解决。我的研究项目集中在三个关键问题上：多线索集成、计划开发和推理以及人机交互。多线索集成：自动机器必须能够从一系列不同的技术中捕获感官数据，以建立一个模型，说明它在哪里，以及它如何在环境中移动。这里的基本方法是了解人类如何构建类似的表示，并利用这些知识为机器解决问题。在过去十年左右的时间里，我一直在与一群国际合作者合作，以了解人类是如何发展出自我定位和自我运动的感觉的。这里的关键是在环境中进行控制实验，在这种环境中，人类被呈现出不寻常的线索组合，以阐明潜在的线索整合过程。随着最近长时间卧床休息工作的完成，我已经帮助开发了一个深入而广泛的人类自我取向感知数据集，我计划用它来构建一个时间依赖的线索整合模型，该模型将使用水下机器人进行评估。计划开发和执行：机器必须能够推断其环境以及如何在该环境中行动。传统上，机器人的长期计划是使用基于逻辑结构分解任务的机制来制定的。尽管在当时是有效的，但这些规划方法已经被现代人工智能方法（例如深度强化学习- DRL）所掩盖，这些方法已经证明了非凡的能力，特别是对于具有合理短期视野的任务，可以进行适当的训练。我们如何才能最好地将从传统规划制度中吸取的经验教训与DRL等机制的能力结合起来？利用一组特定的任务——侵入性水厂监测和室内环境监测——我计划探索如何最好地使用DRL来学习短期计划，这些计划可以被排序，为这些复杂但高度结构化的任务提供任务解决方案。人机交互（HRI）：机器必须能够与环境交互，最重要的是，它必须能够与占据其操作环境的人类交互。我的初步工作表明，特定任务的手势语言对HRI是有效的，但我们能否更好地利用潜在的任务结构来构建有效的HRI系统？利用常见的商业潜水员任务，我计划将我在这里的初步工作推广到HRI表示中的条件语言令牌和会话结构，并利用两者来理解人类与机器人的通信。

Although there have been significant strides in our ability to design and deploy autonomous systems, key problems remain unsolved that are critical to future advances in this field. My research program concentrates on three of these key problems: multi-cue integration, plan development and reasoning, and human-robot interaction. Multi-cue integration: An autonomous machine must be able to capture sensory data from a range of different technologies to build a model of where it is and how it can move within its environment. The underlying approach here is to understand how humans build similar representations and to leverage this knowledge to solve the problem for machines. Over the last ten years or so I have been working with a group of international collaborators to understand how humans develop a sense of self-orientation and self-motion. Key here is performing controlled experiments in environments within which humans are presented with unusual cue combinations to illuminate the underlying cue integration process. With the completion of recent work in long duration bed rest, I have helped develop a deep and wide dataset of human self-orientation perception which I plan to use to construct a time-dependent cue integration model that will be evaluated using underwater robots. Plan development and execution: The machine must be able to reason about its environment and how it can act upon that environment. Traditionally, long term robot plans were developed using mechanisms that decomposed tasks based on logical structures. Although effective in its day, these planning approaches have been eclipsed by modern AI approaches (e.g, Deep Reinforcement Learning - DRL) that have demonstrated extraordinary capabilities, especially for tasks with reasonably short-term horizons for which appropriate training can be performed. How can we best integrate the lessons learned from classic planning regimes with the capabilities of mechanisms like DRL? Utilizing a set of specific tasks - invasive water plant monitoring and indoor environment monitoring - I plan to explore how best to use DRL to learn short-term plans that can be sequenced to provide task solutions for these complex, but highly structured tasks. Human-robot interaction (HRI): The machine must be able to interact with the environment and most critically it must be able to interact with humans that occupy the environment within which it is operating. My preliminary work suggests task-specific gesture language can be effective for HRI, but can we better leverage the underlying task structure to build effective HRI systems? Utilizing common commercial diver tasks, I plan to generalize my preliminary work here to condition language token and conversational structure priors within the HRI representation, and to leverage both in understanding human to robot communication.