Interactive autonomous machines

尽管我们在设计和部署自主系统的能力方面取得了重大进展，但对该领域未来进展至关重要的关键问题仍然没有解决。我的研究项目集中在其中的三个关键问题上：多线索整合、规划开发和推理以及人与机器人的交互。多线索集成：自动机器必须能够从一系列不同的技术中捕获感觉数据，以建立它在哪里以及如何在其环境中移动的模型。这里的基本方法是理解人类如何构建类似的表示，并利用这一知识为机器解决问题。在过去的大约十年里，我一直在与一群国际合作者合作，以了解人类是如何发展出一种自我定位和自我运动的感觉的。这里的关键是在环境中进行受控实验，在这种环境中，向人类呈现不寻常的线索组合，以阐明潜在的线索整合过程。随着最近在长时间卧床休息方面的工作的完成，我帮助开发了一个深入而广泛的人类自我定向感知的数据集，我计划使用它来构建一个依赖时间的线索整合模型，该模型将使用水下机器人进行评估。计划的制定和执行：机器必须能够对其环境进行推理，以及它如何对该环境采取行动。传统上，长期机器人计划是使用基于逻辑结构分解任务的机制来制定的。虽然这些计划方法在当时是有效的，但与现代人工智能方法(例如，深度强化学习-DRL)相比，这些方法已经黯然失色，这些方法已经显示出非凡的能力，特别是对于可以进行适当训练的具有合理短期视野的任务。我们如何才能最好地将从经典规划制度中吸取的经验教训与DRL等机制的能力结合起来？利用一组特定的任务-侵入性水草监测和室内环境监测-我计划探索如何最好地使用DRL来学习可以排序的短期计划，以便为这些复杂但高度结构化的任务提供任务解决方案。人机交互(HRI)：机器必须能够与环境交互，最重要的是，它必须能够与占据其运行环境的人类交互。我的初步工作表明，特定于任务的手势语言对HRI是有效的，但我们能否更好地利用潜在的任务结构来建立有效的HRI系统？利用常见的商业潜水员任务，我计划在这里概括我的初步工作，以在HRI表示中限制语言标记和对话结构的优先顺序，并利用两者来理解人类与机器人的交流。

Although there have been significant strides in our ability to design and deploy autonomous systems, key problems remain unsolved that are critical to future advances in this field. My research program concentrates on three of these key problems: multi-cue integration, plan development and reasoning, and human-robot interaction. Multi-cue integration: An autonomous machine must be able to capture sensory data from a range of different technologies to build a model of where it is and how it can move within its environment. The underlying approach here is to understand how humans build similar representations and to leverage this knowledge to solve the problem for machines. Over the last ten years or so I have been working with a group of international collaborators to understand how humans develop a sense of self-orientation and self-motion. Key here is performing controlled experiments in environments within which humans are presented with unusual cue combinations to illuminate the underlying cue integration process. With the completion of recent work in long duration bed rest, I have helped develop a deep and wide dataset of human self-orientation perception which I plan to use to construct a time-dependent cue integration model that will be evaluated using underwater robots. Plan development and execution: The machine must be able to reason about its environment and how it can act upon that environment. Traditionally, long term robot plans were developed using mechanisms that decomposed tasks based on logical structures. Although effective in its day, these planning approaches have been eclipsed by modern AI approaches (e.g, Deep Reinforcement Learning - DRL) that have demonstrated extraordinary capabilities, especially for tasks with reasonably short-term horizons for which appropriate training can be performed. How can we best integrate the lessons learned from classic planning regimes with the capabilities of mechanisms like DRL? Utilizing a set of specific tasks - invasive water plant monitoring and indoor environment monitoring - I plan to explore how best to use DRL to learn short-term plans that can be sequenced to provide task solutions for these complex, but highly structured tasks. Human-robot interaction (HRI): The machine must be able to interact with the environment and most critically it must be able to interact with humans that occupy the environment within which it is operating. My preliminary work suggests task-specific gesture language can be effective for HRI, but can we better leverage the underlying task structure to build effective HRI systems? Utilizing common commercial diver tasks, I plan to generalize my preliminary work here to condition language token and conversational structure priors within the HRI representation, and to leverage both in understanding human to robot communication.