New Directions in Robotic Environmental Monitoring via Machine Learning

是什么阻止了加拿大景观的机器人环境监测实现自动化？答案是双重的。首先，在硬件健壮性方面存在挑战，例如在恶劣环境条件下运行。其次，也是最关键的一点是，机器人对环境科学家真正需要的数据类型视而不见。他们不知道该把注意力放在哪里。我的首要目标是在几乎不受人类监督和与人类互动的情况下解决后一组挑战。这项研究将开发感知、推理和控制算法，以解决在知情的视觉探索中具有挑战性的机器人问题。主要的应用领域是在我过去在这个领域的工作的基础上，通过在户外操作的机器人来实现环境监测的自动化。这项技术的主要用户是环境科学家。开发的核心算法将直接适用于大量的人-机器人交互场景，并将跨越三个关键方向：(1)基于用户规范的机器人视觉注意力的数据高效学习：我们将开发有效学习图像中哪些视觉内容对用户重要的方法。视觉注意的预测模型将从少量的标记图像和大量的未标记图像中学习。它们将使机器人能够根据用户的喜好进行知情的视觉探索。我们还将发展与奖励学习和图像上的反向强化学习的联系。这是人-机器人交互的一个关键因素，尽管在低维环境中取得了多年的进展，但在高维环境中仍然缺乏强大的实用方法。(D2)基于视觉注意不确定性的机器人在线视觉探索策略：我们将估计视觉注意模型的模型(认知)不确定性，而不仅仅是学习单一的概率模型。这将使机器人能够在线收集信息。它还将允许仅主动选择用户需要在(D1)中标记的少数图像。然而，最重要的是，它将为开发具有不确定性的3D视觉探索策略铺平道路。(D3)用于环境监测的协作人类多机器人视觉探索：我们将设计方法，使一组机器人和人类能够协同对未知环境进行视觉探索。这将允许不同能力的机器人通过对人类的位置、意图和偏好做出明智的短期和长期预测，进行协作探索，而不必不断地看到人类在哪里。在自己探索之后，机器人将需要重新定位他们的人类合作者，以便将他/她的注意力重新引导到任何有趣的观察上。这项研究在人类参与的情况下共同应对这些挑战方面是独一无二的。本文提出的技术将在地面和室外操作的空中机器人上进行验证。

What is preventing robotic environmental monitoring of Canadian landscapes from becoming automated? The answer is twofold. First, there are challenges in hardware robustness, such as operation in adverse environmental conditions. Second, and most crucial, robots are oblivious to the type of data that environmental scientists really need. They do not know where to direct their attention. My overarching goal is to address the latter set of challenges with little supervision from and interaction with humans. This research will develop perception, inference, and control algorithms to address challenging robotics problems in informed visual exploration. The main application domain is the automation of environmental monitoring by robots operating outdoors, building on my past work in this area. The main users of the technology are environmental scientists. The core algorithms developed will be directly applicable to a vast array of human-robot interaction scenarios, and will span three key directions: (D1) Data-Efficient Learning of Robot Visual Attention with User Specifications: we will develop methods that efficiently learn what visual content in an image matters to the user. Predictive models of visual attention will be learned from a small number of labeled and a large number of unlabeled images. They will enable robots to perform informed visual exploration tailored to the user's preferences. We will also develop connections with reward learning and inverse reinforcement learning over images. This is a key enabling factor for human-robot interaction, which despite years of progress in low-dimensional settings, still lacks robust practical methods in high dimensions. (D2) Online Visual Exploration Strategies for Robots via Uncertainty in Visual Attention: we will estimate the model (epistemic) uncertainty of visual attention models, going beyond mere learning of a single probabilistic model. This will enable online, information gathering behaviors from the robot. It will also enable active selection of only a few images that the user needs to label in (D1). Most importantly, however, it will pave the way for the development of uncertainty-aware visual exploration strategies in 3D. (D3) Collaborative Human Multi-Robot Visual Exploration for Environmental Monitoring: we will design methods to enable a team of robots and a human to visually explore an unknown environment in tandem. This will allow robots of different capabilities to explore collaboratively, without constantly having to see where the human is, by making informed short-term and long-term predictions about his/her location, intent, and preferences. After exploring on their own, robots will need to relocate their human collaborator to redirect his/her attention towards any interesting observations. This research is unique in jointly addressing these challenges with a human in the loop. The techniques proposed herein will be validated on ground and aerial robots operating outdoors.