New Directions in Robotic Environmental Monitoring via Machine Learning

是什么阻碍了加拿大景观的机器人环境监测走向自动化？答案是双重的。首先，硬件鲁棒性方面存在挑战，例如在恶劣环境条件下的操作。其次，也是最关键的一点，机器人对环境科学家真正需要的数据类型一无所知。他们不知道该把注意力放在哪里。我的首要目标是解决后一组挑战，几乎没有人类的监督和互动。这项研究将开发感知、推理和控制算法，以解决信息视觉探索中具有挑战性的机器人问题。主要的应用领域是通过在户外操作的机器人实现环境监测的自动化，这是基于我过去在这一领域的工作。这项技术的主要使用者是环境科学家。开发的核心算法将直接适用于大量人机交互场景，并将跨越三个关键方向：（D1）基于用户规范的机器人视觉注意的数据高效学习：我们将开发有效学习图像中对用户重要的视觉内容的方法。视觉注意的预测模型将从少量标记和大量未标记的图像中学习。它们将使机器人能够根据用户的喜好进行知情的视觉探索。我们还将开发图像上的奖励学习和逆强化学习的联系。这是人机交互的关键促成因素，尽管多年来在低维环境中取得了进展，但在高维环境中仍然缺乏强大的实用方法。（D2）基于视觉注意不确定性的机器人在线视觉探索策略：我们将评估视觉注意模型的模型（认知）不确定性，而不仅仅是单一概率模型的学习。这将使机器人能够在线收集信息。它还将允许对用户需要在（D1）中标记的少数图像进行主动选择。然而，最重要的是，它将为3D中不确定性感知视觉探索策略的发展铺平道路。（D3）协同人类多机器人视觉探索环境监测：我们将设计方法，使一队机器人和一个人一起视觉探索未知的环境。这将允许不同能力的机器人通过对人类的位置、意图和偏好做出明智的短期和长期预测，协作探索，而不必经常看到人类在哪里。在自己探索之后，机器人将需要重新安置他们的人类合作者，以将他/她的注意力转移到任何有趣的观察上。这项研究在与人类共同应对这些挑战方面是独一无二的。本文提出的技术将在户外操作的地面和空中机器人上进行验证。

What is preventing robotic environmental monitoring of Canadian landscapes from becoming automated? The answer is twofold. First, there are challenges in hardware robustness, such as operation in adverse environmental conditions. Second, and most crucial, robots are oblivious to the type of data that environmental scientists really need. They do not know where to direct their attention. My overarching goal is to address the latter set of challenges with little supervision from and interaction with humans. This research will develop perception, inference, and control algorithms to address challenging robotics problems in informed visual exploration. The main application domain is the automation of environmental monitoring by robots operating outdoors, building on my past work in this area. The main users of the technology are environmental scientists. The core algorithms developed will be directly applicable to a vast array of human-robot interaction scenarios, and will span three key directions: (D1) Data-Efficient Learning of Robot Visual Attention with User Specifications: we will develop methods that efficiently learn what visual content in an image matters to the user. Predictive models of visual attention will be learned from a small number of labeled and a large number of unlabeled images. They will enable robots to perform informed visual exploration tailored to the user's preferences. We will also develop connections with reward learning and inverse reinforcement learning over images. This is a key enabling factor for human-robot interaction, which despite years of progress in low-dimensional settings, still lacks robust practical methods in high dimensions. (D2) Online Visual Exploration Strategies for Robots via Uncertainty in Visual Attention: we will estimate the model (epistemic) uncertainty of visual attention models, going beyond mere learning of a single probabilistic model. This will enable online, information gathering behaviors from the robot. It will also enable active selection of only a few images that the user needs to label in (D1). Most importantly, however, it will pave the way for the development of uncertainty-aware visual exploration strategies in 3D. (D3) Collaborative Human Multi-Robot Visual Exploration for Environmental Monitoring: we will design methods to enable a team of robots and a human to visually explore an unknown environment in tandem. This will allow robots of different capabilities to explore collaboratively, without constantly having to see where the human is, by making informed short-term and long-term predictions about his/her location, intent, and preferences. After exploring on their own, robots will need to relocate their human collaborator to redirect his/her attention towards any interesting observations. This research is unique in jointly addressing these challenges with a human in the loop. The techniques proposed herein will be validated on ground and aerial robots operating outdoors.