Richer sensors and challenging environments: filling a gap in training field robotic perception systems

为了完成任务，在户外运行的自动驾驶车辆必须能够解读自己周围的世界。例如，它必须找到可驾驶的区域或确定可操作的物品。同时，它必须进行本地化，通常是通过构建环境地图来实现。这个问题被称为人工感知，依赖于主要使用公开可用的数据集开发的算法。因此，这些数据集在很大程度上影响着科学界的研究方向。例如，许多用于室外场景的图像检测算法已经针对中分辨率200万像素的彩色图像进行了调整。同样，为激光雷达开发的算法通常希望出现光滑的平坦表面，因为数据集主要是在城市地区捕获的。这项研究建议寻求在领域机器人的人工感知的背景下，更好地理解算法、传感器和环境之间的密切关系。要做到这一点，我们将首先探索使用更新的传感技术来理解户外场景可以获得什么收益。我们将使用6000万像素的相机、多极相机和高光谱相机收集数据集。多极相机感测入射光线的偏振方向，提供有关光线如何与表面相互作用的信息。一台高光谱相机收集50-100个色带的信息，比标准彩色相机收集的三个色带丰富得多。接下来，我们将利用这些丰富的信息来促进基于人工智能(AI)的感知系统的训练。就像我们依靠更好的数据和人工智能来提高场景理解一样，我们也寻求同样的基于激光雷达的定位。许多已部署的系统，包括自动驾驶汽车，都依赖于一种名为迭代最近点(ICP)的本地化方法。为了将当前的3D激光雷达扫描“锚定”到一个世界模型上，ICP预计会出现巨大的平面。两者之间的差异被用来改进本地化估计。为了避免这种对平面的过度依赖，我们将首先在非结构化环境(如森林)中收集3D激光雷达数据集。然后，我们将通过加入机器可学习的元素来增强ICP定位算法。然后，我们的方法将能够学习如何通过考虑周围场景，即它是森林还是高速公路，来利用激光雷达测量和模型之间的局部差异。在理论方面，这项研究将扩大我们对感知在人工智能中作用的根本性理解。在实践方面，它将促进在非结构化环境中部署的自主系统的开发。反过来，这可能会缓解劳动力短缺的问题，从而使林业和采矿业受益。它还将通过车辆自动化消除工人的危险，从而有助于改善工人的工作条件和安全。

To complete its mission, an autonomous vehicle operating outdoor must be able to interpret the world around itself. For instance, it must find drivable areas or identify actionable items. Simultaneously, it has to localize itself, oftentimes by building a map of the environment. This problem, called artificial perception, rests on algorithms that have been developed primarily using publicly available datasets. Consequently, these datasets heavily influence research directions in the scientific community. For example, many image detection algorithms for outdoor scenes have been tuned for medium-resolution 2-megapixels color images. Likewise, algorithms developed for lidars often expect the presence of smooth flat surfaces, as datasets have been captured chiefly in urban areas. This research proposal seeks to find a better understanding of the intimate relationship between algorithms, sensors and environments, in the context of artificial perception in field robotics. To do so, we will first explore what gains can be achieved by using newer sensing technologies for outdoor scene understanding. We will collect a dataset using a 60-megapixels camera, a multipolar camera and a hyperspectral camera. A multipolar camera senses the direction of polarization of incoming light rays, which provides information about how light interacts with surfaces. A hyperspectral camera gathers information for 50-100 color bands, much richer than the three for a standard color camera. Next, we will leverage this rich information to facilitate the training of perception systems based on artificial intelligence (AI). Just as we relied on better data and AI to improve scene understanding, we seek the same for lidar-based localization. Many deployed systems, including autonomous cars, rely on a localization approach called Iterative Closest Point (ICP). ICP expects large flat surfaces to be present, in order to "anchor" the current 3D lidar scan to a model of the world. The divergence between the two is used to refine the localization estimate. To circumvent this overreliance on flat surfaces, we will first collect 3D lidar datasets in unstructured environments such as forests. Then, we will augment the ICP localization algorithm, by incorporating a machine-learnable element. Our approach would then be able to learn how to exploit the local divergence between lidar measurements and the model, by considering the surrounding scene, i.e., is it a forest or a highway. On the theoretical side, this research will expand our fundamental understanding of the role of sensing in artificial intelligence. On a practical side, it will facilitate the development of autonomous systems deployed in unstructured environments. In turns, this could benefit the forestry and mining industries, by alleviating the problem of labor shortage. It would also contribute to improving the work condition and safety of workers, by removing them from harm's way through vehicle automation.