Richer sensors and challenging environments: filling a gap in training field robotic perception systems

点击翻译按钮获取中文摘要

To complete its mission, an autonomous vehicle operating outdoor must be able to interpret the world around itself. For instance, it must find drivable areas or identify actionable items. Simultaneously, it has to localize itself, oftentimes by building a map of the environment. This problem, called artificial perception, rests on algorithms that have been developed primarily using publicly available datasets. Consequently, these datasets heavily influence research directions in the scientific community. For example, many image detection algorithms for outdoor scenes have been tuned for medium-resolution 2-megapixels color images. Likewise, algorithms developed for lidars often expect the presence of smooth flat surfaces, as datasets have been captured chiefly in urban areas. This research proposal seeks to find a better understanding of the intimate relationship between algorithms, sensors and environments, in the context of artificial perception in field robotics. To do so, we will first explore what gains can be achieved by using newer sensing technologies for outdoor scene understanding. We will collect a dataset using a 60-megapixels camera, a multipolar camera and a hyperspectral camera. A multipolar camera senses the direction of polarization of incoming light rays, which provides information about how light interacts with surfaces. A hyperspectral camera gathers information for 50-100 color bands, much richer than the three for a standard color camera. Next, we will leverage this rich information to facilitate the training of perception systems based on artificial intelligence (AI). Just as we relied on better data and AI to improve scene understanding, we seek the same for lidar-based localization. Many deployed systems, including autonomous cars, rely on a localization approach called Iterative Closest Point (ICP). ICP expects large flat surfaces to be present, in order to "anchor" the current 3D lidar scan to a model of the world. The divergence between the two is used to refine the localization estimate. To circumvent this overreliance on flat surfaces, we will first collect 3D lidar datasets in unstructured environments such as forests. Then, we will augment the ICP localization algorithm, by incorporating a machine-learnable element. Our approach would then be able to learn how to exploit the local divergence between lidar measurements and the model, by considering the surrounding scene, i.e., is it a forest or a highway. On the theoretical side, this research will expand our fundamental understanding of the role of sensing in artificial intelligence. On a practical side, it will facilitate the development of autonomous systems deployed in unstructured environments. In turns, this could benefit the forestry and mining industries, by alleviating the problem of labor shortage. It would also contribute to improving the work condition and safety of workers, by removing them from harm's way through vehicle automation.