CRII: FRR: Semantic Vector Fields for Robot Navigation and Exploration in Unstructured Environments

为了实现机器人更大的机动性和自主性，重要的是要研究强大的感知和导航算法，这些算法不仅可以在铺砌的表面上运行良好，还可以在包括岩石、植被和其他危险的具有挑战性的地形下运行良好。在高速公路和仓库等结构化环境中，机器人可以按照车道标记或预先定义的路径进行导航。然而，在建筑、农业、农村运输和灾难现场等非结构化环境中，机器人需要对周围物体和地形有更深入的了解，才能安全导航。不幸的是，大多数用于导航和探索的移动机器人系统主要使用传感器来收集几何信息，如障碍物和环境中其他物体的形状和位置。需要更丰富的信息，如地形类型和穿越的难度，以方便导航和探索。为了解决这一基础研究缺口，本项目旨在研究一种用于野外机器人的语义感知框架，该框架将几何信息与地形属性的语义信息相补充，以提高机器人的导航和探索能力。这项研究代表了实现具有先进人工智能能力的机器人的重要一步，这些机器人可以在非结构化建筑、采矿或农业环境中很好地工作。特别是，在这些传统劳动密集型产业中，消除自动化技术的进入壁垒对于提高这些产业在未来工作场所的经济竞争力至关重要。本研究将利用基于深度学习的3D语义推理工具与运动反馈的自监督学习相结合，重新审视势场导航方法，为机器人主动感知提供新的视角。语义导航方法包含三个主要组成部分：(i)将原始传感器数据映射到语义特征，并将语义特征映射到导航信号的语义向量场预测网络；（ii）将观测数据库和专家演示的先验知识转移到导航系统的预训练方案；（iii）语义引导的探索方案，使机器人能够在收集周围环境信息的同时承担可计算的风险。该方法将首先在基于物理的MSU自动驾驶车辆模拟器（MAVS）上进行评估，该模拟器具有模拟机器人在具有挑战性的地形和天气下操作的能力。最后，现场实验将在密西西比州立大学独一无二的越野试验场进行。该项目由跨部门机器人基础研究项目支持，由工程（ENG）和计算机与信息科学与工程（CISE）联合管理和资助。该项目也由促进竞争研究的既定计划（EPSCoR）共同资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

To achieve greater mobility and autonomy for robots, it is important to study robust perception and navigation algorithms that can operate well not just with paved surfaces but also under challenging terrain involving rocks, vegetation, and other hazards. In structured environments such as highways and warehouses, robots can navigate by following lane markings or pre-defined paths. However, in unstructured environments, such as construction, agriculture, rural delivery and disaster sites, robots need to have a deeper understanding of the surrounding objects and terrain in order to navigate safely. Unfortunately, most mobile robot systems deployed for navigation and exploration use their sensors to mainly gather geometric information such as the shape and location of the obstacles and other objects in the environment. Richer information is needed, such as type of terrains and how difficult they will be to traversed, to facilitate navigation and exploration. To address this fundamental research gap, this project aims to investigate a semantically-aware framework for field robots that complements the geometric information with semantic information about terrain properties in order to improve the navigation and exploration capabilities of robots. This research represents an important step towards achieving robots with advanced artificial intelligence capabilities that can operate well in unstructured construction, mining, or agriculture environments. In particular, removing barriers of entry for automation technology in these traditionally labor-intensive industries is vital towards increased economic competitiveness of these industries in workplaces of the future.This research will revisit the potential field navigation method using deep learning-based 3D semantic reasoning tools paired with self-supervised learning from motion feedback to provide a fresh perspective on active perception for robots. The semantic navigation method contains three main components: (i) a semantic vector field prediction network to map raw sensor data to semantic features and map semantic features to navigation signals; (ii) a pre-training scheme that transfers prior knowledge from an observation database and expert demonstrations to the navigation system; and (iii) a semantically-guided exploration scheme to enable the robot to take calculated risks while gathering information about the surroundings. The method will initially be evaluated on the physics-based MSU Autonomous Vehicle Simulator (MAVS) which has the capability of simulating robot operation under challenging terrain and weather. Finally, field experiments will be conducted at the one-of-a-kind off-road proving ground at Mississippi State University.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE). This project is also jointly funded by the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.