RI: Small: Representation Learning for Semantic Mapping and Safe Robot Navigation

自主机器人系统为改变包括交通、建筑、采矿和农业在内的各个工业部门提供了巨大的潜力。然而，这些领域的运行条件是无组织的和动态变化的。这给当前的机器人系统设计带来了重大挑战，因为它们依赖于静态的离线环境模型和刚性的动力学模型，而这些模型不会随着操作经验的改善而得到改善。自主系统在这些领域的影响也受到手工设计的安全规则的限制，这些规则未能考虑到现实世界操作的复杂性和不确定性。该项目将开发新的理论和算法工具，以提高自主系统从感官观察中在线理解周围环境的能力，并根据不断变化的条件安全地调整其操作。在教育方面，该项目旨在增加未被充分代表的本科生参与与人类环境中机器人自主相关的教育和研究活动，并在STEM领域培养新人才，这将对美国经济的未来至关重要。该项目的关键创新包括根据感官观察在线推断物体形状和机器人动力学模型的技术，以及受观察物体安全约束的学习机器人动力学的控制设计。首先，提出了一种物体形状的隐式表面模型，该模型用一个隐含特征向量进行紧凑编码。提出了一种联合估计物体姿态、形状和机器人运动的在线优化算法。该算法实现了语义环境的理解和自主导航安全约束的规范。其次，利用Koopman算子的频谱特性和贝叶斯神经网络的通用性，设计了自监督机器人动力学学习算法。这些算法提供了一种从在线数据估计机器人动力学的自适应方法，同时依靠近似误差界来保证控制设计满足感知系统提供的安全约束。这些创新将使自主机器人能够在未知环境中运行，由于使用了学习的机器人和对象模型，并且由于在控制设计中使用了感知和不确定性感知约束，因此是安全的。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Autonomous robot systems offer tremendous potential for transforming various industry sectors, including transportation, construction, mining, and agriculture. The operational conditions in these domains, however, are unstructured and dynamically changing. This poses a major challenge for current robot system designs as they depend on static offline environment models and rigid dynamics models that do not improve with operational experience. The impact of autonomous systems in these domains is also limited by hand-designed safety rules that fail to account for the complexity and uncertainty of real-world operation. This project will develop new theoretical and algorithmic tools for advancing the ability of autonomous systems to comprehend their surroundings online from sensory observations and adapt their operation safely in response to changing conditions. On the educational front, the project aims to increase the participation of underrepresented undergraduate students in education and research activites related to robot autonomy in human environments and develop new talent in the STEM fields, which will be critical for the future of the U.S. economy.The key innovations of the project include techniques for online inference of object shapes and robot dynamics models from sensory observations as well as control design for the learned robot dynamics, subject to safety constraints from the observed objects. First, an implicit surface model of object shape, compactly encoded with a latent feature vector is proposed. An online optimization algorithm to estimate the object poses, shapes, and robot motion jointly is developed. This algorithm enables semantic environment understanding and specification of safety constraints for autonomous navigation. Second, the spectral properties of the Koopman operator and the versatility of Bayesian neural networks are leveraged to design self-supervised robot dynamics learning algorithms. These algorithms provide an adaptive way of estimating robot dynamics from online data, while relying on approximation error bounds to guarantee that the control design satisfies the safety constraints provided by the perceptual system. These innovations will enable autonomous robot operation in unknown environments that is adaptable, due to the use of learned robot and object models, and safe, due to the use of perception- and uncertainty-aware constraints in the control design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.