NRI: Collaborative Research: Task Dependent Semantic Modeling for Robot Perception

该项目的研究使机器人能够更好地处理我们周围复杂杂乱的环境，从开放场景到杂乱的桌面设置，并执行基本的映射、导航、对象搜索，从而实现服务协同机器人应用中最常见的获取和交付任务。该项目的关键贡献是为机器人开发视觉感知系统，该系统可以根据特定机器人任务或人机交互的要求，在多个特定级别上理解视觉世界的语义标签。此外，该项目使机器人感知系统能够通过自动适应现有的学习模型，并通过积极选择如何最好地探索和识别新的视觉空间和物体，更好地理解新的、以前未见过的环境。这些数据集和基准，以及开发的模型，为机器人语义视觉感知的快速发展奠定了基础。开发能够实现主动学习和有效推理的组合表示学习方法是计算机视觉和机器人感知领域长期存在的问题。在室内和室外环境约束的指导下，我们计划利用大量数据，强大的几何和语义先验，并开发新的对象和场景表示。开发的表示由组合结构化概率模型捕获，包括深度卷积网络。为了支持主动的视觉探索以改进空间的语义解析，需要快速地完成这些工作。此外，项目团队收集和传播密集采样的RGBD图像的大型数据集，以支持离线评估和主动视觉语义解析的基准测试。该项目可以在机器人任务的主动分层语义视觉方面取得进展，包括探索、搜索、操作、示例编程，以及通常的人机交互。

The research in this project enables robots to better deal with the complex cluttered environments around us, ranging from open scenes to cluttered table-top settings and to perform the basic mapping, navigation, object search so as to enable fetch and delivery tasks most commonly required in service co-robotics applications. The key contribution of the project is to develop visual perception systems for robots that can understand the semantic labels of the visual world at multiple levels of specificity as required by particular robot tasks or human-robot interaction. In addition, the project enables robot perception systems to better understand new, previously unseen, environments through automatically adapting existing learned models, and by actively choosing how to best explore and recognize novel visual spaces and objects. The datasets and benchmarks, as well as the developed models, form basis for more rapid progress on semantic visual perception for robotics.The development of methodologies for learning compositional representations which enable active learning and efficient inference is a long standing problem in computer vision and robot perception. Guided by the constraints of indoors and outdoors environments, we plan to exploit large amounts of data, strong geometric and semantic priors and develop novel representations of objects and scenes. The developed representations are captured by compositional structured probabilistic models including deep convolutional networks. Doing this rapidly is required to support active visual exploration to improve semantic parsing of a space. Furthermore the project team collects and disseminates a large dataset of densely sampled RGBD imagery to support offline evaluation and benchmarking of active vision for semantic parsing. The project can result in advances in active hierarchical semantic vision for robot tasks including exploration, search, manipulation, programming by example, and generally for human-robot interaction.