NRI: Collaborative Research: Task Dependent Semantic Modeling for Robot Perception

该项目的研究使机器人能够更好地处理我们周围复杂的杂乱环境，从开放场景到杂乱的桌面设置，并执行基本的地图绘制、导航和对象搜索，从而实现服务协作机器人应用中最常见的取回和递送任务。该项目的主要贡献是为机器人开发视觉感知系统，该系统可以根据特定机器人任务或人-机器人交互的要求，在多个特定级别上理解视觉世界的语义标签。此外，该项目通过自动调整现有的学习模型，并通过主动选择如何最好地探索和识别新的视觉空间和对象，使机器人感知系统能够更好地理解新的、以前未见过的环境。这些数据集和基准以及开发的模型为机器人语义视觉感知的更快发展奠定了基础。学习成分表征的方法的发展使其能够进行主动学习和有效的推理是计算机视觉和机器人感知中的一个长期问题。在室内和室外环境的约束下，我们计划利用大量的数据，强烈的几何和语义先验，开发对象和场景的新表示。所开发的表示被包括深卷积网络的组合结构化概率模型捕获。为了支持主动的视觉探索以改进空间的语义解析，需要快速地这样做。此外，项目团队收集和传播密集采样的RGBD图像的大型数据集，以支持语义分析的主动视觉的离线评估和基准测试。该项目可以促进用于机器人任务的主动分层语义视觉的进步，包括探索、搜索、操作、举例编程，以及一般用于人-机器人交互的主动分层语义视觉。

The research in this project enables robots to better deal with the complex cluttered environments around us, ranging from open scenes to cluttered table-top settings and to perform the basic mapping, navigation, object search so as to enable fetch and delivery tasks most commonly required in service co-robotics applications. The key contribution of the project is to develop visual perception systems for robots that can understand the semantic labels of the visual world at multiple levels of specificity as required by particular robot tasks or human-robot interaction. In addition, the project enables robot perception systems to better understand new, previously unseen, environments through automatically adapting existing learned models, and by actively choosing how to best explore and recognize novel visual spaces and objects. The datasets and benchmarks, as well as the developed models, form basis for more rapid progress on semantic visual perception for robotics.The development of methodologies for learning compositional representations which enable active learning and efficient inference is a long standing problem in computer vision and robot perception. Guided by the constraints of indoors and outdoors environments, we plan to exploit large amounts of data, strong geometric and semantic priors and develop novel representations of objects and scenes. The developed representations are captured by compositional structured probabilistic models including deep convolutional networks. Doing this rapidly is required to support active visual exploration to improve semantic parsing of a space. Furthermore the project team collects and disseminates a large dataset of densely sampled RGBD imagery to support offline evaluation and benchmarking of active vision for semantic parsing. The project can result in advances in active hierarchical semantic vision for robot tasks including exploration, search, manipulation, programming by example, and generally for human-robot interaction.