NRI: Collaborative Research: Task Dependent Semantic Modeling for Robot Perception

该项目的研究使机器人能够更好地处理我们周围复杂杂乱的环境，从开放的场景到杂乱的桌面设置，并执行基本的映射，导航，对象搜索，以便实现服务合作机器人应用中最常见的获取和交付任务。 该项目的主要贡献是为机器人开发视觉感知系统，该系统可以根据特定机器人任务或人机交互的要求，在多个特定级别上理解视觉世界的语义标签。 此外，该项目使机器人感知系统能够通过自动适应现有的学习模型，并通过主动选择如何最好地探索和识别新的视觉空间和物体，更好地理解新的，以前看不见的环境。 数据集和基准，以及开发的模型，形成的基础上更快的进展语义视觉感知robotics.The发展的方法学习组合表示，使主动学习和有效的推理是一个长期存在的问题，在计算机视觉和机器人感知。在室内和室外环境的约束下，我们计划利用大量的数据，强大的几何和语义先验知识，并开发新的对象和场景的表示。所开发的表示由包括深度卷积网络的组合结构概率模型捕获。需要快速完成此操作以支持主动视觉探索，从而改进空间的语义解析。 此外，项目团队收集和传播了大量密集采样的RGBD图像数据集，以支持离线评估和语义解析主动视觉基准测试。 该项目可以导致机器人任务的主动分层语义视觉的进步，包括探索，搜索，操纵，编程的例子，并通常用于人机交互。

The research in this project enables robots to better deal with the complex cluttered environments around us, ranging from open scenes to cluttered table-top settings and to perform the basic mapping, navigation, object search so as to enable fetch and delivery tasks most commonly required in service co-robotics applications. The key contribution of the project is to develop visual perception systems for robots that can understand the semantic labels of the visual world at multiple levels of specificity as required by particular robot tasks or human-robot interaction. In addition, the project enables robot perception systems to better understand new, previously unseen, environments through automatically adapting existing learned models, and by actively choosing how to best explore and recognize novel visual spaces and objects. The datasets and benchmarks, as well as the developed models, form basis for more rapid progress on semantic visual perception for robotics.The development of methodologies for learning compositional representations which enable active learning and efficient inference is a long standing problem in computer vision and robot perception. Guided by the constraints of indoors and outdoors environments, we plan to exploit large amounts of data, strong geometric and semantic priors and develop novel representations of objects and scenes. The developed representations are captured by compositional structured probabilistic models including deep convolutional networks. Doing this rapidly is required to support active visual exploration to improve semantic parsing of a space. Furthermore the project team collects and disseminates a large dataset of densely sampled RGBD imagery to support offline evaluation and benchmarking of active vision for semantic parsing. The project can result in advances in active hierarchical semantic vision for robot tasks including exploration, search, manipulation, programming by example, and generally for human-robot interaction.