NRI: Large-Scale Collaborative Semantic Mapping using 3D Structure from Motion

该项目开发技术，以推进最先进的技术，以应对与使用机器人创建此类表示相关的挑战，即与此类地图的可扩展性和语义可解释性相关的问题。研究活动包括多个领域的知识进步，如计算机视觉，运动结构，机器人和语义映射。研究结果具有许多社会应用的潜力，包括城市规划、资产管理、历史记录的创建和对自动驾驶的支持。通过语义地图促进人类和机器人之间的实时交互的开发理论技术的演示，可以实现更大的社会效益，例如应急管理，预防犯罪和交通管理。直接的教育影响，预计研究生和结果通过出版物和软件传播，使社会利用result.This研究计划推进实时大规模分布式室外环境的语义映射。具体来说，研究团队正在通过使用无监督对象发现来实现实时大规模语义映射，从而避免了对每个对象类别的大量注释视频集的需求，这在处理数百个对象类别时变得令人望而却步。研究团队将这一过程框架在结构运动优化框架内，从而利用几何和多视图约束和特征来提高对象轨迹关联和类别聚类的可靠性。除了解决可扩展性问题外，该项目还开发了一个分布式多机器人系统，允许大型空中和地面车辆团队在合理的时间范围内合作构建大型地理区域的地图。此外，该项目还开发了使地图更具语义意义的技术，从而使人类更容易解释。为了实现这一目标，研究小组使用自动技术将语义标签附加到以无监督方式发现的对象上。此外，人类可以在多个层面上与系统交互。人类用户可以细化对象类别和语义标签以提高其准确性，并指定感兴趣的动态目标和任务机器人来跟踪它们。

The project develops techniques to advance the state of the art in tackling the challenges associated with creating such representations using robots, namely issues related to the scalability and semantic interpretability of such maps. The research activities include advancement of knowledge in multiple fields, such as computer vision, structure from motion, robotics, and semantic mapping. The results have the potential for many societal applications including city planning, asset management, creation of historical records, and support for autonomous driving. The demonstration of the developed theoretical techniques for real-time interaction between humans and robots facilitated by a semantic map enables even greater societal benefit, for example for emergency management, crime prevention, and traffic management. Direct educational impact is anticipated for graduate students and the results are disseminated through both publications and software, allowing the community to leverage the results.This research program advances real-time large-scale distributed semantic mapping of outdoor environments. Specifically, the research team is enabling real-time large-scale semantic mapping by using unsupervised object discovery, obviating the need for large sets of annotated videos for each object category which becomes prohibitive when dealing with hundreds of object categories. The research team frames this process within the structure from motion optimization framework, thereby leveraging geometric and multi-view constraints and features to increase reliability of object track association as well as category clustering. In addition to address scalability, the project develops a distributed, multi-robot system, allowing large teams of air and ground vehicles to cooperatively build a map of large geographic areas in reasonable time frames. Furthermore, the project develops techniques to make the maps more semantically-meaningful and hence interpretable by humans. To accomplish this objective, the research team uses automatic techniques to attach semantic labels to objects discovered in an unsupervised manner. Moreover, humans can interact with the system at multiple levels. Human users can refine both the object categories and semantic labels to increase their accuracy, as well as designate dynamic targets of interest and task robots to track them.