NRI: FND: Self-supervised Object Discovery, Detection and Visual Object Search

服务机器人在家庭和服务环境中无处不在的部署依赖于检测和识别感兴趣的对象并向它们导航的能力。在过去的几年里，在机器学习方法的推动下，计算机视觉社区取得了巨大的进步。然而，用于训练和评估的标准数据集通常由来自互联网的静态图像组成，需要大量的手动注释。虽然这种范式对于学习常见的对象类别是有效的，但它并不能推广到服务机器人应用程序中可能存在的数千个感兴趣的对象。开发不需要通过详细的人工注释来监督的学习算法是计算机视觉和人工智能的核心问题之一。我们对人类和生物系统如何在环境中获得关于视觉内容的新知识的理解激发了这一领域的开放问题。这个项目将导致一个新的算法类的对象发现，对象检测，三维环境建模和导航。该研究将为乔治梅森大学的研究生和本科生提供支持，并将进一步推进主动视觉基准数据集，以评估服务机器人的开发和部署。该项目的技术目标侧重于开发学习对象表示的方法，这些方法特定于机器人操作的上下文，可以以自我监督的方式学习，而无需费力的注释，并且可用于多个任务。本研究利用摄像机运动作为一种自我监督的形式来学习新的多视图目标嵌入，然后在很少或不需要标记的情况下对强大的目标检测器模型进行零拍摄或少拍摄的检测训练。物体检测的固有局限性将通过语义目标驱动的导航技术在机器人环境中解决，该技术是在为物体检测开发的表示和体系结构之上的强化学习框架中学习的。这些策略将构成机器人代理的一套基本的视觉引导导航技能，并将与绘图和探索策略相结合。这些方法将受到当前室内场景中具身智能体感知的挑战的推动，但解决方案将广泛适用于需要智能体与动态变化的环境进行长期持续交互的设置。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The ubiquitous deployment of service robots in homes and service environments rests on the ability to detect and recognize objects of interest and navigate towards them. In the past few years, largely enabled by machine-learning approaches, there has seen tremendous progress by the computer vision community. The standard datasets for training and evaluation, however, typically consist of static images curated from the internet and requiring extensive manual annotation. While this paradigm is effective for learning commonly encountered object categories, it does not generalize to possibly thousands of objects of interest in service robotics applications. The development of learning algorithms which do not require supervision through detailed human annotations is one of the central problems in computer vision and artificial intelligence. The open problems in this area are motivated by our understanding how humans and biological systems acquire new knowledge about visual content in the environments. This project will lead to a new class of algorithms for object discovery, object detection, 3-D environment modeling, and navigation. The research will support a cohort of diverse graduate and undergraduate students at George Mason University and will further advance the active vision benchmark dataset for evaluating the development and deployment of service robots.Technical aims of the project focus on the development of methods for learning representations of objects which are specific to the context where the robot operates, can be learned in self-supervised manner without need for laborious annotations, and are reusable for multiple tasks. This research utilizes the camera motion as a form of self-supervision for learning the new multi-view object embeddings, followed by zero-shot or few-shot detection training of powerful object detector models with little or no labelling effort. The inherent limitations of object detection will be tackled in the robotic setting by semantic target driven navigation techniques, learned in a reinforcement learning framework on top of representations and architectures developed for object detection. These policies will constitute a basic set of visually guided navigation skills of the robotic agent and will be integrated with mapping and exploration strategies. The approaches will be motivated by the current challenges of embodied agents' perception in indoors scenes, but the solutions will be broadly applicable in settings which require the long-term on-going interactions of an agent with dynamically changing environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.