NRI: FND: Self-supervised Object Discovery, Detection and Visual Object Search

NRI：FND：自监督对象发现、检测和视觉对象搜索

• 批准号: 1925231
• 负责人: Jana Kosecka
• 金额: $ 50万
• 依托单位: George Mason University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1925231&HistoricalAwards=false
• 关键词: NRI; FND; Self; supervised; Object

The ubiquitous deployment of service robots in homes and service environments rests on the ability to detect and recognize objects of interest and navigate towards them. In the past few years, largely enabled by machine-learning approaches, there has seen tremendous progress by the computer vision community. The standard datasets for training and evaluation, however, typically consist of static images curated from the internet and requiring extensive manual annotation. While this paradigm is effective for learning commonly encountered object categories, it does not generalize to possibly thousands of objects of interest in service robotics applications. The development of learning algorithms which do not require supervision through detailed human annotations is one of the central problems in computer vision and artificial intelligence. The open problems in this area are motivated by our understanding how humans and biological systems acquire new knowledge about visual content in the environments. This project will lead to a new class of algorithms for object discovery, object detection, 3-D environment modeling, and navigation. The research will support a cohort of diverse graduate and undergraduate students at George Mason University and will further advance the active vision benchmark dataset for evaluating the development and deployment of service robots.Technical aims of the project focus on the development of methods for learning representations of objects which are specific to the context where the robot operates, can be learned in self-supervised manner without need for laborious annotations, and are reusable for multiple tasks. This research utilizes the camera motion as a form of self-supervision for learning the new multi-view object embeddings, followed by zero-shot or few-shot detection training of powerful object detector models with little or no labelling effort. The inherent limitations of object detection will be tackled in the robotic setting by semantic target driven navigation techniques, learned in a reinforcement learning framework on top of representations and architectures developed for object detection. These policies will constitute a basic set of visually guided navigation skills of the robotic agent and will be integrated with mapping and exploration strategies. The approaches will be motivated by the current challenges of embodied agents' perception in indoors scenes, but the solutions will be broadly applicable in settings which require the long-term on-going interactions of an agent with dynamically changing environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

服务机器人在家庭和服务环境中的无处不在的部署取决于检测和识别感兴趣的对象并向它们导航的能力。在过去的几年里，在机器学习方法的支持下，计算机视觉社区取得了巨大的进步。然而，用于训练和评估的标准数据集通常由来自互联网的静态图像组成，需要大量的手动注释。虽然这种范例对于学习常见的对象类别是有效的，但它不能推广到服务机器人应用中可能感兴趣的数千个对象。开发不需要通过详细的人类注释进行监督的学习算法是计算机视觉和人工智能的核心问题之一。这一领域的开放性问题的动机是我们理解人类和生物系统如何获得有关环境中视觉内容的新知识。这个项目将导致一类新的算法的对象发现，对象检测，三维环境建模和导航。该研究将支持乔治梅森大学的一群不同的研究生和本科生，并将进一步推进主动视觉基准数据集，用于评估服务机器人的开发和部署。该项目的技术目标侧重于开发用于学习对象表示的方法，这些对象表示特定于机器人操作的环境，可以以自我监督的方式学习，而不需要费力的注释，并且可重复用于多个任务。这项研究利用相机运动作为一种自我监督的形式，用于学习新的多视图对象嵌入，然后对功能强大的对象检测器模型进行零拍摄或少量拍摄检测训练，很少或根本没有标记工作。物体检测的固有局限性将在机器人环境中通过语义目标驱动的导航技术来解决，这些技术是在为物体检测开发的表示和架构之上的强化学习框架中学习的。这些政策将构成一套基本的视觉导航技能的机器人代理，并将与映射和探索策略相结合。这些方法将受到当前在室内场景中体现代理感知的挑战的激励，但这些解决方案将广泛适用于需要代理与动态变化的环境进行长期持续互动的环境。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

项目成果

Learning-Augmented Model-Based Planning for Visual Exploration

基于学习增强模型的视觉探索规划

• 发表时间: 2023
• 期刊: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Yimeng Li;Arnab Debnath;Gregory J. Stein;Jana Košecká
• 通讯作者: Jana Košecká

Object Pose Estimation using Mid-level Visual Representations

• DOI: 10.1109/iros47612.2022.9981452
• 发表时间: 2022-03
• 期刊: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 作者: Negar Nejatishahidin;Pooya Fayyazsanavi;J. Kosecka

Learning View and Target Invariant Visual Servoing for Navigation

学习用于导航的视图和目标不变视觉伺服

• 发表时间: 2020
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Li, Yimeng;Kosecka, Jana
• 通讯作者: Kosecka, Jana

Uncertainty Aware Proposal Segmentation for Unknown Object Detection

• DOI: 10.1109/wacvw54805.2022.00030
• 发表时间: 2021-11
• 期刊: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)
• 作者: Yimeng Li;J. Kosecka