VEC: Small: Collaborative Research: Scene Understanding from RGB-D Images

VEC：小型：协作研究：RGB-D 图像的场景理解

• 批准号: 1539014
• 负责人: Thomas Funkhouser
• 金额: $ 13.5万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1539014&HistoricalAwards=false
• 关键词: VEC; Small; Collaborative; Research; Scene

This project exploits the benefits of RGB-D (color and depth) image collections with extra depth information to significantly advance the state-of-the-art in visual scene understanding, and makes computer vision techniques become usable in practical applications. Recent advance in affordable depth sensors has made depth acquisition significantly easier for ordinary users. These depth cameras are becoming very common in digital devices and help automatic scene understanding. The research team develops technologies to take advantage of depth information. Besides the published research results, the research team plans to distribute source code and benchmark data sets that could benefit researchers in a variety of disciplines. This project is integrated with educational programs, such as interdisciplinary workshops and courses at the graduate, undergraduate, and professional levels and diversity enhancement programs that promote opportunities for disadvantaged groups. The research team is closely collaborating with the industrial partner (Intel), involving interns and technology transfer in real products. The project is also applying the developed algorithms to the assistive technology for the blind and visually impaired.This research develops algorithms required to perform real-time segmentation, labeling, and recognition of RGB-D images, videos, and 3D scans of indoor environments. Specifically, the PIs develop methods to: (1) acquire large labeled RGB-D datasets for training and evaluation, (2) study algorithms to recognize objects and estimate detailed 3D knowledge about the scene, (3) exploit the object-to-object contextual relationships in 3D, and (4) demonstrate applications to benefit the general public, including household robotics and assistive technologies for the blind.

该项目利用具有额外深度信息的 RGB-D（颜色和深度）图像集合的优势，显着推进视觉场景理解的最先进水平，并使计算机视觉技术在实际应用中变得可用。经济实惠的深度传感器的最新进展使普通用户的深度获取变得更加容易。这些深度相机在数字设备中变得非常常见，有助于自动场景理解。研究团队开发了利用深度信息的技术。除了已发表的研究成果外，研究团队还计划分发源代码和基准数据集，这可能使各个学科的研究人员受益。该项目与教育计划相结合，例如研究生、本科生和专业水平的跨学科研讨会和课程，以及促进弱势群体机会的多样性增强计划。研究团队与工业合作伙伴（英特尔）密切合作，涉及实习生和实际产品的技术转让。该项目还将开发的算法应用于盲人和视障人士的辅助技术。这项研究开发了对室内环境的 RGB-D 图像、视频和 3D 扫描进行实时分割、标记和识别所需的算法。具体来说，PI 开发方法来：(1) 获取大型标记 RGB-D 数据集用于训练和评估，(2) 研究识别对象并估计有关场景的详细 3D 知识的算法，(3) 利用 3D 中的对象到对象上下文关系，以及 (4) 演示造福公众的应用程序，包括家用机器人和盲人辅助技术。