VEC: Small: Collaborative Research: Scene Understanding from RGB-D Images

VEC：小型：协作研究：RGB-D 图像的场景理解

• 批准号: 1539099
• 负责人: Jitendra Malik
• 金额: $ 15.2万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1539099&HistoricalAwards=false
• 关键词: VEC; Small; Collaborative; Research; Scene

This project exploits the benefits of RGB-D (color and depth) image collections with extra depth information to significantly advance the state-of-the-art in visual scene understanding, and makes computer vision techniques become usable in practical applications. Recent advance in affordable depth sensors has made depth acquisition significantly easier for ordinary users. These depth cameras are becoming very common in digital devices and help automatic scene understanding. The research team develops technologies to take advantage of depth information. Besides the published research results, the research team plans to distribute source code and benchmark data sets that could benefit researchers in a variety of disciplines. This project is integrated with educational programs, such as interdisciplinary workshops and courses at the graduate, undergraduate, and professional levels and diversity enhancement programs that promote opportunities for disadvantaged groups. The research team is closely collaborating with the industrial partner (Intel), involving interns and technology transfer in real products. The project is also applying the developed algorithms to the assistive technology for the blind and visually impaired.This research develops algorithms required to perform real-time segmentation, labeling, and recognition of RGB-D images, videos, and 3D scans of indoor environments. Specifically, the PIs develop methods to: (1) acquire large labeled RGB-D datasets for training and evaluation, (2) study algorithms to recognize objects and estimate detailed 3D knowledge about the scene, (3) exploit the object-to-object contextual relationships in 3D, and (4) demonstrate applications to benefit the general public, including household robotics and assistive technologies for the blind.

该项目利用RGB-D(颜色和深度)图像集合的优势，提供额外的深度信息，以显著提高视觉场景理解的最新水平，并使计算机视觉技术在实际应用中变得有用。最近在负担得起的深度传感器方面的进步使普通用户获得深度变得非常容易。这些深度相机在数字设备中变得非常常见，有助于自动理解场景。研究团队开发了利用深度信息的技术。除了公布的研究结果外，研究团队还计划发布源代码和基准数据集，这些数据集可能会让不同学科的研究人员受益。该项目与教育计划相结合，如研究生、本科生和专业水平的跨学科研讨会和课程，以及促进弱势群体机会的多样性增强计划。研究团队正在与产业合作伙伴英特尔(Intel)密切合作，涉及实习生和真实产品的技术转移。该项目还将开发的算法应用于盲人和视障人士的辅助技术。这项研究开发了对RGB-D图像、视频和室内环境的3D扫描执行实时分割、标记和识别所需的算法。具体地说，PI开发的方法是：(1)获取用于训练和评估的标记RGB-D大型数据集，(2)研究识别对象并估计关于场景的详细3D知识的算法，(3)在3D中利用对象到对象的上下文关系，以及(4)展示造福普通公众的应用程序，包括家用机器人和盲人辅助技术。