VEC: Small: Collaborative Research: Scene Understanding from RGB-D Images

该项目利用具有额外深度信息的RGB-D（颜色和深度）图像集合的优势，显着推进视觉场景理解的最新技术，并使计算机视觉技术在实际应用中变得可用。负担得起的深度传感器的最新进展使得深度获取对于普通用户而言明显更容易。这些深度相机在数字设备中变得非常普遍，并有助于自动场景理解。研究团队开发了利用深度信息的技术。除了已发表的研究成果外，研究团队还计划分发源代码和基准数据集，这些数据集可以使各个学科的研究人员受益。该项目与教育方案相结合，如研究生、本科生和专业一级的跨学科讲习班和课程，以及促进弱势群体机会的多样性增强方案。研究团队与工业合作伙伴（英特尔）密切合作，包括实习生和真实的产品技术转让。该项目还将开发的算法应用于盲人和视障人士的辅助技术。该研究开发了对RGB-D图像、视频和室内环境的3D扫描进行实时分割、标记和识别所需的算法。具体而言，PI开发方法：（1）获取大型标记的RGB-D数据集用于训练和评估，（2）研究识别对象并估计有关场景的详细3D知识的算法，（3）利用3D中的对象到对象的上下文关系，以及（4）演示使公众受益的应用程序，包括家用机器人和盲人辅助技术。

This project exploits the benefits of RGB-D (color and depth) image collections with extra depth information to significantly advance the state-of-the-art in visual scene understanding, and makes computer vision techniques become usable in practical applications. Recent advance in affordable depth sensors has made depth acquisition significantly easier for ordinary users. These depth cameras are becoming very common in digital devices and help automatic scene understanding. The research team develops technologies to take advantage of depth information. Besides the published research results, the research team plans to distribute source code and benchmark data sets that could benefit researchers in a variety of disciplines. This project is integrated with educational programs, such as interdisciplinary workshops and courses at the graduate, undergraduate, and professional levels and diversity enhancement programs that promote opportunities for disadvantaged groups. The research team is closely collaborating with the industrial partner (Intel), involving interns and technology transfer in real products. The project is also applying the developed algorithms to the assistive technology for the blind and visually impaired.This research develops algorithms required to perform real-time segmentation, labeling, and recognition of RGB-D images, videos, and 3D scans of indoor environments. Specifically, the PIs develop methods to: (1) acquire large labeled RGB-D datasets for training and evaluation, (2) study algorithms to recognize objects and estimate detailed 3D knowledge about the scene, (3) exploit the object-to-object contextual relationships in 3D, and (4) demonstrate applications to benefit the general public, including household robotics and assistive technologies for the blind.