CIF: Small: Image and Video Processing with Depth

CIF：小：深度图像和视频处理

• 批准号: 1218682
• 负责人: Minh Do
• 金额: $ 39.05万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1218682&HistoricalAwards=false
• 关键词: CIF; Small; Image; Video; Processing

This research aims to advance the theory and practice of image and video processing with the emerging 3D cameras. The recent huge commercial success and excitement with low-cost consumer-grade color plus depth cameras such as Microsoft Kinect promise the wide deployment of depth sensors to every computer and display device. This rapid development demands fundamental new methods in processing depth input and its combination with regular color images and videos. Furthermore, this emerging depth-sensing technology has the potential to radically extend the possibilities of image and video processing. Toward these aims, this research involves, (i) calibration and denoising the raw measurements from depth cameras via accurate physical and statistical modeling of the new depth sensors and typical 3D scenes, (ii) improving the quality and resolution of depth images by exploiting the synergy between color and depth cameras, (iii) integration of multiple depth frames as an extension of super-resolution in video from 2D to 3D using computer graphics tools, and, finally, (iv) employing the improved and integrated depth information with color videos in challenging and high-impact applications including color plus depth rendering, relighting, and encoding.

本研究旨在推进新兴3D相机的图像和视频处理的理论和实践。 最近，微软Kinect等低成本消费级彩色加深度相机取得了巨大的商业成功和兴奋，这预示着深度传感器将广泛部署到每台计算机和显示设备上。 这种快速发展需要处理深度输入及其与常规彩色图像和视频的组合的基本新方法。 此外，这种新兴的深度传感技术有可能从根本上扩展图像和视频处理的可能性。为了实现这些目标，本研究涉及：（i）通过对新深度传感器和典型3D场景进行准确的物理和统计建模，对深度相机的原始测量结果进行校准和去噪，（ii）通过利用协同作用提高深度图像的质量和分辨率彩色和深度相机，（iii）使用计算机图形工具将多个深度帧集成为视频中从2D到3D的超分辨率的扩展，以及最后，（iv）在具有挑战性和高影响力的应用（包括彩色加深度渲染、重新照明和编码）中将改进的和集成的深度信息与彩色视频一起使用。