VEC: Small: Collaborative Research: Joint Compressive Spectral Imaging and 3D Ranging Sensing Using a Commodity Time-Of-Flight Range Sensor

VEC：小型：协作研究：使用商品飞行时间距离传感器进行联合压缩光谱成像和 3D 测距传感

• 批准号: 1538950
• 负责人: Gonzalo Arce
• 金额: $ 27.46万
• 依托单位: University of Delaware
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1538950&HistoricalAwards=false
• 关键词: VEC; Small; Collaborative; Research; Joint

The commercial Kinect input device uses both a camera and a time-of-flight depth sensor to capture 3d interactions ? initially for gaming, it has found novel applications, including autism diagnosis and allowing surgeons to control imaging in the operating room. This project aims to integrate the time-of-flight (TOF) depth with Coded Aperture Snapshot Spectral Imaging (CASSI) in a single sensor, giving 3d images not only of visible light, but extending into the infrared. This will have advantages of improving the resolution of the depth information, allowing the two sensors to work together to improve the quality of both. The result will be the first generation of 3D-based ?spectrophotographic? cameras for use on robotic platforms and as the heart of machine vision systems used in manufacturing and metrology. With their higher spatial and spectral resolution, these could open new and unexplored markets. This project develops a time-of-flight, spectral imaging camera by addressing the problems of (a) novel coded aperture design and optimization under light level constraints; (b) fast compressive inverse reconstruction algorithms for real-time implementations; (c) characterization on non-ideal optical elements and calibration mechanisms, and (d) super-resolution enhancement schemes. The PIs will design and assemble a proof-of-concept prototype camera that integrates a DMD array with a CMOS, TOF image sensor in a package comparable to a pico-projector light engine. In order to mitigate the computational complexity of CS inverse image reconstruction algorithms that often preclude their use in real-time implementations, the PIs will build on their history of developing coded aperture schemes that give rise to divide-and-conquer image reconstruction algorithms that can be implemented on GPU and other multi-core processors. This project also incorporates education activities including: (1) the development of a compressive optical imaging course with an emphasis on spectral and TOF modalities; (2) technical seminars at the annual Society of Hispanic Professional Engineers (SHPE) Conference; (3) open-source algorithms and measurement data for the scientific community.

商业Kinect输入设备使用摄像头和飞行时间深度传感器来捕捉3D交互？最初用于游戏，它已经找到了新的应用，包括自闭症诊断和允许外科医生在手术室控制成像。该项目旨在将飞行时间（TOF）深度与编码孔径快照光谱成像（CASSI）集成在单个传感器中，不仅提供可见光的3D图像，还提供扩展到红外的3D图像。 这将具有提高深度信息的分辨率的优点，允许两个传感器一起工作以提高两者的质量。 其结果将是第一代基于3D？分光照相的？用于机器人平台的相机，以及用于制造和计量的机器视觉系统的核心。 凭借其更高的空间和光谱分辨率，这些可以打开新的和未开发的市场。该项目通过解决以下问题开发了一种飞行时间光谱成像相机：（a）新的编码孔径设计和光照水平约束下的优化;（B）用于实时实现的快速压缩逆重建算法;（c）非理想光学元件和校准机制的表征;以及（d）超分辨率增强方案。PI将设计和组装一个概念验证原型相机，该相机将DMD阵列与CMOS，TOF图像传感器集成在一个可与微型投影机光引擎相媲美的封装中。为了减轻CS逆图像重建算法的计算复杂性，这通常会妨碍它们在实时实现中的使用，PI将建立在他们开发编码孔径方案的历史基础上，这些方案产生了可以在GPU和其他多核处理器上实现的分而治之的图像重建算法。 该项目还包括教育活动，包括：（1）开发压缩光学成像课程，重点是光谱和TOF模式;（2）在西班牙裔专业工程师协会（SHPE）年度会议上举办技术研讨会;（3）为科学界提供开源算法和测量数据。