Motion-tolerant coded-exposure-pixel cameras for emerging computational photography applications (Phase I)

用于新兴计算摄影应用的运动容错编码曝光像素相机（第一阶段）

• 批准号: 561521-2021
• 负责人: Genov, Roman
• 金额: $ 9.11万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Idea to Innovation
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=724224
• 关键词: Motion; tolerant; coded; exposure; pixel

Motion causes significant imaging artifacts, which become prohibitively large when complex computational photography algorithms are implemented on conventional cameras, as they often require the use of multiple video frames. Highly-specialized high-frame-rate image sensors can be used to perform CEP-like imaging without such artifacts. But such high-speed cameras are prone to high noise, excessive power consumption, and extremely high output data rate, and thus are typically bulky and very expensive. We are introducing an emerging class of coded-exposure pixel (CEP) image sensors, that keep motion artifacts low and eliminate other existing drawbacks by implementing exposure-time programmability for each individual pixel.Technically speaking, for each (relatively slow) frame readout many (fast) exposures are performed, with the photogenerated charge for each exposure directed to one of several accumulation nodes within the pixel, based on a pixel code. This simple in-pixel charge-sorting function comes at a cost of an inconsequential pixel area overhead (especially for stacked-wafer image sensors), but allows for implementing a very powerful set of computational photography applications, without the drawbacks of the other approaches. The objective of this project is to refine and commercialize our CEP image sensors - microchips that are at the heart of a new class of motion-tolerant computational cameras. CEP image sensors dynamically program the exposure time of each individual pixel as opposed to conventional cameras where all pixels are exposed to light over equal timeintervals.Specifically, we target three imaging capabilities: (1) adaptive-pixel-exposure imaging under wide-ranging illumination conditions, (2) dynamic 3D imaging, and (3) sub-surface imaging of translucent objects, all done for moving scenes but without exacerbating motion artifacts. These and many other transformative imaging technologies are studied within the field of computational photography, an area of computer vision where synergetic use of imaging and computation yields powerful vision capabilities.

运动导致显著的成像伪影，当在传统相机上实现复杂的计算摄影算法时，其变得过大，因为它们通常需要使用多个视频帧。高度专业化的高帧速率图像传感器可用于执行类似CEP的成像，而不会出现这种伪影。但是这种高速相机易于产生高噪声、过度功耗和极高的输出数据速率，因此通常体积庞大且非常昂贵。我们正在引入一类新兴的编码曝光像素（CEP）图像传感器，其通过为每个单独的像素实现实时可编程性来保持低运动伪影并消除其他现有缺点。（相对较慢）帧读出执行多次（快速）曝光，每次曝光的光生电荷被引导到像素内的几个累积节点之一，基于像素代码。这种简单的像素内电荷分选功能是以不重要的像素面积开销为代价的（特别是对于堆叠晶片图像传感器），但是允许实现一组非常强大的计算摄影应用，而没有其他方法的缺点。该项目的目标是完善和商业化我们的CEP图像传感器-微芯片是一类新的运动容忍计算相机的核心。CEP图像传感器动态编程每个像素的曝光时间，而传统相机的所有像素都在相同的时间间隔内曝光。具体来说，我们的目标是三种成像功能：（1）在宽范围照明条件下的自适应像素曝光成像，（2）动态3D成像，以及（3）半透明物体的亚表面成像，所有这些都是针对运动场景而完成的，但不会加剧运动伪像。这些和许多其他变革性的成像技术在计算摄影领域进行了研究，这是一个计算机视觉领域，其中成像和计算的协同使用产生了强大的视觉功能。