Motion-tolerant coded-exposure-pixel cameras for emerging computational photography applications (Phase I)

运动导致明显的成像伪影，当复杂的计算摄影算法在传统相机上实现时，它变得非常大，因为它们通常需要使用多个视频帧。高度专业化的高帧率图像传感器可用于执行类似cep的成像，而无需此类伪影。但是这种高速摄像机容易产生高噪音、过度耗电和极高的输出数据速率，因此通常体积庞大且非常昂贵。我们正在引入一种新兴的编码曝光像素（CEP）图像传感器，通过实现每个单独像素的曝光时间可编程性，使运动伪影保持在较低水平，并消除其他现有的缺点。从技术上讲，对于每个（相对较慢的）帧读出，执行许多（快速）曝光，每次曝光的光产生电荷根据像素代码定向到像素内的几个积累节点之一。这种简单的像素内电荷排序功能的代价是不重要的像素面积开销（特别是对于堆叠晶片图像传感器），但允许实现一组非常强大的计算摄影应用程序，而没有其他方法的缺点。这个项目的目标是改进和商业化我们的CEP图像传感器——微芯片，是一种新的运动容忍计算相机的核心。与传统相机不同，CEP图像传感器对每个像素的曝光时间进行动态编程，而传统相机的所有像素都以相同的时间间隔暴露在光线下。具体来说，我们的目标是三个成像能力：(1)大范围照明条件下的自适应像素曝光成像，(2)动态3D成像，以及(3)半透明物体的亚表面成像，所有这些都是针对运动场景进行的，但不会加剧运动伪影。这些和许多其他变革性成像技术都是在计算摄影领域进行研究的，这是计算机视觉的一个领域，在这个领域中，成像和计算的协同使用产生了强大的视觉能力。

Motion causes significant imaging artifacts, which become prohibitively large when complex computational photography algorithms are implemented on conventional cameras, as they often require the use of multiple video frames. Highly-specialized high-frame-rate image sensors can be used to perform CEP-like imaging without such artifacts. But such high-speed cameras are prone to high noise, excessive power consumption, and extremely high output data rate, and thus are typically bulky and very expensive. We are introducing an emerging class of coded-exposure pixel (CEP) image sensors, that keep motion artifacts low and eliminate other existing drawbacks by implementing exposure-time programmability for each individual pixel.Technically speaking, for each (relatively slow) frame readout many (fast) exposures are performed, with the photogenerated charge for each exposure directed to one of several accumulation nodes within the pixel, based on a pixel code. This simple in-pixel charge-sorting function comes at a cost of an inconsequential pixel area overhead (especially for stacked-wafer image sensors), but allows for implementing a very powerful set of computational photography applications, without the drawbacks of the other approaches. The objective of this project is to refine and commercialize our CEP image sensors - microchips that are at the heart of a new class of motion-tolerant computational cameras. CEP image sensors dynamically program the exposure time of each individual pixel as opposed to conventional cameras where all pixels are exposed to light over equal timeintervals.Specifically, we target three imaging capabilities: (1) adaptive-pixel-exposure imaging under wide-ranging illumination conditions, (2) dynamic 3D imaging, and (3) sub-surface imaging of translucent objects, all done for moving scenes but without exacerbating motion artifacts. These and many other transformative imaging technologies are studied within the field of computational photography, an area of computer vision where synergetic use of imaging and computation yields powerful vision capabilities.