Motion-tolerant coded-exposure-pixel cameras for emerging computational photography applications (Phase I)

点击翻译按钮获取中文摘要

Motion causes significant imaging artifacts, which become prohibitively large when complex computational photography algorithms are implemented on conventional cameras, as they often require the use of multiple video frames. Highly-specialized high-frame-rate image sensors can be used to perform CEP-like imaging without such artifacts. But such high-speed cameras are prone to high noise, excessive power consumption, and extremely high output data rate, and thus are typically bulky and very expensive. We are introducing an emerging class of coded-exposure pixel (CEP) image sensors, that keep motion artifacts low and eliminate other existing drawbacks by implementing exposure-time programmability for each individual pixel.Technically speaking, for each (relatively slow) frame readout many (fast) exposures are performed, with the photogenerated charge for each exposure directed to one of several accumulation nodes within the pixel, based on a pixel code. This simple in-pixel charge-sorting function comes at a cost of an inconsequential pixel area overhead (especially for stacked-wafer image sensors), but allows for implementing a very powerful set of computational photography applications, without the drawbacks of the other approaches. The objective of this project is to refine and commercialize our CEP image sensors - microchips that are at the heart of a new class of motion-tolerant computational cameras. CEP image sensors dynamically program the exposure time of each individual pixel as opposed to conventional cameras where all pixels are exposed to light over equal timeintervals.Specifically, we target three imaging capabilities: (1) adaptive-pixel-exposure imaging under wide-ranging illumination conditions, (2) dynamic 3D imaging, and (3) sub-surface imaging of translucent objects, all done for moving scenes but without exacerbating motion artifacts. These and many other transformative imaging technologies are studied within the field of computational photography, an area of computer vision where synergetic use of imaging and computation yields powerful vision capabilities.