CSR: Small: Computational Support for Time Domain Continuous Imaging

传统的数码相机和摄像机模仿胶片相机：传感器在曝光间隔期间整合光线，然后处理潜像，创建快照或视频帧序列。 相比之下，时域连续成像（TDCI）是一种变革性的新的成像计算方法，基于记录每个像素的连续波形，描述光水平如何随时间变化。 由静止图像或视频帧表示的时间间隔可以在捕获之后指定，并且图像通过计算积分对应于该周期的记录的像素波形的部分来渲染。 此外，曝光参数不再直接受传感器ISO感光度的限制：波形提供低噪声和高动态范围（HDR），与场景中的亮度范围或所使用的表观快门速度无关。实际上，TDCI流是一种“原始”成像表示，除了通常的图像特征之外，还允许对时间属性进行后处理。 通过指定，实验评估和传播TDCI所需的基本计算支持，该项目奠定了传感器系统和采用该模型的新应用程序的未来发展的基础。TDCI传感器系统具有重新定义相机概念的潜力，但其计算密集度极高。 必须在摄像机中进行大量计算，以满足传感器控制、捕获和每个像素波形的压缩编码的严格实时约束，以创建TDCI流。 TDCI流的操作还需要新的计算方法用于其他任务，从从TDCI流的常规图像的有效合成到直接变换TDCI流的算法的实现。 例如，TDCI流中每个波形的准确性可以通过分析附近像素的波形来提高。拟议的工作集中在探索和实验评估支持TDCI所需的计算的各个方面，包括相机内和后处理。 微芯片技术的最新进展表明，这些计算可以在真实的时间内经济地完成。 虽然TDCI的充分利用最终可能会利用集成并行纳米控制器阵列的定制传感器，但该项目中的探索性研究通过使用传统传感器和计算机硬件合成和处理的TDCI流来避免对这种新传感器的依赖。 该项目的目标是更好地理解与TDCI相关的计算挑战和问题，同时产生基本计算支持的参考实现和“项目工具包”，作为促进传播和进一步调查的工件。

Conventional digital still and video cameras mimic film cameras: the sensor integrates light during an exposure interval and then the latent image is processed, creating a snapshot or sequence of video frames. In contrast, time domain continuous imaging (TDCI) is a transformative new computational approach to imaging based on recording, for each pixel, a continuous waveform describing how the light level changes over time. The time interval to be represented by a still image or video frame can be specified after capture and the image rendered by computationally integrating the portion of the recorded pixel waveforms corresponding to that period. Further, the exposure parameters are no longer directly constrained by sensor ISO sensitivity: the waveforms provide low noise and high dynamic range (HDR) independent of the range of brightness in the scene or apparent shutter speed used. In effect, a TDCI stream is a "raw" imaging representation that allows post-processing of temporal properties in addition to the usual image characteristics. By specifying, experimentally evaluating, and disseminating the basic computational support needed for TDCI, this project lays the foundation for future development of sensor systems and new applications employing this model. Success could spawn an entirely new generation of technology that would revolutionize the fields of digital photography, video recording, and remote sensing.TDCI sensor systems have the potential to redefine the concept of a camera, but are extremely compute-intensive. Substantial computation must be done in the camera to meet tight real-time constraints for control of the sensor, capture, and compressed encoding of a waveform per pixel to create a TDCI stream. Manipulation of TDCI streams also requires new computational methods for other tasks ranging from efficient synthesis of conventional images from TDCI streams to implementation of algorithms directly transforming TDCI streams. For example, the accuracy of each waveform within a TDCI stream can be improved using analysis of waveforms for nearby pixels.The proposed work centers on exploring and experimentally evaluating all aspects of computation needed to support TDCI, both in-camera and post-processing. Recent advances in microchip technology suggest that these computations can be done economically in real time. While full exploitation of TDCI may ultimately make use of custom sensors integrating massively-parallel nanocontroller arrays, the exploratory research in this project avoids dependence on such new sensors by using TDCI streams synthesized and processed using conventional sensors and computer hardware. The goal of this project is to better understand the computational challenges and issues associated with TDCI, while producing reference implementations of the basic computing support and "project kits" as artifacts facilitating dissemination and further investigation.