ITR: Sensory Level Computation and Information Encoding for Robust Imaging

这是为期三年的持续奖励的第一年资助。在从传感器芯片读出信息之前，该项目研究用于中间感觉信号表示的片上模拟和混合模式计算。通过适当的信息编码，这种表示将对传感和读出过程的限制具有健壮性，从而能够有效地提取有用的环境信息。片上计算使传感器能够在芯片上做出部分决定，并使用这些决定来创建最佳信号表示和感官信息的稳健提取。该项目将专注于图像传感器。从理论上讲，图像传感器很有趣，因为它们是可扩展的并行系统，需要在大量站点之间进行细粒度、分布式计算和全局数据通信。在大型并行系统中，需要将大量处理器/站点的数据集中在一起会迅速使通信连接饱和，并对计算效率产生不利影响。这个项目的目的不是将传统的芯片上的图像处理微型化，而是获得有关环境的信息，如果不在感官水平上进行计算，则无法获得环境信息。从实用的角度来看，该项目侧重于图像传感器对高动态范围和低动态范围场景的适应能力。这样的场景通常会导致传统传感器出现故障；然而，这样的场景在日常成像应用中无处不在。PI的团队将建造几个高分辨率的CMOS线和面计算图像传感器来测试我们的信号编码技术。最终，芯片将在CMU机器人研究所正在进行的机器人应用中进行测试。一个候选应用是为严重残疾用户提供完全自主或机器人辅助的轮椅的视觉指导，这些用户只能提供对系统的高级控制。这项研究的主要意义在于缓解大型可扩展并行系统的全局数据聚合和通信问题。这项研究的第二个意义在于它在视觉感知系统中的实际应用以及它优越的信息提取能力。

This is the first year funding of a three-year continuing award. The project investigates on-chip analog and mixed mode computation for an intermediate sensory signal representation before the information is read out from the sensor chip. By appropriate information encoding, such representation will be robust to limitations of the sensing and readout process, thus enabling efficient extraction of useful environmental information. On-chip computation enables a sensor to make partial decisions on-chip and to use those decisions to create an optimal signal representation and robust extraction of sensory information. This project will focus on image sensors. From a theoretical point of view, image sensors are interesting because they are scalable parallel systems that require fine-grain, distributed computation and global data communication among a large number of sites. The necessity to bring together data from a large number of processors/sites quickly saturates communication connectivity and adversely affects computing efficiency in large parallel systems. The aim of this project is not to miniaturize conventional imageprocessing on-chip, but rather to obtain information about the environment that is not obtainable if computation is not performed on the sensory level. From a practical point of view, this project focuses on the ability of image sensors to adapt to high and low dynamic range scenes. Such scenes routinely cause conventional sensors to fail; yetsuch scenes are omnipresent in everyday imaging applications. The PI's group willbuild several high resolution CMOS line and area computational image sensors to test our signal encoding techniques. Ultimately chips will be tested in an ongoing robotic application at the CMU Robotics Institute. One candidate application is a visual guidance of fully autonomous or robot-assisted wheelchair for severely disabled users who are able to provide only high-level control to the system. The primary significance of this research is toward alleviating the problem of global data aggregation and communication from large and scalable parallel systems. The secondary significance of this research is in its practical application to visual perception systems and its superior information extraction ability.