ITR: Sensory Level Computation and Information Encoding for Robust Imaging

这是一个为期三年的连续奖励的第一年资助。该项目研究了在从传感器芯片读取信息之前，用于中间感觉信号表示的片上模拟和混合模式计算。通过适当的信息编码，这种表示将对感测和读出过程的限制具有鲁棒性，从而能够有效地提取有用的环境信息。片上计算使传感器能够在片上做出部分决策，并使用这些决策来创建最佳信号表示和传感信息的鲁棒提取。该项目将专注于图像传感器。从理论的角度来看，图像传感器是有趣的，因为它们是可扩展的并行系统，需要细粒度，分布式计算和大量站点之间的全局数据通信。将来自大量处理器/站点的数据汇集在一起的必要性使通信连接性迅速饱和，并对大型并行系统中的计算效率产生不利影响。该项目的目的不是要实现传统的片上图像处理，而是要获得有关环境的信息，如果不进行感官层面的计算，这些信息是无法获得的。从实用的角度来看，该项目主要关注图像传感器适应高动态范围和低动态范围场景的能力。此类场景通常会导致传统传感器出现故障;但此类场景在日常成像应用中无处不在。PI的小组将建立几个高分辨率CMOS线和面积计算图像传感器来测试我们的信号编码技术。最终，芯片将在CMU机器人研究所正在进行的机器人应用中进行测试。一个候选应用是完全自主或机器人辅助轮椅的视觉引导，用于只能对系统进行高级控制的严重残疾用户。这项研究的主要意义是减轻问题的全球数据聚合和通信的大型和可扩展的并行系统。这项研究的第二个意义在于它在视觉感知系统中的实际应用及其优越的上级信息提取能力。

This is the first year funding of a three-year continuing award. The project investigates on-chip analog and mixed mode computation for an intermediate sensory signal representation before the information is read out from the sensor chip. By appropriate information encoding, such representation will be robust to limitations of the sensing and readout process, thus enabling efficient extraction of useful environmental information. On-chip computation enables a sensor to make partial decisions on-chip and to use those decisions to create an optimal signal representation and robust extraction of sensory information. This project will focus on image sensors. From a theoretical point of view, image sensors are interesting because they are scalable parallel systems that require fine-grain, distributed computation and global data communication among a large number of sites. The necessity to bring together data from a large number of processors/sites quickly saturates communication connectivity and adversely affects computing efficiency in large parallel systems. The aim of this project is not to miniaturize conventional imageprocessing on-chip, but rather to obtain information about the environment that is not obtainable if computation is not performed on the sensory level. From a practical point of view, this project focuses on the ability of image sensors to adapt to high and low dynamic range scenes. Such scenes routinely cause conventional sensors to fail; yetsuch scenes are omnipresent in everyday imaging applications. The PI's group willbuild several high resolution CMOS line and area computational image sensors to test our signal encoding techniques. Ultimately chips will be tested in an ongoing robotic application at the CMU Robotics Institute. One candidate application is a visual guidance of fully autonomous or robot-assisted wheelchair for severely disabled users who are able to provide only high-level control to the system. The primary significance of this research is toward alleviating the problem of global data aggregation and communication from large and scalable parallel systems. The secondary significance of this research is in its practical application to visual perception systems and its superior information extraction ability.