CCSS: Programmable Mixed-Signal Vision Sensor for Continuous Mobile Vision

可穿戴设备的出现使计算机能够不断地解读用户环境，或连续的移动视觉。它可以扩展用户的记忆力和注意力，不仅可以实现以前不可能的个性化服务，还可以以前所未有的方式帮助视力或注意力受损的人。虽然现代设备能够捕捉和解释用户看到的东西，但它们面临着一个令人望而生畏的物理障碍：能源效率。例如，执行连续视觉工作负载会在大约40分钟内耗尽谷歌眼镜的电量。虽然工艺技术和系统级优化技术可能会继续提高数字电路的能效，但最近的一项测量研究指出了计算机视觉能效的一个根本瓶颈：图像传感器，特别是其模拟读出电路。该项目的目标是通过设计、原型和评估一种新的视觉传感器体系结构及其优化框架来解决这一瓶颈。通过瞄准计算机视觉，这种视觉传感器架构从根本上不同于现有的针对摄影进行优化的图像传感器设计。它不是产生高质量的图像，而是通过明智地将处理转移到模拟域来输出特定于应用的功能。通过这样做，它保证了计算机视觉工作负载的更高的效率，并缓解了连续移动视觉的隐私问题。本项目将朝着上述目标寻求两个相辅相成、相互关联的方向：首先，混合信号视觉传感器的设计必须在有限的模拟域复杂性下提供足够的可编程性。该项目将利用一种新的硬件体系结构，循环地重复使用模拟模块来实现可编程数据流。该体系结构将采用一种新颖的基于列的拓扑结构，该拓扑结构利用数据局部性来降低数据访问的互连复杂性。该项目还将研究允许在模拟域中处理的效率和精度之间进行可编程折衷的硬件机制。其次，在给定传感器架构的情况下，必须将视觉工作负载仔细划分为模拟和数字阶段。该项目应提供一个优化框架，利用准确的能量模型和视觉工作负荷的噪音容忍度。该项目将进一步贡献拟议的混合信号视觉传感器的新用例，数据隐私、能源消耗和任务性能是可能的优化约束。

The emergence of wearable devices has made it possible for computers to continuously interpret the user environment, or continuous mobile vision. It can extend a user's memory and attention, not only enabling previously impossible, personalized services but also assisting people with vision or attention impairment in an unprecedented way. While modern devices are capable of capturing and interpreting what their users see, they face a daunting physical barrier: energy efficiency. For example, performing continuous vision workloads drains the battery of Google Glass in about 40 minutes. While process technology and system-level optimization techniques may continue to improve the energy efficiency of digital circuits, a recent measurement study has pointed to a fundamental bottleneck to energy efficiency of computer vision: the image sensor, especially its analog readout circuitry. The goal of this project is to tackle this bottleneck by designing, prototyping and evaluating a novel vision sensor architecture along with its optimization framework. By targeting computer vision, this vision sensor architecture radically departs from existing image sensor designs that are optimized for photography. Instead of producing high-quality images, it outputs application-specific features by judiciously shifting processing into the analog domain. In doing so, it promises better efficiency by orders of magnitudes for computer vision workloads and relieves the privacy concern with continuous mobile vision.This project will pursue two complementary, interrelated directions toward the above goal: First, the mixed-signal vision sensor design must provide sufficient programmability under constrained complexity in the analog domain. The project will exploit a novel hardware architecture that cyclically reuses analog modules for a programmable dataflow. This architecture will employ a novel column-based topology that exploits data locality to reduce interconnect complexity for data access. The project will also investigate hardware mechanisms that allow programmable tradeoffs between efficiency and accuracy of processing in the analog domain. Second, vision workloads must be carefully partitioned into analog and digital stages given the sensor architecture. The project shall provide an optimization framework that leverages accurate energy models and noise tolerance of vision workload. The project will further contribute novel use cases of the proposed mixed-signal vision sensor with data privacy, energy consumption, and task performance being the possible optimization constraints.