On-Sensor Computer Vision

由于功率、尺寸和通信带宽的限制，将先进的计算机视觉应用于机器人、消费电子产品或传感器网络等边缘设备具有挑战性。我们建议垂直整合研究到传感器计算机视觉范式中，其中传感和处理统一到单个芯片中，产生抽象的，信息丰富的输出，而不是图像。我们的目标是证明传感器计算机视觉可以比以前的研究更强大和通用，并且正确的硬件设计，软件框架和算法选择允许在单个设备上切换甚至同时计算广泛的视觉能力（如运动估计，分割和场景分类）。我们建议通过从像素处理架构设计和微电子硬件实现，通过软件平台开发，到统一算法设计和实验来确定如何在完整应用中使用该硬件的工作计划来设计传感器计算机视觉系统。这将使相机设备不仅可以捕获图像，而且具有强大的内置视觉能力，可以理解他们所看到的东西，最终可以在单个传感器/处理器芯片上从光到决策，具有前所未有的速度，低功耗和小占地。我们希望开辟一种新的边缘应用，让摄像头更加高效和独立，或者让智能摄像头以前所未有的方式使用。

Bringing advanced computer vision to edge devices such as robots, consumer electronics, or sensor networks is challenging due to the constraints of power, size and communication bandwidth under which they often operate. We propose vertically integrated research into the paradigm of on-sensor computer vision, where sensing and processing are unified into single chip which produces abstract, information-rich output rather than images. We aim to demonstrate that on-sensor computer vision can be much more powerful and general than seen in previous research, and that the correct hardware design, software framework and algorithm choices permit switchable or even simultaneous computation of a broad set of vision competences (such as motion estimation, segmentation and scene classification) on a single device. We propose to work on the design of on-sensor computer vision systems through a programme of work from pixel-processing architecture design and microelectronic hardware implementation, through software platform development, to unified algorithm design and experimentation to determine how to use this hardware in a full application. This will enable camera devices that not only capture images, but have a powerful built-in vision capability to understand what they are looking at, and ultimately can go from light to decision on a single sensor/processor chip, with unprecedented speed, low power consumption and small footprint. We hope to open up a new class of edge applications where cameras can be much more efficient and independent, or for smart cameras to be used in ways never previously considered.