On-Sensor Computer Vision

传感器计算机视觉

• 批准号: EP/Y023048/1
• 负责人: Piotr Dudek
• 金额: $ 169.13万
• 依托单位: University of Manchester
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2024
• 资助国家: 英国
• 起止时间: 2024 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FY023048%2F1
• 关键词: Sensor; Computer; Vision

Bringing advanced computer vision to edge devices such as robots, consumer electronics, or sensor networks is challenging due to the constraints of power, size and communication bandwidth under which they often operate. We propose vertically integrated research into the paradigm of on-sensor computer vision, where sensing and processing are unified into single chip which produces abstract, information-rich output rather than images. We aim to demonstrate that on-sensor computer vision can be much more powerful and general than seen in previous research, and that the correct hardware design, software framework and algorithm choices permit switchable or even simultaneous computation of a broad set of vision competences (such as motion estimation, segmentation and scene classification) on a single device. We propose to work on the design of on-sensor computer vision systems through a programme of work from pixel-processing architecture design and microelectronic hardware implementation, through software platform development, to unified algorithm design and experimentation to determine how to use this hardware in a full application. This will enable camera devices that not only capture images, but have a powerful built-in vision capability to understand what they are looking at, and ultimately can go from light to decision on a single sensor/processor chip, with unprecedented speed, low power consumption and small footprint. We hope to open up a new class of edge applications where cameras can be much more efficient and independent, or for smart cameras to be used in ways never previously considered.

将先进的计算机视觉引入机器人、消费电子产品或传感器网络等边缘设备是一项挑战，因为它们通常在功率、尺寸和通信带宽方面受到限制。我们建议垂直集成研究传感器计算机视觉的范式，其中传感和处理被统一到单个芯片，产生抽象的，信息丰富的输出，而不是图像。我们的目标是证明传感器上的计算机视觉可以比以前的研究中看到的更强大和更通用，并且正确的硬件设计，软件框架和算法选择允许在单个设备上切换甚至同时计算广泛的视觉能力（如运动估计，分割和场景分类）。我们建议通过一个工作方案，从像素处理架构设计和微电子硬件实现，通过软件平台开发，统一的算法设计和实验，以确定如何使用这个硬件在一个完整的应用程序上的传感器计算机视觉系统的设计。这将使相机设备不仅能够捕获图像，而且具有强大的内置视觉功能，可以理解他们正在看什么，最终可以在单个传感器/处理器芯片上从光到决策，具有前所未有的速度，低功耗和小尺寸。我们希望开辟一种新的边缘应用，使摄像头更加高效和独立，或者以以前从未考虑过的方式使用智能摄像头。