LeCA: In-Sensor Learned Compressive Acquisition for Efficient Machine Vision on the Edge
LeCA: In-Sensor Learned Compressive Acquisition for Efficient Machine Vision on the Edge
复制标题
DOI：
10.1145/3579371.3589089
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Tianrui Ma;Adith Boloor;Xiangxing Yang;Weidong Cao;Patrick Williams;Nan Sun;Ayan Chakrabarti;
中科院分区：
文献类型：
--
作者：
Tianrui Ma;Adith Boloor;Xiangxing Yang;Weidong Cao;Patrick Williams;Nan Sun;Ayan Chakrabarti;
随着基于深度学习的计算机视觉（CV）技术的快速发展，数字图像越来越多地被消费，而不是被人类消费，而是被下游的CV算法消费。然而，捕获高保真度和高分辨率图像是能量密集型的。它不仅主导了传感器本身的能耗（即在低功耗边缘设备中），而且还导致了后期存储，处理和通信阶段的重大内存负担和性能瓶颈。在本文中，我们系统地探讨了一种新的范式的传感器处理，称为“学习压缩采集”（LeCA）。LeCA框架针对边缘机器视觉应用，利用传感器自动编码器结构与下游CV算法的联合学习，有效地将原始图像压缩为具有自适应位深的低维特征。我们直接在图像传感器内部采用列并行模拟域处理来执行原始图像的压缩编码，从而实现有意义的硬件节省和能效改进。在现代机器视觉处理管道中进行评估，LeCA在任何数字压缩之前都可以实现4倍、6倍和8倍的压缩比，在ImageNet上的精度损失最小，分别为0.97%、0.98%和2.01%，优于现有方法。与传统的全分辨率图像传感器和最先进的压缩感知传感器相比，我们的LeCA传感器的能效分别提高了6.3倍和2.2倍，同时压缩比提高了2倍。
With the rapid advances of deep learning-based computer vision (CV) technology, digital images are increasingly consumed, not by humans, but by downstream CV algorithms. However, capturing high-fidelity and high-resolution images is energy-intensive. It not only dominates the energy consumption of the sensor itself (i.e. in low-power edge devices), but also contributes to significant memory burdens and performance bottlenecks in the later storage, processing, and communication stages. In this paper, we systematically explore a new paradigm of in-sensor processing, termed "learned compressive acquisition" (LeCA). Targeting machine vision applications on the edge, the LeCA framework exploits the joint learning of a sensor autoencoder structure with the downstream CV algorithms to effectively compress the original image into low-dimensional features with adaptive bit depth. We employ column-parallel analog-domain processing directly inside the image sensor to perform the compressive encoding of the raw image, resulting in meaningful hardware savings, and energy efficiency improvements. Evaluated within a modern machine vision processing pipeline, LeCA achieves 4×, 6×, and 8× compression ratios prior to any digital compression, with minimal accuracy loss of 0.97%, 0.98%, and 2.01% on ImageNet, outperforming existing methods. Compared with the conventional full-resolution image sensor and the state-of-the-art compressive sensing sensor, our LeCA sensor is 6.3× and 2.2× more energy-efficient while reaching a 2× higher compression ratio.