SHF: Small: Collaborative Research: Software-Defined Imaging for Energy-Efficient Visual Computing

SHF：小型：协作研究：用于节能视觉计算的软件定义成像

• 批准号: 1909663
• 负责人: Suren Jayasuriya
• 金额: $ 33.3万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909663&HistoricalAwards=false
• 关键词: SHF; Small; Collaborative; Research; Software

Image sensors are becoming ubiquitous in daily life as they are incorporated in future intelligent systems including autonomous navigation, health monitoring, and robotics. A central challenge in these camera-driven applications is the inflexibility of current sensor designs and their consequent energy cost. This project designs a new category of image sensors which exploit hardware -- software co-design to attain better sensing at lower cost. The project advances a vertically-integrated design from the mixed-signal sensor circuitry to the computational architecture and the operating system software support. The project's impacts are the creation of new, flexible image sensor systems that can be used for a variety of visual computing applications. The project further seeks to include broadening access to education and research through curriculum material which emphasize smart cameras of the future, outreach to middle and high school students in a summer program to discover imaging applications, and industry engagement through workshops on software-defined imaging. The project focuses on designing software-defined image sensors, which offer new dimensions of configurability along with system support and programming abstractions to support application-specific needs. To achieve this, the project focuses on three main objectives: (1) Design and implementation of configurable sensor primitives, including programmable regions of interest with custom exposure, readout, and quantization, (2) architectures to control these sensor primitives, and to accelerate image signal processing and vision workloads, and (3) operating system services for scheduling the memory needs for our new sensor primitives, and OS interfaces that enable low-latency reactive sensor control for key applications. These innovations are evaluated in an integrative evaluation testbed that includes a fabricated software-defined image sensor prototype and a Field Programmable Gate Array (FPGA)-based system to measure energy and performance for a set of end-to-end visual applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

图像传感器在日常生活中变得无处不在，因为它们被纳入未来的智能系统，包括自主导航，健康监测和机器人技术。在这些摄像头驱动的应用中，一个核心挑战是电流传感器设计的可扩展性及其随之而来的能源成本。本项目设计了一种新型的图像传感器，利用硬件-软件协同设计，以较低的成本获得更好的传感。该项目推进了从混合信号传感器电路到计算架构和操作系统软件支持的垂直集成设计。该项目的影响是创建新的，灵活的图像传感器系统，可用于各种视觉计算应用。该项目进一步寻求通过强调未来智能相机的课程材料扩大教育和研究的机会，在暑期课程中向初中和高中学生推广以发现成像应用，并通过软件定义成像研讨会参与行业参与。该项目的重点是设计软件定义的图像传感器，它提供了新的维度的可配置性沿着系统支持和编程抽象，以支持特定于应用的需求。为实现这一目标，该项目侧重于三个主要目标：（1）可配置传感器基元的设计和实现，包括具有自定义曝光、读出和量化的可编程感兴趣区域，（2）控制这些传感器基元并加速图像信号处理和视觉工作负载的架构，以及（3）用于调度我们的新传感器基元的存储器需求的操作系统服务，和OS接口，可实现关键应用的低延迟反应式传感器控制。这些创新在一个综合评估测试平台中进行评估，该平台包括一个制造的软件定义图像传感器原型和一个基于现场可编程门阵列（FPGA）的系统，用于测量一组端到端视觉应用的能量和性能。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Adaptive Resolution-Based Tradeoffs for Energy-Efficient Visual Computing Systems

节能视觉计算系统的基于自适应分辨率的权衡

• DOI: 10.1109/mprv.2021.3052528
• 发表时间: 2021
• 期刊: IEEE Pervasive Computing
• 影响因子: 1.6
• 作者: LiKamWa, Robert;Hu, Jinhan;Kodukula, Venkatesh;Liu, Yifei
• 通讯作者: Liu, Yifei

Rhythmic pixel regions: multi-resolution visual sensing system towards high-precision visual computing at low power

• DOI: 10.1145/3445814.3446737
• 发表时间: 2021-04
• 期刊: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems
• 作者: Venkatesh Kodukula;Alexander Shearer;Van Nguyen;Srinivas Lingutla;Yifei Liu;R. Likamwa

Design and FPGA Implementation of an Adaptive video Subsampling Algorithm for Energy-Efficient Single Object Tracking

• DOI: 10.1109/icip40778.2020.9191146
• 发表时间: 2020-10
• 期刊: 2020 IEEE International Conference on Image Processing (ICIP)
• 作者: Odrika Iqbal;Saquib Siddiqui;Joshua Martin;Sameeksha Katoch;A. Spanias;D. Bliss;Suren Jayasuriya

Software-Defined Imaging: A Survey

• DOI: 10.1109/jproc.2023.3266736
• 发表时间: 2023-05
• 期刊: Proceedings of the IEEE
• 影响因子: 20.6
• 作者: Suren Jayasuriya;Odrika Iqbal;Venkatesh Kodukula;Victor Torres;R. Likamwa;A. Spanias

Optimizing JPEG Quantization for Classification Networks

优化分类网络的 JPEG 量化

• 发表时间: 2020
• 期刊: Resource-Constrained Machine Learning (ReCoML
• 作者: Li, Zhijing De Sa