SHF: Small: Collaborative Research: Software-Defined Imaging for Energy-Efficient Visual Computing

SHF：小型：协作研究：用于节能视觉计算的软件定义成像

• 批准号: 1909073
• 负责人: Adrian Sampson
• 金额: $ 16.6万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909073&HistoricalAwards=false
• 关键词: SHF; Small; Collaborative; Research; Software

Image sensors are becoming ubiquitous in daily life as they are incorporated in future intelligent systems including autonomous navigation, health monitoring, and robotics. A central challenge in these camera-driven applications is the inflexibility of current sensor designs and their consequent energy cost. This project designs a new category of image sensors which exploit hardware -- software co-design to attain better sensing at lower cost. The project advances a vertically-integrated design from the mixed-signal sensor circuitry to the computational architecture and the operating system software support. The project's impacts are the creation of new, flexible image sensor systems that can be used for a variety of visual computing applications. The project further seeks to include broadening access to education and research through curriculum material which emphasize smart cameras of the future, outreach to middle and high school students in a summer program to discover imaging applications, and industry engagement through workshops on software-defined imaging. The project focuses on designing software-defined image sensors, which offer new dimensions of configurability along with system support and programming abstractions to support application-specific needs. To achieve this, the project focuses on three main objectives: (1) Design and implementation of configurable sensor primitives, including programmable regions of interest with custom exposure, readout, and quantization, (2) architectures to control these sensor primitives, and to accelerate image signal processing and vision workloads, and (3) operating system services for scheduling the memory needs for our new sensor primitives, and OS interfaces that enable low-latency reactive sensor control for key applications. These innovations are evaluated in an integrative evaluation testbed that includes a fabricated software-defined image sensor prototype and a Field Programmable Gate Array (FPGA)-based system to measure energy and performance for a set of end-to-end visual applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

图像传感器在日常生活中变得无处不在，因为它们被纳入未来的智能系统，包括自主导航、健康监测和机器人技术。这些相机驱动应用的一个主要挑战是当前传感器设计的不灵活性及其随之而来的能源成本。该项目设计了一种新型图像传感器，它利用硬件-软件协同设计以更低的成本获得更好的传感效果。该项目推进了从混合信号传感器电路到计算架构和操作系统软件支持的垂直集成设计。该项目的影响是创建新的、灵活的图像传感器系统，可用于各种视觉计算应用。该项目还寻求通过强调未来智能相机的课程材料扩大教育和研究的机会，在暑期项目中向中学生和高中生推广以发现成像应用，并通过软件定义成像研讨会进行行业参与。该项目的重点是设计软件定义的图像传感器，它提供了新的可配置性维度以及系统支持和编程抽象，以支持特定于应用程序的需求。为了实现这一目标，该项目重点关注三个主要目标：（1）可配置传感器基元的设计和实现，包括具有自定义曝光、读出和量化功能的可编程感兴趣区域，（2）控制这些传感器基元并加速图像信号处理和视觉工作负载的架构，以及（3）用于调度新传感器基元内存需求的操作系统服务，以及支持低延迟反应的操作系统接口 关键应用的传感器控制。这些创新在综合评估测试台中进行评估，该测试台包括制造的软件定义图像传感器原型和基于现场可编程门阵列 (FPGA) 的系统，用于测量一组端到端视觉应用的能量和性能。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Modular Hardware Design with Timeline Types

具有时间线类型的模块化硬件设计

• DOI: 10.1145/3591234
• 发表时间: 2023
• 期刊: Proceedings of the ACM on Programming Languages
• 作者: Nigam, Rachit;Azevedo de Amorim, Pedro Henrique;Sampson, Adrian
• 通讯作者: Sampson, Adrian

Optimizing JPEG Quantization for Classification Networks

优化分类网络的 JPEG 量化

• 发表时间: 2020
• 期刊: Resource-Constrained Machine Learning (ReCoML
• 作者: Li, Zhijing De Sa