SBIR Phase I: Lightweight Learning-based Camera Image Signal Processing (ISP) for Photon-Limited Imaging

SBIR 第一阶段：用于光子限制成像的轻量级基于学习的相机图像信号处理 (ISP)

• 批准号: 2335309
• 负责人: Preston Rahim
• 金额: $ 27.47万
• 依托单位: DEEPLUX TECHNOLOGY, INC
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-03-01 至 2025-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2335309&HistoricalAwards=false
• 关键词: SBIR; Phase; Lightweight; Learning; based

The broader impact of this Small Business Innovation (SBIR) Phase I project will result from the ability to operate digital image sensors at lower light levels than is currently possible. The technology is expected to be deployable in any mid- to low-level camera device, with potential applications across all industries that leverage camera technology. Consumer applications that would benefit from improved low-light imaging include dashboard cameras and notebook cameras for videoconferencing; military and national security applications include night vision and autonomous navigation; while the technology will also enable improved diagnostic capabilities in medical procedures such as endoscopies. The technology is expected to have a direct impact on workforce development, and deployment of the solution will drive economics in consumer electronics.This Small Business Innovation Research (SBIR) Phase I project aims to achieve photon-limited image denoising using a lightweight algorithm that has the potential to be implemented on a camera chip. Accomplishing this goal requires several technological breakthroughs, collectively leading to a new image signal processor (ISP) known as a Small and Learnable ISP Module (SLIM). The key to SLIM is to identify the bottlenecks of physics-based ISPs and replace them with customized learning-based modules. Specifically, SLIM consists of five innovations: (i) learning-based frequency demodulation, (ii) guided denoising, (iii) learned feature extraction, (iv) learned indexing, and (v) learned filtering. In Phase 1, the team proposes to optimize SLIM and implement it on a field programmable gate array (FPGA). This includes shrinking the size of the filters and streamlining the indexing scheme to further speed up SLIM, introducing new encoders to improve generalization, and optimizing the memory, communication, and processing through improved programming and real-data evaluation and demonstration.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这一小型企业创新(SBIR)第一阶段项目的更广泛影响将来自于在比目前可能的光线更低的情况下操作数字图像传感器的能力。这项技术预计将部署在任何中低级摄像设备中，在利用摄像技术的所有行业都有潜在的应用。将受益于改进的微光成像的消费者应用包括用于视频会议的仪表板摄像头和笔记本摄像头；军事和国家安全应用包括夜视和自主导航；而该技术还将使内窥镜等医疗程序的诊断能力得到改善。该技术预计将对劳动力发展产生直接影响，该解决方案的部署将推动消费电子产品的经济效益。这个小型企业创新研究(SBIR)第一阶段项目旨在使用一种有可能在相机芯片上实现的轻量级算法来实现光子受限的图像去噪。实现这一目标需要几项技术突破，共同导致一种新的图像信号处理器(ISP)，称为小型可学习的ISP模块(SLIM)。SLIM的关键是找出基于物理的互联网服务提供商的瓶颈，并用定制的基于学习的模块来取代它们。具体地说，SLIM包括五个创新：(I)基于学习的频率解调，(Ii)引导去噪，(Iii)学习特征提取，(Iv)学习索引，和(V)学习滤波。在第一阶段，该团队建议优化SLIM并在现场可编程门阵列(FPGA)上实现它。这包括缩小过滤器的尺寸和简化索引方案以进一步加快SLIM的速度，引入新的编码器以改进泛化，并通过改进的编程和真实数据评估和演示来优化存储器、通信和处理。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。