Collaborative Research: RI : Medium: Thermal Computational Imaging

用于光热波段的图像传感器通常有噪声，并且容易受到干扰变化的影响。然而，毋庸置疑的是，热成像有可能刺激大量的科学和工程应用，从而从根本上改变我们的社会。例如，配备了热像仪的自动驾驶车辆可以在黑暗中导航，并在雾和雨中导航。热成像技术的广泛应用可能带来在公共场所进行非侵入性生命体征监测的方法，从而为我们的公共健康武器库提供新的工具，并能够在地方和全球范围内进行环境和生态系统监测。该项目致力于使用价格低廉但噪音较大的热传感器来实现此类应用。在这一进展中取得的研究进展将与教育和推广计划相结合，包括创建新的本科生和研究生课程，让本科生参与研究，并与K-12社区接触。这项研究的重点是通过开发用于热波段的渲染、建模和成像的基础工具来促进对热场景的理解。该项目将开发一种物理上精确的热波段渲染管道，其中包括复杂表面的波长相关热发射率、大气中的热传播以及准确的传感器建模和噪声表征。这种建模将用于设计和开发新型的计算成像系统，允许捕获多维热信号，如多光谱和偏振测量。这项研究还将开发一种可区分的渲染器，并在反向图形的端到端学习框架中对其进行开发。这将使去噪、超分辨率、立体声、目标检测等问题得到显著改善。研究进展将用于探索和推进自主导航、远程健康监测和远程环境监测。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Image sensors for thermal wavebands of light are usually noisy and subject to nuisance variations. Yet, it is indisputable that thermal imaging has the potential to spur numerous scientific and engineering applications that can fundamentally transform our society. For example, autonomous vehicles equipped with thermal cameras can navigate in the dark and through fog and rain. The widespread availability of thermal imaging technology could usher in methods for non-intrusive vital sign monitoring in public spaces, thereby providing a new tool in our public health arsenal, as well as enable environmental and ecosystem monitoring at both local and global scales. This project seeks to enable such applications with inexpensive but noisy thermal sensors. The research advances made in this progress will be integrated with an educational and outreach program that includes creating new undergraduate and graduate courses, engaging undergraduates in research, and engaging with K-12 communities.The focus of this research is to advance thermal scene understanding by developing foundational tools for rendering, modeling and imaging at thermal wavebands. The project will develop a physically accurate rendering pipeline for thermal wavebands, that incorporates wavelength-dependent thermal emissivity of complex surfaces, thermal propagation in atmosphere, and accurate sensor modeling and noise characterization. This modeling will be used to design and develop novel computational imaging systems that allow capture of multi-dimensional thermal signals, such as multi-spectral and polarization measurements. The research will also develop a differentiable renderer and exploit it within an end-to-end learning framework for inverse graphics. This will allow for significant improvements in problems such as denoising, super-resolution, stereo, object detection and others. The advances in research will be used to explore and advance autonomous navigation, remote health monitoring and remote environmental monitoring.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.