CRII: RI: Sub-mm 3D Scanning of Real-World Scenes with Active Multi-View Event Sensing

该奖项全部或部分由2021年美国救援计划法案（公法117-2）资助。最先进的3D成像方法无法同时测量所有可能的物体类别，仍然需要针对特定应用进行量身定制。这就是为什么3D成像在我们的社会中仍然不是无所不在的主要原因之一，并且仍然只有经过培训的专家才能使用特定任务的设备来捕获高质量的3D模型。该项目旨在建立一个“适合所有人”的系统，有可能改变这一点。目标系统可对当今价值数十亿美元的行业中的复杂表面/场景进行精确的3D测量，例如虚拟现实、工业检测、自主导航或医学成像。这些行业中的许多行业通常会遇到3D扫描系统特别具有挑战性的场景。此外，与场景无关且精确的3D感测系统可以具有许多应用。所产生的高质量3D数据集可以引领视觉相关人工智能研究的下一波浪潮，从而产生具有前所未有的检测质量、预测精度或导航精度的算法。鉴于当前现代社会各个领域相关技术的异化，每个人都可以受益。该项目伴随着一个综合教育计划，将3D成像原理纳入芝加哥课外课程的课程中，向高危青少年介绍光学，图像处理和电子学的基本概念。这项研究的重点是解决计算机视觉中一个长期存在的问题：表示“高”之义分辨率主动3D扫描场景，这些场景被混合镜面反射的物体所扰乱，并被不希望的光线贡献（如环境照明或强相互反射针对这一具有挑战性的任务的现有方法提供了相当清醒的结果，或者依赖于大型训练数据集或其他广泛的先验知识，例如场景中对象的几何形状和反射率。一种提供高质量数据的简单而灵活的解决方案引起了更广泛的计算机视觉社区研究人员的极大兴趣。这项研究提炼了研究者和他的同事们过去十年的研究成果。它结合了以前在不同对象类别的主动多视图3D成像概念方面的经验，以及生物启发事件传感器的新型检测模式（其工作原理与传统传感器根本不同）。通过适当地促进现有的3D成像和事件传感的权衡，该团队将开发理论，硬件和算法，导致一个从根本上新型的3D相机。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).State-of-the-art 3D imaging methods are not able to measure all possible classes of objects at once and still need to be tailored to a specific application. This is one of the main reasons why 3D imaging is still not omnipresent in our society, and still solely trained experts with task-specific equipment are able to capture high-quality 3D models. This project seeks to build a “one fits all” system that has the potential to change this. The targeted system enables precise 3D measurements of complicated surfaces/scenes in today’s billion-dollar industries, such as virtual reality, industrial inspection, autonomous navigation, or medical imaging. Many of these industries routinely run into particularly challenging scenarios for 3D scanning systems. Moreover, a scene-independent and precise 3D sensing system can have many applications. The produced sets of high-quality 3D data can usher the next wave in vision-related artificial intelligence research, leading to algorithms with unprecedented detection quality, prediction accuracy, or navigation precision. Given the current dissimilation of related techniques in all sectors of our modern society, everyone can profit. The project is accompanied by a comprehensive education program incorporating 3D imaging principles in a curriculum for Chicago afterschool programs to introduce at-risk youth to basic concepts in optics, image processing, and electronics.The focus of this research is to solve a long-standing problem in Computer Vision: high-resolution active 3D scanning of scenes cluttered with objects of mixed specularity and polluted by undesirable light contributions such as ambient illumination or strong inter-reflections. Existing approaches for this challenging task deliver rather sobering results or rely on large training datasets or other extensive prior knowledge, such as the geometry and reflectance of objects in the scene. An easy and flexible solution that delivers high-quality data is of significant interest for researchers in the broader computer vision community. This research distills the past decade’s research of the investigator and his colleagues. It combines previous experience in active multi-view 3D imaging concepts for different object classes with the novel detection modality of biologically inspired event sensors (which operate on a fundamentally different principle than conventional sensors). By properly facilitating the existing tradeoffs in 3D imaging and event sensing, the team will develop theory, hardware, and algorithms that lead to a fundamentally new type of 3D camera. The developed technique significantly advances the state-of-the-art and our fundamental understanding of limits.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.