CRII: RI: Sub-mm 3D Scanning of Real-World Scenes with Active Multi-View Event Sensing

该奖项的全部或部分资金来自《2021年美国救援计划法案》(公法117-2)。最先进的3D成像方法无法同时测量所有可能的物体类别，仍需针对特定应用进行量身定做。这就是为什么3D成像在我们的社会中仍然不是无所不在的主要原因之一，仍然只有经过培训的专家使用特定任务的设备能够捕获高质量的3D模型。这个项目寻求建立一个“一刀切”的系统，有可能改变这一点。在当今价值数十亿美元的行业中，这种有针对性的系统可以对复杂的表面/场景进行精确的3D测量，如虚拟现实、工业检测、自主导航或医学成像。这些行业中的许多经常遇到3D扫描系统特别具有挑战性的场景。此外，一个与场景无关的、精确的三维传感系统可以有很多应用。产生的高质量3D数据集可以在视觉相关的人工智能研究中迎来下一波浪潮，导致算法具有前所未有的检测质量、预测精度或导航精度。鉴于目前相关技术在我们现代社会所有部门的异化，每个人都可以受益。该项目伴随着一个全面的教育计划，该计划将3D成像原理纳入芝加哥课外计划的课程中，向高危青少年介绍光学、图像处理和电子学的基本概念。这项研究的重点是解决计算机视觉中的一个长期存在的问题：高分辨率主动3D扫描场景，场景中杂乱的物体混合镜面，并被环境照明或强烈相互反射等不良光线污染。这一具有挑战性的任务的现有方法提供了相当令人警醒的结果，或者依赖于大量的训练数据集或其他广泛的先验知识，例如场景中对象的几何和反射比。对于更广泛的计算机视觉领域的研究人员来说，提供高质量数据的简单而灵活的解决方案是非常重要的。这项研究提炼了这位调查员和他的同事过去十年的研究成果。它结合了以前针对不同对象类别的主动多视角3D成像概念的经验，以及受生物启发的事件传感器的新颖检测模式(其工作原理与传统传感器完全不同)。通过适当地促进3D成像和事件传感方面的现有权衡，该团队将开发出理论、硬件和算法，从而产生一种全新的3D相机。开发的技术极大地促进了最先进的技术和我们对限制的基本理解。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).State-of-the-art 3D imaging methods are not able to measure all possible classes of objects at once and still need to be tailored to a specific application. This is one of the main reasons why 3D imaging is still not omnipresent in our society, and still solely trained experts with task-specific equipment are able to capture high-quality 3D models. This project seeks to build a “one fits all” system that has the potential to change this. The targeted system enables precise 3D measurements of complicated surfaces/scenes in today’s billion-dollar industries, such as virtual reality, industrial inspection, autonomous navigation, or medical imaging. Many of these industries routinely run into particularly challenging scenarios for 3D scanning systems. Moreover, a scene-independent and precise 3D sensing system can have many applications. The produced sets of high-quality 3D data can usher the next wave in vision-related artificial intelligence research, leading to algorithms with unprecedented detection quality, prediction accuracy, or navigation precision. Given the current dissimilation of related techniques in all sectors of our modern society, everyone can profit. The project is accompanied by a comprehensive education program incorporating 3D imaging principles in a curriculum for Chicago afterschool programs to introduce at-risk youth to basic concepts in optics, image processing, and electronics.The focus of this research is to solve a long-standing problem in Computer Vision: high-resolution active 3D scanning of scenes cluttered with objects of mixed specularity and polluted by undesirable light contributions such as ambient illumination or strong inter-reflections. Existing approaches for this challenging task deliver rather sobering results or rely on large training datasets or other extensive prior knowledge, such as the geometry and reflectance of objects in the scene. An easy and flexible solution that delivers high-quality data is of significant interest for researchers in the broader computer vision community. This research distills the past decade’s research of the investigator and his colleagues. It combines previous experience in active multi-view 3D imaging concepts for different object classes with the novel detection modality of biologically inspired event sensors (which operate on a fundamentally different principle than conventional sensors). By properly facilitating the existing tradeoffs in 3D imaging and event sensing, the team will develop theory, hardware, and algorithms that lead to a fundamentally new type of 3D camera. The developed technique significantly advances the state-of-the-art and our fundamental understanding of limits.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.